Automated tile grid stitching - how it works

Pff, stitching multiple tiles turned out to be a much harder problem then I thought…

You can find a lot of tutorials how to do simple image stitching using two photos and that is no problem. But what to do when I want to make a panorama from 4-6 images or more?

This question posted on stackoverflow, this article, make clear that robust automated stitching of multiple images presents a much harder problem. In case you are interested to learn how this works. Here follows a brief description of the consecutive steps of automated tile stitching algorithm implemented in gridstitcher.

To get started you need to obtain a list of the tile image files. This is best done with the operating system independent standard package pathlib. Let’s also already import matplotlib here to inspect the result below.

from pathlib import Path
import matplotlib.pyplot as plt

tiles_dir = Path('/home/frank/Work/Werknummers/2024-811_Falnama/data/Weimar-IRR_1600nm/71803-02_IRR 1600 nm/') 
im_files = sorted(tiles_dir.glob('*.tif'))
len(im_files)

To perform the consecutive steps of the automated registration of the image tiles we first need to import the Tilegrid class.

from gridstitcher import Tilegrid

Tile registration is performed in three consecutive steps. In step 1 we instantiate a Tilegrid object with a sorted list of image files, and specify the number of rows and columns for the grid.

tg = Tilegrid(im_files, nrows=6, ncols=6) # 1) load images

Initializing regular 6x6 tile grid...

In step 2 the SIFT-RANSAC algorithm is used to obtain pairs of points that match corresponding features on adjacent tiles. These point pairs are computed and plotted with the Tilegrid.get_ransac_pair_points() method.

tg.get_ransac_pair_points()

Detecting corresponding SIFT RANSAC point pairs...
Could not detect corresponding feature points in tile pair [2, 3] 
Could not detect corresponding feature points in tile pair [3, 4]

For this specific example warnings are given that in the tile overlap pairs [2, 3] and [3, 4] no corresponding feature points were detected. The absence of overlap pair is not a problem as long as each tile is connected at least with one other tile.

In step 3 the sum of distances for all RANSAC point pairs in the tile grid is minimized by shifting each individual tile left-right and up-down. Note that we do not rotate, scale or warp the images.

tg.register_tiles()                          # 3) shift tiles to optimal overlap

Registering tiles...
Sum of tile distances: 4536.0

/home/frank/anaconda3/lib/python3.11/site-packages/scipy/optimize/_minimize.py:705: OptimizeWarning: Desired error not necessarily achieved due to precision loss.
  res = _minimize_bfgs(fun, x0, args, jac, callback, **options)

         Current function value: 4536.434504
         Iterations: 115
         Function evaluations: 15487
         Gradient evaluations: 212

In our example we observe that the registration of the individual tiles is successful. To make the composite image array use the method Tilegrid.make_stitched(). If you specify the option save_to= you can automatically save to an image file.

stitched_im = tg.make_stitched(save_to='../downloads/stitched.png')

Saving image to: ../downloads/stitched.png

Let’s inspect our stitched mosaic image.

fig, ax = plt.subplots()
ax.imshow(stitched_im)

Nicely registered!

FUNCTIONS

source

make_stitched

 make_stitched (ims, extent_list, save_to=None)

*Create a stitched mosaic image from tile images ims and tile positions extent_list.

Returns: stitched_im*

source

remove_outliers

 remove_outliers (rpp_list, oip_list, nrows, ncols, max_delta=20,
                  verbose=True)

Remove vector outliers from rpp and oip lists based on max_delta pixels shift.

source

crop

 crop (ax, extent_list)

source

plot_vectors

 plot_vectors (canvas_rpp_list, oip_list, ax)

Plot vectors from canvas_rpp_list on active canvas ax with overlapping image pair list oip_list.

source

total_distance

 total_distance (b_flat, rpp_list, oip_list)

Compute distances.

source

get_canvas_rpp_list

 get_canvas_rpp_list (rpp_list, oip_list, b_flat)

Compute canvas positions of red and green points.

source

get_extent_list

 get_extent_list (b_flat, tile_h, tile_w)

*Compute tile images extents on canvas.

extent : left, right, bottom, top*

source

get_x0_list

 get_x0_list (nrows, ncols, tile_h, tile_w, margin=20)

Initial flat tile positions list.

source

add_ransac_markers

 add_ransac_markers (axs, oips, ransac_points_pair_list)

Add red and green markers for corresponding points in all tiles.

source

get_ransac_pair_points

 get_ransac_pair_points (oips, kpts_descr_list)

source

make_overlap_image_pairs

 make_overlap_image_pairs (nrows, ncols)

Generate image pair indices for horizontal and vertical overlap pairs.

source

make_kpts_descr_list

 make_kpts_descr_list (ims)

Create keypoints and descriptors list for al images ims.

source

Tilegrid

 Tilegrid (im_files, nrows, ncols, filenames=True, tile_borders=True)

Create a Tilegrid object.