Let’s continue our work on image alignment. We shall cover further details on image warping.

For what comes next, we’ll work a bit in Python. Import the following packages :

import cv2
import numpy as np
from matplotlib import pyplot as plt

So far, we saw how to :

  • detect features (Harris corner detector, Laplacian of Gaussians for blobs, difference of Gaussians for fast approximation of the LOG)
  • the properties of the ideal feature
  • the properties of the feature descriptor
  • matching of local features
  • feature distance metrics

We’ll cover into further details image alignment.

image

I. Image Warping

There is a geometric relationship between these 2 images :

image

An image warping is a change of domain of an image : . This might include translation, rotation or aspect change. These changes are said to be global parametric warping: , since the transformation can easily be described by few parameters and is the same for every input point.

image

To build the transformed image, we usually apply an inverse-warping :

  • for every pixel in :
  • compute the source location
  • resample at location and copy to

This allows to be defined for all pixels in .

1. Linear transformations

We’ll now cover the different types of linear transformations that we can apply to an image using inverse-warping.

a. Uniform Scaling

Scaling by factor :

image

b. Rotation

Rotation by angle :

image

c. 2D Mirror about the Y-axis

d. 2D Miror accross line

e. All 2D linear transformations

In summary, the linear transforms we can apply are :

  • scale
  • rotation
  • shear
  • mirror

The transformation should respect the following properties :

  • origin maps origin
  • lines map to lines
  • parallel lines remain parallel
  • ratios are preserved
  • closed under composition

2. Translation

The trick is to add one more coordinate to build homogenous image coordinates.

image

3. Affine transformation

An affine transformation is any transformation that combines linear transformations and translations. For example :

In affine transformations, the origin does not always have to map the origin.

4. Homography

The Homography transform is also called projective transformation or planar perspective map.

Homographic transformations simply respect the following properties :

  • lines map to lines
  • closed under composition

The different transformations can be summarized this way :

image

With homographies, points at infinity become finite vanishing points.

image

II. Computing transformations

Given a set of matches between images and , we must find the transform that best agrees with the matches.

image

1. Translation

The displacement of match is where and . We want therefore to solve :

image

We face an overdetermined system of equations, which can be solved by least squares.

The goal is to minimize the sum of squared residuals :

We can rewrite the problem in matrix form :

image

And the solution heads :

2. Affine transformation

We can write the residuals as :

And rewrite the cost function as :

Which can be rewritten in matrix form as :

image

Let’s develop a more general formulation. We have a parametric transformation. The Jacobian of the transformation with respect to the motion parameters determines the relationship between the amount of motion and the unknown parameters . We note that .

image

The sum of squared residuals is then :

The solution yields :

where :

  • we define the Hessian
  • and

Up to now, we considered only a perfect matching accuracy. It’s however only rarely the case. We can weight the least squares problem :

. If the are fixed, the solution to apply is : with a matix containing for each observation the noise level.

**Conclusion **: I hope this article on image alignment, transformations, and warping was helpful. Don’t hesitate to drop a comment if you have any question.


Like it? Buy me a coffeeLike it? Buy me a coffee

Leave a comment