Let’s continue our work on image alignment. We shall cover further details on image warping.

For what comes next, we’ll work a bit in Python. Import the following packages :

```
import cv2
import numpy as np
from matplotlib import pyplot as plt
```

So far, we saw how to :

- detect features (Harris corner detector, Laplacian of Gaussians for blobs, difference of Gaussians for fast approximation of the LOG)
- the properties of the ideal feature
- the properties of the feature descriptor
- matching of local features
- feature distance metrics

We’ll cover into further details image alignment.

# I. Image Warping

There is a geometric relationship between these 2 images :

An image warping is a change of domain of an image : . This might include translation, rotation or aspect change. These changes are said to be global parametric warping: , since the transformation can easily be described by few parameters and is the same for every input point.

To build the transformed image, we usually apply an inverse-warping :

- for every pixel in :
- compute the source location
- resample at location and copy to

This allows to be defined for all pixels in .

## 1. Linear transformations

We’ll now cover the different types of linear transformations that we can apply to an image using inverse-warping.

### a. Uniform Scaling

Scaling by factor :

### b. Rotation

Rotation by angle :

### c. 2D Mirror about the Y-axis

### d. 2D Miror accross line

### e. All 2D linear transformations

In summary, the linear transforms we can apply are :

- scale
- rotation
- shear
- mirror

The transformation should respect the following properties :

- origin maps origin
- lines map to lines
- parallel lines remain parallel
- ratios are preserved
- closed under composition

## 2. Translation

The trick is to add one more coordinate to build homogenous image coordinates.

## 3. Affine transformation

An affine transformation is any transformation that combines linear transformations and translations. For example :

In affine transformations, the origin does not always have to map the origin.

## 4. Homography

The Homography transform is also called projective transformation or planar perspective map.

Homographic transformations simply respect the following properties :

- lines map to lines
- closed under composition

The different transformations can be summarized this way :

With homographies, points at infinity become finite vanishing points.

# II. Computing transformations

Given a set of matches between images and , we must find the transform that best agrees with the matches.

## 1. Translation

The displacement of match is where and . We want therefore to solve :

We face an overdetermined system of equations, which can be solved by least squares.

The goal is to minimize the sum of squared residuals :

We can rewrite the problem in matrix form :

And the solution heads :

## 2. Affine transformation

We can write the residuals as :

And rewrite the cost function as :

Which can be rewritten in matrix form as :

Let’s develop a more general formulation. We have a parametric transformation. The Jacobian of the transformation with respect to the motion parameters determines the relationship between the amount of motion and the unknown parameters . We note that .

The sum of squared residuals is then :

The solution yields :

where :

- we define the Hessian
- and

Up to now, we considered only a perfect matching accuracy. It’s however only rarely the case. We can weight the least squares problem :

. If the are fixed, the solution to apply is : with a matix containing for each observation the noise level.

**Conclusion **: I hope this article on image alignment, transformations, and warping was helpful. Don’t hesitate to drop a comment if you have any question.

Like it? Buy me a coffee

## Leave a comment