A panorama is created via pure camera rotation around an axis through the projection center. Given panorama images, our goal is to project them onto one large image.
The problem is that by rotating our camera, we’re rotating the camera coordinate system. To project a point from the second image onto the first, we’ll have to account for the rotation.
We’ll solve the panorama for both planar and spherical projections.
Planar Projection
We’ll first try to project panorama images onto a single image plane called the mosaic. For a point in the world, we have the following:
To go from the second image to the first, we thus have
If we don’t use a relative rotation
Note that this transformation is scene-independent and does not depend on the depth of a point.
Info
If we have multiple images, we can instead compute homographies using point tracking; by tracking at least four points, our problem reduces to the 🖼️ Homography.
Spherical Projection
However, for a full panorama, we can’t project 360 degrees onto a single plane. Our mosaic then needs to be a sphere or cylinder, and we need to map world points onto the unit sphere.
Note that we would need to rotate our world point to the orientation of the sphere coordinate system,
Thus, to go from image to spherical coordinates, we have
where
If we wanted to unwrap the sphere, our coordinates would be
where the second tuple defines the unwrapping center.