🔖 Coplanar PNP

Coplanar PNP is a variant of the standard 📏 Perspective-N-Point problem with the additional guarantee that the 3D points lie on the same plane. If we let them all lie on the ground, , our goal is the following: given correspondences from world points to pixels and the camera intrinsics , find the rotation and translation of the camera.

The World Coordinates to Image Coordinates equation is given by

Assuming , we can write this as

Observe that this is a Projective Transformation with . The determinant is zero only when the camera lines in the ground plane .

We can find by solving the 🖼️ Homography as usual, but our problem now is that must have the first two columns be orthogonal unit vectors. Therefore, we have an extra few steps using 📎 Singular Value Decomposition.

Compute . We want to find

with the constraints and . We can do this by decomposing the first two columns of ,

Then, , , and . Finally, to fill in , we have .