The reprojection error is a geometric error corresponding to the image distance between a projected point and a measured one. It is used to quantify how closely an estimate of a 3D point X ^ {\displaystyle {\hat {\mathbf {X} }}} recreates the point's true projection x {\displaystyle \mathbf {x} }. More precisely, let P {\displaystyle \mathbf {P} } be the projection matrix of a camera and x ^ {\displaystyle {\hat {\mathbf {x} }}} be the image projection of X ^ {\displaystyle {\hat {\mathbf {X} }}}, i.e. x ^ = P X ^ {\displaystyle {\hat {\mathbf {x} }}=\mathbf {P} \,{\hat {\mathbf {X} }}}. The reprojection error of X ^ {\displaystyle {\hat {\mathbf {X} }}} is given by d ( x , x ^ ) {\displaystyle d(\mathbf {x} ,\,{\hat {\mathbf {x} }})}, where d ( x , x ^ ) {\displaystyle d(\mathbf {x} ,\,{\hat {\mathbf {x} }})} denotes the Euclidean distance between the image points represented by vectors x {\displaystyle \mathbf {x} } and x ^ {\displaystyle {\hat {\mathbf {x} }}}.

Minimizing the reprojection error can be used for estimating the error from point correspondences between two images. Suppose we are given 2D to 2D point imperfect correspondences { x i ↔ x i ′ } {\displaystyle \{\mathbf {x_{i}} \leftrightarrow \mathbf {x_{i}} '\}}. We wish to find a homography H ^ {\displaystyle {\hat {\mathbf {H} }}} and pairs of perfectly matched points x i ^ {\displaystyle {\hat {\mathbf {x_{i}} }}} and x ^ i ′ {\displaystyle {\hat {\mathbf {x} }}_{i}'}, i.e. points that satisfy x i ^ ′ = H ^ x ^ i {\displaystyle {\hat {\mathbf {x_{i}} }}'={\hat {H}}\mathbf {{\hat {x}}_{i}} } that minimize the reprojection error function given by

∑ i d ( x i , x i ^ ) 2 + d ( x i ′ , x i ^ ′ ) 2 {\displaystyle \sum _{i}d(\mathbf {x_{i}} ,{\hat {\mathbf {x_{i}} }})^{2}+d(\mathbf {x_{i}} ',{\hat {\mathbf {x_{i}} }}')^{2}}

So the correspondences can be interpreted as imperfect images of a world point and the reprojection error quantifies their deviation from the true image projections x i ^ , x i ^ ′ {\displaystyle {\hat {\mathbf {x_{i}} }},{\hat {\mathbf {x_{i}} }}'}

  • Richard Hartley and Andrew Zisserman (2003). Multiple View Geometry in computer vision. Cambridge University Press. ISBN 0-521-54051-8.