`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2038
`Page 1
`
`

`

`Section 12.1. Reconstruction
`
`325
`
`in the algorithms for establishing stereo correspondences presented in Sections 12.2
`and 12.3.
`
`12.1.2
`
`Image Rectification
`
`The calculations associated with stereo algorithms are often considerably simplified
`when the images of interest have been rectified, i.e., replaced by two projectively
`equivalent pictures with a common image plane parallel to the baseline joining the
`two optical centers (Figure 12.5). The rectification process can be implemented
`by projecting the original pictures onto the new image plane. With an apropriate
`choice of coordinate system, the rectified epipolar lines are scanlines of the new
`images, and they are also parallel to the baseline.
`
`P
`
`O
`
`p
`
`l
`
`e
`
`p
`
`l
`
`O’
`
`p’
`
`l’
`
`’
`
`e’
`
`p’
`
`l’
`
`’
`
`Figure 12.5. A rectified stereo pair: the two image planes Π and Π(cid:1) are reprojected onto
`a common plane ¯Π = ¯Π(cid:1) parallel to the baseline. The epipolar lines l and l
`(cid:1) associated with
`
`(cid:1) in the two pictures map onto a common scanline ¯l = ¯l(cid:1) also parallel
`the points p and p
`to the baseline and passing through the reprojected points ¯p and ¯p(cid:1). The rectified images
`are easily constructed by considering each input image as a polyhedral mesh and using
`texture mapping to render the projection of this mesh into the plane ¯Π = ¯Π(cid:1).
`
`As noted in [?], there are two degrees of freedom involved in the choice of the
`rectified image plane: (1) the distance between this plane and the baseline, which
`is essentially irrelevant since modifying it will only change the scale of the rectified
`pictures, an effect easily balanced by an inverse scaling of the image coordinate
`axes, and (2) the direction of the rectified plane normal in the plane perpendicular
`to the baseline. Natural choices include picking a plane parallel to the line where
`the two original retinas intersect, and minimizing the distortion associated with the
`reprojection process.
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2038
`Page 2
`
`

`

`326
`
`Stereopsis
`
`Chapter 12
`
`In the case of rectified images, the notion of disparity introduced informally
`earlier takes a precise meaning: given two points p and p(cid:5) located on the same
`scanline of the left and right images, with coordinates (u, v) and (u(cid:5), v), the disparity
`is defined as the difference d = u(cid:5) −u. Let us assume from now on normalized image
`coordinates.
`If B denotes the distance between the optical centers, also called
`baseline in this context, it is easy to show that the depth of P in the (normalized)
`coordinate system attached to the first camera is z = −B/d (Figure 12.6).
`
`P
`
`d
`
`Q
`
`-z
`
`O
`
`v
`
`1
`
`u
`
`q
`
`p
`
`b
`
`b’
`
`O’
`
`v’
`
`1
`
`u’
`
`’
`
`q’
`
`p’
`
`Figure 12.6. Triangulation for rectified images: the rays associated with two points p
`(cid:1) on the same scanline are by construction guaranteed to intersect in some point P .
`and p
`As shown in the text, the depth of P relative to the coordinate system attached to the left
`(cid:1) − u. In particular, the preimage
`camera is inversely proportional to the disparity d = u
`of all pairs of image points with constant disparity d is a frontoparallel plane Πd (i.e., a
`plane parallel to the camera retinas).
`
`To show this, let us consider first the points q and q(cid:5) with coordinates (u, 0) and
`(u(cid:5), 0), and the corresponding scene point Q. Let b and b(cid:5) denote the respective
`distances between the orthogonal projection of Q onto the baseline and the two
`optical centers O and O(cid:5). The triangles qQq(cid:5) and OQO(cid:5) are similar, and it follows
`immediately that b = zu and b(cid:5) = −zu(cid:5). Thus B = −zd, which proves the result
`for q and q(cid:5). The general case involving p and p(cid:5) with v (cid:19)= 0 follows immediately
`from the fact that the line P Q is parallel to the two lines pq and p(cid:5)q(cid:5) and therefore
`also parallel to the rectified image plane. In particuliar, the coordinate vector of
`the point P in the frame attached to the first camera is P = −(B/d)p, where
`p = (u, v, 1)T is the vector of normalized image coordinates of p. This provides yet
`another reconstruction method for rectified stereo pairs.
`
`APPLE V COREPHOTONICS
`IPR2020-00905
`Exhibit 2038
`Page 3
`
`