`
`
`Jean-Claude Artonne et al.
`In re Patent of:
`6,844,990 B2 Attorney Docket No.: 50095-0114IP1
`U.S. Patent No.:
`January 18, 2005
`
`Issue Date:
`Appl. Serial No.: 10/706,513
`
`Filing Date:
`November 12, 2003
`
`Title:
`METHOD FOR CAPTURING AND DISPLAYING A VARIABLE
`RESOLUTION DIGITAL PANORAMIC IMAGE
`
`
`Mail Stop Patent Board
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`PETITION FOR INTER PARTES REVIEW OF UNITED STATES PATENT
`NO. 6,844,990 PURSUANT TO 35 U.S.C. §§ 311–319, 37 C.F.R. § 42
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`TABLE OF CONTENTS
`
`I.
`II.
`
`III.
`
`INTRODUCTION ........................................................................................... 1
`IPR REQUIREMENTS ................................................................................... 1
`A. Standing .................................................................................................... 1
`B. Challenge and Relief Requested ............................................................... 1
`C. Claim Construction ................................................................................... 2
`’990 PATENT .................................................................................................. 5
`A. Overview ................................................................................................... 5
`B. Relevant Prosecution History ................................................................. 12
`C. Panasonic IPR and Subsequent Reexamination ..................................... 13
`D. LG IPRs .................................................................................................. 14
`IV. LEVEL OF ORDINARY SKILL IN THE ART ........................................... 15
`V. APPLYING PRIOR ART TO CHALLENGED CLAIMS ........................... 15
`A. Ground 1: Baker in view of Shiota Renders Obvious Claims 2, 4, 27,
`and 29 ...................................................................................................... 15
`1.
`Baker (APPLE-1006) .................................................................... 15
`2.
`Shiota (APPLE-1012) ................................................................... 20
`3.
`Combination of Baker and Shiota ................................................. 22
`4. Analysis ......................................................................................... 27
`B. Ground 2: Baker in view of Shiota and Fisher Renders Obvious Claims
`29 and 30 ................................................................................................. 62
`1.
`Fisher (APPLE-1009) .................................................................... 62
`2.
`Combination of Baker, Shiota, and Fisher .................................... 65
`3. Analysis ......................................................................................... 69
`VI. DISCRETIONARY DENIAL IS NEITHER APPROPRIATE NOR
`EQUITABLE ................................................................................................. 70
`A. §314(a): The Fintiv Factors Favor Institution ........................................ 70
`1.
`Factor 1: Institution Will Increase Likelihood of Stay ................. 71
`2.
`Factor 2: The Final Written Decision will Issue Before Any
`Foreseeable Trial ........................................................................... 72
`Factor 3: Early Stage of Parallel Proceedings ............................... 72
`Factor 4: Under the Current Trial Schedule, No Overlap is
`Possible .......................................................................................... 72
`Factor 5: Apple’s involvement in Parallel Proceedings ................ 73
`Factor 6: Other Circumstances Support Institution ....................... 73
`
`3.
`4.
`
`5.
`6.
`
`i
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`B. §325(d): This Petition Applies New Arguments Not Previously
`Considered or Relied Upon by the Office .............................................. 73
`VII. CONCLUSION AND FEES ......................................................................... 79
`VIII. MANDATORY NOTICES UNDER 37 C.F.R § 42.8(a)(1) ......................... 79
`A. Real Party-In-Interest Under 37 C.F.R. § 42.8(b)(1) .............................. 79
`B. Related Matters Under 37 C.F.R. § 42.8(b)(2) ....................................... 79
`C. Lead And Back-Up Counsel Under 37 C.F.R. § 42.8(b)(3) ................... 80
`D. Service Information ................................................................................ 81
`
`
`
`
`
`
`ii
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`PETITIONER’S EXHIBIT LIST
`
`APPLE-1001
`
`U.S. Patent No. 6,844,990 to Artonne et al. (“the ’990 patent”)
`
`APPLE-1002
`
`Prosecution History of the ’990 Patent (“Original Prosecution
`History”)
`
`APPLE-1003
`
`Declaration of Dr. David Kessler, Ph.D.
`
`APPLE-1004
`
`Curriculum Vitae of Dr. David Kessler, Ph.D.
`
`APPLE-1005
`
`Service of complaint in ImmerVision, Inc. v. Apple, Inc., D.Del.
`1:21-cv-01733-MN-CJB (January 18, 2022)
`
`APPLE-1006
`
`U.S. Patent No. 5,686,957 to Baker (“Baker”)
`
`APPLE-1007
`
`“Fish Eye Lens” by K. Miyamoto (Feb. 19, 1964) (“Miyamoto”)
`
`APPLE-1008
`
`Service of complaint in ImmerVision, Inc. v. Apple, Inc., D.Del.
`1:21-cv-01484-MN-CJB (January 18, 2022)
`
`APPLE-1009
`
`U.S. Patent No. 3,953,111 to Fisher et al. (“Fisher”)
`
`APPLE-1010
`
`U.S. Patent No. 6,128,145 to Nagaoka (“Nagaoka”)
`
`APPLE-1011
`
`Reexamination File History for the ’990 Patent (“Reexam
`History”)
`
`APPLE-1012
`
`EP 1028389 A2 to Shiota et al. (“Shiota”)
`
`APPLE-1013
`
`APPLE-1014
`
`Rebiai et al., “Image Distortion from Zoom Lenses: Modeling
`and Digital Correction,” 1992 IBC International Broadcasting
`Convention (July 1992) (“Rebiai”)
`
`JP 2000-242773 to Matsui et al. (“Matsui”) (original and
`translated documents also provided in APPLE-1011, 43-51, 180-
`198, as provided by Patent Owner during reexamination)
`
`iii
`
`
`
`
`APPLE-1015
`
`APPLE-1016
`
`APPLE-1017
`
`APPLE-1018
`
`
`APPLE-1019
`
`
`APPLE-1020
`
`
`APPLE-1021
`
`
`APPLE-1022
`
`
`
`
`
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`U.S. Patent No. 5,861,999 to Tada (“Tada”)
`
`Zeljko Andreic, “Simple 180o field-of-view F-theta all-sky
`camera,” SPIE Proc. 1500 (Oct. 1, 1991) (“Andreic”)
`
`Abed Kassim et al., “Optical Performance of axial gradient and
`aspheric surface lenses: study and analysis,” SPIE’s 1994
`International
`Symposium
`on Optics,
`Imaging,
`and
`Instrumentation, Vol. 2263 (Sept. 30, 1994) (“Kassim”)
`
`Susan Houde-Walter, “Recent Progress In Gradient-Index
`Optics,” SPIE Proc. 0935 (Apr. 8, 1988) (“Houde-Walter”)
`
`Petition for Inter Partes Review of U.S. Patent No. 6,844,990,
`filed by Panasonic System Networks Co., Ltd. (IPR2014-01438)
`
`Paul K. Manhart et al., “Fundamentals of macro axial gradient
`index optical design and engineering,” SPIE Opt. Eng. 36(6),
`1607-1621 (June 1997) (“Manhart”)
`
`Vidal, “Interim Procedure for Discretionary Denials in AIA Post
`Grant Proceedings with Parallel District Court Litigation”
`(USPTO, June 2021)
`
`Scheduling Order entered in ImmerVision, Inc. v. Apple, Inc.,
`D.Del. 1:21-cv-01733-MN-CJB and ImmerVision, Inc. v. Apple,
`Inc., D.Del. 1:21-cv-01484-MN-CJB (entered on June 21, 2022)
`
`
`
`iv
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`LISTING OF CLAIMS
`
`Claim 1
`
`[1pre] A method for capturing a digital panoramic image
`
`[1a]
`
`[1b]
`
`[1c]
`
`by projecting a panorama onto an image sensor by means of a
`panoramic objective lens,
`
`the panoramic objective lens having an image point distribution
`function that is not linear relative to the field angle of object points of
`the panorama,
`
`the distribution function having a maximum divergence of at least
`±10% compared to a linear distribution function,
`
`[1d]
`
`such that the panoramic image obtained has at least one substantially
`expanded zone and at least one substantially compressed zone.
`Claim 27
`
`[27pre] A method for displaying a digital panoramic image, the method
`comprising:
`
`[27a]
`
`[27b]
`
`[27c]
`
`[27d]
`
`[27e]
`
`obtaining a digital panoramic image by projecting a panorama onto an
`image sensor using a panoramic objective lens,
`
`the panoramic objective lens having an image point distribution
`function that is not linear relative to a field angle of object points of the
`panorama,
`
`the distribution function having a maximum divergence of at least +/-
`10% compared to a linear distribution function,
`
`such that the panoramic image obtained has at least one substantially
`expanded zone and at least one substantially compressed zone, and
`
`displaying the obtained panoramic image by correcting the non-
`linearity of the initial image,
`
`v
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`correcting the non-linearity … performed by retrieving image points on
`the obtained image in a coordinate system of center O’ using at least the
`non-linear distribution function and a size L of the obtained image
`
`[27f]
`
`Claim 2
`
`[2pre] The method according to claim 27,
`
`[2a]
`
`wherein the objective lens has a non-linear distribution function that is
`symmetrical relative to the optical axis of the objective lens, the
`position of an image point relative to the center of the image varying
`according to the field angle of the corresponding object point
`
`Claim 4
`
`[4pre] The method according to claim 27,
`
`[4a]
`
`wherein the objective lens expands the edges of the image and
`compresses the center of the image.
`Claim 29
`
`[29pre] The method according to claim 27,
`
`[29a] wherein the objective lens comprises a set of lenses forming an
`apodizer.
`Claim 30
`
`[30pre] The method according to claim 29,
`
`[30a] wherein the set of lenses forming an apodizer comprises at least one
`aspherical lens.
`
`
`
`
`
`vi
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`I.
`
`INTRODUCTION
`
`Apple, Inc. (“Apple” or “Petitioner”) petitions for Inter Partes Review (“IPR”)
`
`of claims 2, 4, 27, 29, and 30 (“Challenged Claims”) of U.S. Patent 6,844,990 (“’990
`
`Patent”).
`
`II.
`
`IPR REQUIREMENTS
`A.
`
`Standing
`
`Apple certifies that the ’990 Patent is available for IPR. Apple is not barred
`
`or estopped from requesting this review on the grounds identified below. APPLE-
`
`1005; APPLE-1008.
`
`B. Challenge and Relief Requested
`
`Petitioner requests an IPR of the Challenged Claims on the grounds set forth
`
`below. Additional explanation/support for each ground is set forth in Dr. Kessler’s
`
`declaration (APPLE-1003, ¶¶43-259).
`
`Ground
`1
`
`’990 Patent Claims
`2, 4, 27, 29
`
`Basis
`§103: Baker in view of Shiota
`
`2
`
`29, 30
`
`§103: Baker in view of Shiota and
`
`Fisher
`
`
`
`The application that resulted in the ’990 Patent was filed on November 12,
`
` 1
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`2003, and is identified as a continuation of PCT Application No. PCT/FR02/01588,
`
`filed on May 10, 2002. APPLE-1001, Face. The ’990 Patent claims foreign priority
`
`to FR Application 01 06261, filed on May 11, 2001. APPLE-1001, Face. Thus, the
`
`earliest priority date of the ’990 Patent is May 11, 20011 (“Critical Date”)2.
`
`Each of the prior art references applied in Grounds 1-2 qualifies as prior art to
`
`the ’990 Patent on at least the bases shown below:
`
`Reference
`Baker
`(APPLE-1006)
`
`Shiota
`(APPLE-1012)
`
`Fisher
`(APPLE-1009)
`
`Date
`Filed: June 30, 1995
`Issued: November 11, 1997
`
`Filed: January 7, 2000
`Published: August 16, 2000
`
`Filed: November 4, 1974
`Issued: April 27, 1976
`
`
`
`
`C. Claim Construction
`
`Prior Art Status
`102(b)
`
`102(a)
`
`102(b)
`
`For the analysis presented herein, Petitioner is applying the constructions
`
`adopted in IPR2020-00195 for certain applicable claim terms in the Challenged
`
`
`In reexamination, Patent Owner contended that May 10, 2002 is the earliest
`
`1
`
`priority date for the ’990 Patent. APPLE-1011, 227-228.
`
`2
`
`Petitioner does not concede that the ’990 Patent is entitled to this priority date.
`
` 2
`
`
`
`
`
`
`Claims, as reflected below:
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`Claim Term
`
`Construction
`
`“panoramic objective lens”
`
`“a super-wide or ultra-wide objective
`
`lens”
`
`“object points of the panorama”
`
`“points of the object in the panorama
`
`being viewed by the lens”
`
`“image point”
`
`“a point of light projected by the lens
`
`onto an image plane, said light coming
`
`from the corresponding object point of a
`
`viewed object in the panorama”
`
`“field angle of object points”
`
`“the angles of incident light rays passing
`
`through the object points and through
`
`the
`
`center
`
`of
`
`the
`
`panorama
`
`photographed, relative to the optical
`
`axis of the objective lens”
`
`“maximum divergence”
`
`“Defined by
`
`the formula DIVmax
`
`%=[[dr(Pd)-dr(Pdl)]/[dr(Pdl)]]*100,
`
`where dr(Pd) is the relative distance
`
`from the center of the image to the point
`
` 3
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`of maximum divergence Pd, and dr(Pdl)
`
`is the relative distance in relation to the
`
`center of the corresponding point on the
`
`linear image distribution line”
`
`“expanded zone”
`
`“the portion of
`
`the
`
`image point
`
`distribution function where the gradient
`
`is higher than the gradient of the linear
`
`distribution function”
`
`“compressed zone”
`
`“the portion of
`
`the
`
`image point
`
`distribution function where the gradient
`
`is lower than the gradient of the linear
`
`distribution function”
`
`
`
`For the other claims terms of the Challenged Claims, because the evidence
`
`below and the prior art’s description of the claimed elements are similar to that of
`
`the ’990 specification, Petitioner believes that no claim constructions are necessary
`
`“to resolve [a] controversy” as to such claim terms. Wellman, Inc. v. Eastman Chem.
`
`Co., 642 F.3d 1355, 1361 (Fed. Cir. 2011). Petitioner is pursuing prior art-based
`
`invalidity grounds in this IPR and is not waiving arguments concerning invalidity
`
`grounds (e.g., §112) that cannot be raised in this proceeding.
`
` 4
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`III.
`
`’990 PATENT
`A. Overview
`
` The ’990 Patent is directed to “a method for capturing a digital panoramic
`
`image” using “a panoramic objective lens.” APPLE-1001, Abstract; APPLE-1003,
`
`¶¶75-94.
`
`The ’990 Patent’s Figure 5 (reproduced and annotated below) represents a
`
`conventional system “for taking panoramic shots,” and includes a panoramic
`
`objective lens 15 (yellow) having an optical axis OZ (orange) and a digital image
`
`sensor 17 (green) arranged in the image plane of the objective lens 15. APPLE-
`
`1001, 6:66-7:2.
`
`
`As shown, object points (e.g., a, b, c, d – red above) are projected through the
`
`panoramic objective lens and captured as respective image points (e.g., a’, b’, c’, and
`
`d’ – blue above) at the image sensor 17. APPLE-1001, 7:2-14. The field angle of
`
` 5
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`each of the object points is identified above as α1, α2, -α2, -α13 , and represents the
`
`“angle that an incident light ray passing through the object point considered and
`
`through the center of the panorama PM, marked by a point ‘p’ on FIG. 5, has relative
`
`to the optical axis OZ of the objective lens.” APPLE-1001, 7:2-10. On the image
`
`sensor 17, image points a’, b, c’, and d’ are located at distances from the center of
`
`the image, respectively, equal to d1, d2, -d2, -d1. APPLE-1001, 7:11-14.
`
`Per the ’990 Patent, the distribution of image points according to the field
`
`angle of the object points is linear in prior art panoramic objective lenses. APPLE-
`
`1001, 7:14-19. “Linearity” as used in the ’990 specification refers to a ratio of
`
`proportionality between the distance of an image point measured relative to the
`
`center of the image and the field angle of the corresponding object point. APPLE-
`
`1001, 7:24-28. Thus, the distances d1 and d2 are linked by the following relation:
`
`d1/α1=d2/α2. APPLE-1001, 7:20-22. Because the angle α1 for object point a is twice
`
`angle α2 for object point b (i.e., α1 = 2α2), distance d1 of the corresponding image
`
`point a’ will be twice the distance d2 of the corresponding image point b’ (i.e.,
`
`
`In Figure 5, angles α1 and - α1 should have been shown as being between “a”
`
`3
`
`and the axis OZ, and the angle between “d” and the axis OZ, respectively. APPLE-
`
`1003, ¶77 n. 1.
`
` 6
`
`
`
`
`
`
`d1=2d2). Id.
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`Because of such linearity in prior art panoramic objective lenses and the axial
`
`symmetry of panoramic objective lenses, the ’990 Patent states that image points
`
`corresponding to object points having an identical field angle form equally spaced
`
`concentric circles C10, C20 through C90 on an image disk 10, as shown in Figure
`
`4A (below). APPLE-1001, 1:30-32, 2:14-29, 18:17-23.
`
`
`Additionally, because of axial symmetry, the field angle of an object point is between
`
`0° and 90° for an objective lens having an aperture of 180°, e.g., as shown by the
`
`green semicircle below. APPLE-1001, 1:30-32, 2:22-23, 18:17-23, FIG. 4
`
`(annotated).
`
` 7
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`
`The purported invention of the ’990 Patent is a panoramic objective lens
`
`having an image point distribution function that is not linear relative to the field
`
`angle of object points, which creates at least one zone of the image that is expanded
`
`and at least another zone that is compressed. APPLE-1001, Abstract. Per the ’990
`
`Patent, the expanded zones of the image cover more pixels of the image sensor than
`
`if they were not expanded. APPLE-1001, 4:3-6.
`
`The ’990 Patent states that the portions of the image to expand and compress
`
`can be chosen according to the intended application. APPLE-1001, 7:66-8:5, 4:6-
`
`10. Figure 7A (below) illustrates one such application by providing an image
`
`distribution function of a panoramic objective lens that expands the center of the
`
`image:
`
` 8
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`
`APPLE-1001, Fig. 7A, 8:12-14. As seen by comparing Fig. 7A and 4A (the latter is
`
`obtained using a linear panoramic objective lens), the circles C10 and C20 are further
`
`from the center of the image and further from each other in FIG. 7A, while the circles
`
`C30 to C90 are closer to each other. APPLE-1001, 8:21-26. This panoramic image
`
`(shown in FIG. 7A) thus has an expanded zone in the center and a compressed zone
`
`towards the edge of the image disk. APPLE-1001, 8:26-28.
`
`Another way the ’990 Patent illustrates the expanded and compressed zones
`
`is by using a curve of a corresponding image point distribution function as shown in
`
`FIG. 7B (below), where the X-axis represents the field angle of the object point (in
`
`degrees) and the Y-axis represents the relative distance of an image point in relation
`
`to the center of the image. APPLE-1001, 8:29-38; APPLE-1003, ¶86.
`
` 9
`
`
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`
`APPLE-1001, Fig. 7B (annotated). As shown, the curve of the function Fd1 has a
`
`higher gradient than the straight or linear line Fdc for angles α between 0° and 20°,
`
`and a lesser gradient between 20° and 90°. APPLE-1001, 8:38-41. A higher
`
`gradient relative to the linear/straight line represents an expansion of the image and
`
`a lower gradient means a compression of the image. APPLE-1001, 8:41-43;
`
`APPLE-1003, ¶¶86-87; see also APPLE-1001, 9:58-64, Fig. 9, APPLE-1003, ¶¶90-
`
`91 (describing and showing an image distribution for an image with compressed,
`
`expanded, and compressed zones).
`
`The ’990 Patent describes using this curve to calculate a point of maximum
`
`divergence “Pd,” which is the point with the greatest gap in relative distance dr in
`
`relation to the corresponding point on the linear distribution straight line (Fdc).
`
`10
`
`
`
`
`APPLE-1001, 8:44-49.
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`Per the ’990 Patent, the maximum divergence DIVmax of the distribution
`
`function Fd1
`
`is calculated using
`
`the
`
`formula: DIVmax %=[[dr(Pd)-
`
`dr(Pdl)]/[dr(Pdl)]]*100, where dr(Pd) is the relative distance from the center of the
`
`image to the point of maximum divergence Pd, and dr(Pdl) is the relative distance
`
`from the center of the image to the linear distribution straight line, at the same field
`
`angle (here 20°). APPLE-1001, 8:49-65. Applying this formula, the maximum
`
`divergence at the field angle of 20° in Figure 7B is +125% (i.e., [[0.5-
`
`0.222]/[0.222]]*100). APPLE-1001, 9:1-2.
`
`The ’990 Patent states that the non-linearity in the image can be corrected,
`
`e.g., using the non-linear distribution function of the objective lens, to display an
`
`image that is “free from the optical distortion.” APPLE-1001, Abstract, 10:6-17. In
`
`one embodiment, the ’990 Patent’s correction method involves a two-step technique:
`
`(1) projecting the image points of an image sector corresponding to a display
`
`window (referred to as E(i, j), onto a sphere portion of a hemisphere of center O and
`
`of axes OX, OY, OZ, to obtain image points P(px, py, pz); and (2) projecting the
`
`determined image points P (i.e., P(px, py, pz)) onto an image disk (ID1) using the
`
`lens’s non-linear distribution function. APPLE-1001, 12:59-14:35, FIG. 13.
`
`
`11
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`
`As shown above, the projection of the image points P onto the image disk ID1 allows
`
`image points p(pu, pv) to be obtained on the image disk, in a coordinate system of
`
`center O’ (corresponding to the center of the image disk) and of axes O’U and
`
`O’V. APPLE-1001, 12:59-14:35, FIG. 13 (annotated above to show image disk and
`
`its center O’); APPLE-1003, ¶¶92-94.
`
`B. Relevant Prosecution History
`
`During original prosecution, Applicant cited Fisher (APPLE-1009), and
`
`foreign counterparts of Baker (APPLE-1006) and Nagaoka (APPLE-1010) (i.e., EP
`
`
`12
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`0 695 085 and EP 1 004 915, respectively). The Examiner issued a first-action
`
`allowance and noted in the reasons for allowance that the “prior art fails to teach …
`
`the claimed features … which include” the features recited in limitations [1b]-[1d]
`
`(which are similar in some respects to [27b]-[27d]). APPLE-1002, 232-233.
`
`C.
`
`Panasonic IPR and Subsequent Reexamination
`
`In September 2014, Panasonic System Networks Co., Ltd. (“Panasonic”) filed
`
`a petition for inter partes review of the ’990 Patent (“Panasonic IPR”), challenging
`
`claims 1-4, 6-7, 10, 11, 15-20, 22-23, and 25. APPLE-1019. Before an institution
`
`decision was entered, the parties in that proceeding sought—and the Board
`
`ordered—termination of that proceeding due to settlement. IPR2014-01438, Papers
`
`9 and 11.
`
`Patent Owner then requested ex parte reexamination of claims 1-4, 6-7, 10-
`
`11, 15-20, 22-23, and 25 of the ’990 Patent using the same art from the Panasonic
`
`IPR. APPLE-1011, 227-237. With the reexamination request, Patent Owner filed a
`
`preliminary amendment, in which it cancelled claims 1, 6-7, and 22, added new
`
`claims 27-48, and amended claims 2-4 and 15 (where claims 2-4 were amended to
`
`now depend from claim 27). APPLE-1011, 247.
`
`The Office granted the request for ex parte reexamination of the ’990 Patent,
`
`finding that the request raised substantial new questions (SNQ) of patentability with
`
`
`13
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`respect to claims for which reexamination was requested. APPLE-1011, 283-291.
`
`Baker, Fisher, and Shiota were among the references that were applied during
`
`reexamination and which formed, at least in part, the basis for the SNQ finding. Id.
`
`The Office subsequently issued a Notice of Intent to Issue Ex Parte Reexamination
`
`Certification (NIRC), finding, in part, that claims 2-4 (as amended to depend from
`
`new claim 27) and new claims 27-47 were allowable. APPLE-1011, 343-344.
`
`In its reasons for patentability, the Office stated that the “prior art of record,
`
`including … Baker, Fisher, Shiota ... , does not specifically disclose or fairly teach
`
`a method for capturing a digital panoramic image” including the features recited in
`
`limitations [27e]-[27f]. Id. The reexamination certificate then issued. APPLE-
`
`1011, 355-356; APPLE-1001, Ex Parte Reexamination Certificate; see APPLE-
`
`1003, ¶¶96-100.
`
`D. LG IPRs
`
`Petitioner LG Electronics Inc. filed two petitions (in IPR2020-00195 and
`
`IPR2020-00179) challenging the patentability of claims 5 and 21 in view of Tada
`
`(APPLE-1015) alone or in view of either Nagaoka or Baker. APPLE-1002, 342-
`
`420, 348, 387. In the FWDs, the Board found that LG’s petitioned grounds did not
`
`render claims 5 and 21 unpatentable. APPLE-1002, 380, 419. Neither of the LG
`
`IPRs addressed claim 27 or any claim depending therefrom (which are at issue here).
`
`
`14
`
`
`
`
`APPLE-1002, 374-379, 413-419.
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`IV. LEVEL OF ORDINARY SKILL IN THE ART
`
`A POSITA as of the ’990 Patent’s Critical Date would have had at least a
`
`Bachelor’s degree in Physics, Optical Engineering, and/or Electrical Engineering
`
`and at least five years’ experience in developing and designing optical imaging
`
`systems and have familiarity with image processing algorithms and optical design
`
`software. APPLE-1003, ¶¶24-25. Superior education could compensate for a
`
`deficiency in work experience, and vice-versa. Id.
`
`V. APPLYING PRIOR ART TO CHALLENGED CLAIMS
`A. Ground 1: Baker in view of Shiota Renders Obvious Claims 2, 4,
`27, and 29
`1.
`
`Baker (APPLE-1006)
`
`Baker discloses a “video conferencing, voice-directional video imaging
`
`system for automatic electronic video image manipulation of a selected, directional
`
`signal of a hemispheric conference scene ….” APPLE-1006, 5:16-24; APPLE-1003,
`
`¶¶106-115.
`
`Baker’s system includes an “imaging system” that includes a “camera having
`
`a capture lens” that captures a hemispherical or panoramic scene. Id., 5:37-47. The
`
`capture lens in Baker’s system “emphasizes the peripheral content of a
`
`hemispherical field of view” (id., 5:48-54) by “imaging the field of view to the image
`
`
`15
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`plane such that the ratio of the size of the smallest detail contained within the
`
`periphery of the scene to the size of the smallest resolving pixel of an image device
`
`is increased.” Id., 5:57-62. Specifically, Baker explains that the “panoramic image”
`
`provided by its imager “is ideally suited for teleconferencing” where the “important
`
`image information” is located “along a 10 to 30 degree … segment of the horizon,”
`
`which represents a 60° to 80° segment of the field. APPLE-1006, 5:37-47, 1:52-58;
`
`see id., 3-12-16 (“To obtain sufficient detail on the critical objects in the scene, the
`
`technique should differentiate between the relevant visual information along the
`
`horizon and the remaining visual information in the scene in order to provide greater
`
`resolution in areas of higher importance”); APPLE-1003, ¶¶107-113.
`
`An example of Baker’s capture lens as deployed in its imaging system is
`
`shown below:
`
`
`16
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`
`
`APPLE-1006, FIGS. 3BA-3BB. As shown, Baker’s lens includes a “multi-element
`
`refractive lens that is designed to capture a wide field of view” and a “hemispherical
`
`lens” (annotated above in red) that “is made of gradient index material” such that
`
`“points impinging the image plane incur different factors of magnification
`
`depending on what index of refraction and what portion of the hemispherical curve
`
`or lenselets they pass through.” APPLE-1006, 11:52-65; see id., 12:5-14 (explaining
`
`that, with the gradient index hemispherical element, index values decrease from the
`
`center such that n1<n2<n3<n4).
`
`Using Baker’s above lens, the peripheral portion of the scene is “map[ped] to
`
`
`17
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`a larger percentage of a given image detector area and, simultaneously, the mapped
`
`image of the central area of the scene … [is] minimized by the lens so that it does
`
`not interfere with the peripheral content now covering a wider annulus in the image
`
`plane.” Id., 5:55-6:1.
`
`Baker explains that its lens increases resolution in the peripheral portion of an
`
`image compared to a typical fisheye lens: “if an ordinary fisheye lens focuses the
`
`lowest 15 degrees up from the horizon on ten percent of the imager at the imaging
`
`plane and the peripheral-enhancing lens focuses that same 15 degrees on fifty
`
`percent of the imager, there is a five-fold increase in resolution using the same
`
`imaging device” and “[d]epending on the application and exact formulation of the
`
`lens equations, there will be at least a five times increase in resolving power by this
`
`lens/imager combination.” Id., 6:43-56; see id., 11:48-51, 12:33-55, 8:32-40, 5:37-
`
`47, 1:52-58.
`
`Baker’s lens projects an image on an image sensor (e.g., a CCD) of the camera
`
`and this image can be stored in memory or another storage device. APPLE-1006,
`
`13:9-18, 14:9-17. Subsequently, “[w]hen a portion of the scene is to be displayed,”
`
`Baker’s system retrieves the image information relating to the relevant portion of the
`
`scene from memory or another storage device. Id., 6:5-24. Baker explains that the
`
`image captured using its lens is in a “warped form” in which the image “warp[ing]”
`
`
`18
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`is caused in part by the selective magnification along the peripheral field of view.
`
`APPLE-1006, 14:18-26. Baker’s system compensates for this warping and displays
`
`a “corrected” version of the captured image with the “proper aspect of ratio for the
`
`human visual system (i.e., as a perspective corrected view)”:
`
`The transform processor subsystem compensates for the distortion or
`
`difference in magnification between the central and peripheral areas of
`
`the scene caused by the lens by applying appropriate correction criteria
`
`to bring the selected portion of the scene into standard viewing format.
`
`The transform processor subsystem can also more fully compensate for
`
`any aberrations of the enhanced peripheral image because of the
`
`image's improved resolution as it covers a larger portion of the image
`
`device (increased number of pixels used to detect and measure the
`
`smallest detail in the periphery image).
`
`Id., 6:5-24, 14:42-15:59, 14:23-26 (recreating a “proper display of the scene in two
`
`dimensions
`
`for perspective-correct viewing).
`
` Thus,
`
`to perform
`
`this
`
`correction/transformation—i.e., “compensate[] for the distortion or difference in
`
`magnification between the central and peripheral areas of the scene caused by the
`
`lens”—Baker’s transform processor applies “appropriate correction criteria to bring
`
`
`19
`
`
`
`
`Attorney Docket No. 50095-0114IP1
`IPR of U.S. Patent No. 6,844,990
`
`the selected portion of the scene into standard viewing format.” APPLE-1006, 6:12-
`
`16; see id., 6:16-24.
`
`2.
`
`Shiota (APPLE-1012)
`
`Like Baker, Shiota discloses an
`
`image
`
`transformation system for
`
`“transforming a fisheye image obtained by using a fisheye lens” into a “plane image
`
`for display.” APPLE-1012, Abstract
`
`Shiota explains that an “image produced by [a] fisheye lens is … distorted
`
`compared to … an image obtained by using [a] standard lens and is very hard for the
`
`operator to watch it.” Id., [0003]. Thus, Shiota’s system provides a two-step
`
`transformation process that transforms a “circular image obtained by using a fisheye
`
`lens into … a plan[ar] image” (id., [0004]), where the planar image is obtained by
`
`projecting points from the hemispherical objective lens surface onto a plane
`
`intersecting the lens surface at a point identifying the viewing direction from the
`
`origin of the camera. APPLE-1012, [0028]-[0032]; AP