IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`Valeo North America, Inc., Valeo S.A., Valeo GmbH,
`Valeo Schalter und Sensoren GmbH,
`and Connaught Electronics Ltd.
`Petitioners
`
`v.
`
`Magna Electronics, Inc.
`Patent Owner
`
`U.S. Patent No. 8,643,724
`Case IPR2015-To be Assigned
`
`
`Mail Stop PATENT BOARD
`Patent Trial and Appeal Board
`United States Patent and Trademark Office
`Madison Building (East)
`600 Dulany Street
`Alexandria, VA 22313
`
`
`
`PETITION FOR INTER PARTES REVIEW U.S. PATENT 8,643,724
`
`
`
`

`

`TABLE OF CONTENTS
`
`I.
`
`INTRODUCTION ...........................................................................................1
`
`II. MANDATORY NOTICES .............................................................................3
`
`A.
`
`Real Party-In-Interest ............................................................................3
`
`B.
`
`C.
`
`Related Matters .....................................................................................4
`
`Lead and Back-up Counsel ...................................................................4
`
`D.
`
`Service Information ...............................................................................4
`
`III.
`
`PAYMENT OF FEES ......................................................................................5
`
`IV. REQUIREMENTS FOR INTER PARTES REVIEW ......................................5
`
`A. Grounds for Standing ............................................................................5
`
`B.
`
`Identification of Challenge ...................................................................6
`
`1.
`
`2.
`
`The Specific Art and Statutory Ground(s) on Which the
`Challenge Is Based......................................................................6
`
`How the Construed Claims Are Unpatentable and Supporting
`Evidence Relied Upon to Support the Challenge .......................9
`
`V.
`
`FACTUAL BACKGROUND ..........................................................................9
`
`A. Declaration Evidence ............................................................................9
`
`B.
`
`C.
`
`The State of the Art ............................................................................ 13
`
`Summary of the ’724 Patent ............................................................... 16
`
`D.
`
`Summary of the Prosecution History ................................................. 18
`
`VI. BROADEST REASONABLE CONSTRUCTION ...................................... 19
`
`A.
`
`B.
`
`
`
`Construction of terms used in the challenged claims ........................ 20
`
`How the Construed Claims Are Unpatentable and Supporting
`Evidence Relied Upon to Support the Challenge .............................. 23
`
`ii
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`VII. GROUNDS OF UNPATENTABILITY ON WHICH PETITIONER IS
`LIKELY TO PREVAIL ................................................................................ 24
`
`A.
`
`B.
`
`C.
`
`D.
`
`E.
`
`Claims 49-56, 58, 61, 62, 64, 65-71, 73, 75-82, 84 and 86 are obvious
`over Nissan, Hino, Lemelson, Wang, and Aishin .............................. 24
`
`Claims 57, 72 and 83 are obvious over Nissan, Hino, Lemelson,
`Wang, Aishin, and Fuji. ..................................................................... 51
`
`Claim 59 is obvious over Nissan, Hino, Lemelson, Wang, Aishin, and
`Otsuka. ................................................................................................ 53
`
`Claims 60, 74, and 85 are obvious over Nissan, Hino, Lemelson,
`Wang, Aishin, and Paff. ..................................................................... 54
`
`Claim 63 is obvious over Nissan, Hino, Lemelson, Wang, Aishin, and
`King. ................................................................................................... 56
`
`VIII. CONCLUSION ............................................................................................. 57
`
`iii
`
`

`

`EXHIBIT LIST
`EXHIBIT LIST
`
`Ex. 1001
`EX.
`1001
`
`U.S. Pat. No. 8,643,724 to Schofield.
`US. Pat. No. 8,643,724 to Schofield.
`
`Ex. 1002
`EX.
`1002
`
`U.S Pat. No. 6,553,130 to Lemelson.
`US Pat. No. 6,553,130 to Lemelson.
`
`Ex. 1003
`EX.
`1003
`
`Japanese Publication No. JP3099952 assigned to Nissan Motor
`Japanese Publication No. JP3099952 assigned to Nissan Motor
`Co., Ltd.
`Co., Ltd.
`
`Ex. 1004
`EX.
`1004
`
`Certified English Translation of Japanese Publication
`Certified English Translation of Japanese Publication
`No. JP3099952 assigned to Nissan Motor Co., Ltd.
`No. JP3099952 assigned to Nissan Motor Co., Ltd.
`
`Ex. 1005
`EX.
`1005
`
`Japanese Publication No. JPA64-14700 assigned to Aishin Warner
`Japanese Publication No. JPA64—14700 assigned to Aishin Warner
`Kabushiki-Kaisha.
`Kabushiki-Kaisha.
`
`Ex. 1006
`EX.
`1006
`
`Certified English Translation of Japanese Publication No. JPA64-
`Certified English Translation of Japanese Publication No. JPA64—
`14700 assigned to Aishin Warner Kabushiki-Kaisha.
`14700 assigned to Aishin Warner Kabushiki-Kaisha.
`
`Ex. 1007
`EX.
`1007
`
`Japanese Publication No. 62-16073 assigned to Hino Motors Ltd.
`Japanese Publication No. 62-16073 assigned to Hino Motors Ltd.
`
`Ex. 1008
`EX.
`1008
`
`Certified English Translation of Japanese Publication No. 62-
`Certified English Translation of Japanese Publication No. 62-
`16073 assigned to Hino Motors Ltd.
`16073 assigned to Hino Motors Ltd.
`
`Ex. 1009
`EX.
`1009
`
`Wang, G., et al. "CMOS Video Cameras", IEEE, 1991, dated May
`Wang, G., et al. "CMOS Video Cameras", IEEE, 1991, dated May
`27-31, 1991.
`27-31, 1991.
`
`Ex. 1010
`EX.
`1010
`
`Great Britain Patent No. GB 2233530 assigned to Fuji Jukogyo
`Great Britain Patent No. GB 2233530 assigned to Fuji Jukogyo
`Kabushiki Kaisha.
`Kabushiki Kaisha.
`
`Ex. 1011
`EX.
`1011
`
`Japanese Publication No. H2-36417 assigned to Niles Co., Ltd.
`Japanese Publication No. H2—36417 assigned to Niles Co., Ltd.
`
`Ex. 1012
`EX.
`1012
`
`Certified English Translation of Japanese Publication No. H2-
`Certified English Translation of Japanese Publication No. H2—
`36417 assigned to Niles Co., Ltd.
`36417 assigned to Niles Co., Ltd.
`
`Ex. 1013
`EX.
`1013
`
`U.S Patent No. 5,444,478 to Lelong.
`US Patent No. 5,444,478 to Lelong.
`
`Ex. 1014
`EX.
`1014
`
`U.S. Patent No. 4,693,788 to King.
`US. Patent No. 4,693,788 to King.
`
`Ex. 1015
`EX.
`1015
`
`U.S. Patent No. 4,966,441 to Conner.
`US. Patent No. 4,966,441 to Conner.
`
`Ex. 1016
`EX.
`1016
`
`U.S. Patent No. 5,793,420 to Schmidt.
`US. Patent No. 5,793,420 to Schmidt.
`
`iV
`iv
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`Ex. 1017
`
`SAE Paper No. 871288 to Otsuka.
`
`Ex. 1018
`
`U.S. Patent No. 4,833,534 to Paff.
`
`Ex. 1019
`
`Expert Declaration of Dr. George Wolberg
`
`Ex. 1020
`
`Expert Declaration of Dr. Ralph Wilhelm
`
`Ex. 1021
`
`U.S. Patent No. 4,390,895 to Sato.
`
`Ex. 1022
`
`SAE Paper No. 890288 to Goesch.
`
`Ex. 1023
`
`Japanese Article “Television Image Engineering Handbook, The
`Institute of Television Engineers of Japan” (“JP Handbook”).
`
`Ex. 1024
`
`English Translation of Japanese Article “Television Image
`Engineering Handbook, The Institute of Television Engineers of
`Japan” (“JP Handbook”).
`
`Ex. 1025
`
`Dr. George Wolberg Curriculum Vitae.
`
`Ex. 1026
`
`Robert Nathan, Digital Video Data Handling, NASA JPL Tech
`Report 32-877, Pasadena, CA, Jan. 5, 1966.
`
`Ex. 1027
`
`Ex. 1028
`
`P. Burt et al., A Multiresolution Spline with Application to Image
`Mosaics, ACM Transactions on Graphics, Vol. 2. No. 4, Pages
`217-236, October 1983.
`
`Lisa Gottesfeld Brown, A Survey of Image Registration
`Techniques, vol. 24, ACM Computing Surveys, pp. 325-376,
`1992.
`
`Ex. 1029
`
`George Wolberg, Digital Image Warping, IEEE Computer Society
`Press, 1990.
`
`Ex. 1030
`
`N. Greene et al., Creating Raster Omnimax Images from Multiple
`Perspective Views Using the Elliptical Weighted Average Filter,
`IEEE Computer Graphics and Applications, vol. 6, no. 6, pp. 21-
`27, June 1986.
`
`Ex. 1031
`
`Richard Szeliski, Image Mosaicing for Tele-Reality Applications,
`DEC Cambridge Research Laboratory, CRL 94/2, May 1994.
`
`v
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`Ex. 1032
`
`G. Wolberg, “A Two-Pass Mesh Warping Implementation of
`Morphing,” Dr. Dobb’s Journal, no. 202, July 1993.
`
`Ex. 1033
`
`T. Porter and T. Duff, “Compositing Digital Images,” Computer
`Graphics (Proc. Siggraph), vol. 18, no. 3, pp. 253-259, July 1984.
`
`Ex. 1032
`
`G. Wolberg, “A Two-Pass Mesh Warping Implementation of
`Morphing,” Dr. Dobb’s Journal, no. 202, July 1993.
`
`Ex. 1033
`
`T. Porter and T. Duff, “Compositing Digital Images,” Computer
`Graphics (Proc. Siggraph), vol. 18, no. 3, pp. 253-259, July 1984.
`
`Ex. 1034
`
`Dr. Ralph V. Wilhelm Curriculum Vitae.
`
`Ex. 1035
`
`SAE Paper No. 750364 to Nolan.
`
`Ex. 1036
`
`SAE Paper No. 890282 to Corsi.
`
`Ex. 1037
`
`SAE Paper No. 890283 to Brandt.
`
`Ex. 1038
`
`SAE Paper No. 860173 to Ortega.
`
`Ex. 1039
`
`SAE Paper No. 930456 to Gumkowski.
`
`Ex. 1040
`
`U.S. Patent No. 6,693,524 to Payne.
`
`Ex. 1041
`
`SAE Paper No. 770274 to Smith.
`
`vi
`
`

`

`I.
`
`INTRODUCTION
`
`Petitioners Valeo North America, Inc., Valeo S.A., Valeo GmbH, Valeo
`
`Schalter und Sensoren GmbH, and Connaught Electronics Ltd. (collectively
`
`“Valeo” or “Petitioner”) respectfully request inter partes review of claims 49-86
`
`(the “challenged claims”) of U.S. Patent No. 8,643,724 (“the ’724 patent,”
`
`attached as Ex. 1001) in accordance with 35 U.S.C. §§ 311-319 and 37 C.F.R. §
`
`42.100 et seq.
`
`The ’724 patent is generally directed to a vision system for a motor vehicle.
`
`More particularly, the ’724 patent is directed to a multi-camera vision system for a
`
`vehicle that includes two or three vehicle-mounted image capture devices (i.e.,
`
`cameras), an image processor, and a display to provide the driver with perspective
`
`of the vehicle’s surroundings. (Ex. 1001 at 2:23-35; 2:59-3:221 and Fig. 8). The
`
`image processor synthesizes the image portions captured by each of the image
`
`capture devices.
`
` That image processing results in a synthesized image
`
`characterized by the absence of duplicate objects, which might otherwise appear in
`
`
`1 This petition cites to various exhibits by citing page (or column) and line number
`
`references (where applicable), as follows: “Ex. [No.] at [page/column]:[lines].”
`
`Citations to page numbers within non-patent publications refer to the document’s
`
`original pagination as displayed therein.
`
`1
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`images due to the cameras capturing images having overlapping fields of view.
`
`(Id. at 7:5-16; 7:44-57; Fig. 3). Ultimately, the driver views a synthesized image
`
`on the display inside the vehicle on a reconfigurable display device such that
`
`auxiliary information other than the synthesized image may be shown to the
`
`driver. The driver (or another user) may select the types of information displayed
`
`on the reconfigurable display.
`
`As demonstrated by various prior art references, motor vehicle vision
`
`systems and methods for processing image data to display a synthesized image to a
`
`driver of the vehicle and reconfigurable displays were well known to a person
`
`having ordinary skill in the art (hereinafter “PHOSITA”) well before May 22,
`
`1996, the earliest claimed priority date of the ’724 patent. Although some of these
`
`prior art references were made of record via an Information Disclosure Statement
`
`in the application that resulted in the ’724 patent (App. No. 13/800,691), the
`
`Examiner did not specifically cite or discuss any of these references during the
`
`prosecution of the ’724 patent.
`
`For example, JP3099952, assigned to Nissan Motor Co. (“Nissan,” certified
`
`English translation at Ex. 1004), discloses a vehicle surroundings monitoring
`
`system having one or a plurality of cameras installed in a vehicle, converting
`
`images input by the cameras to other coordinates by a perspective conversion,
`
`combining the converted images into one image related to an image of the vehicle
`
`2
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`itself, and a display for displaying the image to the vehicle occupants. Ex. 1004 at
`
`2:5-12. Hino discloses synthesizing image data captured by multiple cameras on a
`
`vehicle to depict a view from the perspective of a virtual camera located directly
`
`above the vehicle. Ex. 1008 at Fig. 3; 4:31-37. Further, Lemelson discloses a
`
`robust driver awareness system having, among other things, a forward-facing
`
`camera with a reconfigurable display on which various auxiliary information may
`
`be displayed, in addition to the video images captured by the camera. See Ex.
`
`1002 at 6:46-54; 15:45-53.
`
`For the reasons discussed in Section VII (below) and in the declarations of
`
`Dr. George Wolberg and Dr. Ralph Wilhelm, submitted in support of this petition,
`
`prior art references Nissan, Hino, and Lemelson, and other references described
`
`below render the challenged claims of the ’724 patent obvious.
`
`II. MANDATORY NOTICES
`
`Pursuant to 37 C.F.R. § 42.8(b), Petitioner provides the following
`
`mandatory disclosures.
`
`A. Real Party-In-Interest
`
`Pursuant to 37 C.F.R. § 42.8(b)(1), Petitioner certifies that Valeo North
`
`America, Inc., Valeo S.A., Valeo GmbH, Valeo Schalter und Sensoren GmbH, and
`
`Connaught Electronics Ltd. are the real parties-in-interest.
`
`3
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`B. Related Matters
`
`Pursuant to 37 C.F.R. § 42.8(b)(2), Petitioner states that the ’724 patent is
`
`involved in the litigation styled Magna Electronics Inc. v. Valeo, Inc., et al.,
`
`No. 2:14-cv-10540 (E.D. Mich. filed Feb. 5, 2014; Amended complaint filed
`
`March 19, 2014). This litigation remains pending. One of the three patents-in-suit
`
`is the ’724 Patent, attached as Ex. 1001.
`
`Petitioner will be filing a motion to stay the corresponding district court
`
`litigation pending the conclusion of the Inter Partes Review proceedings.
`
`C. Lead and Back-up Counsel
`
`Pursuant to 37 C.F.R. § 42.8(b)(3), Petitioner provides the following
`
`designation of counsel:
`
`Lead Counsel
`Tammy J. Terry (Reg. No.: 69,167)
`terry@oshaliang.com
`OSHA LIANG LLP
`909 Fannin Street, Suite 3500
`Houston, TX 77010-1034
`Phone: (713) 228-8600
`Fax: (713) 228-8778
`
`Back-up Counsel
`Seema Mehta (Reg. No.: 56,235)
`mehta@oshaliang.com
`Aly Dossa (Reg. No.: 63,372)
`dossa@oshaliang.com
`Peter C. Schechter (Reg. No.: 31,662)
`schechter@oshaliang.com
`OSHA LIANG LLP
`909 Fannin Street, Suite 3500
`Houston, TX 77010-1034
`Phone: (713) 228-8600
`Fax: (713) 228-8778
`
`Pursuant to 37 C.F.R. § 42.10(b), a Power of Attorney accompanies this
`
`Petition.
`
`D.
`
`Service Information
`
`4
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`Pursuant to 37 C.F.R. § 42.8(b)(4), papers concerning this matter should be
`
`served on lead and backup counsel whose service information is provided above.
`
`Petitioner consents to service of papers in this proceeding by e-mail.
`
`III. PAYMENT OF FEES
`
`The undersigned authorizes the Office to charge $35,800 to the deposit
`
`account designated and used for payment during e-filing as the fee required by 37
`
`C.F.R. §42.15(a) for this Petition for Inter Partes Review. Review of thirty-eight
`
`(38) claims is being requested, so an excess claims fee is included in this fee
`
`calculation. Any additional fees or credit of overpayment that might be due in
`
`connection with this Petition may be charged to Deposit Account No. 50-0591
`
`(Ref. No. 18096/008001).
`
`IV. REQUIREMENTS FOR INTER PARTES REVIEW
`
`As set forth below and pursuant to 37 C.F.R. § 42.104, each requirement for
`
`inter partes review of the ’724 patent is satisfied.
`
`A. Grounds for Standing
`
`Pursuant to 37 C.F.R. § 42.104(a), Petitioner hereby certifies that the ’724
`
`patent is available for inter partes review and that the Petitioner is not barred or
`
`estopped from requesting inter partes review challenging the claims of the ’724
`
`patent on the grounds identified herein. The ’724 patent has not been subject to a
`
`5
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`previous estoppel based proceeding of the America Invents Act (AIA), and the
`
`complaint was served within the last twelve months.
`
`B.
`
`Identification of Challenge
`
`Pursuant to 37 C.F.R. §§ 42.104(b)(1), Petitioner requests inter partes
`
`review and cancellation of claims 49-86 of the ’724 patent.
`
`1. The Specific Art and Statutory Ground(s) on Which the
`Challenge Is Based
`
`Pursuant to 37 C.F.R. § 42.104(b)(2), inter partes review of the ’724 patent
`
`is requested in view of the following references, each of which is prior art to the
`
`’724 patent under 35 U.S.C. § 102(a), (b), and/or (e):2
`
`(1)
`
`Japanese Publication No. JP3099952 assigned to Nissan Motor
`
`Co., Ltd., (“Nissan,” Ex. 1003 (certified English translation at Ex. 1004))
`
`published on April 25, 1991, from an application filed on September 12, 1989.
`
`Because it published more than one year before the filing date for the ’724 patent
`
`(May 22, 1996), Nissan is prior art to the ’724 patent under 35 U.S.C. § 102(b).
`
`See also Ex. 1019 at ¶¶ 62-71.
`
`(2)
`
`Japanese Publication No. 62-16073 assigned to Hino Motors Ltd.
`
`
`2 The pre-AIA versions of 35 U.S.C. §§ 102 and 103 apply to the claims of the
`
`pre-AIA ’724 patent.
`
`6
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`(“Hino,” Ex. 1007 (certified English translation at Ex. 1008)) published on April
`
`10, 1987, from an application filed on December 23, 1981. Because it published
`
`more than one year before the filing date for the ’724 patent, Hino is prior art to
`
`the ’724 patent under 35 U.S.C. § 102(b). See also Ex. 1019 at ¶¶ 72-75.
`
`(3) U.S Patent No. 6,553,130 to Lemelson (“Lemelson,” Ex. 1002) was
`
`filed on June 28, 1996, and claims benefit to a parent application filed on August
`
`11, 1993. Because Lemelson’s effective filing date (August 11, 1993) was before
`
`the ’724 patent’s earliest priority date (May 22, 1996), Lemelson is prior art to the
`
`’724 patent under 35 U.S.C. § 102(e). See also Ex. 1020 at ¶¶ 44-48.
`
`(4)
`
`Japanese Publication No. JP A64-14700 assigned to Aishin Warner
`
`Kabushiki-Kaisha (“Aishin,” Ex. 1005 (certified English translation at Ex. 1006))
`
`published on January 18, 1989, from an application filed on July 8, 1987. Because
`
`it published more than one year before the filing date for the ’724 patent, Aishin is
`
`prior art to the ’724 patent under 35 U.S.C. § 102(b). See also Ex. 1019 at ¶¶ 76-
`
`82.
`
`(5) Wang, G., et al. “CMOS Video Cameras”, IEEE, 1991, p. 100-103,
`
`(“Wang,” Ex. 1009) published on May 27-31, 1991. Because it published more
`
`than one year before the filing date for the ’724 patent, Wang is prior art to the
`
`’724 patent under 35 U.S.C. § 102(b). See also Ex. 1019 at ¶¶ 36, 83-85.
`
`7
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`(6) GB Patent No. 2,233,530 assigned to Fuji Jukogyo Kabushiki Kaisha
`
`(“Fuji,” Ex. 1010) published on January 9, 1991, from an application filed on May
`
`14, 1990. Because it published more than one year before the filing date for the
`
`’724 patent, Fuji is prior art to the ’724 patent under 35 U.S.C. § 102(b). See also
`
`Ex. 1020 at ¶¶ 49-52.
`
`(7) U.S. Patent No. 4,963,788 to King (“King,” Ex. 1014) issued on
`
`October 16, 1990. King is prior art to the ’724 patent under 35 U.S.C. § 102(b).
`
`See also Ex. 1020 at ¶¶ 53-56.
`
`(8) SAE Paper No. 871288 to Otsuka et al. (“Otsuka,” Ex. 1016)
`
`published on November 8, 1987. Because it published more than one year before
`
`the filing date for the ’724 patent, Otsuka is prior art to the ’724 patent under 35
`
`U.S.C. § 102(b). See also Ex. 1020 at ¶¶ 67-69.
`
`(9) U.S. Patent No. 4,833,534 to Paff (“Paff,” Ex. 1017) issued on May
`
`23, 1989. Paff is prior art to the ’724 patent under 35 U.S.C. § 102(b). See also
`
`Ex. 1020 at ¶¶ 64-66.
`
`The Challenged Claims of the’724 patent are unpatentable under 35 U.S.C.
`
`§ 103(a) as being obvious over the prior art. Specifically:
`
`(A) Claims 49, 50, 51, 52, 53, 54, 55, 56, 58, 61, 62, 64, 65, 66, 67, 68, 69, 70,
`
`71, 73, 75, 76, 77, 78, 79, 80, 81, 82, 84 and 86 are obvious over Nissan,
`
`Hino, Lemelson, Wang, and Aishin.
`
`8
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`(B) Claims 57, 72 and 83 are obvious over Nissan, Hino, Lemelson, Wang,
`
`Aishin, and Fuji.
`
`(C) Claim 59 is obvious over Nissan, Hino, Lemelson, Wang, Aishin, and
`
`Otsuka.
`
`(D) Claim 60, 74 and 85 are obvious over Nissan, Hino, Lemelson, Wang,
`
`Aishin, and Paff.
`
`(E) Claim 63 is obvious over Nissan, Hino, Lemelson, Wang, Aishin, and King.
`
`2. How
`the Construed Claims Are Unpatentable and
`Supporting Evidence Relied Upon to Support the Challenge
`
`Pursuant to 37 C.F.R. § 42.104(b)(4), an explanation of how the Challenged
`
`Claims of the ’724 patent are unpatentable under the statutory grounds identified
`
`above, including the identification of where each element of the claim is found in
`
`the prior art, is provided in Section VII below, in the form of claims charts.
`
`Pursuant to 37 C.F.R. § 42.104(b)(5), the Exhibit numbers of the supporting
`
`evidence relied upon to support the challenges and the relevance of the evidence
`
`to the challenges raised, including identifying specific portions of the evidence
`
`that support the challenges, are provided in Section VII below, in the form of
`
`claim charts.
`
`V.
`
`FACTUAL BACKGROUND
`
`A. Declaration Evidence
`
`9
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`This Petition is supported by the declarations of Dr. George Wolberg
`
`(attached as Ex. 1019) and Dr. Ralph Wilhelm (attached as Ex. 1020). Both
`
`experts offer their opinions with respect to the content and state of the prior art, as
`
`well as the subject matter of the ’724 patent.
`
`Dr. Wolberg is a professor of Computer Science at the City College of New
`
`York and has studied and worked in the field of Computer Science, specifically
`
`focusing on Image Processing, Computer Vision, and Computer Graphics, since
`
`1985. After earning his Ph.D. in Computer Science from Columbia University in
`
`1990, Dr. Wolberg joined the City College of New York as an assistant professor,
`
`where he has continued his research and teaching in image processing, computer
`
`graphics, and computer vision to the present day. Dr. Wolberg also is a Senior
`
`Member of the IEEE and a member of ACM SIGGRAPH, as well as a co-founder
`
`and Chief Technology Officer of a private company, Brainstorm Technology.
`
`In 1990, Dr. Wolberg published a leading monograph on Digital Image
`
`Warping, and over his career he has published over sixty (60) academic papers on
`
`image processing, computer vision, and computer graphics. Many of these works
`
`focus on image morphing, warping, registration, and scattered data interpolation,
`
`all of which are relevant to the ’724 patent. Dr. Wolberg also has written source
`
`code and image morphing and warping software that has been used by industry
`
`giants such as Adobe and Apple. Dr. Wolberg offers his opinions in this
`
`10
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`proceeding with respect to subject matter of the ’724 patent as it relates to vision
`
`systems and imaging technology at the time of the filing date of the ’724 patent.
`
`Dr. Ralph Wilhelm is currently the President of Wilhelm Associates, LLC,
`
`a consulting firm that founded in 2001 by Dr. Wilhelm. The firm specializes in
`
`automotive electronics, telematics, systems engineering, data communications
`
`between systems and devices, and product/market and business strategies. In this
`
`role, Dr. Wilhelm provides advice and assistance in the development and use of
`
`market assessment methodologies, product requirement definitions, product
`
`design, product and market strategy, and product implementation in his areas of
`
`technical expertise.
`
`Dr. Wilhelm received a Bachelor of Science degree in Electrical
`
`Engineering from Cornell University in 1967, a Doctor of Philosophy degree in
`
`Ceramic Engineering/Material Science from Rutgers University in 1972, an
`
`Executive Management Program certificate from the University of Illinois in 1985,
`
`and a Master of Business Administration degree in Operations and Strategy from
`
`the University of Michigan in 1987. He was a Senior Research Scientist from
`
`1971 to 1978 at General Motors Research Laboratories. Thereafter, from 1978 to
`
`1984, Dr. Wilhelm worked in General Motors Corporation’s AC Spark Plug
`
`Division as the Supervisor and Department Head of Materials Development.
`
`11
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`From 1984 to 2001, Dr. Wilhelm worked at and held various positions in
`
`the AC Spark Plug Division and Delphi Delco Electronics Corporation. Further,
`
`Dr. Wilhelm has authored dozens of published technical papers and delivered
`
`several keynote addresses concerning automotive electronic systems. Dr. Wilhelm
`
`also is a named inventor on three issued U.S. patents directed to methods of
`
`constructing automotive sensors. In the area of automotive display technologies
`
`for use in monitoring real time operating conditions in vehicles, Dr. Wilhelm has
`
`been involved in the development of and/or has experience with providing sensor
`
`alerts from oil pressure sensors, engine temperature sensors, door sensors, fuel
`
`sensors, engagement/disengagement of stability control systems, anti-skid braking
`
`(ABS) systems, traction control (TCS) systems, and others. In the area of
`
`automotive display technologies, Dr. Wilhelm has worked specifically with
`
`Vacuum Fluorescent Displays (VF), Cathode Ray Tubes (CRT), flat panel displays
`
`of different construction (e.g., LCD, etc.), Head Up Displays (HUD), and others.
`
`In the area of vehicle Central Processing Units (CPUs) and various control
`
`systems, Dr. Wilhelm has experience with navigation systems, audio systems,
`
`telematics systems, stability control systems, anti-skid braking systems, traction
`
`control systems, collision avoidance and collision warning systems, night vision
`
`systems, among others. In the area of electric vehicle (EV) systems, Dr. Wilhelm
`
`has led and supervised product teams developing and manufacturing a significant
`
`12
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`number of different EV products and systems, including those that focused on
`
`vehicle range and power consumption. Dr. Wilhelm offers his opinions with
`
`respect to subject matter of the ’724 patent as it relates to vision systems and
`
`reconfigurable display technology at the time of the filing date of the ’724 patent.
`
`B.
`
`The State of the Art
`
`The ’724 patent is directed to a “multi-camera vision system for a vehicle,”
`
`for capturing image data using at least two image capture devices that are mounted
`
`on the vehicle and have overlapping fields of view. Ex. 1001 at Abstract. In the
`
`’724 patent, outputs of the image capture devices are provided to an image
`
`processor that produces a synthesized image for display to the driver of the
`
`vehicle. Id. All of the features described in the ’724 patent were well known in
`
`the art prior to May 22, 1996, the earliest priority date for the ’724 patent. Ex.
`
`1020 at ¶ 22.
`
`The issues that arise when images are taken from different image capture
`
`devices with overlapping fields of view are as old as photography itself. “Image
`
`mosaicing” refers to the process of stitching together multiple input images to
`
`form one composite output image, while “image compositing” refers to the
`
`blending of overlapping regions between images. Mosaicing originated shortly
`
`after the development of the photographic process around 1840 and continued to
`
`13
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`develop throughout the 1900s with advances in aerial and satellite photography.
`
`Ex. 1019 at ¶ 31.
`
`As early as 1983, artisans in the field began developing various techniques
`
`to accomplish seamless stitching of images in mosaics, including image
`
`registration, image warping, and image morphing. Ex. 1019 at ¶¶ 32-35. Michael
`
`Jackson’s 1991 music video, “Black or White,” which exhibits a series of seamless
`
`transitions from one face to another, is a famous example of image morphing, a
`
`technique that maximizes the use of geometric transformations for image
`
`alignment that may occur by blending overlapping regions among multiple
`
`images. Id. at ¶ 34. Failure to accurately warp frames into alignment would have
`
`produced two sets of facial features upon blending. Id.
`
`By the 1990s, skilled artisans had recognized that CCD and CMOS cameras,
`
`among others, could be used on vehicles to capture video image data. See Ex.
`
`1009 and Ex. 1019 at ¶ 36. By that time, it also was well known that fisheye
`
`lenses on cameras would achieve wide angles of view that spanned in a variety of
`
`directions from the camera’s position, particularly since Nikon had released a
`
`camera with a fisheye lens decades earlier and since the predecessor to IMAX
`
`films had debuted in the 1970s. Id.
`
`Skilled artisans had also already discovered that cameras with overlapping
`
`fields of view would need to share a common center of projection to avoid
`
`14
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`parallax (the effect of the position or direction of an object appearing to differ
`
`when viewed from different positions). Various solutions for such issues were
`
`well-developed in the art long before May 1996. Id. at ¶¶ 38-39.
`
`In terms of displays used in vehicle applications, the first technologies used
`
`in fixed segment electronic displays included plasma displays in the form of Nixie
`
`Tubes and Light Emitting Diodes (LEDs), as used in electronic calculators and
`
`electronic watches. The 1960s and various market demands brought newer
`
`technologies such as Vacuum Fluorescent Displays (VFD) and Liquid Crystal
`
`Displays (LCD). Ex. 1020 at ¶¶ 23-24.
`
` The first reconfigurable displays were Cathode Ray Tubes (CRTs) and
`
`then, eventually, to address issues such as weight, depth, and heat generation,
`
`CRTs were replaced with various types and sizes of flat panel displays. Ex. 1019
`
`at ¶41; Ex. 1020 at ¶ 25. As information content increased and the capability of
`
`the reconfigurable displays improved, Automotive design engineers focused on
`
`developing reconfigurable displays that used a variety of different methods for the
`
`driver or passenger to interact with the display itself, including various “touch”
`
`mechanisms and “hands-free” or speech mechanisms. Id. Reconfigurable displays
`
`were connected to the camera system using serial or parallel bus structures. Ex.
`
`1020 at ¶ 27.
`
`15
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`In the early to mid-1980s, numerous technologies were considered as
`
`alternatives to the bulky CRT (e.g., active matrix liquid crystal displays, thin film
`
`electroluminescent displays, and light emitting diode displays, among others). Ex.
`
`1020 at ¶ 28. The choice often came down to a tradeoff between various
`
`considerations, including operating characteristics, the use of the reconfigurable
`
`flat panel, the amount of data to be shown, and human factors that guided the
`
`driver input devices, among others. Id at ¶ 26. By May 1996, skilled artisans
`
`were experienced in dealing with these characteristic factors of the available and
`
`emerging technologies. Id at ¶ 29. Those of ordinary skill in the art also were
`
`familiar with the various features of flat panel displays, including brightness,
`
`dimmability, and reflections, and the various options that had been developed to
`
`deal with these issues, including films and polarizers. The choices between these
`
`options were common and obvious design choices influenced by cost and
`
`consumer demand, not inventive events.
`
`For at least these reasons, and as described in more detail below and in the
`
`declarations of Dr. Wolberg (Ex. 1019) and Dr. Wilhelm (Ex. 1020), the state of
`
`the art was such that the system described in the ’724 patent would have been
`
`obvious to the PHOSITA in the art in early 1996.
`
`C.
`
`Summary of the ’724 Patent
`
`The ’724 patent generally describes a driver assist vision system for a motor
`
`16
`
`

`

`Petition for Inter Partes Review of U.S. Patent No. 8,643,724
`
`vehicle. Ex. 1001 at 2:59-3:22 and Fig. 1. More particularly, the ’724 patent
`
`descri