UNITED STATES PATENT AND TRADEMARK OFFICE
`_____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`VALEO NORTH AMERICA, INC., VALEO S.A., VALEO GMBH,
`VALEO SCHALTER UND SENSOREN GMBH,
`and CONNAUGHT ELECTRONICS LTD.
`Petitioners
`v.
`
`MAGNA ELECTRONICS INC.
`Patent Owner
`____________
`
`Case IPR2015-014101
`Patent 8,643,724
`__________________
`
`DECLARATION OF DR. RALPH ETIENNE-CUMMINGS
`IN SUPPORT OF PATENT OWNER RESPONSE
`
`
`
`
`
`Mail Stop “PATENT BOARD”
`Patent Trial and Appeal Board
`U.S. Patent & Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`
`1 Case IPR2015-01414 has been consolidated with this proceeding.
`
`
`Magna 2004
`Valeo v. Magna
`IPR2015-01410
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`
`TABLE OF CONTENTS
`Introduction ...................................................................................................... 1
`I.
`Qualifications ................................................................................................... 1
`II.
`III. Materials Considered ....................................................................................... 4
`IV. Overview Of The Law Used In This Declaration ........................................... 6
`A.
`Level of Skill in the Art ......................................................................... 7
`B.
`Obviousness ........................................................................................... 8
`C.
`Claim Construction..............................................................................10
`Instituted Grounds .........................................................................................10
`V.
`VI. Vehicle vision system technology background .............................................12
`VII. The ’724 patent ..............................................................................................13
`A. A synthesized image is generated without duplication of objects ......15
`B.
`Approximates a view as would be seen by a virtual camera at a single
`location ................................................................................................19
`VIII. The inventors of the ’724 patent constructively reduced the claimed
`invention to practice prior to Yamamoto’s date of availability as prior art. .21
`IX. The combination of Yamamoto, Mitsubishi, and Lemelson and the
`combination of Yamamoto, Mitsubishi, Lemelson, Wang, and Aishin fail to
`render the independent claims obvious. ........................................................31
`A.
`The cameras of Yamamoto will produce images that, when combined,
`exhibit the effects of parallax. .............................................................32
`Combined images that exhibit the effects of unresolved parallax will
`not be a synthesized image generated without duplication of objects.
` .............................................................................................................34
`Combined images that exhibit the effects of unresolved parallax will
`not be a synthesized image that approximates a view as would be seen
`by a virtual camera at a single location exterior of the equipped
`vehicle..................................................................................................40
`Experimentation confirms that the techniques of Yamamoto and
`Mitsubishi would not lead to the claimed synthesized image. ............44
`The combination of Yamamoto, Mitsubishi, Lemelson, and Goesch fails to
`render claim 45 obvious. ...............................................................................51
`XI. Conclusion .....................................................................................................53
`
`D.
`
`B.
`
`C.
`
`X.
`
`- i -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`
`Introduction
`
`I.
`I, Dr. Ralph Etienne-Cummings, hereby declare as follows:
`
`1. I understand that in response to Petitions submitted by Valeo North America,
`
`Inc., Valeo S.A., Valeo GmbH, Valeo Schalter und Sensoren GmbH, and
`
`Connaught Electronics Ltd. (collectively “Valeo”), the Patent Trial and Appeal
`
`Board (“Board”) instituted an inter partes review as to claims 1, 3–12, 14, 15,
`
`17, 19–52, 54–67, 69–79, and 81–86 (“instituted claims”) of U.S. Patent No.
`
`8,643,724 (“the ’724 patent”). I understand that the ’724 patent is titled “Multi-
`
`Camera Vision System for a Vehicle” by Kenneth Schofield, Mark L. Larson,
`
`and Keith J. Vadas and that the ’724 patent is currently assigned to Magna
`
`Electronics Inc. (“Magna”).
`
`2. I have been retained on behalf of Magna to provide expert opinions in
`
`connection with this inter partes review proceeding. Specifically, I have been
`
`asked to provide my expert opinion relating to the patentability of claims 1, 3–
`
`12, 14, 15, 17, 19–52, 54–67, 69–79, and 81–86 of the ’724 patent) relative to
`
`the instituted grounds.
`
`II. Qualifications
`3. I am an expert in the field of computer vision, having designed and
`
`implemented image sensors and vision algorithms hardware and software.
`
`
`
`- 1 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`4. Currently, I am Chairman and Professor of Electrical and Computer
`
`Engineering at the Johns Hopkins University in Baltimore, MD.
`
`5. I am also the Director of the Computational Sensory Motor Systems Lab at the
`
`Johns Hopkins University.
`
`6. I am a founding member of the Laboratory for Computational Sensing and
`
`Robotics at the Johns Hopkins University.
`
`7. I received my B.S. degree in Physics in 1988, from Lincoln University,
`
`Pennsylvania. I completed my M.S.E.E. and Ph.D. degrees in Electrical
`
`Engineering at the University of Pennsylvania in 1991 and 1994, respectively.
`
`8. From August 1998 to July 2002, I was an Assistant Professor of Electrical and
`
`Computer Engineering at the Johns Hopkins University. From July 2002 to July
`
`2008, I was an Associate Professor. During my first four years, I was Director
`
`of Computer Engineering at Johns Hopkins University and the Institute of
`
`Neuromorphic Engineering. I was promoted to Professor in July 2008.
`
`9. I am a recipient of the National Science Foundation’s Career and Office of
`
`Naval Research Young Investigator Program, Kavli Frontiers Fellowship and
`
`Fulbright Fellowship Awards.
`
`
`
`- 2 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`10. I have won numerous best paper awards from the Institute of Electrical and
`
`Electronic Engineering (IEEE) for articles in IEEE journals and conferences for
`
`my work on computer vision systems, robotics and neuroprosthetics.
`
`11. I am an IEEE Fellow, an honor bestowed on the top 0.1% of IEEE members,
`
`for contributions to “neuromorphic sensory-motor systems.”
`
`12. I am a former Topic Editor of the IEEE Sensors Journal and the former Deputy
`
`Editor in Chief of the IEEE Transactions on Biomedical Circuits and Systems. I
`
`am an Associate Editor of IEEE Transactions on Biomedical Circuits and
`
`Systems, Frontiers in Neuromorphic Engineering and Journal of Low Power
`
`Electronics and Applications.
`
`13. I have expertise in mixed signal VLSI systems, CMOS image sensors,
`
`computational sensors, computer vision, neuromorphic engineering, smart
`
`structures, mobile robotics, legged locomotion, and neuroprosthetics based on
`
`education, research, and industrial experience.
`
`14. The vision systems that I have developed can be used in the automotive context.
`
`I have tested my vision systems in vehicles operating in real-world conditions.
`
`
`
`- 3 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`
`III. Materials Considered
`15. In forming my expert opinions expressed in this declaration, I have considered
`
`and relied upon my education, background, and experience. I reviewed the
`
`Petitions filed by Valeo along with relevant exhibits to the Petitions.
`
`16. I have reviewed the specification of the ’724 patent. I understand that the ’724
`
`patent is a continuation of U.S. Appl. No. 12/688,146, filed January 15, 2010,
`
`now U.S. Pat. No. 8,842,176, which is a continuation of U.S. Appl. No.
`
`12/496,357, filed July 1, 2009, now U.S. Pat. No. 8,462,204, which is a
`
`continuation of U.S. Appl. No. 11/122,880, filed May 5, 2005, now U.S. Pat.
`
`No. 7,561,181, which is a continuation of U.S. Appl. No. 10/324,679, filed
`
`December 20, 2002, now U.S. Pat. No. 6,891,563, which is a continuation of
`
`U.S. Appl. No. 08/952,026, filed as application no. PCT/US96/07382 on May
`
`22, 1996, now U.S. Pat. No. 6,498,620.
`
`17. I understand that the ’724 patent has been provided as Exhibit 1001. I will cite
`
`to the specification using the following format: (’724 patent, 1:1–10). This
`
`example citation points to the ’724 patent specification at column 1, lines 1–10.
`
`18. I have also reviewed and am familiar with the following documents:
`
`
`
`
`
`- 4 -
`
`

`
`1001
`1003
`
`1005
`
`1006
`1008
`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`
`Description
`
`Paper /
`Exhibit #
`-1410 1 Petition for Inter Partes Review (Pet.)
`-1410 6 Patent Owner Preliminary Response (POPR)
`-1414 1 Petition for Inter Partes Review (1414 Pet.)
`-1414 7 Patent Owner Preliminary Response (1414 POPR)
`-1410 7 Decision on Institution of Inter Partes Review (dated December
`28, 2015) (Inst. Dec.)
`U.S. Patent No. 8,643,724 to Schofield et al. (’724 patent)
`Certified English Translation of Japanese Publication No. JP H7-
`30149 (Yamamoto)
`Certified English Translation of Japanese Publication No. H2-
`117935 (Mitsubishi)
`U.S. Patent No. 6,553,130 (Lemelson)
`Certified English Translation of Japanese Publication No. JPA64-
`14700 (Aishin)
`1009 Wang, G., et al. “CMOS Video Cameras” (Wang)
`1010
`Great Britain Patent No. GB 2233530 (Fuji)
`Certified English Translation of Japanese Publication No. H2-
`1012
`36417 (Niles)
`1013
`U.S. Patent No. 4,963,788 (King)
`1014
`U.S. Patent No. 4,966,441 (Conner)
`1015
`U.S. Patent No. 5,793,420 (Schmidt)
`1016
`SAE Paper No. 871288 (Otsuka)
`1017
`U.S. Patent No. 4,833,534 (Paff)
`1018
`U.S. Patent No. 4,390,895 (Sato)
`1019
`SAE Paper No. 890288 (Goesch)
`1020
`Expert Declaration of George Wolberg (Wolberg Decl.)
`1022
`Expert Declaration of Ralph Wilhelm (Wilhelm Decl.)
`2001 Magna’s Constructive Reduction to Practice Chart
`2002
`U.S. Patent No. 5,670,935 (Schofield)
`2003
`File Wrapper of U.S. Patent No. 8,643,724
`2005
`Inventor Declaration for U.S. Patent No. 5,670,935
`
`
`
`- 5 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`
`
`19. I have been asked to provide my technical review, analysis, insights, and
`
`opinions regarding the ’724 patent and the above-noted references that form the
`
`basis for the grounds of unpatentability set forth in the Petitions and the
`
`Institution Decision.
`
`IV. Overview Of The Law Used In This Declaration
`20. When considering the ’724 patent and stating my opinions, I am relying on
`
`legal principles that have been explained to me by counsel.
`
`21. I understand that for a claim to be found patentable, the claims must be, among
`
`other requirements, novel and nonobvious from what was known at the time of
`
`the invention. While the time of invention is presumptively the earliest effective
`
`filing date, i.e., the earliest priority date of the ’724 patent—May 22, 1996, the
`
`time of invention can be shown to be sooner based on actual or constructive
`
`reduction to practice. Constructive reduction to practice occurs with a filing of a
`
`patent application that discloses the invention.
`
`22. I understand that the information that is used to evaluate whether a claim is
`
`novel and nonobvious is referred to as prior art.
`
`
`
`- 6 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`23. I understand that in this proceeding Valeo has the burden of proving that the
`
`claims of the ’724 patent are rendered obvious by the alleged prior art
`
`references.
`
`A. Level of Skill in the Art
`24. I have been asked to consider the level of ordinary skill in the art that someone
`
`would have had in May 1995—due to a May 1995 constructive reduction to
`
`practice—and in May 1996—due to a May 1996 earliest priority date. In my
`
`opinion, the level of skill in May 1995 and May 1996 was largely the same and
`
`my opinion is the same for either date. With over 25 years of experience in
`
`electrical engineering and imaging systems and having educated individuals at
`
`the bachelor’s, master’s, Ph.D., and post-doctoral levels, I am well informed
`
`with the level of ordinary skill, which takes into consideration:
`
`• levels of education and experience of persons working in the field;
`
`• types of problems encountered in the field; and
`
`• sophistication of the technology.
`
`
`
`25. Based on the technologies disclosed in the ’724 patent and the considerations
`
`listed above, a person having ordinary skill in the art (“POSA”) would have had
`
`at least a bachelor’s degree in electrical, electronic, or mechanical engineering,
`
`
`
`- 7 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`or computer science, or equivalent experience, and at least two years of
`
`experience in the relevant field, such as vision systems for real-world
`
`applications, such as encountered in vehicular vision systems. Less education
`
`could be compensated by more direct experience and vice versa.
`
`26. Throughout my declaration, even if I discuss my analysis in the present tense, I
`
`am always making my determinations based on what a POSA would have
`
`known at the time of invention. Additionally, throughout my declaration, even
`
`if I discuss something stating “I,” I am referring to a POSA’s understanding.
`
`B. Obviousness
`27. It is my understanding that the claims may be “anticipated” and thus
`
`unpatentable if a single prior art reference teaches each and every limitation
`
`recited in the claim. It is my understanding that the proposed ground in this
`
`inter partes review, however, is based on obviousness, not anticipation. I
`
`understand that a patent claim is invalid if the claimed invention would have
`
`been obvious to a POSA at the time of invention. This means that even if all of
`
`the requirements of the claim cannot be found in a single prior art reference that
`
`would anticipate the claim, the claim can still be invalid.
`
`28. I understand that to obtain a patent, a claimed invention must have, as of the
`
`time of invention, been nonobvious in view of the prior art in the field. I
`
`
`
`- 8 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`understand that an invention is obvious when the differences between the
`
`subject matter sought to be patented and the prior art are such that the subject
`
`matter as a whole would have been obvious at the time the invention was made
`
`to a person having ordinary skill in the art.
`
`29. I understand that to prove that prior art or a combination of prior art renders a
`
`patent obvious, it is necessary to (1) identify the particular references that,
`
`singly or in combination, make the patent obvious; (2) specifically identify
`
`which elements of the patent claim appear in each of the asserted references;
`
`and (3) explain how the prior art references could have been combined in order
`
`to create the inventions recited in the patent claims.
`
`30. I understand that certain objective indicia can be important evidence regarding
`
`whether a patent is obvious or nonobvious. Such indicia include: commercial
`
`success of products covered by the patent claims; a long-felt need for the
`
`invention; failed attempts by others to make the invention; copying of the
`
`invention by others in the field; unexpected results achieved by the invention as
`
`compared to the closest prior art; praise of the invention by the infringer or
`
`others in the field; the taking of licenses under the patent by others; expressions
`
`of surprise by experts and those skilled in the art at the making of the invention;
`
`and the patentee proceeding contrary to the accepted wisdom of the prior art.
`
`
`
`- 9 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`31. I also understand that in this proceeding, Valeo has the burden of proving the
`
`obviousness of the challenged claims by a preponderance of the evidence. In
`
`other words, Valeo must prove that the challenged claims are more likely to be
`
`obvious than not.
`
`C. Claim Construction
`32. I understand that words of the claims are generally given their ordinary and
`
`customary meaning in the art field as understood by a POSA at the time of
`
`invention. In concluding how a POSA would interpret words of the claims, I
`
`have considered the words of the claims themselves, their context, the
`
`remainder of the specification, and the prosecution history.
`
`Instituted Grounds
`
`V.
`33. I understand that the Board has instituted trial of claims 1, 3–6, 10–12, 14, 15,
`
`17, 23, 25, 29–32, 41–43, and 46–48 under 35 U.S.C. § 103(a) as unpatentable
`
`over Yamamoto, Mitsubishi, and Lemelson; claims 49–52, 54–56, 58, 61, 62,
`
`64–67, 69–71, 73, 75–79, 81, 82, 84, and 86 under 35 U.S.C. § 103(a) as
`
`unpatentable over Yamamoto, Mitsubishi, Lemelson, Wang, and Aishin; claim
`
`19 under 35 U.S.C. § 103(a) as unpatentable over Yamamoto, Mitsubishi,
`
`Lemelson, and Wang; claims 7–9 and 20–22 under 35 U.S.C. § 103(a) as
`
`unpatentable over Yamamoto, Mitsubishi, Lemelson, and Aishin; claim 24
`
`under 35 U.S.C. § 103(a) as unpatentable over Yamamoto, Mitsubishi,
`
`
`
`- 10 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`Lemelson, and Niles; claim 26 under 35 U.S.C. § 103(a) as unpatentable over
`
`Yamamoto, Mitsubishi, Lemelson, Aishin, and Schmidt; claims 27 and 28
`
`under 35 U.S.C. § 103(a) as unpatentable over Yamamoto, Mitsubishi,
`
`Lemelson, and Fuji; claims 33 and 35–38 under 35 U.S.C. § 103(a) as
`
`unpatentable over Yamamoto, Mitsubishi, Lemelson, and Otsuka; claim 34
`
`under 35 U.S.C. § 103(a) as unpatentable over Yamamoto, Mitsubishi,
`
`Lemelson, Otsuka, and Conner; claim 39 under 35 U.S.C. § 103(a) as
`
`unpatentable over Yamamoto, Mitsubishi, Lemelson, Otsuka, and Sato; claim
`
`40 under 35 U.S.C. § 103(a) as unpatentable over Yamamoto, Mitsubishi,
`
`Lemelson, Otsuka, and Paff; claim 44 under 35 U.S.C. § 103(a) as unpatentable
`
`over Yamamoto, Mitsubishi, Lemelson, and King; claim 45 under 35 U.S.C. §
`
`103(a) as unpatentable over Yamamoto, Mitsubishi, Lemelson, and Goesch;
`
`claims 57, 72, and 83 under 35 U.S.C. § 103(a) as unpatentable over
`
`Yamamoto, Mitsubishi, Lemelson, Wang, Aishin, and Fuji; claim 59 under 35
`
`U.S.C. § 103(a) as unpatentable over Yamamoto, Mitsubishi, Lemelson, Wang,
`
`Aishin, and Otsuka; claims 60, 74, and 85 under 35 U.S.C. § 103(a) as
`
`unpatentable over Yamamoto, Mitsubishi, Lemelson, Wang, Aishin, and Paff;
`
`and claim 63 under 35 U.S.C. § 103(a) as unpatentable over Yamamoto,
`
`Mitsubishi, Lemelson, Wang, Aishin, and King.
`
`
`
`- 11 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`
`VI. Vehicle vision system technology background
`34. Vision systems are generally designed for a specific application and thus must
`
`be adapted to the environment of that specific application. Due to the ever-
`
`changing environment of vehicular vision systems, these systems must account
`
`for numerous application-specific constraints, including weather conditions
`
`(e.g., fog, rain, snow), dynamic lighting (e.g., shadows, daytime, nighttime,
`
`streetlamps), complex surroundings, and safety concerns. Vehicular vision
`
`systems capture information about the vehicle surroundings to assist the vehicle
`
`operator in driving safely.
`
`35. A purpose of vehicular vision systems is to “reduce the amount of time spent
`
`gathering information of the condition around the vehicle.” (’724 patent, 1:28–
`
`30.) “One difficulty with proposed systems has been that they present a large
`
`amount of visual information in a manner which is difficult to comprehend.”
`
`(Id. at 1:61–64.) For example, to provide the driver with an increased field of
`
`view, some vehicular camera systems utilized a wide-angle lens. However, the
`
`wide-angle lens “introduce[d] distortion of the scene and further impair[ed] the
`
`ability of the driver to judge distances of objects displayed.” (Id. at 2:29–32.)
`
`36. An alternative way to provide the driver with an increased field of view is to
`
`utilize multiple cameras, with each camera positioned at a different location
`
`around the vehicle. Under this approach, one object may be captured by
`
`
`
`- 12 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`multiple cameras. (Id. at 1:64–2:2.) But with cameras “positioned at different
`
`longitudinal locations on the vehicle, objects behind the vehicle are at different
`
`distances from the image capture devices” leading to “different image sizes for
`
`the same object.” (Id. at 2:4–8.)
`
`37. The different positions of the cameras result in disparities, distortions, and
`
`duplications in the content of the images produced by the individual cameras.
`
`Thus, combining these images to increase the field of view produced confusing
`
`and often contradictory image portions. These combined images made it harder
`
`for the driver to understand the environment, in part due to duplications and
`
`distortions. Hence, the problem of reducing the amount of time spent gathering
`
`information about the condition around the vehicle remained unresolved.
`
`VII. The ’724 patent
`38. The ’724 patent relates generally to multi-camera vision systems for vehicles.
`
`(Id. at Abstract.) More particularly, the ’724 patent is directed towards rearview
`
`imaging systems that utilize image capture devices to provide scenic
`
`information rearward of the vehicle to the vehicle operator. (Id. at 1:22–27.)
`
`The systems resolved the long-felt need in the art by providing a synthesized
`
`image from the multiple cameras without duplication of objects that
`
`approximated a view as would be seen by a single virtual camera. (See id. at
`
`5:63–6:5.)
`
`
`
`- 13 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`39. The ’724 patent provided a system that included “at least two side image
`
`capture devices 14 positioned, respectively, on opposite sides of vehicle 10 and
`
`a center image capture device 16 positioned on the lateral centerline of the
`
`vehicle.” (Id. at 5:54–57.) The system also included an “image processor 18 for
`
`receiving data signals from image capture devices 14, 16 and synthesizing, from
`
`the data signals, a composite image 42 which is displayed on a display 20.” (Id.
`
`at 5:59–62.) The resulting image “approximates the view from a single virtual
`
`image capture device” and “provides a substantially seamless panoramic view
`
`rearwardly of the vehicle without duplicate or redundant images of objects.”
`
`(Id. at 5:65–6:5.)
`
`40. The system of the ’724 patent partially compensates for duplication of objects
`
`by adjusting a field of view of image capture device 16 (id. at 6:66–7:43), and
`
`the system also compensates for duplication of objects by processing the image
`
`from image capture device 16 differently than image capture devices 14 (id. at
`
`14:52–15:44). The combination of these techniques allows for the image
`
`processor to generate a synthesized image without duplication of objects that
`
`approximates a view from a single virtual camera. The image from the virtual
`
`camera position therefore became less confusing to the driver, allowing the
`
`driver to more quickly understand the conditions around the vehicle.
`
`
`
`- 14 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`A. A synthesized image is generated without duplication of objects
`41. Each independent claim recites “a synthesized image is generated without
`
`duplication of objects.” I understand that the Board construed “synthesized
`
`image” to mean “the image generated by combining the received image data
`
`captured by the image capture devices.” (Institution Decision, p. 11.) But the
`
`Board’s construction does not include the term “is generated without
`
`duplication of objects,” which modifies the synthesized image. Valeo, however,
`
`suggested that “without duplication of objects” should mean “there is minimal
`
`multiple exposure of objects appearing in overlap zones in the synthesized
`
`image.” (Petition, p. 11.)
`
`42. For purposes of this Declaration, I address the Board’s construction of
`
`synthesized image in conjunction with Valeo’s proposed construction for
`
`without duplication of objects. I understand that these constructions were made
`
`under a different standard than will be applied for the final decision. Thus,
`
`while I construe “a synthesized image is generated without duplication of
`
`objects” to mean the image generated by combining the received image data
`
`captured by the image capture devices where there is minimal multiple
`
`exposure of objects appearing in overlap zones in the synthesized image, I
`
`provide the following context from the claim language, specification, and
`
`
`
`- 15 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`prosecution history of the ’724 patent to clarify the proper construction for this
`
`claim term.
`
`43. My proposed construction addresses the purpose of the inventions disclosed in
`
`the ’724 patent, which is to overcome the distortion and duplication present in
`
`other vehicular vision systems so as to reduce the amount of time spent
`
`gathering information. Based on the plain language of the claim in the context
`
`of the ’724 patent, a POSA would have understood “a synthesized image is
`
`generated without duplication of objects” to mean that the generated
`
`synthesized image shows primarily one representation of each object in the
`
`captured scene.
`
`44. The context of this claim term within the claim itself provides further insight.
`
`The synthesized image, for example, is not generated from image data collected
`
`from cameras at any location. Instead, the claim recites image capture devices
`
`disposed at a driver-side portion, a passenger-side portion, and a rear portion of
`
`the equipped vehicle. (See, e.g., ’724 patent, 23:65–24:6.) Thus, the synthesized
`
`image generated without duplication of objects must be synthesized from image
`
`capture devices at the claimed locations. As Dr. Wolberg has already opined,
`
`the claimed locations lead to additional difficulties in generating a synthesized
`
`image without duplication of objects. Specifically, because the cameras do not
`
`
`
`- 16 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`share a common center of projection, the views from the image data will exhibit
`
`the effects of parallax. That is, each image sensor will “see” a different view of
`
`the object, typically from a different distance, which will cause the object to
`
`appear rotated, dilated, shifted, and/or sheared when compared to each other. As
`
`a result, when these images are combined, unless special processing precautions
`
`are taken, objects may be duplicated, among other effects, in the combined
`
`image. Thus, considering the claimed locations of the image capture devices,
`
`the generated synthesized image should show primarily one representation of
`
`each object in the captured scene.
`
`45. This construction is consistent with the ’724 patent’s specification and captures
`
`a significant function of the ’724 patent.
`
`46. As discussed above, the ’724 patent is directed to providing a system to address
`
`problems with distortion and duplication of objects in previous systems. As a
`
`result, “[v]ision system 12 provides a substantially seamless panoramic view
`
`rearwardly of the vehicle without duplicate or redundant images of objects.”
`
`(’724 patent, 6:2–5.) In part, the ’724 patent accounts for duplication of objects
`
`by adjusting the field of view of the rear image capture device. (Id. at 6:66–
`
`7:43.) But this alone will not lead to a synthesized image without duplication of
`
`objects because there still needs to be an overlap zone. And the claims recite
`
`
`
`- 17 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`“without duplication of objects present in said first overlap zone and in said
`
`second overlap zone.” (’724 patent, 24:43–44 (emphasis added).)
`
`47. Accordingly, in addition to the field of view adjustment technique, the ’724
`
`patent also provides processing techniques used by the image processor that
`
`account for duplication of objects within the overlap zone. Specifically, the
`
`image processor processes the image from image capture device 16 differently
`
`than the images from image capture devices 14. (Id. at 15:19–23.) Thus, even
`
`though the camera locations lead to “different image sizes for the same object”
`
`(id. at 2:7–8) and to the same object being located on different rows of pixels
`
`between multiple image capture devices (id. at 14:59–63), i.e., the images are
`
`different due to parallax, the processing of the system disclosed in the ’724
`
`patent provides for a synthesized image that is generated without duplication of
`
`objects. That is, the ’724 patent provides a way to reduce the effects of parallax,
`
`leading to minimal multiple exposure of objects.
`
`48. In this case, the prosecution history does not change this construction.
`
`Accordingly, in the context of the ’724 patent, the proper construction of “a
`
`synthesized image is generated without duplication of objects” requires that the
`
`generated synthesized image shows primarily one representation (with minimal
`
`
`
`- 18 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`multiple exposure) of each object in the captured scene despite different
`
`locations of image capture devices.
`
`B. Approximates a view as would be seen by a virtual camera at a
`single location
`
`49. Each independent claim also recites that the synthesized image “approximates a
`
`view as would be seen by a virtual camera at a single location.” I understand
`
`that Valeo does not offer a construction of this claim feature. (Petition, pp. 10–
`
`14.)
`
`50. Based on the claim language, a POSA would have understood “approximates a
`
`view as would be seen by a virtual camera at a single location” to mean a view
`
`that appears to be from a single virtual camera, i.e., the view would not include
`
`distortion caused by combining multiple views. Displaying multiple views tiled
`
`side-by-side together on a single display screen does not necessarily
`
`approximate the claimed view as would be seen by a virtual camera, even if the
`
`multiple views cover the same field of view as the virtual camera.
`
`51. As with “without duplication of objects,” other claim limitations affect this
`
`construction, such as the claimed location of the cameras. Accordingly, the
`
`synthesized image that approximates a view as would be seen by a virtual
`
`camera at a single location must be synthesized from image sensors at the
`
`
`
`- 19 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`claimed locations, which will lead to additional difficulties in accomplishing the
`
`claimed view.
`
`52. The specification further supports this construction. The ’724 patent provided a
`
`system to address problems with distortion and duplication of objects in
`
`previous systems. As a result, “elongated, laterally-extending, objects, such as
`
`the earth’s horizon, appear uniform and straight across the entire displayed
`
`image. The displayed image provides a sense of perspective, which enhances
`
`the ability of the driver to judge location and speed of adjacent trailing
`
`vehicles.” (’724 patent, 6:5–10.) This sense of perspective is accomplished by
`
`the processing techniques that account for different image sizes for the same
`
`object and for disjointed boundaries. (See id. at 14:52–16:14.)
`
`53. In this case, the prosecution history does not change this construction.
`
`Accordingly, in the context of the ’724 patent, the proper construction of
`
`“approximates a view as would be seen by a virtual camera” requires that the
`
`synthesized image shows a view that appears to be from a single virtual camera
`
`without different image sizes and disjointed boundary lines despite different
`
`locations of image capture devices.
`
`
`
`- 20 -
`
`

`
`IPR2015-01410
`U.S. Pat. No. 8,643,724
`VIII. The inventors of the ’724 patent constructively reduced the claimed
`invention to practice prior to Yamamoto’s date of availability as prior art.
`
`54. I have been informed that filing a non-provisional, utility U.S. patent
`
`application is considered a “constructive reduction to practice” for the
`
`inventions supported and claimed therein, so long as the application complies
`
`with the requirements of pre-AIA 35 U.S.C. § 112, first paragraph. In addition,
`
`I understand that the application must contain a disclosure of the invention
`
`sufficiently adequate to enable a POSA to practice the invention without
`
`excessive experimentation.
`
`55. With that understanding, and in support of the above-captioned IPR to show
`
`prior invention to one of the references therein (Ex. 1003; “Yamamoto”