IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`Valeo North America, Inc., Valeo S.A., Valeo GmbH,
`VALEO Schalter und Sensoren GmbH,
`and Connaught Electronics Ltd.
`Petitioners,
`
`v.
`
`Magna Electronics, Inc.
`Patent Owner
`
`
`Case IPR2015-____
`U.S. Patent No. 8,643,724
`
`
`
`
`Mail Stop PATENT BOARD
`Patent Trial and Appeal Board
`United States Patent and Trademark Office
`Madison Building (East)
`600 Dulany Street
`Alexandria, VA 22313
`
`
`DECLARATION OF DR. GEORGE WOLBERG REGARDING
`U.S. PATENT NO. 8,643,724
`
`
`
`
`
`
`
`
`VALEO EX. 1019_001
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`I, Dr. George Wolberg, do hereby declare and state, that all statements made
`
`herein of my own knowledge are true and that all statements made on information
`
`and belief are believed to be true; and further that these statements were made with
`
`the knowledge that willful false statements and the like so made are punishable by
`
`fine or imprisonment, or both, under Section 1001 of Title 18 of the United States
`
`Code.
`
`
`
`Dated: November 7, 2014
`
`
`
`
`
`
`
`________________________
`
`
`
`VALEO EX. 1019_002
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`TABLE OF CONTENTS
`
`I.
`
`INTRODUCTION .......................................................................................... 1
`
`A.
`
`B.
`
`C.
`
`D.
`
`Engagement .......................................................................................... 1
`
`Background And Qualifications .......................................................... 1
`
`Compensation and Prior Testimony .................................................... 4
`
`Information Considered ....................................................................... 5
`
`II.
`
`LEGAL STANDARDS FOR PATENTABILITY ........................................ 6
`
`A.
`
`B.
`
`The Person Having Ordinary Skill In The Art .................................... 7
`
`The State of the Art – Description of Background Technology .......... 9
`
`III. THE ’724 PATENT .....................................................................................16
`
`A.
`
`B.
`
`C.
`
`D.
`
`E.
`
`Technical Overview Of The ’724 Patent ...........................................16
`
`Prosecution History Of The ’724 Patent ............................................20
`
`Claim Construction ............................................................................20
`
`Patentability Analysis of the ’724 Patent ...........................................23
`
`Discussion of Relevant Patents and Publications ..............................24
`
`1.
`
`2.
`
`3.
`
`Japanese Publication No. H3-99952 (“Nissan”) .....................24
`
`Japanese Publication No. S62-16073 (“Hino”) .......................27
`
`Japanese Publication No. 64-14700 (“Aishin”) .....................29
`
`4. Wang Publication on CMOS Video Cameras .........................31
`
`5.
`
`6.
`
`Japanese Publication No. 59-114139 assigned to Niles Co.,
`Ltd. (“Niles”) ...........................................................................31
`
`U.S. Patent No. 5,444,478 to Lelong (“Lelong”) ....................33
`
`F. Motivations to Combine ....................................................................35
`
`1.
`
`Claims 1-18, 20, 22, 23, 25, 29-32, 41, 42, 43, 45 and 48 are
`obvious over Nissan, Hino, and Lemelson ..............................38
`
`a.
`
`b.
`
`Nissan renders obvious the limitation “without
`duplication of objects” ..................................................38
`
`Nissan and Hino render obvious “wherein said
`synthesized image approximates a view as would be
`seen by a virtual camera at a single location exterior of
`the equipped vehicle” ....................................................42
`i
`
`
`
`VALEO EX. 1019_003
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`2.
`
`3.
`
`4.
`
`5.
`
`6.
`
`7.
`
`8.
`
`9.
`
`Claims 2, 15, 16, and 18 of the ’724 patent are obvious in view
`of Nissan, Hino, and Lemelson ...............................................46
`
`Claim 13 is rendered obvious by Nissan, Hino, and Lemelson
` .................................................................................................51
`
`Claim 19 is rendered obvious by Nissan, Hino, Lemelson, and
`Wang ........................................................................................54
`
`Claim 21 is obvious over Nissan, Hino, Lemelson, and Aishin
` .................................................................................................56
`
`Claim 24 is obvious over Nissan, Hino, Lemelson, and Niles 59
`
`Claims 46 and 47 are obvious over Nissan, Hino, Lemelson,
`and Lelong ...............................................................................62
`
`Claims 49, 65, and 78 are obvious in view of Nissan, Hino,
`Lemelson and Aishin ...............................................................64
`
`Claims 53, 68 and 80 are obvious in view of Nissan, Hino,
`Lemelson, Aishin, and Wang ..................................................66
`
`IV. CONCLUSION ............................................................................................67
`
`
`
`
`
`ii
`
`
`
`VALEO EX. 1019_004
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`I.
`
`INTRODUCTION
`
`A. Engagement
`
`1.
`
`I have been retained by counsel for Valeo North America Inc., Valeo S.A.,
`
`Valeo GmbH, Valeo Schalter und Sensor GmbH and Connaught Electronics Ltd., as an
`
`expert witness in the above-captioned proceeding. I have been asked to render an
`
`opinion regarding the validity of claims 1-86 (the “Challenged Claims”) of U.S. Patent
`
`No. 8,643,724 (“the ’724 patent), which is submitted herewith as Exhibit 1001. The
`
`following is my written report on that topic.
`
`B.
`
`2.
`
`Background And Qualifications
`
`I am currently a Professor of Computer Science at the City College of New
`
`York in the department of Computer Science. I have studied and worked in the field of
`
`computer science and engineering since 1985. My experience includes research and
`
`teaching, with research interests in image processing, computer graphics, and computer
`
`vision.
`
`3.
`
`I have been working in the field of Computer Science, specifically Image
`
`Processing, Computer Vision, and Computer Graphics since 1985. I have a Bachelor’s
`
`Degree and Master’s Degree in Electrical Engineering, both from Cooper Union in
`
`1985. I have a Ph.D. in Computer Science from Columbia University in 1990.
`
`1
`
`VALEO EX. 1019_005
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`4.
`
`After earning my PhD, I joined City College of New York as an assistant
`
`professor in the Computer Science department, where I continued performing research
`
`in image processing, computer graphics, and computer vision.
`
`5.
`
`In 1994, I became an associate professor, and in 1998, I became a full
`
`professor at the City College of New York, where I have continued my research ever
`
`since.
`
`6.
`
`I have published a leading monograph on Digital Image Warping in 1990.
`
`Example source code from that text was used in Adobe Photoshop in their perspective
`
`cloning tool. I have published over sixty (60) academic papers on image processing,
`
`computer vision, and computer graphics. Many of them revolve around image
`
`morphing, warping, registration, and scattered data interpolation. My image morphing
`
`and warping software was used by Apple in their Shake program. The following is a
`
`list of selected representative journal publications [relevant specifically to the subject
`
`matter of patent]:
`
`a.
`
`b.
`
`c.
`
`G. Wolberg, Digital Image Warping, IEEE Computer Society Press, 1990.
`
`T. Boult and G. Wolberg, “Local Image Reconstruction and Subpixel
`
`Restoration Algorithms,” CVGIP: Graphical Models and
`
`Image
`
`Processing, vol. 55, no. 1, pp. 63-77, January 1993.
`
`G. Wolberg, “A Two-Pass Mesh Warping Implementation of Morphing,”
`
`Dr. Dobb’s Journal, no. 202, July 1993.
`
`2
`
`VALEO EX. 1019_006
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`d.
`
`e.
`
`f.
`
`g.
`
`h.
`
`i.
`
`7.
`
`S. Lee, K.-Y. Chwa, S.Y. Shin, and G. Wolberg, “Image Metamorphosis
`
`Using Free-Form Deformations and Snakes,” Computer Graphics (Proc.
`
`Siggraph ’95), pp. 439-448, August 1995.
`
`S. Lee, G. Wolberg, K.-Y. Chwa, and S.Y. Shin, “Image Metamorphosis
`
`With Scattered Feature Constraints,” IEEE Trans. Visualization and
`
`Computer Graphics, vol. 2, no. 4, pp. 337-354, December 1996.
`
`S. Lee, G. Wolberg, and S.Y. Shin, “Scatter Data Interpolation Using
`
`Multilevel B-Splines,” IEEE Trans. Visualization and Computer Graphics,
`
`vol. 3, no. 3, pp. 228-244, July-September 1997.
`
`S. Lee, G. Wolberg, and S.Y. Shin, “Polymorph: Morphing Among
`
`Multiple Images,” IEEE Computer Graphics and Applications, vol. 18, no.
`
`1, pp. 58-71, January-February 1998.
`
`G. Wolberg, “Image Morphing: A Survey”, Visual Computer, vol. 14, pp.
`
`360-372, 1998.
`
`S. Zokai and G. Wolberg, “Image Registration Using Log-Polar Mappings
`
`for Recovery of Large-Scale Similarity and Projective Transformations,”
`
`IEEE Trans. Image Processing, vol. 14, no. 10, pp. 1422-1434, October
`
`2005.
`
`I co-founded Brainstorm Technology in 2003, where I hold the title of
`
`Chief Technology Officer.
`
`8.
`
`I am a Senior Member of the IEEE and a member of ACM SIGGRAPH.
`
`3
`
`VALEO EX. 1019_007
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`9.
`
`In 1991, I won the NSF Presidential Young Investigator Award. In 1998, I
`
`won the CCNY Outstanding Teacher Award and in 2000 I won the Mayor's Award for
`
`Excellence in Science and Technology from NYC Mayor Guliani.
`
`10. Over the past 23 years, I have obtained approximately $4,500,000 in grant
`
`funding from NSF, NIST, NASA, ONR, AFRL, DOE, CUNY, Securics, Xerox, and
`
`Google.
`
`11.
`
`I am a reviewer of many different journals, including IEEE Trans. Image
`
`Processing, IEEE Trans. Pattern Analysis and Machine Intelligence, IEEE Computer,
`
`IEEE Computer Graphics and Applications, IEEE Trans. Visualization and Computer
`
`Graphics, CVGIP: Graphical Models and Image Processing, Journal of Visualization
`
`and Computer Animation, and Computer Graphics (SIGGRAPH). I also am reviewing
`
`for the top computer vision journals and conferences including IEEE Computer Vision
`
`and Pattern Recognition, ACM Siggraph, and Computer Graphics International.
`
`12. A copy of my curriculum vitae, which describes in further detail my
`
`qualifications, responsibilities, employment history, honors, awards, professional
`
`associations, invited presentations, and publications is at Ex. 1025.
`
`C. Compensation and Prior Testimony
`
`13.
`
`I am being compensated at a rate of $350 per hour for my study and
`
`testimony in this matter. I am also being reimbursed for reasonable and customary
`
`expenses associated with my work and testimony in this investigation. My
`
`4
`
`VALEO EX. 1019_008
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`compensation is not contingent on the outcome of this matter or the specifics of my
`
`testimony.
`
`14. During the previous four years, I have not testified as an expert at trial or
`
`in deposition. However, I testified in a jury trial as an image processing expert in the
`
`United States District Court for the Western District of Michigan in “Computer Design,
`
`Inc. vs. Nedgraphics Inc,” Civil Action No. 1:97-cv-00014.
`
`D.
`
`Information Considered
`
`15. My opinions are based on my years of education, research and experience,
`
`as well as my investigation and study of relevant materials. In forming my opinions, I
`
`have considered the materials referred to herein or listed in – “Appendix A” at the end
`
`of this declaration.
`
`16.
`
`I may rely upon these materials and/or additional materials to rebut
`
`arguments raised by the patent owner. Further, I may also consider additional
`
`documents and information in forming any necessary opinions – including documents
`
`that may not yet have been provided to me.
`
`17. My analysis of the materials produced in this investigation is ongoing and
`
`I will continue to review any new material as it is provided. This report presents only
`
`those opinions I have formed to date. I reserve the right to revise, supplement, and/or
`
`5
`
`VALEO EX. 1019_009
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`amend my opinions stated herein based on new information and on my continuing
`
`analysis of the materials already provided.
`
`II. LEGAL STANDARDS FOR PATENTABILITY
`
`18.
`
`In expressing my opinions and considering the subject matter of the claims
`
`of the ’724 patent, I am relying upon certain basic legal principles that counsel has
`
`explained to me.
`
`19.
`
`It is my understanding that a patent claim is unpatentable if the claimed
`
`invention as a whole would have been obvious to a person having ordinary skill in the
`
`art (“PHOSITA”) at the time of the invention, in view of the prior art in the field and
`
`analogous fields. This means that even if all of the elements of the claim are not
`
`described or disclosed in a single prior art reference that would anticipate the claim, the
`
`claim can still be unpatentable.
`
`20.
`
`I understand that in order to prove that a claimed invention is unpatentable
`
`for obviousness, it is necessary to (1) identify the differences between the claim and
`
`particular disclosures in the prior art references, singly or in combination, (2)
`
`specifically explain how the prior art references could have been combined in order to
`
`arrive at the subject matter of the claimed invention, and (3) specifically explain why a
`
`PHOSITA would have been motivated to so combine the prior art references.
`
`6
`
`VALEO EX. 1019_010
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`21.
`
`I also understand that certain objective indicia can be important evidence
`
`of obviousness or nonobviousness of a claimed invention. Such indicia may include,
`
`under appropriate circumstances: (a) commercial success of products covered by the
`
`patent claims, where that success is not attributable to factors or features other than
`
`those recited in the claim; (b) a long-felt and unmet need for the invention; (c) copying
`
`of the invention by others in the field; (d) unexpected results achieved by the claimed
`
`invention as compared to the closest prior art; (e) praise of the invention by others; (f)
`
`expressions of surprise by experts and PHOSITAs at the making of the invention; (g) a
`
`history of licensing the claimed invention, again where such licensing is not attributable
`
`to factors or features other than those recited in the claim; and, (h) the inventor having
`
`proceeded against the accepted wisdom in the art at the time of the invention.
`
`22.
`
`In the present case, I am not aware of any evidence of objective indicia of
`
`nonobviousness of the claims of the ’724 patent.
`
`23.
`
`I have been asked to express my opinions regarding patentability of the
`
`Challenged Claims and whether or not, in my view, a given claim is more likely
`
`obvious than not.
`
`A. The Person Having Ordinary Skill In The Art
`
`24.
`
`I have been informed that “a person having ordinary skill in the relevant
`
`art (PHOSITA)” is a hypothetical person considered to have the normal skills and
`
`7
`
`VALEO EX. 1019_011
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`knowledge in a particular technical field, without being a genius. This person is one to
`
`whom an expert in the relevant field could assign a routine task with reasonable
`
`confidence that the task would be successfully carried out. I have been informed that
`
`the level of skill in the relevant art is evidenced by the prior art references.
`
`25.
`
`I believe, and the prior art discussed herein demonstrates, that a PHOSITA
`
`in the field of the ’724 patent would be someone who was familiar with basic computer
`
`vision systems and the concepts of image stitching and image compositing documented
`
`in the vision systems literature prior to May of 1996. That literature would have
`
`included books and/or publications that had been written on image stitching, image
`
`compositing, image morphing, image warping, artificial vision, as well as academic
`
`papers published at conferences such as the IEEE CVPR, ICCV, ACM SIGGRAPH,
`
`and Pattern Recognition and Image Processing; and in such journal publications as
`
`CVGIP: Computer Vision, Graphics, and Image Processing, CVGIP: Graphical
`
`Models and Image Processing, IEEE Computer Graphics and Applications, IEEE
`
`Trans. Image Processing, IEEE Trans. Visualization and Computer Graphics, ACM
`
`Transactions on Graphics, Image and Vision Computing Journal, Visual Computer, Intl.
`
`Journal of Computer Vision, Pattern Recognition, Journal of Visualization and
`
`Animation, and IEEE Computer.
`
`8
`
`VALEO EX. 1019_012
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`26. The PHOSITA may have been an undergraduate student in mathematics,
`
`engineering, or computer science, and should have had at least one course in image
`
`processing, computer graphics, or computer vision.
`
`27. The PHOSITA may have worked in the fields of image processing,
`
`computer graphics, or computer vision in academia (either as a professor or a student),
`
`for a technology company, or for a government.
`
`28. Based on my experience, I have an understanding of the capabilities of a
`
`person of ordinary skill in the relevant field. I have supervised and directed many such
`
`persons over the course of my career. Further, I had those capabilities myself at the
`
`time the ’724 patent was filed.
`
`B.
`
`The State of the Art – Description of Background Technology
`
`29. The ’724 patent is directed to a “multi-camera vision system for a vehicle,”
`
`for capturing image data using at least three image capture devices that are mounted on
`
`the vehicle and have overlapping fields of view. Ex. 1001 at Abstract. Outputs of the
`
`at least three image capture devices are provided to an image processor that produces a
`
`composite image for display to the driver of the vehicle. Id.
`
`30.
`
`Image mosaicing refers to the process of stitching together multiple input
`
`images to form one composite output image. Generally, the input images differ by
`
`some geometric transformations such as rotation, scale, translation (shift), perspective,
`
`and camera lens distortion. Mosaicing thereby requires an image registration stage to
`
`9
`
`VALEO EX. 1019_013
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`infer and implement the geometric transformation necessary for image alignment.
`
`Image compositing is a related term that refers to the blending of the overlapping
`
`regions. If image registration is performed effectively, then the blending of the
`
`multiple input images will diminish the appearance of any misaligned multiple
`
`exposure effect. That is, the overlapping pixels among two or more images must be in
`
`geometric correspondence otherwise they will be blended together to form an unsharp
`
`image with the same image features appearing in multiple shifted positions in the
`
`output image.
`
`31. To put image mosaicing in historical context, it is worth noting that image
`
`mosaicing originated shortly after the development of the photographic process around
`
`1840 for topographical mapping applications. Images acquired from balloons or hill
`
`tops were manually pieced together. This pace accelerated after aerial photography
`
`became possible in the early 1900s, and after satellite imagery became widespread in
`
`the 1970s with the Landsat program. The early work in computer techniques for image
`
`mosaicing therefore had its roots in stitching together aerial and satellite photography
`
`for agencies such as NASA. There are technical reports from NASA dating back to the
`
`mid-1960s that report on related work in this area, such as, for example, Robert
`
`Nathan’s, “Digital Video Data Handling,” NASA JPL Tech Report 32-877, Pasadena,
`
`CA, Jan. 5, 1966. See Ex. 1026.
`
`10
`
`VALEO EX. 1019_014
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`32. Seams can naturally appear between the images stitched together in a
`
`mosaic. A landmark paper addressing this problem is “A Multiresolution Spline with
`
`Application to Image Mosaics” by Peter Burt and Edward Adelson, published in ACM
`
`Transactions on Graphics in 1983. See Ex. 1027. Their approach, based on
`
`decomposing the constituent images into the frequency domain and then blending
`
`across frequency bands, delivered exceptional results as exemplified in their apple-
`
`orange hybrid image that was prominently displayed on the cover of the journal in
`
`which the publication appeared. This paper represented cutting-edge work in the area
`
`of image compositing since it addressed the problem of seamlessly stitching together
`
`images that are already in geometric alignment.
`
`33. A critical stage before such seams can be eliminated is to align the images
`
`such that image features in the overlapping areas lie on top of each other. This process,
`
`known as image registration, was well-known before 1996. Indeed, a classic survey in
`
`the field of image registration remains: Lisa Gottesfeld Brown, “A Survey of Image
`
`Registration Techniques”, vol. 24, ACM Computing Surveys, pp. 325-376, 1992. See
`
`Ex. 1028. It continues to be one of the most cited registration surveys in the field.
`
`34. Other techniques that were well established by 1996 were image morphing
`
`and image warping, both of which were well documented in my Digital Image Warping
`
`monograph, published in 1990. See Ex. 1029. Image morphing is a famous application
`
`11
`
`VALEO EX. 1019_015
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`that demonstrates the importance of performing geometric transformations for image
`
`alignment to remove artifacts that may occur by blending overlapping regions among
`
`multiple images. For instance, Michael Jackson’s “Black or White” music video of
`
`1991 exhibits a series of seamless transitions from one face to another. This involved
`
`geometric correspondence between successive key frames of the video to be specified
`
`manually. For all computed frames between one key frame and the next, both key
`
`frames were warped into alignment for some intermediate instance of the animation and
`
`then these warped frames were blended. Failure to accurately warp frames into
`
`alignment would have produced two sets of eyes, noses, mouths, and other facial
`
`features upon blending. This demonstrates how critical the geometric transformation
`
`stage is in reducing artifacts when blending overlapping regions.
`
`35.
`
`Image warping refers to the geometric transformation of images. This
`
`includes perspective transformations and homographies, which map the four corners of
`
`an image to a new set of positions. This quadrilateral-to-quadrilateral transformation
`
`proves valuable in accounting for perspective deformations in image alignment. This is
`
`central to image mosaicing work done in aerial and satellite photography, which is the
`
`domain in which mosaicing was first applied. Id.
`
`36. By the 1990s, those skilled in the art recognized that arrays of
`
`photosensing pixels, such as CCD and CMOS cameras as used in the claims of the ’724
`
`12
`
`VALEO EX. 1019_016
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`patent, could be used on vehicles to capture video image data. See Ex. 1009. By that
`
`time, it was also well known that fisheye lenses on cameras would achieve wide angles
`
`of view that spanned rearward, sideward, and forward of the position at which the
`
`camera is installed, particularly since Nikon had released a camera back in 1962 with a
`
`fisheye lens. The use of fisheye lenses on cameras became even more well known in
`
`1973, when the San Diego Hall of Science premiered a dome system called Omnimax,
`
`which projected film that was shot with a camera equipped with a fisheye lens. That
`
`system, later renamed IMAX Dome, became a popular attraction for projecting movies
`
`onto domed planetariums.
`
`37. A computer graphics technique to create large-format Omnimax images
`
`was discussed in a well-known paper: Greene, Ned and Paul Heckbert, “Creating Raster
`
`Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted
`
`Average Filter,” IEEE Computer Graphics and Applications, vol. 6, no. 6, pp. 21-27,
`
`June 1986. See Ex. 1030. As in the ’724 patent, Greene and Heckbert’s paper
`
`recognized that to create large-format Omnimax images, the relative position of the
`
`cameras was known in advance. However, a key point to the work of Greene and
`
`Heckbert is that they recognized that when attempting to merge images from multiple
`
`cameras, the cameras must share a common center of projection to avoid parallax.
`
`13
`
`VALEO EX. 1019_017
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`38.
`
` A common center of projection can be achieved by rotating a camera on
`
`top of a tripod. Each position of the rotated camera shares the same center of
`
`projection. The purpose of doing this is to avoid parallax, which is a displacement in
`
`the apparent position of an object viewed along two different lines of sight. Thus,
`
`cameras cannot be placed in arbitrary positions and orientations around the scene to
`
`create a single mosaic. In particular, it is not enough that the cameras have overlapping
`
`fields of view. Instead, the center of projection for all the cameras must be the same
`
`point so that parallax can be avoided. There is no mention anywhere in the ’724 patent
`
`about the role that parallax plays, particularly since the multiple cameras described in
`
`the patent clearly do not share a single center of projection.
`
`39. When multiple cameras are used to create image mosaics in the literature,
`
`they either lie far away from the scene such that the scene appears flat, or they share the
`
`same center of projection so that panoramic image mosaicing is achieved. These two
`
`cases are documented in an excellent survey of image mosaics: Szeliski, Richard,
`
`“Image Mosaicing for Tele-Reality Applications”, DEC Cambridge Research
`
`Laboratory, CRL 94/2, May 1994. See Ex. 1031. Note that the option of positioning
`
`the cameras far from the scene follows the assumptions and approach used in virtually
`
`all image mosaicing work that is performed in aerial and satellite photography, which
`
`was the original domain in which image mosaicing was applied. As a result, there is no
`
`14
`
`VALEO EX. 1019_018
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`parallax and it is sufficient to perform perspective transformations to map one image to
`
`another to create larger mosaic images from the constituent images. The methods for
`
`doing this are explained in my Digital Image Warping book, as well as the Szeliski tech
`
`report cited above, and the classic registration survey by Brown cited earlier.
`
`40. Another technique that was well established by 1996 was antialiasing.
`
`Most image processing books, including my Digital Image Warping text (1990),
`
`devoted chapters to this problem. Antialiasing refers to techniques employed to combat
`
`aliasing, which are artifacts due to under-sampling the input signal. This happens when
`
`resizing an image to smaller dimensions, because only then are pixels discarded leading
`
`to information loss. The general solution for this problem is to bandlimit (or blur) the
`
`input before subsampling it. It is this averaging that the ’724 patent refers to as “pixel
`
`adjustment.”
`
`41. Prior to May 1996, there were many known forms of displaying images to
`
`viewers, beginning with television monitors using cathode ray tubes (CRTs) in the
`
`1930s. CRT displays were the most popular display technology, used for the majority
`
`of the 20th century, until due to their bulk and weight, they were superseded by flat
`
`panel display technologies, including various types of Liquid Crystal Displays (LCDs),
`
`plasma displays, and Organic Light-Emitting Diode (OLED) displays. By the time of
`
`the filing date for the ’724 patent, the person of ordinary skill in the art had many
`
`15
`
`VALEO EX. 1019_019
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`display technologies to choose from, including, reconfigurable displays, which are
`
`discussed by Dr. Wilhelm in his declaration at Ex. 1020.
`
`III. THE ’724 PATENT
`
`A. Technical Overview Of The ’724 Patent
`
`42. The ’724 patent generally describes a driver assist vision system for a
`
`motor vehicle. Ex. 1001 at 2:59-3:22 and Fig. 1. More particularly, the ’724 patent
`
`describes a multi-camera vision system for a vehicle in which three image capture
`
`devices (e.g., cameras) are mounted on a vehicle with overlapping fields of views
`
`exterior of the vehicle. Ex. 1001 at Abstract; Fig. 1. The at least two cameras are
`
`placed at the side and rear center of the vehicle. Ex. 1001 at Fig. 1, cameras 14, 16.
`
`The cameras each capture image data based on the respective fields of view.
`
`43. These image capture devices capture images and an image processor
`
`processes the images in such a way as to display the vehicle’s surroundings in a
`
`synthesized single image to the driver of as the driver operates the vehicle. Ex. 1001 at
`
`2:59-3:22 and Fig. 8. Each of the three image capture devices captures a scene exterior
`
`of the vehicle and has zones of overlap with another one of the three image capture
`
`devices. Ex. 1001 at Fig. 1; 6:66-7:5. The displayed image viewable by the driver is
`
`the result of image processing performed by an image processor, whereby the images
`
`captured by each of the image capture devices are merged into a single image, called a
`
`synthesized image in the ’724 patent. Ex. 1001 at Fig. 3; 7:44-57. The synthesized
`
`16
`
`VALEO EX. 1019_020
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`image has little or no duplication of objects that may otherwise be present as a result of
`
`the overlapping portions of the fields of views of the image capture devices. Ex. 1001
`
`at 7:5-16. Further, the synthesized image is seen from the perspective of a virtual
`
`camera at a single location exterior of the equipped vehicle. Ex. 1001 at 5:64-6:2. The
`
`synthesized image is displayed on a reconfigurable display having the capability of
`
`selectively displaying camera images as well as various auxiliary information of
`
`interest to the driver. Ex. 1001 at 12:49-64. The challenge in this problem is to
`
`properly blend the overlapping fields of views and stitch together the images taken by
`
`cameras at different positions such that the image that is ultimately displayed to the
`
`driver of the vehicle has minimal multiple exposure (duplication).
`
`44. The drafters of the ’724 patent wrote a very lengthy specification with
`
`many different embodiments directed to different aspects of a generally conventional
`
`driver assist vision system, and then filed a series of continuation applications to cover
`
`the various embodiments of this unpatentable automobile vision system, each with an
`
`unusually large number of claims. The ’724 patent has 86 claims, some of which recite
`
`several alternatives, and all of which add conventional, well-known features to an
`
`otherwise unpatentable core invention. Due to its length and the number of different
`
`embodiments described in the ’724 patent specification, it is necessary to describe the
`
`state of the art of various features that were well-known much before the time of the
`
`17
`
`VALEO EX. 1019_021
`
`

`

`Declaration of Dr. George Wolberg
`Regarding U.S. Patent No. 8,643,724
`
`
`’724 patent. It is well known, however, that claiming a conventional feature from
`
`standard product configurations does not lend to patentability. Indeed, the PHOSITA
`
`would expect such features to be practical to include in a driver assist system, to make
`
`the system useful.
`
`45. A fundamental flaw to the ’724 patent is that it never once mentions the
`
`problem of parallax that is well-known to any researcher in the field before 1996.
`
`Parallax presents disparity among pixels of the same scene point taken from multiple
`
`images. This makes it impossible to blend overlapping images together with common
`
`geometric features because the ge