UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`VALEO NORTH AMERICA, INC., VALEO S.A., VALEO GMBH,
`VALEO SCHALTER UND SENSOREN GMBH,
`AND CONNAUGHT ELECTRONICS LTD.,
`
`Petitioners,
`
`v.
`
`MAGNA ELECTRONICS, INC.,
`
`Patent Owner.
`____________________
`
`Case IPR2015-014101
`Patent 8,643,724 B2
`____________________
`
`
`
`
`
`SUPPLEMENTAL DECLARATION OF DR. GEORGE WOLBERG
`
`
`1 Case IPR2015-01414 has been consolidated with this proceeding.
`
`
`
`____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`VALEO NORTH AMERICA, INC., VALEO S.A., VALEO GMBH,
`VALEO SCHALTER UND SENSOREN GMBH,
`AND CONNAUGHT ELECTRONICS LTD.,
`
`Petitioners,
`
`v.
`
`MAGNA ELECTRONICS, INC.,
`
`Patent Owner.
`____________________
`
`Case IPR2015-014101
`Patent 8,643,724 B2
`____________________
`
`
`
`
`
`SUPPLEMENTAL DECLARATION OF DR. GEORGE WOLBERG
`
`
`1 Case IPR2015-01414 has been consolidated with this proceeding.
`
`
`
`
`
`I, Dr. George Wolberg, do hereby declare and state, that all statements made
`
`herein of my own knowledge are true and that all statements made on information
`
`and belief are believed to be true; and further that these statements were made with
`
`the knowledge that willful false statements and the like so made are punishable by
`
`fine or imprisonment, or both, under Section 1001 of Title 18 of the United States
`
`Code.
`
`
`
`Dated: May ___, 2016
`
`
`
`
`
`
`
`
`2
`
`
`
`
`
`TABLE OF CONTENTS
`
`I.
`
`INTRODUCTION ........................................................................................... 4
`
`II.
`
`LEGAL STANDARDS ................................................................................... 4
`
`A. Obviousness ............................................................................................ 5
`
`B.
`
`C.
`
`Reasons to Combine ............................................................................... 5
`
`Claim Construction................................................................................. 6
`
`III.
`
`SUPPLEMENTAL OPINIONS ...................................................................... 7
`
`A.
`
`Claim Construction................................................................................. 7
`
`1.
`
`2.
`
`“a synthesized image is generated without duplication of
`objects” ......................................................................................... 8
`
`“approximates a view as would be seen by a virtual
`camera at a single location” ....................................................... 13
`
`B.
`
`The Prior Art Renders the Challenged Claims Obvious. ..................... 16
`
`1.
`
`2.
`
`Yamamoto and Mitsubishi teach a synthesized image with
`minimal multiple exposure and that approximates a view
`from a single location. ................................................................ 17
`
`A PHOSITA would have found it obvious to scale images
`as part of an image synthesis process. ....................................... 24
`
`
`
`3
`
`
`
`
`
`
`
`I.
`
`INTRODUCTION
`
`
`
`1.
`
`I submit this Supplemental Declaration to offer my opinions regarding
`
`the patentability and validity of claims 1, 3–12, 14, 15, 17, 19–52, 54–67, 69–79,
`
`and 81–86 of U.S. Patent No. 8,643,724 (“the ’724 Patent”). More specifically, I
`
`provide my opinions regarding Patent Owner’s Response (Paper 14), Dr. Etienne-
`
`Cummings’s Declaration (Exhibit 2004) and other exhibits submitted by Patent
`
`Owner.
`
`2.
`
`I set forth my professional qualifications and experience and attached
`
`my curriculum vitae with my declaration submitted on June 15, 2015. (Ex. 1020-
`
`21).
`
`3.
`
`In forming the opinions I express in this Supplemental Declaration, I
`
`considered the materials cited in Patent Owner’s Response, Dr. Etienne-
`
`Cummings’s Declaration, and any materials I cite in this Supplemental Declaration.
`
`I have further considered all materials that were cited in my original declaration
`
`submitted on June 15, 2015. I also rely on my years of education, research and
`
`experience, as well as my investigation and study of relevant materials.
`
`II. LEGAL STANDARDS
`
`4.
`
`In expressing my opinions and considering the subject matter of the
`
`claims of the ’724 Patent, I am relying upon certain basic legal principles that
`
`counsel has explained to me.
`
`
`
`4
`
`
`
`
`
`A. Obviousness
`
`5.
`
`It is my understanding that a patent claim is unpatentable if the claimed
`
`invention as a whole would have been obvious to a person having ordinary skill in
`
`the art (“PHOSITA”) at the time of the invention, in view of the prior art in the field
`
`and analogous fields. This means that even if all of the elements of the claim are
`
`not described or disclosed in a single prior art reference, the claim can still be
`
`unpatentable. I understand that in order to prove that a claimed invention is
`
`unpatentable for obviousness, it is necessary to (1) identify the differences between
`
`the claim and particular disclosures in the prior art references, singly or in
`
`combination; (2) specifically explain how the prior art references could have been
`
`combined in order to arrive at the subject matter of the claimed invention; and (3)
`
`specifically explain why a PHOSITA would have been motivated to so combine the
`
`prior art references. I understand the claims are unpatentable if it is more likely
`
`than not that the claims are obvious.
`
`B. Reasons to Combine
`
`6.
`
`To determine whether a claim is obvious, I understand that I may
`
`combine multiple references if a PHOSITA would have had apparent reasons to
`
`combine the references at the time of the alleged invention. I have been informed
`
`that reasons to combine references can include: (a) combining prior art elements
`
`according to known methods to yield predictable results; (b) simple substitution of
`
`
`
`5
`
`
`
`
`
`one known element for another to obtain predictable results; (c) use of a known
`
`technique to improve similar devices (methods, or products) in the same way;
`
`(d) applying a known technique to a known device (method, or product) ready for
`
`improvement to yield predictable results; (e) choosing from a finite number of
`
`identified, predictable solutions that are “obvious to try” and having a reasonable
`
`expectation of success; (f) known work in one field of endeavor may prompt
`
`variations of it for use in either the same field or a different one based on design
`
`incentives or other market forces if the variations are predictable to a PHOSITA;
`
`(g) some teaching, suggestion, or motivation in the prior art that would have led a
`
`PHOSITA to modify the prior art reference or to combine prior art reference
`
`teachings to arrive at the claimed invention.
`
`C. Claim Construction
`
`7.
`
`I understand that the Board has invited the parties to propose claim
`
`constructions for the ’724 Patent claims under the standard established by Phillips
`
`v. AWH Corp., 415 F.3d 1303 (Fed. Cir. 2005). I have been informed by counsel
`
`that under the Phillips standard, a claim term should be interpreted based on its
`
`ordinary and customary meaning, which is the meaning the term would have had to
`
`a PHOSITA at the time of the invention. I understand that to determine the proper
`
`meaning of a claim term, one primarily looks to “intrinsic evidence,” which
`
`includes patent claims, specification, and prosecution history. I understand that a
`
`
`
`6
`
`
`
`
`
`claim term should be interpreted based on the context of the entire claim, without
`
`rendering any claims terms or words meaningless, as well as the specification of the
`
`patent. I further understand that claims generally should not be interpreted to
`
`exclude embodiments or examples of the claimed invention that are set forth in the
`
`specification. I have also been informed by counsel that evidence such as expert
`
`and inventor testimony, dictionaries, and treatises may be relevant to interpretation
`
`of the claim language, although this type of evidence, sometimes called “extrinsic
`
`evidence,” cannot be used to contradict the claim language or the specification.
`
`III. SUPPLEMENTAL OPINIONS
`
`A. Claim Construction
`
`8.
`
`I provide my opinions on the construction of the terms “a synthesized
`
`image is generated without duplication of objects” and “approximates a view as
`
`would be seen by a virtual camera at a single location” as they appear in every
`
`independent claim of the ’724 Patent. Based on my review of the claim language,
`
`the specification, and my understanding of a PHOSITA at the time of the alleged
`
`invention, it is my opinion that the term “a synthesized image is generated without
`
`duplication of objects” should be interpreted as “the synthesized image is generated
`
`with minimum multiple exposure of objects appearing in overlap zones.” Further, it
`
`is also my opinion that a PHOSITA would understand the meaning of
`
`“approximates a view as would be seen by a virtual camera at a single location,” as
`
`
`
`7
`
`
`
`
`
`it is used in this patent, in the plain language in which it is written.
`
`1. “a synthesized image is generated without duplication of objects”
`
`9.
`
`In my opinion, the term “a synthesized image is generated without
`
`duplication of objects” should be interpreted as “the synthesized image is generated
`
`with minimum multiple exposure of objects appearing in overlap zones.” This
`
`construction is consistent with the claim language and specification of the ’724
`
`Patent. By contrast, Patent Owner has proposed in its Response that this term
`
`should be interpreted as “the generated synthesized image shows primarily one
`
`representation (with minimal multiple exposure) of each object in the captured
`
`scene despite the claimed locations of image capture device.” (Paper 14, Patent
`
`Owner Resp. at 11.) In my opinion, the claim language and specification of the
`
`’724 Patent do not support Patent Owner’s proposed construction.
`
`10. As an initial matter, Patent Owner seeks to add new requirements to
`
`the claims such that they require the synthesized image to show “primarily one
`
`representation . . . of each object.” In my opinion, Patent Owner’s proposal
`
`unnecessarily adds a layer of subjectivity and ambiguity to the claims. Indeed,
`
`those of ordinary skill in the art may reasonably disagree as to whether each object
`
`in a given synthesized image is shown in “primarily one representation.” Patent
`
`Owner further applies its proposed construction in a way that requires the complete
`
`resolution of parallax. For example, Patent Owner states that “images that exhibit
`
`
`
`8
`
`
`
`
`
`the effects of unresolved parallax will not be a synthesized image generated without
`
`duplication of objects, as claimed.” (Paper 14, Patent Owner Resp. at 22.)
`
`11. Patent Owner, however, fundamentally confuses the term “without
`
`duplication of objects” with resolution of parallax. For example, duplication of
`
`objects occurs in images synthesized from multiple cameras to the extent the fields
`
`of view of the cameras overlap. As I explained in my original declaration, the field
`
`of view of each camera can be envisioned as a cone that emanates from that camera,
`
`and when there are multiple cameras with overlapping fields of view, these cones
`
`overlap. (Ex. 1020, Wolberg Decl. at ¶ 65.) The result is that cameras with
`
`overlapping fields of view may capture the same objects multiple times, resulting in
`
`multiple exposure. Duplication, however, is different from parallax. For example,
`
`when each of the cameras have a common center of projection, there will be little or
`
`no parallax (Ex. 2004, Etienne-Cummings Decl. at ¶ 44), but given their
`
`overlapping fields of view, the cameras may nonetheless capture the same objects
`
`multiple times resulting in duplication. Thus, in my opinion, the claim language
`
`does not support adding new requirements pertaining to addressing parallax.
`
`12. Moreover, the specification ’724 Patent does not mention parallax or
`
`methods of resolving parallax, and provides only one method of reducing
`
`duplication of objects in the synthesized image, namely adjusting the field of view
`
`of the center (rear) camera to reduce the overlap zones between the fields of view of
`
`
`
`9
`
`
`
`
`
`the side cameras and the center camera. (Ex. 1001, ’724 Patent at 7:11-64.) Again,
`
`the field of view of each camera can be envisioned as a cone that emanates from
`
`that camera. To minimize overlap, the ’724 Patent discloses a method of changing
`
`the radius of those cones. (Ex. 1001 ’724 Patent at 7:11-64.) This technique
`
`reduces overlap because it effectively makes the cone narrower. As the cones are
`
`narrowed, the overlap zones become smaller and more distant from the vehicle. But
`
`this technique can never eliminate overlap zones. Moreover, narrowing the field of
`
`view for the center camera also increases the blind spot areas behind the vehicle
`
`that are not captured by any of the cameras.
`
`13. To illustrate, the overlap zones in Figure 4 of the ’724 Patent
`
`(reproduced below) are annotated in red. Objects that appear in this zone are
`
`captured by the side camera and the center (rear) camera, and therefore would
`
`appear twice in the synthesized image. The area shaded in gray represents blind
`
`zones behind the car that none of the cameras will image.
`
`
`
`10
`
`
`
`
`
`(Ex. 1001 ’724 Patent at Fig. 4 (annotated).)
`
`
`
`14. To reduce the likelihood that an object will appear in this overlap zone,
`
`the specification of the ’724 Patent discloses a method to narrow the field of view
`
`of the center (rear) camera. As shown in revised Figure 4 (below), narrowing the
`
`field of view of the center camera would certainly reduce the overlap zones. But
`
`this technique cannot totally eliminate the overlap zones because the cone from the
`
`center camera will inevitably intersect with the cones from the side cameras. Any
`
`objects in the resulting overlap zones would appear twice in the synthesized image.
`
`As annotated in Figure 4 (below), this technique also increases the blind zones
`
`(showing in grey) behind the vehicle that are not captured by any of the cameras.
`
`(Ex. 1001 ’724 Patent at Fig. 4 (annotated).)
`
`
`
`15.
`
`In support of its proposed construction, Patent Owner asserts that
`
`“[t]he ’724 patent further accounts for duplication of objects and generates a
`
`synthesized image without duplication of objects by processing the image from
`
`
`
`11
`
`
`
`
`
`image capture device 16 differently than the images from image capture devices
`
`14.” (Paper 14, Patent Owner Resp. at 11.) The cited portion of the specification
`
`discloses a method for scaling, or compressing, the image captured by the center
`
`(rear) camera. (Ex. 1001 ’724 Patent at 15:19-26.) This process adjusts the size of
`
`the center image so that all three images appear in approximately the same scale.
`
`16.
`
`Image compression, or scaling, however, has nothing to do with
`
`reducing or eliminating duplication in the synthesized image. Rather, this
`
`technique only addresses the scale changes between the center (rear) camera and the
`
`two side cameras. To illustrate using Figure 7 of the ’724 Patent, an object at
`
`position P2, is closer to the center (rear) camera 16 than the side camera 14. Thus,
`
`this object would appear somewhat larger in the image captured by the center
`
`camera 16. To compensate for this difference, the center image is compressed so
`
`that objects in the image become smaller. As I explain below, this is a well-known
`
`technique in image processing, but does nothing to resolve duplications from the
`
`overlapping fields of view and certainly would not resolve parallax.
`
`(Ex. 1001 ’724 Patent at Fig. 7.)
`
`
`
`12
`
`
`
`
`
`
`
`17. Further, the ’724 Patent specification recognizes that adjusting the field
`
`of view for the center camera can only reduce or minimize, but cannot eliminate,
`
`duplication of objects. For example, the specification explains that the claimed
`
`system discuses “reduce[s] redundant or duplicative images” in the image portions.
`
`(Ex. 1001, ’724 Patent at 7:60-64.) Likewise, the specification explains that the
`
`reduction in overlap zones, and hence “the presentation of redundant information to
`
`the driver,” “may be accomplished to a satisfactory extent.” (Ex. 1001, ’724 Patent
`
`at 7:7-14.) Patent Owner’s proposed construction is not consistent with the
`
`specification because it implies eliminating duplication and parallax, whereas the
`
`specification cannot eliminate duplication and does not addresses parallax.
`
`Therefore, in my opinion, “a synthesized image is generated without duplication of
`
`objects” should be interpreted as “a synthesized image is generated with minimum
`
`multiple exposure of objects appearing in overlap zones.”
`
`2.
`
`“approximates a view as would be seen by a virtual camera
`at a single location”
`
`18.
`
`I also understand that Patent Owner proposes the term “approximates a
`
`view as would be seen by a virtual camera at a single location” should be
`
`interpreted as “a view that appears to be from a single virtual camera without
`
`different image sizes and disjointed boundary lines.” (Paper 14, Patent Owner
`
`Resp. at 13.) Patent Owner proposes to add new requirements to the claim that
`
`would require the view to be “without different image sizes and disjointed boundary
`
`
`
`13
`
`
`
`
`
`lines” in order to suggest a relationship between the claim language and resolution
`
`of parallax. In my opinion, however, a PHOSITA would have understood the
`
`meaning of “approximates a view as would be seen by a virtual camera at a single
`
`location,” as it is used in this patent, in the plain language in which it is written.
`
`That is, the claimed view is from a single virtual camera and does not have to
`
`perfectly or precisely represent the actual view as would be seen from such a virtual
`
`camera. In fact, I find that the specification of the ’724 Patent contradicts Patent
`
`Owner’s proposed interpretation because it discloses examples that would include
`
`disjointed boundary lines or other distortions during typical driving situations.
`
`19. For example, as I explain above and in my original declaration, the
`
`’724 Patent does not at all discuss parallax and does not suggest any requirement
`
`that the claimed system resolve parallax. Rather, claims simply require a
`
`synthesized image that “approximates a view from a single location.” The
`
`specification also explains that claimed view need not be perfect, and explains that
`
`the synthesized image need only “reduce redundant and duplicative images” and
`
`that this “may be accomplished to a satisfactory extent.” (Ex. 1001, ’724 Patent at
`
`7:11-14, 7:60-64.) Thus, I find no support in the ’724 Patent for requiring a view
`
`“without different image sizes and disjointed boundary lines.”
`
`20. To the contrary, the specification of the ’724 Patent provides examples
`
`of camera-based systems that represent the claimed invention and that would result
`
`
`
`14
`
`
`
`
`
`in disjointed boundary lines, different image sizes, and other distortions. In one
`
`example, the disclosed system compresses only the bottom-half of the central
`
`image, whereas the top-half of the center image remains unchanged. (Ex. 1001,
`
`’724 Patent at 15:62-16:4.) The underlying rationale is that objects in the top-half
`
`of the image are far away during most driving situations. Therefore, any difference
`
`in scale between the images captured by the side cameras and the top-half of the
`
`center (rear) camera are expected to be negligible in most driving situations. As the
`
`’724 Patent explains:
`
`In an alternative embodiment, the vertical compression technique may
`
`be applied to only a lower vertical portion of central image portion
`
`48’. In most driving situations, objects imaged by rearward-facing
`
`image capture devices above the horizon are at a long distance from
`
`the vehicle while those below the horizon get progressively closer to
`
`the vehicle in relation to the distance below the horizon in the
`
`displayed image. Therefore, compression of the upper vertical portion
`
`of the central image portion may be eliminated without significant
`
`reduction in performance.
`
`(Ex. 1001, ’724 Patent at 15:62-16:4.)
`
`21. This technique may reduce disjointed boundary lines or different
`
`image sizes in some driving situations, such as when a car is driving down an open
`
`road with few tall structures around. In other common driving situations, however,
`
`the disclosed technique would leave disjointed boundary lines, different image
`
`
`
`15
`
`
`
`
`
`sizes, and other distortions in the synthesized image. For example, the synthesized
`
`image would be particularly distorted if there was a large object directly behind the
`
`vehicle, such as another car or building, which occupies the full field of view of the
`
`center (rear) camera. In this situation, the bottom half of the object would be drawn
`
`at one scale while the top half would be drawn at another scale. The resulting
`
`synthesized image would inevitably include disjointed boundary lines and other
`
`distortions. Nonetheless, the ’724 Patent specification states that such a technique
`
`will generate a single view “without significant reduction in performance.” (Ex.
`
`1001, ’724 Patent at 16:2-4.) Therefore, the specification contemplates that the
`
`synthesized image only needs to approximate the view from a single location and
`
`can include images with some disjointed boundaries or other distortions.
`
`B.
`
`The Prior Art Renders the Challenged Claims Obvious.
`
`22. As I explained in my original declaration, it is my opinion that it would
`
`have been obvious to a PHOSITA to combine the teachings of Yamamoto,
`
`Mitsubishi and Lemelson. I understand the Patent Owner asserts that the
`
`combination of Mitsubishi and Yamamoto would not produce a synthesized image
`
`“without duplication of objects . . . [that] approximates a view as would be seen by
`
`a virtual camera at a single location exterior of the equipped vehicle.” (Paper 14,
`
`Patent Owner Resp. at 28.) Instead, Patent Owner argues that Mitsubishi and
`
`Yamamoto do not meet the claim because they do not disclose how to resolve
`
`
`
`16
`
`
`
`
`
`parallax. (Paper 14, Patent Owner Resp. at 26-27.) But as I explain above, the
`
`claims do not require the elimination of parallax, but only require the synthesized
`
`image to have minimal multiple exposures and approximate a view from a single
`
`virtual camera. In my opinion, Patent Owner’s arguments are incorrect for the
`
`reasons below.
`
`1.
`
`Yamamoto and Mitsubishi teach a synthesized image with
`minimal multiple exposure and that approximates a view
`from a single location.
`
`23.
`
`I understand that Patent Owner criticizes my opinions for “gloss[ing]
`
`over the problem of parallax when asserting that Yamamoto and Mitsubishi disclose
`
`a synthesized image without duplication of objects.” (Paper 14, Patent Owner
`
`Resp. at 26). Patent Owner further asserts that Mitsubishi does not disclose a
`
`synthesized image as claimed in the ’724 Patent and that “Dr. Wolberg admits that
`
`the images must first be properly scaled and aligned, but . . . does not assert that
`
`Mitsubishi discloses how to properly scale and align the images to overcome
`
`Yamamoto’s parallax.” (Paper 14, Patent Owner Resp. at 27.) In my opinion, even
`
`if Yamamoto, Mitsubishi and Lemelson do not expressly address parallax issues,
`
`they still teach a synthesized image with minimal multiple exposure and that
`
`approximates a view from a single location.
`
`24. As I explain above, the claims of the ’724 Patent do not require the
`
`synthesized image to resolve parallax. Nor does the specification disclose any
`
`
`
`17
`
`
`
`
`
`technique for resolving parallax. Instead, the ’724 Patent only teaches how to
`
`minimize duplication of objects in the synthesized image by adjusting the field of
`
`view of the center (rear) camera, as well as aligning the scale of all three images by
`
`compressing the center image. Neither of these techniques resolves parallax. Yet,
`
`according to the specification, the resulting image nonetheless approximates the
`
`view as would be seen by a virtual camera at a single location. Thus, a PHOSITA
`
`would have understood that the claimed synthesized image need not be a perfect
`
`reproduction of the view from a single virtual camera, but rather an approximation
`
`that may contain some artifacts or distortions from parallax.
`
`25.
`
`In my original declaration, I explained that Figure 5 of Yamamoto
`
`depicts an image that mimics the view from the perspective of a camera positioned
`
`exterior to the vehicle. (Ex. 1020, Wolberg Decl. at 47-48.) I understand that
`
`Patent Owner asserts that “Yamamoto does not disclose how the images from these
`
`cameras are combined into a synthesized image that approximates a view as would
`
`be seen by a virtual camera.” (Paper 14, Patent Owner Resp. at 29.) Techniques
`
`for generating the synthesized image shown in Yamamoto, however, were known to
`
`those of ordinary skill in the art and disclosed by Mitsubishi. In particular,
`
`Mitsubishi explains:
`
`The image correcting unit 13 corrects the images in order to join
`
`(synthesize) them by performing a coordinate conversion on data with
`
`the same patterns (for example, PTA and PTA’) retrieved from
`
`
`
`18
`
`
`
`
`
`overlapping portions c, d in P1 image 14 and P2 image 15. . . . The
`
`overlapping regions of P2 image 15 and P3 image 16 are corrected in
`
`the same way to join and synthesize P2 image 15 and P3 image 16
`
`. . . . Because the display areas 1a, 2a, 3a in images P1, P2 and P3
`
`cover the entire field of view from cameras 1, 2 and 3, the desired
`
`portions (denoted by 7’) are included in the series of images that are
`
`displayed on the image display screen 7 by the display output unit 6.
`
`In other words, the area denoted by 7’ in FIG. 3 is displayed on the
`
`image display unit 7.
`
`(Ex. 1005, Mitsubishi at 7, Fig 3.)
`
`
`
`Therefore, Mitsubishi discloses a system that matches the patterns in the
`
`overlapping portions and joins the three separate images. Because the system aligns
`
`the overlap regions, it will minimize any duplication from the overlap zones
`
`between each camera. Mitsubishi further discloses an arithmetic processing unit for
`
`merging the three images, which results in a synthesized image that approximates
`
`the view from a single virtual camera.
`
`
`
`19
`
`
`
`
`
`26.
`
` My opinion is further supported by the declaration of Dr. Etienne-
`
`Cummings. In his declaration, Dr. Etienne-Cummings performs a demonstration in
`
`which he took three photographs at three positions around a vehicle, and merged the
`
`images using various techniques. (Ex. 2004, Etienne-Cummings Decl. at 45-51.)
`
`In my opinion, Dr. Etienne-Cummings’ demonstration shows that a synthesized
`
`image generated using the technique taught by Mitsubishi will approximate the
`
`view from a single virtual camera.
`
`27. According to Dr. Etienne-Cummings, he took pictures from three
`
`locations around a Honda Accord: one at the driver side mirror, another from the
`
`passenger side mirror, and one from the rear of the car. (Ex. 2004, Etienne-
`
`Cummings Decl. at 45-46.) The three images are shown below, respectively.
`
`
`
`
`
`
`
`20
`
`
`
`
`
`
`
`(Ex. 2004, Etienne-Cummings Decl. at 45-46 (top is image from driver side mirror;
`
`middle is image from passenger side mirror; and bottom is image from rear).)
`
`28. Dr. Etienne-Cummings merges the three images in several different
`
`ways and asserts that no arrangement of the three images would result in an
`
`approximation of a view from a single virtual camera. (Ex. 2004, Etienne-
`
`Cummings Decl. at 50-51.) I disagree with Dr. Etienne-Cummings conclusion.
`
`Although none of the merged images are perfect, his demonstration confirms that an
`
`image approximating the view from a virtual camera can be generated using the
`
`technique disclosed in Mitsubishi.
`
`29. For example, in one of his demonstrations, Dr. Etienne-Cummings
`
`creates a synthesized image by scaling the center image and matching the
`
`overlapping portions of each image. (Ex. 2004, Etienne-Cummings Decl. at 45-51.)
`
`The result is reproduced below.
`
`
`
`21
`
`
`
`
`
`
`
`(Ex. 2004, Etienne-Cummings Decl. at 50.)
`
`30. As shown in the following side-by-side comparison, Dr. Etienne-
`
`Cummings’ demonstration mirrors
`
`the
`
`technique disclosed
`
`in Mitsubishi.
`
`Mitsubishi also explains that only “the area denoted by 7’ in FIG. 3 is displayed on
`
`the image display unit 7.” (Ex. 1005, Mitsubishi at 7.) Thus, I have highlighted the
`
`portion of Dr. Etienne-Cummings’ merged image that approximates the area 7’ in
`
`Figure 3 of Mitsubishi.
`
`Fig. 3 of Mitsubishi
`
`Dr. Etienne-Cummings
`
`31. The figure below shows the synthesized composite image from
`
`
`
`
`
`Dr. Etienne-Cummings’ declaration that would be displayed to the driver according
`
`
`
`22
`
`
`
`
`
`to the technique disclosed by Mitsubishi, prior to any processing that may be done
`
`to smooth the images. In this resulting image, there are some minor distortions near
`
`the boundaries between the images. In fact, it appears Dr. Etienne-Cummings has
`
`attempted to complicate the resulting image by using three images taken from a car
`
`parked on a busy street with multiple houses and cars in the immediate background,
`
`even though the specification explains that “[i]n most driving situations, objects
`
`imaged by rearward-facing image capture devices above the horizon are at a long
`
`distance from the vehicle.” (Ex. 1001, ’724 Patent at 15:64-16:2.) Nonetheless, the
`
`resulting image as a whole approximates the view from a single virtual camera and
`
`would provide a useful rearview to the driver of the vehicle. Further, the resulting
`
`image would not have discouraged or otherwise led a PHOSITA to abandon the
`
`three-camera arrangement of Yamamoto. Therefore, Dr. Etienne-Cummings’
`
`declaration confirms my opinion that the technique disclosed by Mitsubishi and
`
`Yamamoto would generate a synthesized image that approximates the view from a
`
`single virtual camera.
`
`32. Further, although Dr. Etienne-Cummings asserts that “[t]he combined
`
`image continues
`
`to have many
`
`instances of
`
`incongruity, distortions and
`
`
`
`
`
`23
`
`
`
`
`
`discontinuities,” he fails to identify anything in the ’724 Patent that resolves such
`
`imperfections. (Ex. 2004, Etienne-Cummings Decl. at 50-51.) Again, the ’724
`
`Patent only discloses compressing the center image, which may help scale the
`
`center image but does not resolve parallax. And the claims specify a synthesized
`
`image that approximates the view from a virtual camera rather than a perfect image.
`
`2.
`
`A PHOSITA would have found it obvious to scale images as
`part of an image synthesis process.
`
`33.
`
`I understand that Patent Owner argues Mitsubishi fails to disclose
`
`scaling the images to produce a synthesized image. (Paper 14, Patent Owner Resp.
`
`at 26-28.) I disagree with Patent Owner’s assertion. In my opinion, a PHOSITA
`
`would have recognized that Mitsubishi teaches a general “image synthesis process”
`
`for combining multiple images was based on the registration, or alignment, of the
`
`images (Ex. 1005, Mitsubishi at 5), and it was well known at the time that scaling is
`
`an important element of image registration (Ex. 1026, Brown at 10). Thus, in my
`
`opinion, a PHOSITA would have found it obvious to scale the images produced by
`
`the three-camera arrangement in Yamamoto to generate a synthesized image as
`
`shown in Figure 5 of Yamamoto.
`
`34. Significantly, Mitsubishi generally
`
`teaches
`
`the use of
`
`image
`
`registration to merge or synthesize the images captured by three different cameras.
`
`For example, Mitsubishi explains that “[t]he image correcting unit 13 corrects the
`
`images in order to join (synthesize) them by performing a coordinate conversion on
`
`
`
`24
`
`
`
`
`
`data with the same patterns . . . retrieved from overlapping portions . . . .” (Ex.
`
`1005, Mitsubishi at 7.) In my opinion, a PHOSITA would have readily understood
`
`that the image correcting unit described in Mitsubishi performs image registration,
`
`i.e., image alignment, to merge the three images.
`
`35. Further, a PHOSITA would have recognized that image registration
`
`techniques for aligning images from several cameras include basic methods such as
`
`translation, scaling and rotation. For example, a highly cited 1992 survey paper on
`
`image registration by Brown et al. describes scaling as one of the most commonly
`
`used Cartesian operations for image matching:
`
`The most commonly used registration transformation is the affine
`
`transformation which is sufficient to match two images of a scene
`
`taken from the same viewing angle but from a different position, i.e.,
`
`the camera can be moved, and it can be rotated around its optical axis.
`
`This affine transformation is composed of the Cartesian operations of
`
`a scaling, a translation, and a rotation.
`
`(Ex. 1026, Brown at 10
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
