`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.,
`Petitioner,
`
`v.
`
`COREPHOTONICS, LTD.,
`Patent Owner.
`____________
`
`Case No. IPR2020-00905
`U.S. Patent No. 10,225,479
`____________
`
`
`PATENT OWNER’S RESPONSE TO
`PETITION FOR INTER PARTES REVIEW
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`TABLE OF CONTENTS
`
`“fused image with a point of view (POV) of the Wide camera”
`
`INTRODUCTION .................................................................. 1
`I.
`SUMMARY OF ARGUMENT ................................................ 1
`II.
`III. LEVEL OF ORDINARY SKILL IN THE ART (POSITA) ....... 3
`IV. LEGAL STANDARDS ........................................................... 4
`V. OVERVIEW OF THE ’479 PATENT ...................................... 5
`VI. CLAIM CONSTRUCTION ..................................................... 8
`A.
`(claims 1 and 23) ............................................................................. 8
`VII. OVERVIEW OF SELECTED PRIOR ART ........................... 14
`A.
`Parulski .......................................................................................... 14
`VIII. OBVIOUSNESS ................................................................... 26
`A.
`Over the Combination of Parulski and Konno (Ground 1) ........... 26
`B.
`Parulski, Konno and Szeliski (Ground 2) ..................................... 31
`C.
`Parulski, Konno, Szeliski and Segall (Ground 3) ......................... 32
`D.
`Parulski, Konno and Stein (Ground 4) .......................................... 33
`IX. SECONDARY CONSIDERATIONS / OBJECTIVE INDICIA
`OF NON-OBVIOUSNESS .................................................... 35
`
`Claims 5–9 and 27–31 Are Not Obvious Over the Combination of
`
`Claims 1, 10–14, 16, 18, 23, 32–26, 38, and 40 Are Not Obvious
`
`Claims 2–4 and 24–26 Are Not Obvious Over the Combination of
`
`Claims 15 and 37 Are Not Obvious Over the Combination of
`
`i
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Industry Praise / Licensing ............................................................ 37
`A.
`Commercial Success ..................................................................... 44
`B.
`Failure of Others / Copying ........................................................... 45
`C.
`X. CONCLUSION .................................................................... 47
`
`
`
`ii
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Cases
`
`TABLE OF AUTHORITIES
`
`Ariosa Diagnostics v. Verinata Health, Inc.,
`805 F.3d 1359 (Fed. Cir. 2015) ................................................................. 5
`
`Graham v. John Deere Co. of Kansas City,
`383 U.S. 1 (1966). ................................................................................... 35
`
`Harmonic Inc. v. Avid Tech., Inc.,
`815 F.3d 1356 (Fed. Cir.2016) .................................................................. 4
`
`Heidelberger Druckmaschinen AG v. Hantscho Commercial Products, Inc.,
`21 F.3d 1068 (Fed. Cir. 1994) ................................................................. 45
`
`In re Magnum Oil Tools Int’l, Ltd.,
`829 F.3d 1364 (Fed. Cir. 2016) ................................................................. 4
`
`Mintz v. Dietz & Watson, Inc.,
`679 F.3d 1372 (Fed. Cir. 2012) ............................................................... 35
`
`SAS Inst., Inc. v. Iancu,
`138 S. Ct. 1348 (2018) .............................................................................. 4
`
`Wasica Finance GMBH v. Continental Auto. Systems,
`853 F.3d 1272 (Fed. Cir. 2017) ................................................................. 5
`
`
`
`
`
`
`iii
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`PATENT OWNER’S EXHIBIT LIST
`
`2005
`
`2006
`
`Exhibit No Description
`2001
`Declaration of John C. Hart, Ph.D.
`2002
`Fredo Durand, Presentation Titled “Photography 101”
`2003
`Curriculum Vitae of John C. Hart, Ph.D.
`2004
`Complaint for Patent Infringement, Dkt. No. 1, Case No.
`19-cv-4809 (United States District Court, Northern District
`of California)
`Answer to Complaint for Patent Infringement, Dkt. No. 17,
`Case No. 19-cv-4809 (United States District Court, Northern
`District of California)
`Corephotonics Proposal: “Dual Aperture Image Fusion
`Technology, Proposed Engagement Framework” (June 22,
`2014)
`Email chain with emails dating from July and August 2014
`Email chain with emails dating from March 2015
`Email dated December 21, 2015
`Email chain with emails dating from August 2016
`Email dated May 23, 2013
`Email dated May 23, 2013
`Declaration of Eran Kali
`Transcript of January 21, 2021 Video-Recorded Deposition
`of Fredo Durand, Ph.D.
`Declaration of Duncan Moore, Ph.D.
`Rudolf Kingslake, “Optics in Photography” (1992)
`Curriculum Vitae of Duncan Moore, Ph.D.
`Email chain with emails dating from June and July 2013
`Email chain with emails dating from June and July 2013
`Email chain with emails dating from October 2013
`Technology Evaluation Agreement dated August 8, 2013
`Email chain with emails dating from September 18, 2013
`Email dated May 21, 2014
`Reserved
`Reserved
`Deposition transcript of José Sasián, November 9, 2020
`José Sasián, Introduction to Lens Design (2019), hardcopy
`
`2007
`2008
`2009
`2010
`2011
`2012
`2013
`2014
`
`2015
`2016
`2017
`2018
`2019
`2020
`2021
`2022
`2023
`2024
`2025
`2026
`2027
`
`iv
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Tigran V. Galstian, Smart Mini-Cameras (2014)
`Dmitry Reshidko and Jose Sasián, “Optical analysis of min-
`iature lenses with curved imaging surfaces,” Applied Optics,
`Vol. 54, No. 28, E216-E223 (October 1, 2015)
`José Sasián, Introduction to Aberrations in Optical Imaging
`Systems (2013), hardcopy
`Yufeng Yan and Jose Sasián, “Miniature Camera Lens De-
`sign with a Freeform Surface,” Design and Fabrication Con-
`gress (2017)
`Peter Clark, “Mobile platform optical design,” Proc. SPIE
`9293, International Optical Design Conference 2017,
`92931M (17 December 2014)
`Jane Bareau and Peter P. Clark, “The Optics of Miniature
`Digital Camera Modules,” SPIE Vol. 6352, International Op-
`tical Design Conference 2006, 63421F.
`Yufeng Yan, “Selected Topics in Novel Optical Design,”
`Ph.D. Dissertation (2019)
`Declaration of Jose Sasián, Ph.D. from IPR2020-00489
`Transcript of January 26, 2021 Video-Recorded Deposition
`of Fredo Durand, Ph.D.
`U.S. Patent No. 8,989,517 (“Morgan-Mar”)
`Forsyth and Ponce, “Computer Vision: A Modern Ap-
`proach” (1st ed.) (2003)
`
`2028
`2029
`
`2030
`
`2031
`
`2032
`
`2033
`
`2034
`
`2035
`2036
`
`2037
`2038
`
`
`
`
`v
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`I.
`
`INTRODUCTION
`
`Patent Owner Corephotonics, Ltd. submits this response to the Petition
`
`filed by Apple Inc., requesting inter partes review of claims 1–16, 18, 23–36–
`
`38, and 40 of U.S. Patent No. 10,225,479 (Ex. 1001, ’479 patent). The Board
`
`granted institution on four grounds of obviousness, each involving a combi-
`
`nation including at least Parulski (Ex. 1005) with Konno (Ex. 1015). Core-
`
`photonics submits that the arguments presented herein and the additional
`
`evidence submitted, such as the testimony from Patent Owner’s expert witness
`
`John Hart (Ex. 2001), demonstrate that Apple has failed to establish obvious-
`
`ness of the challenged claims and that Apple’s grounds should be rejected.
`
`II. SUMMARY OF ARGUMENT
`
`Apple’s petition suffers from multiple flaws that affect every ground of
`
`its petition. Most fundamentally, its obviousness analysis is premised on a
`
`plainly incorrect construction of the term “point of view” (POV). As ex-
`
`plained below, Apple’s construction rewrites the term “point of view” in the
`
`claims to mean “field of view” (FOV). However, these two terms refer to two
`
`distinct concepts in the art. “Point of view” is a concept that relates to how the
`
`shapes and positions of objects within images differ when those images are
`
`taken by different cameras, with different locations and orientations. (Ex.
`
`1
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`1001 at 5:10–33.) “Field of view,” by contrast is the angle that defines how
`
`wide a portion of a scene a given camera and lens can capture. (Id. at 7:2–22.)
`
`It is a property of the camera and lens that is independent of how the camera
`
`is located and oriented and of what scene it is directed toward.
`
`The specification uses the terms POV and FOV to refer to these two
`
`very different concepts, consistent with the use of these terms in the art. Fur-
`
`ther, the challenged claims each use both POV and FOV and use these terms
`
`to refer to distinct concepts.
`
`Apple’s plainly erroneous claim construction, rewriting POV to mean
`
`FOV renders each of its grounds incomplete. Apple has attempted to show
`
`that the prior art discloses the limitation “fused image with a point of view
`
`(POV) of the Wide camera” (id. at 13:47–48, 15:65–66), by showing the pur-
`
`ported fused image has the FOV of the Wide camera. Apple has provided no
`
`evidence that the prior art references, alone or in combination, provide a fused
`
`image with the POV of the wide camera, under any proper construction of
`
`POV. This is because Apple entirely fails to demonstrate anything about the
`
`shapes or positions of objection in the “fused image,” which are exactly the
`
`things that it must show match the POV of the wide camera, as the ’479 patent
`
`uses the term POV.
`
`2
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Even setting this fundamental flaw aside, Apple’s petition improperly
`
`mixes and matches elements from unrelated embodiments of Parulski and
`
`other prior art, without a convincing explanation for why a POSITA would
`
`have been motivated to do so.
`
`These and other flaws in Apple’s obviousness grounds are explained
`
`further below.
`
`In addition, there exist considerable objective evidence confirming the
`
`non-obviousness of the challenged claims. Corephotonics has, in a non-litiga-
`
`tion context, licensed the technology claimed in the ’479 patent to numerous
`
`companies, including some of the largest smartphone makers in the world. In
`
`2019, Samsung acquired Corephotonics and its camera technologies for
`
`$155m. And, critically, Apple itself asked Corephotonics for its patented tech-
`
`nology, evaluated and studied it for years, and then asked for a portfolio li-
`
`cense from Corephotonics.
`
`III. LEVEL OF ORDINARY SKILL IN THE ART (POSITA)
`
`For purposes of this proceeding, Patent Owner accepts Petitioner’s def-
`
`inition of the level of ordinary skill, namely that a POSITA “would include
`
`someone who had, as of the claimed priority date of the ’479 Patent, a bache-
`
`lor’s or the equivalent degree in electrical and/or computer engineering or a
`
`3
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`related field and 2-3 years of experience in imaging systems including image
`
`processing and lens design,” and that “that someone with less formal educa-
`
`tion but more experience, or more formal education but less experience could
`
`have also met the relevant standard for a POSITA.” Ex. 1003, ¶ 13. See also
`
`Declaration of John C. Hart, Ph.D. (“Hart Declaration”) (Ex. 2001), ¶¶ 14-18.
`
`IV. LEGAL STANDARDS
`
`The petitioner has the burden to clearly set forth the basis for its chal-
`
`lenges in the petition. Harmonic Inc. v. Avid Tech., Inc., 815 F.3d 1356, 1363
`
`(Fed. Cir.2016) (citing 35 U.S.C. § 312(a)(3) as “requiring IPR petitions to
`
`identify ‘with particularity ... the evidence that supports the grounds for the
`
`challenge to each claim’”). A petitioner may not rely on the Board to substi-
`
`tute its own reasoning to remedy the deficiencies in a petition. SAS Inst., Inc.
`
`v. Iancu, 138 S. Ct. 1348, 1355 (2018) (“Congress chose to structure a process
`
`in which it’s the petitioner, not the Director, who gets to define the contours
`
`of the proceeding.”); In re Magnum Oil Tools Int’l, Ltd., 829 F.3d 1364, 1381
`
`(Fed. Cir. 2016) (rejecting the Board’s reliance on obviousness arguments that
`
`“could have been included” in the petition but were not, and holding that the
`
`Board may not “raise, address, and decide unpatentability theories never pre-
`
`sented by the petitioner and not supported by the record evidence”); Ariosa
`
`4
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Diagnostics v. Verinata Health, Inc., 805 F.3d 1359, 1367 (Fed. Cir. 2015)
`
`(holding that “a challenge can fail even if different evidence and arguments
`
`might have led to success”); Wasica Finance GMBH v. Continental Auto. Sys-
`
`tems, 853 F.3d 1272, 1286 (Fed. Cir. 2017) (holding that new arguments in a
`
`reply brief are “foreclosed by statute, our precedent, and Board guidelines”).
`
`V. OVERVIEW OF THE ’479 PATENT1
`
`The ’479 patent describes and claims techniques for making “thin dig-
`
`ital cameras with optical zoom operating in both video and still mode.” (Ex.
`
`1001, ’479 patent at 3:27–28.) As the patent explains, zoom is “commonly
`
`understood as a capability to provide different magnifications of the same
`
`scene and/or object by changing the focal length of an optical system.” (Ex.
`
`1001, ’479 patent at 1:44–49.) Traditionally, this was accomplished by me-
`
`chanically moving lens elements relative to one another. (Ex. 1001, ’479 pa-
`
`tent at 1:49–51.) Another approach is “digital zooming,” where the focal
`
`length of the lens is kept unchanged, but the image is cropped and digitally
`
`manipulated to produce an image that is magnified but has a lower resolution.
`
`(Ex. 1001, ’479 patent at 1:55–38.)
`
`
`1 See Hart Decl., ¶¶ 31-35.
`
`5
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`The ’479 patent describes an approach to approximating the effect of a
`
`zoom lens (which varies its focal length) by using two lens systems (a “wide”
`
`and a “tele” lens system) with different fixed focal lengths. (Ex. 1001, ’479
`
`patent at 3:34–54.) Various computational means are used to take the images
`
`from these two lenses to produce an output that approximate a system with
`
`mechanical zoom. This approach can produce a device that is smaller, lower
`
`cost, and more reliable than devices that use mechanical zoom. (Ex.
`
`1001, ’479 patent at 1:51–53.)
`
`
`
`(Ex. 1001, ’479 patent, Fig. 1B)
`
`Relevant to the claims of the ’479 patent, the specification describes
`
`combining still images using the technique of “fusion.” (Ex. 1001, ’479 patent
`
`at 3:48–54.) A “fused” image includes information from both the wide and
`
`tele images. (Id.) One approach to performing fusion is shown in Figure 5:
`
`6
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`
`
`(Ex. 1001, ’479 patent, Fig. 5.)
`
`Making a compact, high-quality dual-aperture zoom system requires
`
`lenses with particular characteristics. The ’479 patent teaches lens designs for
`
`the tele lens which provide a small “total track length” relative to their focal
`
`length, which means that they have a compact size in light of the degree of
`
`magnification that they provide. (Ex. 1001, ’479 patent at 12:38–53.) One of
`
`7
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`the lens designs taught by the ’479 patent and covered by several of the chal-
`
`lenged claims is shown in Figure 9:
`
`(Ex. 1001, ’479 patent, Fig. 9.)
`
`
`
`The lens aspects of the ’479 patent are described further in Dr. Moore’s
`
`declaration. (E.g., Ex. 2015, Moore Decl., ¶¶ 31–34.)
`
`VI. CLAIM CONSTRUCTION
`
`A.
`
`“fused image with a point of view (POV) of the Wide camera”
`(claims 1 and 23)
`
`Petitioner and Dr. Durand contend that this term should be construed as
`
`“a fused image that maintains the Wide camera’s field of view or both the
`
`8
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Wide camera’s field of view and position.” (Ex. 1003, ¶¶ 29–33.) Patent
`
`Owner disagrees, for the reasons explained below. See Hart Decl., ¶ 36.
`
`Under this construction there are two ways to meet the “point of view”
`
`requirement. Either, the fused image can maintain the Wide camera’s (a) field
`
`of view or (b) field of view and position. However, the second of these two
`
`options is superfluous, as if the image has both the field of view and position
`
`of the Wide camera, then it also necessarily has the field of view of the Wide
`
`camera. Apple’s construction is logically equivalent to the construction “a
`
`fused image that maintains the Wide camera’s field of view.” See Hart Decl.,
`
`¶ 37.
`
`Even the superfluous “Wide camera’s . . . position” portion of the con-
`
`struction does not line up with the term “position” as it is used in the ’479
`
`patent’s discussion of “POV.” During his deposition, Dr. Durand confirmed
`
`that he understood the “Wide camera’s . . . position” to refer to the “3D XYZ
`
`location of the camera.” (Ex. 2036, Durand Depo. at 21:3–7.) But when the
`
`specification refers to “position POV” in its discussion of “combination”
`
`POVs, it is referring to the “position of either sub-camera image.” (Ex.
`
`1001, ’479 patent at 5:14–16.) That is, “position POV” is based on the posi-
`
`tions of images, not the positions of cameras. An image position may differ
`
`9
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`because the camera is located in a different position, but it also may differ
`
`because camera, located in the same position, has been pointed in a different
`
`direction. For this reason as well, Petitioner’s proposed construction is incon-
`
`sistent with how the patent specification uses the relevant terms. See Hart
`
`Decl., ¶ 38.
`
`The effect of Petitioner’s construction is to replace the term “point of
`
`view” in the claims with the term “field of view.” This is not consistent with
`
`how a POSITA would understand these phrases or with how they are used in
`
`the ’479 patent. For example, claim 1 refers to both “a field of view FOVW”
`
`of the wide camera and “a point of view (POV)” of the wide camera, with no
`
`suggestion they are the same thing or that one term is the antecedent basis for
`
`the other. (Ex. 1001, ’479 patent at 13:25–26, 13:48.) See Hart Decl., ¶ 39.
`
`In the specification, the ’479 patent clearly defines “FOV” as a planar
`
`angle, representable in degrees: “As used herein, the FOV is measured from
`
`the center axis to the corner of the sensor (i.e. half the angle of the normal
`
`definition).” (Ex. 1001 at 7:11–13.) Examples of FOV values are given in de-
`
`grees (id. at 7:20–22), and FOV is used as a parameter to the tangent function,
`
`further confirming that it is a simple angle (id. at 7:7–8). See Hart Decl., ¶ 40.
`
`10
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Dr. Durand agreed during his deposition that his construction of “POV”
`
`matches what the patent calls field of view:
`
`Q. So when you’re using the term “field of view” in this con-
`struction, you’re reviewing -- you're referring to how much of
`the scene is captured by the camera; is that right?
`
`A. This is a vague version of the definition, I would say one
`definition of the field of view. For example, the horizontal field
`of view is to look at the angle between the two edges of the --
`of the image.
`
`(Ex. 2036 at 22:4–12.)
`
`This definition matches what the ’479 patent specification calls the
`
`“normal definition” of FOV (the ’479 patent uses half that “normal” value in
`
`its formulas). (Ex. 1001, ’479 patent at 7:11–13.) See Hart Decl., ¶¶ 41-42. As
`
`Dr. Durand testified, this FOV is an inherent property of the camera and lens,
`
`and independent of where they are pointed or what they see:
`
`Q. Would you agree that a camera’s field of view is a property
`of the camera that’s independent of what direction the camera is
`pointing?
`
`A. So one definition or understanding of field of view would be
`-- would indeed be just an angle that’s a property of the combi-
`nation of a camera and the lens.
`
`(Ex. 2036 at 22:25–23:6.)
`
`POV is defined in the specification quite differently. It refers to how
`
`objects are “seen by each sub-camera,” i.e., how objects “with be shifted and
`
`11
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`have different perspective (shape)” for the two cameras. (Ex. 1001 at 5:10–
`
`14.) This POV depends on the position and orientation of the camera and can-
`
`not be expressed fully by a single numerical angle. Rather, as the ’479 patent
`
`explains, using a camera with a different POV can both shift an object (change
`
`its position in the image) and change the perspective of an object (changes its
`
`apparent shape in the image). (Ex. 1001 at 5:10–16.) See Hart Decl., ¶ 43.
`
`Examples of changing POV can be seen in image pairs (a)-(b) and (d)-
`
`(e) from Szeliski Figure 1.1:
`
`(Ex. 1013, Szeliski at 468.) See also Hart Decl., ¶ 44.
`
`
`
`12
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`The ’479 patent refers to “combination” possibilities where an output
`
`image reflects only some aspects of a given POV, such as “Wide perspective
`
`POV” or “Wide position POV.” (Ex. 1001, ’479 patent at 5:15–19.) But, when
`
`it refers to “Wide POV,” without qualification, it is referring to the complete
`
`Wide POV, both perspective and position. (Ex. 1001, ’479 patent at 5:10–14;
`
`5:23–26.) See Hart Decl., ¶ 45.
`
`In summary, a POSITA would not agree that the term POV in the
`
`phrase “fused image with a point of view (POV) of the Wide camera” can be
`
`replaced with the distinct term FOV. Further, a POSITA would understand
`
`that POV of the Wide camera in this phrase refers to the full Wide camera
`
`POV and not to “combination” outputs that have a Wide “perspective POV”
`
`and Tele “position POV” or vice versa. (Ex. 1001, ’479 patent at 5:13–23.)
`
`Therefore, a POSITA would understand this term to mean “fused image in
`
`which the positions and shapes of objects reflect the POV of the Wide cam-
`
`era.” See Hart Decl., ¶ 46.
`
`13
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`VII. OVERVIEW OF SELECTED PRIOR ART
`
`A.
`
`Parulski2
`
`The Parulski patent was published as U.S. Patent No. 7,859,588 and
`
`issued on December 28, 2010. (Ex. 1005.) It was filed on March 9, 2007. (Ex.
`
`1005, Parulski, at 1.)
`
`Parulski at the Summary of the Invention includes an overview of the
`
`preferred embodiments and their motivations at 7:54 – 8:19. These embodi-
`
`ments include the use of the secondary image from the additional lens “to
`
`sharpen portions of the primary image … where the secondary output image
`
`is captured … at a different focus position … ; to modify the dynamic range
`
`of the primary image … ; to provide scene analysis data for setting the capture
`
`parameters for the primary image; or to replace portions of the primary image
`
`… with corresponding portions of [a longer exposure] secondary image.” Id.
`
`at 7:56-8:5. As this list suggests, these various preferred embodiments are de-
`
`signed to achieve different results, and they take different approaches to doing
`
`so. A POSITA would not understand all of Parulski’s specification (or all of
`
`the portions cited by Apple and Dr. Durand) to be part of the same embodi-
`
`ment or even to be compatible with one another. See Hart Decl., ¶ 52.
`
`
`2 See Hart Decl., ¶¶ 51-71.
`
`14
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`Parulski discloses a camera system comprising “the use of two (or
`
`more) image capture stages, wherein an image capture stage is composed of a
`
`sensor, a lens and a lens focus adjuster, in a multi-lens digital camera in which
`
`the two (or more) image capture stages can be used to separately capture im-
`
`ages of the same scene so that one image capture stage can be used for auto-
`
`focus and other purposes while the other(s) is used for capturing an image.”
`
`Id. at 8:6-13. “More specifically, the non-capturing image stage may advan-
`
`tageously be used to provide a secondary image that can be used to modify or
`
`otherwise augment, e.g., the focus or dynamic range of the primary image.”
`
`Id. at 8:16-19. See Hart Decl., ¶ 53.
`
`Parulski uses Figure 1 reproduced below to illustrate an “image capture
`
`assembly” including “two imaging stages 1 and 2.” Id. at 12:42-43. The image
`
`capture stages 1 and 2 comprise the zoom lenses 3 and 4 and the image sensors
`
`12 and 14… .” Id. at 12:66-67. Lenses 3 and 4 “have different focal lengths to
`
`provide and extended optical zoom range for the image capture assembly.” Id.
`
`at 10:15-17. See Hart Decl., ¶ 54.
`
`15
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`
`
`Parulski discloses that this design can facilitate autofocusing. “The con-
`
`trol processor and timing generator 40 controls the digital multiplexers 34 and
`
`36 in order to select one of the sensor outputs (12e or 14e) as the captured
`
`image signal, and to select the other sensor output (14e or 12e) as the autofo-
`
`cus image signal.” Id. at 14:1-5. “Briefly summarized, the image processor 50
`
`produces the focus detection signals that drive the first and second focus ad-
`
`justers, that is, the zoom and focus motors 5a and 5b.” See Hart Decl., ¶ 55.
`
`Parulski uses Figure 3, reproduced below, to show how the image cap-
`
`ture assembly in Figure 1 is used to capture images.
`
`16
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`
`
`The decision at block 102 uses the zoom position to determine whether
`
`the first stage (image capture stage 1 in Fig. 1) or the second stage (image
`
`capture stage 2 in Fig. 1) has the more appropriate focal length for that zoom
`
`setting. As an example, we can assume that the zoom position is not greater
`
`17
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`than X and the steps on the left hand side of Fig. 3 starting with 104 are se-
`
`lected, previewing images from the first stage and using the second stage to
`
`assist the autofocus of the first stage.3 See Hart Decl., ¶ 57.
`
`Block 108 represents the step in Fig. 3 that represents the action of the
`
`image processor (block 50 in Fig. 1) that accesses the images captured by both
`
`stage 1 and stage 2. “In block 104 … the first image capture stage 1 is used to
`
`capture images in the preview mode, while the second image capture stage 2
`
`is used to capture autofocus images. The first image capture stage 1 continues
`
`to capture images for preview on the display 70 (block 110) while, in block
`
`106, the second image capture stage 2 is used to capture autofocus images for
`
`autofocus of the first image capture stage 1, which are processed by the image
`
`processor 50 and used in block 108 to focus the first image capture stage 1.”
`
`Id. at 15:57-67. See Hart Decl., ¶ 58.
`
`Parulski discloses three options for block 108: “rangefinder,” “hill
`
`climbing” and “rangemap.” See Hart Decl., ¶ 59.
`
`
`3 During his deposition, Dr. Durand testified that the only situation where he
`had offered an opinion that Parulski satisfied the necessary claim elements
`was when the zoom position equals 1 (no zoom) and the output field of
`view equals the wide image view of view. (Ex. 2036, Durand Depo. at
`64:20–65:3, 65:18–67:5.) Given that this is Dr. Durand’s position, the case
`where the zoom position is greater than X is not relevant to Dr. Durand’s
`opinions.
`
`18
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`The “rangefinder” option is shown in Fig. 4, reproduced below Step
`
`258 indicates that the shutter button is pressed halfway down (S0 à S1), ini-
`
`tiating autofocus. “The cropped and upsampled autofocus image is then cor-
`
`related with the other autofocus image to identify the pixel shift between the
`
`two autofocus images (block 264) and thereby produce the focus detection
`
`signal.” Id. at 16:54-58. Step 266 indicates a “rangefinder calibration curve”
`
`is used to convert the “focus detection signal” into the single value sent by
`
`step 268 to focus the “first image” in block 108 of Fig. 3. See Hart Decl., ¶
`
`60.
`
`The “hill climbing” option is illustrated in Fig. 5 and disclosed by Id.
`
`at 17:7-56. It uses the second capture stage to experimentally adjust its focus
`
`
`
`19
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`to maximize contrast to find the optimal focus setting for the first capture
`
`stage. The advantage of this approach is that the iterated adjustments in the
`
`second capture stage can remain hidden while the user observes the preview
`
`image updated in the first capture stage, even while adjusting zoom settings
`
`or reorienting the camera to different focal points in the scene. See Hart Decl.,
`
`¶ 61.
`
`The “rangemap” option is illustrated in Fig. 11, reproduced below, and
`
`disclosed by Id. at 21:49-22:49. The “rangemap” option uses the rangefinder
`
`calibration curve in block 482. Whereas a single pixel offset is used to produce
`
`a single range value in the “rangefinder” option, block 482 shows that the
`
`“rangemap” option determines “the distances to different portions of the im-
`
`ages.” Id. at 20:15. See Hart Decl., ¶ 62.
`
`
`
`20
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`This range map is described as being usable “for a variety of purposes”
`
`(id. at 20:51–21:6), but it is noteworthy that none of the example uses listed
`
`in the specification involves “fusing” or otherwise combining image data from
`
`the two images. The first three example all involve identifying object bound-
`
`aries or motion tracking of objects, which does not have anything to do with
`
`fusion, per se. (Id. at 20:54–62.) The fourth example describes blurring por-
`
`tions of the output image. (Id. at 20:63–65.) The last three examples describe
`
`increasing or decreasing the brightness of portions of the image. (Id. at 20:66–
`
`21:6.) See Hart Decl., ¶ 63.
`
`A POSITA would not understand the discussion of “blurring” in con-
`
`nection with Fig. 11 (id. at 20:63, 21:36–44) to be referring to fusing two im-
`
`ages. See Hart Decl., ¶ 64. Rather a single image (or portions of the image)
`
`can be digitally blurred using a variety of techniques. Id. Generally speaking,
`
`blurring an image involves reducing the magnitude of the high-frequency
`
`components of a image, while leaving the low-frequency components alone.
`
`Id. This has a similar effect to that of averaging the brightness values of the
`
`pixels in each local portion of the image. Id. One approach to blurring is to
`
`calculate the Fourier transform of an image to compute its frequency compo-
`
`nents, reduce the high frequency components using a filter, and then perform
`
`21
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`an inverse Fourier transform on the result. Id. This general approach is de-
`
`scribed, for example, in the Morgan-Mar reference that Apple relies on in the
`
`-00906 IPR. (Id.; Ex. 2037, Morgan-Mar at 3:36–54.)
`
`Likewise, the discussion of the dog being “sharpened” would not be
`
`understood by a POSITA to refer to fusing two images. (Ex. 1005 at 21:30–
`
`31.) See Hart Decl., ¶ 65. The same Fourier transform techniques used to blur
`
`can be used instead to sharpen—e.g., making edges in the image more prom-
`
`inent—by increasing the high-frequency components rather than decreasing
`
`them. Id. This is also explained in Morgan-Mar: “In the Fourier domain, N is
`
`not constrained to being an integer. As long as N>1, the blurring of the back-
`
`ground is increased. If N<1, the blurring of the background is reduced; in other
`
`words the background is sharpened, mimicking the effect of a greater depth
`
`of field than the original images.” (Id.; Ex. 2037, Morgan-Mar at 11:33–38.)
`
`Whereas “Fig. 3 depicts a flow diagram showing a method for perform-
`
`ing autofocus and for capturing digital still images according to a first embod-
`
`iment of the digital camera shown in Fig. 1” and “Fig. 8 depicts a flow
`
`diagram showing a method for performing autofocus and for capturing digital
`
`video images according to a first embodiment of the digital camera shown in
`
`Fig. 1[,]” “Fig. 14 depicts a flow diagram showing a method for enhancing
`
`22
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`the depth of field of an image by using images from both image capture stages
`
`according to an embodiment of the invention.” Ex. 1005 at 8:34-37, 48-51 and
`
`9:1-4. Parulski identifies a special, different method for “enhancing the depth
`
`of field of an image” than was disclosed for “performing autofocus and for
`
`capturing digital still images.”
`
`
`
`23
`
`
`
`Case Nos. IPR2020-00905
`U.S. Patent No. 10,225,479
`
`As before, we can focus on a case that the zoom position is not greater
`
`than X and focus the discussion on the left side of the flow diagram starting
`
`with block 504. See Hart Decl., ¶ 67. In blocks 504 and 506, the first image
`
`capture stage is used to capture and preview images while the second
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
