UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`APPLE INC.,
`Petitioner
`v.
`COREPHOTONICS, LTD.
`Patent Owner
`
`_______________
`U.S. Patent No. 10,230,898
`_______________
`DECLARATION OF FRÉDO DURAND, PH.D.
`UNDER 37 C.F.R. § 1.68 IN SUPPORT OF PETITION
`FOR INTER PARTES REVIEW
`
`APPL-1003 / Page 1 of 109
`Apple v. Corephotonics
`
`

`

` Declaration of Frédo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,230,898
`TABLE OF CONTENTS
`
`I.  
`INTRODUCTION ........................................................................................... 1 
`II.   QUALIFICATIONS ........................................................................................ 3 
`III.  LEVEL OF ORDINARY SKILL IN THE ART ............................................. 6 
`IV.   RELEVANT LEGAL STANDARDS ............................................................. 8 
`A.   Obviousness ........................................................................................... 8 
`V.   THE ’898 PATENT ....................................................................................... 10 
`A. 
`Summary of the ’898 Patent ................................................................ 10 
`B. 
`Prosecution History of the ’898 Patent ............................................... 15 
`VI.   CLAIM CONSTRUCTION .......................................................................... 17 
`VII.  GROUNDS .................................................................................................... 17 
`A.  Ground 1: Claims 1, 4, 8, 12, and 15 are unpatentable under
`35 U.S.C. § 103 over Golan in view of Martin and Togo. .................. 17 
`1. 
`Summary of Golan .................................................................... 17 
`2. 
`Summary of Martin ................................................................... 20 
`3. 
`Reasons to Combine Golan and Martin .................................... 23 
`4. 
`Summary of Togo ..................................................................... 28 
`5. 
`Reasons to Combine Togo with Golan and Martin .................. 35 
`6. 
`Claim 1 ...................................................................................... 39 
`7. 
`Claim 4 ...................................................................................... 72 
`8. 
`Claim 8 ...................................................................................... 75 
`9. 
`Claim 12 .................................................................................... 77 
`10.  Claim 15 .................................................................................... 78 
`Ground 2: Claim 9 is unpatentable under 35 U.S.C. § 103 over
`Golan in view of Martin, Togo, and Levey. ........................................ 79 
`1. 
`Summary of Levey .................................................................... 79 
`2. 
`Reasons to Combine Levey, Golan, and Martin, and Togo...... 80 
`3. 
`Claim 9 ...................................................................................... 82 
`
`B. 
`
`
`
`- i -
`
`APPL-1003 / Page 2 of 109
`
`

`

` Declaration of Frédo Durand, Ph.D.
`
` Inter Partes Review of U.S. Patent 10,230,898
`Ground 3: Claims 11 and 19 is unpatentable under 35 U.S.C. § 103
`C. 
`over Golan in view of Martin, Togo, and Border. .............................. 85 
`1. 
`Summary of Border ................................................................... 85 
`2. 
`Reasons to Combine Border with Golan, Martin, and Togo .... 87 
`3. 
`Claim 11 .................................................................................... 89 
`4. 
`Claim 19 .................................................................................... 94 
`D.  Ground 4: Claims 10 and 20 are unpatentable under 35 U.S.C.
`§ 103 over Golan in view of Martin, Togo, and Parulski. .................. 94 
`1. 
`Summary of Parulski ................................................................. 94 
`2. 
`Reasons to Combine Parulski, Golan, Martin, and Togo ......... 97 
`3. 
`Claim 10 ..................................................................................100 
`4. 
`Claim 20 ..................................................................................102 
`VIII.  DECLARATION ......................................................................................... 106 
`
`
`
`
`- ii -
`
`APPL-1003 / Page 3 of 109
`
`

`

`
`
`I.
`
`1.
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`INTRODUCTION
`
`I, Frédo Durand, have been retained by counsel for Apple Inc.
`
`(“Apple” or “Petitioner”) as a technical expert in connection with the proceeding
`
`identified above. I submit this declaration in support of Apple’s Petition for Inter
`
`Partes Review of U.S. Patent No. 10,230,898 (“the ’898 Patent”).
`
`2.
`
`Compensation for my work in this matter is based on an hourly rate.
`
`In addition, reasonable and customary expenses associated with my work and
`
`testimony in this matter are reimbursed. This compensation is not contingent on
`
`the outcome of this matter, nor is it contingent on the specifics of my testimony. I
`
`have no personal or financial stake, nor any interest in the outcome of the present
`
`proceeding.
`
`3.
`
`In the preparation of this declaration, I have studied:
`
`(1) The ’898 Patent, APPL-1001;
`
`(2) The prosecution file history of the ’898 Patent (’720 App), APPL-
`
`1002;
`
`(3) U.S. Patent Application Publication No. 2012/0026366 to Golan, et al.
`
`(“Golan”), APPL-1005;
`
`(4) U.S. Patent No. 8,081,206 to Martin et al. (“Martin”), APPL-1006;
`
`(5) U.S. Patent No. 7,990,422 to Ahiska et al. (“Ahiska”), APPL-1007;
`
`(6) U.S. Patent No. 7,859,588 to Parulski et al. (“Parulski”), APPL-1008;
`
`
`
`- 1 -
`
`APPL-1003 / Page 4 of 109
`
`

`

`
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`(7) U.S. Patent Application Publication No. 2008/0030592 to Border et
`
`al. (“Border”), APPL-1009;
`
`(8)
`
`J.P. Patent Application Publication No. JP 2011-55246 to Togo
`
`(“Togo”), APPL-1010;
`
`(9) U.S. Patent No. 8,553,106 to Scarff (“Scarff”), APPL-1012;
`
`(10) Richard Szeliski, Computer Vision: Algorithms and Applications,
`
`2011 (“Szeliski”), APPL-1013;
`
`(11) U.S. Patent No. 8,854,432 to Orimoto (“Orimoto”), APPL-1014;
`
`(12) U.S. Patent Application Publication No. 2012/0019704 to Levey
`
`(“Levey”), APPL-1015;
`
`(13) Xiong, et al., “A critical review of image registration methods,”
`
`International Journal of Image and Data Fusion, June 2010 (“Xiong”), APPL-1016;
`
`(14) Ralph E. Jacobson et al., The Manual of Photography: photographic
`
`and digital imaging, 9th Edition, 2000 (“Jacobson”), APPL-1017;
`
`(15) Hansen, et al., “Online continuous stereo extrinsic parameter
`
`estimation,” 2012 IEEE Conference on Computer Vision and Pattern Recognition,
`
`June 2012 (“Hansen”), APPL-1019;
`
`(16) U.S. Patent No. 9,571,731 to Shabtay, et al. (“’731 Patent”), APPL-
`
`1020;
`
`
`
`- 2 -
`
`APPL-1003 / Page 5 of 109
`
`

`

`
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`(17) U.S. Patent Application Publication No. 2014/0362274 to Christie, et
`
`al. (“Christie”), APPL-1021;
`
`(18) Warren J. Smith, MODERN LENS DESIGN (1992) (“Smith”), APPL-
`
`1023;
`
`(19) U.S. Patent No. 7,777,972 to Chen et al. (“Chen”), APPL-1024; and
`
`(20) Jacobs et al., “Focal Stack Compositing for Depth of Field Control,”
`
`Stanford Computer Graphics Laboratory Technical Report 2012-1 (“Jacobs”),
`
`APPL-1026.
`
`4.
`
`In forming the opinions expressed below, I have considered:
`
`(1) The documents listed above;
`
`(2) Any additional documents discussed below; and
`
`(3) My own knowledge and experience based upon my work in the fields
`
`of imaging systems as described below.
`
`II. QUALIFICATIONS
`
`5. My qualifications and professional experience are described in my
`
`Curriculum Vitae, a copy of which can be found in exhibit APPL-1004. The
`
`following is a brief summary of my relevant qualifications and professional
`
`experience.
`
`6.
`
`I earned my Bachelor’s degree in Math and Computer Science from
`
`École Normale Superieure of Paris, France in 1993, Master of Science degree in
`
`
`
`- 3 -
`
`APPL-1003 / Page 6 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`Computer Science from Grenoble Institute of Technology, Grenoble, France in
`
`
`
`1994, and Ph.D. degree in Computer Science from Joseph Fourier University,
`
`Grenoble, France in 1999. My doctoral thesis focused on 3D visibility and lighting
`
`simulation.
`
`7.
`
`For more than 25 years, I have been developing professional and
`
`academic experience in the field of imaging systems, including integration of
`
`optics, sensors, and digital processing in imaging systems. My research interests
`
`span most aspects of picture generation and creation, and one of the major themes
`
`of my research has been directed to computational photography, which combines
`
`expertise in optics design and image processing.
`
`8.
`
`I am a tenured full Professor in the Electrical Engineering and
`
`Computer Science Department of the Massachusetts Institute of Technology, and a
`
`member of the Computer Science and Artificial Intelligence Laboratory.
`
`9.
`
`As a professor, I teach in the area of computational photography. In
`
`the courses of Computational Photography, I teach principles of computational
`
`photography through a series of hands on projects, including applications of
`
`computational photography in high-dynamic range photography, photomontage,
`
`panoramas, image resampling, foreground extraction, Bayer sensor demosaicing,
`
`optical aberration correction, background defocusing, and morphing.
`
`10.
`
`I have authored and co-authored over two hundred journal
`
`
`
`- 4 -
`
`APPL-1003 / Page 7 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`publications, conference proceedings, technical papers, and technical presentations
`
`
`
`in the area of imaging system technologies, including optics design, image
`
`processing, and computational photography.
`
`11.
`
`In 2004, I received an inaugural Eurographics Young Researcher
`
`Award. I received a National Science Foundation (NSF) Faculty Early Career
`
`Development (CAREER) award in 2005. The NSF CAREER award is to support
`
`my research project “Transient Signal Processing for Realistic Imagery,” which is
`
`NSF’s most prestigious award in support of early-career faculty who has the
`
`potential to serve as academic role models in research and education and to lead
`
`advances in the mission of their department or organization. The goal of this
`
`project is to characterize light transport from a signal-processing perspective with
`
`applications to image synthesis and material-appearance acquisition. I received an
`
`inaugural Microsoft Research New Faculty Fellowship in 2005, a Sloan fellowship
`
`in 2006, a Spira award for distinguished teaching in 2007. I received Association
`
`for Computing Machinery's (ACM’s) Special Interest Group on Computer
`
`Graphics and Interactive Techniques (SIGGRAPH) Computer Graphics
`
`Achievement Award in 2016, which is given by the organization each year to
`
`recognize an individual for an outstanding achievement in computer graphics and
`
`interactive techniques. I became an ACM fellow in 2016, which is ACM’s most
`
`prestigious member grade that recognizes the top 1% of ACM members for their
`
`
`
`- 5 -
`
`APPL-1003 / Page 8 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`outstanding accomplishments in computing and information technology and/or
`
`
`
`outstanding service to ACM and the larger computing community.
`
`12. My involvement in the research community extends to several
`
`organizations, journals, and conferences. Over the years, I have organized and
`
`served in the Program Committee of a variety of conferences, including IEEE
`
`International Conference on Computational Photography, Symposium on
`
`Computational Photography and Video, ACM SIGGRAPH, Eurographics
`
`Symposium on Rendering (EGSR), Graphics Interface, Eurographics, Non-
`
`Photorealistic Animation and Rendering (NPAR), Symposium on Point-Based
`
`Rendering, ACM Transactions on Graphics, Foundations and Trends in Computer
`
`Graphics and Computer Vision. I was a Member of the advisory board of Image
`
`and Meaning 2, an interdisciplinary conference on scientific illustration and
`
`education.
`
`13. A list of my publications and patents is contained in my CV at exhibit
`
`APPL-1004.
`
`III. LEVEL OF ORDINARY SKILL IN THE ART
`
`14.
`
`I understand that the level of ordinary skill may be reflected by the
`
`prior art of record, and that a Person of Ordinary Skill in The Art (“POSITA”) to
`
`which the claimed subject matter pertains would have the capability of
`
`understanding the scientific and engineering principles applicable to the pertinent
`
`
`
`- 6 -
`
`APPL-1003 / Page 9 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`art. I understand that a POSITA has ordinary creativity, and is not an automaton.
`
`
`
`15.
`
`I understand that there are multiple factors relevant to determining the
`
`level of ordinary skill in the pertinent art, including (1) the levels of education and
`
`experience of persons working in the field at the time of the invention; (2) the
`
`sophistication of the technology; (3) the types of problems encountered in the field;
`
`and (4) the prior art solutions to those problems.
`
`16.
`
`I am familiar with the imaging system art pertinent to the ’898 Patent.
`
`I am also aware of the state of the art at the time the application resulting in the
`
`’898 Patent was filed. I have been informed by counsel that the earliest claimed
`
`priority date for the ’898 Patent is August 13, 2015, although any given claim of
`
`the ’898 Patent may or may not be entitled to the earliest claimed date.
`
`17. Based on the technologies disclosed in the ’898 Patent, I believe that a
`
`POSITA would include someone who had, as of the claimed priority date of the
`
`’898 Patent, a bachelor’s or the equivalent degree in electrical and/or computer
`
`engineering or a related field and 2-3 years of experience in imaging systems
`
`including optics and image processing. In addition, I recognize that someone with
`
`less formal education but more experience, or more formal education but less
`
`experience could have also met the relevant standard for a POSITA. I believe that
`
`I am at least a POSITA and, furthermore, I have supervised students and engineers
`
`who were also POSITAs. Accordingly, I believe that I am qualified to opine
`
`
`
`- 7 -
`
`APPL-1003 / Page 10 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`from the perspective of a POSITA regarding the ’898 Patent.
`
`
`
`18. For purposes of this Declaration, unless otherwise noted, my opinions
`
`and statements, such as those regarding the understanding of a POSITA (and
`
`specifically related to the references I consulted herein), reflect the knowledge that
`
`existed in the art before the earliest claimed priority date of the ’898 Patent.
`
`IV. RELEVANT LEGAL STANDARDS
`
`19.
`
`I have been asked to provide my opinions regarding whether claims 1,
`
`4, 8-12, 15, 19, and 20 (the “Challenged Claims”) of the ’898 Patent would have
`
`been obvious to a POSITA at the time of the alleged invention in light of the prior
`
`art.
`
`20.
`
`I am not an attorney. In preparing and expressing my opinions and
`
`considering the subject matter of the ’898 Patent, I am relying on certain legal
`
`principles explained to me by counsel.
`
`21.
`
`I understand that a claim is unpatentable if it is anticipated under 35
`
`U.S.C. § 102 or obvious under 35 U.S.C. § 103.
`
`A. Obviousness
`22.
`
`I have been informed and I understand that a claimed invention is
`
`unpatentable under 35 U.S.C. § 103(a) if the differences between the subject matter
`
`sought to be patented and the prior art are such that the subject matter as a whole
`
`would have been obvious to a POSITA at the time the invention was made. I
`
`
`
`- 8 -
`
`APPL-1003 / Page 11 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`understand that the appropriate analysis for determining obviousness of a claimed
`
`
`
`invention takes into account factual inquiries, including the level of ordinary skill
`
`in the art, the scope and content of the prior art, and the differences between the
`
`prior art and the claimed subject matter as a whole.
`
`23.
`
`I have been informed and I understand that the United States Supreme
`
`Court has recognized several rationales for combining references or modifying a
`
`reference to show obviousness of claimed subject matter. Some of these rationales
`
`include the following: (a) combining prior art elements according to known
`
`methods to yield predictable results; (b) simple substitution of one known element
`
`for another to obtain predictable results; (c) use of a known technique to improve a
`
`similar device (method, or product) in the same way; (d) applying a known
`
`technique to a known device (method, or product) ready for improvement to yield
`
`predictable results; (e) choosing from a finite number of identified, predictable
`
`solutions, with a reasonable expectation of success; and (f) some teaching,
`
`suggestion, or motivation in the prior art that would have led a POSITA to modify
`
`the prior art reference or to combine prior art reference teachings to arrive at the
`
`claimed invention. I have also been informed and I understand that a
`
`demonstration of obviousness does not require a physical combination or bodily
`
`incorporation, but rather may be found based on consideration of what the
`
`combined teachings would have suggested to a POSITA at the time of the alleged
`
`
`
`- 9 -
`
`APPL-1003 / Page 12 of 109
`
`

`

`
`invention.
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`V. THE ’898 PATENT
`
`A.
`Summary of the ’898 Patent
`24. The ’898 Patent is titled “Dual Aperture Zoom Camera with Video
`
`Support and Switching / Non-Switching Dynamic Control” and was issued on
`
`March 12, 2019. (APPL-1001), ’898 Patent, Title. The ’898 Patent is directed to a
`
`“dual-aperture zoom digital camera operable in both still and video modes.”
`
`(APPL-1001), ’898 Patent, Abstract. In its background, the ’898 Patent
`
`acknowledges that using digital zooming is an “alternative approach [to optical
`
`zooming] for approximating the zoom effect,” and use of “multi-aperture imaging
`
`systems to approximate the effect of a zoom lens are known.” (APPL-1001), ’898
`
`Patent, 1:46-53. For example, the ’898 Patent acknowledges that US Patent
`
`Application Publication No. 2008/0030592 to Border et al. (“Border”, APPL-1009)
`
`describe a digital camera including “two lenses having different focal lengths” with
`
`corresponding Wide and Tele sensors providing “Wide” and “Tele” images with
`
`different fields of view (FOVs) respectively. (APPL-1001), ‘898 Patent, 2:8-20.
`
`However, the ’898 Patent alleges that Border “requires, in video mode, very large
`
`processing resources in addition to high frame rate requirements and high power
`
`consumption (since both cameras are fully operational).” Id., 2:30-33. For further
`
`example, the ’898 Patent acknowledges that US Patent Application Publication No.
`
`
`
`- 10 -
`
`APPL-1003 / Page 13 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`2010/0277619 to Scarff (“Scarff”, APPL-1012) describes a camera with two
`
`
`
`lens/sensor combinations, where “the zoomed image is provided from the
`
`lens/sensor combination having a FOV that is next larger than the requested FOV”
`
`based on a zoom amount requested by a user, but alleges that Scarff “leads to
`
`parallax artifacts when moving to the Tele camera in video mode.” (APPL-1001),
`
`’898 Patent, 2:38-50.
`
`25. The ’898 Patent alleges that neither Border nor Scarff deal with
`
`registration errors and that none of the known art references “provide a thin (e.g.,
`
`fitting in a cell-phone) dual-aperture zoom digital camera with fixed focal length
`
`lenses, the camera configured to operate in both still mode and video mode to
`
`provide still and video images, wherein the camera configuration does not use any
`
`fusion to provide a continuous, smooth zoom in video mode.” (APPL-1001), ’898
`
`Patent, 2:58-59, 3:11-17.
`
`26. As an alleged solution to this problem, the ’898 Patent describes a
`
`dual-aperture digital camera including “a Wide camera and a Tele camera.”
`
`(APPL-1001), ’898 Patent, 3:28-31. “In video mode, optical zoom is achieved
`
`‘without fusion,’ by, in some embodiments, switching between the W and T
`
`images to shorten computational time requirements, thus enabling high video rate.”
`
`(APPL-1001), ’898 Patent, 3:38-41. The ’898 Patent describes video mode zoom
`
`operation from low zoom factor (ZF) to higher ZFs above a switch point (described
`
`
`
`- 11 -
`
`APPL-1003 / Page 14 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`variously as Zswitch or ZFT or up-transfer ZF), with “[processing] applied to
`
`
`
`eliminate the changes in the image during crossover from one camera to the other.”
`
`(APPL-1001), ’898 Patent, 7:57-8:31, 8:18-34.
`
`27.
`
`“[A]t low ZF up to slightly above ZFT (the zoom factor that enables
`
`switching between Wide and Tele outputs) the output image is the digitally
`
`zoomed, unchanged Wide camera output.” (APPL-1001), ’898 Patent, 8:18-21.
`
`“[F]or higher ZF than the up-transfer ZF, the output is the transformed Tele camera
`
`output, digitally zoomed. However, in other embodiments, for higher ZF than the
`
`up-transfer ZF, there will be no switching from the Wide to the Tele camera
`
`output, i.e. the output will be from the Wide camera, digitally zoomed.” (APPL-
`
`1001), ’898 Patent, 9:60-66. “Switching from the Wide camera output to the
`
`transformed Tele camera output will be performed unless some special condition
`
`(criterion), determined based on inputs obtained from the two camera images,
`
`occurs. In other words, switching will not be performed only if [a] no-switching
`
`criteria is fulfilled.” (APPL-1001), ’898 Patent, 10:1-7.
`
`28. FIG. 1A of the ’898 Patent illustrates a dual-aperture Zoom imaging
`
`system 100 including a Wide imaging section and a Tele imaging section, each
`
`having a respective lens with respect field of view (FOV) and respective image
`
`sensor to provide image data of an object or scene, and FIG. 2 of the ’898 Patent
`
`illustrates Wide and Tele sensors and their respective FOVs.
`
`
`
`- 12 -
`
`APPL-1003 / Page 15 of 109
`
`

`

`
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`
`
`(APPL-1001), ’898 Patent, FIGS. 1A and 2
`
`
`
`- 13 -
`
`APPL-1003 / Page 16 of 109
`
`

`

`
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`29. FIG. 2 illustrates by way of exemplary images, a larger FOV for the
`
`Wide image provided by the Wide sensor 202 and a smaller FOV for the
`
`corresponding Tele image provided by the Tele sensor.
`
`30.
`
`Independent claim 1 of the ’898 Patent is representative:
`
`1. A zoom digital camera comprising:
`a) a Wide imaging section that includes a fixed focal length Wide lens
`with a Wide field of view (FOV) and a Wide sensor, the Wide imaging
`section operative to provide Wide image data of an object or scene;
`b) a Tele imaging section that includes a fixed focal length Tele lens
`with a Tele FOV that is narrower than the Wide FOV and a Tele sensor, the
`Tele imaging section operative to provide Tele image data of the object or
`scene; and
`c) a camera controller operatively coupled to the Wide and Tele
`imaging sections and configured to evaluate if a no-switching criterion is
`fulfilled or not fulfilled, wherein if the no-switching criterion is fulfilled in a
`zoom-in operation between a lower zoom factor (ZF) value and a higher ZF
`value at a zoom factor (ZF) higher than an up-transfer ZF, the camera
`controller is further configured to output a zoom video output image that
`includes only Wide image data, and wherein if the no-switching criterion is
`not fulfilled, the camera controller is further configured to output a zoom
`video output image that includes only transformed, digitally zoomed Tele
`image data.
`
`
`(APPL-1001), ’898 Patent, 12:23-45.
`
`31. As I discuss below in more detail, the system and method presented in
`
`the ’898 Patent, namely, a multiple aperture zoom digital camera using 1) Wide
`
`and Tele imaging sections to provide images of an object or scene, and 2) a camera
`
`controller to evaluate if a no-switching criterion is fulfilled or not fulfilled, wherein
`
`if the no-switching criterion is fulfilled in a zoom-in operation at a zoom factor
`
`(ZF) higher than an up-transfer ZF, the camera controller outputs a zoom video
`- 14 -
`
`
`
`APPL-1003 / Page 17 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`output image that includes only Wide image data, and wherein if the no-switching
`
`
`
`criterion is not fulfilled, the camera controller outputs a zoom video output image
`
`that includes only transformed, digitally zoomed Tele image data was well known
`
`to persons of ordinary skill in the art before the earliest priority date of the ’898
`
`Patent.
`
`B.
`Prosecution History of the ’898 Patent
`32. U.S. Patent Application No. 15/324,720 (“’720 App”), which
`
`ultimately issued as the ’898 Patent, was filed on January 8, 2017, and contained
`
`21 claims. (APPL-1002), ’720 App, 37-40. The ’720 App was filed under 35
`
`U.S.C. § 371 from International Patent Application PCT/IB2016/053803, filed
`
`June 26, 2016, and claims priority from US Provisional Application No.
`
`62/204,667, filed August 13, 2015. Id., 21.
`
`33. On June 19, 2018, a non-final Office Action was issued. Claims 1, 2,
`
`8-14, 20, and 21 were rejected under 35 U.S.C. 102(a)(1) as being anticipated by
`
`WO 2014/199338 to Shabtay et al.1 Id., 198-205. Remaining claims were
`
`objected to as dependent on a rejected base claim, but allowable if rewritten.
`
`According to the Examiner, Shabtay and the art then of record did not provide
`
`
`
`1 The international application that published as WO 2014/199338 eventually
`
`issued as US Patent 9,185,291 to Shabtay, Goldenberg, Gigushinski and Cohen.
`
`
`
`- 15 -
`
`APPL-1003 / Page 18 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`disclosure of the no-switching criterion setting defined in dependent claims 3-7 and
`
`
`
`15-19.
`
`34. On September 23, 2018, Applicant filed an Information Disclosure
`
`Statement disclosing 103 US Patent references (including Golan), 16 foreign patent
`
`references, and 16 non-patent literature documents. Id., 151-175.
`
`35. On October 10, 2018, Applicant filed an Response amending the
`
`specification to recite “in other embodiments, for higher ZF than the up-transfer
`
`ZF, there will be no switching from the Wide to the Tele camera output” and
`
`amending independent claims 1 and 13 to recite “if the no-switching criterion is
`
`fulfilled, configuring the camera controller to output at a zoom factor (ZF) higher
`
`than an up-transfer ZF a zoom video output image that includes only Wide image
`
`data” and “if the no-switching criterion is not fulfilled, configuring the camera
`
`controller to output a zoom video output image that includes only transformed,
`
`digitally zoomed Tele image data.” Id., 141-145. Claim 11 was canceled. Id.,
`
`293. Applicant’s argument against the Shabtay anticipation rejection was based
`
`on the amendment and, more specifically, that Shabtay did not itself teach “a
`
`camera controller configured to output, at a ZF higher than an up-transfer ZF, a
`
`zoom video output image that includes only Wide image data in a zoom-in
`
`operation between a lower ZF value and a higher ZF value,” but instead taught
`
`“that any ZF higher than the up-transfer ZF a camera outputs a Tele image.” Id.,
`
`
`
`- 16 -
`
`APPL-1003 / Page 19 of 109
`
`

`

`
`147-148.
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`37. On November 16, 2018, a Notice of Allowance issued. Id., 301.
`
`Shortly after allowance, on November 25, 2018, Applicant filed an additional
`
`Information Disclosure Statement disclosing 16 US Patent references and 3
`
`additional foreign patent references. Id., 38-42. Thereafter, on December 10,
`
`2018, the Office issued a notice of Non-Compliant IDS, indicating that the
`
`November 25, 2018 IDS would be placed in the file, but had not been considered.
`
`Id., 37.
`
`38. The ’898 Patent issued on March 12, 2019. Id., 435. The allowed
`
`claims 1-10 and 12-21 of the ’720 Application were issued as claims 1-20 of the
`
`’898 Patent, respectively.
`
`VI. CLAIM CONSTRUCTION
`
`39.
`
`I have reviewed the claim language, the specification, and the
`
`prosecution history. For the purposes of my analysis below, I do not believe any of
`
`the claim terms require a specific construction beyond the plain and ordinary
`
`meaning as would be understood by one of ordinary skill in the art.
`
`VII. GROUNDS
`
`A. Ground 1: Claims 1, 4, 8, 12, and 15 are unpatentable under
`35 U.S.C. § 103 over Golan in view of Martin and Togo.
`
`1.
`
`Summary of Golan
`
`40. U.S. Patent Application Publication No. 2012/0026366 to Golan et al.
`
`
`
`- 17 -
`
`APPL-1003 / Page 20 of 109
`
`

`

`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`(“Golan”) was published on February 2, 2012. Golan is titled “Continuous Electronic
`
`
`
`Zoom for an Imaging System with Multiple Imaging Devices Having Different Fixed
`
`FOV,” and discloses providing a video output with “a continuous electronic zoom for
`
`an image acquisition system, the system including multiple imaging devices having
`
`different fixed FOV.” (APPL-1005), Golan, FIG. 1, Title, [0002].
`
`41. Golan teaches use of wide and tele lenses and employs wide and tele
`
`images during digital zooming, which “facilitates a light weight electronic zoom with
`
`a large lossless zooming range.” (APPL-1005), Golan, [0009]. Specifically, as
`
`illustrated in FIG. 1 below, Golan discloses zoom control sub-system 100 for an
`
`image acquisition system including “multiple image sensors, each with a fixed and
`
`preferably different FOV.” (APPL-1005), Golan, [0036]. Golan’s zoom control sub-
`
`system 100 includes a tele image sensor 110 coupled with a narrow lens 120 having a
`
`tele FOV 140, a wide image sensor 112 coupled with a wide lens 122 having a wide
`
`FOV 142, a zoom control module 130 and an image sensor selector 150. (APPL-
`
`1005), Golan, FIG. 1, [0037].
`
`
`
`- 18 -
`
`APPL-1003 / Page 21 of 109
`
`

`

`
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`
`(APPL-1005), Golan, FIG. 1
`
`
`
`42. Golan teaches that, in embodiments of FIGS. 1 and 2, each image frame
`
`of video output is generated based on an acquired image frame from “the relevant
`
`image sensor” of an image acquisition device selected based on the user input zoom
`
`factor. (APPL-1005), Golan, FIGS. 1-2, [0039]. Specifically, Golan teaches that
`
`zoom control circuit 130 receives a required zoom from an operator of the image
`
`acquisition system, and selects the relevant image sensor (110 and 112) by activating
`
`image sensor selector 150 position. (APPL-1005), Golan, [0039].
`
`
`
`- 19 -
`
`APPL-1003 / Page 22 of 109
`
`

`

`
`
`
`
`Declaration of Frédo Durand, Ph.D.
`Inter Partes Review of U.S. Patent 10,230,898
`43. Golan teaches that “an electronically calibrating is performed to
`
`determine the alignment offsets between wide image sensor array 110 and tele image
`
`sensor array 112,” ((APPL-1005), Golan, [0038]) and that the “calibration of the
`
`alignment, between the first image sensor array and the second image sensor array,
`
`facilitates continuous electronic zoom with uninterrupted imaging, when
`
`switching back and forth between the first image sensor array and the second image
`
`sensor array.” (APPL-1005), Golan, [0015]. The electronic calibration is performed
`
`preferably with sub-pixel accuracy. Id.
`
`2.
`
`Summary of Martin
`
`44. U.S. Patent No. 8,081,206 to Martin et