UNITED STATES PATENT AND TRADEMARK OFFICE
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`APPLE, INC.
`Petitioner
`
`v.
`
`IMMERVISION, INC.
`(“ImmerVision”),
`Patent Owner
`
`Case IPR2023-00471
`Patent No. 6,844,990
`
`DECLARATION OF DR. DAVID KESSLER, Ph.D.
`
`1
`
`APPLE 1003
`
`

`

`TABLE OF CONTENTS
`
`Introduction ....................................................................................................... 5
`
`I.
`
`II. Background and Qualifications ........................................................................ 6
`
`III. Level of Ordinary Skill in the Art ................................................................... 10
`
`IV. Materials Considered and Relied Upon .......................................................... 11
`
`V. Legal Standards ............................................................................................... 13
`
`A.
`
`Legal Standards for Prior Art .............................................................. 13
`
`B.
`
`C.
`
`Legal Standard for Priority Date ......................................................... 15
`
`Legal Standard for Obviousness ......................................................... 15
`
`VI. Overview of the Relevant Technology ........................................................... 18
`
`A.
`
`Rectilinear Imaging ............................................................................. 18
`
`B.
`
`C.
`
`Panoramic/Fisheye Lenses .................................................................. 21
`
`Optical Designers Understood How to Implement a Particular
`
`Image Distribution Function in their Optical Designs ........................ 26
`
`D. Using Aspherics to Control Distortion ................................................ 28
`
`E.
`
`F.
`
`Post-Capture Image Processing ........................................................... 29
`
`Panoramic Imaging and Digital Image Processing ............................. 29
`
`VII. The ’990 Patent ............................................................................................... 31
`
`A. Overview ............................................................................................. 31
`
`B.
`
`C.
`
`File History of the ’990 Patent (APPLE-1002) ................................... 41
`
`Reexamination Summary (APPLE-1011) ........................................... 41
`
`D.
`
`Interpretation of the ’990 Patent Claims at Issue ................................ 43
`
`VIII. Overview of the Cited References .................................................................. 46
`
`
`
`2
`
`

`

`A.
`
`Baker (APPLE-1006) .......................................................................... 46
`
`B.
`
`C.
`
`Shiota (APPLE-1012) ......................................................................... 51
`
`Fisher (APPLE-1009) .......................................................................... 52
`
`IX. Overview of Unpatentability Grounds ............................................................ 55
`
`X. Ground 1: Baker in View of Shiota Renders OBvious CLaims 27, 2, 4,
`
`and 29 ............................................................................................................. 56
`
`A.
`
`Combination of Baker and Shiota ....................................................... 56
`
`B.
`
`Claim 27 .............................................................................................. 60
`
`[27pre] A method for displaying a digital panoramic image, the
`method comprising:................................................................... 61
`
`[27a]: obtaining a digital panoramic image by projecting a
`panorama onto an image sensor using a panoramic
`objective lens, ............................................................................ 67
`
`[27b]: the panoramic objective lens having an image point
`distribution function that is not linear relative to a field
`angle of object points of the panorama, .................................... 71
`
`[27c] the distribution function having a maximum divergence
`of at least +/-10% compared to a linear distribution
`function, .................................................................................... 75
`
`[27d]: such that the panoramic image obtained has at least one
`substantially expanded zone and at least one substantially
`compressed zone, and ............................................................... 78
`
`[27e]: displaying the obtained panoramic image by correcting
`the non-linearity of the initial image, ........................................ 81
`
`[27f]: performed by retrieving image points on the obtained
`image in a coordinate system of center O’ using at least
`the non-linear distribution function and a size L of the
`obtained image. ......................................................................... 84
`
`C.
`
`Claim 2 ................................................................................................ 90
`
`[2pre] The method according to claim 27: .......................................... 91
`
`
`
`3
`
`

`

`[2a] wherein the objective lens has a non-linear distribution
`function that is symmetrical relative to the optical axis of
`the objective lens, the position of an image point relative
`to the center of the image varying according to the field
`angle of the corresponding object point; ................................... 91
`
`D.
`
`Claim 4 ................................................................................................ 94
`
`[4pre]: The method according to claim 27: ......................................... 94
`
`[4a]: wherein the objective lens expands the edges of the image
`and compresses the center of the image. ................................... 95
`
`E.
`
`Claim 29 .............................................................................................. 96
`
`[29pre]: The method according to claim 27: ....................................... 97
`
`[29a] wherein the objective lens comprises a set of lenses
`forming an apodizer. ................................................................. 97
`
`XI. Ground 2: Baker in view of Shiota and Fisher Render Obvious Claims
`
`29 and 30 ........................................................................................................ 99
`
`A.
`
`Combination of Baker, Shiota, and Fisher .......................................... 99
`
`B.
`
`Claims 29 and 30 ...............................................................................104
`
`[29pre]/[30pre]: The method according to claim 27, .......................104
`
`[29a]: wherein the objective lens comprises a set of lenses
`forming an apodizer / [30a] wherein the set of lenses
`forming an apodizer comprises at least one aspherical
`lens. .........................................................................................105
`
`XII. Additional Remarks ......................................................................................106
`
`
`
`
`
`
`
`
`
`4
`
`

`

`I.
`
`INTRODUCTION
`
` My name is Dr. David Kessler, and I have been retained by counsel for
`
`Petitioner Apple, Inc. (“Apple” or “Petitioner”) as an expert witness to provide
`
`assistance regarding U.S. Patent No. 6,844,990 (the “’990 Patent”). Specifically, I
`
`have been asked to consider the validity of claims 27, 2, 4, 29, and 30 of the ’990
`
`Patent (“Challenged Claims”) in view of prior art, obviousness considerations, and
`
`understanding of a person of ordinary skill in the art at the time of the invention
`
`(“POSITA”) as it relates to the ’990 Patent
`
`
`
`I am being compensated for my time at my standard consulting rate. I
`
`am also being reimbursed for expenses that I incur during the course of this work.
`
`My compensation is not contingent upon the results of my study, the substance of
`
`my opinions, or the outcome of any proceeding involving the challenged claims. I
`
`have no financial interest in the outcome of this matter or on the pending litigation
`
`between Petitioner and Patent Owner.
`
` My analysis here is based on my years of education, research and
`
`experience, as well as my investigation and study of relevant materials, including
`
`those cited herein.
`
`
`
`I may rely upon these materials, my knowledge and experience, and/or
`
`additional materials to rebut arguments raised by the Patent Owner. Further, I may
`
`
`
`5
`
`

`

`also consider additional documents and information in forming any necessary
`
`opinions, including documents that may not yet have been provided to me.
`
` My analysis of the materials produced in this proceeding is ongoing and
`
`I will continue to review any new material as it is provided. This declaration
`
`represents only those opinions I have formed to date. I reserve the right to revise,
`
`supplement, and/or amend my opinions stated herein based on new information and
`
`on my continuing analysis of the materials already provided.
`
`
`
`II. BACKGROUND AND QUALIFICATIONS
`
` My education and experience are described more fully in the attached
`
`curriculum vitae (APPLE-1004). For ease of reference, I have highlighted certain
`
`information below.
`
`
`
`I have over forty years of academic and industry experience in optics.
`
`I am proficient in several topics/fields, including, e.g., aberrations theory, physical
`
`optics, optical systems architecture and design, and major optical CADs.
`
`
`
`Since 2006, I have been the owner of Kessler Optics & Photonics
`
`Solutions Ltd., which provides consulting services relating to optical designs and
`
`optical systems architecture primarily in imaging systems; projection systems;
`
`display technologies; AR/VR systems; medical imaging systems like digital retina
`
`
`
`6
`
`

`

`and cornea microscopes; intra oral cameras based on mobile phone cameras; military
`
`digital simulators and more.
`
`
`
`For most of the period between 1982 and 2006, I managed an advanced
`
`optical design team at Kodak Research Labs. During my time at Kodak Research
`
`Labs, my projects included, among others, designing and developing digital cameras,
`
`designing of anti-aliasing filters for cameras, designing medical digital imaging
`
`systems like digital fundus CCD cameras, CCD scanners, laser systems including
`
`some using f-theta lenses, designing scan/process/print systems (where films were
`
`scanned, digitally processed to correct color, grain, exposure and artifacts, and then
`
`printed to paper at high speeds).; The first digital cinema projectors; Extended depth
`
`of focus (EDOF) cameras (where phase filters are introduced into the camera lens ,
`
`images captured and algorithms used to resurrect the image with a resulting extended
`
`focus range;
`
`
`
`I am a recipient of a Tech Emmy Award for the high-speed CCD
`
`scanner for real time telecine (system for conversion of a movie from film to HD
`
`video).
`
` My full portfolio and my current projects are on my web site at
`
`www.KesslerOptics.com.
`
`
`
`7
`
`

`

` While at Kodak, I taught advanced optical design over two academic
`
`years as a visiting professor at Tel-Aviv University. More recently, I have conducted
`
`a number of advanced optical designs courses for corporate optical design teams.
`
`
`
`I have a Bachelor of Science (BSc.) in physics and mathematics from
`
`The Hebrew University of Jerusalem. I have a Master of Science degree (MSc.).in
`
`physics and electro-optics from the School of Applied Sciences and Technology at
`
`The Hebrew University of Jerusalem. I have a Ph.D. in optical sciences from the
`
`Optical Sciences Center at the University of Arizona. My dissertation title is “Image
`
`quality criteria in the presence of moderately large aberrations” (which are the
`
`criteria to be chosen for usage in the merit function while optimizing lenses and other
`
`optical systems).
`
` Additionally, I have familiarity with image processing algorithms. For
`
`example, as part of my Ph.D. program at the Optical Sciences Center at the
`
`University of Arizona, I studied under Prof. J. Gaskill who is the author of “Linear
`
`Systems Fourier Transformations in Optics.” Indeed, my dissertation deals with
`
`merit functions used in optical CAD programs that are associated with processing
`
`algorithms for image improvement.
`
` Further still, a number of projects I led and/or participated in required
`
`post-capture image processing, some of which are referenced below:
`
`
`
`8
`
`

`

` Central photofinishing labs at Kodak where film was scanned and the
`
`scanned image was then processed using image processing algorithms to correct for
`
`defects in, e.g., correct exposure, low contrast, color gamut, grain, and resolution
`
`and printed at rates of 10,000 an hour;
`
`
`
`I also analyzed and worked on Extended Depth of Focus (EDOF)
`
`cameras. These cameras, such as by CDM, DBLUR, XCEED and others, use phase
`
`elements in the camera to intentionally aberrate the captured images, which are then
`
`processed using image processing algorithms to retrieve the image at an increased
`
`depth of focus.
`
`
`
`In lenticular imaging (also referred to as dynamic imaging) where about
`
`30 images are combined as thin stripes at a focal plane of the lenticular lenses, we
`
`used image processing algorithms for the assembly of the images to provide a motion
`
`sensation;
`
`
`
`In the laser printer space—which was one of my major activities in the
`
`first 14 years at Kodak—image interpolation algorithms were routinely used to
`
`facilitate correction for distortion.
`
` Additionally, in a paper I published with colleagues, which is titled
`
`“Continuous tone printing using LCOS” (Digest of the Technical Papers, SID
`
`International Symposium, Vol. XXXVII, Page 174-177, June 2006), we described
`
`dithering and interpolation algorithms to significantly increase the resolution.
`
`
`
`9
`
`

`

`Relatedly,
`
`there
`
`are
`
`additional
`
`four
`
`papers
`
`on my
`
`site
`
`at
`
`http://www.kessleroptics.com/publications, dealing with multi spot laser printers or
`
`multichannel printers where interleaving was used and image processing was center
`
`to the ability to produce clean contiguous prints. In another paper on my website
`
`under “presentations,” and which is titled ”Image Quality Issues: the Optical
`
`Designer Point of View,” the issue of artifacts is covered and I discuss there the
`
`mitigation of such artifacts by either optical means or by image processing.
`
`
`
`I also have designed and analyzed a number of Light Field (LF)
`
`cameras and microscopes for the two modalities of LF systems, namely temporal
`
`and spatial. Notably, LF systems rely heavily on image processing algorithms to
`
`capture images over a large depth of field.
`
`
`
`I am a named inventor on 104 U.S. patents almost all related to optical
`
`systems and imaging systems and have been a listed author on 25 papers.
`
`
`
`I am a fellow of the SPIE, the International Society of Optical
`
`Engineering, and I am a member of organizations such as OPTICA and SID.
`
`
`III. LEVEL OF ORDINARY SKILL IN THE ART
`
`
`
`In my opinion, a person of ordinary skill in the art (“POSITA”) relating
`
`to, and at the time of, the ’990 Patent’s Critical Date would have had at least a
`
`bachelor’s degree in Physics, Optical Engineering, and/or Electrical Engineering and
`
`at least five years of experience in developing and designing optical imaging systems
`
`
`
`10
`
`

`

`and have familiarity with image processing algorithms and optical design software.
`
`Additional education in a relevant field, or relevant industry experience may
`
`compensate for a deficit in one of the other aspects of the requirements stated above.
`
` Based on my knowledge, skill, and experience, I have a good
`
`understanding of the capabilities of the POSITA. Indeed, I have taught, participated
`
`in organizations, recruited, and worked closely with many such persons over the
`
`course of my career. For example, from my industry experience, I am familiar with
`
`what an engineer would have known and found predictable in the art. From teaching
`
`and supervising my post-graduate students, I also have an understanding of the
`
`knowledge that a person with this academic experience possesses. Furthermore, I
`
`possess those capabilities myself.
`
`
`
`IV. MATERIALS CONSIDERED AND RELIED UPON
`
`
`
`In reaching the conclusions described in this declaration, I have relied
`
`on the documents and materials referenced below as well as those identified in this
`
`declaration, including the ’990 Patent, the prosecution history of the ’990 Patent, and
`
`prior art references cited herein. These materials include patents, related documents,
`
`and printed publications. Each of these materials is a type of document that experts
`
`in my field would have reasonably relied upon when forming their opinions.
`
`
`
`11
`
`

`

`
`
`I have also relied on my education, training, research, knowledge, and
`
`personal and professional experience in the relevant technologies and systems that
`
`were already in use prior to, and within the timeframe of the earliest priority date of
`
`the claimed subject matter in the ’990 Patent, which I have been informed by counsel
`
`is May 11, 2001 (“Critical Date”).
`
` APPLE-1001: U.S. Patent No. 6,844,990 (“the ’990 patent”)
`
` APPLE-1002: Prosecution History of
`
`the ’990 Patent (“Original
`
`Prosecution History”)
`
` APPLE-1006: U.S. Patent No. 5,686,957 to Baker (“Baker”)
`
` APPLE-1007: “Fish Eye Lens” by K Miyamoto (Feb. 19, 1964)
`
`(“Miyamoto”)
`
` APPLE-1009: U.S. Patent No. 3,953,111 to Fisher et al. (“Fisher”)
`
` APPLE-1010: U.S. Patent No. 6,128,145 to Nagaoka (“Nagaoka”)
`
` APPLE-1011: Reexamination File History of the ’990 Patent (“Reexam
`
`History”)
`
` APPLE-1012: EP 1028389 to Shiota et al. (“Shiota”)
`
` APPLE-1013: Rebiai et al., “Image Distortion from Zoom Lenses:
`
`Modeling and Digital Correction,” 1992 IBC International Broadcasting
`
`Convention (July 1992) (“Rebiai”)
`
` APPLE-1014: JP 2000-242773 to Matsui et al. (“Matsui”)
`
` APPLE-1015: U.S. Patent No. 5,861,999 to Tada (“Tada”)
`
` APPLE-1016: Zeljko Andreic, “A simple 180o field-of-view F-theta all-
`
`sky camera,” SPIE Proc. 1500 (Oct. 1, 1991) (“Andreic”)
`
`
`
`12
`
`

`

` APPLE-1017: Abed Kassim et al., “Optical Performance of axial gradient
`
`and aspheric surface lenses: study and analysis,” SPIE’s 1994 International
`
`Symposium on Optics, Imaging, and Instrumentation, Vol. 2263 (Sept. 30,
`
`1994) (“Kassim”)
`
` APPLE-1018: Susan Houde-Walter, “Recent Progress In Gradient-Index
`
`Optics,” SPIE Proc. 0935 (Apr. 8, 1988) (“Houde-Walter”)
`
` APPLE-1020: Paul K. Manhart et al., “Fundamentals of macro axial
`
`gradient index optical design and engineering,” SPIE Opt. Eng. 36(6),
`
`1607-1621 (June 1997) (“Manhart”)
`
`
`
`V. LEGAL STANDARDS
`
`A. Legal Standards for Prior Art
`
`
`
`I understand that a patent or other publication must first qualify as prior
`
`art before it can be used to invalidate a patent claim.
`
`
`
`I understand that a U.S. or foreign patent qualifies as prior art to a patent
`
`if the date of issuance of the U.S. or foreign patent is prior to the invention of the
`
`patent. I further understand that a printed publication, such as an article published
`
`in a magazine or trade publication, qualifies as prior art to a patent if the date of
`
`publication is prior to the invention of the patent. My understanding is that, for such
`
`prior art references, a patentee may attempt to show that the claimed invention was
`
`conceived prior to the issuance of the U.S. foreign patent or publication of the printed
`
`materials. To do so, it is my understanding that patentee must prove with
`
`corroborating evidence that the named inventors conceived of the complete claimed
`
`
`
`13
`
`

`

`invention before the prior art and were diligent in reducing the claimed inventions
`
`to practice.
`
`
`
`I understand that, regardless of the date of invention of the patent, a U.S.
`
`or foreign patent qualifies as prior art to a patent if the date of issuance of the U.S.
`
`or foreign patent is more than one year before the earliest effective filing date of the
`
`patent. I further understand that a printed publication, such as an article published
`
`in a magazine or trade publication, constitutes prior art to a patent if the publication
`
`occurs more than one year before the earliest effective filing date of the patent, again
`
`regardless of the date of invention of the patent.
`
`
`
`I understand that a U.S. patent or published U.S. application qualifies
`
`as prior art to a patent if the application for that patent was filed in the United States
`
`before the invention of the patent. My understanding is that, for such prior art
`
`references, a patentee may attempt to show that the claimed invention was conceived
`
`prior to the filing in the United States of the purported prior art U.S. patent or
`
`application. To do so, it is my understanding that patentee must prove with
`
`corroborating evidence that the named inventors conceived of the complete claimed
`
`invention before the prior art and were diligent in reducing the claimed inventions
`
`to practice.
`
`
`
`14
`
`

`

`
`
`I understand that to qualify as prior art, a reference must contain an
`
`enabling disclosure that allows one of ordinary skill to practice the claims without
`
`undue experimentation.
`
`
`
`I understand that documents and materials that qualify as prior art can
`
`be used to invalidate a patent claim as anticipated or as obvious.
`
`B.
`
`Legal Standard for Priority Date
`
`
`
`I understand that the “priority date” or “earliest effective filing date” of
`
`a patent is the date on which it is filed, or the date on which an earlier-filed U.S. or
`
`international patent application was filed if the patentee claims the benefit of priority
`
`to that earlier-filed U.S. or international patent application. I further understand that
`
`although a foreign priority document may be used to try to overcome certain prior
`
`art references, the effective filing date is not the filing date of a foreign priority
`
`document.
`
`C. Legal Standard for Obviousness
`
` My understanding is that a patent claim is invalid as obvious only if the
`
`subject matter of the claimed invention “as a whole” would have been obvious to
`
`one of ordinary skill at the time the invention was made. To determine the
`
`differences between a prior art reference (or a proposed combination of prior art
`
`references) and the claims, the question of obviousness is not whether the differences
`
`themselves would have been obvious, but whether the claimed invention as a whole
`
`
`
`15
`
`

`

`would have been obvious. Also, obviousness grounds cannot be sustained by mere
`
`conclusory statements. Rather, it is necessary to provide some articulated reasoning
`
`with rational underpinning to support the legal conclusion of obviousness.
`
`
`
`I understand that a patent claim that comprises several elements is not
`
`proved obvious by simply showing that each of its elements was independently
`
`known in the prior art. In my evaluation of whether any claim of the ’990 Patent
`
`would have been obvious, I considered whether the Petition, or any evidence
`
`submitted in this proceeding, presented an articulated reason with a rational basis
`
`that would have motivated one of ordinary skill to combine the elements or concepts
`
`from the prior art in the same way as in the claimed invention.
`
`
`
`It is my understanding that there is no single way to define the line
`
`between true inventiveness on one hand—which is patentable—and the application
`
`of common sense and ordinary skill to solve a problem on the other hand—which is
`
`not patentable. For instance, factors such as market forces or other design incentives
`
`may be the source of what produced a change, rather than true inventiveness.
`
`
`
`I understand that the decision-maker may consider whether the change
`
`was merely the predictable result of using prior art elements according to their
`
`known functions, or whether it was the result of true inventiveness. And, the
`
`decision-maker may also consider whether there is some teaching or suggestion in
`
`the prior art to make the modification or combination of elements recited in the claim
`
`
`
`16
`
`

`

`at issue. Also, the decision-maker may consider whether the innovation applies a
`
`known technique that had been used to improve a similar device or method in a
`
`similar way. The decision-maker may also consider whether the claimed invention
`
`would have been obvious to try, meaning that the claimed innovation was one of a
`
`relatively small number of possible approaches to the problem with a reasonable
`
`expectation of success by those skilled in the art.
`
`
`
`I have been instructed by counsel that if any of these considerations are
`
`relied upon to reach a conclusion of obviousness, the law requires that the analysis
`
`of such a consideration must be made explicit. I understand that the decision-maker
`
`must be careful not to determine obviousness using the benefit of hindsight and that
`
`many true inventions might seem obvious after the fact. I understand that the
`
`decision-maker should consider obviousness from the position of one of ordinary
`
`skill at the time the claimed invention was made, and that the decision-maker should
`
`not consider what is known today or what is learned from the teaching of the patent.
`
`
`
`I understand that in order to determine whether a patent claim is
`
`obvious, one must make certain factual findings regarding the claimed invention and
`
`the prior art. Specifically, I understand that the following factors must be evaluated
`
`to determine whether a claim is obvious: the scope and content of the prior art; the
`
`difference or differences, if any, between the claim of the patent and the prior art;
`
`the level of ordinary skill in the art at the time the claimed invention was made; and,
`
`
`
`17
`
`

`

`if available, the objective indicia of non-obviousness, also known as “secondary
`
`considerations.”
`
`
`
`I understand that the secondary considerations include: commercial
`
`success of a product due to the merits of the claimed invention; a long felt need for
`
`the solution provided by the claimed invention; unsuccessful attempts by others to
`
`find the solution provided by the claimed invention; copying of the claimed
`
`invention by others; unexpected and superior results from the claimed invention;
`
`acceptance by others of the claimed invention as shown by praise from others in the
`
`field or from the licensing of the claimed invention; teaching away from the
`
`conventional wisdom in the art at the time of the invention; independent invention
`
`of the claimed invention by others before or at about the same time as the named
`
`inventor thought of it; and other evidence tending to show obviousness.
`
`
`
`I understand that, to establish a secondary consideration, the evidence
`
`must demonstrate a nexus between that secondary consideration and the claimed
`
`invention.
`
`VI. OVERVIEW OF THE RELEVANT TECHNOLOGY
`
`A. Rectilinear Imaging
`
` Since near the beginning of photography (around the year 1840), lenses
`
`were designed and used to provide images. Lenses used in photography have to
`
`focus a beam of light coming from an object placed at a distance away from the
`
`
`
`18
`
`

`

`smallest spot/point at the image plane, where an image capture device is positioned
`
`(e.g., photographic film or an image sensor such as a charge-coupled device (CCD))).
`
`
`
`In addition to getting the small focused spot/point at the image plane,
`
`optical designers of imaging lenses generally had to consider the location of this spot
`
`at the sensor. Their main concern was and still is to control distortion in the lenses.
`
` Distortion is the deviation of the spot at the image plane from the
`
`position it should be at.
`
` However, when distortion is present, it can be readily seen, as shown in
`
`
`
`the images below, which show a comparison of an image with no distortion relative
`
`to images with distortion, namely barrel and pincushion distortion.
`
`
`
`19
`
`
`
`

`

`
`
`APPLE-1013, 1 (showing no distortion in first image from left; showing barrel and
`
`pincushion distortion in second and third images from left).
`
`
`
`In other words, before 2001, optical lens designers were striving and
`
`largely still are, to provide distortion free rectilinear imaging. In a rectilinear
`
`imaging lens with no distortion present, the ideal position ri of an image point as a
`
`function of the field angle  is given by the following function:
`
`ri () = f * tan()
`
`
`
` (1)
`
`where f is the focal length of the lens and  is the field angle in degrees. This
`
`function is the necessary and sufficient condition for distortion free rectilinear
`
`imaging.
`
`
`
`In reality though, lenses have some distortion and the actual position of
`
`an image point as a function of the field angle is given by r(). The default definition
`
`of distortion Dist() in optics is the difference of the actual position r() of the image
`
`at the image sensor relative to the ideal position ri(), which can be represented using
`
`the following equation:
`
`
`
`20
`
`

`

`𝐷𝑖𝑠𝑡(∅) = 100 ∗
`
`𝐫(∅) − 𝐫𝒊(∅)
` (2)
`𝐫𝒊(∅)
`
` Until the inception of digital cameras in the mid-1980s, cameras
`
`generally used film to capture images. Lenses in such film cameras had to provide
`
`rectilinear imaging since there was no easy way to correct pincushion or barrel
`
`distortion other than with the lenses. Thus, almost all lenses at the time were using
`
`the f*tan() function as the design reference for the image distribution curve and
`
`designers generally strived to achieve a certain limited amount of distortion in the
`
`captured images.
`
`B.
`
`Panoramic/Fisheye Lenses
`
` Panoramic imaging began shortly after the invention of photography in
`
`1839. The desire to show overviews of cities and landscapes prompted
`
`photographers to create panoramas. Cameras capable of capturing panoramic scenes
`
`were either swing-lens cameras, where the lens rotated while the film remained
`
`stationary, or 360-degree rotation cameras, where both the camera and the film
`
`rotated.
`
` Another way to create a panoramic image in “one shot” without
`
`swinging the camera was by using a fisheye lens. A fisheye lens covers a
`
`hemispherical field of view and when pointed up, the rays shown in red below can
`
`in a single shot capture a panoramic image onto a ring on the image plane. See also
`
`APPLE-1007.
`
`
`
`21
`
`

`

`
`
`
`
` Fisheye lenses date back to the inventor Robert W. Woods who
`
`developed a lens based on how a fish would view the world beneath the water.
`
`However, fisheye lenses did not rise to prominence until the 1960s when the fisheye
`
`lens was finally mass-produced.
`
` Fisheye lenses cannot be rectilinear. They have large distortion against
`
`the f*tan() curve needed for rectilinear imaging. This distortion effectively stems
`
`from flattening the hemisphere. See APPLE-1007, 1. Fisheye lenses thus use a
`
`different reference function, which is represented using the following function:
`
`
`
`22
`
`

`

`ri () = f *  
`
`
`
`
`
`
`
`
`
`See id. This function represents “linear” imaging in this domain and is referred to
`
`as the “f-theta” condition. Thus, distortion in fisheyes can be represented using the
`
`same equation as shown in equation (1) but with a different reference function,
`
`namely the f-theta reference shown above in equation (3).
`
` Fisheye lenses are also panoramic lenses because they capture the 360°
`
`view when pointing up. Thus, a POSITA would have understood that panoramic
`
`lenses include hemispherical and fisheye lenses.
`
` Most panoramic lenses use “f-theta” mapping as their design
`
`reference and the distortion they aim to control is relative to this linear mapping
`
`function. However, there were, and are, applications where lenses with different
`
`“image distribution curves” have been implemented.
`
` Although the term “image distribution function” is not commonly used
`
`in the field, a POSITA would have understood it to be the same as or similar to a
`
`lens’s mapping or projection. In 1964, Miyamoto identified different image
`
`projections in panoramic lenses. APPLE-1007, 1-2. Miyamoto identified four
`
`different image distribution functions or projections, as shown below:
`
`
`
`23
`
`

`

` Miyamoto provided the following graphical representation of the image
`
`
`
`distribution curves for these four projections:
`
`Fig. 1: Curves of different projections. (1) Ordinary projection,
`
`92) stereographic projection, (3) equidistance projection, (4)
`
`equisolid angle projection
`
`
`
`
`
` Miyamoto notes that curve 1 is for “camera lenses” and refers to this as
`
`an “ordinary projection.” This image distribution is typically used for rectilinear
`
`imaging, as I have discussed above. Miyamoto refers to the second curve (curve (2))
`
`
`
`24