USOO7206136B2
`
`(12) United States Patent
`Labaziewicz et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,206,136 B2
`Apr. 17, 2007
`
`(54) DIGITAL CAMERA USING MULTIPLE
`LENSES AND IMAGE SENSORS TO
`PROVIDE AN EXTENDED ZOOM RANGE
`
`(75) Inventors: Peter Labaziewicz, Tokyo (JP);
`Wilbert F. Janson, Jr., Shortsville, NY
`(US); Kenneth A. Parulski, Rochester,
`NY (US)
`
`(73) Assignee: Eastman Kodak Company, Rochester,
`NY (US)
`
`(*) Notice:
`
`-
`0
`Subject to any disclaimer, the term of this
`past its, adjusted under 35
`M
`YW-
`y U days.
`
`6, 1978 Engelsmann ................. 396/73
`4,097,882 A
`4, 1980 McCullough et al. ....... 348,704
`4,199,785 A
`1/1991 Paxton ........................ 348/47
`4,989,078 A *
`9, 1991 von Hoessle
`... 348,335
`5,051,830 A
`5,164,831 A 1 1/1992 Kuchta et al. ........... 348,231.7
`5,668,597 A
`9, 1997 Parulski et al. ............. 348,350
`5,929,903. A
`7, 1999 Keisow ............
`348,97
`6,288,742 B1* 9/2001 Ansari et al. .......... 348,211.14
`6,611,289 B1* 8/2003 Yu et al. ..................... 348,265
`6,804,460 B1
`10/2004 Oshima et al. ............... 396,74
`2002/0163582 A1 11/2002 Gruber et al. ........... 348.218.1
`2003/0020814 A1
`1/2003 Ono ........................ 348,220.1
`2003/0160886 A1
`8/2003 Misawa et al. ............. 348/347
`2003/0202113 A1 10, 2003 Yoshikawa .................. 348,272
`
`(21) Appl. No.: 11/062,174
`(22) Filed:
`Feb. 18, 2005
`
`(65)
`
`Prior Publication Data
`US 2006/01873 12 A1
`Aug. 24, 2006
`9.
`
`(51) Int. Cl.
`(2006.01)
`GO2B 5/14
`(2006.01)
`G03B 4I/00
`(2006.01)
`H04N 5/225
`(52) U.S. Cl. .................. 359/676: 396/322; 348/207.99
`(58) Field of Classification Search ........ 35 9/676–692,
`359/72, 85, 60,322,435, 436; 348/240.99,
`348/240.1, 239, 240.3, 2622, 207.99,
`348/34, 369,360, 361, 344; 396/72, 85,
`396/60, 322. 435, 436
`See application file for complete search history.
`References Cited
`U.S. PATENT DOCUMENTS
`
`(56)
`
`* cited by examiner
`Primary Examiner Scott J. Sugarman
`Assistant Examiner M. Hasan
`(74) Attorney, Agent, or Firm—Pamela R. Crocker
`
`(57)
`
`ABSTRACT
`
`A digital camera includes a first image sensor, a first wide
`le lens for formi
`first i
`f
`the first
`as the 'S', getti's
`f g
`s
`9.
`s
`orming a second image of the same Scene on the Second
`image sensor, a control element for selecting either a first
`sensor output from the first image sensor or a second sensor
`output from the second image sensor, and a processing
`section for producing the output image from the selected
`sensor output. In one variation of this embodiment, the first
`lens is also a Zoom lens, where the maximum focal length of
`the first lens is less than or equal to the minimum focal
`length of the second Zoom lens.
`
`3,497,614 A
`
`2, 1970 Petrocelli et al. ........... 348,704
`
`55 Claims, 22 Drawing Sheets
`
`K-12 :
`1ST IMAGE
`S
`as
`SENSOR:
`I
`--
`14 E-3
`DRIVERS
`
`- - - - - -
`
`- - - - - - - - - - - - - - -
`
`2
`| FIXED FOCAL
`LENGTH LENS
`
`3
`
`
`
`:
`
`|
`:
`
`)
`ag.Af
`-
`I
`---
`
`---
`
`in host
`INTERFACE
`52
`
`Host
`PC
`
`64
`
`54
`
`-104
`
`REMOVABLE
`AEMORY
`CARD
`
`MEMORY
`CARD
`INTERFACE
`56
`RAM
`MEMORY
`
`58
`
`FIRMARE
`MEMORY
`
`34
`
`S
`ANA-96
`MUX
`
`A/0
`CONVERTER
`
`iMAGE
`PROCESSOR
`
`- J6
`DRAM
`BUFFER-s-
`ag
`
`!---------------- --J
`1-1
`
`CONTROL
`40--PROCESSOR AND
`AUTOFOCUS AND
`iiiNG
`GENERATOR
`46-AUTOEXPOSURE
`DETECTORS
`
`38
`
`48--- FLASH-s-
`
`USER
`CONTROLS
`
`E:
`WO---
`DISPLAY
`
`
`
`so-callAR “Cillar
`PROCESSOR:
`MODEM
`
`APPL-1018 / Page 1 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 1 of 22
`
`US 7,206,136 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IS!
`
`80SNES | 30/W||
`
`| | |
`
`
`
`r--r-------------- –––––– – – –)
`
`APPL-1018 / Page 2 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 2 of 22
`
`US 7,206,136 B2
`
`
`
`
`
`FIG. 2A
`
`APPL-1018 / Page 3 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`APPL-1018 / Page 4 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 4 of 22
`
`US 7,206,136 B2
`
`- - - - -
`
`s s
`N l
`CS
`
`| |---------------1
`
`No.ss,
`
`
`
`(107 }|0700
`
`99
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`}007
`
`SJEMIJO || 91
`
`X0070
`
`
`
`W007 (|NZ #
`
`SN37
`
`APPL-1018 / Page 5 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 5 of 22
`
`US 7,206,136 B2
`
`FIG. 5A
`
`
`
`FIG. 5B
`
`APPL-1018 / Page 6 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 6 of 22
`
`US 7,206,136 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`APPL-1018 / Page 7 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 7 of 22
`
`US 7,206,136 B2
`
`! |Iš?e
`
`99
`
`J. ‘OIH
`
`L
`
`r-----
`
`99
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`W007
`
`APPL-1018 / Page 8 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 8 of 22
`
`US 7,206,136 B2
`
`
`
`APPL-1018 / Page 9 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`APPL-1018 / Page 10 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 10 of 22
`
`US 7,206,136 B2
`
`FIG. OA
`
`
`
`FIG 10C
`
`APPL-1018 / Page 11 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 11 of 22
`
`US 7,206,136 B2
`
`
`
`* FIG. OE
`
`10/1
`12d
`2a a(1)
`1a(4)-NUR-2
`2d (O) (CD)
`
`-
`
`
`
`-/
`
`12d
`
`12 FIG. OF
`
`APPL-1018 / Page 12 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 12 of 22
`
`US 7,206,136 B2
`
`
`
`204
`
`APPL-1018 / Page 13 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 13 of 22
`
`US 7,206,136 B2
`
`
`
`
`
`- R-R-R-R-
`
`- - - -
`L--J - -
`
`- -
`- -
`
`-
`
`FIG. 12
`
`
`
`210
`
`FIG. 13
`
`APPL-1018 / Page 14 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 14 of 22
`
`US 7,206,136 B2
`
`10B
`
`
`
`l
`
`54
`
`15B
`
`fb
`
`43
`
`15B
`
`10C
`
`204
`
`1C
`FIG. 15
`
`FIG 15B
`
`APPL-1018 / Page 15 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 15 of 22
`
`US 7,206,136 B2
`
`FIG. 16A
`
`
`
`9
`
`I
`
`
`
`APPL-1018 / Page 16 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 16 of 22
`
`US 7,206,136 B2
`
`
`
`16.9
`
`WDE ADJUSTMENT
`18b.
`
`18O
`TELE ADJUSTMENT
`
`---12
`
`APPL-1018 / Page 17 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 17 of 22
`
`US 7,206,136 B2
`
`ISOH
`
`0d
`
`99
`
`90
`
`
`
`(107 }}0700
`
`30/W||
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`APPL-1018 / Page 18 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`APPL-1018 / Page 19 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 19 of 22
`
`US 7,206,136 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`APPL-1018 / Page 20 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 20 of 22
`
`US 7,206,136 B2
`
`s
`
`SS.
`
`S
`
`S
`
`S
`
`S3
`
`S
`
`GS
`
`S.
`
`S
`
`S
`
`S
`
`S
`
`S
`
`APPL-1018 / Page 21 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`Sheet 21 of 22
`
`US 7,206,136 B2
`
`
`
`600
`
`iPROCESSOR:
`
`CEllular
`
`PROCESSOR
`- - - - - - - - - -
`
`Y
`
`a
`
`FIG. 23B
`
`APPL-1018 / Page 22 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`U.S. Patent
`
`Apr. 17, 2007
`
`US 7,206,136 B2
`
`
`
`
`
`DDDDDDDD
`
`019
`
`APPL-1018 / Page 23 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`US 7,206,136 B2
`
`1.
`DIGITAL CAMERAUSING MULTIPLE
`LENSES AND IMAGE SENSORS TO
`PROVIDE AN EXTENDED ZOOM RANGE
`
`FIELD OF THE INVENTION
`
`The present invention relates to a digital camera that
`produces digital image files and, more particularly, to a
`digital camera that uses multiple lenses and image sensors to
`provide an extended Zoom range.
`
`10
`
`BACKGROUND OF THE INVENTION
`
`2
`published patent application, a large format digital camera
`exposes multiple detector arrays using multiple lens systems
`to acquire Sub-images of overlapping Sub-areas of large area
`objects. The Sub-images are Stitched together to form a large
`format digital macro-image. However, all of the lenses have
`the same focal length, and all are used simultaneously to
`capture the different Sub-areas of the image.
`Stereo film cameras and stereo electronic cameras are
`known in the prior art. These cameras typically have two
`horizontally separated lenses of the same focal length, which
`focus two slightly different images of the scene onto two
`image sensors or two frames of film. Such a system is
`disclosed in commonly assigned U.S. Pat. No. 4,989,078.
`entitled “Still Video Camera for Recording Stereo Images on
`a Video Disk' and issued on Jan. 21, 1991 in the name of K.
`Bradley Paxton. The two images provide a so-called “stereo
`pair, which simulates the slightly different perspectives that
`a person’s left and right eyes would see when viewing the
`scene. In the aforementioned patent disclosure, the two
`lenses are designed to provide the same magnification, and
`both are used to simultaneously capture the left and right eye
`images on a pair of image sensors in order to achieve a stereo
`effect.
`Film cameras that use multiple lenses to capture multiple
`images at the same time are also known in the prior art. For
`example, some instant film cameras used to produce iden
`tification pictures can capture four Small images simulta
`neously on the same piece of instant film. The four lenses in
`these cameras provide the same magnification, and all are
`used to simultaneously capture the four images.
`According to the second category of prior art, film cam
`eras that include two or more lenses to provide two or more
`different focal lengths are also known in the prior art. For
`example, such cameras can use two different fixed focal
`length lenses which are slid in front of the same film plane.
`This provides an inexpensive “two-position Zoom’ capabil
`ity, that is, two fixed focal length lenses that provide, e.g.,
`the wide angle and telephoto angle settings of a correspond
`ing Zoom lens. In another example, in U.S. Pat. No. 4,097,
`882, entitled “Multiple Lens Camera Having Lens-position
`Controlled Focal-length Adjustment” and issued Jun. 27.
`1978 in the name of Engelsmann, a “110 size pocket film
`camera has a carrier mounting three or more lenses of
`different focal lengths that can be selectively moved trans
`verse to the optical axis of the camera So as to place any one
`of the lenses in an operating position relative to a film plane.
`Digital cameras that include two lenses to provide two
`different focal lengths are also known in the prior art. A lens
`turret is popularly used to obtain multiple focal lengths in a
`camera. However, in the case of a digital still camera or
`especially in the case of mobile phone digital camera, lens
`modules are required to be extremely small due to the
`limited space for the lens module. U.S. Pat. No. 6,804,460,
`entitled “Lens Turret with Back Focal Length Adjustment'
`and issued Oct. 12, 2004 in the names of Oshima et al.,
`describes a lens turret that is said to be extremely compact
`and flat in size and Suitable for digital still cameras and
`mobile phone digital cameras. The lens turret is rotatable
`around an axis and has a wide-angle lens and a telephoto
`angle lens mounted thereon, and a driving mechanism
`rotates the lens turret so that one of the lenses can be set at
`a picture taking position opposite an image sensor. By means
`of back focal length adjustment, the position of the lens with
`the shorter focal length can be fixed on the lens turret at the
`same level thereon as the lens with the longer focal length.
`It is also known to use a two lens arrangement in a film
`scanner, where two lenses with different combinations of
`
`Currently, most digital cameras use a Zoom lens and a
`single color image sensor to capture still and motion images.
`The captured images are then digitally processed to produce
`digital image files, which are stored in a digital memory in
`the camera. The digital image files can then be transferred to
`a computer, displayed, and shared via the Internet. The
`digital camera can be included as part of a mobile telephone,
`to form a so-called "camera phone'. The camera phone can
`transmit the digital image files to another camera phone, or
`to service providers, via a mobile telephone network.
`Small camera size and a large "optical Zoom range' are
`two very important features of digital cameras. Users prefer
`to have a large Zoom range (e.g. 5:1 or greater) rather than
`a limited Zoom range (e.g. 3:1 or Smaller). Unfortunately,
`providing a large Zoom range lens, without sacrificing the
`quality of the captured images, increases the size of the
`digital camera. Large Zoom range lenses are also more
`costly. Thus, there are fundamental trade-offs between small
`camera size, large Zoom range, and low camera cost which
`must be made when designing a digital camera. With higher
`cost cameras, Such as single lens reflex cameras, these
`problems are sometimes addressed by using multiple inter
`changeable Zoom lenses, such as two 3:1 Zoom lenses, e.g.,
`a 28-70 mm Zoom and a 70–210 Zoom. Such an option,
`which has its own problems in user inconvenience, is
`nonetheless not available for low cost digital cameras.
`The prior art of most interest can be separated into two
`categories: image capture systems that use multiple lenses,
`usually two, having the same focal length and image capture
`systems that utilize multiple lenses, also usually two, having
`different focal lengths.
`Addressing the first category, Some digital cameras use
`multiple image sensors to form a color image. In most
`cameras of this type, a single lens is used to provide an
`image of the scene, which is then separated into multiple
`colors by a prism beam splitter. Multiple monochrome
`image sensors are used to capture red, green, and blue color
`separation images. However, as disclosed in U.S. Pat. No.
`6,611,289, entitled “Digital Cameras Using Multiple Sen
`sors with Multiple Lenses” and issued Aug. 26, 2003 in the
`name of Yu et al., it is possible to use multiple image sensors
`and multiple lenses to provide color separation. However,
`this patent disclosure teaches that the lenses all have the
`same focal length, and are all used together, in order to
`simultaneously capture the different color components of the
`image
`Some digital imaging systems also use multiple image
`sensors and multiple lenses to capture different portions of
`the digital image. Such a system is disclosed in U.S.
`Published Patent Application No. US20020163582 A1,
`entitled “Self-calibrating, Digital, Large Format Camera
`with Single or Multiple Detector Arrays and Single or
`Multiple Optical Systems” and published Nov. 7, 2002 in the
`names of Gruber et al. In one embodiment disclosed in this
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`APPL-1018 / Page 24 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`US 7,206,136 B2
`
`3
`focal lengths are used to capture variable sized images. For
`instance, in commonly assigned U.S. Pat. No. 5,929,903,
`entitled “Multiposition Lens Mechanism for a Scanner and
`issued Jul. 27, 1999 in the name of R. H. Kiesow, a
`removable digital camera, which is tethered to a computer, 5
`is Supported in a housing in a film Scanner in the optical path
`of a multiposition lens assembly having a single focal length
`lens and a Zoom lens. The lens assembly positions the lenses
`in the optical path of the camera for Scanning different sized
`images, e.g., two or more film format sizes. However, these 10
`cameras, both film and digital, that use multiple focal length
`lenses share the characteristic of using only a single 'sen
`Sor, that is, a single film or a single electronic image sensor.
`In U.S. Pat. No. 6,288,742, entitled “Video Camera
`Including Multiple Image Sensors' and issued Sep. 11, 2001 15
`in the names of Ansariet el., a digital motion camera useful
`in teleconferencing includes two lenses and two image
`sensors. As disclosed in this patent, the first lens is an 8 mm
`fixed focus lens for providing a relatively wide-angle view
`of a room and the second lens is a 16 mm lens with manual 20
`focus control for providing high resolution document trans
`mission capability. The first lens is oriented for a room view
`of a conference participant to provide face-to-face commu
`nication during a videotelephone conference, and the second
`lens is oriented at a substantial angle to the first lens for 25
`viewing a document, e.g., on a table. During a videotele
`phone conference, Such a camera permits fast Switching
`between an image of the room as seen through the first lens
`or an image of a document as seen through the second lens,
`without the need for expensive and tediously slow moving 30
`pan/tilt stages and/or a plurality of complete camera units.
`Another camera, the Sanyo S750 UMTS cellphone camera,
`has a similar kind of dual imaging capability, where an
`inwardly facing VGA imager captures an image of the caller
`using the cellphone while an outwardly facing 1 megapixel 35
`imager captures an image of a scene that the caller is looking
`at. Such cameras, however, are not useful in the environment
`of the present invention because the lenses are not collecting
`images from the same scene.
`In U.S. Pat. No. 4,199,785, entitled “Electronic Zoom 40
`Feature' and issued Apr. 22, 1980 in the name of
`McCullough et al., a television system employs two (or
`more) fixed focal length Vidicon cameras, one camera with
`a wide angle field of view and the other camera with a
`narrow angle field of view, and an electronic Zoom feature 45
`for Zooming between the two fields of view. The cameras are
`boresighted such that the field of view of the smaller field
`camera is within, and usually centered in the field of view of
`the larger field camera. The "Zoom’ is accomplished by
`manipulating the scan generators of the two cameras and 50
`expanding the central portion of the display with the image
`from the Smaller field camera as the Zoom amount is
`progressively increased. This system, of course, is an alter
`native to a single optical Zoom lens, whose usage the patent
`disclosure discourages as they (optical Zooms) are lower 55
`quality, more expensive and mechanically more complex
`than fixed focal length lenses. However, the inherent draw
`back of an electronic Zoom is also low quality since the
`resolution of the electronic Zoom feature ordinarily is lim
`ited by the number of scan lines available for Zooming. 60
`Consequently, this patent disclosure is devoted to control
`ling the scan lines of the two Vidicon cameras So as to be able
`to Zoom without an effective loss of resolution.
`In U.S. Pat. No. 5,051,830, entitled “Dual Lens System
`for Electronic Camera' and issued Sep. 24, 1991 in the name 65
`of Hoessle, a double focal length electronic camera (used on
`board a guided missile) includes a single lens system com
`
`4
`ponent having a short focal length section integrated into the
`center of a Surrounding lens section having a long focal
`length, where each focal length section has its own dedicated
`picture array sensor. Here too, this lens system is a Substitute
`for a single motor driven Zoom, which the disclosure deni
`grates because of size, expense, heaviness, inherent com
`plexity; furthermore, an optical Zoom is “which is impor
`tant much too slow with respect to its use” (col. 2, line 1
`of the Hoessle patent).
`None of these prior art systems, and especially the mul
`tifocal length prior art systems, provide a Sufficiently com
`pact, low cost, large Zoom range optical system for a small,
`lightweight and relatively inexpensive consumer digital
`camera. As especially pointed out in the aforementioned
`Hoessle patent, it is additionally desirable to avoid the
`slowness so typical of Zoom usage and to be able to traverse
`a large Zoom range quickly. What is therefore needed is a
`digital camera that provides a rapidly-operating extended
`Zoom range without unduly increasing the size or cost of the
`digital camera.
`
`SUMMARY OF THE INVENTION
`
`The object of this invention is to provide an extended
`Zoom range in a digital camera without unduly increasing
`the size or cost of the camera.
`Another object of this invention is to provide an extended
`optical Zoom range in a digital camera by means of a
`plurality of separate lenses and corresponding image sen
`SOS.
`Another object of this invention is to provide an extended
`optical Zoom range in a digital camera where the movement
`between user-requested Zoom positions may be undertaken
`in an expedited manner.
`The present invention is directed to overcoming one or
`more of the problems set forth above. Briefly summarized,
`the invention comprises an electronic camera for producing
`an output image of a scene, where the camera comprises: a
`first image sensor for generating a first sensor output: a first
`lens for forming a first image of the scene on the first image
`sensor; a second image sensor for generating a second
`sensor output; a Zoom lens for forming a second image of the
`same scene on the second image sensor, wherein the Zoom
`lens is adjustable between a minimum focal length and a
`maximum focal length to provide the second image; a
`control element for selecting either the first sensor output
`from the first image sensor or the second sensor output from
`the second image sensor, thereby providing a selected sensor
`output; and a processing section for producing the output
`image from the selected sensor output. In one variation of
`this embodiment, the first lens is also a Zoom lens, where the
`maximum focal length of the first lens is less than or equal
`to the minimum focal length of the second Zoom lens.
`These various aspects of the invention provide significant
`technical advantages. By providing a plurality of optical
`image capture modalities within a digital camera, wherein
`each modality includes a lens-sensor combination with a
`distinctive different focal length or combination of focal
`lengths (i.e., a Zoom), the conflicted requirements (namely,
`large size, high cost and compromised optical quality)
`engendered by digital camera consumer desire for a large
`Zoom ratio, e.g., 10:1, can be accomplished in a smaller
`scale space at lower cost with higher quality optical results
`than heretofore achieved.
`These and other aspects, objects, features and advantages
`of the present invention will be more clearly understood and
`appreciated from a review of the following detailed descrip
`
`APPL-1018 / Page 25 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`US 7,206,136 B2
`
`5
`tion of the preferred embodiments and appended claims, and
`by reference to the accompanying drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`10
`
`15
`
`FIG. 1 depicts a block diagram of a first embodiment of
`a digital camera using a fixed focal length, wide-angle lens
`with a first image sensor, and a Zoom lens with a second
`image sensor according to the invention.
`FIGS. 2A and 2B are two perspective views of the digital
`camera shown in FIG. 1.
`FIG. 3 depicts a flow diagram showing a method for
`capturing digital images using the digital camera shown in
`FIG 1.
`FIG. 4 depicts a block diagram of a second embodiment
`of a digital camera using a first Zoom lens with a first image
`sensor, and a second Zoom lens with a second image sensor
`according to the invention.
`FIGS.5A and 5B are two perspective views of the digital
`camera shown in FIG. 4.
`FIG. 6 depicts a flow diagram showing a method for
`capturing digital images using the digital camera shown in
`FIG. 4.
`FIG. 7 depicts a block diagram of a third embodiment of
`25
`a digital camera using a first Zoom lens with a first image
`sensor, a second Zoom lens with a second image sensor and
`a fixed focal length lens with a third image sensor according
`to the invention.
`FIGS. 8A and 8B are two perspective views of the digital
`camera shown in FIG. 7.
`FIG. 9 depicts a flow diagram showing a method for
`capturing digital images using the digital camera shown in
`FIG. 7.
`FIGS. 10A, 10B, 10C, 10D, 10E and 10F diagram the
`optical layout of several embodiments of the image capture
`assembly used in the cameras shown in FIGS. 1, 4, 7, 19 and
`21.
`FIG. 11 is a frontal view of a digital camera employing
`two image capture assemblies, one fixed focal length and the
`other Zoom, of the type shown in FIGS. 10A-10F.
`FIG. 12 is a top view of the digital camera shown in FIG.
`11.
`FIG. 13 is a side view of the digital camera shown in FIG.
`11.
`FIGS. 14A, 14B and 14C are three views of a digital
`camera employing two image capture assemblies, both
`Zoom, of the type shown in FIGS. 10A-10F.
`FIGS. 15A, 15B and 15C are three views of a digital
`camera employing three image capture assemblies of the
`type shown in FIGS. 10A-10F.
`FIGS. 16A and 16B show two views of the optical relay
`subassembly shown in the various embodiments of FIGS.
`10A-10F for supporting a fixed focal length lens in relation
`to an image sensor along a folded optical path.
`FIG. 17 shows the optical relay subassembly shown in the
`various embodiments of FIGS. 10A-10F for supporting a
`Zoom lens in relation to an image sensor along a folded
`optical path.
`FIGS. 18A and 18B show two imagers with different
`panoramic aspect ratios and the effect obtained by changing
`the aspect ratio.
`FIG. 19 depicts a block diagram of a further embodiment
`of a digital camera using a first fixed focal length lens with
`a first sensor, a second fixed focal length lens with a second
`sensor, and a third fixed focal length lens with a third sensor.
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`FIG. 20 depicts a flow diagram showing a method for
`capturing digital images using the digital camera shown in
`FIG. 19.
`FIG. 21 depicts a block diagram of a further embodiment
`of a digital camera using a first lens with a first sensor having
`pixels of one size. e.g., three micron pixels, and a second
`lens with a second sensor having pixels of another size, e.g.,
`five micron pixels.
`FIG. 22 is a diagram useful for explaining an express
`Zooming feature.
`FIGS. 23A and 23B are perspective views of the front and
`back of a cell phone including a camera with multiple lenses
`and multiple sensors.
`FIGS. 24A and 24B are two views of the image capture
`assembly used in the cell phone shown in FIGS. 23A and
`23B.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`Because digital cameras employing imaging devices and
`related circuitry for signal processing are well known, the
`present description will be directed in particular to elements
`forming part of, or cooperating more directly with, apparatus
`in accordance with the present invention. Elements not
`specifically shown or described herein may be selected from
`those known in the art. Certain aspects of the embodiments
`to be described may be provided in software. Given the
`system as shown and described according to the invention in
`the following materials, Software not specifically shown,
`described or Suggested herein that is useful for implemen
`tation of the invention is conventional and within the ordi
`nary skill in Such arts.
`Each of the several embodiments of the present invention
`include an image capture assembly having multiple lenses
`and multiple image sensors mounted within a digital camera
`in order to provide an extended Zoom range. This can reduce
`the cost and size of the camera, and improve its optical
`performance, compared with a camera having a single
`sensor and a large range Zoom lens (e.g. having a 10:1 zoom
`range). While not in an exactly coaxial arrangement with
`respect to each other, the multiple lenses and sensors are
`generally aligned with respect to each other so as to be
`viewing substantially the same object, albeit with different
`fields of view. Each image capture assembly comprises two
`or more optical relay Subassemblies having a lens and an
`image sensor disposed at opposing ends thereof and a folded
`optical path for directing the light from the lens to the sensor.
`This configuration can further reduce the size of the optical
`components, thereby enabling the design and manufacture
`of a very thin and compact camera. While the folded optics
`are used in many of the preferred embodiments, a folded
`optical path is not generally necessary for practice of the
`invention. This is particularly true for the wide angle optical
`Subassemblies since the focal length of Such wide angle
`lenses is very short to begin with. Also, the size of the sensor,
`and consequently the size of the image that must be pro
`duced to fill the sensor, may be small enough to reduce the
`focal length to an acceptable front-to-back dimension—even
`for normal and short telephoto focal lengths.
`In each embodiment, the camera includes a control ele
`ment for selecting either the first sensor output from the first
`image sensor or a sensor output from one of the other image
`sensors, thereby providing a selected sensor output that may
`be provided to a processing section in the camera for
`producing an output image. Moreover, each embodiment
`includes some type of user control that allows a user to select
`
`APPL-1018 / Page 26 of 40
`APPLE INC. v. COREPHOTONICS LTD.
`
`

`

`US 7,206,136 B2
`
`7
`a focal length, either directly or via Some marking (e.g.,
`“panoramic' or “portrait”) indicative of a focal length; the
`aforementioned control element is then responsive to the
`user control for selecting a corresponding sensor output. In
`Some embodiments, a single "Zoom lens' user control is
`used, e.g., where the “wide setting selects a wide angle
`fixed focal length lens and the “tele' setting(s) select various
`positions of a Zoom lens. The user control output is then
`provided to the control element, which selects the image
`sensor that is used to produce the output image. When the
`selected sensor is for the Zoom lens, the user control (or the
`control element) also enables the Zoom and focus motors for
`the Zoom lens to drive the Zoom lens to the selected focal
`length. In addition, digital Zooming may be used to Zoom
`“up' from the wide angle setting to the minimum focal
`length setting of the Zoom lens. All this, of course, may be
`transparent to the user, who simply manipulates the "Zoom
`lens' user control between the “wide” and “tele' settings.
`Referring first to FIGS. 10A-10F, several diagrams are
`shown of the optical layout of several embodiments of an
`image capture assembly 1, which is included within the
`various embodiments of a digital camera (which will be
`described later). These diagrams include the optical relay
`Subassemblies 1a, 1b and 1c containing the aforementioned
`folded optical elements. In each of the FIGS. 10A-10F, a
`circle 1d delineates an optical profile of a front surface of a
`digital camera through which the respective lenses protrude.
`The optical relay subassemblies 1a, 1b and 1c are folded
`behind the lenses and, as will be described, covered by the
`front surface of the camera.
`In a first embodiment of the present invention, a digital
`camera employs a first fixed focal length wide angle lens 2
`with a first image sensor 12, and a Zoom lens 3 with a second
`image sensor 14. In the first embodiment as shown in FIG.
`10A, an image capture assembly 1 includes the first lens 2
`and the first image sensor 12 mounted at opposing ends of
`a first optical relay Subassembly 1a having a folded optical
`path arranged between the first image sensor 12 and the lens
`2. The first lens 2, which preferably is a fixed focal length
`wide angle lens, forms a first image of a scene on the first
`image sensor 12. The image capture assembly 1 also
`includes the Zoom lens 3 and the second image sensor 14
`mounted at opposing ends of a second optical relay Subas
`sembly 1b having a folded optical path arranged between the
`second image sensor 14 and the Zoom lens 3. The Zoom lens
`3, which has a range of focal lengths adjustable between a
`minimum focal length and a maximum focal length, forms
`a second image of the scene on the second image sensor 14.
`In this embodiment, the first lens 2 is a wide angle lens
`having a focal length less, and preferably Substantially less,
`than the minimum focal length of the Zoom lens 3.
`In a second embodiment of the present invention, a digital
`camera employs a first Zoom lens 3 with an image sensor 14,
`and a second Zoom lens 4 with an image sensor 16. In the
`second embodiment as shown in FIG. 10B, the image sensor
`14 will be characterized as the first image sensor 14 and the
`image sensor 16 will be characterized as the second image
`sensor 16. Accordingly, the image capture assembly 1
`includes the first Zoom