(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2012/0026366 A1
`
` Golan et al. (43) Pub. Date: Feb. 2, 2012
`
`
`US 20120026366A1
`
`(54) CONTINUOUS ELECTRONIC ZOOM FOR AN
`IMAGING SYSTEM WITH MULTIPLE
`{:DI/ifianNrg‘UEVICES HAVING DIFFERENT
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`H04N 5/262
`(52) US. Cl. .............................. 348/2402; 348/E05.055
`
`(75)
`
`Inventors:
`
`Chen Golan, Ein Vered (IL); Boris
`Kipnis, Tel-av1v (IL)
`
`(57)
`
`ABSTRACT
`
`(73) Assignee:
`
`Nextvision Stabilized Systems
`Ltd., Raanana (IL)
`
`(21) Appl. No.:
`
`13/262,842
`
`(22) PCT Filed:
`
`Apr. 6, 2010
`
`(86) PCT No .
`"
`§371 (c)(l)
`(2), (4) Date:
`
`PCT/IL1 0/00281
`
`Oct. 43 2011
`
`Related US. Application Data
`
`(60) Provisional application No. 61 /1 67,226, filed on Apr.
`7, 2009.
`
`A method for continuous electronic zoom in a computerized
`image acquisition system, the system having a wide image
`acquisition device and a tele image acquisition device having
`a tele image sensor array coupled with a tele lens having a
`narrow FOV, and a tele electronic zoom. The method includes
`providing a user of the image acquisition device with a zoom
`selecting control, thereby obtaining a requested zoom, select-
`ing one of the image acquisition devices based on the
`requested zoom and acquiring an image frame,
`thereby
`obtaining an acquired image frame, and performing digitally
`zoom on the acquired image frame, thereby obtaining an
`acquired image frame with the requested zoom. The align-
`ment between the wide image sensor array and the tele image
`sensor array is computed, to facilitate continuous electronic
`zoom with uninterrupted imaging, when switching back and
`forth between the wide image sensor array and the tele image
`sensor array.
`
`700
`
`752
`
`750
`
`Sam: Spam megmteén 5
`
`RQB ta mach
`'
`
`770
`
`fix
`/
`
`,
`Siting out
`»
`Chm?
`
`780
`
`
`
`
`
`Zoom {infirm}
`
`20m
`
`630
`
`stator Space Com-«ammo
`mam-hm we
`
`754
`
`'
`
`APPL-1005/ Page 1 of 13
`Apple v. Corephotonics
`
`APPL-1005 / Page 1 of 13
`Apple v. Corephotonics
`
`

`

`Patent Application Publication
`
`Feb. 2, 2012 Sheet 1 0f 7
`
`US 2012/0026366 A1
`
`0;
`
`142
`
`122
`
`D ‘Iul'iden out}: Znnm Cnntrul
`
`130
`
`Fig 1
`
`APPL-1005/ Page 2 of 13
`
`APPL-1005 / Page 2 of 13
`
`

`

`Patent Application Publication
`
`Feb. 2, 2012 Sheet 2 of 7
`
`US 2012/0026366 A1
`
`/ 200
`
`210
`
`Providing a wide image
`acquisition device and a narrow
`
`image acquisition device
`
`
`
` 220
`
`230
`
`
`
`Determining alignment offsets
`
` Selecting an image / 240
`
`acquisition device
`
`250
`
`Acquiring an image frame
`
`260
`
`Resampling the acquired image
`frame to the requested zoom
`
`Fig 2
`
`APPL-1005/ Page 3 0f 13
`
`APPL-1005 / Page 3 of 13
`
`

`

`Patent Application Publication
`
`Feb. 2, 2012 Sheet 3 0f 7
`
`US 2012/0026366 A1
`
`310
`
`~-._
`
`340
`
`CL;
`
`Digital
`
`‘Iul'ideu nut _
`Ennm
`
`
`_
`Elnffiklp
`
`Cnntininus
`Enum
`
`Zuum Contrul
`
`
`
`330
`
`Fig 3
`
`APPL-1005/ Page 4 of 13
`
`APPL-1005 / Page 4 of 13
`
`

`

`Patent Application Publication
`
`Feb. 2, 2012 Sheet 4 0f 7
`
`US 2012/0026366 A1
`
`/ 400
`
`Prov1d1ng a W1de 1mage
`acquisition device and a narrow
`
`image acquisition device
`
`
`
`Determining alignment offsets
`
`
`
`410
`
`420
`
`430
`
`
`
`Zoom selection
`
`
`
`Selecting bin/skip
`
`functions parameters
`
`
`
`435
`
`Selecting an image
`acquisition device
`
`
`
`Acquiring an image frame
`
`440
`
`450
`
`460
`
`
`
`Resampling the acquired image
`
`frame to the requested zoom
`
`Fig 4
`
`APPL-1005/ Page 5 0f 13
`
`APPL-1005 / Page 5 of 13
`
`

`

`Patent Application Publication
`
`Feb. 2, 2012 Sheet 5 0f 7
`
`US 2012/0026366 A1
`
` p70
`
`510\
`U”
`575
`
`512
`
`_
`Fig 5b
`
`580
`
`{If
`
`570
`
`.
`
`.
`
`HER—ah
`.I
`hwy 574
`
`APPL-1005/ Page 6 0f 13
`
`APPL-1005 / Page 6 of 13
`
`

`

`Patent Application Publication
`
`Feb. 2, 2012 Sheet 6 0f 7
`
`US 2012/0026366 A1
`
`600
`
`630
`
`xx&\\\\\\\\\\\\\\\\\\\\\\\\\\
`«“3
`
`640
`
`642
`
`APPL-1005/ Page 7 of 13
`
`€50
`“'x'x \‘x‘x‘x‘x‘x‘xfim‘x‘kfi‘kfi-
`
`VII/111111111’wx/x/x/x/x/x/m/mIV
`
`660
`I“
`“as?
`
`111W,111,1/1/1/1/1/1/«4444/4 31111111111111111111111111
`§\\\\\\\\\§§..\
`
`Sxx\\\\\\§
`
`
` 1 f
`1',
`
`§«(Wtiiiuw' '<<<<1
`
`'//////t/I/I/I/I/I/I//////////////////}
`‘V§§\xxxx\§\\\\\\\\\\\\\\\\\\\\§z~
`
`632
`
`APPL-1005 / Page 7 of 13
`
`

`

`Patent Application Publication
`
`Feb. 2, 2012 Sheet 7 0f 7
`
`US 2012/0026366 A1
`
`642
`
`700
`
`/
`
`RGB m Y,c.r,*c.h
`
`770
`
`752
`
`750
`
`X mags: {wt .
`
`,M :Cemr Spam Csnmmim 3
` 760
`
`780
`
`,
`
`‘
`>
`‘
`msgom .
`
`«:E ‘23 “if? E
`
`1 SEQ: 53$an
`
`754
`
`Kama {inmmi
`
`:5an
`
`630
`
`Fig 7
`
`APPL-1005/ Page 8 0f 13
`
`APPL-1005 / Page 8 of 13
`
`

`

`US 2012/0026366 A1
`
`Feb. 2, 2012
`
`CONTINUOUS ELECTRONIC ZOOM FOR AN
`IMAGING SYSTEM WITH MULTIPLE
`IMAGING DEVICES HAVING DIFFERENT
`FIXED FOV
`
`RELATED APPLICATION
`
`[0001] The present application claims the benefit of US.
`provisional application 61/1 67,226 filed on Apr. 7, 2009, the
`disclosure of which is incorporated herein by reference.
`
`FIELD OF THE INVENTION
`
`[0002] The present invention relates to an electronic zoom
`for imaging systems, and more particularly, the present inven-
`tion relates to a continuous electronic zoom for an image
`acquisition system, the system including multiple imaging
`devices having different fixed FOV.
`
`BACKGROUND OF THE INVENTION AND
`PRIOR ART
`
`[0003] Digital zoom is a method of narrowing the apparent
`angle ofview ofa digital still or Video image. Electronic zoom
`is accomplished by cropping an image down to a centered
`area of the image with the same aspect ratio as the original,
`and usually also interpolating the result back up to the pixel
`dimensions of the original. It is accomplished electronically,
`without any adjustment of the camera’s optics, and no optical
`resolution is gained in the process. Typically some informa-
`tion is lost in the process.
`[0004]
`In Video streams (such as PAL, NTSC, SECAM,
`656, etc.) the image resolution is known, and by using image
`sensors having substantially higher resolution, one can per-
`form lossless electronic zoom. The ratio between the image
`sensor resolution and the output resolution dictates the loss-
`less electronic zoom range. For example, having a 5 Mega-
`pixel, 2592x1944, image sensor array and an output resolu-
`tion frame of 400x300 yields maximal lossless electronic
`zoom of 6.48:
`
`2592/400:6.48,
`[0005]
`1944/300:6.48.
`[0006]
`[0007] Typically, a camera with a large dynamic zoom
`range requires heavy and expensive lenses, as well as com-
`plex design. Electronic zoom does not need moving mechani-
`cal elements, as does optical zoom.
`[0008] There is a need for and it would be advantageous to
`have image sensors, having static, light weight electronic
`zoom and a large lossless zooming range.
`
`SUMMARY OF THE INVENTION
`
`[0009] The present invention describes a continuous elec-
`tronic zoom for an image acquisition system, having multiple
`imaging devices each with a different fixed field of view
`(FOV). Using two (or more) image sensors, having different
`fixed FOV, facilitates a light weight electronic zoom with a
`large lossless zooming range. For example, a first image
`sensor has a 60° angle of view and a second image sensor has
`a 600 angle ofview. Therefore, Wide_FOV:Narrow_FOV*6.
`Hence, switching between the image sensors provide a loss-
`less electronic zoom of 62:36. This lossless electronic zoom
`is also referred to herein, as the optimal zoom:
`Optimal,Zoom:(Wide7FOV/Narrow7FOV)2.
`
`It should be noted that to obtain similar zoom (x36)
`[0010]
`by optical means, for an output resolution frame of 400x300,
`the needed image sensor array is:
`[0011]
`36*400:l4400,
`[0012]
`36*300:10800.
`[0013]
`l4400*10800:155,520,000.
`Hence, to obtain a zoom of x36 by optical means, for an
`output resolution frame of 400x300, one needs a 155 Mega-
`pixel, l4400><10800, image sensor array.
`[0014] According to teachings of the present invention,
`there is provided a method for continuous electronic zoom in
`a computerized image acquisition system, the system having
`multiple optical image acquisition devices each with a FOV.
`The method includes providing a first image acquisition
`device having a first image sensor array coupled with a first
`lens having a first FOV, typically a wide FOV , and a first
`electronic zoom. The method further includes providing a
`second image acquisition device having a second image sen-
`sor array coupled with a second lens having a second FOV,
`typically a narrow FOV, and a second electronic zoom. Typi-
`cally, the angle ofview ofthe first FOV is wider than the angle
`of view of the second FOV. At least a portion of the environ-
`ment, viewed from within the second FOV of the second
`image acquisition device, overlaps the environment viewed
`from within the first FOV ofthe first image acquisition device.
`The method further
`includes computing the alignment
`between the first image sensor array and the second image
`sensor array, whereby determining an X-coordinate offset, a
`Y—coordinate offset and optionally, a Z-rotation offset of the
`correlation between the first image sensor array and the sec-
`ond image sensor array.
`[0015] The method further includes the steps ofproviding a
`user of the image acquisition device with a zoom selecting
`control, thereby obtaining a requested zoom, selecting one of
`the image acquisition devices based on the requested zoom,
`acquiring an image frame with the selected image acquisition
`device, thereby obtaining an acquired image frame, and per-
`forming digitally zoom on the acquired image frame, thereby
`obtaining an acquired image frame with the requested zoom.
`The calibration of the alignment, between the first image
`sensor array and the second image sensor array, facilitates
`continuous electronic zoom with uninterrupted imaging,
`when switching back and forth between the first image sensor
`array and the second image sensor array. Preferably the elec-
`tronic calibration is performed with sub-pixel accuracy.
`[0016] Optionally, the computerized image acquisition sys-
`tem is configured to provide zooming functions selected from
`the group consisting of a bin function and a skip function. The
`selecting of the image acquisition device includes selecting
`the parameters of the bin and/or skip functions, wherein the
`method further includes the step of applying the selected
`bin/skip functions to the acquired image frame, before the
`performing of the digital zoom step.
`[0017]
`In variations of the present invention, the image
`sensor arrays are focused to the infinite.
`[0018] Optionally, the first lens is a focus adjustable lens.
`[0019] Optionally, the second lens is a focus adjustable
`lens.
`
`[0020] Optionally, the second lens is a zoom lens.
`[0021]
`In image acquisition systems having more than two
`imaging devices, the electronic calibration step is performed
`on each pair of adjacently disposed image sensor arrays.
`[0022]
`In variations ofthe present invention, the first image
`acquisition device and the second image acquisition device
`APPL-1005/ Page 9 0f 13
`
`APPL-1005 / Page 9 of 13
`
`

`

`US 2012/0026366 A1
`
`Feb. 2, 2012
`
`are coupled with a mutual front lens and a beam splitter,
`wherein one portion of the light reaching the beam splitter is
`directed towards the first image sensor array and the remain-
`der portion of the light reaching the beam splitter is directed
`towards the second image sensor array.
`[0023]
`In embodiments of the present invention, the first
`image sensor array is a color sensor and the second image
`sensor array is a monochrome sensor, wherein a colored
`image frame is acquired by the first image sensor array, a
`monochrome image frame is acquired by the second image
`sensor array, wherein the colored image frame and the mono-
`chrome image frame are fused to form a high resolution
`colored image frame.
`In preferred embodiments of the
`present invention, the angle of view of the first FOV is wider
`than the angle of view of the second FOV. However, in varia-
`tion ofthe present invention, the angle ofview ofthe first FOV
`is substantially equal to the angle of view of the second FOV.
`[0024] Optionally, the fusion of the colored image frame
`and the monochrome image frame includes the step of com-
`puting color values for the high resolution pixels ofthe mono-
`chrome image frame from the respective low resolution pixels
`of the colored image frame. Optionally, the computing of
`color values is performed in sub pixel accuracy.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0025] The present invention will become fully understood
`from the detailed description given herein below and the
`accompanying drawings, which are given by way of illustra-
`tion and example only and thus not limitative of the present
`invention, and wherein:
`[0026]
`FIG. 1 is a block diagram illustration of another
`zoom control sub-system for an image acquisition system,
`according to variations of the present invention;
`[0027]
`FIG. 2 is a schematic flow diagram chart that out-
`lines the successive steps of the continuous zoom process,
`according to embodiments of the present invention;
`[0028]
`FIG. 3 is a block diagram illustration of a zoom
`control sub-system for an image acquisition system, accord-
`ing to variations of the present invention;
`[0029]
`FIG. 4 is a schematic flow diagram chart that out-
`lines the successive steps of the continuous zoom process,
`according to variations ofthe present invention, include using
`bin/skip functions;
`[0030]
`FIGS. 5a and 5b illustrate examples of beam splitter
`configurations for image acquisition systems, according to
`embodiments of the present invention;
`[0031]
`FIG. 6 is a block diagram illustration of a camera
`system, according to embodiments of the present invention,
`including a color image sensor having wide FOV and a color
`image sensor having narrow FOV; and
`[0032]
`FIG. 7 is a block diagram illustration of another
`zoom control sub-system for a color image acquisition sys-
`tem, according to variations of the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`[0033] Before explaining embodiments of the invention in
`detail, it is to be understood that the invention is not limited in
`its application to the details of construction and the arrange-
`ment of the components set forth in the host description or
`illustrated in the drawings.
`[0034] Unless otherwise defined, all technical and scien-
`tific terms used herein have the same meaning as commonly
`
`understood by one of ordinary skill in the art of the invention
`belongs. The methods and examples provided herein are
`illustrative only and not intended to be limiting.
`[0035]
`It should be noted that in general, the present inven-
`tion is described, with no limitations, in terms of an image
`acquisition system having two image acquisition devices. But
`the present invention is not limited to two image acquisition
`devices, and in variations of the present invention, the image
`acquisition system can be similarly embodied with three
`image acquisition devices and more.
`[0036] Reference is made to FIG. 1, which is a block dia-
`gram illustration of a zoom control sub-system 100 for an
`image acquisition system, according to preferred embodi-
`ments ofthe present invention. Zoom control sub-system 100
`includes multiple image sensors, each with a fixed and pref-
`erably different FOV, configured to provide continuous elec-
`tronic zoom capabilities with uninterrupted, when switching
`back and forth between the image sensors.
`[0037] Zoom control sub-system 100 includes a tele image
`sensor 110 coupled with a narrow lens 120 having a prede-
`signed FOV 140, a wide image sensor 112 coupled with a
`wide lens 122 having a predesigned FOV 142, a zoom control
`module 130 and an image sensor selector 150. An object 20 is
`viewed from both tele image sensor 110 and wide image
`sensor 112, whereas the object is magnified in tele image
`sensor 110 with respect to wide image sensor 112, by a
`predesigned factor. In the optimal configuration, the FOV of
`wide image sensor 112 can be calculated by multiplying the
`FOV of tele image sensor 110 by the optimal zoom of image
`sensors 110 and 112. Tele image sensor 110 and wide image
`sensor 112 are adj acently disposed, such that at least a portion
`of the environment viewed from within the narrow FOV of
`
`tele image acquisition device 110 overlaps the environment
`viewed from within the wide FOV of wide image acquisition
`device 112.
`
`[0038] Before using zoom control sub-system 100, an elec-
`tronically calibrating is performed to determine the alignment
`offsets between wide image sensor array 110 and tele image
`sensor array 112. Typically, since the spatial offsets between
`wide image sensor array 110 and tele image sensor array 112
`are fixed, the electronic calibration step is performed one
`time, after the manufacturing ofthe image acquisition system
`and before the first use. The electronic calibration yields an
`X-coordinate offset, a Y—coordinate offset and optionally, a
`Z-coordinate rotational offset ofthe correlation between wide
`
`image sensor array 110 and tele image sensor array 112.
`Preferably, all three aforementioned offset values are com-
`puted in sub-pixel accuracy. It should be noted that for image
`acquisition systems with more than two image sensors, the
`electronic calibration step is performed on each pair of adja-
`cently disposed image sensor arrays.
`[0039] Zoom control circuit 130 receives a required zoom
`from an operator of the image acquisition system, and selects
`the relevant image sensor (110 and 112) by activating image
`sensor selector 150 position. The relevant camera zoom factor
`is calculated by zoom control unit 130.
`[0040] An aspect of the present invention is to provide
`methods facilitating continuous electronic zoom capabilities
`with uninterrupted imaging, performed by an image acquisi-
`tion system having multiple image sensors, each with a fixed
`and preferably different FOV. The continuous electronic
`zoom with uninterrupted imaging is also maintained when
`switching back and forth between adjacently disposed image
`sensors.
`
`APPL—1005 / Page 10 of 13
`
`APPL-1005 / Page 10 of 13
`
`

`

`US 2012/0026366 A1
`
`Feb. 2, 2012
`
`[0041] Reference is also made to FIG. 2, which is a sche-
`matic flow diagram chart that outlines the successive steps of
`an example continuous zoom process 200, according to
`embodiments of the present invention, performed on image
`acquisition system, having a zoom control sub-system such as
`zoom control sub-system 100. Process 200 includes the flow-
`ing steps:
`Step 210: providing a wide image acquisition device and a
`tele image acquisition device.
`[0042] Multiple optical image acquisition devices can be
`used, but for description clarity, with no limitation, the
`method will be described in terms of two image acqui-
`sition devices: wide image acquisition device and a tele
`image acquisition device.
`[0043] Both image acquisition devices (110 and 112)
`include an image sensor array coupled with a lens (120
`and 122, respectively), providing a fixed FOV (tele FOV
`140 and wide FOV 142, respectively). Preferably, wide
`FOV 142 is substantially wider than narrow FOV 140.
`[0044] The image acquisition devices are adjacently dis-
`posed, such that at least a portion of the environment,
`viewed from within narrow FOV 140 of the tele image
`acquisition device 110, overlaps
`the environment
`viewed from within the wide FOV 142 of wide image
`acquisition device 112.
`Step 220: determining alignment offsets.
`[0045] Before using zoom control sub-system 100, an
`electronically calibrating is performed to determine the
`alignment offsets between wide image sensor array 110
`and tele image sensor array 112. Typically, since the
`spatial offsets between wide image sensor array 110 and
`tele image sensor array 112 are fixed, the electronic
`calibration step is performed one time, after the manu-
`facturing of the image acquisition system and before the
`first use. The electronic calibration yields an X-coordi-
`nate offset and a Y—coordinate offset of the correlation
`
`between wide image sensor array 110 and tele image
`sensor array 1 12. Preferably, the X-coordinate offset and
`the Y—coordinate offset are computed in sub-pixel accu-
`racy. It should be noted that for image acquisition sys-
`tems with more than two image sensors, the electronic
`calibration step is performed on each pair of adjacently
`disposed image sensor arrays.
`Step 230: zoom selection.
`[0046] A user of the image acquisition selects the
`required zoom.
`Step 240: selecting an image acquisition device.
`[0047] The zoom control 130 selects an image acquisi-
`tion device with the having a zoom more proximal to the
`requested zoom.
`Step 250: acquiring an image frame.
`[0048] An image frame is acquired by the selected image
`acquisition device.
`Step 260:
`resampling the acquired image frame to the
`requested zoom.
`[0049] The zoom control 130 computes the zoom factor
`between the fixed zoom of the selected image acquisi-
`tion device and the requested zoom. Based on the com-
`puted factor, zoom control 130 performs electronic
`zoom on the acquired image frame to meet the requested
`zoom.
`
`[0050] Reference is made back to FIG. 1 and referring also
`to FIGS. 5a and 5b, which illustrates examples of beam
`splitter configurations for image acquisition systems, accord-
`
`ing to embodiments of the present invention. In variations of
`the present invention, wide image acquisition device 112 and
`tele image acquisition device 110 are coupled with a mutual
`front lens 570 and a beam splitter 580, wherein one portion of
`the light reaching beam splitter 580 is directed towards wide
`image sensor array 112 and the remainder portion ofthe light
`reaching beam splitter 580 is directed towards tele image
`sensor array 110. In FIG. 5a, the beam splitter configuration
`includes a wide angle lens 572, to provide image sensor 510
`a wider FOV with respect to image sensor 512. In FIG. 5b, the
`beam splitter configuration includes wide angle lens 572, to
`provide image sensor 510 a wide FOV, and a narrow angle
`lens 574, to provide image sensor 512 a narrow FOV, relative
`to the FOV ofimage sensor 512.
`[0051] Reference is now made to FIG. 3, which is a block
`diagram illustration of zoom control sub-system 300 for an
`image acquisition system, according to some embodiments of
`the present invention. Zoom control sub-system 300 includes
`an image sensor 310 having a lens module 320 with a fixed
`focal length lens or a zoom lens, a zoom control module 330
`and a digital-zoom module 340. An object 20 is captured by
`image sensor 310 through lens module 320. Zoom control
`unit 330 calculates the most optimal values for image sensor
`310, binning/skip factors and continuous digital-zoom values
`that are provided to digital-zoom unit 340. Setting the bin-
`ning/skip factor and windowing of image sensor 310 allows
`to keep a suitable frame refresh rate, while digital-zoom unit
`340 provides continuous zoom.
`[0052] A binning function, which function is optionally
`provided by the sensor array provider, is a zoom out function
`that merges 2x2, or 4x4, or 8x8 pixels pixel array, or any other
`square array of pixels, into a single pixel, whereby reducing
`the image frame dimensions. The binning function may be
`refined by using algorithms such as “bi-linear” interpolation,
`“bi-cubic” interpolation and other commonly used digital
`zoom algorithms. A skip function, which function is option-
`ally provided by the sensor array provider, is a zoom out
`function that allows skipping pixels while reading frame out,
`whereby reducing the image frame dimensions and decrease
`the image acquisition time.
`[0053]
`In variations of the present invention, zoom control
`sub-system 100 of a image acquisition system includes the
`binning/skip function capabilities as in zoom control sub-
`system 300.
`[0054] Reference is also made to FIG. 4, which is a sche-
`matic flow diagram chart that outlines the successive steps of
`an example continuous zoom process 400, according to
`embodiments of the present invention, performed on image
`acquisition system, having a zoom control sub-system such as
`zoom control sub-system 100. Process 400 includes the flow-
`ing steps:
`Step 410: providing a wide image acquisition device and a
`tele image acquisition device.
`[0055] Multiple optical image acquisition devices can be
`used, but for description clarity, with no limitation, the
`method will be described in terms of two image acqui-
`sition devices: wide image acquisition device and a tele
`image acquisition device.
`[0056] Both image acquisition devices (110 and 112)
`include an image sensor array coupled with a lens (120
`and 122, respectively), providing a fixed FOV (tele FOV
`140 and wide FOV 142, respectively). Preferably, wide
`FOV 142 is substantially wider than narrow FOV 140.
`APPL—1005 / Page 11 of 13
`
`APPL-1005 / Page 11 of 13
`
`

`

`US 2012/0026366 A1
`
`Feb. 2, 2012
`
`[0057] The image acquisition devices are adjacently dis-
`posed, such that at least a portion of the environment,
`viewed from within narrow FOV 140 of the tele image
`acquisition device 110, overlaps
`the environment
`viewed from within the wide FOV 142 of wide image
`acquisition device 112.
`Step 420: determining alignment offsets.
`[0058] Before using zoom control sub-system 100, an
`electronically calibrating is performed to determine the
`alignment offsets between wide image sensor array 110
`and tele image sensor array 112. Typically, since the
`spatial offsets between wide image sensor array 110 and
`tele image sensor array 112 are fixed, the electronic
`calibration step is performed one time, after the manu-
`facturing of the image acquisition system and before the
`first use. The electronic calibration yields an X-coordi-
`nate offset, aY-coordinate offset and optionally, a Z-co-
`ordinate rotational offset of the correlation between
`
`wide image sensor array 110 and tele image sensor array
`1 12. Preferably, all three aforementioned coordinate off-
`set values are computed in sub-pixel accuracy. It should
`be noted that for image acquisition systems with more
`than two image sensors, the electronic calibration step is
`performed on each pair of adjacently disposed image
`sensor arrays.
`Step 430: zoom selection.
`[0059] A user of the image acquisition selects the
`required zoom.
`Step 435: bin/skip function selection.
`[0060] The zoom control 130 selects the bin/skip func-
`tion, typically provided by the image sensor provider,
`bringing the combination of the optical zoom and the
`binning/skip magnification selection, to a zoom value
`most proximal to the requested zoom.
`Step 440: selecting an image acquisition device.
`[0061] The zoom control 130 selects an image acquisi-
`tion device, bringing the combination of the optical
`zoom and the binning/skip magnification selection, to a
`zoom value most proximal to the requested zoom.
`Step 450: acquiring an image frame.
`[0062] An image frame is acquired by the selected image
`acquisition device.
`Step 460: performing electronic zoom on the acquired image
`frame to meet the requested zoom.
`[0063] The zoom control 130 computes the zoom factor
`between the fixed zoom of the selected image acquisi-
`tion device, combined with the selected by bin/skip fac-
`tor, and the requested zoom. Based on the computed
`factor, zoom control 130 performs electronic zoom on
`the acquired image frame to meet the requested zoom.
`[0064] Reference is now made to FIG. 6, which is a block
`diagram illustration of a camera system 600, according to
`embodiments of the present invention,
`including a color
`image sensor 612 having wide FOV 642 and a monochrome
`image sensor 610 having narrow FOV 640. The angle ofview
`of wide FOV 142 is typically wider than the angle of view of
`narrow FOV 140. In some variations of the present invention,
`the angle of view of wide FOV 142 is substantially equal to
`the angle of view of narrow FOV 140.
`invention
`[0065] A principal
`intention of the present
`includes providing a camera system 600 and a method of use
`thereof, wherein the output image frame 650 has the resolu-
`tion of image sensor 610, having narrow FOV 640, and the
`color of image sensor 612, having wide FOV 642.
`
`[0066] Reference is now made to FIG. 7, which is a block
`diagram illustration of another zoom control sub-system 700
`for a color image acquisition system, according to variations
`of the present invention. A colored image frame 632 is
`acquired by wide image sensor array 612, and a monochrome
`image frame 630 is acquired by narrow image sensor array
`610. When image sensor selector 750 closes contact 752,
`monochrome image sensor 610 is bypassed and only color
`image sensor 612 having is in operation.
`[0067] When image sensor selector 750 closes contact 754,
`both monochrome image sensor 610 and color image sensor
`612 are in operation, whereas image frames are acquired by
`monochrome image sensor 610 and color of image sensor
`612, synchronously. Fusion module 660 extracts the color
`information from color image frame 632 and fuses the
`extracted color information with monochrome image frame
`630 to form a high resolution, colored image frame 650. The
`fusion includes computing color values for the high resolu-
`tion pixels of monochrome image frame 630 from the respec-
`tive low resolution color image frame 632. Preferably, the
`computation and alignment of the color values is performed
`in sub pixel accuracy.
`[0068]
`In some variations of the present invention, the out-
`put colored image frame 650 is provided with RGB informa-
`tion. In other variations of the present invention, fusion mod-
`ule 760 transmits
`the Y information, obtained from
`monochrome image sensor 610 covered with color (Cr, Cb)
`information obtained from color image sensor 612. The color
`information obtained from color image sensor 612 via a color
`space. Then, fusion module 760 merges the Y information,
`obtained from monochrome image sensor 610, and the color
`(Cr, Cb) information. Then, color space conversion module
`770 converts the image back to an RGB color space, creating
`colored output image frame 650. Optionally, the (Y, Cr, Cb)
`image information is transmitted in separate channels to an
`image receiving unit, bypassing color space conversion mod-
`ule 770.
`
`[0069] The invention being thus described in terms of
`embodiments and examples, it will be obvious that the same
`may be varied in many ways. Such variations are not to be
`regarded as a departure from the spirit and scope of the
`invention, and all such modifications as would be obvious to
`one skilled in the art are intended to be included within the
`
`scope of the claims.
`1. In a computerized image acquisition system, having
`multiple optical image acquisition devices each with a fixed
`field ofview (FOV), a method for continuous electronic zoom
`comprising the steps of:
`a) providing a first image acquisition device including:
`i) a first image sensor array coupled with a first lens
`having a first FOV; and
`ii) a first electronic zoom;
`b) providing a second image acquisition device including:
`i) a second image sensor array coupled with a second
`lens having a second FOV; and
`ii) a second electronic zoom;
`wherein at least a portion of the environment, viewed from
`within said second FOV of said second image acquisi-
`tion device, overlaps the environment viewed from
`within said first FOV of said first image acquisition
`device;
`c) electronically calibrating the alignment between said
`first image sensor array and said second image sensor
`array, whereby determining an X-coordinate offset and a
`APPL—1005 / Page 12 of 13
`
`APPL-1005 / Page 12 of 13
`
`

`

`US 2012/0026366 A1
`
`Feb. 2, 2012
`
`Y—coordinate offset of the correlation between said first
`
`image sensor array and said second image sensor array;
`d) providing a user of the image acquisition device with a
`zoom selecting control, thereby obtaining a requested
`zoom;
`e) selecting one of said image acquisition devices based on
`said requested zoom;
`f) acquiring an image frame with said selected image
`acquisition device, thereby obtaining an acquired image
`frame; and
`g) performing digitally zoom on said acquired image
`frame, thereby obtaining an acquired image frame with
`said requested zoom,
`wherein said calibrating of said alignment between said first
`image sensor array and said second image sensor array, facili-
`tates continuous electronic zoom with uninterrupted imaging,
`when switching back and forth between said first image sen-
`sor array and said second image sensor array.
`2. The method as in claim 1, wherein the computerized
`image acquisition system is configured to provide zooming
`functions selected from the group consisting of a bin function
`and a skip function; wherein said selecting of said image
`acquisition device includes selecting the parameters of said
`bin and/or skip functions; and wherein said method further
`includes the step of applying said selected bin/skip functions,
`with said selected parameters, to said acquired image frame,
`before said performing of said digital zoom step.
`3. The method as in claim 1, wherein said image sensor
`arrays are focused to the infinite.
`4. The method as in claim 1, wherein a lens, selected from
`the group consisting of said first lens and said second lens, is
`a