PROVISIONAL PATENT APPLICATION
`
`DUAL APERTURE ZOOM DIGITAL CAMERA
`
`FIELD
`
`Embodiments disclosed herein relate in general to digital cameras and in particular to
`
`thin zoom digital cameras with both still image and video capabilities
`
`10
`
`BACKGROUND
`
`15
`
`20
`
`25
`
`30
`
`Digital camera modules are currently being incorporated into a variety of host devices.
`
`Such host devices include cellular telephones, personal data assistants (PDAs), computers,
`
`and so forth. Consumer demand for digital camera modules in host devices continues to grow.
`
`Host device manufacturers prefer digital camera modules to be small, so that they can
`
`be incorporated into the host device without increasing its overall size. Further, there is an
`
`increasing demand for cameras in host devices to have higher-performance characteristics.
`
`One such characteristic that many higher-performance cameras (e.g., standalone digital still
`
`cameras) have is the ability to vary the focal length of the camera to increase and decrease the
`
`magnification of the image, typically accomplished with a zoom lens, now known as optical
`
`zooming. Optical zooming is typically accomplished by mechanically moving lens elements
`
`relative to each other. Such zoom lens are typically more expensive, larger, and less reliable
`
`than fixed focal length lenses. An alternative approach for approximating the zoom effect is
`
`achieved with what is known as digital zooming. With digital zooming, instead of varying the
`
`focal length of the lens, a processor in the camera crops the image and interpolates between
`
`the pixels of the captured image to create a "magnified" but lower-resolution image.
`
`Attempts to use two different lenses to approximate the effect of a zoom lens are
`
`known, see e.g. US Patent Publications No. 2008/0030592 and 2010/0277619.
`
`In US
`
`2008/0030592,
`
`the two sensors are operated simultaneously to capture an image. The
`
`respective lenses have different focal lengths, so even though each lens/sensor combination is
`
`aligned to look in the same direction, each will capture an image of the same subject but with
`
`two different fields of view (FOVs). One sensor is commonly called “Wide” and the other
`
`“Tele”. Each sensor provides a separate image, referred to respectively as “Wide” (or “W”)
`
`and “Tele” (or “T”) images. The W-image reflects a wider FOV and has lower resolution than
`
`1
`
`Ex. 3002
`
`Ex. 3002
`IPR2020-00488
`
`IPR2020—00488
`
`

`

`the T-image. The images are then stitched (fused) together to form a composite image. In the
`
`composite image, the central portion is formed by the relatively higher-resolution image taken
`
`by the lens/sensor combination with the longer focal length, and the peripheral portion is
`
`formed by a peripheral portion of the relatively lower-resolution image taken by the
`
`lens/sensor combination with the shorter focal length. The user selects a desired amount of
`
`zoom and the composite image is used to interpolate values from the chosen amount of zoom
`
`to provide a respective zoom image. The solution offered by US 2008/0030592 requires, in
`
`video mode, very large processing resources in addition to high frame rate requirements, high
`
`power consumption (since both cameras are fully operational). It does not provide reference
`
`to disparity range and does not provide reference of how to avoid registration errors.
`
`US 2010/0277619 teaches a camera with a pair of lens/sensor combinations, the two
`
`lenses having different focal lengths, so that the image from one of the combinations has a
`
`field of view approximately 2-3 times greater than the image from the other combination. As
`
`a user of the camera requests a given amount of zoom, the zoomed image is provided from the
`
`lens/sensor combination having the field of view that is next larger than the requested field of
`
`view. Thus, if the requested field of view is less than the smaller field of view combination,
`
`the zoomed image will be created from the image captured by that combination, using
`
`cropping and interpolation if necessary. Similarly, if the requested field of view is greater
`
`than the smaller field of view combination, the zoomed image will be created from the image
`
`captured by the other combination, using cropping and interpolation if necessary. The solution
`
`offered by US 2010/0277619 leads to parallax artifacts when moving to the Tele camera in
`
`video mode. It does not provide reference to disparity range and its heavy computational
`
`requirements and does not provide reference of how to avoid registration errors.
`
`In both US 2008/0030592 and US 2010/0277619, different focal length systems cause
`
`Tele and Wide matching FOVs to be exposed at different times using CMOS sensors, which
`
`degrades the overall image quality. Different optical F numbers (”F#”) cause image intensity
`
`differences. Working with such a dual sensor system requires double bandwidth support, i.e.
`
`additional wires from the sensors to the following HW component. Neither US 2008/0030592
`
`nor US 2010/0277619 deal with false registration.
`
`Therefore there is a need for, and it would be advantageous to have thin digital
`
`cameras with optical zoom operating in both video and still mode that do not suffer from the
`
`commonly encountered problems and disadvantages, some of which are listed above.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`

`

`SUMMARY
`
`Embodiments disclosed herein teach the use of dual-aperture (also referred to as dual-
`
`lens or two-sensor) optical zoom digital cameras. The cameras include two camera “subsets”,
`
`a Wide subset and a Tele subset, each subset including a lens, an image sensor and an image
`
`signal processor (ISP). The Tele subset is the higher zoom subset and the Wide subset is the
`
`lower zoom subset. In some embodiments, the lenses are thin lenses with short optical paths
`
`of less than about 9mm. The thickness/effective focal length (EFL) ratio of the Tele lens is
`
`smaller than 0.9. Such a camera includes two sensors or a single sensor divided into at least
`
`two areas. Hereinafter, the description refers to “two sensors”, with the understanding that
`
`they may be sections of a single physical sensor (imager chip). The camera may be operated
`
`in both still and video modes. In still mode, zoom is achieved by fusing W and T images, with
`
`the resulting “fused” image including always information from both images. In video mode,
`
`full and smooth optical zoom is achieved by switching between the W and T images. To
`
`avoid discontinuities in video mode, the switching includes applying additional processing
`
`blocks.
`
`In order to reach optical zoom capabilities, a different magnification image of the
`
`same scene is captured by each camera subset, resulting in FOV overlap between the two
`
`subsets processing is applied on the two images grabbed by the multi-aperture imaging
`
`system to fuse and output one image processed according to a user zoom factor request. As
`
`part of the fusion procedure, up-sampling may be applied on one or both of the images to
`
`scale it to the image grabbed by the Tele subset or to a scale defined by the user. The fusion
`
`or up-sampling may be applied to only some of the pixels of a sensor. Down-sampling can be
`
`performed as well if the output resolution is smaller than the sensor resolution.
`
`The cameras and associated methods disclosed herein address and correct many of the
`
`problems and disadvantages of known dual aperture optical zoom digital cameras. They
`
`provide an overall zoom solution that refers to all aspects: optics, algorithmic processing and
`
`system hardware (HW). The proposed solution distinguishes between video and capture mode
`
`in the processing flow, and specifies the optical requirements and HW requirements.
`
`In
`
`addition, it provides an innovative optical design which enables low TTL focal length ratio
`
`using a specific lens curvature order.
`
`In addition to enabling optical zoom, a dual-aperture zoom system described herein can
`
`also enable a unique photography feature which is typically not available in compact camera
`
`modules currently available in cell-phones. This feature is very shallow depth of focus (DOF),
`
`10
`
`15
`
`20
`
`25
`
`30
`
`

`

`particularly suited for portrait photos. The DOF of a lens decreases as the focal
`
`length
`
`increases.
`
`When portrait photos are captured with large cameras (such as DSLRs), usually a 50-
`
`80mm lens is used, coupled with a wide-open aperture setting to create a shallow DOF effect.
`
`Due to the large focal length, objects that are in front or behind the plane of focus appear very
`
`blurry, and a nice foreground-to-background contrast is achieved. However, it is difficult to
`
`create such a blur using a compact camera with a relatively short focal length and small
`
`aperture size, such as a cell-phone camera. In some embodiments, a dual-aperture zoom
`
`system disclosed herein can be used to capture a shallow-DOF photo by taking advantage of
`
`the longer focal length of the Tele lens. The reduced DOF effect provided by this relatively
`
`long focal length can be further enhanced in the final image by fusing data from an image
`
`captured with the Wide-lens at the same time. Depending on the object distance, while the
`
`Tele lens is focused on the subject of the photo, the Wide lens can be focused to a very near
`
`distance, so that objects behind the subject appear very blurry. Once the two images are
`
`captured, information from the out-of—focus blurred background in the Wide image is fused
`
`with the original Tele image background information, to result in a blurrier background and
`
`even shallower DOF.
`
`In an embodiment there is provided a dual-aperture zoom digital camera comprising a
`
`Wide imaging section operative to provide a Wide image of an object, the Wide imaging
`
`section including a lens with a wide FOV, a Wide sensor and a Wide image signal processor
`
`(ISP), a Tele imaging section operative to provide a Tele image of the object,
`
`the Tele
`
`imaging section including a lens with a narrow FOV, a Tele sensor and a Tele ISP, and a
`
`camera controller coupled to the Wide and Tele imaging sections and operative to process
`
`Wide and Tele sensor data to obtain a fused image of the object in a capture or stills mode,
`
`and to process without fusion Wide and Tele sensor data to obtain, in a video mode, images
`
`with an output resolution and a smooth transition when switching between a lower zoom
`
`factor (ZF) value and a higher ZF value or vice versa, wherein at the lower ZF the output
`
`resolution is determined by the Wide sensor and wherein at the higher ZF value the output
`
`resolution is determined by the Tele sensor, whereby the processing provides a continuous
`ZOOIII.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`

`

`In an embodiment there is provided a method for providing continuous optical zoom in
`
`a digital camera that includes a Wide imaging section including a lens with a wide FOV, a
`
`Wide sensor and a Wide ISP and a Tele imaging section including a lens with a narrow FOV,
`
`a Tele sensor and a Tele ISP, the method comprising the steps of:
`
`imaging an object to obtain
`
`Wide sensor data and Tele sensor data, processing with fusion the Wide and Tele sensor data
`
`to obtain a fused image of the object in a capture or stills mode, and processing without fusion
`
`the Wide and Tele sensor data to obtain, in a video mode, images with an output resolution
`
`and a smooth transition when switching between a lower zoom factor (ZF) value and a higher
`
`ZF value or vice versa, wherein at the lower ZF the output resolution is determined by the
`
`Wide sensor and wherein at the higher ZF value the output resolution is determined by the
`
`Tele sensor.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Non-limiting examples of embodiments disclosed herein are described below with
`
`reference to figures attached hereto that are listed following this paragraph.
`
`Identical
`
`structures, elements or parts that appear in more than one figure are generally labeled with a
`
`same numeral in all the figures in which they appear. The drawings and descriptions are
`
`meant to illuminate and clarify embodiments disclosed herein, and should not be considered
`
`limiting in any way.
`
`FIG. 1 shows schematically a block diagram illustrating a dual-aperture zoom imaging
`
`system disclosed herein;
`
`FIG. 2 shows an example of Wide sensor, Tele sensor and their respective FOVs;
`
`FIG. 3 shows a schematically embodiment of CMOS sensor image grabbing vs. time;
`
`FIG. 4 shows schematically a sensor time configuration which enables sharing one
`
`sensor interface using dual sensor zoom system;
`
`FIG. 5 shows an embodiment of a method disclosed herein for acquiring a zoom
`
`image in capture mode;
`
`FIG. 6 shows an embodiment of a method disclosed herein for acquiring a zoom
`
`image in video/preview mode;
`
`FIG. 7 shows a graph illustrating an effective resolution zoom factor;
`
`FIG. 8 shows one embodiment of a lens block in a thin camera disclosed herein;
`
`FIG. 9 shows another embodiment of a lens block in a thin camera disclosed herein.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`

`

`DETAILED DESCRIPTION
`
`FIG.
`
`1 shows schematically a block diagram illustrating an embodiment of a dual-
`
`aperture zoom imaging system (also referred to simply as “camera”) disclosed herein and
`
`numbered 100. Camera 100 comprises a Wide imaging section (“subset”) that includes a
`
`Wide lens block 102, a Wide image sensor 104 and a Wide image processor 106. Camera 100
`
`further comprises a Tele imaging section (“subset”) that includes a Tele lens block 108, a Tele
`
`image sensor 110 and a Tele image processor 112. The image sensors may be physically
`
`separate or may be part of a single larger image sensor. The Wide sensor pixel size can be
`
`equal to or different from the Tele sensor pixel size. Camera 100 further comprises a camera
`
`fusion processing core (also referred to as “controller”) 114 that includes a sensor control
`
`module 116, a user control module 118, a Video processing module 126 and a capture
`
`processing module 128, all operationally coupled to sensor control block 110. User control
`
`module 118 comprises an operational mode function 120, a region of interest (ROI) function
`
`122 and a zoom factor (ZF) function 124.
`
`Sensor control module 116 is connected to the two sub-cameras and to the user control
`
`module 118 and used to choose, according to the zoom factor, which of the sensors is
`
`operational and control
`
`the exposure mechanism and the sensor readout. Since zoom is
`
`achieved by sensor oversampling, for most zoom factors, only one sensor is operational in
`
`Video mode. This is also true for the Auto Focus (AF) mechanism. Mode choice function 120
`
`is used for choosing capture/Video modes. The capture mode may include a “burst mode”.
`
`ROI function 122 is used to choose region of interest. Zoom factor function 124 is used to
`
`choose a zoom factor. Video processing module 126 is connected to the mode choice function
`
`120 and used for high frame rate Video processing. Capture processing module 128 is
`
`connected to the mode choice function 120 and used for high image quality still mode images.
`
`The Video processing module is applied when the user desires to shoot in Video mode.
`
`The capture processing module is applied when the user wishes to shoot still pictures.
`
`Following is a detailed description and examples of different methods of use of camera 100.
`
`Design for continuous Zoom in Video Mode
`
`In order to reach high quality optical continuous zooming in Video mode while
`
`reaching real optical zoom, optical system design is taken into account as follows:
`
`10
`
`15
`
`20
`
`25
`
`30
`
`

`

`Tan (FOVWIDE)/Tan (FOVTELE) = PLWIDF/ PLvideo
`
`(1)
`
`"Tan" refers to “tangent”, while FOVWIDE and FOVTELE refer respectively to Wide and Tele
`
`lens fields of view (in degrees). The FOV is measured from the center axis to the corner of the
`
`sensor. PLWIDE and PLvideo refer respectively to the “in-line” (i.e. in a line) number of sensor
`
`pixels and in-line number of output video format pixels. For example, in order to get full
`
`optical zoom continuous experience with a l2Mp sensor (sensor dimensions: 4000x3000) and
`
`a required 1080p (dimension: 1920x1080) video format,
`
`the FOV ratio should be
`
`4000/l920=2.083. Moreover,
`
`if the Wide lens FOV is given as FOVWIDE = 37.5
`
`,
`
`the
`
`required Tele lens FOV is 20.2 . The zoom switching point is set according to the ratio
`
`between sensor pixels in-line and the number of pixels in-line in the video format and defined
`
`as:
`
`stitch=PLWIDF/ PLvideo
`
`Maximum optical zoom is reached according to the following formula:
`
`Zmax= Tan (FOVWIJDE)/Tan (FOVTELE)-* PLTELE/ PLvideo
`
`(2)
`
`(3)
`
`For example: for the configuration defined above and assuming PLTele=4000 and PLvidw:
`
`1920, Zmax=4.35.
`
`In an embodiment, the sensor control module has a setting that depends on the Wide
`
`and Tele FOVs and on a sensor oversampling ratio, the setting used in the configuration of
`
`each sensor. In an embodiment, the Wide and Tele FOVs and the oversampling ratio satisfy
`
`the condition 0.8*PLWIDF/ PLVid60 < Tan (FOVWIDE)/Tan (FOVTELE) <l.2*PLw1DE/ PLvideo.
`
`“Oversampling ratio” is defined as the ratio between the number of pixels in the sensor vs the
`
`number of pixels in the output image. For example, when using a 4000x3000 sensor and when
`
`outputting a 1920x1080 image, the oversampling ratio is 4000/l920=2.0833
`
`Still camera mode operation/function
`
`In still camera mode, the obtained image is fused from information obtained by both
`
`sub-cameras at all zoom levels, see FIG. 2, which shows a first (”Wide") sensor 202 and a
`
`second ("Tele") sensor 204 and their respective FOVs. Exemplarily, as shown, the Tele sensor
`
`FOV is half the Wide sensor FOV. The still camera mode processing includes two stages: (1)
`
`setting HW settings and configuration, where a first objective is to control the sensors in such
`
`a way that matching FOVs in both images (Tele and Wide) are scanned at the same time. A
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`

`

`second objective is to control the relative exposures according to the lens properties. A third
`
`objective is to minimize the required bandwidth from both sensors for the ISPs; and (2) image
`
`processing that fuses the Wide and the Tele images to achieve optical zoom, improves SNR
`
`and provides wide dynamic range.
`
`FIG. 3 shows image line numbers vs. time for an image section captured by CMOS
`
`sensors. A fused image is obtained by line (row) scans of each image. To prevent matching
`
`FOVs in both sensors to be scanned at different time, a special configuration is applied on
`
`both image sensors while keeping the same frame rate. The difference in FOV between the
`
`sensors determines the relationship between the rolling shutter time and the vertical blanking
`
`time for each sensor. The scanning is synchronized such that the same points of the object in
`
`each view are obtained simultaneously.
`
`Specifically with reference to FIG. 3 and according to an embodiment of a method
`
`disclosed herein, the fused image acquisition process includes setting the Tele sensor vertical
`
`blanking time VBTele to equal the Wide sensor vertical blanking time VBWide plus half the
`
`Wide sensor rolling shutter time RSTWide, setting the Tele and Wide sensor exposure times
`
`ETTele and ETWide to be equal or different, setting the Tele sensor rolling shutter time RSTtele
`
`to be half the Wide sensor rolling shutter time RSTWide and setting the frame rates of the two
`
`sensors to be equal. This procedure results in identical image pixels in the Tele and Wide
`
`sensor images being exposed at the same time.
`
`The exposure times applied to the two sensors could be different, for example, in order
`
`to reach same image intensity using different F# and different pixel size for the Tele and Wide
`
`systems.
`
`In this case,
`
`the relative exposure time should be configured according to the
`
`formula below:
`
`ETTele= ETWide'(F#Tele/F#Wide)2 ‘ (PIXCl SlZCTele/PIXCI SIZCWide)2
`
`(4)
`
`Other exposure time ratios may be applied to achieve wide dynamic range and improved
`
`SNR. Fusing two images with different intensities will result in wide dynamic range image.
`
`In more detail with reference to FIG. 3, in the first stage, after the user chooses a
`
`required zoom factor ZF, the sensor control module configures each sensor as follows:
`
`1)
`
`Cropping index Wide sensor:
`
`YWMe Sm: 1/2.PCWide(l- l/ZF)
`
`YWide end: 1/2‘PCWide(1+1/ZF)
`
`where PC is the number of pixels in a column, and Y is the row number
`
`10
`
`15
`
`20
`
`25
`
`30
`
`

`

`2)
`
`Cropping index Tele sensor:
`
`If ZF> Tan (FOVWide)/Tan (FOVTele), then
`
`YTele Sm: l/2~PCTele(l-(l/ZF) ~ Tan (FOVTele)/Tan (FOVWide))
`
`YTele end: 1/2'PCTele(1+(1/ZF) ‘ Tan (FOVTele)/Tan (FOVWide))
`
`If ZF < Tan (FOVWide)/Tan (FOVTele), then
`
`Y Tele staItO
`
`Y Tele end: PCTCIC
`
`10
`
`15
`
`20
`
`25
`
`30
`
`This will result in start exposing the Tele sensor with a delay of:
`
`(l-ZF/((Tan (FOVWide)/Tan (FOVTele))). l/(2'PFS)
`
`(5)
`
`where FPS is the sensor's frame per second configuration.
`
`In cases where ZF> Tan
`
`(FOVWide)/Tan (FOVTele), no delay will be introduced between Tele and Wide exposure
`
`starting point. For example, for a case where Tan (FOVWide)/Tan (FOVTele)=2 and ZF=l, the
`
`Tele image first pixel
`
`is exposed l/4~(l/PFS) sec after the Wide image first pixel was
`
`exposed.
`
`After applying the cropping according to the required zoom factor, the sensor rolling
`
`shutter time and the vertical blank should be configured in order to satisfy the equation to
`
`keep the same frame rate
`
`VBWide+RSTWide= VBTele+RSTTele
`
`(6)
`
`FIG. 3 exemplifies Eq. (6), One way to satisfy Eq. (6) is to increase the RSTWide.
`
`Controlling the RSTWide may be done by changing the horizontal blanking (HB) of the Wide
`
`sensor. This will cause a delay between the data coming out from each row of the Wide
`sensor.
`
`Generally, working with a dual-sensor system requires multiplying the bandwidth to
`
`the following block, for example the ISP. For example, using 12Mp working at 30fps, 10bit
`
`per pixel requires working at 3.6Gbit/sec. In this example, supporting this bandwidth requires
`
`4 lanes from each sensor to the respective following ISP in the processing chain. Therefore,
`
`working with two sensors requires double bandwidth (7.2Gbit/sec) and 8 lanes connected to
`
`the respective following blocks. The bandwidth can be reduced by configuring and
`
`

`

`synchronizing the two sensors. Consequently,
`
`the number of lanes can be half that of a
`
`conventional configuration (3.6Gbit/sec).
`
`FIG. 4 shows schematically a sensor time configuration that enables sharing one
`
`sensor interface using a dual-sensor zoom system. For simplicity, assuming the Tele sensor
`
`image is magnified by a factor of 2 compared with the Wide sensor image, the Wide sensor
`
`horizontal blanking time HBWide is set to twice the Wide sensor line readout time. This causes
`
`a delay between output Wide lines. This delay time matches exactly the time needed to output
`
`two lines from the Tele sensor. After outputting two lines from the Tele sensor, the Tele
`
`sensor horizontal blanking time HBTele is set to be one Wide line readout time, so, while the
`
`Wide sensor outputs a row from the sensor, no data is being output from the Tele sensor. For
`
`this example, every 3rd line in the Tele sensor is delayed by an additional HBTele, In this delay
`
`time, one line from the Wide sensor is output from the dual-sensor system. After the sensor
`
`configuration stage, the data is sent in parallel or by using multiplexing into the processing
`
`section.
`
`FIG. 5 shows an embodiment of a method disclosed herein for acquiring a zoom
`
`image in capture mode. In ISP step 502, the data of each sensor is transferred to the respective
`
`ISP component, which performs on the data various processes
`
`such as denoising,
`
`demosaicing, sharpening, scaling, etc, as known in the art. After the processing in step 502,
`
`all following actions are performed in capture processing core 128: in rectification step 504,
`
`both Wide and Tele images are aligned to be on the epipolar line; in registration step 506,
`
`mapping between the Wide and the Tele aligned images is performed to produce a registration
`
`map; in resampling step 508, the Tele image is resampled according to the registration map,
`
`resulting in a re-sampled Tele image; in decision step 510, the re-sampled Tele image and the
`
`Wide image are processed to detect errors in the registration and to provide a decision output;
`
`in fusion step 512, the decision output, re-sampled Tele image and the Wide image are fused
`
`into a single zoom image.
`
`To reduce processing time and power, steps 506, 508, 510, 512 could be bypassed by
`
`not fusing the images in non-focused areas. In this case, all steps specified above should be
`
`applied on focused areas only. Since the Tele optical system will introduce shallower depth of
`
`field than the Wide optical system, defocused areas will suffer from lower contrast in the Tele
`
`system.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`10
`
`

`

`Video mode operation/function
`
`In this mode, sensor oversampling is used to enable continuous zoom experience in
`
`video mode. Processing is applied to eliminate the changes in the image during crossover
`
`from one sub-camera to the other. Zoom from 1 to stitch is performed using the Wide sensor
`
`only. From stitch and on, it is performed mainly by the Tele sensor. To prevent “jumps”
`
`(roughness in the image), switching to the Tele image is done using a zoom factor which is a
`
`bit higher (stitch +AZoom) than stitch. AZoom is determined according to the system's
`
`properties and is different for cases where zoom-in is applied and cases where zoom-out is
`
`applied (AZoomian AZoomout). This is done to prevent residual jumps artifacts to be visible at
`
`a certain zoom factor .The switching between sensors, for an increasing zoom and for
`
`decreasing zoom, is done on a different zoom factor.
`
`The zoom video mode operation includes two stages:
`
`(1)
`
`sensor control and
`
`configuration, and (2) image processing.
`
`In the range from 1 to stitch, only the Wide sensor
`
`is operational, hence, power can be supplied only to this sensor. Similar conditions hold for a
`
`Wide AF mechanism. From stitch+AZoom to Zmax only the Tele sensor is operational, hence,
`
`power is supplied only to this sensor. Similarly, only the Tele sensor is operational and power
`
`is supplied only to it for a Tele AF mechanism. Another option is that the Tele sensor is
`
`operational and the Wide sensor is working in low frame rate. From stitch to stitch+AZoom,
`
`both sensors are operational.
`
`FIG. 6 shows an embodiment of a method disclosed herein for acquiring a zoom
`
`image in video/preview mode for 3 different zoom factor (ZF) ranges: (a) ZF range = l
`
`:
`
`stitch; (b) ZF range = stitch : stitch + AZoomin: and (c) Zoom factor range = stitch +
`
`AZoomin : Zmax. The description is with reference to a graph of effective resolution vs. zoom
`
`value (FIG. 7). In step 602, sensor control module 116 chooses (directs) the sensor (Wide,
`
`Tele or both) to be operational. Specifically, if the ZF range = 12mm, module 116 directs the
`
`Wide sensor to be operational and the Tele sensor to be non-operational. If the ZF range is
`
`stitch : stitch + AZoomin, module 116 directs both sensors to be operational and the zoom
`
`image is generated from the Wide sensor. If the ZF range is stitch + AZoomin : Zmax, module
`
`116 directs the Wide sensor to be non-operational and the Tele sensor to be operational. After
`
`the sensor choice in step 602, all following actions are performed in video processing core
`
`126. Optionally, in step 604, color balance is calculated if two images are provided by the
`
`two sensors. Optionally yet, in step 606, the calculated color balance is applied in one of the
`
`images (depending on the zoom factor). Further optionally,
`
`in step 608, registration is
`
`10
`
`15
`
`20
`
`25
`
`30
`
`11
`
`

`

`performed between the Wide and Tele images to output a transformation coefficient. The
`
`transformation coefficient can be used to set an AF position in step 610. In step 612, an output
`
`of any of steps 602-608 is applied on one of the images (depending on the zoom factor) for
`
`image signal processing that may include denoising, demosaicing, sharpening, scaling, etc. In
`
`step 614, the processed image is resampled according to the transformation coefficient, the
`
`requested ZF (obtained from zoom function 124) and the output video resolution (for example
`
`1080p). To avoid a transition point to be executed at the same ZF, AZoom can change while
`
`zooming in and while zooming out. This will result in hysteresis in the sensor switching point.
`
`10
`
`15
`
`In more detail, for ZF range 1
`
`: stitch, for ZF < stitch, the Wide image data is
`
`transferred to the ISP in step 612 and resampled in step 614. For ZF range = stitch : stitch +
`
`AZoomin, both sensors are operational
`
`and the zoom image is generated from the Wide
`
`sensor. The color balance is calculated for both images according to a given ROI. In addition,
`
`for a given ROI, registration is performed between the Wide and Tele images to output a
`
`transformation coefficient. The transformation coefficient is used to set an AF position. The
`
`transformation coefficient includes the translation between matching points in the two images.
`
`This translation can be measured in a number of pixels. Different translations will result in a
`
`different number of pixel movements between matching points in the images. This movement
`
`can be translated into depth and the depth can be translated into an AF position. This enables
`
`to set the AF position by only analyzing two images (Wide & Tele). The result is fast
`
`20
`
`focusing.
`
`25
`
`30
`
`Both color balance ratios and transformation coefficient are used in the ISP step.
`
`In
`
`parallel, the Wide image is processed to provide a processed image, followed by resampling.
`
`For ZF range = stitch + AZoomin : Zmax and for Zoom factor > stitch, + AZoomin, the color
`
`balance calculated previously is now applied on the Tele image. The Tele image data is
`
`transferred to the ISP in step 612 and resampled in step 614. To eliminate crossover artifacts
`
`and to enable smooth transition to the Tele image, the processed Tele image is resampled
`
`according to the transformation coefficient, the requested ZF (obtained from zoom function
`
`124) and the output video resolution (for example 1080p).
`
`FIG. 7 shows the effective resolution as a function of the zoom factor for a zoom-in
`
`case and for a a zoom-out case AZoomup is set when we zoom in, and AZoomdown is set when
`
`we zoom out. Setting AZoomup to be different from AZoomdown will result in transition
`
`between the sensors to be performed at different zoom factor (“hysteresis”) when zoom-in is
`
`used and when zoom-out is used. This hysteresis phenomenon in the video mode results in
`
`smooth continuous zoom experience.
`
`12
`
`

`

`Optical Design
`
`Additional optical design considerations were taken into account to enable reaching
`
`optical zoom resolution using small total track length (TTL). These considerations refer to the
`
`Tele lens.
`
`In an embodiment, the camera is “thin” in the sense that is has an optical path of
`
`less than 9mm and a thickness/focal length (FP) ratio smaller than about 0.85. Exemplarily, as
`
`shown in FIG. 8, such a thin camera has a lens block that includes (along an optical axis
`
`starting from an object) five lenses: a first lens element 802 with positive power and two
`
`lenses 804 and 806 and with negative power, a fourth lens 808 with positive power and a fifth
`
`lens 810 with negative power. In the embodiment of FIG. 8, the effective focal length (EFL)
`
`is 7 mm and the optical total track length (TTL) is 5.8 mm. Thus the Tele lens TTL/EFL ratio
`
`is smaller than 0.9.
`
`In another embodiment of a lens block in a thin camera, shown in FIG. 9, the camera
`
`has a lens block that includes (along an optical axis starting from an object) a first lens
`
`element 902 with positive power a second lens element 904 with negative power, a third lens
`
`element with positive power 906 and a fourth lens element with negative power 908, and a
`
`fifth filed lens element 910 with positive or negative power. Lens data is also shown in FIGS.
`
`8 and 9.
`
`In conclusion, dual aperture optical zoom digital cameras and associate methods
`
`disclosed herein reduce the amount of processing resources, lower frame rate requirements,
`
`reduce power consumption, remove parallax artifacts and provide continuous focus (or
`
`provide loss of focus) when changing from Wide to Tele in video mode. They provide a
`
`dramatic reduction of the disparity range and avoid false registration in capture mode. They
`
`reduce image intensity differences and enable work with a single sensor bandwidth instead of
`
`two, as in known cameras.
`
`While this disclosure has been described in terms of certain embodiments and
`
`generally associated methods, alterations and permutations of the embodiments and methods
`
`will be apparent to those skilled in the art. T