Application Note
`
`MTD/PS-0259
`
`
`
`
`
`
`
`
`Shutter Operations
`for
`CCD and CMOS Image Sensors
`
`
`
`
`
`
`
`
`Image Sensor Solutions
`Eastman Kodak Company
`Rochester, New York 14650-2010
`
`
`
`October 23, 2001
`Revision 1
`
`
`Eastman Kodak Company - Image Sensor Solutions - Rochester, NY 14650-2010
`Phone (716) 722-4385 Fax (716) 477-4947
`Web: www.kodak.com/go/imagers E-mail: imagers@kodak.com
`
`
`
`Ex.1008 / Page 1 of 5Ex.1008 / Page 1 of 5
`
`TESLA, INC.TESLA, INC.
`
`

`

`Application Note
`
`MTD/PS-0259
`
`
`Introduction:
`In digital cameras, an image is captured by converting the light from an object into an electronic signal at the photosensitive
`area (photodiode) of a solid-state CCD or CMOS image sensor. The amount of the signal generated by the image sensor
`depends on the amount of the light that falls on the imager, both in terms of intensity and duration. Therefore, like
`conventional film cameras, digital cameras require a shutter to control exposure for image quality. This is generally done
`either by incorporating an external mechanical shutter in front of the image sensor or by incorporating an on-chip electronic
`shutter.
`
`For digital cameras equipped with a mechanical shutter, the photodiode integration time depends on the duration of the
`opening of the mechanical shutter. Some image sensors provide electronic means for controlling integration time. These
`types of image sensors may allow the elimination of the mechanical shutter.
`
`In this application note three electronic shutter mechanisms are described:
`•= CCD image sensor with electronic shutter;
`•= CMOS image sensor with a conventional rolling shutter;
`•= CMOS image sensor with a Kodak patented global shutter.
`
`
`CCD Imager with Electronic Shutter:
`
`Some types of CCD image sensors provide an electronic shutter mechanism. In a CCD imager with a built in electronic
`shutter, the entire imager is reset before integration to remove any residual signal in the photodiodes. The photodiodes then
`accumulate charge for some period of time. At the end of the integration period, all charges are simultaneously transferred to
`light shielded areas of the sensor. The charges are then shifted out of the light shielded areas of the sensor and read out.
`
`An example of a CCD image sensor with this capability is the so-called interline CCD image sensor in which columns of
`photodiodes are interleaved with light shielded columns - vertical CCDs. The vertical CCDs are used to shift the charge
`vertically out of the imaging area. The light shield prevents further accumulation of charge during the readout process. The
`exposure time is the time between the reset of the imager and the transfer of charge from the photodiodes to the vertical
`CCDs.
`
`Depending upon the design and the pixel size, even an interline CCD may require a shutter; this is the case with many
`contemporary consumer digital cameras.
`
`
`CMOS Imager with Rolling Shutter:
`
`The rolling shutter in a CMOS image sensor works analogously to a focal plane shutter in a film camera (indeed, the rolling
`shutter is sometimes referred to as an electronic focal plane shutter). Typically, the rows of pixels in the image sensor are
`reset in sequence, starting at the top of the image and proceeding row by row to the bottom. When this reset process has
`moved some distance down the image, the readout process begins: rows of pixels are read out in sequence, starting at the top
`of the image and proceeding row by row to the bottom in exactly the same fashion and at the same speed as the reset process.
`
`The time delay between a row being reset and a row being read is the integration time. By varying the amount of time
`between when the reset sweeps past a row and when the readout of the row takes place, the integration time (hence, the
`exposure) can be controlled. In the rolling shutter, the integration time can be varied from a single line (reset followed by
`read in the next line) up to a full frame time (reset reaches the bottom of the image before reading starts at the top) or more.
`The Kodak KAC1310 1.3Mpixel image sensor provides a rolling shutter mode of operation.
`
`
`Eastman Kodak Company - Image Sensor Solutions - Rochester, NY 14650-2010
`Phone (716) 722-4385 Fax (716) 477-4947
`Web: www.kodak.com/go/imagers E-mail: imagers@kodak.com
`
`2
`
`
`Revision No. 1
`
`Ex.1008 / Page 2 of 5Ex.1008 / Page 2 of 5
`
`TESLA, INC.TESLA, INC.
`
`

`

`Application Note
`
`MTD/PS-0259
`
`
`Since the integration process moves through the image over some length of time, some motion artifacts may become
`apparent. For example, if a vehicle is moving through the image during capture, then light from the top of vehicle will be
`integrated at some earlier time than light from the bottom of the vehicle, causing the bottom of the vehicle to appear slanted
`forward in the direction of motion. While this may cause the vehicle to look somewhat sporty, it is not a fully accurate
`representation of the vehicle. Depending on the relationship between moving objects in the image and the rolling shutter,
`various distortions may occur.
`
`
`Note:
`A similar effect may be noted in conventional film cameras that employ a focal plane shutter.
`
`
`
`CMOS Imager with Global Shutter:
`
` A
`
` CMOS image sensor employing the Kodak patented global shutter controls exposure in a similar fashion to an interline
`CCD described above. In the global shutter operation, the entire imager is reset before integration. The pixels are allowed to
`accumulate charge during the integration time. At the end of the integration time, the accumulated charge in each pixel is
`transferred to a light shielded storage area. Then the signals are read out from the light shielded area. Since all the pixels are
`reset at the same time, integrate over the same interval, and are transferred to a light shielded storage area at the same time,
`there is no potential for motion artifacts. The Kodak KAC0311 VGA CMOS imager provides both rolling shutter and global
`shutter modes of operation.
`
`Application of Photoflash in CMOS Sensors:
`
`Similarly to film cameras, photoflash is commonly used in digital cameras to provide sufficient light to expose the image
`properly in a low light environment. The duration of an electronic flash is generally very short (on the order of hundreds of
`microseconds to a millisecond or two), but the readout time for a CMOS image sensor may be much longer. Using a
`photoflash with a CMOS image sensor therefore requires special consideration.
`
`The following cases are described below:
`•= CMOS image sensor (rolling or global electronic shutter) with a mechanical shutter;
`•= CMOS imager with a global shutter;
`•= CMOS imager with a rolling shutter.
`
`Photoflash Used with CMOS Sensor and Mechanical Shutter:
`
`If a mechanical shutter is used in conjunction with a CMOS image sensor, the photoflash operation is as follows:
`1. The integration time of the CMOS imager’s electronic shutter is adjusted so that all the pixels are integrating light at the
`same time;
`2. The mechanical shutter is opened;
`3. The photoflash is fired;
`4. The mechanical shutter is closed;
`5. The imager is read out.
`
`The Kodak KAC1310 image sensor provides a signal, STROBE, which indicates, in some modes of operation, when the reset
`operation has reached the bottom of the image; the signal therefore provides an indication that all the pixels are integrating
`light. This convenient signal may be used to begin the sequence of operations starting with opening the mechanical shutter
`(Step 2 above).
`
`
`
`
`Eastman Kodak Company - Image Sensor Solutions - Rochester, NY 14650-2010
`Phone (716) 722-4385 Fax (716) 477-4947
`Web: www.kodak.com/go/imagers E-mail: imagers@kodak.com
`
`3
`
`
`Revision No. 1
`
`Ex.1008 / Page 3 of 5Ex.1008 / Page 3 of 5
`
`TESLA, INC.TESLA, INC.
`
`

`

`Application Note
`
`MTD/PS-0259
`
`
`Photoflash Used with CMOS Sensor with Global Shutter:
`
` A
`
` CMOS sensor with global shutter can accommodate the photoflash quite easily. In this case, the photoflash operation is as
`follows:
`1. All the pixels of the imager are reset simultaneously;
`2. The photoflash is fired;
`3. The accumulated charges at each pixel are transferred simultaneously to light shielded storage areas;
`4. The light shielded areas of the imager are read out.
`
`The time between step 1 and step 3 needs to be only as long as the duration of the photoflash. As mentioned above, this can
`be a very short time. The resulting exposure will primarily due to the illumination provided by the photoflash, and will
`depend a little on the ambient light illumination.
`
`Photoflash Used with CMOS Sensor with Rolling Shutter:
`
`It is more challenging to operate a photoflash with a CMOS image sensor operating in a rolling shutter mode. Since the
`duration of the photoflash is very short compared to the overall readout time of the imager, the photoflash can only be used
`when all the pixels of the imager are integrating at the same time. To see why this is, consider the case when the time
`between reset and readout is less than a full frame time, say one-quarter the number of lines of the imager. In this case, as the
`integration process moves through the imager area, only those areas of the imager that are integrating during the photoflash
`will be affected by the photoflash. Depending on the timing of the photoflash, this will lead to a properly exposed band in the
`middle of the image (exposed by the photoflash) with underexposed areas above and below (integrating before and after the
`photoflash and hence integrating only available light).
`
`Based on these considerations, the operation of a photoflash with a CMOS imager operating in rolling shutter mode is as
`follows:
`1. The integration time of the imager is adjusted so that all the pixels are integrating simultaneously for the duration of the
`photoflash;
`2. The reset process progresses through the image row by row until the entire imager is reset;
`3. The photoflash is fired;
`4. The imager is read out row by row until the entire imager is read out.
`
`
`Note:
`The Kodak KAC1310’s STROBE signal (mentioned in the mechanical shutter discussion above) may be used here as
`well. In this case, it provides an indication of when to fire the photoflash (Step 3).
`
`
`Since all the pixels of the image sensor must be integrating simultaneously, the amount of time for the reset process to sweep
`completely through the image plus some amount of time for the photoflash, sets the minimum exposure time to use for
`photoflash operation. For a KAC-1310 sensor operated at 10MHz, the minimum exposure time at this mode will be greater
`than 130ms. At this minimum exposure time, there is some ambient light level, Iamb, that will properly expose the image; no
`photoflash is required at this light level. Hence, the ambient light level should be less than this level (Iamb)in order to use
`photoflash.
`
`It is clear from this discussion that the net exposure in this mode will result from integrating both ambient light and the light
`from the photoflash. As previously mentioned, to obtain the best image quality, the ambient light level should probably be
`significantly below the minimum light level at which the photoflash can be used so that the photoflash contributes a
`significant portion of the exposure illumination. Depending on the speed at which the reset and readout processes can take
`place, the minimum exposure time to use with photoflash may be sufficiently long to open the possibility of image blur due
`to camera or subject motion during the exposure. To the extent that the exposure light is provided by the short duration
`photoflash, this blur may be minimized.
`
`
`Eastman Kodak Company - Image Sensor Solutions - Rochester, NY 14650-2010
`Phone (716) 722-4385 Fax (716) 477-4947
`Web: www.kodak.com/go/imagers E-mail: imagers@kodak.com
`
`4
`
`
`Revision No. 1
`
`Ex.1008 / Page 4 of 5Ex.1008 / Page 4 of 5
`
`TESLA, INC.TESLA, INC.
`
`

`

`Application Note
`
`MTD/PS-0259
`
`
`For example, if the photoflash provides 75% of the exposure light, then only 25% of the light from ambient sources is
`integrated over the entire integration time; that 25% is the only light which can contribute to image blur. The ratio of the flash
`to ambient exposure light, is dependent upon the system configuration, including the sensor type, the sensor filter type (RGB
`or CYM for example), the IR cutoff filter, operating speed, and flash type. And, of course, the ambient light level.
`
`
`
`
`Eastman Kodak Company - Image Sensor Solutions - Rochester, NY 14650-2010
`Phone (716) 722-4385 Fax (716) 477-4947
`Web: www.kodak.com/go/imagers E-mail: imagers@kodak.com
`
`5
`
`
`Revision No. 1
`
`Ex.1008 / Page 5 of 5Ex.1008 / Page 5 of 5
`
`TESLA, INC.TESLA, INC.
`
`