IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`Confirmation No.: TBD
`
`Inventor(s)
`
`Serial No.
`
`Filed
`
`For
`
`Group
`
`Examiner
`
`Docket No.
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`T. NAKANO et al.
`
`To be assigned
`
`December 21, 2016
`
`ELECTRIC CAMERA
`
`To be assigned
`
`To be assigned
`
`ASA-9606—08
`
`Customer No.:
`
`24956
`
`PRELIMINARY AMENDMENT
`
`Mail Stop Amendments
`Commissioner for Patents
`PO. Box 1450
`
`Alexandria, VA 22313-1450
`
`Sir:
`
`December 21, 2016
`
`Prior to examination, please amend the above-identified application as follows.
`
`Amendments to the Specification begin on page 2;
`
`Remarks are included following the amendments.
`
`IPR2020-00597
`
`Apple EX1024 Page 1
`
`IPR2020-00597
`Apple EX1024 Page 1
`
`
`
`Confirmation No.: TBD
`
`Inventor(s)
`
`Serial No.
`
`Filed
`
`For
`
`Group
`
`Examiner
`
`Docket No.
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`T. NAKANO et al.
`
`To be assigned
`
`December 21, 2016
`
`ELECTRIC CAMERA
`
`To be assigned
`
`To be assigned
`
`ASA-9606—08
`
`Customer No.:
`
`24956
`
`PRELIMINARY AMENDMENT
`
`Mail Stop Amendments
`Commissioner for Patents
`PO. Box 1450
`
`Alexandria, VA 22313-1450
`
`Sir:
`
`December 21, 2016
`
`Prior to examination, please amend the above-identified application as follows.
`
`Amendments to the Specification begin on page 2;
`
`Remarks are included following the amendments.
`
`IPR2020-00597
`
`Apple EX1024 Page 1
`
`IPR2020-00597
`Apple EX1024 Page 1
`
`
`
`Serial No. To be assigned
`Preliminary Amendment filed December 21 , 2016
`
`ASA-9606-08
`
`IN THE SPECIFICATION:
`
`Please amend the first paragraph on page 1 of the specification following the
`
`heading "CROSS REFERENCE TO RELATED APPLICATION" as follows:
`
`CROSS REFERENCE TO RELATED APPLICATION
`
`This application is a continuation of U.S. patent application No.
`
`
`14/661 227 filed March 18 2015 which is a continuation of U.S. patent
`
`application No. 14/264,243, filed April 29, 2014, now U.S. Patent No.
`
`9,100,604, which is a continuation of U.S. application serial no. 13/681 ,495,
`
`filed November 20, 2012, now U.S. Patent No. 8,736,729, which is a
`
`continuation of U.S. application serial no. 12/845,266, filed July 28, 2010, now
`
`U.S. Patent No. 8,339,493, issued December 25, 2012, which is a
`
`continuation of U.S. application serial no. 10/660,710, filed September 12,
`
`2003, now U.S. Patent No. 8,059,177, issued November 15, 2011, and is
`
`related to U.S. application serial no. 10/660,711, filed September 12, 2003,
`
`now U.S. Patent No. 7,403,226, issued July 22, 2008, both of which are
`
`divisional applications of U.S. application Serial No. 09/520,836, filed March 8,
`
`2000, now U.S. Patent No. 6,765,616, issued July 20, 2004, the subject
`
`matter of all the above is incorporated by reference herein.
`
`Page 2 of 3
`
`|PR2020-00597
`
`Apple EX1024 Page 2
`
`IPR2020-00597
`Apple EX1024 Page 2
`
`
`
`Serial No. To be assigned
`Preliminary Amendment filed December 21 , 2016
`
`ASA-9606-08
`
`REMARKS
`
`Entry of the above amendments prior to examination is respectfully requested.
`
`No new matter is added by the amendments.
`
`Please charge any shortage in fees due in connection with the filing of this
`
`paper, or credit any overpayment of fees, to the deposit account of MATTING LY &
`
`MALUR, PC, Deposit Account No. 50-1417.
`
`Respectfully submitted,
`
`MATTINGLY & MALUR, PC
`
`/John R. Mattin l /
`
`John R. Mattingly, Reg. No. 30,293
`(703) 684-1 120
`
`Page 3 of 3
`
`|PR2020-00597
`
`Apple EX1024 Page 3
`
`IPR2020-00597
`Apple EX1024 Page 3
`
`
`
`
`CROSS REFERENCE TO RELATED APPLICATION
`
`_1_
`
`ELECTRIC CAMERA
`
`This application is a continuation of U.S. application serial no. 13/681,495,
`
`filed November 20, 2012, which is a continuation of U.S. application serial no.
`
`12/845,266, filed July 28, 2010, now U.S. Patent No. 8,339,493,
`
`issued December 25,
`
`2012, which is a continuation of U.S. application serial no. 10/660,710, filed
`
`September 12, 2003, now U.S. Patent No. 8,059,177,
`
`issued November 15, 2011, and is
`
`10
`
`related to U.S. application serial no. 10/660,711, filed September 12, 2003, now U.S.
`
`Patent No. 7,403,226,
`
`issued July 22, 2008, both of which are divisional applications
`
`of U.S. application Serial No. 09/520,836, filed March 8, 2000, now U.S. Patent No.
`
`6,765,616,
`
`issued July 20, 2004,
`
`the subject matter of all the above is incorporated
`
`by reference herein.
`
`15
`
`BACKGROUND OF THE INVENTION
`
`The present
`
`invention relates to a photography related to video cameras,
`
`camcorders, digital still cameras and others using a solid—state image sensing device,
`
`and more particularly to an electric camera using a solid—state image sensing device
`
`with a large number of pixels.
`
`20
`
`
`Electric cameras using solid—state image sensors such as CCDs
`
`(charge—coupled
`
`devices)
`
`include a soecalled video camera or camcorder for taking moving images and a
`
`so—called digital still camera for taking still images.
`
`In recent years, video cameras
`
`with a still image taking function and digital still cameras with a moving image
`
`taking function have become available.
`
`25
`
`In a video camera to photograph moving images, it is generally assumed that the
`
`video is viewed on a display such as television monitor and thus the camera is
`
`designed to produce output signals conforming to a television system such as NTSC and
`
`|PR2020—OO597
`
`Apple EX1024 Page 4
`
`IPR2020-00597
`Apple EX1024 Page 4
`
`
`
`_1b_
`
`PAL. Therefore,
`
`the effective number of vertically arranged pixels or picture elements
`
`on the image sensing device used in such a camera is determined to enable
`
`television signals to be generated. The NTSC system, for example, performs
`
`interlaced scanning on two fields, each of which has an effective scanning
`
`line number of about 240 lines (the number of scanning lines actually
`
`|PR2020-00597
`
`Apple EX1024 Page 5
`
`IPR2020-00597
`Apple EX1024 Page 5
`
`
`
`-2—
`
`displayed on the monitor which is equal to the number of
`
`scanning lines in the vertical blanking period subtracted
`
`from the total number of scanning lines in each field).
`
`To
`
`realize this,
`
`the image sensing device has about 480 pixel
`
`rows as the standard effective number of vertically
`
`arranged pixels. That is, the signals of two vertically
`
`adjoining pixels in each field are mixed together inside or
`
`outside the image sensing device to generate about 240
`
`scanning lines, and the combinations of pixels to be
`
`10
`
`cyclically mixed together are changed from one field to
`
`another to achieve the interlaced scanning.
`
`15
`
`20
`
`25
`
`Some image sensing devices to take moving images
`
`according to the NTSC system have an area of pixels for
`
`image stabilization added to the area of effective pixel
`
`area,
`
`thus bringing the effective number of vertically
`
`arranged pixels to about 480 or more.
`
`In this case, an
`
`area beyond 480th pixels is read out at high speed during
`
`the vertical blanking period and therefore the signals thus
`
`read out are not used as effective signals. Therefore,
`
`the
`
`Video signals can only be generated from these signals
`
`coming from the area of about 480 vertically arranged
`
`pixels. When such a camera is used to photograph a still
`
`image, it is relatively easy to generate a static image
`
`signal conforming to, for example,
`
`JPEG (Joint Photographic
`
`Expert Group)
`
`from the signals coming from the same pixel
`
`area that is used to take a moving image.
`
`A problem
`
`remains, however, that the number of vertically arranged
`
`pixels obtained is limited to around 480, making it
`
`|PR2020-00597
`
`Apple EX1024 Page 6
`
`IPR2020-00597
`Apple EX1024 Page 6
`
`
`
`_ 3 _
`
`impossible to produce more detailed static image signals.
`
`In a camera having an image sensing device with
`
`the area of pixels for image stabilization mentioned above,
`
`a method of alleviating this problem may involve using the
`
`entire area of effective pixels including the area of image
`
`stabilization pixels in photographing a still image.
`
`Even
`
`when photographing a still image, however,
`
`the photographed
`
`image needs to be monitored for check and, for that
`
`purpose, it is necessary to generate signals conforming to
`
`the television system from signals read out from all effec-
`
`tive pixels.
`
`An example of such a conventional camera has been
`
`proposed in JP—A—11—187306.
`
`In the camera disclosed in
`
`this publication, signals from all the effective pixels are
`
`read out taking two or more times the field period of the
`
`television system, stored in a memory means such as a field
`
`memory, and then subjected to interpolation processing for
`
`transformation into signals conforming to the field cycle
`
`and horizontal scan cycle of television.
`
`This conventional camera, however, requires a
`large processing circuit, such as field memory, for signal
`
`conversion. Another drawback is that the image sensing
`
`device readout cycle is a plurality of times the field
`
`cycle, degrading the dynamic resolution.
`
`Even with the use
`
`of this circuit,
`
`the number of pixels obtained as the
`
`static image signals is limited to the number of effective
`
`pixels used for moving videos plus the area of image
`
`stabilization pixels.
`
`5
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 7
`
`IPR2020-00597
`Apple EX1024 Page 7
`
`
`
`_ 4 _
`
`In a digital still camera designed for taking
`
`still images,
`
`there has been a trend in recent years toward
`
`an increasing number of pixels used on the moving video
`
`image sensing device in order to obtain higher resolution
`
`5
`
`static image signals. When taking a moving image or
`
`monitoring the video, it is necessary to generate signals
`
`that conform to the television system.
`
`The number of
`
`pixels on such an image sensing device, however, does not
`
`necessarily match the number of scanning lines of the
`
`10
`
`television system and therefore some form of conversion
`
`means is required.
`
`The conversion means may involve, as in the video
`
`camera with the area of image stabilization pixels, reading
`
`out signals from the image sensing device taking a longer
`
`15
`
`time than the field cycle and interpolating them to
`
`generate television signals. This method has,
`
`in addition
`
`to the problem described above, a drawback that the readout
`
`cycle increases as the number of pixels increases, further
`
`degrading the dynamic resolution.
`
`20
`
`To mitigate this problem, JPuAu9=270959 discloses
`
`an apparatus which mixes together or culls the pixel
`
`signals inside the image sensing device to reduce the
`
`number of signals to be read and therefore the read cycle.
`
`Although this apparatus alleviates the problem of the
`
`25
`
`degraded dynamic resolution, it requires a large processing
`
`circuit such as field memory to perform time—axis transfor-
`
`mation to generate signals conforming to the television
`
`system and the image sensing device itself needs to have a
`
`|PR2020-00597
`
`Apple EX1024 Page 8
`
`IPR2020-00597
`Apple EX1024 Page 8
`
`
`
`_ 5 _
`
`special structure for performing desired mixing and
`
`culling.
`
`SUMMARY OF THE INVENTION
`
`The present invention relates to a photography of
`
`5 video cameras, camcorders, digital still cameras and others
`
`using a solid—state image sensing device, and more particu—
`
`larly to an electric camera using a solid—state image
`
`sensing device with a large number of pixels.
`
`The conventional electric cameras, as described
`
`10
`
`above, have drawbacks that when taking a still picture by
`
`using a video camera,
`
`the number of pixels is not suffi—
`
`cient and that when taking a moving image with a still
`
`camera,
`
`the associated circuit inevitably increases and the
`
`dynamic image quality deteriorates. Taking both moving and
`
`15
`
`static images of satisfactory quality with a single camera
`
`is difficult to achieve.
`
`In addition to solving the above
`
`problems,
`
`to obtain good dynamic picture quality by using
`
`an image sensing device having a large number of pixels
`
`intended for still images requires extracting a pixel area
`
`20
`
`that is used to realize an image stabilizing function.
`
`The
`
`conventional art and cameras do not offer a means to
`
`accomplish this function.
`
`An object of the present invention is to provide
`
`an electric camera that solves these problems and which
`
`25
`
`uses an image sensing device with a sufficient number of
`
`pixels for still images and enables taking of highly
`
`detailed still images and a moving video taking with
`
`|PR2020-00597
`
`Apple EX1024 Page 9
`
`IPR2020-00597
`Apple EX1024 Page 9
`
`
`
`_5__.
`
`reduced image quality degradation without
`
`increasing circuitry
`
`such as field memory. It is also an object of the present
`
`invention to provide an electric camera that can also realize
`
`the image stabilizing function.
`
`According to one aspect of this invention,
`
`the
`
`electric camera to realize the above objectives has: an image
`
`sensing device with a light receiving surface having N
`
`vertically arranged pixels and an arbitrary number of pixels
`
`arranged horizontally, N being equal to or more than three times
`
`the number of effective scanning lines M of a display screen of
`
`a television system; a driver to drive the image sensing device
`
`to vertically mix or cull signal charges accumulated in
`
`individual pixels of every K pixels to produce a number of lines
`
`of output signals which corresponds to the number of effective
`
`scanning lines M, K being at least one of integers equal to or
`
`less than an integral part of a quotient of N divided by M (a
`
`number of lines of output signals corresponds to l/K the number
`
`of vertically arranged pixels N of the image sensing device);
`
`and a signal processing unit to generate image signals by using
`
`the output signals of the image sensing device.
`
`As explained above, since this invention eliminates
`
`the limit on the number of vertically arranged pixels, an
`
`electric camera can be provided which enables taking of highly
`
`detailed still images and a satisfactory moving video taking by
`
`using an image sensing device with a large enough pixel number
`
`even for still images.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Figure l is a block diagram showing the configu
`
`|PR2020-00597
`
`Apple EX1024 Page 10
`
`IPR2020-00597
`Apple EX1024 Page 10
`
`
`
`_ 7 _
`
`ration of a first embodiment of an electric camera accord-
`
`ing to the present invention.
`
`Figure 2 is a schematic diagram showing the
`
`structure of an image sensing device in the first embodi—
`
`ment of the electric camera of the invention.
`
`Figure 3 is a drive pulse timing diagram in the
`
`first embodiment of the electric camera of the invention.
`
`Figure 4 is a schematic diagram showing a mixing
`
`operation in the first embodiment of the electric camera of
`
`10
`
`the invention.
`
`Figure 5 is a schematic diagram showing a readout
`
`area in the first embodiment of the electric camera of the
`
`invention.
`
`15
`
`20
`
`Figure 6 is a schematic diagram showing a mixing
`
`operation in the first embodiment of the electric camera of
`
`the invention.
`
`Figure 7 is a block diagram showing the configu-
`
`ration of a second embodiment of an electric camera accord-
`
`ing to the present invention.
`
`Figure 8 is a schematic diagram showing a mixing
`
`operation in the second embodiment of the electric camera
`
`of the invention.
`
`Figure 9 is a schematic diagram showing a readout
`
`area in the second embodiment of the electric camera of the
`
`25
`
`invention.
`
`Figure 10 is a schematic diagram showing the
`
`structure of an image sensing device in a third embodiment
`
`of the electric camera according to the present invention.
`
`|PR2020-00597
`
`Apple EX1024 Page 11
`
`IPR2020-00597
`Apple EX1024 Page 11
`
`
`
`- 3 _
`
`Figure 11 is a drive pulse timing diagram in the
`
`third embodiment of the electric camera of the invention.
`
`Figure 12 is a schematic diagram showing an
`
`interpolation operation in the third embodiment of the
`
`electric camera of the invention.
`
`Figures 13A and 13B are schematic diagrams show-
`
`ing the arrangement of color filters in the image sensing
`
`device in a fourth embodiment of the electric camera
`
`according to the present invention.
`
`10
`
`DESCRIPTION OF THE EMBODIMENTS
`
`Now embodiments of the present invention will be
`
`described by referring to the accompanying drawings.
`
`Figure l is a block diagram showing the configuration of
`
`one embodiment of an electric camera according to the
`
`15
`
`invention.
`
`In Figure 1, reference number 1 represents a
`
`lens,
`
`2 an aperture,
`
`3 an image sensing device, 4 a drive
`
`circuit,
`
`5 a gain adjust circuit,
`
`6 an analog-digital
`
`(A/D)
`
`conversion circuit, 7 a signal processing circuit,
`
`8 a
`
`vertical interpolation circuit to perform interpolation in
`
`a vertical direction,
`
`9 a horizontal interpolation circuit
`
`to perform interpolation in a horizontal direction, 10 a
`
`recording unit including recording media such as magnetic
`
`tape, semiconductor memory and optical disk to record a
`
`video signal, 11 a control circuit to control these
`
`constitutional elements according to the operating state,
`
`12 an encoder circuit to modulate the video signal into a
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 12
`
`IPR2020-00597
`Apple EX1024 Page 12
`
`
`
`-9-
`
`standard television signal, 13 a digital—analog (D/A)
`
`conversion circuit, 14'a mode selector switch to change
`
`over the operation mode between the moving video taking and
`
`the still image taking, 15 a record button to start or stop
`
`the recording, 16a and 16b gyro sensors to detect vertical
`
`image—unstability and lateral image—unstability, respec-
`
`tively, and 17 an image—unstability decision circuit to
`
`determine the image—instability from signals output from
`
`the gyro sensors.
`
`10
`
`15
`
`20
`
`25
`
`In the above configuration,
`
`light coming from the
`
`lens 1 through the aperture 2 is focused on a light receiv—
`
`ing surface of the image sensing device 3 where it is
`
`converted into an electric signal.
`
`In this embodiment the
`
`image sensing device 3 is of a CCD type. Figure 2 shows
`
`the structure of this image sensing device 3.
`
`In Figure 2,
`
`denoted 30 are pixels each formed of a photodiode, which
`
`are arranged horizontally and vertically in a grid pattern.
`
`On these grid—arrayed pixels three types of color filters
`
`that pass yellow (Ye), green (G) and cyan (CY),
`
`respec—
`
`tively, are arranged in such a way that the combination of
`
`these three colors is repeated horizontally every three
`
`pixels and that the filters of the same colors are lined
`
`vertically in so—called vertical stripes. Although an
`
`arbitrary number of pixels may be used, this embodiment has
`
`an array of 1200 pixels vertically and 1600 pixels horizon—
`
`tally.
`
`A vertical transfer unit 32 is a CCD which is
`
`driven by three phase pulses V1, V2, V3. This CCD has a
`
`three—gate structure in which each pixel corresponds to
`
`|PR2020-00597
`
`Apple EX1024 Page 13
`
`IPR2020-00597
`Apple EX1024 Page 13
`
`
`
`-10-
`
`three phase pulses and thus can vertically transfer a
`
`signal charge of each pixel independently. Transfer gates
`
`31 for transferring the charge of each pixel to the
`
`vertical transfer unit 32 are commonly connected to a gate
`
`of the vertical transfer unit 32 that corresponds to the V2
`
`pulse. An operation to transfer the charge from each pixel
`
`to the vertical transfer unit 32 in reSponse to a peak
`
`value of the pulse applied to the commonly connected gate
`
`and an operation to transfer the charge vertically are
`
`performed separately.
`
`A horizontal transfer unit 33
`
`horizontally transfers the charges supplied from the
`
`vertical transfer units 32 and outputs them successively
`
`through an output amplifier 34 from the output terminal.
`
`Referring back to Figure l, the operation
`
`performed when the moving video mode is selected by the
`
`mode selector switch 14 will be explained. The number of
`
`vertically arranged pixels on the image sensing device in
`
`this embodiment is 1200, so if the number of effective
`
`scanning lines in the field of the NTSC system is assumed
`
`to be 240 lines,
`
`then vertically mixing five pixels (= 1200
`
`pixel rows/240 scanning lines) can match the number of
`
`lines of output signals from the image sensing device to
`
`the number of effective scanning lines.
`
`However,
`
`in this embodiment,
`
`to realize the image
`
`stabilizing function described later, four vertically
`
`arranged pixels are mixed together during motion image
`
`taking mode. When four vertically arranged pixels are to
`
`be cyclically mixed together,
`
`the signals from the area of
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 14
`
`IPR2020-00597
`Apple EX1024 Page 14
`
`
`
`_ 11 -
`
`960 pixels (= 240 scanning lines X 4 pixels) out of the
`
`1200 vertically arranged pixels are used as effectiVe
`
`signals and the remaining 240 pixels (= 1200 (all pixels) —
`
`960 (effective pixels)) are not used for image forming.
`
`Figure 3 shows the timing of a vertical drive pulse for the
`
`image sensing device in this operation mode, with V1, V2
`
`and V3 representing three phase drive pulses applied to
`
`each gate of the CCD or vertical transfer unit 32.
`
`In Figure 3,
`
`in a period T1 included in the
`
`10
`
`15
`
`20
`
`vertical blanking period,
`
`the drive pulse V2 is held high
`
`to transfer the signal charge accumulated in each pixel to
`
`under the V2 gate of the vertical CCD. Next,
`
`in a period
`
`T2, while the V2 pulse is still at middle level,
`
`the V3
`
`pulse is raised from low level to middle level; next, while
`
`the V3 pulse is at middle level,
`
`the V2 pulse is changed
`
`from middle level to low level, after which the V1 pulse is
`
`changed from low level to middle level; next, while the V1
`
`pulse is at middle level, the V3 pulse is changed from
`
`middle level to low level, after which the V2 pulse is
`
`changed to middle level and finally the V1 pulse is changed
`
`from middle level to low level. With this sequence of
`
`pulse operations, the signal charges under the V2 gate for
`
`one pixel row are transferred and held again under the V2
`
`gate.
`
`25
`
`By repeating this series of operations,
`
`the
`
`signal charges for a desired number of pixel rows can be
`
`transferred.
`
`In Figure 3, during a period T3 included in
`
`the vertical blanking period before the vertical effective
`
`|PR2020-00597
`
`Apple EX1024 Page 15
`
`IPR2020-00597
`Apple EX1024 Page 15
`
`
`
`- 12 _
`
`scanning period (the vertical scanning period minus the
`
`vertical blanking period which corresponds to the actually
`
`displayed image) and during a period T4 included in the
`
`vertical blanking period after the vertical effective
`
`scanning period,
`
`the above transfer operation for one pixel
`
`row is repeated a total of 240 times to transfer the signal
`
`charges of the 240 pixel rows not used for image generation
`
`to the horizontal transfer unit 33 during the vertical
`
`blanking period.
`
`For example, if this transfer operation
`
`10
`
`is performed 120 times during the period T3 and 120 times
`
`during the period T4,
`
`the signal charges from upper 120
`
`pixel rows and lower 120 pixel rows on the light receiving
`
`surface are transferred to the horizontal transfer unit 33
`
`during the period T3 and period T4 within the vertical
`
`15
`
`blanking period. During each of the subsequent periods T5
`
`and T6 in the vertical blanking period,
`
`the horizontal
`
`transfer unit 33 is driven for a predetermined period to
`
`output the charges transferred to the horizontal transfer
`
`unit 33 from the output terminal. These charges are not
`used as valid signals as they are output during the
`
`2O
`
`vertical blanking period.
`
`Next,
`
`in the vertical effective scanning period
`
`of Figure 3,
`
`the above one—pixel-row transfer operation is
`
`performed four times during each horizontal blanking period
`
`25
`
`to transfer the signal charges of four pixel rows to the
`
`horizontal transfer unit 33 where they are mixed together.
`
`Then, during a horizontal effective scanning period (the
`
`horizontal scanning period minus the horizontal blanking
`
`|PR2020-00597
`
`Apple EX1024 Page 16
`
`IPR2020-00597
`Apple EX1024 Page 16
`
`
`
`_ 13 _
`
`period which corresponds to the actually displayed image),
`
`the horizontal transfer unit 33 is driven to read out the
`
`signal charges from the horizontal transfer unit to produce
`
`an output signal conforming to the television system.
`
`If
`
`the above operation is performed on the A field and if, on
`
`the B field,
`
`the number of pixel rows transferred during
`
`the period T3 is set to 122 rows and that during the period
`
`T4 is set to 118 rows,
`
`then the combination of four pixel
`
`rows to be cyclically mixed together shifts by two rows
`
`between the two fields,
`
`thus allowing the interlaced scan~
`
`ning to be performed as shown in Figure 4.
`
`(Figure 4 shows
`
`the light receiving surface of the image sensing device and
`
`its relation to the displayed screen is vertically
`
`inverted.)
`
`Let us return to Figure l.
`
`The output signal
`
`from the image sensing device 3 is adjusted in gain by the
`
`gain adjust circuit 5 and then converted by the A/D conver—
`
`sion circuit 6 into a digital signal.
`
`The digital signal
`
`is then processed by the signal processing circuit 7 that
`
`performs color signal processing and luminance signal
`
`processing, such as generation of color signals, gamma
`
`correction, white balance processing and outline enhance—
`
`ment.
`
`The image sensing device in this embodiment has an
`
`array of vertical stripes of yellow (Ye), green (G) and
`
`cyan (Cy) color filters, so the color signals for Ye, G and
`
`Cy are obtained as a series of color points from one line
`
`of output signals at all times no matter how many pixels
`
`are vertically combined.
`
`From these color signals three
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 17
`
`IPR2020-00597
`Apple EX1024 Page 17
`
`
`
`_ 14 -
`
`primary color signals R, G, B can be obtained from the
`
`following calculations.
`
`R = Ye - G
`
`B = Cy - G
`
`G = G
`
`The R, G and B signals undergoes the white
`
`balance processing and gamma correction processing in the
`
`signal processing circuit 7 and are then converted into
`
`color difference signals such as R—Y, B—Y or U and V.
`
`The
`
`luminance signals and the color difference signals are then
`entered through the vertical interpolation circuit 8 into
`
`the horizontal interpolation circuit 9.
`
`In this operation
`
`state the signals just pass through the vertical interpola—
`
`tion circuit 8 without being processed.
`
`The horizontal
`
`interpolation circuit 9 performs interpolation on the
`
`signals in the horizontal direction.
`
`Figure 5 shows the light receiving surface of the
`
`image sensing device. As described above,
`
`in the operating
`
`state of this embodiment,
`
`the signals read out during the
`
`vertical effective scanning period correspond to an area
`
`having 960 of the 1200 vertically arranged pixels and a
`
`horizontal width of 1600 pixels, as shown shaded at A in
`
`Figure 5.
`
`If the entire light receiving surface of the
`
`image sensing device has a 4-to—3 (width to height) aspect
`
`ratio,
`
`the shaded area A is more laterally elongate than
`
`this aspect ratio. Hence, if the signals of all horizontal
`
`pixels of the light receiving surface are displayed, for
`
`example, on an NTSC standard television monitor with the
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 18
`
`IPR2020-00597
`Apple EX1024 Page 18
`
`
`
`_ 15 _
`
`4—to—3 aspect ratio,
`
`the image displayed is compressed
`
`horizontally and looks vertically elongate, compared with
`
`the original image.
`
`It is therefore necessary to output
`
`during the horizontal effective scanning period only those
`
`signals coming from a pixel area with the horizontal width
`
`conforming to the aspect ratio of the television system, as
`
`shown by a shaded area B. When the television system has
`
`an 4-to-3 aspect ratio,
`
`the number of pixels in the
`
`horizontal width of the shaded area B is 1280 (= 960
`
`(vertical effective pixels) X 4/3).
`
`Returning back to Figure l,
`
`the horizontal
`
`interpolation circuit 9 performs interpolation processing
`
`on the signals from the horizontal 1280 pixels to expand
`
`the signals so that they can be output over the entire
`
`horizontal effective scanning period.
`
`It also performs
`
`switching among different clocks as required. With the
`
`above operation, an area having 960 pixels in height and
`
`1280 pixels in width is demarcated from the light receiving
`
`surface as signals conforming to the television system.
`
`Then,
`
`the luminance signal and the color difference signal
`
`are encoded by the encoder circuit 12 into television
`
`signals, which are then converted by the D/A conversion
`
`circuit 13 into analog signals for output. When the
`
`recording is specified by the record button 15,
`
`the signals
`
`are recorded by the recording unit 10. At this time, the
`
`signals may be compressed in the MPEG (Moving Picture
`
`Expert Group) format.
`
`Next,
`
`the image stabilizing operation will be
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 19
`
`IPR2020-00597
`Apple EX1024 Page 19
`
`
`
`-16..
`
`explained.
`
`Image—unstability information obtained by the
`
`gyro sensors 16a, 16b that detect vertical and horizontal
`
`image-unstabilities is entered into the image-unstability
`
`decision circuit 17, which checks the received information
`
`for the amount and direction of the image—unstability and
`
`converts them into the number of pixels in vertical and
`
`horizontal directions on the light receiving surface of the
`
`image sensing device. Based on the converted pixel
`
`numbers,
`
`the position of an extracted area (effective pixel
`
`area) on the light receiving surface is shifted in a
`
`direction that cancels the image—unstability. This can
`
`correct the image—unstability.
`
`The positional shifting of
`
`the extracted area is performed as follows.
`
`The shifting
`
`in the vertical direction can be made by changing the
`
`number of pixel rows transferred during the periods T3 and
`
`T4 in Figure 3 and the shifting in the horizontal direction
`
`made by changing the interpolation start position in the
`
`horizontal interpolation circuit 9.
`
`The operation during the moving video mode has
`
`been described above. Next,
`
`the operation performed when
`
`the static image mode is selected by the mode selector
`
`switch 14 will be explained.
`
`In the static image mode,
`
`too, until the record—
`
`ing is requested by the record button 15,
`
`the camera
`
`outputs signals compatible with the television system to
`
`monitor the angle of view. Unlike the moving video photo—
`
`graphing, all of the effective pixels on the image sensing
`
`device are used in this embodiment during the still image
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 20
`
`IPR2020-00597
`Apple EX1024 Page 20
`
`
`
`_ 17 _
`
`photographing to produce signals with as high a resolution
`
`as possible. Hence, during the monitoring the television
`
`signals need to be generated from the signals coming from
`
`the entire pixel area.
`
`The image sensing device of this embodiment has
`
`1200 vertically arranged pixels, and the number of lines of
`
`output signals from the image sensing device can be made to
`
`match the number of effective scanning lines of NTSC
`
`system, which is assumed to have 240 scanning lines, by
`
`vertically mixing five pixels (= 1200/240).
`
`To make the
`
`image sensing device operate in this manner,
`
`the one—pixel-
`
`row transfer operation is performed five times during each
`
`horizontal blanking period in the vertical effective
`
`scanning period shown in the pulse timing diagram of Figure
`
`3. With this operation,
`
`the signal charges of five pixel
`
`rows can be mixed by the horizontal transfer unit 33. As
`
`for the transfer operations during the periods T3 and T4 in
`
`the vertical blanking period, because the interlaced scan—
`
`ning is carried out, only two pixel rows are transferred
`
`during the period T3 on the B field, with no transfer
`
`operations performed in other vertical blanking periods (In
`
`this embodiment, 1200/240
`
`5 with no remainder produced,
`
`so no further transfer is necessary; if, however, a
`
`remainder occurs,
`
`the remaining pixels need only be trans—
`
`ferred during the periods T3 and T4).
`
`The charges mixed by the horizontal transfer unit
`
`33 are read out by driving the horizontal transfer unit 33
`
`during the horizontal effective scanning period. With the
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 21
`
`IPR2020-00597
`Apple EX1024 Page 21
`
`
`
`- 18 -
`
`above operations,
`
`the signal charges of all pixels on the
`
`image sensing device can be read out in a manner conforming
`
`to the television system.
`
`The output signal from the image
`
`sensing device 3 is, as during the moving image photo-
`
`graphing, adjusted in gain by the gain adjust circuit 5 and
`
`converted by the A/D conversion circuit 6 into a digital
`
`signal, which is then subjected to the color signal
`
`processing and the luminance signal processing in the
`
`signal processing circuit 7 before being entered into the
`
`vertical interpolation circuit 8. During the static image
`
`monitoring,
`
`the vertical interpolation circuit 8 performs a
`
`vertical gravity center correction on the received signals.
`
`Figure 6 shows combinations of pixels to be
`
`cyclically mixed on the A field and the B field and also
`
`the vertical position of the gravity center of the mixed
`
`signals.
`
`In the interlaced scanning, scanning lines of the
`
`A field and the B field are located at the centers of
`
`adjoining scanning lines on other field. Hence,
`
`the signal
`
`samplings in the camera system for the two fields must be
`
`180 degrees out of phase in the vertical direction.
`
`In the
`
`operating state of this embodiment, however, because five
`
`pixels are mixed together,
`
`the gravity centers of the
`
`output signals for the A field and the B field are deviated
`
`36 degrees (= 1/2 pixel or l/lO the line-to—line distance
`
`on the same field) from the ideal sampling phase difference
`
`of 180 degrees, as shown in Figure 6.
`
`To correct this
`
`requires generating a signal from two adjoining output
`
`lines by interpolation.
`
`For example, if we let an nth
`
`10
`
`15
`
`20
`
`25
`
`|PR2020-00597
`
`Apple EX1024 Page 22
`
`IPR2020-00597
`App
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
