Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 1 of 19 PageID #: 220
`Case 5:23-cv-00152-RWS Document1-8 Filed 12/29/23 Page 1 of 19 PagelD #: 220
`
`EXHIBIT 7
`EXHIBIT 7
`
`

`

`US008736729B2
`
`(12) United States Patent
`Nakano et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,736,729 B2
`*May 27, 2014
`
`(54) ELECTRIC CAMERA
`(71) Applicant: Hitachi, Ltd., Tokyo (JP)
`(72) Inventors: Takahiro Nakano, Tokyo (JP); Ryuji
`Nishimura, Tokyo (JP); Toshiro
`Kinugasa, Tokyo (JP)
`
`(*) Notice:
`
`(73) Assignee: Hitachi Consumer Electronics Co.,
`Ltd., Tokyo (JP)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`This patent is Subject to a terminal dis
`claimer.
`
`(21) Appl. No.: 13/681,495
`
`(22) Filed:
`
`Nov. 20, 2012
`
`(65)
`
`Prior Publication Data
`US 2013/OO93933 A1
`Apr. 18, 2013
`
`Related U.S. Application Data
`(60) Continuation of application No. 12/845,266, filed on
`Jul. 28, 2010, now Pat. No. 8,339,493, which is a
`continuation of application No. 10/660,710, filed on
`Sep. 12, 2003, now Pat. No. 8,059,177, which is a
`division of application No. 09/520,836, filed on Mar. 8,
`2000, now Pat. No. 6,765,616.
`Foreign Application Priority Data
`
`(30)
`
`Jan. 11, 2000 (JP) ................................. 2000-006064
`
`(51) Int. Cl.
`H04N 5/335
`(52) U.S. Cl.
`USPC ............................ 348/294; 348/312: 348/322
`(58) Field of Classification Search
`CPC .......... H04N 5/23245; H04N 5/23248; H04N
`
`(2011.01)
`
`5/23274; H04N 5/23258; H04N 5/343; H04N
`5/345; H04N 5/3452
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,054,915 A 10/1977 Sugihara
`5,170,249 A 12/1992 Ohtsubo et al.
`5,187,569 A
`2, 1993 Tani
`5,287,192 A * 2/1994 Iizuka ........................... 348,311
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`
`11, 1992
`04-323973
`10, 1997
`9-270959
`(Continued)
`Primary Examiner — Loung TNguyen
`(74) Attorney, Agent, or Firm — Antonelli, Terry, Stout &
`Kraus, LLP.
`
`ABSTRACT
`(57)
`An electric camera includes an image sensing device with a
`light receiving Surface having N vertically arranged pixels
`and an arbitrary number of pixels arranged horizontally, N
`being equal to or more than three times the number of effec
`tive Scanning lines M of a display screen of a television
`system, a driver to drive the image sensing device to vertically
`mix or cull signal charges accumulated in individual pixels of
`K pixels to produce, during a vertical effective scanning
`period of the television system, a number of lines of output
`signals which corresponds to 1/K the number of vertically
`arranged pixels N of the image sensing device, K being an
`integer equal to or less than an integral part of a quotient of N
`divided by M, and a signal processing unit having a function
`of generating image signals by using the output signals of the
`image sensing device.
`
`4 Claims, 8 Drawing Sheets
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 2 of 19 PageID #: 221
`
`16a
`2
`
`GYRO
`SENSO
`
`17
`-
`
`MAGE
`UNSABILTY
`DECISION
`CRCU
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`RWE
`CERCUIT
`
`18
`
`14
`
`5
`
`VEGF
`
`SWITCH
`
`Mode
`SELECTOR
`SWTC
`
`RECORD
`BUtoN
`
`CONTROL
`CIRCUT
`
`RECORING
`UN
`
`
`
`EMAGE
`
`GAN
`
`SIGNAL
`
`WERICAL
`
`HORIZONTA
`
`SENSN go PENG- E - El-SEF-DA
`
`CRC
`
`CRC
`
`OUPU
`
`N 3
`LENs 2 APERTURE
`
`5
`
`6
`
`7
`
`8
`
`9
`
`2
`
`13
`
`

`

`US 8,736,729 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,734,424 A * 3/1998 Sasaki ........................ 348.222.1
`5,828,406 A 10/1998 Parulski et al.
`5,847,758 A 12/1998 Iizuka
`6, 195,125 B1
`2/2001 Udagawa et al.
`6,519,000 B1
`2/2003 Udagawa
`6,661.451 B1
`12/2003 Kijima et al.
`6,765,616 B1* 7/2004 Nakano et al. ................ 348/322
`6,798.448 B1* 9/2004 Motono et al. ............. 348.222.1
`6,906,746 B2
`6/2005 Hijishiri et al.
`
`6,970,191 B1 * 1 1/2005 Sase et al. .................. 348,220.1
`7,154,539 B1
`12/2006 Nishimura et al.
`7,403.226 B2
`7/2008 Nakano et al.
`8,059,177 B2 * 1 1/2011 Nakano et al. ................ 348,294
`
`FOREIGN PATENT DOCUMENTS
`
`11-004456
`JP
`11-1873.06
`JP
`11-355.665
`JP
`* cited by examiner
`
`1, 1999
`7, 1999
`12/1999
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 3 of 19 PageID #: 222
`
`

`

`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 4 of 19 PageID #: 223
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 4 of 19 PagelD #: 223
`
`U.S. Patent
`
`May27, 2014
`
`Sheet 1 of 8
`
`US 8,736,729 B2
`
`Sls
`
`elol
`
`LINDHID
`
`vl
`
`3CON
`
`YdOLOATAS
`
`HOLIMS
`
`INOHIO
`
`|
`
`JOHLNOO
`
`
`
`HNALNOvolYACOON]
`
`
`LINOUIOMoebeyNOLLIOdasnrayv|}ONISNSSonigga90ud1a
`WINOZIHOH
`HALHAINTWNOIS
`
`
`LINDYIOLINOUID
`WOILLYSAAAVIA|LINDHID
`Linawio
`auNLYsdyZoNaSDIAAG
`
`ZtBgl
`
`ADVAN!
`
`ALNISV.LSNN
`
`NOIsIoad
`
`LINDE
`
`|HOSNSS
`
`OAD
`
`HOSNSS
`
`SAG
`
`v
`
`G9L
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`May 27, 2014
`
`Sheet 2 of 8
`
`US 8,736,729 B2
`
`FIG.2
`
`32
`V1 V2 V3
`
`
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 5 of 19 PageID #: 224
`
`FG.4
`
`s
`
`A FIELD (n+1)TH LINE
`
`b. BFIELD nTH INE
`
`was
`
`A FELD nTH LINE
`
`

`

`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 6 of 19 PageID #: 225
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 6 of 19 PagelD #: 225
`
`US 8,736,729 B2
`
`U.S. Patent
`
`May27, 2014
`
`Sheet 3 of 8
`
`yin
`
`Jen
`
`eOld
`
`TWLINOZIHOH
`
`
`
`QOldWAdONINNVOS
`
`
`
`QOldadONDINV1E
`
`
`
`ol
`
`
`
`
`
`
`
` TWOLLYSA COldadWOLLYSA GOIHAdwgGONAdONDNVTEONINNVOSSAILO3453IWOILLESAONDINVTSTWOLLUSA
`
`
`
`
`
`GOldadONINNVOS
`
`
`
`
`
`
`
`
`
`
`
`4h|UU
`IL
`Ud
`TAAT)FIGAIN|ZA
`
`
`TSARFIGCIN
`
`WASTTIGA
`SASHOI
`WAT]M01
`TSAS1MO
`WATTM01
`
`
`

`

`U.S. Patent
`
`May 27, 2014
`
`Sheet 4 of 8
`
`US 8,736,729 B2
`
`ººººººººººººººººººººººº Ø d
`&&&&&&&&&&&&&&&&&&&&&&
`
`X RS
`
`
`
`SR
`&
`O O
`
`FIG.5
`
`AREAB
`
`AREA A
`
`SRS O
`C
`d
`
`s O O
`s C
`9. s
`C
`9.
`C
`S. O
`O
`() 8 0. O SRS
`SS
`O
`O
`O
`O
`O
`O
`s
`X) 888
`8.
`X
`O
`d SRs
`s
`C
`O
`X XC S.
`S. O
`SR 8.
`
`C
`C
`O d
`
`
`
` d O d
`Y
`
`ŠAREA A
`
`1280
`-600
`
`
`
`
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 7 of 19 PageID #: 226
`
`A FIELD GRAVITY CENTER OF (n+1)TH LINE
`
`• • • • • • • • • • • • • •
`
`- - - - - - - - BFIELD GRAVITY CENTER OF nTH LINE
`
`A FIELD GRAVITY CENTER OF nTH LINE
`
`

`

`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 8 of 19 PageID #: 227
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 8 of 19 PagelD #: 227
`
`U.S. Patent
`
`May27, 2014
`
`Sheet 5 of 8
`
`LOIGlvlSt
`
`qHOosY
`
`NOLLNG
`
`AGOW
`
`HOLOAIAS
`
`HOLIMS
`
`STONYMIA
`
`|
`
`JONVHO
`
`HOLIMS
`
`ZtBOL
`
`ADVANI
`
`ALMGVLSNN
`
`NOISIOSG
`
`OAD
`
`YOSNAS
`
`LhYOSNAS
`
`
`
`LINDHIDOYAD
`
`
`
`ONIGHOOSYJOHLNOO
`
`
`
`OL|LINN_|LInoHIOqgt
`
`SANG
`
`
`
`
`[,LNdino|WidHACOONa-HSINI“HSNoescedLsnravONISNAS
`a)LINDYIDNOLLW10dNOILV1Od
`
`
`LINDHIDLINDHID
`
`WOLLYSANIVDADVINI|LINDYID
`
`LINDHIOLINDO
`
`SOIC|
`
`WILNOZIHOH
`
`
`
`
`
`US 8,736,729 B2
`
`ELZk68Z9Sg\\
`
`sunivsdygSNEL
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`May 27, 2014
`
`Sheet 6 of 8
`
`US 8,736,729 B2
`
`- A FIELD GRAVITY CENTER OF (n+1)TH LINE
`
`- -
`as
`a
`lar
`- - -
`all BFIELD GRAVITY CENTER OF nh LINE
`J - A FIELD GRAVITY CENTER OF nTH LINE
`
`a a 3 e is
`
`
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 9 of 19 PageID #: 228
`
`AREA A : 96OHGH x 128OWIDE
`
`AREA B : 72OHIGH x 96OWDE
`
`AREA C : 480HIGH x 64OWIDE
`
`

`

`U.S. Patent
`
`May 27, 2014
`
`Sheet 7 of 8
`
`US 8,736,729 B2
`
`
`
`F.G. 10
`
`- nTH INE - n'TH LINE
`on-- (n'+1)TH LINE
`(n+1)TH LINE
`ab -- (n'+2)TH LINE
`-- (n'+3)TH LINE
`- (n+2)TH LINE - (n'+4)TH LINE
`(BEFORE INTERPOLATION)
`(AFTER INTERPOLATION)
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 10 of 19 PageID #: 229
`
`
`
`F.G. 13A
`
`

`

`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 11 of 19 PageID #: 230
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 11 of 19 PagelD #: 230
`
`U.S. Patent
`
`May27, 2014
`
`Sheet 8 of 8
`
`US 8,736,729 B2
`
`|LOLLILeoLALee"JenI|
`onTaAT1HOI
`
`“WINOZIOH
`
`
`
`GOldadSNINNVOS
`
`
`
`QGOlWadONDINY14
`
`TWLNOZIHOH
`
` LEDs»=abl
`
`
`
`TATSTGIN
`
`TAA]HOIH
`
`TSASICGIN
`
`ASTM01
`
`VaAaTHOI
`
`oeaori.LILsrsrcaneJ”
`
`ULLPeeelyOOLfUUreansconJ"-ppfpLLJUUsaxon
`SA
`
`
`
`
`
`
`
`OldDNINNVOS
`
`
`
`
`
`
`
`
`
` TVOLLHSA COlWSdONDINYTEWOLLYSA|GOivadONINNVOSSALLOASAaWOUWSA__GOldadONDINYTETWOLLYSA
`
`
`

`

`US 8,736,729 B2
`
`1.
`ELECTRIC CAMERA
`
`CROSS REFERENCE TO RELATED
`APPLICATION
`
`This application is a continuation of U.S. application Ser.
`No. 12/845,266, filed Jul. 28, 20102, which is a continuation
`of U.S. application Ser. No. 10/660,710, filed Sep. 12, 2003,
`now U.S. Pat. No. 8,059,177, issued Nov. 15, 2011, and is
`related to U.S. application Ser. No. 10/660,711, filed Sep. 12,
`2003, now U.S. Pat. No. 7,403.226, issued Jul. 22, 2008, both
`of which are divisional applications of U.S. application Ser.
`No. 09/520,836, filed Mar. 8, 2000, now U.S. Pat. No. 6,765,
`616, issued Jul. 20, 2004, the subject matter of which is
`incorporated by reference herein.
`
`10
`
`15
`
`BACKGROUND OF THE INVENTION
`
`2
`arranged pixels obtained is limited to around 480, making it
`impossible to produce more detailed Static image signals.
`In a camera having an image sensing device with the area of
`pixels for image stabilization mentioned above, a method of
`alleviating this problem may involve using the entire area of
`effective pixels including the area of image stabilization pix
`els in photographing a still image. Even when photographing
`a still image, however, the photographed image needs to be
`monitored for check and, for that purpose, it is necessary to
`generate signals conforming to the television system from
`signals read out from all effective pixels.
`An example of Such a conventional camera has been pro
`posed in JP-A-11-187306. In the camera disclosed in this
`publication, signals from all the effective pixels are read out
`taking two or more times the field period of the television
`system, stored in a memory means such as a field memory,
`and then subjected to interpolation processing for transfor
`mation into signals conforming to the field cycle and hori
`Zontal scan cycle of television.
`This conventional camera, however, requires a large pro
`cessing circuit, such as field memory, for signal conversion.
`Another drawback is that the image sensing device readout
`cycle is a plurality of times the field cycle, degrading the
`dynamic resolution. Even with the use of this circuit, the
`number of pixels obtained as the static image signals is lim
`ited to the number of effective pixels used for moving videos
`plus the area of image stabilization pixels.
`In a digital still camera designed for taking still images,
`there has been a trend in recent years toward an increasing
`number of pixels used on the moving video image sensing
`device in order to obtain higher resolution static image sig
`nals. When taking a moving image or monitoring the video, it
`is necessary to generate signals that conform to the television
`system. The number of pixels on Such an image sensing
`device, however, does not necessarily match the number of
`scanning lines of the television system and therefore some
`form of conversion means is required.
`The conversion means may involve, as in the video camera
`with the area of image stabilization pixels, reading out signals
`from the image sensing device taking a longer time than the
`field cycle and interpolating them to generate television sig
`nals. This method has, in addition to the problem described
`above, a drawback that the readout cycle increases as the
`number of pixels increases, further degrading the dynamic
`resolution.
`To mitigate this problem, JP-A-9-270959 discloses an
`apparatus which mixes together or culls the pixel signals
`inside the image sensing device to reduce the number of
`signals to be read and therefore the read cycle. Although this
`apparatus alleviates the problem of the degraded dynamic
`resolution, it requires a large processing circuit such as field
`memory to perform time-axis transformation to generate sig
`nals conforming to the television system and the image sens
`ing device itself needs to have a special structure for perform
`ing desired mixing and culling.
`
`SUMMARY OF THE INVENTION
`
`The present invention relates to a photography of video
`cameras, camcorders, digital still cameras and others using a
`Solid-state image sensing device, and more particularly to an
`electric camera using a Solid-state image sensing device with
`a large number of pixels.
`The conventional electric cameras, as described above,
`have drawbacks that when taking a still picture by using a
`video camera, the number of pixels is not sufficient and that
`when taking a moving image with a still camera, the associ
`
`The present invention relates to a photography related to
`Video cameras, camcorders, digital still cameras and others
`using a solid-state image sensing device, and more particu
`larly to an electric camera using a solid-state image sensing
`device with a large number of pixels.
`Electric cameras using solid-state image sensors such as
`CCDs (charge-coupled devices) include a so-called video
`camera or camcorder for taking moving images and a so
`called digital still camera for taking still images. In recent
`years, video cameras with a still image taking function and
`digital still cameras with a moving image taking function
`have become available.
`In a video camera to photograph moving images, it is
`generally assumed that the video is viewed on a display Such
`as television monitor and thus the camera is designed to
`produce output signals conforming to a television system
`such as NTSC and PAL. Therefore, the effective number of
`Vertically arranged pixels or picture elements on the image
`sensing device used in Such a camera is determined to enable
`television signals to be generated. The NTSC system, for
`example, performs interlaced scanning on two fields, each of
`40
`which has an effective scanning line number of about 240
`lines (the number of scanning lines actually displayed on the
`monitor which is equal to the number of scanning lines in the
`vertical blanking period subtracted from the total number of
`scanning lines in each field). To realize this, the image sensing
`45
`device has about 480 pixel rows as the standard effective
`number of Vertically arranged pixels. That is, the signals of
`two vertically adjoining pixels in each field are mixed
`together inside or outside the image sensing device to gener
`ate about 240 Scanning lines, and the combinations of pixels
`to be cyclically mixed together are changed from one field to
`another to achieve the interlaced scanning.
`Some image sensing devices to take moving images
`according to the NTSC system have an area of pixels for
`image stabilization added to the area of effective pixel area,
`thus bringing the effective number of vertically arranged
`pixels to about 480 or more. In this case, an area beyond 480th
`pixels is read out at high speed during the vertical blanking
`period and therefore the signals thus read out are not used as
`effective signals. Therefore, the video signals can only be
`generated from those signals coming from the area of about
`480 vertically arranged pixels. When such a camera is used to
`photograph a still image, it is relatively easy to generate a
`static image signal conforming to, for example, JPEG (Joint
`Photographic Expert Group) from the signals coming from
`65
`the same pixel area that is used to take a moving image. A
`problem remains, however, that the number of vertically
`
`25
`
`30
`
`35
`
`50
`
`55
`
`60
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 12 of 19 PageID #: 231
`
`

`

`3
`ated circuit inevitably increases and the dynamic image qual
`ity deteriorates. Taking both moving and static images of
`satisfactory quality with a single camera is difficult to
`achieve. In addition to solving the above problems, to obtain
`good dynamic picture quality by using an image sensing
`device having a large number of pixels intended for still
`images requires extracting a pixel area that is used to realize
`an image stabilizing function. The conventional art and cam
`eras do not offer a means to accomplish this function.
`An object of the present invention is to provide an electric
`camera that solves these problems and which uses an image
`sensing device with a sufficient number of pixels for still
`images and enables taking of highly detailed still images and
`a moving video taking with reduced image quality degrada
`tion without increasing circuitry such as field memory. It is
`also an object of the present invention to provide an electric
`camera that can also realize the image stabilizing function.
`According to one aspect of this invention, the electric cam
`era to realize the above objectives has: an image sensing
`device with a light receiving surface having N vertically
`arranged pixels and an arbitrary number of pixels arranged
`horizontally, N being equal to or more than three times the
`number of effective scanning lines Mofa display screen of a
`television system; a driver to drive the image sensing device
`to vertically mix or cull signal charges accumulated in indi
`vidual pixels of every Kpixels to produce a number of lines of
`output signals which corresponds to the number of effective
`scanning lines M. K being at least one of integers equal to or
`less than an integral part of a quotient of N divided by M (a
`number of lines of output signals corresponds to 1/K the
`number of vertically arranged pixels N of the image sensing
`device); and a signal processing unit to generate image sig
`nals by using the output signals of the image sensing device.
`As explained above, since this invention eliminates the
`limit on the number of Vertically arranged pixels, an electric
`camera can be provided which enables taking of highly
`detailed still images and a satisfactory moving video taking
`by using an image sensing device with a large enough pixel
`number even for still images.
`
`5
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`US 8,736,729 B2
`
`4
`FIG.11 is a drive pulse timing diagram in the third embodi
`ment of the electric camera of the invention.
`FIG. 12 is a schematic diagram showing an interpolation
`operation in the third embodiment of the electric camera of
`the invention.
`FIGS. 13A and 13B are schematic diagrams showing the
`arrangement of color filters in the image sensing device in a
`fourth embodiment of the electric camera according to the
`present invention.
`
`DESCRIPTION OF THE EMBODIMENTS
`
`Now embodiments of the present invention will be
`described by referring to the accompanying drawings. FIG. 1
`is a block diagram showing the configuration of one embodi
`ment of an electric camera according to the invention.
`In FIG. 1, reference number 1 represents a lens, 2 an
`aperture, 3 an image sensing device, 4 a drive circuit, 5 again
`adjust circuit, 6 an analog-digital (A/D) conversion circuit, 7
`a signal processing circuit, 8 a vertical interpolation circuit to
`perform interpolation in a vertical direction, 9 a horizontal
`interpolation circuit to perform interpolation in a horizontal
`direction, 10 a recording unit including recording media Such
`as magnetic tape, semiconductor memory and optical disk to
`record a video signal, 11 a control circuit to control these
`constitutional elements according to the operating State, 12 an
`encoder circuit to modulate the video signal into a standard
`television signal, 13 a digital-analog (D/A) conversion cir
`cuit, 14 a mode selector Switch to change over the operation
`mode between the moving video taking and the still image
`taking, 15 a record button to start or stop the recording, 16a
`and 16b gyro sensors to detect vertical image-unstability and
`lateral image-unstability, respectively, and 17 an image-un
`stability decision circuit to determine the image-instability
`from signals output from the gyro sensors.
`In the above configuration, light coming from the lens 1
`through the aperture 2 is focused on a light receiving Surface
`of the image sensing device 3 where it is converted into an
`electric signal. In this embodiment the image sensing device
`3 is of a CCD type. FIG. 2 shows the structure of this image
`sensing device 3. In FIG.2, denoted 30 are pixels each formed
`of a photodiode, which are arranged horizontally and verti
`cally in a grid pattern. On these grid-arrayed pixels three types
`of color filters that pass yellow (Ye), green (G) and cyan (Cy),
`respectively, are arranged in Such a way that the combination
`of these three colors is repeated horizontally every three pix
`els and that the filters of the same colors are lined vertically in
`so-called vertical stripes. Although an arbitrary number of
`pixels may be used, this embodiment has an array of 1200
`pixels vertically and 1600 pixels horizontally. A vertical
`transfer unit 32 is a CCD which is driven by three phase pulses
`V1,V2, V3. This CCD has a three-gate structure in which
`each pixel corresponds to three phase pulses and thus can
`Vertically transfera signal charge of each pixel independently.
`Transfer gates 31 for transferring the charge of each pixel to
`the vertical transfer unit 32 are commonly connected to a gate
`of the vertical transfer unit 32 that corresponds to the V2
`pulse. An operation to transfer the charge from each pixel to
`the vertical transfer unit 32 in response to a peak value of the
`pulse applied to the commonly connected gate and an opera
`tion to transfer the charge vertically are performed separately.
`A horizontal transfer unit 33 horizontally transfers the
`charges supplied from the vertical transfer units 32 and out
`puts them Successively through an output amplifier 34 from
`the output terminal.
`Referring back to FIG. 1, the operation performed when
`the moving video mode is selected by the mode selector
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 13 of 19 PageID #: 232
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a block diagram showing the configuration of a
`first embodiment of an electric camera according to the
`present invention.
`FIG. 2 is a schematic diagram showing the structure of an
`image sensing device in the first embodiment of the electric
`camera of the invention.
`FIG. 3 is a drive pulse timing diagram in the first embodi
`ment of the electric camera of the invention.
`FIG. 4 is a schematic diagram showing a mixing operation
`in the first embodiment of the electric camera of the invention.
`FIG.5 is a schematic diagram showing a readout area in the
`first embodiment of the electric camera of the invention.
`FIG. 6 is a schematic diagram showing a mixing operation
`in the first embodiment of the electric camera of the invention.
`FIG. 7 is a block diagram showing the configuration of a
`second embodiment of an electric camera according to the
`present invention.
`FIG. 8 is a schematic diagram showing a mixing operation
`in the second embodiment of the electric camera of the inven
`tion.
`FIG.9 is a schematic diagram showing a readout area in the
`second embodiment of the electric camera of the invention.
`FIG. 10 is a schematic diagram showing the structure of an
`image sensing device in a third embodiment of the electric
`camera according to the present invention.
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`US 8,736,729 B2
`
`5
`
`10
`
`15
`
`6
`television system. If the above operation is performed on the
`A field and if, on the B field, the number of pixel rows
`transferred during the period T3 is set to 122 rows and that
`during the period T4 is set to 118 rows, then the combination
`of four pixel rows to be cyclically mixed together shifts by
`two rows between the two fields, thus allowing the interlaced
`scanning to be performed as shown in FIG. 4. (FIG. 4 shows
`the light receiving Surface of the image sensing device and its
`relation to the displayed screen is vertically inverted.)
`Let us return to FIG.1. The output signal from the image
`sensing device 3 is adjusted in gain by the gain adjust circuit
`5 and then converted by the A/D conversion circuit 6 into a
`digital signal. The digital signal is then processed by the
`signal processing circuit 7 that performs color signal process
`ing and luminance signal processing. Such as generation of
`color signals, gamma correction, white balance processing
`and outline enhancement. The image sensing device in this
`embodiment has an array of vertical stripes of yellow (Ye).
`green (G) and cyan (Cy) color filters, so the color signals for
`Ye, G and Cy are obtained as a series of color points from one
`line of output signals at all times no matter how many pixels
`are vertically combined. From these color signals three pri
`mary color signals R,G,B can be obtained from the following
`calculations.
`
`5
`switch 14 will be explained. The number of vertically
`arranged pixels on the image sensing device in this embodi
`ment is 1200, so if the number of effective scanning lines in
`the field of the NTSC system is assumed to be 240 lines, then
`vertically mixing five pixels (=1200 pixel rows/240 scanning
`lines) can match the number of lines of output signals from
`the image sensing device to the number of effective scanning
`lines.
`However, in this embodiment, to realize the image stabi
`lizing function described later, four vertically arranged pixels
`are mixed together during motion image taking mode. When
`four vertically arranged pixels are to be cyclically mixed
`together, the signals from the area of 960 pixels (=240 scan
`ning linesx4 pixels) out of the 1200 vertically arranged pixels
`are used as effective signals and the remaining 240 pixels
`(=1200 (all pixels)-960 (effective pixels)) are not used for
`image forming. FIG. 3 shows the timing of a vertical drive
`pulse for the image sensing device in this operation mode,
`with V1,V2 and V3 representing three phase drive pulses
`applied to each gate of the CCD or vertical transfer unit 32.
`In FIG. 3, in a period T1 included in the vertical blanking
`period, the drive pulse V2 is held high to transfer the signal
`charge accumulated in each pixel to under the V2 gate of the
`vertical CCD. Next, in a period T2, while the V2 pulse is still
`at middle level, the V3 pulse is raised from low level to middle
`level; next, while the V3 pulse is at middle level, the V2 pulse
`is changed from middle level to low level, after which the V1
`pulse is changed from low level to middle level; next, while
`the V1 pulse is at middle level, the V3 pulse is changed from
`middle level to low level, after which the V2 pulse is changed
`to middle level and finally the V1 pulse is changed from
`middle level to low level. With this sequence of pulse opera
`tions, the signal charges under the V2 gate for one pixel row
`are transferred and held again under the V2 gate.
`By repeating this series of operations, the signal charges
`for a desired number of pixel rows can be transferred. In FIG.
`3, during a period T3 included in the vertical blanking period
`before the vertical effective scanning period (the vertical
`scanning period minus the vertical blanking period which
`corresponds to the actually displayed image) and during a
`period T4 included in the vertical blanking period after the
`Vertical effective scanning period, the above transfer opera
`tion for one pixel row is repeated a total of 240 times to
`transfer the signal charges of the 240 pixel rows not used for
`image generation to the horizontal transfer unit 33 during the
`Vertical blanking period. For example, if this transfer opera
`tion is performed 120 times during the period T3 and 120
`times during the period T4, the signal charges from upper 120
`pixel rows and lower 120 pixel rows on the light receiving
`surface are transferred to the horizontal transfer unit 33 dur
`ing the period T3 and period T4 within the vertical blanking
`period. During each of the subsequent periods T5 and T6 in
`the vertical blanking period, the horizontal transfer unit 33 is
`driven for a predetermined period to output the charges trans
`ferred to the horizontal transfer unit 33 from the output ter
`minal. These charges are not used as valid signals as they are
`output during the vertical blanking period.
`Next, in the vertical effective scanning period of FIG.3, the
`above one-pixel-row transfer operation is performed four
`times during each horizontal blanking period to transfer the
`signal charges of four pixel rows to the horizontal transfer unit
`33 where they are mixed together. Then, during a horizontal
`effective Scanning period (the horizontal scanning period
`minus the horizontal blanking period which corresponds to
`the actually displayed image), the horizontal transfer unit 33
`is driven to read out the signal charges from the horizontal
`transfer unit to produce an output signal conforming to the
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`The R, G and B signals undergoes the white balance pro
`cessing and gamma correction processing in the signal pro
`cessing circuit 7 and are then converted into color difference
`signals such as R-Y. B-Y or U and V. The luminance signals
`and the color difference signals are then entered through the
`vertical interpolation circuit 8 into the horizontal interpola
`tion circuit 9. In this operation state the signals just pass
`through the vertical interpolation circuit 8 without being pro
`cessed. The horizontal interpolation circuit 9 performs inter
`polation on the signals in the horizontal direction.
`FIG. 5 shows the light receiving surface of the image
`sensing device. As described above, in the operating state of
`this embodiment, the signals read out during the vertical
`effective scanning period correspond to an area having 960 of
`the 1200 vertically arranged pixels and a horizontal width of
`1600 pixels, as shown shaded at A in FIG. 5. If the entire light
`receiving Surface of the image sensing device has a 4-to-3
`(width to height) aspect ratio, the shaded area A is more
`laterally elongate than this aspect ratio. Hence, if the signals
`of all horizontal pixels of the light receiving surface are dis
`played, for example, on an NTSC standard television monitor
`with the 4-to-3 aspect ratio, the image displayed is com
`pressed horizontally and looks vertically elongate, compared
`with the original image. It is therefore necessary to output
`during the horizontal effective scanning period only those
`signals coming from a pixel area with the horizontal width
`conforming to the aspect ratio of the television system, as
`shown by a shaded area B. When the television system has an
`4-to-3 aspect ratio, the number of pixels in the horizontal
`width of the shaded area B is 1280 (=960 (vertical effective
`pixels)x4/3).
`Returning back to FIG. 1, the horizontal interpolation cir
`cuit 9 performs interpolation processing on the signals from
`the horizontal 1280 pixels to expand the signals so that they
`can be output over the entire horizontal effective scanning
`period. It also performs Switching among different clocks as
`required. With the above operation, an area having 960 pixels
`
`Case 5:23-cv-00152-RWS Document 1-8 Filed 12/29/23 Page 14 of 19 PageID #: 233
`
`

`

`US 8,736,729 B2
`
`10
`
`15
`
`25
`
`30
`
`8
`7
`3 is, as during the moving image photographing, adjusted in
`in height and 1280 pixels in width is demarcated from the
`light receiving Surface as signals conforming to the television
`gain by the gain adjust circuit 5 and converted by the A/D
`conversion circuit 6 into a digital signal, which is then Sub
`system. Then, the luminance signal and the color difference
`jected to the color signal processing and the luminance signal
`signal are encoded by the encoder circuit 12 into television
`signals, which are then converted by the D/A conversion 5 processing in the signal processing circuit 7 before being
`circuit 13 into analog signals for output. When the recording
`entered into the vertical interpolation circuit 8. During the
`static image monitoring, the vertical interpolation circuit 8
`is specified by the record button 15, the signals are recorded
`by the recording unit 10. At this time, the signals may be
`performs a vertical gravity center correction on the received
`signals.
`compressed in the MPEG (Moving Picture Expert Group)
`FIG. 6 shows combinations of pixels to be cyclically mixed
`format.
`Next, the image stabilizing operation will be explained.
`on the A field and the B field and also the vertical position of
`Image-unstability information obtained by the gyro sensors
`the gravity center of the mixed signals. In the interlaced
`scanning, Scanning lines of the A field and the B field are
`16a, 16b that detect vertical and horizontal image-unstabili
`located at the centers