111111
`
`1111111111111111111111111111111111111111111111111111111111111111111111111111
`US 20040174444Al
`
`(19) United States
`(12) Patent Application Publication
`Ishii
`
`(10) Pub. No.: US 2004/0174444 Al
`Sep. 9, 2004
`(43) Pub. Date:
`
`(54) SIGNAL PROCESSING APPARATUS AND
`IMAGE DATA GENERATION APPARATUS
`
`(52) U.S. Cl. ................... 348/240.1; 348/240.2; 348/252;
`348/220.1; 348/333.02
`
`(76)
`
`Inventor: Yoshiki Ishii, Kanagawa (JP)
`
`(57)
`
`ABSTRACT
`
`Correspondence Address:
`MORGAN & FINNEGAN, L.L.P.
`345 PARKAVENUE
`NEW YORK, NY 10154 (US)
`
`(21) Appl. No.:
`
`10/790,930
`
`(22) Filed:
`
`Mar. 1, 2004
`
`(30)
`
`Foreign Application Priority Data
`
`Mar. 4, 2003
`
`(JP) ...................................... 2003-057628
`
`Publication Classification
`
`(51)
`
`Int. CI? ......................... H04N 5/262; H04N 5/208;
`H04N 5!222
`
`A recording mode control circuit controls a first recording
`mode in which still image data having a first image size
`corresponding to the number of pixels of an image sensing
`element is generated and recorded and a second recording
`mode in which moving image data having a second image
`size smaller than the first image size is generated and
`recorded. In generating the moving image data in the second
`recording mode, a camera signal processing circuit executes
`electrical zoom processing to obtain the second image size.
`At this time, a maximum magnification ratio setting circuit
`sets the maximum variable magnification ratio of electrical
`zoom processing in the camera signal processing circuit.
`With this processing, an image recording apparatus which
`allows a user to select a zoom mode with a small degradation
`in image quality in electrical zoom processing can be
`implemented.
`
`MAXIMUM
`MAGNIFICATION 214
`RATIO
`SELECTION
`INPUT
`ZOOM
`OPERATION
`KEY INPUT
`
`213
`
`r216
`
`CONTROL -CIRCUIT
`
`~
`
`OPTICAL
`ZOOM
`
`RECORDING
`MODE INPUT
`
`210
`
`'
`
`ZOOM MAGNIFICATION
`RATIO INDICATION
`215
`OUTPUT
`
`~
`
`211
`
`!212
`ELECTRONIC
`ZOOM
`VARIABLE
`CONTROL f---+ MAGNIFICATION f--
`CIRCUIT
`CONTROL
`CIRCUIT
`
`t
`
`t
`
`r202
`
`IMAGE
`SENSING
`DATA INPUT
`
`~
`
`IMAGE
`SENSING
`ME~ORY -~
`SIGNAL
`PROCESSING
`CIRCUIT
`
`r203 - PROCESSING
`- PROCESSING
`
`r2os
`REDUCTION
`
`CIRCUIT
`
`r206
`ENLARGEMENT
`
`CIRCUIT
`
`104
`
`T
`
`r204
`VARIABLE
`MAGNIFICATION
`PROCESSING CIRCUIT
`
`r207
`
`+
`VARIABLE
`MAGNIFICATION
`PROCESSING
`SELECTION CIRCUIT
`t
`
`r2os 20
`9
`-- ME~ORY~
`
`AVER EXHIBIT 1006
`Page 1 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~
`N c c
`
`'"""'
`~ >
`
`'Jl
`Cj
`
`-..J
`N
`
`'"""' 0 ......,
`~
`
`'Jl =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'Jl
`
`~ .... 0 =
`""C g. -....
`~ .... 0 =
`'t:l -....
`>
`~ = .....
`~
`""C
`
`(')
`
`'t:l
`
`(')
`
`VIEWFINDER
`
`1 j114
`
`CIRCUIT
`CONTROL
`DISPLAY
`
`1 f 113
`
`~
`~2
`
`I
`,. y y r111
`
`L___. ...,..--fo
`I
`
`,___.
`
`INDICATION
`
`CIRCUIT
`
`MAGNIFICATION
`
`RATIO
`
`ZOOM
`
`.. 1
`
`I
`
`I
`
`r--+1 MAGNIFICATION I
`
`MAXIMUM
`
`RATIO SETTING
`
`CIRCUIT
`
`f 110
`
`. f 109 .
`
`f 106
`
`( 105
`
`•
`
`• ( 104
`
`( 103
`
`.--------------1 MODE CONTROL 1------,
`
`CIRCUIT
`
`RECORDING
`
`f 108
`
`FIG. 1 m07
`
`1----+
`
`ELEMENT
`SENSING
`IMAGE
`t )10~
`
`SYSTEM
`OPTICAL~
`'\ LENS ~
`t ( 101
`
`PROCESSING
`SIGNAL
`CAMERA ~ RECORDING
`
`A/D CIRCUIT 1-----.j PROCESSING
`
`CIRCUIT
`
`CIRCUIT
`
`SIGNAL
`
`AVER EXHIBIT 1006
`Page 2 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~ c
`N c c
`
`'"""'
`~ >
`
`'JJ.
`Cj
`
`-..J
`N
`0 ......,
`N
`~ .....
`'JJ. =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'JJ.
`
`.... 0 =
`0' -....
`~
`.... 0 =
`~ 't:l -....
`~ = .....
`~ .....
`""C
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`~ PROCESSING 1-----.J
`
`ENLARGEMENT
`
`CIRCUIT
`
`f--.1 ME~ORY H
`~ 299
`
`I 207
`
`1
`
`VARIABLE
`+
`
`SELECTION CIRCUIT
`
`PROCESSING
`MAGNIFICATION
`
`-""
`
`~ PROCESSING I
`f 205
`
`REDUCTION
`
`CIRCUIT
`
`r-• __ ........... ___,, PROCESSING CIRCUIT
`MAGNIFICATION
`VARIABLE
`j204
`
`MAGNIFICATION I-
`
`.-
`
`CIRCUIT
`CONTROL
`
`ELECTRONIC
`
`VARIABLE
`
`T
`
`)
`104
`
`1212
`
`211
`
`1
`
`FIG. 2
`
`1215
`
`......
`OUTPUT
`RATIO INDICATION
`ZOOM MAGNIFICATION
`
`CIRCUIT
`PR5~E~~~NG
`SENSING
`IMAGE
`
`j_203
`
`CIRCUIT
`CONTROL I----
`
`ZOOM
`
`~0
`
`2
`
`MEMO
`1
`
`IMAGE (201
`
`SENSING ll-(cid:173)
`
`DATAINPUT
`
`~1
`
`CIRCUIT
`
`1 CONTR
`ZOOM
`OPTIC
`
`6
`
`210
`
`J,.
`217
`
`213
`
`214
`
`MODE INPUT
`RECORDING
`
`KEY INPUT
`OPERATION
`ZOOM
`INPUT
`SELECTION
`RATIO
`MAGNIFICATION
`MAXIMUM
`
`AVER EXHIBIT 1006
`Page 3 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 3 of 27
`
`US 2004/0174444 Al
`
`FIG. 3
`
`301
`~
`.
`..
`
`.I
`
`r302
`REDUCTION
`CONTROL
`CIRCUIT
`
`r (304
`
`LPF
`COEFFICIENT
`SETIING
`CIRCUIT
`
`• (305
`
`303
`~ -
`,., - SPATIALLPF
`
`VARIABLE
`
`f 205
`
`307
`-()
`
`~
`
`(306
`- DOWN-SAMPLING
`
`CIRCUIT
`
`AVER EXHIBIT 1006
`Page 4 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 4 of 27
`
`US 2004/0174444 Al
`
`FIG. 4
`
`GAIN
`
`HORIZONTAL
`SPATIAL FREQUENCY
`
`AVER EXHIBIT 1006
`Page 5 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 5 of 27
`
`US 2004/0174444 Al
`
`FIG. 5
`
`f 206
`
`r502
`ENLARGEMENT
`CONTROL
`CIRCUIT
`
`(505
`CORRECTION
`COEFFICIENT
`SETIING CIRCUIT
`
`.
`.
`
`r
`
`(504
`
`UP-SAMPLING
`CIRCUIT
`
`,
`
`(506
`APERTURE
`CORRECTION
`CIRCUIT
`
`507.
`-cY
`
`501
`~
`..
`
`,/
`
`503
`..
`~
`
`,/
`
`AVER EXHIBIT 1006
`Page 6 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 6 of 27
`
`US 2004/0174444 Al
`
`FIG. 6
`
`EDGE ENHANCEMENT
`. SIGNAL GAIN
`
`M
`
`o~-----------------------------------
`N
`1.0
`
`ENLARGEMENT RATIO
`
`AVER EXHIBIT 1006
`Page 7 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 7 of 27
`
`US 2004/0174444 Al
`
`FIG. 7
`
`f 204a
`
`r702
`701
`.. MAGNIFICATION
`VARIABLE
`~
`,
`
`CONTROL CIRCUIT
`
`703
`~
`rJ
`
`, r704
`
`.. RE-SAMPLING
`...
`FILTE
`
`705
`-d
`
`AVER EXHIBIT 1006
`Page 8 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 8 of 27
`
`US 2004/0174444 Al
`
`FIG. 8
`
`0
`
`802
`
`0
`
`STILL IMAGE
`MODE
`
`MOVING IMAGE
`MODE
`
`AVER EXHIBIT 1006
`Page 9 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~
`N c c
`
`'"""'
`~ >
`
`'JJ.
`Cj
`
`-..J
`N
`0 ......,
`~
`~
`
`'JJ. =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'JJ.
`
`912
`
`911
`
`ZOOM WIDE-ANGLE STATE
`VIEW ANGLE OF OPTICAL:
`
`MAGNIFICATION
`EXTRACTION AT
`
`RATIO 0Fx1
`
`~ .... 0 =
`""C g. -....
`~ .... 0 =
`'t:l -....
`>
`~ = ......
`~
`""C
`
`(')
`
`(')
`
`'t:l
`
`910
`
`ZOOM
`OPTICAL
`
`ZOOM TELEPHOTO STATE
`VIEW ANGLE OF OPTICAL
`
`FIG. 9
`
`AVER EXHIBIT 1006
`Page 10 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 10 of 27
`
`US 2004/0174444 A1
`
`FIG. 10·
`
`1002
`
`1001
`
`w
`
`T
`
`114
`
`AVER EXHIBIT 1006
`Page 11 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 11 of 27
`
`US 2004/0174444 Al
`
`FIG. 11A w I ~
`
`1101
`
`1001
`)
`
`1102 1103
`
`I \ I I I T
`
`F I G. 11 8 w 1,....-------,~oo~n-----~,-+--,l I T
`
`1101
`
`1102
`
`1001.
`)
`
`F I G. 11 c w L--1 ___..LL,;~a..._______,JI T
`
`1101
`
`1001
`)
`
`AVER EXHIBIT 1006
`Page 12 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 12 of 27
`
`US 2004/0174444 A1
`
`FIG. 12A wl ~
`
`1201
`
`1001
`)
`
`1202 1203
`1\1\lr
`
`1201
`
`1202
`
`1001
`)
`
`F I G. 1 2 B w lr--...,.-bo-,~r-----r-1-f--1\ I T
`
`1201
`
`1001
`)
`
`F I G. 12C w .....__I """""'-~ __.I T
`
`AVER EXHIBIT 1006
`Page 13 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 13 of 27
`
`US 2004/0174444 A1
`
`FIG. 13
`
`f 1301
`MENU
`
`CAMERA SHAKE CORRECTION
`
`!
`
`13
`03
`
`ELECTRICAL ZOOM
`
`WHITE.BALANCE
`
`t
`r
`1304
`
`r1302
`OFF
`
`X2
`
`XB
`
`AVER EXHIBIT 1006
`Page 14 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 14 of 27
`
`US 2004/0174444 A1
`
`FIG. 14
`
`SET MAGNIFICATION RATIO
`(SELECT MODE)
`
`S1
`
`INITIALIZE MAGNIFICATION
`RATIO TO WIDE-ANGLE SIDE
`
`82
`
`NO
`
`DETECT TELEPHOTO
`OR WIDE-ANGLE STATE
`
`EXECUTE FIRST
`SIGNAL PROCESSING
`
`813
`
`814
`
`NO
`
`NO
`
`AVER EXHIBIT 1006
`Page 15 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 15 of 27
`
`US 2004/0174444 A1
`
`FIG. 15
`
`SET SECOND MODE
`
`84
`
`DETECT TELEPHOTO
`OR WIDE-ANGLE STATE
`
`85
`
`EXECUTE FIRST
`SIGNAL PROCESSING
`
`86
`
`NO
`
`NO
`
`YES
`
`INHIBIT SECOND
`SIGNAL PROCESSING
`
`89
`
`AVER EXHIBIT 1006
`Page 16 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 16 of 27
`
`US 2004/0174444 A1
`
`FIG. 16
`
`EXECUTE SECOND
`SIGNAL PROCESSING
`
`S17
`
`DETECT TELEPHOTO
`OR WIDE-ANGLE STATE
`
`S18
`
`EXECUTE SECOND
`SIGNAL PROCESSING
`
`S19
`
`INHIBIT SECOND
`SIGNAL PROCESSING
`
`S23
`
`AVER EXHIBIT 1006
`Page 17 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~
`N c c
`
`'"""'
`~ >
`
`'JJ.
`Cj
`
`-..J
`N
`0 ......,
`'"""'
`-..J
`~
`
`'JJ. =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'JJ.
`
`~ .... 0 =
`""C g. -....
`~ .... 0 =
`'t:l -....
`>
`~ = .....
`~
`""C
`
`(')
`
`'t:l
`
`(')
`
`114
`
`CIRCUIT
`CONTROL
`DISPLAY
`
`' s 113
`
`MAGNIFICATION H
`
`I
`
`INDICATION
`
`CIRCUIT
`
`RATIO
`
`ZOOM
`
`s 110
`
`RECORDING
`
`MEDIUM
`
`PROCESSING
`
`RECORDING
`
`SIGNAL
`
`CIRCUIT
`
`rvf-fl2
`
`T
`
`{)
`• y y r111
`
`1
`
`PROCESSING
`
`SIGNAL
`CAMERA
`
`CIRCUIT
`
`1
`
`~ AJD CIRCUIT 1-----.1
`
`RATIO SETTING
`r---+1 MAGNIFICATION
`
`MAXIMUM
`
`CIRCUIT
`
`109
`
`£13
`
`\I SYSTEM
`OPTICAL
`
`LENS
`
`105
`
`104a
`
`( 103
`
`I 102
`
`.L
`
`...-----------~ MODE CONTROL t-----,
`
`CIRCUIT
`
`RECORDING
`
`o:t
`
`~ 108
`
`FIG. 17
`
`AVER EXHIBIT 1006
`Page 18 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~ c
`N c c
`
`'"""'
`~ >
`
`'Jl
`Cj
`
`-..J
`N
`0 ......,
`'"""' 00
`~ .....
`'Jl =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'Jl
`
`MEMORY
`
`2
`
`209
`
`208
`
`.... 0 =
`0' -....
`~
`.... 0 =
`~ 't:l -....
`~ = .....
`~ .....
`""C
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`PROCESSING CIRCUIT
`MAGNIFICATION
`VARIABLE
`r204
`
`'
`
`1
`
`SELECTION CIRCUIT
`
`PROCESSING
`MAGNIFICATION
`
`VARIABLE
`
`207
`
`~ PROCESSING
`ENLARGEMENT
`206
`
`CIRCUIT
`
`PROCESSING
`REDUCTION
`
`CIRCUIT
`
`f 205
`
`MAGNIFICATION ~--------,
`
`r-
`
`+
`CIRCUIT
`CONTROL
`
`ELECTRONIC
`
`VARIABLE
`
`)
`104a
`
`r212,
`
`18
`
`FIG.
`
`( 215
`
`OUTPUT
`RATIO INDICATION
`ZOOM MAGNIFICATION
`
`, t t t I (211~
`
`6
`
`213
`
`214
`
`1516
`
`KEY INPUT
`ZOOM OPERATION
`INPUT
`·SELECTION
`RATIO
`MAGNIFICATION
`MAXIMUM
`FIXING INPUT
`x1 MAGNIFICATION
`
`PROCESSING
`
`CIRCUIT
`
`SIGNAL
`SENSING
`IMAGE
`
`203
`
`j_.
`
`CIRCUIT
`CONTROL j---.
`
`ZOOM
`
`SENSING c>-.J MEMORY
`
`1
`
`DATAINPUT~ I
`
`!
`
`201
`
`IMAGE
`
`~2
`
`210
`
`J
`
`MODE INPUT
`RECORDING
`
`21~ CIRCUIT
`
`CONTR
`ZOOM
`OPTIC
`
`~1
`
`AVER EXHIBIT 1006
`Page 19 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 19 of 27
`
`US 2004/0174444 A1
`
`_J
`
`-<~ uo
`r-o
`a..N
`0
`
`oq-
`0
`<0
`T"""
`
`T"""
`
`-.JW
`-<1--
`u-<
`-r-
`1--CI)
`a..o
`o._
`LLo
`O:::c
`wa..
`-.JW
`(!}-.J
`zW
`-< ......
`!$:~
`wO
`-o
`>N
`
`-.Jw
`-<1--
`(_)~
`1--CI)
`a..w
`0--.J
`LL<!J
`oz
`w4=
`-.JW
`<!JO
`z-
`-<!$:
`!$:~
`wo
`-o
`>N
`
`0)
`
`,....
`•
`G -u.
`
`AVER EXHIBIT 1006
`Page 20 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~ c
`N c c
`
`'"""'
`~ >
`
`'JJ.
`Cj
`
`-..J
`N
`0 ......,
`N c
`~ .....
`'JJ. =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'JJ.
`
`.... 0 =
`~ -a -....
`.... 0 =
`~ 't:l -....
`~ = .....
`~ .....
`""C
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`VIEWFINDER
`
`_1114
`
`~
`I ~2
`
`... y y r111a
`
`'-------~0
`I
`
`CIRCUIT
`CONTROL
`DISPLAY
`
`+ 5113
`
`~
`j106
`
`RECORDING
`
`MEDIUM
`
`RATIO ~
`
`INDICATION
`
`CIRCUIT
`
`MAGNIFICATION
`511 o
`
`ZOOM
`
`PROCESSING
`RECORDING
`
`SIGNAL
`
`CIRCUIT
`
`... 1
`
`I
`
`~ ~~¥~~~~Ni~~ 1-----'
`
`CIRCUIT
`
`MAXIMUM
`
`} 109a
`
`CIRCUIT
`
`AID CIRCUIT ~ PROCESSING
`
`SIGNAL
`CAMERA
`
`r---
`
`j105
`
`f104b
`
`(103
`
`.-------------1 MODE CONTROL t-----,
`
`CIRCUIT
`
`RECORDING
`
`FIG. 20 o::t
`
`)108
`
`AVER EXHIBIT 1006
`Page 21 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~
`N c c
`
`'"""'
`~ >
`
`'JJ.
`Cj
`
`-..J
`N
`
`'"""' 0 ......,
`
`N
`~
`
`'JJ. =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'JJ.
`
`~ .... 0 =
`""C g. -....
`~ .... 0 =
`'t:l -....
`>
`~ = .....
`~
`""C
`
`(')
`
`'t:l
`
`(')
`
`9
`
`(207
`
`VARIABLE
`+
`
`PROCESSING
`MAGNIFICATION ~~ MEMORY
`
`2
`
`SELECTION CIRCUIT
`
`_t
`
`----~~~---~
`
`~ PROCESSING
`ENLARGEMENT
`(206
`
`CIRCUIT
`
`PROCESSING CIRCUIT
`MAGNIFICATION
`VARIABLE
`(204
`
`r
`
`(205
`
`ELECTRICAL
`
`CIRCUIT
`CONTROL
`
`ZOOM
`
`)
`104b
`
`11811
`
`1
`
`FIG. · 21
`
`~.J
`
`. 215
`
`RATIO INDICATION
`ZOOM MAGNIFICATION
`
`_.,
`
`....
`210
`
`J
`
`./
`
`213
`
`"
`r 214
`
`MODE INPUT
`RECORDING
`
`KEY INPUT
`OPERATION
`ZOOM
`INPUT
`SELECTION
`RATIO
`MAGNIFICATION
`MAXIMUM
`
`'(203 ~ PROCESSING
`REDUCTION
`
`CIRCUIT
`
`PROCESSING
`
`CIRCUIT
`
`SIGNAL
`SENSING
`IMAGE
`
`MEMORY
`
`1
`
`201
`
`"' ""
`
`DATA INPUT
`SENSING
`IMAGE
`
`(202
`
`AVER EXHIBIT 1006
`Page 22 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 22 of 27
`
`US 2004/0174444 Al
`
`<X)
`0
`0>
`.,....
`_.
`<(
`(_)~
`a: a"~
`1-o
`ON w
`_.
`w
`
`.,....
`
`.......
`0
`0>
`.,....
`_.
`<(
`(_)~
`g:o..-
`oO
`wN
`_.
`w
`
`1-Z <eo..-
`z1-X
`0<CLL
`-oo
`I-_ ou.o
`< ( - -
`a:ZI-
`!;:<(.!)<(
`<(0:
`w~
`
`C\1
`C\1
`
`.,....
`
`•
`CJ
`
`-
`
`LL
`
`AVER EXHIBIT 1006
`Page 23 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 23 of 27
`
`US 2004/0174444 Al
`
`F I G. 23 A w r--1· -r~:hr-----rl-f--,1 I T
`
`2000
`)
`
`2001
`
`2002
`
`2001
`F 1 G. 23 8 w L - - 1 ____._.._m.....________,, T
`
`2000
`)
`
`AVER EXHIBIT 1006
`Page 24 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 24 of 27
`
`US 2004/0174444 Al
`
`2101
`
`2102
`
`2000
`)
`
`F I G. 24 A wl ,...----, ~r-+7r--~l---t---1\ I T
`
`2101
`
`'2000
`)
`
`F I G. 248 w L . . . . . . - .11 ~~--.~IT
`
`AVER EXHIBIT 1006
`Page 25 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 25 of 27
`
`US 2004/0174444 Al
`
`FIG. 25
`
`' 2201
`
`MENU
`
`CAMERA SHAKE CORRECTION
`
`!
`
`03
`22
`
`ELECTRICAL ZOOM
`
`WHITE BALANCE
`
`~
`(
`2204
`
`,2202
`X2
`xa
`
`AVER EXHIBIT 1006
`Page 26 of 46
`
`

`

`~
`~
`~
`'"""'
`-..J
`~ c
`N c c
`
`'"""'
`~ >
`
`'Jl
`Cj
`
`-..J
`N
`0 ......,
`0'1
`N
`~ .....
`'Jl =(cid:173)~
`
`~
`
`N c c
`
`~~
`~ '?
`'Jl
`
`~ ..... .... 0 =
`0' -....
`~
`~ ..... .... 0 =
`~ 't:l -....
`~ = .....
`~ .....
`""C
`
`(')
`
`(')
`
`-
`
`RECORDING
`
`MEDIUM
`
`....
`
`::;,
`r2314
`
`~
`
`PROCESSING
`
`RECORDING
`
`SIGNAL
`
`CIRCUIT
`
`r2313
`
`MEMORY2
`
`)2312
`
`PROCESSING
`ENLARGEMENT
`r f 2309
`
`CIRCUIT
`
`PROCESSING
`
`CIRCUIT
`
`SIGNAL
`CAMERA
`
`2305
`
`r2304
`
`2303
`
`2302
`
`FIG. 26
`
`ON
`I ELECTRICAL ZOOM OFF
`
`2311
`f
`
`[w T J
`r231o
`
`52soa
`
`CIRCUIT
`CONTROL
`
`ZOOM
`
`r2307
`
`OPTICAL ZOOM
`r2306
`
`CIRCUIT
`CONTROL
`
`MEMORY1
`
`CIRCUIT
`
`AID
`
`-
`
`ELEMENT
`SENSING
`IMAGE
`
`-
`
`-
`
`SYSTEM
`OPTICAL
`. LENS
`
`),._....,"'.
`
`\
`
`{
`
`AVER EXHIBIT 1006
`Page 27 of 46
`
`

`

`Patent Application Publication Sep. 9, 2004 Sheet 27 of 27
`
`US 2004/0174444 Al
`
`r--..
`0
`"'¢
`C\J
`
`....J
`<2::
`(.)~
`a: a
`t-o
`ON
`w
`....J w
`
`co
`0
`"'¢
`C\J
`
`....J
`<C~
`oo
`1--0
`O..N
`0
`
`1--Z
`<eo .....
`Zt-X
`0<2::L.L.
`-uo
`I--_
`OL.L.Q
`<Czt-
`a:CJ<2::
`~<2::a:
`w~
`
`1'-
`C\1
`
`•
`CJ
`
`-
`
`LL
`
`AVER EXHIBIT 1006
`Page 28 of 46
`
`

`

`US 2004/017 4444 Al
`
`Sep.9,2004
`
`1
`
`SIGNAL PROCESSING APPARATUS AND IMAGE
`DATA GENERATION APPARATUS
`
`control circuit 2307. That is, the zoom control circuit 2307
`allows optical zoom or electrical zoom processing.
`
`FIELD OF THE INVENTION
`
`[0001] The present invention relates to a signal processing
`apparatus and an image data generation apparatus that have
`both optical and electrical zoom means.
`
`BACKGROUND OF THE INVENTION
`
`In recent years, image recording apparatuses (or
`[0002]
`signal processing apparatuses or image data generation
`apparatuses) such as digital video cameras and digital (still)
`cameras are widely used in the ordinary household. Such
`image recording apparatuses generally have a zoom function
`capable of continuously changing the photographing view
`angle from a wide-angle mode to a telephoto mode by user
`operation. Many image recording apparatuses have even an
`electrical zoom function of electronically variably magni(cid:173)
`fying photographing image data in addition to a conven(cid:173)
`tional optical zoom function using a zoom lens optical
`system.
`
`[0003] FIG. 26 is a block diagram showing the schematic
`arrangement of a conventional image recording apparatus
`having an optical zoom function and an electrical zoom
`function. Referring to FIG. 26, a lens optical system 2301
`includes an optical lens which forms an image of incident
`light on an image sensing element 2302, an aperture, a focus
`controller, various kinds of optical filters, and a driving
`mechanism for them. The lens optical system 2301 can
`execute optical zoom for optically changing the view angle
`and includes a driving mechanism for it.
`
`[0004] The above-described optical mechanism in the lens
`optical system 2301 of the image recording apparatus shown
`in FIG. 26 is a common mechanism, and a description
`thereof will be omitted. The image sensing element 2302
`converts the optical signal of an image formed on the
`light-receiving surface by the lens optical system 2301 into
`an electrical signal. For example, a CCD (Charge Coupled
`Device) and a CMOS (Complementary Metal Oxide Semi(cid:173)
`conductor) sensor are known.
`
`[0005] An AID (Analog-to-Digital conversion) circuit
`2303 converts the image sensing signal, which has been
`converted into an electrical signal by the image sensing
`element 2302, into digital image sensing data (to be referred
`to as image sensing data hereinafter). A memory 1 (2304)
`stores the image sensing data converted by the AID circuit
`2303. At this point of time, the image sensing data is digital
`data in an output format unique to the image sensing element
`(the output format changes depending on the pixel array and
`the color filter array). A camera signal processing circuit
`2305 executes various processing operations for converting
`the image sensing data supplied from the memory 1 (2304)
`into image data. A zoom control circuit 2307 controls an
`optical zoom control circuit 2306 in accordance with input
`from a zoom operation key 2310. The zoom control circuit
`2307 also controls an enlargement processing circuit 2309
`through a switch 2308. The optical zoom control circuit
`2306 controls the lens optical system 2301 to perform
`optical zoom under the control of the zoom control circuit
`2307. The enlargement processing circuit 2309 executes
`electrical zoom processing under the control of the zoom
`
`[0006] An electrical zoom ON/OFF switch 2311 turns
`on/off the electrical zoom function. When the electrical
`zoom ON/OFF switch 2311 is OFF, the switch 2308 is
`turned off to inhibit the enlargement processing circuit 2309
`from executing enlargement processing. When the electrical
`zoom ON/OFF switch 2311 is ON, the switch 2308 is turned
`on to cause the enlargement processing circuit 2309 to
`execute enlargement processing. The electrical zoom
`ON/OFF switch 2311 is prepared for users who dislike a
`degradation in image quality due to enlargement by electri(cid:173)
`cal zoom so that photographing can be executed while
`setting the zoom control mode for only optical zoom.
`
`[0007] A memory 2 (2312) stores image data which has
`undergone camera signal processing and enlargement pro(cid:173)
`cessing. A recording signal processing circuit 2313 executes
`compression-coding, recording format generation, error-cor(cid:173)
`recting coding, or recording modulation coding for the
`image data read out from the memory 2 (2312). A recording
`medium 2314 records the image data processed by the
`recording signal processing circuit 2313.
`
`[0008] The optical zoom and electrical zoom operations in
`the image recording apparatus shown in FIG. 26 will be
`described next.
`
`[0009] FIG. 27 is a view showing the optical zoom and
`electrical zoom operations in the image recording apparatus
`shown in FIG. 26. A frame 2401 indicates the range of the
`object (including a tree, a skier, and a cloud) of image data
`stored in the memory 1 (the frame 2401 indicates the view
`angle). The frame 2401 becomes large as the optical zoom
`shifts to the wide-angle side. More specifically, in a zoom
`control region 2406 for optical zoom, when the lens optical
`system 2301 is controlled to the maximum wide-angle state,
`the frame 2401 becomes large. At that view angle, a wide(cid:173)
`angle image 2403 is obtained. Accordingly, the image data
`of the object indicated by the wide-angle image 2403 is
`stored in the memory 1. In the zoom control region 2406,
`when the lens optical system 2301 is controlled to the
`maximum telephoto state, a telephoto image 2404 is
`obtained at the view angle corresponding to the frame 2401.
`Accordingly, the image data of the object indicated by the
`telephoto image 2404 is stored in the memory 1. As
`described above, the image recording apparatus shown in
`FIG. 26 executes the zoom operation by changing the
`photographing view angle (the range of the frame 2401) by
`controlling the lens optical system 2301.
`
`[0010] On the other hand, in a zoom control region 2407
`for electrical zoom, a partial area indicated by a frame 2402
`is extracted and enlarged from the image data of the object
`within the frame 2401 in the maximum telephoto state,
`thereby obtaining an electrical zoom image 2405.
`
`[0011] At this time, the image quality degrades to some
`extent in electrical zoom processing, as described above.
`Especially, when the magnification ratio of electrical zoom
`is high, the image quality largely degrades. To prevent this,
`the magnification ratio of electrical zoom is generally lim(cid:173)
`ited by defining an upper limit value.
`
`[0012] There is also disclosed an information recording
`apparatus (image recording apparatus) capable of preventing
`any unintentional degradation in image quality by excessive
`
`AVER EXHIBIT 1006
`Page 29 of 46
`
`

`

`US 2004/017 4444 Al
`
`Sep.9,2004
`
`2
`
`electrical zoom processing (e.g., Japanese Patent Laid-Open
`No. 2001-197347). In this apparatus, when electrical zoom
`processing may cause an undesirable degradation in image
`quality, the electrical zoom processing is restricted, or a
`warning is displayed.
`
`[0013] More specifically, the information recording appa(cid:173)
`ratus comprises an image sensing unit which senses an
`image, and an electrical zoom processing unit which
`acquires a selected image within a predetermined range from
`the image obtained by the image sensing unit and converts
`the selected image into an output image having a predeter(cid:173)
`mined number of pixels. The electrical zoom processing unit
`has a selection range condition setting function of setting a
`condition to be satisfied by the selected image by referring
`to the number of pixels of the output image. The apparatus
`further comprises a notification unit which notifies a user
`that the condition is not satisfied when the selected image
`does not satisfy the condition. With the above-described
`arrangement, when electrical zoom processing may cause an
`undesirable degradation in image quality, the electrical
`zoom processing can be restricted, or a warning can be
`displayed.
`
`[0014] Some signal processing apparatuses and image
`data generation apparatuses, which have come along in
`recent years, record reduced image data smaller than the
`image size of the image sensing element in some recording
`modes, as in the moving image photographing mode of a
`digital video camera having a still image photographing
`function. In such signal processing apparatuses and image
`data generation apparatuses, even when photographing is
`performed by setting for only optical zoom, the image data
`is further reduced electronically and recorded in some
`recording modes. That is, setting for electrical zoom is not
`done in consideration of the electronic reduction. In other
`words, it is inconsistent with the primary purpose of the
`photographing mode which prevents, by electrical zoom
`ON/OFF setting, the degradation in image quality by
`enlargement processing.
`
`SUMMARY OF THE INVENTION
`
`[0015] The present invention has been made in consider(cid:173)
`ation of the above-described problem, and has as its main
`object to provide a signal processing apparatus and an image
`data generation apparatus, which allow a user to select a
`zoom mode with a small degradation in image quality in
`electrical zoom processing.
`
`[0016] According to an aspect of the present invention,
`signal processing apparatus which changes a magnification
`ratio of an image acquired by image sensing means, char(cid:173)
`acterized by comprising: first signal processing means for
`executing first signal processing for reducing an image
`sensing signal with respect to a scanning range of an
`obtained image sensing surface when the magnification ratio
`of the image is to be increased; second signal processing
`means for executing second signal processing different from
`the first signal processing to change the magnification ratio
`of the image; first detection means for detecting selection of
`one of a telephoto side and a wide-angle side; second
`detection means for detecting a limit of an increase in
`magnification ratio of the image by the first signal process(cid:173)
`ing; and selection means for selecting one of a first mode and
`a second mode; wherein when the first mode is selected by
`
`the selection means, and the first detection means detects
`that the telephoto side is continuously selected, the first
`signal processing is executed, and when the second detec(cid:173)
`tion means detects that the increase in magnification ratio of
`the image by the first signal processing has reached the limit,
`the second signal processing is subsequently executed to
`enlarge the image, and when the second mode is selected by
`the selection means, even when the second detection means
`detects that the increase in magnification ratio of the image
`by the first signal processing has reached the limit, and the
`first detection means then detects that the telephoto side is
`continuously selected, the second signal processing is inhib(cid:173)
`ited.
`[0017] According to another aspect of the present inven(cid:173)
`tion, an image recording apparatus characterized by com(cid:173)
`prising a signal processing apparatus of the present inven(cid:173)
`tion; and optical variable magnification means for changing
`a view angle of an object image formed on a light-receiving
`surface of image sensing means, wherein variable magnifi(cid:173)
`cation by the optical variable magnification means is
`executed in first signal processing.
`[0018] According to further aspect of the present inven(cid:173)
`tion, an image data generation apparatus which generates
`image data on the basis of image sensing data photographed
`by an image sensing element, characterized by comprising:
`recording mode control means for controlling a first record(cid:173)
`ing mode in which image data in a first image format having
`a first image size corresponding to the number of pixels of
`the image sensing element is generated and recorded, and a
`second recording mode in which image data in a second
`image format having a second image size smaller than the
`first image size is generated and recorded; electronic vari(cid:173)
`able magnification means for executing electronic variable
`magnification processing in generating the image data in the
`second image format in the second recording mode; maxi(cid:173)
`mum variable magnification ratio setting means for setting a
`maximum variable magnification ratio in the electronic
`variable magnification means; and variable magnification
`ratio change indication means for indicating a change in
`variable magnification ratio while defining, as a maximum
`value, the maximum variable magnification ratio set by the
`maximum variable magnification ratio setting means,
`wherein the maximum variable magnification ratio setting
`means sets, as the maximum variable magnification ratio, a
`maximum variable magnification ratio candidate selected by
`a user from a plurality of types of maximum variable
`magnification ratio candidates, and the variable magnifica(cid:173)
`tion ratio change indication means indicates, as a boundary
`line, a value of a maximum variable magnification ratio
`candidate whose variable magnification ratio is lower than
`that of the maximum variable magnification ratio.
`[0019] According to yet further aspect of the present
`invention, a signal processing method of changing a mag(cid:173)
`nification ratio of an image acquired by image sensing
`means, characterized by comprising: a first signal processing
`step of executing first signal processing for reducing an
`image sensing signal with respect to a scanning range of an
`obtained image sensing surface when the magnification ratio
`of the image is to be increased; a second signal processing
`step of executing second signal processing different from the
`first signal processing step to change the magnification ratio
`of the image; a first detection step of detecting selection of
`one of a telephoto side and a wide-angle side; a second
`
`AVER EXHIBIT 1006
`Page 30 of 46
`
`

`

`US 2004/017 4444 Al
`
`Sep.9,2004
`
`3
`
`detection step of detecting a limit of an increase in magni(cid:173)
`fication ratio of the image by the first signal processing step;
`and selection step of selecting one of a first mode and a
`second mode; wherein in the selection step, when the first
`mode is selected, and it is detected in the first detection step
`that the telephoto side is continuously selected, the first
`signal processing is executed, and when it is detected in the
`second detection step that the increase in magnification ratio
`of the image by the first signal processing has reached the
`limit, the second signal processing is subsequently executed
`to enlarge the image, and when the second mode is selected,
`even when it is detected in the second detection step that the
`increase in magnification ratio of the image by the first
`signal processing has reached the limit, and it is then
`detected in the first detection step that movement from the
`wide-angle side to
`the telephoto side is continuously
`selected, the second signal processing is inhibited.
`[0020] According to another aspect of the present inven(cid:173)
`tion, an image data generation method for generating image
`data on the basis of image sensing data photographed by an
`image sensing element, characterized by comprising: a
`recording mode control step of controlling a first recording
`mode in which image data in a first image format having a
`first image size corresponding to the number of pixels of the
`image sensing element is generated and recorded, and a
`second recording mode in which image data in a second
`image format having a second image size smaller than the
`first image size is generated and recorded; an electronic
`variable magnification step of executing electronic variable
`magnification processing in generating the image data in the
`second image format in the second recording mode; a
`maximum variable magnification ratio setting step of setting
`a maximum variable magnification ratio in the electronic
`variable magnification step; and a variable magnification
`ratio change indication step of indicating a change in vari(cid:173)
`able magnification ratio while defining, as a maximum
`value, the maximum variable magnification ratio set in the
`maximum variable magnification ratio setting step, wherein
`in the maximum variable magnification ratio setting step, a
`maximum variable magnification ratio candidate selected by
`a user from a plurality of types of maximum variable
`magnification ratio candidates is set as the maximum vari(cid:173)
`able magnification ratio, and in the variable magnification
`ratio change indication step, a value of a maximum variable
`magnification ratio candidate whose variable magnification
`ratio is lower than that of the maximum variable magnifi(cid:173)
`cation ratio is indicated as a boundary line.
`[0021] According to another aspect of the present inven(cid:173)
`tion., a program for a signal processing apparatus which
`changes a magnification ratio of an image acquired by image
`sensing means, characterized by comprising: a first signal
`processing step of executing first signal processing for
`reducing an image sensing signal with respect to a scanning
`range of an obtained image sensing surface when the mag(cid:173)
`nification ratio of the image is to be increased; a second
`signal processing step of executing second signal processing
`different from the first signal processing step to change the
`magnification ratio of the image; a first detection step of
`detecting selection of one of a telephoto side and a wide(cid:173)
`angle side; a second detection step of detecting a limit of an
`increase in magnification ratio of the image by the first
`signal processing step; and selection step of selecting one of
`a first mode and a second mode; wherein the signal pro(cid:173)
`cessing apparatus is caused to, in the selection step, when the
`
`first mode is selected, and it is detected in the first detection
`step that the telephoto side is continuously selected, execute
`the first signal processing, and when it is detected in the
`second detection step that the increase in magni