`
`Ex. GOOG 1030
`
`EX. GOOG 1030
`
`
`
`
`
`

`

`United States Patent [19]
`Matsumoto et 31.
`
`US005796428A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,796,428
`Aug. 18, 1998
`
`[54] ELECTRONIC PHOTOGRAPHY SYSTEM
`
`[75] Inventors: Kiyoshi Matsumoto. Kokubunji:
`Yuichi Yagawa‘ chigasakt Kazuto
`Senda~ HaChiOjL all of Japan
`
`.. 348/‘279
`3/1995 Tagami et a1. ..
`5.402.171
`348/231
`8/1995 Aoki
`5,438.359
`353/342
`5449401 3/1995 Pamlski e131 -
`369/272
`5.448.550
`9/1995 Mok et a1.
`$600,844 2/1997 Shaw et a1. ........................... .. 395/800
`
`[73] Assignee: Hitachi, Ltd.. Tokyo. Japan
`
`[21] APPL NOJ 325,095
`
`[22] Filed:
`Oct. 17, 1994
`[30]
`Foreign Application Priority Data
`
`Oct. 21, 1993
`
`[JP]
`
`Japan .................................. .. 5-263354
`
`FOREIGN PAT
`A-3-26l935 of0000 Japan.
`406110107 4/1994 Japan ................................... .. 348/820
`
`DOCUMENTS
`
`OTHER PUBLICATIONS
`“A Microcomputer-Based Image Database Management
`System”; Prasad et :11; IEEE Transactions. vol. IE-34 #1.
`Feb‘ .87. pp. 83_88_
`
`[51] Int. Cl.6 ................................................... .. H04N 5/222
`[52] US. Cl. ........................ .. 348/231; 348/588; 348/722;
`35819091
`
`_
`l
`_
`Pm'W-Y EMMEPY'C‘OI R- KOS‘f‘k
`Attorney, Agent, or Fzrm—Antonell1. Terry. Stout & Kraus.
`
`[58] Field of Search ............................... .. 358/9091. 906.
`358/335. 310; 348/232. 208. 231. 239.
`589. 722. 578. 563. 588. 714. 718: H04N 5/255.
`5/262_ 5/265_ 5/272. 5/273
`
`[56]
`
`References Cited
`
`U'S' PATENT DOCUMENTS
`1/1988 Sawyer .................................. .. 358/342
`4,717,971
`4.888.648 12/1989 Takeuchi et a].
`358/335
`4,943,866 7/1990 Barker et a1.
`348/722
`5,012,334
`4/1991 Etra ............... ..
`348/722
`343/231
`59437816 8/1991 Nam‘)? 31- -
`1?;
`5222115131 ‘
`358/906
`6/1993 Sakai ............. ..
`512311501
`.. 358/403
`5,270,831 12/1993 Pamlski eta]
`$295,077 3/1994 Fukuoka ................................ .. 348/231
`
`LLP
`[57]
`
`ABSTRACT
`
`An electronic photography system which displays an image
`shot by an electronic image capturing unit. The electronic
`image capturing unit includes an optical path. an image
`pickup for converting a focused image to electronic image
`data. an information inputting device for adding attribute
`data to the image data- and a Home6 for Storing thc imag?
`data and attribute data in mutually associated relationship.
`The system further includes the unit for classifying the
`image data on the basis of the attribute data for gouping the
`image data into album groups. and selecting. the album
`group to display the image data included in the selected
`album‘
`
`19 Claims, 46 Drawing Sheets
`
`103
`S
`mar
`PlCKUP
`
`101
`
`104
`
`108
`
`109
`
`STORAGE .......... .49 [NTERFACE
`UNIT
`=
`
`STSSGGE
`
`102
`-'
`
`1%2
`DlSPLAY
`Como/115R
`
`123
`
`D‘SPW
`
`.
`
`-
`E
`107/9.) CLOCK
`i
`
`:
`‘-
`1052
`:
`
`COMMAND
`[NPUT
`
`l
`
`[again/009.9111 ------
`
`|MAGRENG
`=
`CAPTU
`“2/106
`UNIT
`CONTROLLER
`
`,
`
`COMMAND
`INPUT
`
`.
`';
`;
`'a
`
`L ........
`...............
`110
`
`/
`
`=
`
`STORAGE
`USPLAY
`cow/R0018
`(
`
`a
`*MAGE STORAGE/L.‘?l$?P/5X.P'FFT.3
`
`.
`
`.................. -.
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 1 0f 46
`
`5,796,428
`
`N2
`
`m:N:_,_me8.
`
`>5mw5
`
`mmjomEoo
`
`.2,de\
`
`mmjomkzoo
`
`mw<mo._.w
`
`......................................................................................
`
`Eo:
`
`uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu
`
`532.
`
`0253.55
`
`:2:
`
`Ejomkzoo
`
`m9
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`
`
`
`

`

`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 2 0f 46
`
`5,796,428
`
`FIG. 2
`
`207
`S
`EVF
`(ELECTRIC VIEW FINDER)
`
`2m
`
`202
`
`203
`Q
`
`—* ' * ' — ' a '
`
`' * ' w" CCD
`
`T0 STORAGE
`
`UNIT104
`
`DlAPHRAGM
`CONTROLLER
`
`FOCUS
`CONTROLLER
`
`coo
`CONTROLLER
`
`>
`
`204
`
`1
`
`S
`
`205
`
`1
`
`S
`
`206
`
`I
`
`4
`
`FROM CONTROLLER 106
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 3 0f 46
`
`5,796,428
`
`FIG. 3
`
`302
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 4 0f 46
`
`5,796,428
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 5 of 46
`
`5,796,428
`
`FIG.
`
`5
`501
`
`r "@1502
`
`m: C552:
`
`282E201
`+11% ~.
`
`28% Egg Z55
`
`5%
`205% as;
`
`
`
`.‘ Magi .
`
`452mm
`
`Z5282
`
`@2528
`
`cooznjiu
`
`to @1052 m
`
`
`
`“=5. wmomsm V
`
`‘
`
`' END
`
`Z
`\
`)
`505
`
`503 504
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 6 0f 46
`
`5,796,428
`
`HONEY MOON
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 7 of 46
`
`5,796,428
`
`NARITA AIRPORT
`
`1_]l[ mus WITH MICKEY MOUSE
`
`40].
`
`7
`
`1
`
`
`
`IN sAN FRANCISCO GRAND CANYON HAVING A GAME AT CARDS ENJOYING THE suNusHT
`
`
`
`
`
`
`
`25 s 15] 2E]
`
`
`
`
`
`
`
`
`
`
`
`
`
` 55} IN sAN DIEGO :N A PLANE FOR MEXICO so ON MY WAY HOME 72 Y
`
`SHOPPING IN BEVERLEY HILLS 24]
`YEARNING HOLLYWOOD
`3g
`
`E
`
`1;)
`
`604
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 8 0f 46
`
`5,796,428
`
`
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 9 0f 46
`
`5,796,428
`
`FIG. 9
`
`\
`
`i?éfNFR/XNCISCO @ mg
`
`I --
`11%
`n A
`nmmouuun Q * j
`
`_1
`
`’
`
`IN CHAINA TOWN
`
`mum!
`
`TO THE GRAND CANYON
`
`:75‘ "
`
`b 1}.
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 10 of 46
`
`5,796,428
`
`
`
`\ \ m <55 $561
`
`v P00?
`
`2 .GI
`
`
`
`_ (H4O mmDhuE
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 11 0f 46
`
`5,796,428
`
`m2;><oIHZOEm<m>mmDHoE
`
`I.O_u_
`
`<55mo<§mSmEC<
`
`5:
`
`(.20mehoE
`
`
`
`
`
`~o_rcorpmop?«oppmop—
`
`<H<Q
`
`mum—>52
`
`No:
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`
`
`

`

`US. Patent
`
`Aug. 13, 1998
`
`Sheet 12 0f 46
`
`5,796,428
`
`FIG.
`
`12
`
`55351325
`01
`02
`03
`04
`05
`06
`07
`08
`09
`10
`1 1
`12
`13
`14
`15
`16
`11
`13
`19
`20
`21
`22
`' 23
`24
`25
`26
`
`YEAR
`92
`92
`92
`92
`92
`92
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`93
`
`MONTH
`12
`12
`12
`12
`12
`12
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`01
`02
`02
`
`DAY
`21
`24
`24
`24
`25
`23
`01
`01
`01
`01
`02
`02
`02
`13
`'13
`13
`14
`14
`14
`14
`14
`14
`15
`15
`05
`05
`
`TIME
`1344
`2132
`2150
`2210
`0753
`0926
`0534
`0627
`1009
`1013
`1326
`1342
`1441
`1022
`1635
`1922
`0855
`1042
`1 155
`1331
`1911
`2033
`1125
`1356
`1013
`0332
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 13 of 46
`
`5,796,428
`
`
`
`
`
`5W P wow F mow P wow: wow:
`
`\ \
`
`L \ ‘AI,
`V 62
`
`
`
`
`2 E3 #555 K 3 m <20 @355 N <20 $56K _ <EQ EDGE 59m: 2%
`
`V 5: 2.5
`N? V
`52
`
`
`
`
`
`
`
`1 >3 I502 Em; mwmznz M2365 .ZEL E5232 2.5
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 14 0f 46
`
`5,796,428
`
`FIG. 14
`
`TOTAL
`HLM
`NUMBER PICTURE
`NUMBER
`56
`329
`145
`51
`238
`32
`26
`1 70
`261
`82
`
`01
`02
`03
`04
`05
`06
`0?
`08
`09
`10
`
`YEAR
`
`MONTH
`
`DAY
`
`92
`92
`92
`92
`92
`92 v
`93
`93
`93
`93
`
`04
`07
`08
`08
`I 0
`12
`02
`03
`05
`06
`
`12
`28
`I 0
`1 8
`25
`02
`06
`1 5
`08
`30
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 13, 1998
`
`Sheet 15 0f 46
`
`5,796,428
`
`2 PM: mug
`
`7
`
`\ N 5: M65
`
`mp .OE
`
`mom?
`
`\
`
`
`
`_ 5: 25.2 _
`
`59
`
`mom, V
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 16 0f 46
`
`5,796,428
`
`Pom:
`
`wow?
`
`mow?
`
`zmwti 128 IE; moi IE5, moi mwmssz
`
`
`
`
`
`
`
`
`
`
`5Z6 59m E50 59: e?zoumoz ._<Q_Em> M65 .59 w2<z 25? E522 23,2
`
`
`
`
`
`
`
`2. .0E
`
`mom?
`
`
`
`V V v SW: 8%: SW: 5W:
`
`
`
`
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 17 of 46
`
`5,796,428
`
`\ F
`
`M65. } J \
`
`
`
`T 5: EDGE Q \ _~ 5: E392 _ 5: $581 595:
`
`C .QE
`
`Not 5: wo<¢
`
`mom?
`
`
`
`
`
`10,60 harm 592.. ws_<z $5 59232 wo<¢
`
`
`
`
`8V: 3V: mwt
`
`
`
`Ex. GOOG 1030
`
`

`

`U.S. Patent
`
`Aug. 18, 1998
`
`Sheet 18 0f 46
`
`5,796,428
`
`2 .0E
`
`zoEwom >
`
`29.58 x
`
`
` @2325 E5 222E E5 @252": #56: $5232 2E mwmznz mzém V V V V V V 82 82 8E 82 8m: 6m:
`mum-232
`
`6: $56: V 8:
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 19 0f 46
`
`5,796,428
`
`FIG. 19
`
`SETTING OF CONDITIONS ON
`I901 M WHICH ALBUM IS GENERATED
`
`1902M SELECTION OF PICTURES
`
`1903 N GENERATION OF ALBUM LIST
`
`SAVING OF ALBUM LIST
`
`1905
`
`Y
`
`1906 /\~ / DISPLAY OF ALUBM
`
`I 907
`
`HAS ALBUM LIST
`BEEN CHANGED’?
`
`I908 "~ / SAVING OF ALBUM LIST
`
`I
`
`1
`
`END
`
`I
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 13, 1998
`
`Sheet 20 of 46
`
`5,796,428
`
`FIG. 20
`
`2001
`
`2002
`
`2003
`
`READ OUT OF ALBUM LIST
`
`DISPLAY OF ALBUM
`
`HAS ALBUM LIST
`BEEN CHANGED?
`
`Y
`
`
`
`
`
`
`2004
`
`SAVING OF ALBUM LIST
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 21 0f 46
`
`5,796,428
`
`120
`
`120
`
`120
`
`120
`
`120
`
`120
`
`310
`
`310
`
`110
`
`310
`
`310
`
`
`
`IDLD
`
`mm 480
`
`480
`
`
`
`OSITIONYPOSITION
`DISPLAY
`
`ANGLEXPmun-ml
`
`
`SCALINGROTATIONALDISPLAYFACTOR
`
`480Inna-mm“ 480m 480mm
`TRIMMINGTRIMMINGTRIMMINGTRIMMING ’5
`
`
`
`
`
`
`
`
`
`NUMBERNUMBER
`
`
`
`
`
`720
`
`720
`
`720
`
`720
`
`720
`
`
`
`o—Nm
`vm >00
`'—-—
`u—v— NNNN
`
`r~ ts r~ r- N r» 1~ rx
`0 o oo o o o 0
`
`EE
`:58mo.
`
`XE
`EE
`:8mo.
`
`m(
`
`I:
`3
`}—-
`2o.
`
`FILM
`
`
`
`FIG.21
`
`I'll“
`FRAMENUMBER
`
`"I'llll"
`PAGENUMBER
`
`
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1993
`
`Sheet 22 0f 46
`
`5,796,428
`
`
`nggg?
`PAGE
`PAGE
`COLOR
`NUMBER ATTRIBUTE
`
`“-—
`
`
`“mu-—
`”m
`
`
`
`
`
`
`
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 23 0f 46
`
`5,796,428
`
`2mm.E.<n_
`
`:_<o_._.mm>
`
`met<m
`
`mm>Oo.rZOmEmw>oohzomn.IZFZONEOI._<o_kmm>
`
`
`
`
`$0.50I._.O_\$mo<mIPQS>w0<mmumEDZ
`
`
`
`
`m=2<z23m4<mumEDZ23m4<
`
`mm.07.
`
`m0<m._<._.O._.
`
`mem—Dm
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`
`
`
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 24 of 46
`
`5,796,428
`
`FIG. 24
`
`93.01.14
`
`93.01.15
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 25 of 46
`
`5,796,428
`
`t unmunmmn
`
`93.01.14 ;;
`
`2502
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 26 of 46
`
`5,796,428
`
`FIG. 26
`
`104
`
`107
`
`2601
`
`A
`
`
` CONTROLLER
`
`IMAGE
`CAPT URiNG
`UNIT
`
`....................................................................
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 27 of 46
`
`5,796,428
`
`
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 28 0f 46
`
`5,796,428
`
`FIG. 28
`
`2801
`
`YEAR
`
`NUMBER
`
`TI00HSGINuHmm00HMSGNTO0HSERUTm0:
`
`GNTOOHSG.N.
`TIME
`DAY
`
`SHOOTING
`ATTITUDE
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 29 0f 46
`
`5,796,428
`
`310
`
`IIO 3IO
`
`
`
`FACTORANGLEXPOSITIONYPOSITION
`
`
`
`
`
`
`
`SCALINGROTATIONALDISPLAYDISPLAY
`
`2901
`
`FIG.29
`
`720
`
`720
`
`720
`
`mm:-mm“mm:-
`“mun-
`nun-u
`
`mun
`
`EX. GOOG 1030
`
`mummm PAGE
`
`HEIGHT (H)
`
`0 Z2 g[
`
`I
`I.
`(D
`.2.
`
`22 E'
`
`— 0 Z2 :
`
`2:[C
`P-
`
`
`STARTXSTARTYPOSITIONPOSITION
`
`
`
`FILMPICTURENUMBERNUMBERNUMBERNUMBER
`FRAME
`
`0 Z2 g(
`
`I:
`1...
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 30 0f 46
`
`5,796,428
`
`FIG. 30
`
`”"93.
`
`34'
`
`[IIIIIIIIIIIIIIIU
`
`I22 @-
`m2-
`
`:
`._..
`
`_nnunnuu
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 31 0f 46
`
`5,796,428
`
`FIG. 31
`
`104
`
`...................................................................
`
`IMAGE
`PICKUP
`
`STORAGE
`UNIT
`
`
`
`
`
`
`
`3101
`
`107
`
`INPUT DEVICE
`OF CAPTURING
`POSITION
`
`COMMAND
`INPUT
`
`6
`
`
`IMAGE
`CAPTURING
`UNIT
`CONTROLLER
`
`
`
`
`
`
`
`
`IMAGE CAPTURING UNITE
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 32 of 46
`
`5,796,428
`
`02:.OOIm
`
`maxim
`
`mmDHUE
`mSE.HIkZOEm<m>
`
`mumEDz
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 33 0f 46
`
`5,796,428
`
`FIG. 33
`
`3301
`
`PICTURE
`NUMBER
`
`YEAR
`
`MONTH
`
`—I N
`
`O—a
`
`GODMN
`
`(D(O(DDm(A)DJOJ0)M
`
`(D0.)
`
`93
`
`CD0)
`
`(O(A)
`
`CD0.)
`
`tono
`
`(O(.0
`
`d.n‘
`
`‘ N
`
`NO
`
`Nm
`
`NA
`
`NU1
`
`N5’
`
`_| w
`
`—l A
`
`_A A.
`
`
`
`(D0.) %%%E%EI%IEI%
`
`m—
`
`m—
`m—
`
`d9m(A)N
`
`m—
`__
`Isaa—
`
`
`
`
`
`
`
`
`
`Ina--
`
`I922
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Ex. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 34 of 46
`
`5,796,428
`
`
`
`OZEEEH02:25:”;02.22;:
`
`hoGI
`
`
`
`0235.5.I._b_>>
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 35 of 46
`
`5,796,428
`
`FIG. 35
`
`104
`
`
`
`
`INPUT
`
`DEVICE OF
`CAPUTURER
`
`
`6
`
`3501
`
`107
`
`
`IMAGE
`
`CAPTURING
`UNIT
`
`CONTROLLER
`
`
`
`
`....................................................................
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 13, 1998
`
`Sheet 36 of 46
`
`5,796,428
`
`
`
`
`
`room<._.<D
`
`
`<._.<om0<$=mHDmEt/x1‘
`
`mm.0_u_
`
`
`
`<._.<n_meHUE
`
`all
`
`
`
`mmIm<m00kOImmi;aIHZOEm<m>mm3._.o_n_
`
`mum—232
`
`EX. GOOG 1030
`
`
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 13, 1998
`
`Sheet 37 of 46
`
`5,796,428
`
`FIG. 37
`
`
`
`TIME
`
`83RS'3"‘
`
`1013
`
`1326
`
`1312
`
`—l
`
`1022
`
`11135
`
`1922
`
`PICTURE
`NUMBER
`
`E3
`
`-A N
`
`['0OJ
`
`N{b
`
`
`
`AY
`
`D
`
`.a
`
`24
`
`24
`
`28
`
`02
`
`02
`
`13
`
`13
`
`_s_n#1)
`
`
`
`
`
`
`
`YEAR
`
`MONTH
`
`92
`
`92
`
`92
`
`92
`
`92
`
`12
`
`12
`
`12
`
`12
`
`93 m-
`as m-
`
`93
`
`(.0OJ
`
`(D0.)
`
`0—-§
`
`
`
`
`
`3701
`
`PHOTOGRAPHER
`
`
`
`
`
`
`
`
`EX. GOOG 1030
`
`
`
`1|1II11111
`
`
`21133
`
`1125
`
`1013
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1993
`
`Sheet 38 0f 46
`
`5,796,428
`
`ANGLEXPOSITIONYPOSITIONSCALINGFIOTATIONALDISPLAYDISPLAYFACTOR
`
`
`
`
`
`
`
`TRIMMINGTRIMMINGTRIMMINGTRIMMING
`
`
`’5
`SEE
`(tn
`I—O
`(On.
`
`
`
`
`
`FIG.38
`
`
`
`310
`
` mm“mm 110
`
`mm“Emmi-mm
`
`«mm-n“mmm
`
`
`
`*5
`EE
`:58L00.
`L110:
`$.23
`:52 8' 8
`_D
`0.2
`m
`
`'5
`
`3 (“3:
`
`ELL:
`4% N N N N N N N
`E:2
`C:
`mm
`Eco
`<2
`03::
`“-2
`
`N m V
`
`N m
`
`
`
`LU
`33
`2%
`l3
`2
`
`N N N
`
`
`
`
`
`
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 39 0f 46
`
`5,796,428
`
`
`
`
`
`..._..._Z.3..>.<J.n._.m._O\w0<mOHm.w0¢.2_....................................................
`
`
`
`
`
`:23m$5.058%.zzzémta"EdmafioéofimWmo<§OH...........m.............
`
`.n.................................................................w.........................T.::EE.
`533.“;:2:mommEzoumoEEmo
`
`
`
`
`
`
`
`
`commFowmmommvommmommNomm
`
`A........$395.85,:
`
`92228m
`
`5062k5%
`
`.........................................................................................................
`
`
`
`
`
`momm.:2:ozEDES83>:......
`
`£955
`
`$440528
`
`mo<mohm
`
`:23
`
`moi):
`
`mmemmE—zoowo
`
`8552.m...........:23ozasES
`
`
`
`m0<_2_EOE”.
`
`Eama
`
`5.30528
`
`5&2.
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`
`
`
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 40 of 46
`
`5,796,428
`
`FIG. 40
`
`SHUTTER 1
`
`SHUTTER 2
`
`
`
`2'""""""""""" .MAGE IMAGE REFERENCE
`_______________________ -2 -I IMAGE
`
`
`
`
`
`IMAGE
`+1
`
`IMAGE s
`+2
`
`__________
`
`s
`
`4003
`
`4001
`
`4004
`
`4005
`
`FIG. 41
`
`SHUTTER
`
`
`
`........................
`
`i REFERENCE
`........................
`E
`IMAGE
`
`
`
`-1 -2 —2
`IMAGE IMAGE IMAGE IMAGE a
`
`a
`
`............
`
`4101
`
`4102
`
`4103
`
`4104
`
`4105
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 41 0f 46
`
`5,796,428
`
`womv N+<._.<n_T(bed
`
`
`
`momvvamvmomv
`
`4<F2mmmn§04<F2mmmmm5u_<_.r2mmwn_m_o._<F2mmmmn:0
`
`
`
`
`
`m=>:._.aIHZOEm<m>mmDHUE
`
`mum—232
`
`
`
`EX. GOOG 1030
`
`NV.
`OE
`
`<H<QmeFOE
`
`
`mo<§_ Fowv
`
`
`“(CodmQmemEmmv<._.<Q
`
`
`
`
`
`wk:91...:
`
`<._.<Q
`
`Ex. GOOG 1030
`
`
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 42 0f 46
`
`5,796,428
`
`
`
`AL ANGL
`
`ROTA
`
`S F
`
`
`
`
`
`
`DISPLAYPOSITIONYPOSITIONDISPLAYNUMBER
`ACTOREXlag-mum
`
` TRIMMINGTRIMMINGTRIMMINGTRIMMINGSTARTXSTARTYPOSITION
`
`FILM
`
`o
`N
`N
`
`IL0
`
`120
`
`310120
`
`CO
`NN
`{‘1‘
`
`WEI-III!mm"
`"mum-In
`
`EX. GOOG 1030
`
`POSITION
`
`3:
`Lu
`a:
`
`Lu
`c:
`
`D’
`
`2:
`— 2a
`
`.
`
`1
`:z
`1:
`Lu
`:0
`5.
`:3
`:2
`a:
`UJ
`CD
`12
`:3
`‘2
`
`NUMBER
`
`4301
`
`FIG.43
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 43 of 46
`
`5,796,428
`
`FIG. 44
`
`4401
`4405
`I:
`TO IMAGE
`:— ----------------------------------’-----------------------------------5
`:
`E STRAGE/DISPLAY
`_§_______U_EIT
`
`IMAGE
`PICK UP
`
`
`
`
`
`
`COMMAND
`INPUT
`
`
`
`IMAGE
`CAPTURING
`UNIT
`
`CWTROLLER
`
`........................................................................
`
`FROM
`IMAGE
`CAPTURING
`UNIT
`
`5
`
`;
`
`
`
` DISPLAY
`CONTROLLER
`
`
`AUDIO
`OUTPUT
`
`i
`g
`
`I
`
`IMAGE STRAGE/DISPLAY UNIT
`
`STORAGE/
`
`CONTROLLER
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 44 of 46
`
`5,796,428
`
`FIG. 45
`
`SHUTTER 1
`
`SHUTTER 2
`
`
`
`............................................... ................................................
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`US. Patent
`
`Aug. 18, 1998
`
`Sheet 45 of 46
`
`5,796,428
`
`3.0E
`
`(.53mm:POE
`
`
`
`.<53
`
`
`<._.<Dm0<_2_5.39””.th
`
`,
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`
`

`

`4701
`
`l‘
`<1"
`.
`(9
`E
`
`US. Patent
`
`Aug. 18, 1993
`
`Sheet 46 of 46
`
`5,796,428
`
`
`
`VOICE DATA
`
`SCALINGFACTOR 5 5 5
`
`2 2 2
`
`25
`
`
`DISPLAYxPOSITIONYPOSITION
`DISPLAY
`
`.m-.m-In. no I“.In. 310 no 310 no 310
`
`ROTATION AL ANGLE:-nnInIn25n
`
`
`{552
`
`
`
`
`
`
`
`
`
`0 E E
`
`EP
`0
`Z
`
`E2
`
`E 2
`
`*5
`2'51:
`2(7)
`{5'58
`(3x52
`_,___
`£255
`
`um:
`“25%
`r-E
`23
`0.2
`
`
`
`51' 9: a
`
`9:
`
`95
`32“
`“-3
`
`E.
`
`
`
`Ex. GOOG 1030
`
`2 c
`
`c
`mm
`303
`m?)
`H-2
`
`
`
`
`E
`Lam
`<2
`m:
`2
`
`Ex. GOOG 1030
`
`

`

`1
`ELECTRONIC PHOTOGRAPHY SYSTEM
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates to a so—called electronic
`photography system which can electronically record images
`on such a memory device as a semiconductor memory or a
`magnetic disk without using any films.
`Silver halide film is usually used for photograph. The
`exposures are developed and printed on a photographic
`paper for its appreciation. There exists a technique for
`displaying images printed on a photographic paper on a TV
`display. For example. disclosed in JP-A-3—261935 is a
`system in which picture images printed on a photographic
`paper are displayed on a monitor device together with
`information associated with the picture images. such as
`shooting days and shutter speeds.
`Meanwhile. an electronic still image capturing unit has
`been put in practical use in these years. This image capturing
`unit is designed to electronically record picture images on a
`memory device such as a semiconductor memory or a
`magnetic disk without using any films. In a reproduction
`mode of the image capturing unit.
`the recorded picture
`images can be reproduced on an external 'I'V display or can
`be transferred to a computer to be used for desk—top pub-
`lishing (DTP).
`Photographs for use by a single individual have been
`rapidly increased lately. Typically. pictures taken by an
`individual amount to several
`tens a year. Thus. several
`hundreds of pictures are produced in a family. In particular.
`a newly-wedded couple. a family having a newly born baby
`or a person who goes abroad. has many chances of taking
`pictures and thus the number of pictures taken per year
`amounts nearly to 500.
`It is highly troublesome to re-arrange and keep these
`pictures. which results in may cases in that such pictures are
`merely inserted into a pocket-type album or kept in different
`places. For this reason. even when it is desired to find some
`pictures. such finding is difficult. In addition. when pictures
`are kept in different places. such pictures tend to be lost
`undesirably.
`For the purpose of re-arranging and keeping pictures. in
`general. there is used such an album of a type that pictures
`are bonded on layout sheets. In this case. it is also possible
`to divide the pictures according to the theme or subject
`matter or to add some cements to them However. rear-
`ranging and keeping pictures requires a lot of labors. includ-
`ing classifying and rearranging the pictures and arranging
`the pictures on layout sheets in a beautiful and balanced
`layout. Further. such cements as associated dates and
`places are added to the pictures. When an user after taking
`pictures immediately conducts the above works. the works
`might not
`impose so burdensome feeling on the user.
`However. once such pictures are left alone and accumulated
`as not rearranged.
`it becomes troublesome to rearrange
`them. And such pictures contain many pictures which are not
`taken well. that is. out of focus or have improper exposure.
`It is an object of the present invention to provide an image
`recording/reproducing system which can eliminate the
`above problems in the prior art and can easily rearrange and
`keep photographic images and also can display the images
`for appreciation.
`
`SUMMARY OF THE INVENTION
`
`In order to solve the above problems. in accordance with
`the present invention. there is provided an image recording/
`
`5 796.428
`
`2
`
`reproducing system which comprises an image capturing
`unit. an image storage/display unit and means for applying
`attribute data to data of a picture image at the time of
`shooting the picture image is provided in the image captur-
`ing unit. Further. means for displaying the picture image on
`the basis of the attribute data in a picture display mode is
`provided in the image storage/display unit. The image
`capturing unit is based on such a system that a device such
`as a charge coupled device (CCD) is employed in place of
`silver halide film to capture the image data in the image
`capturing unit in the form of electronic data. Since the
`attribute data is made also in the form of electronic data. the
`
`image and attribute data can be systematically handled. The
`image storage/display unit includes a data storage for storing
`the image data therein and an image display for electroni-
`cally displaying the image data in the form of an album
`thereon.
`
`In accordance with first one of preferable aspects of the
`present invention. means for applying a date and a time of
`capturing the image data to the picture image data as the
`attribute data is provided in the image capturing unit.
`Further. the image storage/display unit classifies and dis—
`plays the image data on the basis of this attribute data.
`In accordance with second one of the preferable aspects of
`the present invention. an attitude detector for detecting an
`attitude of an image pickup is provided in the image cap-
`turing unit. and means for applying as attribute data to the
`image data the attitude of the image pickup at the time of
`capturing the image data is also provided in the image
`capturing unit. Further.
`the image storage/display unit
`changes an orientation of the image data to be displayed on
`the basis of the attitude attribute data.
`
`In accordance with third one of the preferable aspects of
`the present invention. a shooting place input part is provided
`in the image capturing unit. and means for applying the
`picture shooting place to the picture image data as attribute
`data is also provided in the image capturing unit. Further. the
`image storage/display unit classifies and displays the image
`data on the basis of the shooting place attribute data.
`In accordance with fourth one of the preferable aspects of
`the present invention. a image input part for inputting an
`image of the operator of the system is provided in the image
`capturing unit. and means for appending the image of the
`operator to the picture image data as an operator attribute
`data is also provided in the image capturing unit. Further. the
`image storage/display unit classifies and displays the image
`data on the basis of the attribute data.
`
`In accordance with fifth one of the preferable aspects of
`the present invention. means for shooting sub picture images
`at times in the vicinity of the moment that a main picture
`image is shot and applying the sub picture image data to the
`main picture image data as attribute data of the main image
`data. Further. in the image storage/display unit. a desired
`image is selectively displayed among the main or sub picture
`image data.
`In accordance with sixth one of the preferable aspects of
`the present invention. there is provided in the image cap—
`turing means for shooting sub images with shifted focuses at
`times in the vicinity of the moment of shooting a main
`picture image and applying the sub image data to the main
`picture image data as its attribute data. Further. in the image
`storage/display unit. a desired image is selectively displayed
`among the main or sub picture image data.
`In accordance with seventh one of the preferable aspects
`of the present invention.
`there is provided in the image
`capturing means for shooting sub images with shifted aper-
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`5 796.428
`
`3
`tures at times in the vicinity of the moment of shooting a
`main picture image and applying the sub image data to the
`main picture image data as its attribute data. Further. in the
`image storage/display unit. a desired image is selectively
`displayed among the main or sub picture image data.
`In accordance with eighth one of the preferable aspects of
`the present invention. there is provided in the image cap—
`turing means for recording voice data of a shooting place at
`times in the vicinity of the moment of shooting a picture
`image and applying the voice data to the picture image data
`as its attribute data. Further. when it is desired to display the
`picture image data.
`the image storage/display unit repro-
`duces the voice data of its attribute data.
`
`At the time of shooting the picture image. the attribute
`data on the picture image is applied to the picture image
`data. These picture and attribute data are temporarily stored
`in the storage within the image capturing unit and then
`transferred to the image storage/display unit. In the image
`storage/display unit. the received picture and attribute data
`are once stored. Further. the image storage/display unit
`generates album data for classification and display on the
`basis of the attribute data. When it is desired to display the
`picture data. the data is classified and displayed in the form
`of an album on the basis of the attribute and album data.
`Further. when image display is desired. the display format is
`changed on the basis of the attribute data.
`Upon arranging pictures. it is often that the pictures are
`classified according to themes such as. athletic meeting or
`travel abroad. However. as long as the pictures are treated by
`an individual. the pictures can be arranged on a time series
`basis. In other words. it is impossible to have such pictures
`of the same date on. e.g.. athletic meeting and travel abroad.
`Accordingly. when the pictures are arranged in its shooting
`date or time order. the pictures can also be automatically
`classified according to themes.
`In the first aspect of the invention. each time a picture
`image is shot. its date and time are previously recorded as
`attribute data. The image storage/display unit automatically
`classifies and displays the image data in the form of an
`album on the basis of the attribute data and album data
`generated based on the attribute data.
`A picture image has vertical and horizontal orientations.
`Thus. when it is desired to display such pictures in the form
`of an album. the orientations of pictures must be aligned so
`that the pictures can be displayed in their erecting form.
`In the second aspect of the invention. when a picture
`image is shot. an attitude of the image pickup is previously
`recorded as its attribute data. The image display displays
`such pictures thereon in such a manner that the pictures are
`automatically displayed in their erecting form.
`Although picture classification has been carried out based
`on the date and time attribute data in the first aspect. it is also
`considered to classify such pictures on the basis of their
`shooting places.
`In the third aspect. when a picture image is shot. its
`shooting place is previously recorded as its attribute data.
`The image storage/display unit automatically classifies and
`displays such pictures on the basis of the shooting place
`attribute data and album data generated based on the
`attribute data. For example. when it is desired to make an
`album of travel abroad. an entire album is produced in
`accordance with the first embodiment and pictures in the
`album are classified according to the shooting places. e.g..
`streets by means of the third embodiment.
`In the case where pictures are taken by different members
`in a family and left mixed. albums for the individual
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`4
`photographers can be made. In the fourth aspect. when a
`picture image is shot.
`its photographer is previously
`recorded as its attribute data. The image storage/display unit
`automatically display such pictures in the form of each of the
`individual photographer albums on the basis of the attribute
`data and album data generated based on the attribute.
`There often occurs such a case that some of persons as
`subjects close their eyes at
`the moment
`that
`the user
`depresses the shutter release. that is. missing of a good
`shutter-release timing. In the fifth aspect. sub picture images
`at times before and after the moment of the shutter release
`
`are stored and applied to the main picture image data of
`depression of the shutter release. When the main picture
`image taken at the moment of the shutter release is not
`acceptable. the image storage/display unit selects suitable
`one from the sub picture images and records in the album
`data indicative of the selection of the sub picture image in
`place of the main one. Thus. in the next display. the suitable
`sub picture image is displayed based on the album data.
`Some of pictures are sometimes taken with improper
`focus. In the sixth aspect. after a main picture is taken. sub
`picture images are taken with focuses shifted from the focus
`of the main picture image and added to the main picture
`image data as its attribute data. When the main picture image
`is taken with improper focus. the image storage/display unit
`selects suitable one of the sub picture images with focuses
`shifted and records data indicative of the selection of the
`suitable sub picture image in the album data in place of the
`main picture image. Thus. when the image is displayed in
`next time. the suitable sub picture image is displayed based
`on the album data.
`
`Some of pictures are sometimes taken with improper
`apertures. In the seventh aspect. after a main picture is taken.
`sub picture images are taken with apertures shifted from the
`aperture of the main picture image and added to the main
`picture image data as its attribute data. When the main
`picture image has an improper aperture. the image storage!
`display unit selects suitable one of the sub picture images
`and records data indicative of the selection of the suitable
`sub picture image in the album data in place of the main
`picture image. Thus. when the image is displayed in next
`time. the suitable sub picture image is displayed based on the
`album data.
`
`invention are still
`Pictures dealt with in the present
`pictures. However. when ambient sound at
`the time of
`shooting the picture or a voice comment is added to the
`picture. the picture becomes more impressive and fascinat-
`ing when reproduced later. In the eighth aspect. ambient
`sound ranging between the before and after the shutter
`release is stored and added to the picture image data as its
`attribute data. When the picture image is displayed. the
`image storage/display unit reproduces the voice data as the
`attribute data together with the picture image.
`The first to eighth aspects can be carried out respectively
`independently or in combination.
`In accordance with other aspect of the present invention.
`there is provided an image display method for displaying an
`image shot by an electronic image capturing unit. the image
`capturing unit comprising an optical imaging system. image
`pickup means for converting a focused image to electronic
`image data. data application means for applying attribute
`data to the image data. and storage means for storing therein
`the image data and attribute data in mutually associated
`relationship; the method comprising the steps of classifying
`the image data on the basis of the attribute data for album
`grouping. and selecting.
`in an image display mode.
`the
`
`EX. GOOG 1030
`
`Ex. GOOG 1030
`
`

`

`5.796.428
`
`5
`
`image data on every album basis to display a selected image
`data. In general. the attribute data include a picture number
`for uniquely designating the image.
`When displaying the image. the image and attribute data
`are collectively transferred from the storage means of the
`electronic image capturing unit to the second storage of the
`display device. a plurality of the image and attribute data
`collectively transferred are grouped into a single file list. and
`a file number for identifying a file is added to the file list.
`It is practical that an album includes a plurality of picture
`lists and each of the picture lists has at least the file number
`and picture number. When displaying pictures in each
`album. the image data are automatically retrieved from the
`film list and displayed on the display.
`Further. an album includes a plurality of pages and the
`picture lists are grouped on every page basis. Thus the user
`can see pictures on every page basis with the same feeling
`as to actually see a physical album.
`At the time of displaying the images. a plurality of the
`image data are retrieved from a file list on the basis of the
`file number and picture number of the picture list is dis~
`played on a display with respect to each page. at which time
`the image data to be displayed on the screen are processed
`on the basis of control data contained in the picture list.
`More in detail. the control data includes at least one of
`trimming control data for designation of image trimming.
`scaling control data for enlargement or reduction of the
`image and display position control data for control of a
`display position of the image in the display screen. Thus. the
`user can rearrange pictures and change the size or orienta—
`tion of the pictures in each page according to user’s pref—
`erence.
`
`The attribute data includes at least one of image shooting
`condition data. image shooting time data and user input
`identification data. the film number and picture number