Ex. GOOG 1031
`
`
`
`
`
`

`
`United States Patent [19]
`Parulski et al.
`
`US005633678A
`Patent Number:
`[11]
`[45] Date of Patent:
`
`5,633,678
`May 27, 1997
`
`[54] ELECTRONIC STILL CAMERA FOR
`CAPTURING AND CATEGORIZING IMAGES
`
`[75] Inventors: Kenneth A. Parulski; Thomas A.
`Napoli. both of Rochester; David M.
`Lewis, Waterport, all of NY.
`
`[73] Assignee: Eastman Kodak Company, Rochester.
`NY.
`
`"I‘IFF/EP, A Flexible Image Format for Electronic Still
`Cameras”, by Ken Parulski and George Lathrop, Eastman
`Kodak Company. Rochester, NY. Preprint from IS&T’s 48th
`Annual Conference, May 1995.
`
`Operation and Maintenance Manual for the Kodak Hawkeye
`II Imaging Accessory, Part No. 636-101. Catalogue No. 144
`7903. Eastman Kodak Company, Federal Systems Division.
`
`[21] Appl. No.: 574,756
`[22] Filed:
`Dec. 20, 1995
`
`[51] Int. Cl.‘ ..................................................... .. H04N 5/30
`[52] US. Cl.
`. 348/232; 348/231; 348/239;
`348/552
`[58] Field of Search ................................... .. 348/231. 232,
`348/239, 222. 552, 207; 358/335; H04N 5/30.
`5/225
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCU'MENTS
`
`6/1985 Konishi ................................... .. 358/29
`4,524,381
`3/1986
`358/335
`4,574,319
`5/1991 Sasson et al. ......................... .. 358/209
`5,016,107
`5/1991 Sasaki et al. ......................... .. 358/209
`5,018,017
`5,153,729 10/1992
`358/209
`5,335,072
`8/1994
`3481232
`5,477,264 12/1995
`348/231
`5,502,486
`3/1996
`348/239
`5,515,101
`5/1996 Yoshida ................................. .. 348/232
`
`FOREIGN PATENT DOCUMENTS
`
`0 640 938 A2 8/1994 European Pat. Off. ..... .. 606K 17/00
`
`OTHER PUBLICATIONS
`Kodak Professional Digital Camera System (DSC 100)
`User’s Manual (excerpts). Eastman Kodak Company. 1991,
`1992.
`
`Primary Examiner-Glenton B. Burgess
`Attorney, Agent, or Firm—David M. Woods
`
`[57]
`
`ABSTRACT
`
`An electronic camera captures images representing a variety
`of subjects and categorizes the image according to subject
`matter. The camera comprises an image sensor for capturing
`an image. a converter stage for converting the image into
`digital image data, and a memory for storing a plurality of
`categories providing classi?cation of the images by subject.
`A processor in the camera has the capability of assigning the
`plurality of categories to the images captured by the camera,
`with each category providing a subject classi?cation for the
`images. A user selects one or more categories for a plurality
`of images prior to capture, and an output image signal is then
`generated including the digital image data corresponding to
`a captured image and the particular category selected by the
`user. The categories can be default identi?ers stored in the
`memory, or can be names. text (i.e., account number). and/or
`graphics overlays (i.e., company logo) entered via a host
`computer and uploaded to the camera memory before the
`pictures are taken.
`
`30 Claims, 6 Drawing Sheets
`
`INPUT SECTION (A)
`
`COMPRESSION AND RECORDING
`SECTION (B)
`
`,4
`/
`
`‘12
`
`EXPOSU E
`SECTIO'L [
`,o/
`I]
`f
`Fblelsrn
`
`1s |-~-——_—_'
`I / [IMAGE BUFFER I/IB
`I
`an
`4/0
`'
`L
`3
`, » PROCESSOR
`2
`1
`I
`I
`I /30a
`28
`; FRAME!
`I
`I
`L____ ____.
`/
`PROCESSING
`I r_____ ALGORITHMS
`y
`w'DATE/TIME I/ZOG
`
`PROCESSOR ~
`
`20,
`
`PC
`
`~19
`
`SHUTTER
`BUTTON \ 2 ,
`
`mmm 1mm 2%
`FILES
`/
`24
`IMAGE
`/
`DATA
`FILES
`
`/-—24a
`
`"age
`
`|
`
`,4
`_, I
`3;
`
`FIRMWARE
`MEMORY
`
`_“__""'-‘ ' “ F _ _ ~ “ ___‘I
`
`7’ STATUS
`DISPLAY ~30
`
`[#29
`USER
`:1
`CONTROL SWITCHES
`RAM
`—>
`uvsmucnom
`MEMORY
`
`_'/ 32b
`I
`
`L. _ _ _ _ _ _ _ _ ___ _ _ _ _ _ _ ___l
`
`Ex. GOOG 1031
`
`

`
`U.S. Patent
`
`May 27, 1997
`
`Sheet 1 of 6
`
`5,633,678
`
`/
`
`qgqemmqz.
`2+“.
`qmqzzqmecmq
`mommmoezq
`
`mmss2mr*
`
`xmozmz
`
`mzmd
`
`qgqomuqs.
`
`mmqgguom
`
`ms::moo4q
`
`mm~2.mm
`
`D
`
`:55:3~.9u
`
`EX. GOOG 1031
`
`Ex. GOOG 1031
`
`
`
`
`

`
`U.S. Patent
`
`May 27, 1997
`
`Sheet 2 of 6
`
`5,633,678
`
`Svmozammoommulll.II!
`
`
`............=....u4
`
`
`
`
`
` mm._Sm.mzqm..._.======_==._298%ImommaSE_mm.=\2m.vu.m.mmsmomxmL....
`mm9x_mmfismmoqz._[; rl1!II|.!...L9§zecomm
` ozamoma9%29mEE:8E225%5%.
`
`vmom
`
`mzxtmofiq
`
`6%mommmoomm
`
`onSims5mEEm22:5
`
`Etzxm
`
`mm
`
`
`
`8:023SE28
`
`_|IIIIIlIlIIIIIIIIIIJ
`
`Essa:
`
`
`
`awn29osfiwmmz.mmqgsmtNo:
`Essa:_In.%\.
`
`EX. GOOG 1031
`
`Ex. GOOG 1031
`
`
`
`

`
`US. Patent
`
`May 27, 1997
`
`Sheet 3 0f 6
`
`5,633,678
`
`50 / S”
`
`@m A
`
`52c’
`
`52b
`
`FIG. 3
`
`6O
`/
`USER TURNS ON CAMERA
`/61
`
`USER SELECTS APPROPRIATE
`CATEGORY ICON AND
`CHOOSES DESIRED CATEGORY
`I
`62
`v
`/ I
`R, uSER TAKES PICTURE BY
`DEPRESSING SHUTTER BUTTON
`
`63
`/
`. IMAGE Is STORED IN FILE ‘ALONG
`gy??gm
`>
`
`E
`ANOTH R
`PHOTOGRAPHIC
`GROUP P
`
`USER CONNECTS CAMERA
`OR MEMORY CARD To
`THE HOST COMPUTER
`65
`\
`uSER OPENS CAMERA
`APPLICATION AND CHOOSES
`DESIRED CATEGORIIESI
`
`66\
`IMAGES FROM APPROPRIATE
`CATEGORIES ARE DOWNLOADED
`TO PROPER FOLDERS
`ACCORDING To THE CATEGORY
`IN EACH IMAGE FILE
`
`MORE
`YES IMAGES EORTHIS \NO
`CATE?GORY
`
`FIG.4
`
`Ex. GOOG 1031
`
`

`
`US. Patent
`
`May 27, 1997
`
`Sheet 4 of 6
`
`5,633,678
`
`USER INPUTS CATEGORIES, AND
`RELATED TEXT/ GRAPHICS ON HOST
`
`V... 70
`
`I
`USER DOWNLOADS CATEGORIES/TEXT/GRAPHICS F/7I
`TO CAMERA USING CABLE OR MEMORY CARD
`
`USER TURNS ON CAMERA
`
`“72
`
`I
`73
`USER SELECTS APPROPRIATE CATEGORY ICON /
`AND CHOOSES DESIRED CATEGORY
`
`L
`
`I
`/74
`USER. TAKES PICTURE BY
`OEPRESSING SHUTTER BUTTON 7
`5
`/
`‘I
`As IMAGE IS PROCESSED, OPTIONAL TExT
`GRAPHICS ANO/OR OATE TIME IS
`OvERLAYEO INTO IMAGE IF ENABLED FOR
`THE CATEGORY CHOSEN BY USER
`
`/76
`I
`IMAGE IS STOREO IN FILE ALONG wITH CATEGORY
`NAME AND OPTIONAL TExT FOR THE CATEGORY
`CHOSEN BY THE USER
`
`ANOTHER
`PHOTOGRAPHIC
`GROUP P
`/
`NO
`
`MORE IMAGES FOR
`THIS CATEGORY
`
`64
`
`USER CONNECTS CAMERA OR MEMORY
`CARD TO THE HOST COMPUTER
`
`65\ USER OPENS CAMERA APPLICATION AND
`CHOOSES OESIREO CATEGORIIESI
`
`F 5
`'
`
`66
`
`IMAGES FROM APPROPRIATE CATEGORIES ARE 7
`DOWNLOADED TO PROPER FOLDERS ACCORDING
`TO THE CATEGORY NAME IN EACH IMAGE FILE
`
`Ex. GOOG 1031
`
`

`
`U.S. Patent
`
`May 27, 1997
`
`Sheet 5 of 6
`
`5,633,678
`
`
`
`
`
`
`
`E.:o.:..H2___<2_.§§5.._o..sg.___§.......§.s.:o.:.....2___:2§_§_a.:o.:..u2_E2_._.:§
`
`
`
`mo§naeoz..Y1..Em~_u_$=_..z_r,2:-_:...2_:.z"Na2:E:.z_
`
`
`=$c__fi..:5HE;...N:_,581.2mm:sea“:8___.w_$.._ommm
`.m.Q»32.2.uozaewu2.3.5"2329uu___._E.o
`.o8§=§__.88=E=m=_.8855...:
`
`
`
`Ew_=-3-f:8:=e_em:8:.§_2m
`.
`
`mGI
`
`on2:mgm__....S»2:Q2.1..N»2:E2:
`$42.w9;__542.m9;__$42.54;:
`
`
`
`u.m#Q:2:5“E;8;:252EE:“£52_.___¢sa_ou:§
`
`sousé.H.o2:2528me;u2522:.me:"2522.»E:252B
`
`
`uo=..:2_5song»$_2_5
`
`N.9u5.35solmfl___¢e__5
`
`EX. GOOG 1031
`
`Ex. GOOG 1031
`
`
`

`
`US. Patent
`
`May 27, 1997
`
`Sheet 6 of 6
`
`5,633,678
`
`H
`STORED IMAGE FOB
`SELECTED CATEGORY F123
`J
`
`CAPTURED
`IMAGE
`é» \ Fly-by-night
`
`Insurance Co.
`
`CLAIM #I23 L
`
`GRAPHIC
`>-)~
`HWY-nigh? if’ NOV.9, I995
`lnsuronceCo. / I045 Om
`DATE/TIME
`
`FIG. 8
`
`Claim #123
`
`DATE
`TIME
`50/ TEXT
`
`ON
`ON
`ON
`
`OFF
`OFF
`OFF
`
`v
`
`56
`0%,
`_/-29
`
`f
`54
`
`Ex. GOOG 1031
`
`

`
`5,633,678
`
`1
`ELECTRONIC STILL CAMERA FOR
`CAPTURING AND CATEGORIZING IMAGES
`
`FIELD OF THE INVENTION
`
`This invention pertains to the ?eld of electronic imaging
`systems. and in particular to a digital electronic still camera
`of the type that is used with a host computer.
`
`2
`images. a small image display. and a keypad for manipulat
`ing the images. For images that had already been captured
`by the camera. the system included a “tag” feature that
`allowed a user to ?ag selected images for later display or for
`uploading from the DSU to another computer. Suppose that
`a user wanted to tag a couple of already-captured images for
`later viewing or uploading. The ?rst of the images would be
`brought up on the display and a “tag” button on the keypad
`would be pressed. and so on with the other images that are
`to be tagged. Then if the user presses a “?nd tags” key on the
`keypad. the ?rst of the tagged images is displayed (and/or
`uploaded). With a second press of the “?nd tags” key. the
`second tagged image is displayed (and/or uploaded). and so
`on.
`Another embodiment of this camera. the Kodak Hawkeye
`II camera. also allowed the user to type in a name for any one
`image after the image was taken. and to add a “note”
`text-string to the image ?le. Both cameras allowed a simple
`division of already-captured images into “tagged” and “not
`tagged” images and required a pattern of keystrokes subse
`quent to capture for every image of interest. In particular. the
`tagging feature functioned more as a post-capture data
`gatekeeper than as a useful tool for image handling. Any
`thing more complex in the way of data organization requires.
`as shown in FIG. 1. the use of a special application program
`in the host computer 4 to organize and sort the image data.
`Consequently. it would be desirable to have an electronic
`still system that could handle image content more el?ciently
`in the ?eld than is presently the case.
`
`15
`
`20
`
`25
`
`35
`
`BACKGROUND OF THE INVENTION
`As understood in the prior art. a digital electronic still
`camera is a device which uses an electronic sensor to capture
`an image; signal processing to represent the captured signal
`numerically; and some storage device to preserve the
`numerical image data. It is further known for a digital
`electronic still camera to use a removable storage device.
`such as an integrated circuit memory card. to store images.
`For instance. US. Pat. No. 5.016.107 describes an electronic
`still camera utilizing image compression and providing
`digital storage in a removable memory card having a static
`random access memory. In this camera. the integrated cir
`cuits in the removable memory card store image data and a
`directory locating the data.
`Coordination between a digital camera and a computer is
`generally lmown in the prior art. FIG. 1 shows a known
`electronic still photography system useful for inputting
`images to a computer. The system includes an electronic
`camera 1 with an electronic sensing section 1a. a digital
`processing section lb. a memory card slot 2, and a remov
`able memory card 3; and a host computer 4 with a built-in
`memory card reader 5. a printer 6a. a display 6b. a keyboard
`data entry device 7 and a disk drive 8. Images are captured
`by the sensing section 1a in the camera 1. manipulated in the
`processing section 1b. stored on the memory card 3. and
`transferred from the card 3 to the host computer 4. The
`software used to control the sensing and processing sections
`1a and 1b in the camera is located in ?rmware memory 1c
`in the camera 1. The application software used to control the
`playback operations of the host computer is supplied. at least
`in part. by the camera manufacturer. For instance, the
`manufacturer may provide a special disk or memory card
`that comes with the camera. The application software is
`transferred to the computer hard drive 8. and then down
`loaded to a computer random access memory (RAM)
`memory 9a. where the program is used by a central pro
`cessing unit (CPU) 9b.
`Digital cameras. such as the Kodak Digital Science
`DC4OTM camera sold by Eastman Kodak Company. allow
`almost a hundred images to be stored. Because the images
`are stored as sequentially numbered images. it can be
`di?icult and time consuming to locate a group of images of
`interest from all of the images stored in the camera. In a
`typical usage. an insurance adjuster may take a number of
`photos in processing a number of different claims during one
`day’s operations. It would be desirable to be able to identify
`the images of each group. for example by claim number or
`client name. It may be further desirable to allow this
`information. along with the date and time or a company
`logo. to be optionally included with the image data. This
`ensures that any prints made from the camera can be easily
`identi?ed. Such processing of the captured images is today
`performed in a host computer. such as the host computer 4
`in FIG. 1.
`The Kodak Professional Digital Camera System (model
`DCS-lOO) included a digital camera tethered by cable to a
`separate digital storage unit (DSU). The DSU. similar to the
`computer 4 of FIG. 1. contained a hard drive for storing
`
`SUMMARY OF THE INVENTION
`The present invention is directed to overcoming one or
`more of the problems set forth above. Brie?y summarized.
`according to one aspect of the present invention. an elec
`tronic camera captures images representing a variety of
`subjects and categorizes the image according to subject
`matter. The camera comprises an image sensor for the
`images. a converter stage for converting the images into
`digital image data. and a memory for storing a plurality of
`categories providing classi?cation of the images by subject.
`A processor in the camera has the capability of assigning the
`plurality of categories to the images captured by the camera.
`with each category providing subject classi?cation of one or
`more images. A user then interacts with a user control on the
`camera in order to select one or more of the categories for
`the images. and an output image signal is then generated
`including the digital image data corresponding to the images
`and the one or more categories selected by the user control.
`The advantage of the invention is that the known provi
`sion of a simple post-capture “tag” is enlarged into useful
`pre-capture categories that can e?ectively manage images
`according to variable subject content. According to this
`invention. a digital camera includes a display-based “tag”
`icon/feature which the user can set to an appropriate cat
`egory before taking a group of pictures. The category names
`can be defaults. or can be names and optional text (i.e..
`account number) and/or graphics overlays (i.e.. company
`logo) entered via a host computer and uploaded to the
`camera before the pictures are taken. When the user selects
`a particular category. the category name is stored along with
`the image data in the image ?le. and any text and graphics
`logo. as appropriate. are overlaid onto the image. When
`downloading the images to a host computer. the user can
`select a category and download only the images which have
`a particular category name. or the images can be stored in
`folders labeled with each category name.
`These and other aspects. objects. features and advantages
`of the present invention will be more clearly understood and
`
`45
`
`55
`
`65
`
`Ex. GOOG 1031
`
`

`
`5,633,678
`
`3
`appreciated from a review of the following detailed descrip
`tion of the preferred embodiments and appended claims. and
`by reference to the accompanying drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a diagram of a known electronic imaging system
`incorporating a digital camera for inputting images to a host
`computer.
`FIG. 2 is a block diagram of an electronic camera con
`?gured according to the invention to process and categorize
`images.
`FIG. 3 is a diagram of the status display panel and the user
`control switches used in the camera shown in FIG. 2.
`FIG. 4 is a process diagram for categorizing images
`according to default parameters stored in the camera shown
`in FIG. 2.
`FIG. 5 is a process diagram for categorizing images
`according to externally generated parameters supplied to the
`camera shown in FIG. 2.
`FIG. 6 is an example of categories that are customized
`from a host computer.
`FIG. 7 is an example of a ?le format resulting from
`customized categories.
`FIG. 8 is an example of an image display with customized
`category information overlaid into the image.
`FIG. 9 is an additional diagram of the status display panel
`and the user control switches set up for activating an overlay
`feature.
`FIG. 10 is an example of an image ?le management
`structure using categories.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`15
`
`25
`
`30
`
`35
`
`4
`corresponding to respective picture elements of the particu
`lar image captured.
`The analog image signals are applied to an A/D converter
`16. which generates digital image data from the analog input
`signals for each picture element. The digital image data is
`applied to an image buffer 18 for storing one or more
`images, albeit shown in FIG. 2 as a random access memory
`(RAM) with storage capacity for a plurality of still images.
`A programmed control processor 20 generally controls the
`input section A of the camera by initiating and controlling
`exposure (by operation of the diaphragm and shutter (not
`shown) in the exposure section 10), by controlling the ?ash
`unit 11, by generating the horizontal and vertical clocks
`needed for driving the sensor 12 and for clocking image
`information therefrom. and by enabling the A/D converter
`16 in conjunction with the image bu?’er 18 for each signal
`segment relating to a picture element. Exposure control
`would typically be effected by using ambient light informa
`tion from a photocell 19, and a shutter button 21 allows the
`user to initiate a capture cycle. The processor 20 also
`receives date and time information from a date/time gen
`erator 20a.
`Once a certain amount of digital image data has accumu
`lated in the image buffer 18, the stored data is applied to a
`programmed digital signal processor 22. which controls the
`throughput processing rate for the compression and record
`ing section B of the camera. The algorithms and other
`operating code used by the processor 22 are stored in an
`algorithm memory 28. The digital signal processor 22 com
`presses each still image stored in the image buffer 18
`according to any one of a number of known image com
`pression algorithms, such as the well-known JPEG (Joint
`Photographic Experts Group) discrete cosine
`transformation-based compression algorithm. The processor
`22 applies the compression algorithm from the memory 28
`to the digital image data, and sends the compressed data to
`a removable storage device via a signal port 26. However,
`while compression is shown in the camera of FIG. 2,
`compression is not a necessary feature, and uncompressed
`data could be stored instead in the removable storage device.
`In this example, the signal port 26 is a card interface and
`the removable storage device is a memory card 24. While a
`memory card 24 is shown in this system as the storage
`device, other devices may be used, such as a ?oppy disk
`magnetic medium, a small hard drive, or optical storage (in
`the latter cases, suitable conventional reading/writing appa
`ratus would be provided in the camera, e.g., magnetic or
`optical read/write head. etc.). Arepresentative memory card
`is a card adapted to the PCMCIA card interface standard,
`such as described in the PC Card Standard, Release 2.0,
`published by The Personal Computer Memory Card Inter
`national Association. Sunnyvale, Calif. September 1991.
`The memory card 24 accordingly contains solid state
`memory 24a. such as Flash EPROM memory, which the
`card uses to store image data ?les.
`According to this invention. the digital camera includes a
`categorization feature which enables a user to establish an
`appropriate category, providing classi?cation of the images
`by subject. before taking a group of pictures. The category
`information can be default code symbols (A,B,C, etc.), or
`can be externally-generated category information, such as
`names and optional text (i.e.. account numbers) and/or
`graphics overlays (i.e.. company logos), which is entered via
`a host computer and uploaded to the camera through the
`signal port 26 before the pictures are taken or via cable
`connection. After the user selects a particular category. the
`image is captured and the category information is stored in
`
`40
`
`45
`
`Because electronic still cameras employing electronic
`sensors are well known, the present description will be
`directed in particular to elements forming part of, or coop
`erating more directly with, apparatus in accordance with the
`present invention. Elements not speci?cally shown or
`described herein may be selected from those known in the
`art Certain aspects of the embodiments to be described are
`provided in software. Given the system description as
`described in the following materials, all such software
`implementation is conventional and within the ordinary skill
`in such arts.
`FIG. 2 shows an electronic still camera capable of cat
`egorizing images as taught by the invention. The electronic
`still camera is divided generally into an input section A and
`50
`a compression and recording section B. The input section A
`includes an exposure section 10 for directing image light
`from a subject (not shown) toward an image sensor 12. A
`?ash unit 11 may be used to illuminate the subject. Although
`not shown. the exposure section 10 includes conventional
`optics for directing the image light through a diaphragm,
`which regulates the optical aperture, and a shutter. which
`regulates exposure time. The sensor 12. which includes a
`two-dimensional array of photosites corresponding to pic
`ture elements of the image. is a conventional charge-coupled
`device (CCD) using. e.g., either well-known interline trans
`fer or frame transfer techniques. The sensor 12 is exposed to
`image light to capture a particular image; accordingly,
`analog image charge information corresponding to the par
`ticular image is generated in respective photosites. The
`charge information is applied to an output diode 14, which
`converts the charge information to analog image signals
`
`55
`
`65
`
`Ex. GOOG 1031
`
`

`
`5,633,678
`
`15
`
`25
`
`5
`the removable storage device along with the image data in
`the image ?le, and any text and graphics logo, as
`appropriate. are overlaid onto the image. When downloading
`the images to a host computer, the user can select a particular
`category and download only the images which have a
`particular category identi?er. or the images can all be
`downloaded and stored in ?le folders labeled with each
`category name.
`Consequently, the memory card 24 contains additional
`memory to store header ?les 24b containing category infor
`mation helpful in providing classi?cation of the images by
`subject matter. Such category information may include
`default tag codes. such as simple alphabetic identi?ers A. B.
`C, D, etc.. or alphanumeric names that can be associated
`with speci?c images according to subject matter. The header
`?les 24b can also contain strings of text that identify
`particular images and separate graphics images that signify
`or otherwise identify subject matter. The header ?les 24b can
`also contain instructions to overlay all or part of the category
`information into the image data ?les 24a, or. alternatively,
`the digital signal processor 22 can access an algorithm in the
`algorithm memory 28 which accomplishes the overlay, and
`then stores all or part of the category information with the
`image data. It is a further feature of this invention that the
`externally generated category information can be supplied to
`the digital camera through the signal port 26. Such exter
`nally generated category information can be stored in the
`Card Information Structure (CIS) of the PCMCIA Format
`(as described in the aforementioned Release 2.0). in the
`header data ?le 24b. or in a portion of the memory space of
`the memory 24a used to store the image data.
`The camera also contains a memory section 31, including
`?rmware memory 32a with operating code for the camera,
`as well as RAM instruction memory 32b connected to
`receive code from the memory card 24. When the memory
`card 24 is inserted into the camera. the processor 20 uses the
`?rmware algorithms in the memory 32a to determine if the
`memory card 24 contains externally generated ?les which
`can be used by the camera. If not. the ?rmware algorithms
`supplied with the camera in the memory 32a are used by the
`programmable processor 20. In particular. the default cat
`egory parameters are stored in the ?rmware memory 32a
`and accessed by the processor 20. If the card does contain
`appropriate externally generated category information.
`however. these are downloaded ?om the header ?le section
`24b of the card 24 to the RAM instruction memory 32b and
`used by the programmable processor 20 in place of some
`portion of the ?rmware algorithms supplied with the camera
`from the ?rmware memory 32a. In this way the camera can
`execute customized category de?nitions which provide
`higher quality or more complex categorization of pictures.
`User control is provided by a status display 30 connected
`to the control processor 20 for displaying information useful
`in operation of the camera. and by a set of user control
`switches 29. The set of user control switches 29 are also
`connected to the processor 20 for entering. highlighting, and
`selecting information displayed on the status display 30. In
`accordance with invention. the status display 30 is a liquid
`crystal display (LCD) having a panel con?guration 50 such
`as shown in FIG. 3. which includes a “tag” icon 52a, and an
`alphanumeric segmented section 52b. (The panel con?gu
`ration includes other typical icons, such as a battery level
`icon 520. a ?ash mode icon 52d, a self-timer icon 52e. and
`an image delete icon 52f.) The user control switches 29
`include a toggle switch 54 and a select button 56. The toggle
`switch 54 is used to toggle to a setting once an icon has been
`activated by the select button 56. For example. if the ?ash
`
`6
`mode icon 52d is highlighted and showing that it is the
`currently active control. pressing the select button would
`activate the tag icon 52a and make it the currently active
`control.
`While not important to this invention, other information
`displayed on the status display 30 might include typical
`photographic data, such as shutter speed. aperture. exposure
`bias. color balance (auto. tungsten. ?uorescent, daylight),
`high/low resolution. low light. exposure modes (aperture
`preferred, shutter preferred). and so on. Moreover, other
`information unique to this type of camera may be displayed.
`For instance. the memory card 24 would ordinarily include
`a directory signifying the beginning and ending of each
`stored image. This would show on the display 30 as either
`(or both) the number of images stored or the number of
`image spaces remaining. or estimated to be remaining.
`If externally generated category information is present in
`the header ?le 24b on the card 24. it can also be identi?ed
`on the status display 30. For instance, the externally gener
`ated category information, such as names, text strings, and
`graphics images can be identi?ed and displayed if the status
`display 30 has the appropriate display capability.
`Alternatively. the toggle switch 54 can be used to cycle
`through alphanumeric names that are provided from the host
`computer and displayed on the segmented section 52b.
`Selecting any one name then automatically accesses any text
`string or graphics image that might be associated with the
`name in the instruction memory 321:, and stores such strings
`or graphics image with the digital image data. The camera
`may also have the additional feature of overlaying a text
`string. or date and/or time from the date/time generator 200.
`into the image. Whether or not the date/time or text string is
`overlaid in the images of any particular category can be
`controlled by overlay control information for each category
`present in the header ?le 24b. Alternately. this can be done
`from additional icons in the panel con?guration 50 and from
`the user control switches 29. as shown in FIG. 9. Date, time,
`name, and text are separately activated by the toggle switch
`54. and “on” (overlay active) or “off” icons are activated by
`the select switch 56. It is assumed that an uploaded graphics
`image would always overlay into the image, but if this is not
`always desirable, then a separate graphics icon may be
`provided for overlay control.
`The process for categorizing images according to default
`parameters is shown in FIG. 4. and the process for catego
`rizing images according to customized. externally generated
`parameters is shown in FIG. 5. Referring ?rst to FIG. 4.
`when the camera is ?rst turned on. the “tag” icon 520 will
`be shown only in outline form (step 60). The inner part of the
`“tag” icon 52a will ?ll to indicate when the user has
`positively selected the tag feature by activation of the user
`control switches 29. In the default mode. there are a prede
`termined plurality of possible categories. for example. ?ve
`alphabetic categories A.B.C.D.E and NONE (or OFF). With
`the tag icon 52a activated. the user will select categories
`A.B.C.D.E,NONE (repeat sequence AB. etc.) by pressing
`the toggle switch 54 (step 61). Each press of the switch 54
`causes a di?ierent tag code to be displayed on the alphanu
`meric segment 52b.
`After the tag is selected, the user can activate other icons
`by use of the select button 56. Then, when the shutter button
`21 is pressed. the selected category (tag) is associated with
`the digital image data (steps 62. 63). Then the steps 62. 63
`are repeated for any additional images in the same category.
`If a new category is desired, the process reverts to step 61.
`When all the images are captured. the user connects the
`camera (by cable) or the memory card 24 to a host computer
`
`35
`
`45
`
`55
`
`65
`
`Ex. GOOG 1031
`
`

`
`7
`(step 64), such as the host computer 4 illustrated in FIG. 1.
`Using software supplied by the camera manufacturer, the
`user opens a camera application and chooses the desired
`category (step 65). Then the images from appropriate cat
`egories stored in the camera. or the memory card 24, are
`downloaded to proper ?le folders according to the category
`data stored with the digital image data in each ?le (step 66).
`The user may also customize the tags using the host
`computer. As shown in the process diagram of FIG. 5, a user
`inputs category information. such as category names, and
`related text or graphics images, ?om the host computer (step
`70). The user then downloads the category information to
`the camera (step 71). This may be done through a tethered
`connection to the camera, or by writing appropriate ?les into
`the memory card 24. The user turns on the camera (step 72).
`and selects an appropriate category (step 73). At this stage,
`the user can also decide to activate the overlay capability for
`date, time, text, and graphics images. The picture is taken
`(step 74), and the category information is optionally overlaid
`into the image (step 75). The digital image data is then stored
`in the memory card 24 together with category information,
`either separately or as overlaid into the image data (step 76).
`The remaining steps are substantially the same as in FIG. 4.
`FIGS. 6, 7, and 8 show examples of categories customized
`from a host computer, following on the previously men
`tioned claims adjuster example, and the resulting ?le format
`and image display. As shown in FIG. 6, the categories (tags)
`are given names, optional text strings, and graphics. The
`category information also includes overlay control informa
`tion indicating whether or not the text string. date, and time
`should be overlaid into the image. In this example, the user,
`a free-lance claims adjuster, has created four different cat
`egories (H111 and H122) for different insurance claims from
`a ?rst insurance company, one (F123) for a claim from a
`second insurance company, and one (IGDS) for personal
`images. The user also indicates whether the text string
`should be overlaid into the image, and whether the camera
`should overlay the date and/or time in the image. This
`category information, including names, text, and graphics, is
`then downloaded to the host computer from the memory
`card 24. A typical technique for such downloading is
`described in US. Ser. No. 08/219,608, entitled “Electronic
`Imaging System Using a Removable Software-Enhanced
`Storage Device”, which is to issue as US. Pat. No. 5.477,
`264 on Dec. 19, 1995.
`When programmed by the host or memory card 24, the
`number of tag categories that can be selected in the camera,
`and the four-letter name of each category. is as set by the
`user via the host computer. The user selects the appropriate
`category before taking a sequence of images. The tag names
`and text are then included in the image ?les. as shown in
`FIG. 7. In this example, the user selected Tag name H111
`and took images #1-2. The user then selected Tag name
`KIDS and took images 3-5. Finally. the user selected tag
`name F123 and took image #6. As the images are captured
`and processed. the appropriate overlay information for the
`tag chosen is used. For the category “F123”. the tag indicates
`that the image should include a graphics overlay, the text
`“Claim #123”, and the date/time, as shown in FIG. 8. In a
`furth