United States Patent [191
`Allen et al.
`
`IIHI
`
`~111111111111111111 11~111~1111111111111~1111111111111
`US005737491A
`5,737,491
`[111 Patent Number:
`Apr. 7, 1998
`[45] Date of Patent:
`
`[54] ELECTRONIC IMAGING SYSTEM CAPABLE
`OF IMAGE CAPTURE, LOCAL WIRELESS
`TRANSMISSION AND VOICE
`RECOGNITION
`
`[75]
`
`Inventors: James D. Allen; Omid A. Moghadam;
`Donna M. Romer, all of Rochester,
`N.Y.
`
`[73] Assignee: Eastman Kodak Company, Rochester,
`N.Y.
`
`[21] Appl. No.: 672,773
`
`Jun. 28, 1996
`
`[22] Filed:
`Int. CL 6
`•••••••••••••••••••••••••••••••••••••••••••••••••••••••• GlOL 3/00
`[51]
`[52] U.S. Cl •....................... 395/2.79; 395/2.81; 395/2.84;
`396/283; 348/211
`[58] Field of Search .................................. 395/2.79, 2.81,
`395/2.84; 396/283, 287; 348/211, 223
`
`[56]
`
`References Cited
`
`U.S. PATENf DOCUMENTS
`
`4,502,771
`4,726,065
`4,757,541
`4,856,066
`4,870,686
`5,027,149
`5,179,446
`
`3/1985 Katsuma et al. ..........•.•........... 354/289
`211988 Froessl ...................................... 381/41
`7/1988 Beadles ..................................... 381/43
`8/1989 Lemelson .................................. 381/36
`9/1989 Gerson et al. . ........................... 381/43
`6/1991 Hoshino et al. . ....................... 3541412
`111993 Hong ....................................... 3581224
`
`5,633,678
`
`5/1997 Parulski et al .......................... 348/232
`
`OTHER PUBLICATIONS
`
`Pepper, Jon "Instant Images for Your omputer." FamilyPC,
`World Wide Web,
`http://wwwl.zdnet.com/familypc/
`comtent/960617/fthw/index.html, 1996.
`
`Pri111flry Emminer-Allen R. MacDonald
`Assistant Examiner-Alphonso A. Collins
`Attorney, Agent, or Firm-Thomas H. Close
`
`[57]
`
`ABSTRACT
`
`A system for digital image capture and transmission,
`includes a digital camera that has an electronic image sensor
`for sensing an image and producing a digital image; a
`memory for storing digital images produced by the image
`sensor in digital image files, the digital image files having
`associated information for controlling a remote image ful(cid:173)
`fillment server; a voice recorder for digitizing voice com(cid:173)
`mands relating to control of the image fulfillment server; and
`a transmitter for transmitting the digital image file to the
`image fulfillment server. Either the camera or the fulfillment
`server includes a voice recognition module responsive to the
`digitized voice commands for producing control signals for
`the image fulfillment server. The image fulfillment server
`includes a receiver for receiving the digital image file and
`control signals; a memory for storing the received digital
`image file; and a file manager for managing the digital image
`file in response to the control signals.
`
`7 Claims, 3 Drawing Sheets
`
`27
`EXTERNAL INPUT
`. . - - - - - - - - - - -1
`DEVICE
`
`IMAGE FULFILLMENT
`SERVER (34)
`
`35
`
`~
`
`I
`L ___
`
`- - -
`
`-
`
`-1
`I
`~46
`____ J
`
`I 1--------------
`
`DIGITAL CAMERA(10)
`
`VOICE RECOGNITION
`MODULE(30)
`
`REMOVABLE
`MEDIA
`
`1
`SECONDARY
`TRANSMISSION
`CHANNELS
`
`1
`
`Petition for Inter Partes Review
`of U.S. Pat. No. 7,477,284
`IPR2013‐00219
`EXHIBIT
`Sony‐
`
`

`
`27
`
`EXTERNAL INPUT
`DEVICE
`
`IMAGE FULFILLMENT
`SERVER (34)
`
`38
`
`WIRELESS
`TRANSCEIVER
`
`CENTRAL
`PROCESSING
`UNIT
`
`36
`
`~
`37
`
`42
`
`35 \
`
`16
`
`SHUTTER
`RELEASE I r 18
`SWITCH
`
`14
`-
`
`r
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`I CODE BOOK k I
`
`I 1 MICROPHONE
`ACTIVATION
`SWITCH
`
`-
`
`-
`
`-....-----''----.
`
`I DSP
`
`UNIT
`
`WIRELESS
`TRANSCEIVER
`
`32
`
`I
`
`I
`
`L_ __ __ _____ ___ _ _J
`\
`
`-
`DIGITAL CAMERA(10)
`
`\
`VOICE RECOGNITION
`MODULE(30)
`
`REMOVABLE
`MEDIA
`RECORDING
`DEVICE
`
`48
`
`44
`
`50
`REMOVABLE
`MEDIA
`
`FIG. 1
`
`SECONDARY
`TRANSMISSION
`MODULE
`
`PRINTS
`
`,... r-
`: 0 LJ LJ LJ !__.,/46
`
`L--- 1 - - - _J
`SECONDARY
`TRANSMISSION
`CHANNELS
`
`~ • 00
`• ;p
`t""f'(cid:173)n> a
`
`~
`t'1
`... ""-~
`""""' ~
`
`~
`!
`""""'
`~
`w
`
`u-.
`~ .....:a
`
`-..
`~
`\0
`~
`
`2
`
`

`
`U.S. Patent
`
`Apr. 7, 1998
`
`Sheet 2 of 3
`
`5,737,491
`
`DIGITAL IMAGE IS V52
`CAPTURED
`
`6{
`
`ERASE DIGITAL
`IMAGE
`
`ENGAGE
`MICROPHONE _r54
`SWITCH
`j_
`ENTER VOICE ..J56
`COMMANDS
`i
`
`IF
`COMMAND=
`"ERASE"
`
`COMPARE DIGITIZED
`VOICE COMMANDS
`AGAINST THE
`CODEBOOK
`+
`VOICE COMMAND IS
`MATCHED
`f ELSE
`SET A FLAG IN THE _J64
`IMAGE FILE HEADER
`
`~58
`
`l-J60
`
`'
`
`_..£66
`
`rJ70
`
`COMPOSE IMAGE FILE
`AND HEADER INTO ONE FILE
`t
`68"\
`I RECOGNIZE VOICE COMMAND "TRANSMIT"I
`t
`TRANSMIT IMAGE FILE TO IMAGE
`FULFILLMENT SERVER
`
`72
`
`7~
`TRANSMIT
`IMAGE
`USING A
`SECONDARY
`TRANSMISSION
`CHANNEL
`
`TRANSLATE THE IMAGE FILE HEADER
`INCLUDING t UNIQUE ID & COMMANDS 17~
`
`'
`
`PRINT IMAGES WITH/ WRITE IMAGES TO
`WITHOUT ANNOTATED REMOVABLE MEDIA
`TEXT ON THE
`PHOTOGRAPHIC
`PRINTER
`
`~76
`
`~o
`I
`
`I
`
`BILL PHOTOGRAPHERS ACCOUNT
`
`FIG. 2
`
`3
`
`

`
`U.S. Patent
`
`Apr. 7, 1998
`
`Sheet 3 of 3
`
`5,737,491
`
`DIGITAL IMAGE IS J82
`CAPTURED
`
`ENGAGE
`MICROPHONE lJ84
`SWITCH
`
`ENTER VOICE .F86
`COMMANDS
`
`DIGITIZE VOICE
`COMMANDS & STORE w-aa
`IN IMAGE FILE
`
`'
`'
`HEADER '
`
`APPEND IMAGE FILE HEADER _..r90
`TO THE DIGITAL IMAGE FILE
`t
`TRANSMIT IMAGE FILE TO IMAGE v
`FULFILLMENT SERVER
`t
`COMPARE HEADER DIGITIZED
`VOICE OAT A AGAINST THE v94
`CODEBOOK
`
`92
`
`I RECOGNIZE VOICE COMMAND
`
`96
`
`'
`'
`
`FIG. 3
`
`4
`
`

`
`5,737,491
`
`1
`ELECTRONIC IMAGING SYSTEM CAPABLE
`OF IMAGE CAPTURE, LOCAL WIRELESS
`TRANSMISSION AND VOICE
`RECOGNITION
`
`FIELD OF THE INVENTION
`
`The present invention relates in general to the field of
`electronic photography, and in particular to an electronic
`image capture and transmission system capable of wireless
`transmission and voice recognition.
`
`BACKGROUND OF THE INVENTION
`
`2
`vacation spot. The image may be transmitted via a wireless
`connection such as a cellular phone service to a fulfillment
`server where the images are printed according to the pho(cid:173)
`tographers voice instructions and the printed images are
`5 forwarded to a desired address. The digital images may also
`be stored for future access by the photographer, or further
`transmitted to designated e-mail addresses.
`These and other aspects, objects, features and advantages
`of the present invention will be more clearly understood and
`10 appreciated from a review of the following detailed descrip(cid:173)
`tion of the preferred embodiments and appended claims, and
`by reference to the accompanying drawings.
`
`In the field of professional photography especially in the
`field of photo journalism and sports photography, speedy 15
`delivery of photographs of an event to the photo editor is an
`important factor. In many instances today, the film is shipped
`directly to the news photo editor via courier. Another method
`is to use a digital camera such as the Kodak DCS 420 or
`Kodak DC50 which are capable of capturing images by 20
`using a CCD sensor and storing them in a digital file. The
`digital images are stored in solid state memory, a hard drive
`on the camera or a removable solid state memory card such
`as a PCMCIA memory card. These files can then be down(cid:173)
`loaded to a portable computer and transmitted to a remote 25
`computer via a telecommunication connection such as a
`modem. Although this solution offers the photojournalist
`and his editor a faster turnaround time than the traditional
`route of a courier delivering film and prints, it still means
`that he must leave the scene to find a phone.
`
`BRIEF DESCRIPTION OF THE DRAWlNGS
`
`FlG. 1 is a schern.atic diagram illustrating a digital wire(cid:173)
`less photography and image transmission system with
`speech recognition capabilities according to the present
`invention;
`FlG. 2 is a flow chart illustrating the method of image
`capture, storage and transmission according to an embodi(cid:173)
`ment of the present invention, with the speech recognition
`subsystem in the camera; and
`FlG. 3 is a flow chart illustrating the method of image
`capture, storage and transmission according to an embodi(cid:173)
`ment of the present invention with the speech recognition
`subsystem in the image fulfillment server.
`To facilitate understanding, identical reference numerals
`have been used, where possible, to designate identical
`30 sections that are common to the figures.
`
`SUMMARY OF THE INVENTION
`
`DETAilED DESCRIPTION OF THE
`INVENTION
`
`35
`
`The present invention is directed to overcoming one or
`more of the problems set forth above. Briefly summarized,
`according to one aspect of the present invention, a system
`for digital image capture and transmission, includes a digital
`camera that has an electronic image sensor for sensing an
`image and producing a digital image; a memory for storing
`digital images produced by the image sensor in digital image
`files, the digital image files having associated information
`for controlling a remote image fulfillment- server; a voice
`recorder for digitizing voice commands relating to control of
`the image fulfillment server; and a transceiver for transmit(cid:173)
`ting the digital image file to the image fulfillment server.
`Either the camera or the fulfillment server includes a voice 45
`recognition module responsive to the digitized voice com(cid:173)
`mands for producing control signals for the image fulfill(cid:173)
`ment server. The image fulfillment server includes a trans(cid:173)
`ceiver for receiving the digital image file and control signals;
`a memory for storing the received digital image file; and a 50
`file manager for managing the digital image file in response
`to the control signals.
`
`40
`
`Referring to FlG. 1, a digital camera generally designated
`10, includes a lens 12, an image sensor 14located behind the
`lens 12, a viewfinder 16, a shutter release 18, a micropro(cid:173)
`cessor 20 for controlling the camera, a temporary memory
`22 for storing digital images produced by the image sensor
`14 and digitized sound signals and spoken commands sensed
`by a microphone 24. The microphone 24, an analog to digital
`converter 26 for digitizing the sounds sensed by the micro(cid:173)
`phone 24, and a sound recording microphone activation
`switch 28 for activating the sound recording function of the
`camera, are all included in a voice recognition module 30.
`The voice recognition module 30 recognizes voice com(cid:173)
`mands and produces control signals for use by an image
`fulfillment server 34 as described below. The digital camera
`10 also includes a transceiver 32 for transmitting the digital
`images, and control signals to the image fulfillment server
`34.
`The voice recognition module 30 includes a code book 25
`provided by the camera manufacturer and customized by the
`photographer and a digital signal processing unit (DSP) 29
`55 for comparing the photographer's digitized voice commands
`to digital voice templates stored in code book 25 and
`generating control signals to be appended to the digital
`image file. The DSP 29 may comprise, for ·example, a
`TMS3477 digital signal processing chip, manufactured by
`60 Texas Instruments Inc., Dallas, Tex., or an RSC-164 digital
`signal processing chip manufactured by Sensory Circuits
`Inc., San Jose, <:;alit.
`The camera 10 includes an interface, such as a SCSI port,
`for connecting to an external input device 1:1 such as a
`65 keyboard or LCD touch screen. The external input device 27
`may be used to enter information such as text annotation,
`electronic addresses or file names that are to be associated
`
`ADVANI'AGEOUS EFFECf OF THE
`INVENTION
`
`The system of the present invention has the advantage of
`providing for easy control over fast delivery of digital
`images in the field that allows a choice of different com(cid:173)
`munication relay services. For example a sports photogra(cid:173)
`pher can transmit his digital images locally via wireless
`transmission to a local image fulfillment server at the
`stadium with instructions to make prints and transmit images
`via an ISDN connection to his magazine's photo editor or to
`a remote image fulfillment server computer.
`Using the present invention, an amateur photographer can
`capture an electronic image at a special event or scenic
`
`5
`
`

`
`5,737,491
`
`4
`-continued
`
`VERBAL COMMAND
`
`FUNCTION
`
`"Print (nwnber)"
`
`"Write (media type)"
`"Send Prints (address)"
`
`"Bill (accmmt)"
`
`"Transmif'
`
`-Make a designated nwnber of prints of
`the digital image of a particular size at the
`image fulfilhnent server
`-Write digital images to removable media
`.Send the prints to a designated address
`pre recorded by the photographer
`.Charge the services to a designated
`account
`·Transmit image with appended control
`signals to the image fulfilhnent server
`
`10
`
`3
`with photographer's utterances. An example of this would
`be an e-mail address such as "moghadam@kodak. com"
`which is associated with a spoken command such as
`"address 1", or "grandma".
`The transceiver 32 is a wireless communication system 5
`such as a cellular telephone or a digital wireless communi(cid:173)
`cation system such as the personal handy phone system
`(HPS). The microprocessor 20 is programmed to attach a
`unique identification code (e.g. a camera serial number) to
`each digital image produced by the camera.
`The image fulfillrnent server 34 receives transmissions
`from the digital camera 10, for example by a wireless
`transceiver 36 or by connection to a telecommunications
`network that includes a wireless transceiver. The image
`fulfillment server 34 includes a central processor 37; and a
`memory 38 for receiving and storing digital images and
`control signals that are transmitted from the camera 10. The
`central processor 37 executes the control signals that are
`received with the digital images. The image fulfillment
`server includes a data file relating each identification code
`(e.g. camera serial number) with information relating to its
`owner, such as the owners phone number, credit card
`number, name, address, or e-mail address. The central
`processing unit37 is programmed to read identification code
`associated with the digital images that are received and to
`retrieve the owner information to charge the owner for
`services provided. For example, the services provided may
`be charged to the owners phone number, credit card number,
`or a bill sent to the owners address.
`A printer 42 for producing prints 43 from the digital
`images and annotated text is connected to the image fulfill(cid:173)
`ment server 34. The printer may be a thermal printer such as
`the XLS8600 manufactured by the Eastman Kodak
`Company, Rochester, N.Y., or a digital silver halide printer.
`A secondary transmission module 44 is connected to the
`image fulfillrnent server for transmitting the digital images
`over any one or more of a plurality of secondary commu(cid:173)
`nication channels 46 in response to control signals received
`with the digital images. The secondary communication
`channels may be, for example, connections to the Internet,
`satellite communication systems, ISDN, or conventional
`telephone lines. A label printer 40 is provided for printing
`instructions to the operator of the image fulfillment server,
`such as: mailing instructions for prints; identification of the
`owners of the prints; and billing notices for services pro(cid:173)
`vided by the image fulfillment server. A removable media
`writer 50, such as a CD-R recorder or a floppy disc writer is
`connected to the central processing unit 37 for recording the
`digital images on digital recording medium for returning to
`the customer, or forwarding to a designated location.
`In operation, a photographer captures the image of a scene
`with the digital camera 10, and verbally instructs the camera
`to perform one or more of the command functions listed in
`Table 1.
`
`The verbal instructions are digitized and compared to the
`15 previously digitized voice commands stored in the code
`book 25. When a match is made between the digitized
`spoken command and the digitized command stored in the
`code book, the command is "recognized" by the DSP unit 29
`in the voice recognition module 30 to produce a control
`20 signal for the camera 10 or the image fulfillment server 34.
`Control signals for the image fulfillment server 34 are
`appended to the digital image in an image file header. At the
`verbal command of the photographer, the digital images and
`appended control signals are transmitted to the image ful(cid:173)
`fillment server 34 by transceiver 32. At the image fulfillment
`25 server, the transceiver 36 receives the digital images and the
`control signals and stores them temporarily in memory 38.
`The central processor 37 responds to the control signals to
`effect requested services related to the digital image.
`Alternatively, the digital signal processor 29 and code
`30 book 25 of the voice recognition module 30 may be located
`at the image fulfillment server 34 rather than in the camera.
`In this case, the camera includes a manual user interface for
`effecting the camera operations and for appending the digi(cid:173)
`tized sound annotations to the image and transmitting the
`35 annotated image to the image fulfi1lment server 34.
`The operation of the system with the voice recognition
`capability in the camera 10 will now be described with
`reference to FIG. 2. The photographer actuates the camera
`10 to capture a digital image 52. Next, the photographer
`40 engages the microphone switch 54 and enters a voice
`command 56. The camera 10 digitizes the voice command
`and compares 58 the digitized voice command with the
`commands stored in the codebook. When the digitized voice
`command is matched 60 to a command in the codebook, the
`45 voice command is "recognized". If the command is "erase",
`the digital image stored in the camera is erased 62. Com(cid:173)
`mands that include associated voice data, such as the ''text"
`command are recognized and a command flag corresponding
`to the command is set in the image file header and digitized
`so voice data following the command is added to the image file
`header 64. The image file header is then appended to the
`digital image 66. When the voice command ''transmit" is
`recognized 68, the camera 10 transmits 70 the digital image
`file to the image fulfi1lment center 34.
`55 At the fulfillment center 34, the central processing unit 37
`reads the image file header 72, which includes the LD. of the
`camera, command flags and the digital voice data; and
`effects the action indicated by the command flags that are
`set. For example, if the "send to" flag is set, the image data
`60 is transmitted 74 via secondary transmission channel 46
`through transmission module 44 to the electronic address
`recorded in the digitized voice data. If the ''text" flag is set,
`the recognized voice data and associated information is
`converted to ASCII code and the AS CIT code is stored in the
`65 header of the digital image.
`If the "print" flag is set, the digital image is printed on the
`printer 42, making the number of prints indicated by the
`
`VERBAL COMMAND
`
`FUNCTION
`
`"Erase"
`
`"Send to (address X channel)"
`
`"Name (file name)"
`
`"Text (message)"
`
`-Erase the image from the camera
`memory
`-'fransmit the image to the image
`fulfillment server with appended
`command to forward image over one of
`the secondary communication channels to
`a designated address
`-Appended a file name to image from a
`list of file names recorded by the
`photographer
`-Annotate the digital image with a text
`message from a list of text messages
`recorded by the photographer
`
`6
`
`

`
`5,737,491
`
`6
`-continued
`
`PAIITSLlST
`
`92
`94
`96
`
`transmit image file to fulfillment server step
`compare header data step
`recognize voice commands step
`
`We claim:
`1. A system for digital image capture and transmission,
`comprising:
`
`5
`digitized voice data in the image header 76. Finally, the
`central processing unit 37 bills the photographers account 80
`for the services rendered.
`Turning now to FIG. 3, the operation of the system with
`the voice recognition capability in the image fulfillment 5
`server will be described. First, the photographer captures an
`image 82. Next, the photographer engages the microphone
`switch 84 and enters a voice command 86. The camera 10
`digitizes the voice command and stores the digitized voice
`command in a file header 88. The file header is appended to 10
`the digital image 90. When the microphone switch is
`released, the camera 10 transmits the image file to the image
`fulfillment server 34.
`At the image fulfillment server, the digitized voice data
`stored in the file header is recognized by comparing to the
`text code book 94, and when a match is made 96, the
`commands are executed as described above.
`The invention has been described with reference to a
`preferred embodiment. However, it will be appreciated that 20
`variations and modifications can be effected by a person of
`ordinary skill in the art without departing from the scope of
`the invention.
`
`15
`
`PARTS LIST
`
`camera
`lens
`image sensor
`AID converter
`view finder
`shutter release
`micro processor
`temporary memory
`microphone
`code book
`AID converter
`extemal input device
`microphone activation switch
`digital signal processing(DSP) unit
`voice recognition module
`transceiver( camera)
`image fulfillment server
`wireless transmission link
`transceiver(image fuJfillment server)
`central prooessing unit(CPU)
`memory
`label printer
`printer
`prints
`transmission module
`secondary communication channels
`removable media recording device
`removable media
`digital image capture step
`engage micropl:xme step
`enter voice command step
`compare voice command step
`match voice command step
`erase digital-image step
`set flag step
`compose image file and header step
`recognize "transmif' step
`transmit step
`translate image file header step
`transmit image step
`priut step
`write image to removable media step
`bill photographer's account step
`capture digital image step
`engage microphone step
`enter voice command step
`digitize voice commands and store in file header step
`append image file header to digital image ,step
`
`10
`12
`14
`15
`16
`18
`20
`22
`24
`25
`26
`27
`28
`29
`30
`32
`34
`35
`36
`37
`38
`40
`42
`43
`44
`46
`48
`50
`52
`54
`56
`58
`60
`62
`64
`66
`68
`70
`72
`74
`76
`78
`80
`82
`84
`86
`88
`90
`
`25
`
`30
`
`35
`
`40
`
`45
`
`a) a digital camera having:
`i) an electronic image sensor for sensing an image and
`producing a digital image;
`ii) a memory for storing digital images produced by the
`image sensor in digital image files, the digital image
`files having associated information for controlling a
`remote image fulfillment server;
`iii) a voice recorder for digitizing voice commands
`relating to control of the image fulfillment server;
`and
`iv) a transmitter for transmitting the digital image file
`to the image fulfillment server;
`b) a voice recognition module responsive to the digitized
`voice commands for producing control signals for the
`image fulfillment server; and
`c) an image fulfillment server, having:
`i) a receiver for receiving the digital image file and
`control signals;
`ii) a memory for storing the received digital image file;
`and
`iii) a file manager for managing the digital image file in
`response to the control signals.
`2. The system for digital image capture and transmission
`claimed in claim 1, wherein the voice recognition module is
`located in the camera and the control signals are transmitted
`with the digital image file to the image fulfillment server.
`3. The system for digital image capture and transmission
`claimed in claim 1, wherein the voice recognition module is
`located in the image fulfillment server and the digitized
`voice commands are transmitted to the image fulfillment
`server.
`4. The system for digital image capture and transmission
`claimed in claim 1, wherein the image fulfillment server
`includes a telecommunications processor having a connec(cid:173)
`tion to a plurality of secondary communications channels
`and the file manager manages the digital image file by
`50 storing the digital image file in the memory or causing the
`digital image file to be transmitted over one or more of the
`communications channels according to the associated infor(cid:173)
`mation.
`5. The system for digital image capture and transmission
`55 claimed in claim 1, wherein the image fulfillment server
`includes a printer for printing the digital image and an
`annotation on the print, and the control signals include
`instructions for printing.
`6. The system for digital image capture and transmission
`60 claimed in claim 1, wherein the control signals include an
`image file name.
`7. The system for digital image capture and transmission
`claimed in claim 5, wherein the control signals include an
`address for sending the print.
`
`* * * * *
`
`7