Ex. GOOG 1005
`
`EX. GOOG 1005
`
`
`
`
`
`

`
`United States Patent [19]
`Bernardi et a1.
`
`[54] CAMERA ON-BOARD VOICE
`RECOGNITION
`
`[75] Inventors: Bryan D. Bernardi, Rochester; Dale F.
`1134:1235; fijz'i'fgg'lilgzkwyv‘ahmu
`Honeoye Falls, all of NY.
`
`’
`
`’
`
`'
`
`’
`
`-
`
`[73] Assignee: Eastman Kodak Company, Rochester,
`N_YI
`
`21 A . N .:
`14
`l
`1
`P111
`0 2984“
`[22] Filed:
`Aug. 30, 1994
`
`Int. Cl.6 .......................... .. G03B 29/00; 6033 17/24
`[51]
`[521 US. (:1. ............................................. .. 354/76; 354/106
`[58] Fi?lli of Search ................................... .. 354/105, 106,
`354/75, 76
`
`[56]
`
`References Cited
`
`2,950,971
`3,439,593
`4,270,853
`4,270,854
`4,344,632
`
`U'S' PATENT DOCUMENTS
`8/1960 Lewin ................................... .. 430/140
`4/1969 Weitzner at B1,
`____ __ 95/12
`6/1981 Hatada et al.
`354/76
`6/1981 Slemrrt? 61 al- -
`354/76
`354/62
`8/1932 Hallo“ ~_ ---- -
`lanllg‘zchl e‘ “L
`9/1986 03;; W """ "
`4’6l3‘911
`4’642'717 M987 Matsu'éz'i'g'gl
`4:742:369
`5,1938 Ishii a aL
`4305.029 2/1990 Kelley __________ __
`4,951,079
`8/1990 Hoshino et al.
`4,965,626 10/1990 Robison et a1.
`4,977.419 12/1990 Wash et al
`4’933'996
`1/1991 Kinoshim
`:e?‘m
`o 13.0“ "" "
`'
`’
`.. 354/412
`6/1991 Hoshino et a].
`5,027,149
`5070 355 1211991 [1101.18 613.1. ........................... .. 354/106
`5:097:278
`3/1992 Tamamura et a1. ................... .. 354/105
`5,099.262
`3/1992 Tanaka et a1.
`.. 354/105
`5,128,700
`7/1992 Inoue et a1. ............................. .. 354/76
`
`360/3
`“560/105
`354/106
`354/75
`.. 354/412
`355/40
`354/76
`354/76
`
`lllllllllllllllllllllllllllllllllllllllllllllllllllllllll|||l|||l|l|l|||ll|
`US005546145A
`[11] Patent Number:
`5,546,145
`[45] Date of Patent:
`Aug. 13, 1996
`
`8/1992 Taniguchi et al.
`5,142,310
`9/1992 Hoda et a1. ..... ..
`5,146,249
`5,160,952 11/1992 Iwashita et al.
`303M111; ----- '-
`5/1994 lnoue eta]
`5,313,235
`Ba" et a1
`5’ 6 7 5
`FOREIGN PATENT DOCUMENTS
`
`s
`
`v
`
`e B
`
`354/106
`354/21
`354/76
`32‘;
`.. 354/76
`
`42 , 4
`
`6/1995
`
`"
`
`............................. .. 395/375
`
`0342628 11/1989 European Pat. O?". . ..... _. G03B 17/30
`
`OTHER PUBLICATIONS
`.
`.
`Patent Abstract of Japan; Kazuya, Nomura; Shutter Dnvmg
`Device; Sep. 25, 1991; vol. 15 No. 380 (p.-1256).
`
`ABSTRACT
`
`Primary Examiner—Michael L. Gellner
`Assist/ml Examiner—Mi¢ha¢1 Dalakis
`Mame?’ 488"" 0’ Fi""—David A- Howlcy; Joseph F
`Breimayer
`57
`[
`1
`In a camera, apparatus and methods for recording or storing
`voiced in messages in message memory associated with an
`BXPOS?d im?s? frame for reproduction as an annotation with
`prints made from the exposed image frames. The camera
`includes a voice recognition system for voice recognition of
`words spoken by the user before the words are stored in
`message memory. Fixed and adjustable vocabularies are
`provided for use in the voice recognition. The adjustable
`vocabulary may be loaded into the camera by a vocabulary
`memory card or through an interface with a docking station
`of a personal computer or vending machine so that an
`adjustable vocabulary of words of interest to the camera user
`or related to a scene or event of photographic interest may
`be employed in the voice recognition. The messages may be
`recorded on magnetic ?lm layers or stored in camera on
`board memory or in detachable message memory modules to
`be forwarded with the exposed ?lm strip to a photo?nisher
`for read out and printing on the prints. The messages may be
`.
`.
`.
`“med "1 ‘ecmded by ‘he use‘ 1“ real “me Wm‘ each
`exposure Or a‘ an garner or later time
`
`22 Claims, 10 Drawing Sheets
`
`22
`
`50
`
`DIGITAL
`MEMORY
`
`25
`
`DOCKING STATION
`(VENDING MACHINE. PC)
`1
`I
`.
`CAMERA SYSTEM
`EXTERNAL INTERFACE
`(IE. RS232 0R KEYBOARD) _~ MICROCONTROLU-IR
`\
`1”
`32
`23
`/
`/
`
`I
`
`\
`\
`
`l
`
`2o
`
`17
`
`24
`/
`ANALOG
`AMPLIFIER
`AND DATA __
`PROCESSING
`p
`+
`
`25 f “$220,, “_
`‘C
`
`l
`l
`voIcE
`40f RECOGNITION
`UNIT
`
`I
`FILM
`READ/WRITE
`INTERFACE
`/ CIRCUITS
`27
`l
`l
`f MAGNETIC
`13
`LAYER
`ON FILM _-
`"a
`
`DISPLAY
`uNIT
`LCD
`
`30
`
`23
`/
`FILM
`ADvANcE
`MOTOR
`CONTROLLER
`l
`FILM
`ADvANcE
`l/IOTDR \29
`ADJUSTABLE
`V0?ABULARY
`W SPEAKER
`mm
`VOCABULARY DEFENDANT 0R
`W/SPEAKER
`INDEPENDANT
`DEFENDANT
`TRAINING
`TRAINING (LE.
`(LE. RAM GR
`AND RAN)
`EEPROM)
`l l
`l 1 \
`f
`46
`
`42
`\
`FIXED
`vocAauLARY
`w/sPEAKER
`INDEPENDANT
`TRAINING
`(IE ROM)
`l l
`1
`
`Ex. GOOG 1005
`
`

`
`US. Patent
`
`Aug. 13, 1996
`
`Sheet 1 of 10‘
`
`5,546,145
`
`l/S
`
`Ex. GOOG 1005
`
`

`
`US. Patent
`
`Aug. 13, 1996
`
`Sheet 2 of 10
`
`5,546,145
`
`mm :1 n
`
`_ .
`
`d \\l.rll " ~ ,
`_ _ I: _
`u u _ II. - - I _
`
`_ _ a s
`
`Q0“ 0
`
`noooooo
`.- on...‘
`
`00000-00000
`0000.
`
`
`
`\ III.\\
`
`N .01
`
`in
`
`Ex. GOOG 1005
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 3 of 10
`
`5,546,145
`
`
`
`
`
`mo25.as.n._._Vuz_z_<Eoz_z_<E
`
`cgomnmm
`
`A2<moz<
`
`38.H._._V
`
`mmjofizoo
`
`
`
`
`
`
`uo<.._~m:z_\:2:moz<>o<H_~s<o<m_mmmmm+><#_ma
`2......ofi._zu<2o9<z<8I5.
`m:8m_oR9:~68:
`
`2......3....
`
`
`
`
`
` E..._oEz8omo_283rl2Hm>m(EsIIon8.&o2n._z ._<:o_oNu
`
`zoE:mozcaoo
`
`
`
`Gm.mz_:o<:oz_ozH._>V
`
`
`
`u._o<..EHz_ézmfixm
`
`m<om»mv_moNnmmmas
`
`
`
`
`
`mu«Eb:2......2o<55oz<
`
`uoz<>o<E5:2~m_::n_2<
`
`u._m<»m:3<
`
`>m<.5m<oo>
`
`¢¢
`
`
`
`N.‘ozmmmoowa
`
`
`
`
`»m<.sm<oo>E<Sm<oo>mommmooma:,_<ozm_&oz_9»z<ozmnmaz_~mv_<Em\;~m_v_<mn_m\;mmmoPz<ozmnmo
`
`
`~_mv_<mn_m\;Bx:Bx:ozaom
`oz_z_<EE<ozunm_o
`
`
`5ON
`
`we.Ill
`
`zoEzooom_m9.
`M.o_..._
`
`EX. GOOG 1005
`
`Ex. GOOG 1005
`
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`hS
`
`
` o3<z<mon53,...m1.E._..oEz8mcaoma\\/Im.8..mo.5s_
`05232moz<>o<~m_><._E_..__._n_s_<%_2._:
`
`
`
`
`H.:_m;\o<mmmmmm++><._n_m5E:
`mo<.EEz_\/A:2:moz<>o<
`
`
`
`mmmohoz_2.__.._2oIo<zwm_%¢_Dom
`
`mm
`
`
`
`moEEE_ézfibm
`
`afiomfixmoNmmmmas
`
`8.E05: ._<:u_omm
`
`8;.mz_:o<2oz_ozu>V
`
`zoE:mozcaoo
`
`5,546,145
`
`4E<.Sm<oo>Bx:Bx:mH.:mEm2.o<
`
`
`
`.mmn._v_<mn_m\;E5:m<oo>§5:m<oo>ozaom
`
`
`
`»z<ozm%az_o_mmo:,_<ozm_nm_o~mx<H._n_m\;~mv_<mn_m\;mommmoowamm
`
`
`
`.m._voz_z_<m._..m._Voz_z_<Eoz_z_<E
`
`§<mz_8.:sex2.on:3...”.M3
`
`czomnfimmoC2/xxoz<§om.z_
`E<ozH._nmaz_Hz<ozm&o
`
`.5—zoEzooomm3
`H55.n.o_.._
`
`EX. GOOG 1005
`
`Ex. GOOG 1005
`
`
`
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 5 of 10
`
`5,546,145
`
`><._n_m_o
`
`:2:
`
`00..
`
`on
`
`mm
`
`~m:._oEz8
`
`mtnoma
`
`:.=..._
`
`uoz<>o<
`
`58:
`
`H§<nm_m
`
`Eozmz
`
`M32902
`
`m4m<hm:=o<
`
`>m<4:m<oo>
`
`mmx<u¢m\;
`
`Bx:Bx_..._
`
`
`
`E<.sm<oo>E<Sm<oo>
`
`
`
`~65:E_~§\a<um\/m_oz<>o<Ego:numm++3:zo:<.6zz<
`
`arl
`
`_2u.:m»m<E:<o
`
`
`~m_joEzooomo_:m<o@@_moNnumm.m._V
`
`
`
`n._o<.._~m:z_4<z~mEn._
`
`.8
`
`mu>mo_2m_s_ ._<:oaNu
`
`
`Ga.n._z__._o<2oz_ozH._>V
`
`
`
`2925ozcaoo
`
`on
`
`o
`
`mo»z<ozu¢uo
`
`$v_<mEm\;$xfi.Em\;
`
`._.z<ozm_n_u
`
`oz.
`
`3
`
`Azomnmm
`
`zé.m._
`
`oz_z_<E
`
`W5
`
`A2<moz<
`
`
`
`zom.m_._V929..m.o
`
`
`
`oz_z_<Eoz_z_<E
`
`
`
`Eézunmopz<ozm.maz_
`
`oo._<z<
`
`E_..__._n_:<
`
`<._.<ooz<
`
`ozzom
`
`mommuooma
`
`O.
`
`:2:
`
`zoEzooH._m
`2m.o_.._
`
`EX. GOOG 1005
`
`Ex. GOOG 1005
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 6 of 10
`
`5,546,145
`
`E<Sm<oo>Bx:
`
`
`
`
`mmx<mmm\3>m<4:m<oo>»m<4:m<oo>
`
`omx:
`
`
`
`
`mo._.z<ozmn_m_ommx<mn_m\>>mn._x<mn_m\>>
`
`MSoz_z_<Easoz_z_<E.oz_z_<E
`
`
`SE3%oz<Eomz_waxz_Hwm_.m_zom2.on:on:as
`
`
`Hz<ozmmmoz_+z<ozmmmoFz<ozmmmoz_
`
`
`
`
`
`:2:moz<>o<Esme:
`
`
`
`2IEjofizoumtaogo
`
`
`
`Q3mopozH_%<o<mm
`
`mm
`
`
`
`_3_.._E<m<nmm
`
`
`
`moz<>o<»mo:m2
`
`mm
`
`m4m<»m:ao<
`
`am«So:
`
`¢¢.
`
`m4:ooz
`
`N¢
`
`
`
`
`
`>5n_m_oE.__.._zo:<5zz<
`
`aIII.
`
`on
`
`o
`
`-8
`
`
`
`moEmEz_._<z~mEa
`
`m<om>m§moNmmmmEV
`
`mmE052 ._<:o_oNu
`
`
`Sn..mz_.._o<2oz_ozm>V
`
`zoE§moziooo
`
`oo4<z<
`
`mm::m:<
`
`<H<Qoz<
`
`o_
`
`mommmooma
`
`mm
`
`E23
`
`zozgzooomm
`2m.0:
`
`EX. GOOG 1005
`
`Ex. GOOG 1005
`
`
`
`
`
`
`
`
`
`
`

`
`US. Patent
`
`Aug. 13, 1996
`
`Sheet 7 of 10
`
`5,546,145
`
`S10
`
`—-——-> SPEAK SINGLE
`DESIRED WORD
`
`%
`
`ATTEMPT To I512
`RECOGNIZE WORD
`
`YES
`
`IS WORD
`RECOGNIZED?
`
`SPEAK ANOTHER
`WORD?
`
`FIG. 7
`
`Ex. GOOG 1005
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 8 of 10
`
`5,546,145
`
`Q1?
`
`f S20
`
`ANNOTATION READ
`AT PHOTO—FINISHER
`I
`PHOTO-FINISHER
`f $22
`TRANSFERS ANNOTATION
`TO PRINT
`I
`TRANSFER AVAILABLE
`SOUND BITES TO
`
`FIG. 8
`
`AUDIO MEDIA 69
`
`@ TAKE PICTURE
`
`T—
`
`S30 \L
`
`FIG.
`
`VOICE
`RECOGNITION
`UNIT
`I
`STORE PHRASES TO
`FILM OR SEPARATE
`MODULE
`
`S36
`
`CONTINUE WITH
`THIS ROLL?
`
`TRANSFER TO PRINT
`
`@ID
`
`Ex. GOOG 1005
`
`

`
`US. Patent
`
`Aug. 13, 1996
`
`Sheet 9 of 10
`
`5,546,145
`
`i
`
`TAKE PICTURE
`
`f 540
`
`S41
`
`IF USING PREVIOUS
`ANNOTATION
`
`VOICE
`RECOGNITION I342 -
`UNIT
`I
`SELECT NUMBER OF
`FRAMES TO APPLY I543
`THIS PARTICULAR
`ANNOTATION TO
`
`STORE PHRASE TO I844
`FILM OR SEPARATE
`MODULE
`
`CONTTNUE WITH‘
`THIS ROLL?
`
`TRANSFER TO PRINT
`
`I545
`
`FIG. 10
`
`@b
`
`Ex. GOOG 1005
`
`

`
`US. Patent
`
`Aug. 13, 1996
`
`Sheet 10 of 10
`
`5,546,145
`
`a“ TAKE PICTURE
`
`f S50
`
`I
`I
`SPEAK DESIRED
`851
`PHRASE AND
`STORE THE f
`SPEECH
`
`S52
`
`CONTINUE PICTURE
`TAKING?
`
`S53
`PLAYBACK SPEECH f
`OF COMPLETED FRAMES
`AS A REMINDER
`I
`$54,
`VOICE
`RECOGNITION f
`um
`I
`STORE PHRASE TO I555
`FILM OR SEPARATE
`MODULE
`
`CONTINUE WITH
`THIS ROLL?
`
`S57
`
`FIG. 11
`
`TRANSFER TO PRINT —@
`
`Ex. GOOG 1005
`
`

`
`5,546,145
`
`1
`CAMERA ON-BOARD VOICE
`RECOGNITION
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`Cross-reference is hereby made to commonly assigned
`U.S. patent application Ser. No. 08/298,101 ?led on even
`date herewith to VOICE RECOGNITION OF RECORDED
`MESSAGES FOR PHOTOGRAPHIC PRINTERS in the
`names of Dale F. McIntyre, Bryan D. Bemardi and Dana W.
`Wolcott.
`
`FIELD OF THE INVENTION
`
`This invention relates to the ?eld of photographic ?lm
`cameras and ?lm processing, and particularly to apparatus
`for providing voice information storage and retrieval capa
`bilities employing speech recognition.
`
`BACKGROUND OF THE INVENTION
`
`A variety of advanced still camera systems have been
`recently disclosed in which data related to the scenes pho
`tographed is input into memory associated with the ?lm strip
`to be forwarded to the photo?nisher to aid in making prints,
`and, in some instances including messages or annotations to
`be associated with the prints. As set forth in commonly
`assigned U.S. Pat. No. 5,276,472, (incorporated herein by
`reference in its entirety) systems for providing voice record
`ing in cameras in association with the exposure of each ?lm
`image frame have become known in the art. In the ’472
`patent, a system is provided for recording a message in
`transparent magnetic ?lm tracks in a magnetics-on-?lm
`(MOF) layer on the non-emulsion side of the ?lm overlying
`the image frame area. The user may speak words into a
`camera microphone/speaker that are processed into digital
`signals stored temporarily in memory. When the message is
`completed, the user may command the camera to play it
`back audibly for review and editing as considered necessary.
`When the message content is considered satisfactory, the
`digitized annotation may be recorded in the MOF layer
`tracks of the exposed image frame during advancement of
`the ?lm to the next image frame. The user may continue
`recording messages for each image frame exposed, and the
`fully exposed ?lm is processed so that the resulting prints
`carry the annotations corresponding to the messages
`recorded.
`The ‘472 patent is directed to such a system where the
`coding of each message on the associated print is done
`during the photo?nishing operation in such a way that the
`machine readable coded information allows the audible
`reproduction of the message through the use of a special
`hand-held scanner. The coded information is in the form of
`a bar code, a blister spot pattern or the like that may be
`scanned and translated into an audio voice reproduction by
`the hand-held scanner moved over the coded information. A
`brief alphanumeric place and date annotation may also be
`printed in the border of the print.
`The system disclosed in the ’472 patent thus simply
`phonetically processes the speech that is recorded into coded
`information, and the hand-held scanner phonetically repro
`duces the words. A brief, alphanumeric place and date
`annotation may also be printed in the border of the print by
`the user or the photo?nisher from listening to and manually
`transcribing the recorded message. The quality and accuracy
`of reproduction of the message depends on how carefully
`
`20
`
`25
`
`35
`
`50
`
`55
`
`60
`
`2
`and clearly the words are spoken by the user during the
`editing and re-recording operation.
`In order to store such information, as well as the image
`frame identi?cation to which it pertains, and other informa
`tion automatically entered from the camera operating system
`or optionally entered by the camera user, it is necessary to
`employ such a ?irther writable and readable media in
`association with either the ?lm (as disclosed in the ’472
`patent) or in some other storage media or memory.
`With respect to recording camera operating conditions
`and time and date for each image frame exposure other than
`on the ?lm itself, it has been proposed to magnetically read
`and write data on magnetic strips formed on the sides or an
`end of the ?lm cartridge, as described, for example, in U.S.
`Pat. No. 4,443,077. More recently, it has been suggested that
`such data may be stored and retrieved from non-volatile
`memory chips, e.g. an EEPROM, incorporated in an inte
`grated circuit chip “card” as set forth in U.S. Pat. No.
`5,128,700. The card may be separable from the camera and
`?lm cartridge, or a similar EEPROM card may be attached
`to the ?lm cartridge as set forth generally in U.S. Pat. No.
`5,070,355. Alternatively, the storage of such information in
`“ROM-ICs” attached permanently or releasibly to the sides
`or ends of ?lm cartridges is disclosed in U.S. Pat. No.
`5,142,310.
`The ‘700 patent also stores sound or voice messages
`related to the image frames in the removable sound cards
`that are intended to accompany the ?lm when it is sent in for
`processing, so that the message may be reproduced as a
`sound code with the print made from the negative ?lm image
`frame for phonetic playback of the message. Alternatively,
`the card itself is read out phonetically. As in the ‘472 patent,
`the sound is reproduced phonetically as the coded informa
`tion or recorded data is scanned.
`In another embodiment disclosed in the ’700 patent and in
`a further U.S. Pat. No. 4,344,682, a camera is described for
`recording information related to each image frame as small
`alphanumeric characters exposed in a comer portion of the
`image frame for photographic reproduction on the print
`made from the negative image frame. The information may
`be recorded or stored in temporary memory prior to making
`the exposures of the image frames and exposed on the image
`frames in conjunction with the image frame exposure. As
`each message is inputted into memory, it is displayed and
`may be edited. The input mechanism depicted is a keyboard,
`but it is suggested that other input means, including a voice
`recognition device, may be substituted for the keyboard.
`In an electronic still camera disclosed in U.S. Pat. No.
`4,742,369, it is also suggested that a keyboard or a voice
`recognition circuit be employed to input spoken information
`to be stored regarding each image that is captured by the
`camera.
`In a further camera system, e.g. that disclosed in U.S. Pat.
`No. 5,027,149, voiced commands are given by the user to
`command each camera operation. A voice recognition circuit
`is employed in a system for training the camera to recognize
`and respond accurately to the spoken commands.
`
`Problems to be Solve by the Invention
`
`It is di?icult to accurately reproduce spoken words stored
`in analog or digital form in a non-photographic media
`associated with ?lm strip image frames as alphanumeric,
`readable annotations on the prints made from the image
`frames. It is desirable to make the reproduction without
`human inspection of the annotation before it is printed in the
`
`Ex. GOOG 1005
`
`

`
`5,546,145
`
`3
`border of the print. It is not possible to rely simply on the
`editing process ‘and careful pronunciation of the words to
`ensure that they are processed accurately before they are
`stored for later read out and printing as annotation on the
`prints made from the exposures.
`
`SUMMARY OF THE INVENTION
`
`It is therefore an object of the present invention to provide
`a method and apparatus for ensuring the accurate processing
`of spoken words that can be accurately reproduced by
`photo?nishing equipment.
`These and other objects of the invention are realized in a
`photographic camera including an optical lens, a photo
`graphic ?lmstrip transport mechanism for advancing the
`?lmstrip in a path of travel to and through an image frame
`exposure gate with respect to said optical lens, and an
`exposure system for making an exposure of the ?lmstrip
`image frame in the exposure gate, apparatus for recording a
`voice message related to the exposure made or to be made
`for playback in conjunction with making prints from the
`photographic images captured in the image frames of the
`?lmstrip to provide for the printing of the voice message
`therewith comprising: speech input means into which a
`camera user may speak words of the message to be stored
`with respect to the ?lmstrip image frames; sound processing
`means for processing the words spoken into the speech input
`means as voice digital data; means for providing reference
`voice digital data corresponding to a reference word vocabu
`lary; speech recognition means for comparing the processed
`voice digital data to the reference voice digital data and
`recognizing processed voice digital data corresponding to
`the reference vocabulary voice digital data; message
`memory means having memory locations related to each
`image frame of the ?lmstrip for storing recognized voice
`digital data; and means for storing the recognized voice
`digital data in said message memory means.
`A variety of vocabulary sources may be employed to load
`in a ?xed vocabulary and adjustable vocabulary of voice
`digital data corresponding to commonly used words and
`words speci?c to an event or attraction of photographic
`interest. The vocabulary sources may be detachable vocabu
`lary memory cards insertable into the camera for connection
`with the camera system or may comprise an interface for
`down loading vocabulary words from a camera docking
`station.
`Similarly, the message memory means may comprise
`memory media associated with the ?lm strip and detachable
`with the ?lm cartridge for transfer to the photo?nisher or
`may remain in the camera if the camera is a single use,
`recyclable camera returned with the ?lm cartridge to the
`photo?nisher for processing.
`The user may employ methods of recording or storing the
`messages in real time with the exposure of each image frame
`employing voice recognition or at a later time.
`ADVANTAGES OF THE INVENTION
`
`The invention advantageously results in the storage of
`accurate word messages in relation to ?lm strip image
`frames that may be automatically read out and accurately
`printed as readable annotations on the associated print
`without requiring the photo?nisher to interpret and correct
`the message before it is printed.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`These and other objects, advantages and features of the
`invention will be become apparent from the detailed descrip
`
`20
`
`25
`
`30
`
`45
`
`50
`
`55
`
`65
`
`4
`tion given hereinafter in relation to the accompanying draw
`ings, in which:
`FIG. 1 is a diagram illustrating partial dedicated tracks in
`a virtually transparent MOF layer and a cross section of the
`layers of ?lm particularly adapted for use in a camera of a
`?rst embodiment of the invention;
`FIG. 2 is a schematic illustration of a camera having
`speech recording apparatus in accordance with the various
`embodiments of the invention;
`FIG. 3 is a schematic block diagram of a system for
`recording speech in a camera in accordance with the ?rst
`embodiment of the invention;
`FIG. 4 is a schematic block diagram of a system for
`recording speech in a camera in accordance with a ?rst
`variation on the ?rst embodiment of the invention;
`FIG. 5 is a schematic block diagram of a system for
`recording speech in a camera in accordance with the second
`embodiment of the invention;
`FIG. 6 is a schematic block diagram of a system for
`recording speech in a camera in accordance with a ?rst
`variation on the second embodiment of the invention;
`FIG. 7 is a ?ow chart of the camera based voice recog
`nition steps taken to accurately store and record each word
`of a message voiced by the user;
`FIG. 8 is a ?ow chart of the steps of the transfer to print
`operation undertaken by the photo?nisher;
`FIG. 9 is a ?ow chart of the combined steps of recording
`a message in real time relation to the exposure of each image
`frame and the ?nishing of the prints with the message
`appearing as an annotation on the print;
`FIG. 10 is a flow chart of the combined steps of recording
`a message using terms common to a series of exposures and
`the ?nishing of the prints with the common terms appearing
`as an annotation on each print; and
`FIG. 11 is a flow chart of the combined steps of recording
`a voice message temporarily for each image frame and
`editing and re-recording the edited messages at a later time
`and the ?nishing of the prints with the message appearing as
`an annotation on the print.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS OF THE
`INVENTION
`
`In accordance with an aspect of the invention, the camera
`operating system includes speech recognition of spoken
`words which are compared to an on-camera word vocabu
`lary stored in ?xed vocabulary ROM and adjustable vocabu
`lary RAM or EEPROM or the like as described hereafter.
`Spoken words are processed and compared to the vocabu
`lary. The acceptance of the word is indicated by displaying
`it to the user on the camera LCD display or by audibly
`playing back the closest matching word. Rejection of the
`word may also be indicated to the user. The user may either
`speak the word or an alternate word and repeat the process
`until the word is matched and accepted. In this aspect of the
`invention, the recognized words form a message that is
`stored in memory associated with the ?lm image frame so
`that the memory accompanies the ?lm to the photo?nisher
`where each message can be read out and printed along with
`the respective print made from the ?lm image frame. The
`speech recognition operation results in a more automated
`photo?nishing operation not requiring constant operator
`monitoring and translation of the messages into more read
`able text.
`
`Ex. GOOG 1005
`
`

`
`5,546,145
`
`5
`Due to the space and power supply limitations inherent in
`miniaturized still cameras, it is not possible at this time to
`provide a large scale vocabulary covering all words of a
`given language in the camera or to provide the processing
`speed su?icient to effect the comparison of the voice digital
`data to the memorized word data in a reasonable time period.
`In a further re?nement of the invention, it is proposed that
`the vocabulary of words likely to be spoken by the user to
`describe the scene being photographed include a ?xed
`vocabulary and an adjustable vocabulary word set. The
`speech recognition operating system is constructed to
`accommodate the ?xed vocabulary of common words stored
`in ROM likely to be used in most situations, e.g. the months
`of the year. The system accommodates an adjustable
`vocabulary or vocabularies of other words selected by the
`user and stored in RAM or EEPROM in advance of using the
`camera and which may be related to speci?c persons, events
`or attractions and events. The manner of storing an adjust
`able vocabulary may include a variety of means and sources,
`including words keyed in on a camera mounted keyboard,
`words stored in a personal computer and downloaded
`through a connector interface, or words stored in plug-in
`RAM or EEPROM cards inserted into special slots of the
`camera.
`Voice recognition systems that are available commer
`cially function in a variety of ways to process speech or
`voiced words to derive a match to a pre-stored word in a
`vocabulary memory unit. Voice recognition systems may be
`used with vocabulary memory units which involve either
`“speaker independent training" or "speaker dependent train
`ing“. Speaker independent training means that each word in
`a given vocabulary has enough training patterns stored in
`memory in association with it to distinguish that word from
`any other word in the vocabulary regardless of the pronun~
`ciation of the word by a general population of potential
`users. The training patterns for the various pronunciations of
`such words are determined in advance and stored for the
`words of the vocabulary memory unit before it is supplied to
`the user in the camera or otherwise as described below. Such
`words should be recognized by the system when spoken by
`a fairly broad spectrum of the population and do not require
`the user to undertake a personalized training regimen of the
`vocabulary words prior to voicing the words. However,
`storing a sufficient number of training patterns for each word
`to ensure reliable recognition is expensive and does take up
`memory space.
`Speaker dependent training means that the words of the
`vocabulary are trained to be recognized when spoken by a
`speci?c person, that is, the trainer and user of the vocabu
`lary. In a training session initiated by the camera and the
`user, the words in the vocabulary may be displayed one at a
`time, and the user voices the words into the camera micro
`phone. A training pattern is generated and stored as the user
`voices each word in the course of completing the training
`session. For example, if a ?xed vocabulary is subjected to
`speaker dependent training, the speaker is prompted by a
`displayed or aural command to voice each word of the
`vocabulary one or more times in sequence. Each spoken
`word is then processed and the processed signal is stored as
`the pattern to be recognized for that word in the future.
`While the process may be time consuming, the memory
`space required for the unique training pattern associated
`with the word is reduced. And, of course, the speech
`recognition system for the speaker dependent vocabulary
`words may well be unusable by other persons.
`Commencing with the various embodiments of the inven
`tion, FIGS. 1 and 2 depict, in conjunction with the camera
`
`20
`
`25
`
`30
`
`50
`
`65
`
`6
`circuits of FIGS. 3 and 4, a ?rst embodiment of the invention
`in which the recognized voice digital data may be stored in
`the MOF layer tracks, and the ?xed and adjustable vocabu
`lary may be stored in the camera in a variety of ways. In this
`embodiment, the recognized words are stored in certain of
`the MOF layer tracks and are read out at the photo?nisher to
`be printed along with the prints made from the ?lm image
`frames.
`The camera circuits of FIGS. 3 and 4 dilfer in the type of
`vocabulary memories employed, as described in detail
`below. FIGS. 5 and 6 depict a further embodiment employ
`ing camera circuits similar to FIGS. 3 and 4 but for recording
`the voice recognized memories in separate memory mod~
`ules, rather than on the MOF layer of FIG. 1, as described
`below. In all of the FIGS. 1-6, the source of or means for
`providing the vocabularies may take a variety of forms as
`listed above. FIGS. 7-11 illustrate possible voice recogni
`tion and storage methods employing the apparatus of FIGS.
`1-6 for incamera voice recognition prior to permanently
`recording the messages.
`Referring ?rst to FIG. 1, a strip 10 of magnetically coated
`color negative ?lm, 35 millimeters wide, useful in the
`present invention includes a base 11, various well-known
`photo-chemical layers 12 on one side of the base 11 and a
`virtually transparent MOF layer 13 on the other side. An
`anti-static and lubricating layer 14 overlies the magnetic
`layer 13. The ?lm strip 10 includes perforations 15 along the
`?lm edge at regular intervals matching the pitch of a
`metering pawl in a camera adapted to use the ?lm strip 10.
`For purposes of recording data in the MOF layer 13, each
`frame of the ?lm strip 10 may be formatted as shown in FIG.
`1 and more fully described in commonly assigned US. Pat.
`No. 4,977,419, the disclosure of which is incorporated
`herein by reference. More speci?cally, the frame area is
`divided into a plurality of predetermined longitudinal track
`locations designated in the drawing as outermost tracks
`C0—C3 and innermost tracks Foil-F29. As described more
`fully in the ‘419 patent, certain of the tracks may be reserved
`for recording of information in the camera using magnetic
`recording means included in the camera. In addition, other
`tracks may be reserved for use by the photo?nisher. Addi
`tionally, the ‘419 patent indicates that certain of the tracks
`may be used for recording of audio information. Apparatus
`for magnetically recording information in the camera is
`more fully described in the ‘419 patent and is not repeated
`here except to the extent elements thereof are relevant to an
`understanding of the present invention.
`Referring to FIG. 2, a camera 16 is schematically illus
`trated with a variety of features and components usable
`separately or in various combinations in the systems and
`methods described below. In a ?rst embodiment of the
`invention, the camera 16 is speci?cally adapted to receive
`and function with ?lm having the MOF layer 13 of FIG. 1.
`Camera 16 is provided with a built-in audio transducer, e. g.
`microphone 17, an internally mounted micro-chip 18, a
`magnetic recording head 19 and a miniature speaker 20.
`Camera 16 may also be provided with an LCD panel 30 for
`displaying various camera settings and conditions of the
`type well known in the art. In addition, the LCD panel 30
`may display words that it has recognized to the user for
`con?rmation of the recognition as described below. Various
`conventional user inputs 22 are also provided on the camera
`16.
`The camera 16 may also have an external interface 32 for
`receiving and transmitting vocabulary words to memory in
`micro-chip 18 or for reading out message words stored in
`
`Ex. GOOG 1005
`
`

`
`5,546,145
`
`7
`such memory. For example, the external interface 32 may
`include an RS-232 port, so that camera memory in micro
`chip 18 may be accessed through a computer based docking
`station 50 to load an adjustable vocabulary of words chosen
`by the user or obtained at a particular attraction or event by
`inserting the camera 16 into a docking station of a vending
`machine or the like. In the event that the camera is recy
`clable, the interface 32 may be employed by a docking
`station 50 operated by the photo?nisher to read out the
`messages stored in camera on-board memory. In such a case,
`it would not be necessary that the camera also employ the
`MOF layer 13 on the ?lm strip 10 or the cartridge related
`memory module described below to record or store the
`message the user wishes to appear on the prints.
`Alternatively, the external interface 32 may comprise a
`keyboard on the camera of the type described above in
`reference to the ’682 patent for keying in vocabulary words
`for voice recognition as one manner of storing the adjustable
`vocabulary words.
`FIG. 2 also schematically illustrates a message memory
`module 38 that may be inserted into a slot or attached to a
`?lm cartridge 36 in the camera 16 so that the recording of the
`image frame related messages may be made in the memory
`module 38 rather than in the MOF layer of the ?lm strip 10.
`Such a memory module 38 may take the form of the ?lm
`cartridge end attached modules of commonly assigned,
`co-pending U.S. patent application Ser. No. 071,084 entitled
`ORIENTATION INDEPENDENT, DETACHABLE FILM
`CARTRIDGE, MEMORY MODULE ?led on Jun. 4, 1993,
`in the name of J. David Cocca and Ser. No. 071,096 entitled
`ORIENTATION INDEPENDENT, DETACI'IABLE FILM
`CARTRIDGE, MEMORY MODULE ?led on Jun. 4, 1993
`in the name of Robert S. Bryant. Alternatively, the memory
`module into which the messages are recorded may be a
`plug-in 1C card of the type described in the above-referenced
`‘700 patent or the cartridge mounted ROM-[Cs described in
`the above-referenced ’310 patent or any other convenient
`form
`Finally, FIG. 2 also shows a separate vocabulary memory
`card 34 that may be inserted into a slot connector in the
`camera body for loading a particular ?xed vocabulary. The
`?xed vocabulary memory card 34 may take the form of a
`ROM-1C card of the type described in the above-referenced
`‘700 patent. Again, the vocabulary memory card 34 may be
`vended to the user at an event or attraction of photographic
`interest to customize and expand the vocabulary of ?xed
`words that may be recognized for a particular event or
`attraction of photographic interest. Ideally, the ?xed vocabu
`lary memory card 34 would have speaker independent
`training so that the purchaser