`
`
`as) United States
`
`
`
`
`
`
`
`
`a2) Patent Application Publication (10) Pub. No.: US 2003/0063321 Al
`
`
`
`
`
`(43) Pub. Date: Apr.3, 2003
`
`
`
`Inoueet al.
`
`US 20030063321A1
`
`
`
`(54)
`
`
`
`(75)
`
`
`
`
`
`
`
`IMAGE MANAGEMENTDEVICE, IMAGE
`
`
`
`
`MANAGEMENT METHOD, STORAGE AND
`PROGRAM
`
`
`
`
`
`
`
`
`Inventors: Daisuke Inoue, Kanagawa (JP); Naoki
`
`
`
`
`
`Shimada, Tokyo (JP); Takahiro Onsen,
`
`
`
`
`Kanagawa (JP); Koji Yoshida,
`
`
`Kanagawa (JP)
`
`Publication Classification
`
`
`
`
`
`
`
`
`
`
`
`(51) Ute C17 caccccsscsecsessenseee HO4N 1/21; HO4N 1/23
`
`
`
`
`(52) US. Ch.
`cecesesssssssssstsesiessetnstvasnetnatnssvesve 358/302
`
`
`
`
`(57)
`
`
`
`ABSTRACT
`
`
`
`
`
`
`
`
`
`
`
`Correspondence Address:
`HOGAN & HARTSONL.L.P.
`
`
`
`
`
`
`
`
`
`
`An image management apparatus that transmits image data
`500 S. GRAND AVENUE
`
`
`
`
`
`
`
`
`
`
`
`to an image processing apparatus is provided. The image
`SUITE 1900
`
`
`
`
`
`
`
`
`
`management apparatus includes a sound input unit
`that
`
`
`
`LOS ANGELES, CA 90071-2611 (US)
`
`
`
`
`
`
`
`inputs voice message relating to image data photographed
`
`
`
`
`
`
`
`by a digital camera. When oneof the image data is selected
`
`
`
`
`(73) Assignee: CANON KABUSHIKI KAISHA
`
`
`
`
`
`
`
`
`and a voice message relating to the selected image data is
`
`
`
`
`
`
`
`
`
`
`
`
`(21) Appl. No.:
`input via the sound inputunit, a translation unit of the image
`
`
`
`
`
`management apparatus automatically extracts keywords
`
`
`
`
`
`Sep. 25, 2002
`
`
`
`
`
`
`
`
`
`from the voice message. Thetranslation unit determines one
`
`
`
`
`
`
`
`
`
`
`of the keywordsasatitle, and sets thetitle as a file name of
`
`
`
`
`
`Foreign Application Priority Data
`(30)
`
`
`
`
`
`
`
`
`
`
`the image data. The extracted keywordsare set as data for
`
`
`
`
`
`
`
`
`searching images, and transmitted together with the selected
`
`
`
`
`
`
`image data to the image processing apparatus.
`
`10/254,612
`
`
`
`(22)
`
`Filed:
`
`
`
`
`Sep. 28, 2001
`
`
`
`Sep. 20, 2002
`
`
`
`(IP) eececssscsecseecnecneeeneess 303230/2001
`
`
`(IP) .eececcsscssscssecsescnseeneeeneess 274500/2002
`
`103
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Adapter
`
`Voice Processing
`
`
`Section
`
`
`
`
`
`
`
`
`
`
`
`Data Recognition Data Base 304
`
`}
`305
`302
`
`
`
`
`
`Voice Recognition-
`
`
`Voice Information
`
`
`
`
`
`
`
`
`Keyword Extraction
`
`‘
`
`
`Setting Section
`
`
`Section
`
`
`
`
`204
`
`
`
`
`
`
`
`Transmission File
`
`
`
`
`
`Image Information
`Digital Camera
`’
`
`
`Control Section
`
`
`Storage Section
`
`
`
`AdapterInformation - Communication
`
`102
`
`
`
`
`
` Control Section
`
`
`ManagementSection
`
`
`Sew eee ee ren te ee nee ee ee en renee eee
`
`Portable Communication
`
`Terminal 104
`
`
`
`Page 1 of 26
`
`Netskope Exhibit 1012
`
`Page 1 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`
`Patent Application Publication
`
`
`
`
`
`Apr.3, 2003 Sheet 1 of 11
`
`
`
`US 2003/0063321 Al
`
`
`Fig. 1
`
`
`
`107
`
`
`
`102
`
`
`
`103
`
`
`
`104
`
`
`
`Page 2 of 26
`
`Netskope Exhibit 1012
`
`Page 2 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`Patent Application Publication
`
`
`
`
`
`
`Apr.3, 2003 Sheet 2 of 11
`
`
`
`US 2003/0063321 Al
`
`81qeWod
`
`“NWO
`
`uoyeou
`
`jeususs3}
`
`df]eu}
`
`-nwiwoy
`
`uoiesiu
`
`je3161q
`
`OLC
`
`Lic
`
`£01
`
`
`
`
`
`
`
`
`
`
`
`Z‘Bis
`
`
`
`
`
`
`
`e13awes
`
`Page 3 of 26
`
`Netskope Exhibit 1012
`
`Page 3 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`Patent Application Publication
`
`
`
`
`
`
`Apr.3, 2003 Sheet 3 of 11
`
`
`
`US 2003/0063321 Al
`
`£0E
`
`a|l4uolssiwsueay
`
`
`
`unasafelojs
`
`
`
`UOIJEWUOJUYBIIOA,
`
`uonpagBulyjes
`
`
`
`uopoesxgpiomkay
`
`-UO}}UBOsay
`
`uojj9aS
`
`
`
`uonesiunWWoyajqeyod
`
`
`
`POPjeulunol
`
`uonedunuwoy
`
`
`
`UOdDagJO13;U0D
`
`
`
`uoroasjuswabeuew
`
`
`
`UOI}eWUOjJU]Ja}depy
`
`Bd1I0A
`
`
`
`
`
`Jaj}depy
`
`cOE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`uoljoegBuruie}qo
`
`v0T
`
`uoljaag
`
`ejeqad10A
`
`HuISS8d01dBDIOA
`
`
`
`UOI}DaS[OIJUOD
`
`uoewsojuyeBew|
`
`
`essweyjey!biq
`
`cOL
`
`
`
`Page 4 of 26
`
`Netskope Exhibit 1012
`
`Page 4 of 26
`
`Netskope Exhibit 1012
`
`
`
`
`
`
`
`
`

`

`
`
`
`Patent Application Publication
`
`
`
`
`Apr.3, 2003 Sheet 4 of 11
`
`
`
`US 2003/0063321 Al
`
`
`Fig. 4
`
`
`A
`
`—™S
`
`"Photograph of Nytview of Yofohama”
`
`
`
`
`Cc
`
`
`
`b
`
`
`
`a
`
`
`
`
`
`
`401
`
` <Key Word>
`
`
`
`Yokohama
`
`
`402
`
`Nightview
`Photograph
`
`
`
`403
`
`
`
` <Title>
`
`
`Yokohama
`
`
`
`
`
`<Image File Name>
`
`<Before>
`
`
`
`
`20001201.JPG
`
` <After>
`Yokohama.JPG
`
`
`
`
`
`
`
`404
`
`
`
`
`
`
`
`
`
`405
`
`
`
`06
`
`
`
`Page 5 of 26
`
`Netskope Exhibit 1012
`
`Page 5 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`Patent Application Publication
`
`
`
`
`
`
`Apr.3, 2003 Sheet 5 of 11
`
`
`
`US 2003/0063321 Al
`
`
`Fig. 5
`
`START
`
`
`
`
`
`Obtaina list of image fileg
`
`
`Obtain an image
`
`
`
`
`Voice Input Button
`
`
`
`
`
`Button selected?
`
`
`
`
`
`Image Selection
`
`
`
`
`
`Transmission Button
`Button
`S504
`
`
`
`
`
`Voice Input / Recording
`Obtain a file to be
`$510
`
`
`transmitted
`Process
`
`
`
`
`
`
`
`
`
`gs
`
`Connect to Application
`
`Server
`
`
`
`$511
`
`
`
`$512
`
`
`
`
`Voice Recognition
`
`Process
`
`
`
`Set Keyword
`
`
`
`
`
`
`
`
`Create Voice Information
`File
`
`
`
`
`
`
`
`
`Set Image Information
`
`
`
`Page 6 of 26
`
`Netskope Exhibit 1012
`
`Page 6 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`
`
`
`
`
`
`
`Apr.3, 2003 Sheet 6 of 11
`
`
`
`US 2003/0063321 Al
`
`LO9JBAIBSJJEMFI5
`
`
`
`}9UJ9}U|
`
`
`
`JaAIagUuoIeoddy
`
`Patent Application Publication p09UO!1}D9S
`
`
`
`
`
`uonesiddy
`
`
`
`UIBIAJAAIaS
`
`¢09Apog
`
`
`
`Buissa90j1dadl0A
`
`9‘Biy
`
`
`
`Page 7 of 26
`
`Netskope Exhibit 1012
`
`Page 7 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`Patent Application Publication
`
`
`
`Apr.3, 2003 Sheet 7 of 11
`
`
`
`US 2003/0063321 Al
`
`<a.
`
`
`
`uodI}iUBovayadIoA
`
`902Bseqejeq
`
`$02OL£02LOZ
`
`Zbl
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`puomAay
`
`uoI}!UBODaYSdIOA
`
`BJeQVDIOA
`BujuleyqQ
`
`
`
`U01}98SJ9e1}XA
`
`
`
`UOIPULOJU]BIIOA,
`
`uonoasBuyes
`
`UuO1}998S
`
`uol1}99¢g
`
`
`
`oljeuuojuyeBew|
`
`Bujuleyqo
`
`uol}Ia¢g
`
`TOL
`
`
`
`JaAlaSguoleoddy
`
`
`
`Apogulelyj
`
`q|sbew)
`
`UOH}EOUSA
`
`BuroyuUoW;aurq
`UdI}e.IUNWIWO4
`
`uo1}98S
`
`YIOMJON
`
`Page 8 of 26
`
`Netskope Exhibit 1012
`
`Page 8 of 26
`
`Netskope Exhibit 1012
`
`
`
`
`
`
`
`

`

`
`
`
`Patent Application Publication
`
`
`
`
`Apr.3, 2003 Sheet 8 of 11
`
`
`
`US 2003/0063321 Al
`
`
`Fig. 8
`
`START
`
`
`
`S802
`
`
`
`$803
`
`
`
`$804
`
`
`
`$805
`
`
`
`ane
`
`
`
`aad
`
`
`
`Voice Input/
`
`
`
`Recording Process
`
`
`
`Voice Recognition
`
`
`Process
`
`
`
`
`Set Title
`
`
`
`
`Information File
`
`
`
`Page 9 of 26
`
`Netskope Exhibit 1012
`
`Page 9 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`
`Patent Application Publication
`
`
`
`
`Apr. 3, 2003 Sheet 9 of 11
`
`
`
`US 2003/0063321 Al
`
`
`Fig. 9
`
`S901
`
`
`
`files
`
`
`
`Obtain a list of image
`
`
`
`
`$902
`
`
`
`
`
`
`
`
`
`®@
`
`
`
`
`
`
`
`
`$903
`
`
`
`Voice Recognition
`
`
`Database downloaded?—>
`$908
`
`
`@
`
`
`Voice Input Button
`
`Button selected?
`
`
`
`
`
`
`Transmission Button
`
`
`Obtain File to be
`
`
`transmitted
`
`
`
`Voice Input / Record
`
`
`Process
`
`
`
`
`Connect to
`
`Application Server
`
`
`
`
`Voice Recognition
`
`Process
`
`
`
`
`Transmit
`
`
`
`
`
`
`Set keyword
`
`y
`
`
`
`
`
`
`
`
`
`
`
`Operate Voice
`Information File
`
`
`
`
`
`
`Set Image Information
`
`
`
`
`
`
`
`
`
`
`
`$909
`
`
`
`$910
`
`
`
`soit
`
`
`
`$912
`
`
`
`913
`
`
`
`$914
`
`
`
`
`
`
`Obtain Date Data $905
` $906
`
`
`
` Transmit Date Data
` Receive Voice
`
`
`Recognition Database
`
`
`
`
`Image Selection
`
`Button
`
`
`
`
`
`
`
`
`
`
`
`
`® 391
`
`
`Connectto
`
`
`Application Server
`
`
`
`
`
`
`
`$916
`
`
`
`$904
`
`
`
`
`
`
`
`
`$907,
`
`
`
`
`
`
`
`
`
`@
`
`Page 10 of 26
`
`Netskope Exhibit 1012
`
`Page 10 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`
`Patent Application Publication
`
`
`
`
`Apr.3, 2003 Sheet 10 of 11
`
`US 2003/0063321 Al
`
`£001
`
`alqeuod
`
`uonesjunwiu0e9
`udIJeLUOJU]
`
`
`d/|JEU]
`Bulssad0ld
`
`UOo
`
`Jeuctysod
`
`
`
` UISS3D01d7|BION4/|esaweD\0G202\10
`
`5uonses
`
`-O191IN
`
`auoyd
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 11 of 26
`
`Netskope Exhibit 1012
`
`Page 11 of 26
`
`Netskope Exhibit 1012
`
`
`
`
`
`

`

`
`
`Patent Application Publication
`
`
`
`
`
`
`
`Apr.3, 2003 Sheet 11 of 11
`
`
`
`US 2003/0063321 Al
`
`Fig. 11
`
`
`
`
`Obtainalist of
`image files
`
`
`
`START
`
`
`
`|
`
`
`
`
`
`
`Obtain an image
`
`
`
` $1101
`
`
`
`84103 Voice Recognition
`
`
`Database downloaded?
`-
`
`$1108
`
`
`
`
`Image
`
`Selection B
`
`ction
`
`$1104
`Doi
`
`
`
`Obtain Positional
`
`Information
`
`
`
`$1108
`Connectto
`
`
`
`
`
`Application Server
`
`
`
`
`$1106
`Information
`Transmit Positional
`
`
`
`$1107
`
`
`Received Voice
`
`
`
`
`Recognition Database
`
`
`
`
`
`@
`
`
`
`
`@
`Voice Input Button
`
`
`p
`
`
`
`
`
`
`
`
`Button
`
`
`
`Button selected?
`
`
`
`
`
`
`Transmittion Button
`
`Obtain File to be
`
`
`transmitted
`
`
`
`
`Voice Input /
`
`
`Record Processing
`
`
`
`
`
`$1116
`
`$111
`
`
`
`Connectto
`
`
`Application Server
`
`
`
`Voice Recognition
`
`Process
`
`
`
`
`
`
`
`
`
`Set Keyword
`
`
`
`
`
`
`
`Create Voice
`
`
`Information File
`
`
`
`
`
`
`Set ImageInformation
`
`
`
`
`
`$1109
`
`
`
`81110
`
`
`
`$1111
`
`
`
`§1112
`
`
`
`89113
`
`
`
`S114
`
`
`Page 12 of 26
`
`Netskope Exhibit 1012
`
`Page 12 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`US 2003/0063321 Al
`
`
`Apr. 3, 2003
`
`
`
`
`
`
`
`IMAGE MANAGEMENTDEVICE, IMAGE
`
`
`
`
`MANAGEMENT METHOD, STORAGE AND
`PROGRAM
`
`
`FIELD OF THE INVENTION
`
`
`
`
`
`
`
`
`
`[0001] The present invention relates primarily to a device
`
`
`
`
`
`
`
`and a method for managing image data in photographing
`
`
`
`
`
`
`
`devices and computers, and to an image data management
`
`
`
`
`
`
`technology to manage photographed image data using a
`server on a network.
`
`
`
`
`
`DESCRIPTION OF RELATED ART
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`information processing systems
`[0002] Conventionally,
`
`
`
`
`
`
`
`
`
`that have been knownallow imagedata, which are electronic
`
`
`
`
`
`photographs photographed using image photographing
`
`
`
`
`
`
`
`devices such as digital cameras, to be shared, referred to and
`
`
`
`
`
`
`
`
`edited by a plurality of users by storing the image data in a
`server connected to the Internet.
`
`
`
`
`
`
`
`
`
`
`
`[0003]
`In such information processing systems, a user can
`
`
`
`
`
`
`
`designate on a Web browser the image data that he or she
`
`
`
`
`
`
`
`
`wishes to store, add a title or a message to the image data,
`
`
`and uploadit.
`
`
`
`
`
`
`
`[0004]
`In addition, image photographing devices such as
`
`
`
`
`
`
`
`
`
`digital cameras that allow input of titles and messages for
`
`
`
`
`
`
`
`
`
`
`image data are known;as for uploading imagedata, there are
`
`
`
`
`
`
`
`
`
`terminal devices knownthat allow image data to be sent via
`
`
`
`
`
`
`
`a network to a specific location by connecting an image
`
`
`
`
`
`
`photographing device, such as a digital camera,to a portable
`
`
`
`
`
`communication terminal, such as a cellular telephone or a
`
`
`
`
`
`PHS(personal handy phone system).
`
`
`
`
`
`
`[0005] Furthermore, information processing systems that
`correlate additional information such as voice data with
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`image data and store them together are also known. In such
`
`
`
`
`
`
`information processing systems, the speech vocalized by a
`
`
`
`
`
`
`
`
`user can be recorded and stored as a message with an image
`
`
`
`
`
`
`
`
`data, or the speech vocalized by a user can be recognized
`
`
`
`
`
`
`
`
`with a voice recognition device, and the recognition result
`
`
`
`
`
`
`
`
`
`
`converted into text data, correlated to an image data and
`stored.
`
`
`
`
`
`
`
`[0006] Among voice recognition technologies, a word
`
`
`
`
`
`
`spotting voice recognition technology is known, in which a
`
`
`
`
`
`
`
`sentence a user speaks is recognized using a voice recogni-
`
`
`
`
`
`
`
`tion dictionary and a sentence analysis dictionary, and a
`
`
`
`
`
`
`
`plurality of words included in the sentenceis extracted.
`
`
`
`
`
`
`[0007] However, as image photographing devices such as
`
`
`
`
`
`
`
`
`digital cameras become widely used, the number of image
`
`
`
`
`
`
`data such as electronic photographs is becoming enormous;
`
`
`
`
`
`
`
`
`the user must attach a title, a text message or a voice
`
`
`
`
`
`
`
`message individually to each image data photographed,
`
`
`
`
`
`
`
`
`whichresults in having to invest a huge amount of time and
`
`
`
`
`
`
`effort in organizing and storing imagedata.
`
`
`
`
`
`
`
`
`[0008] When keywords used in searches are set and cor-
`
`
`
`
`
`
`
`
`related with an image data, along with a title or a message
`
`
`
`
`
`
`
`
`
`attached to the image data, the title, the message and the
`
`
`
`
`
`
`search keywords, each consisting of one or more keywords,
`
`
`
`
`
`
`
`
`must be input individually for each image data, even though
`
`
`
`
`
`
`
`
`
`
`in manycases they are very similar to each other;this results
`
`
`
`
`
`
`in a waste in terms of repeated input operations of similar
`words.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SUMMARYOF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`[0009] The present invention was conceivedin view ofthe
`
`
`
`
`problemsentailed in prior art.
`
`
`
`
`
`
`
`
`[0010] The present invention primarily relate to an appa-
`
`
`
`
`
`
`ratus and a methodto efficiently set additional information
`
`
`
`
`
`
`to image data in order to manage images.
`
`
`
`
`
`
`[0011]
`In view of the above, an embodimentof the present
`
`
`
`
`
`
`
`invention pertains to an image management apparatus that
`
`
`
`
`
`
`
`transmits image data to an image processing apparatus, the
`
`
`
`
`
`
`
`image management apparatus comprising: an image input
`
`
`
`
`
`
`
`
`unit that inputs image data to be transmitted; a sound input
`
`
`
`
`
`
`
`
`
`unit that inputs voice information relating to the image data
`
`
`
`
`
`
`
`
`
`input via the image input unit; a translator that voice-
`
`
`
`
`
`
`
`
`
`recognizes the voice information input via the sound input
`
`
`
`
`
`
`
`
`
`unit and converts the voice information into keyword infor-
`
`
`
`
`
`
`
`mation containing at least one keyword; and a transmission
`
`
`
`
`
`
`
`
`
`
`unit that adds the keyword information to the image data and
`
`
`
`
`
`
`
`
`transmits the image data with the keyword information to
`
`
`
`
`the image processing apparatus.
`
`
`
`
`
`
`
`[0012] The present invention also relates to an apparatus
`
`
`
`
`
`
`
`
`
`and a method that are capable of setting additional infor-
`
`
`
`
`
`
`
`mation using more appropriate expression. In this respect,in
`
`
`
`
`
`
`
`
`one aspect of the present invention, the image management
`
`
`
`
`
`
`
`
`apparatus may further include an obtaining unit that obtains
`
`
`
`
`
`
`
`time information correlated to the image data to be trans-
`
`
`
`
`
`
`
`mitted, wherein the translator extracts keywords based on
`the voice information and the time information.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`{0013] Furthermore,
`in another aspect of the present
`
`
`
`
`
`
`invention,
`the image management apparatus may further
`
`
`
`
`
`
`comprises an obtaining unit that obtains geographical posi-
`
`
`
`
`
`
`
`
`information correlated to the imaged data to be
`tional
`
`
`
`
`
`
`transmitted, wherein the translator extracts keywords based
`
`
`
`
`
`
`
`
`on the voice information and the positional information.
`
`
`
`
`
`
`
`[0014] Other purposes and features of the present inven-
`
`
`
`
`
`
`tion shall becomeclear in the description of embodiments
`
`
`
`and drawings below.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`
`
`
`
`
`
`
`[0015] FIG. 1 shows a system configuration diagram
`
`
`
`
`
`indicating the general configuration of an information pro-
`
`
`
`
`
`
`cessing system in accordance with a first embodimentof the
`
`
`present invention.
`
`
`
`
`
`
`[0016] FIG. 2 shows a block diagram indicating the
`
`
`
`electrical configuration of an adaptor.
`
`
`
`
`
`
`[0017] FIG. 3 shows a diagram indicating the configura-
`
`
`
`
`
`
`tion of software installed on the adaptor.
`
`
`
`
`
`[0018] FIG. 4 shows a schematic illustrating information
`
`
`
`
`
`set in a voice information setting file.
`
`
`
`
`
`[0019] FIG. 5 shows a flowchart indicating a processing
`
`
`
`
`unique to the first embodiment.
`
`
`
`
`
`[0020] FIG. 6 shows a configuration diagram indicating
`
`
`
`
`
`the general configuration of an application server according
`
`
`
`
`
`
`
`to the second embodimentof the present invention.
`
`
`
`
`
`
`
`[0021] FIG. 7 shows a schematic indicating the configu-
`
`
`
`
`
`
`ration of software installed on a voice processing section of
`
`
`
`
`the application server in FIG.6.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 13 of 26
`
`Netskope Exhibit 1012
`
`Page 13 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`US 2003/0063321 Al
`
`
`Apr. 3, 2003
`
`
`
`
`
`
`
`
`[0022] FIG. 8 shows a flowchart indicating a processing
`
`
`
`
`unique to the second embodiment.
`
`
`
`
`
`[0023] FIG. 9 shows a flowchart indicating a processing
`
`
`
`
`unique to the third embodiment.
`
`
`
`
`
`
`
`[0024] FIG. 10 shows a block diagram indicating the
`
`
`
`
`
`electrical configuration of an adaptor according to the fourth
`embodiment.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0025] FIG. 11 shows a flowchart indicating a processing
`
`
`
`
`unique to the fourth embodiment.
`DETAILED DESCRIPTION OF THE
`
`
`PREFERRED EMBODIMENT
`
`
`
`
`
`
`
`
`
`[0026] Below, embodiments of the present invention will
`
`
`
`
`
`
`
`be described with reference to the accompanying drawings.
`
`
`
`[0027]
`[First Embodiment]
`
`
`
`
`
`
`[0028] FIG. 1 shows a system configuration diagram
`
`
`
`
`
`
`indicating the general configuration of an information pro-
`
`
`
`
`
`
`
`cessing system in accordance with the first embodiment of
`
`
`
`the present invention.
`
`
`
`
`
`
`[0029] The information processing system includesa ter-
`
`
`
`
`
`
`minal device 101, an external provider 106, an application
`
`
`
`
`
`
`server 108, an information terminal device 109, a commu-
`
`
`
`
`
`
`
`nication network 105 that connects the foregoing compo-
`
`
`
`
`
`
`
`
`
`
`nents so that they can send and receive data, and the Internet
`107.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0030] The terminal device 101 has a digital camera 102,
`
`
`
`
`
`
`
`
`an adaptor 103 and a portable communication terminal 104.
`
`
`
`
`
`
`
`
`
`The digital camera 102 has a display panel to check photo-
`
`
`
`
`
`
`
`
`
`graphed images, and the display panel
`in the present
`
`
`
`
`
`
`
`
`embodimentis used to select image data that are to be sent
`
`
`
`
`
`to the application server 108.
`
`
`
`
`
`
`[0031]
`Images photographedbythe digital camera 102 are
`
`
`
`
`
`
`assigned filenames and stored according to predetermined
`
`
`
`
`
`
`
`
`rules. For example, they are stored according to a DCF
`
`
`
`
`
`
`
`(Design rule for Camera Format). Detailed description of the
`
`
`
`
`
`DCFis omitted, since it is known.
`
`
`
`
`
`
`
`
`
`[0032] The adaptor 103 has a function unique to the
`
`
`
`
`
`
`
`
`present embodiment as described later, in addition to its
`
`
`
`
`
`
`
`
`
`fundamental function of relaying image data that are sent
`
`
`
`
`
`
`
`
`from the digital camera 102 to the portable communication
`
`
`
`
`
`
`
`terminal 104. The portable communication terminal 104 is
`
`
`
`
`
`
`
`provided to send the image data photographed bythe digital
`
`
`
`
`
`
`
`
`camera 102 to the application server 108 and functions as a
`wireless communication terminal. The communication net-
`
`
`
`
`
`
`
`
`
`
`
`
`
`work 105 comprises a public telephone line, ISDN or
`
`
`
`
`
`
`satellite communication network;
`in the present embodi-
`
`
`
`
`
`
`
`ment, however, it is conceived to be a public telephone line
`network that includes wireless network.
`
`
`
`
`
`
`
`
`
`
`
`
`[0033] The external provider 106 intercedes between the
`
`
`
`
`
`
`
`Internet 107 and the communication network 105; it pro-
`
`
`
`
`
`
`vides a dial-up connection service to the information termi-
`
`
`
`
`
`
`
`
`
`nal device 109 and manages and operates user accounts for
`Internet connection.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0034] The application server 108 communicates accord-
`
`
`
`
`
`
`
`ing to a predetermined protocol and has functions to receive,
`
`
`
`
`
`
`
`
`
`
`store, refer to, search and deliver image data and/or voice
`
`
`
`
`
`
`
`data. The information terminal device 109 comprises a
`
`
`
`
`
`personal computer or a portable communication terminal
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and has functions to search, refer to, edit, receive and print
`
`
`
`
`
`
`
`
`via the communication network 105 the image data and/or
`
`
`
`
`
`
`
`the voice data managed by the application server 108.
`
`
`
`
`
`
`
`[0035] Next,
`the adaptor 103, which is unique to the
`
`
`
`
`present embodiment, is described below.
`
`
`
`
`
`
`[0036] FIG. 2 is a block diagram indicating the electrical
`
`
`
`
`configuration of the adaptor 103.
`
`
`
`
`
`
`
`
`[0037] The adaptor 103 according to the present embodi-
`
`
`
`
`
`
`ment is connected to the portable communication terminal
`
`
`
`
`
`
`
`104 via a communication terminal interface 208, which in
`turn is connected to an internal bus 216.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0038] The adaptor 103 is also connected to the digital
`
`
`
`
`
`
`
`
`camera 102 via a camera interface 201, which in turn is
`
`
`
`
`
`
`
`
`connected to the internal bus 216. In the present embodi-
`
`
`
`
`
`
`
`
`
`
`the adaptor 103 and the digital camera 102 are
`ment,
`
`
`
`
`
`
`
`
`
`connected by a USB (universal serial bus), so that
`the
`
`
`
`
`
`
`
`
`
`
`adaptor 103 can obtain, via the USB and the camerainter-
`
`
`
`
`
`
`
`
`face 201, image data photographed by the digital camera
`102.
`
`
`
`
`
`
`
`
`
`[0039]
`To the internal bus 216 are also connected a CPU
`
`
`
`
`
`
`
`
`
`202 that controls the overall operation of the adaptor 103, a
`
`
`
`
`
`
`
`
`ROM 205 that stores an internal operation program and
`
`
`
`
`
`
`settings, a RAM 206 that
`temporarily stores a program
`
`
`
`
`
`
`
`
`
`execution region and data received or to be sent, a user
`
`
`
`
`
`
`
`
`interface (U/1) 209, a voice processing section 204, and a
`
`
`
`
`
`
`
`
`power source 207. The voice processing section 204 is
`
`
`
`
`
`
`configured so that a microphone 203 can be connectedto it.
`
`
`
`
`
`
`
`[0040] A program that controls the present embodimentis
`stored in the ROM 205.
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0041] The U/I 209 has a power source button 210 that
`
`
`
`
`
`
`
`
`turns on and off power supplied by the power source 207, a
`transmission button 201 that instructs the transmission of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`image data, a voice input button 212 that starts voice input
`
`
`
`
`
`
`
`
`processing, and an imageselection button 213 that instructs
`
`
`
`
`
`
`
`
`
`
`to take into the adaptor 103 the image data displayed on the
`
`
`
`
`
`
`
`
`
`display panel of the digital camera 102. In addition, the U/I
`
`
`
`
`
`
`
`
`
`
`
`209 has three-color LEDs 214 and 215 that notify the user
`
`
`
`
`
`
`
`
`of the status of the adaptor 103. The voice processing section
`
`
`
`
`
`
`
`
`
`204 controls the microphone 203 to begin and end taking in
`
`
`
`speech and to record.
`
`
`
`
`
`
`[0042] The ROM 205 comprises a rewritable ROM and
`
`
`
`
`
`
`allows software to be added or changed. In the ROM 205 are
`
`
`
`
`
`
`
`stored software (a control program) shownin FIG.3, as well
`
`
`
`
`
`
`
`
`as various programs, the telephone numberof the portable
`
`
`
`
`
`
`
`
`communication terminal 104 and an adaptor ID. The pro-
`
`
`
`
`
`
`
`
`
`grams stored in the ROM 205 can be rewritten by new
`
`
`
`
`
`
`
`
`
`programsthat are downloaded via the camera interface 201
`
`
`
`
`
`
`
`
`or the communication terminal interface 208. The telephone
`
`
`
`
`
`
`
`number of the portable communication terminal 104 that is
`
`
`
`
`
`
`stored in the ROM 205 can be similarly rewritten.
`
`
`
`
`
`
`
`[0043] The CPU 202 controls the portable communication
`
`
`
`
`
`
`
`terminal 104 in terms of making outgoing calls, receiving
`
`
`
`
`
`
`
`incoming calls and disconnecting based on the programs
`
`
`
`
`
`
`
`stored in the ROM 205. The portable communication ter-
`
`
`
`
`
`
`
`
`
`minal 104 outputs to the adaptor 103 its own telephone
`
`
`
`
`
`
`
`number and information concerning incoming calls (ring
`
`
`
`
`
`
`
`information,
`telephone numbers of incoming calls, and
`
`
`
`
`
`
`
`the portable communication terminal 104).
`status of
`
`
`
`
`
`
`
`
`
`Throughthis, the adaptor 103 can obtain information such as
`
`
`
`
`
`
`
`the telephone numberof the portable communication termi-
`nal 104.
`
`
`
`Page 14 of 26
`
`Netskope Exhibit 1012
`
`Page 14 of 26
`
`Netskope Exhibit 1012
`
`

`

`
`
`US 2003/0063321 Al
`
`
`Apr. 3, 2003
`
`
`
`
`
`
`
`
`
`
`
`[0044] The adaptor 103 has the following function as a
`
`
`
`
`
`
`
`
`function unique to the present embodiment: the adaptor 103
`
`
`
`
`
`
`has a function to voice-recognize a voice message input
`
`
`
`
`
`
`
`
`
`through the microphone 203, extract words from the mes-
`
`
`
`
`
`
`
`
`
`
`
`sage, convert the wordsinto text data, and attach them to the
`
`
`
`
`
`
`
`
`image data as keywords for image searches anda title.
`
`
`
`
`
`
`
`
`[0045] The electrical configuration of the adaptor 103 has
`
`
`
`
`
`
`
`
`been indicated as illustrated in FIG. 2, but different con-
`
`
`
`
`
`
`
`figurations may be used as long as the configuration allows
`
`
`
`
`
`
`
`
`
`the control of the digital camera 102, voice processing, the
`
`
`
`
`
`
`
`
`control of the portable communication terminal 104, and the
`
`
`
`transmission of specific files.
`
`
`
`
`
`
`FIG.3 is a functional block diagram indicating the
`[0046]
`
`
`
`
`
`
`configuration of software that is installed on the adaptor 103
`
`
`
`
`
`
`
`
`and that realizes the function unique to the present embodi-
`ment.
`
`
`
`
`
`
`
`
`[0047] Reference numeral 301 denotes an image informa-
`
`
`
`
`
`
`
`
`
`tion control section that obtains, via the camera interface
`
`
`
`
`
`
`
`
`
`201, list information of image data or specific image data
`
`
`
`
`
`
`
`
`
`
`that are stored in the digital camera 102, and stores them. In
`
`
`
`
`
`
`
`
`other words, when the imageselection button 213is pressed,
`
`
`
`
`
`
`
`
`
`the image information control section 301 obtains and stores
`
`
`
`
`
`
`
`
`the image data displayed on the display panel of the digital
`
`
`
`
`
`
`
`
`
`camera 102. The image information control section 301 also
`
`
`
`
`
`
`
`performs change processing to changethe filename of image
`data obtained.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0048] Reference numeral 302 denotes a voice data
`
`
`
`
`
`
`
`
`
`obtaining section that records voice data taken in via the
`
`
`
`
`
`
`
`
`
`microphone 203 and the voice processing section 204, and
`
`
`
`
`
`
`
`
`
`
`after converting the voice data into digital data that can be
`
`
`
`
`
`
`
`
`processed by the CPU 202, transfers the digital data to a
`
`
`
`
`
`
`voice recognition/keyword extraction section 303, which is
`
`
`
`
`
`
`
`
`described later. The input processing of voice data by the
`
`
`
`
`
`
`
`
`
`voice data obtaining section 302 begins whenthe voice input
`
`
`
`
`
`
`
`
`button 212 is pressed. The recorded voice data is transferred
`
`
`
`
`
`
`
`
`to a transmissionfile storage section 306, which is described
`
`
`
`
`later, as a voicefile.
`
`
`
`
`
`
`
`
`[0049] Reference numeral 303 denotes the voice recogni-
`
`
`
`
`
`
`
`tion/keyword extraction section that uses a voice recognition
`
`
`
`
`
`
`
`
`
`database 304 to analyze the voice data it receives from the
`
`
`
`
`
`
`
`
`voice data obtaining section 302. In the voice recognition
`
`
`
`
`
`
`
`processing, one or more keywords (words) can be extracted
`
`
`
`
`
`
`
`
`
`from the input voice data using a word spotting voice
`
`
`recognition technology.
`
`
`
`
`
`
`
`
`[0050]
`In the voice recognition database 304is registered
`
`
`
`
`
`
`
`information required for the voice recognition processing
`
`
`
`
`
`
`
`and the keyword extraction processing. There may be a
`
`
`
`
`
`
`
`
`
`plurality of the voice recognition databases 304, and they
`
`
`
`
`
`
`
`
`
`may also be downloaded via the camera interface 201 or the
`
`
`
`
`
`
`
`communication terminal interface 208 and registered. The
`
`
`
`
`
`results of analysis by the voice recognition/keyword extrac-
`tion section 303 are transferred to a voice information
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`setting section 305, which is described later.
`
`
`
`
`
`
`[0051] For
`example,
`the voice
`recognition/keyword
`
`
`
`
`
`
`
`
`extraction section 303 analyzes the voice data it receives by
`
`
`
`
`
`
`
`using a phonemic model, a grammaranalysis dictionary and
`
`
`
`
`
`
`
`
`recognition grammarthat are registered in the voice recog-
`nition database 304 and discriminates the voice data into a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`word section and an unnecessary word section. Those parts
`
`
`
`
`
`
`
`
`determined to belong to the word section are converted into
`
`
`
`
`
`
`
`
`
`
`
`character string data, which serve as keywords, and trans-
`
`
`
`
`
`
`
`ferred to the voice information setting section 305.
`
`
`
`
`
`
`
`
`[0052] The voice information setting section 305 corre-
`
`
`
`
`
`
`
`
`
`lates the image data stored in the image information control
`
`
`
`
`
`
`
`
`section 301 with a title and keywords based ontheresults of
`
`
`
`
`
`
`
`analysis (extracted keywords) it receives from the voice
`
`
`
`
`
`
`recognition/keyword extraction section 303. In other words,
`
`
`
`
`
`
`
`
`the voice information setting section 305 correlates one or
`
`
`
`
`
`
`
`
`more extracted keywords (character string data) with the
`
`
`
`
`
`
`
`
`
`
`image data as the image data’s keywords, and sets one of the
`
`
`
`
`
`
`
`
`
`keywords as thetitle (the part preceding the extension (for
`
`
`
`
`
`
`
`
`example, “.jpg”) in filenames) of the image data. The
`
`
`
`
`
`
`
`
`
`
`contents of the title set and the keywords are stored as a
`voice information file. The voice information file will be
`
`
`
`
`
`
`
`
`described later with reference to FIG.4.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0053] Whensetting the title of an image data, a list of
`
`
`
`
`
`
`
`
`
`imagefilenamesin the digital camera 102 and thatis stored
`
`
`
`
`
`
`
`
`
`in the image information control section 301 is referred to,
`
`
`
`
`
`
`
`
`
`
`and the title is set so as not to duplicate any existing image
`
`
`
`
`
`
`
`
`
`filenamesreferred to. Thetitle (character string data) set by
`
`
`
`
`
`
`
`
`the voice information setting section 305 is transferred to the
`
`
`
`
`
`
`
`image information control section 301 and communicated to
`
`
`
`
`
`the corresponding digital camera 102.
`
`
`
`
`
`
`
`
`
`[0054] The filenames of image data within the digital
`
`
`
`
`
`
`
`
`
`camera 102 (i.c., the filenames that were assigned according
`
`
`
`
`
`
`
`
`
`
`to the DCFin the digital camera 102) may be rewritten as the
`
`
`
`
`
`
`
`
`character string data expressed astitles, butit 1s preferable
`
`
`
`
`
`
`
`
`not to change the filenames themselves and instead to store
`
`
`
`
`
`
`
`the filenames as auxiliary information correlated with cor-
`
`
`
`
`
`
`
`
`
`responding image data. The reasonsfor this are to eliminate
`
`
`
`
`
`
`
`the inconvenience of not being able to manage images as a
`
`
`
`
`
`
`
`
`
`result of having filenames in formats other than the DCF, and
`
`
`
`
`
`
`
`
`
`to be able to recognize the image data with new filenames
`
`
`
`
`
`
`
`
`assigned at the destination, which can be doneas long as the
`
`
`
`
`
`filenames are stored as auxiliary information.
`
`
`
`
`
`
`
`[0055] More preferably, new filenames may be stored as
`
`
`
`
`
`
`
`auxiliary information along with information used to rec-
`
`
`
`
`
`
`
`
`ognize the destination. By doing this, even if different
`
`
`
`
`
`
`
`
`filenames are assigned for a single image data by various
`
`
`
`
`
`
`
`
`destinations, the image data with new filenames assigned at
`
`
`
`
`
`various destinations canstill be recognized.
`
`
`
`
`
`
`
`[0056] Reference numeral 306 denotes the transmission
`
`
`
`
`
`
`
`file storage section. When the transmission button 211 is
`
`
`
`
`
`
`
`
`
`pressed, the transmissionfile storage section 306 obtains the
`
`
`
`
`
`
`
`
`
`image data (an image file) from the image information
`
`
`
`
`
`
`
`
`
`
`control section 301,
`the voice file from the voice data
`
`
`
`
`
`
`
`
`
`obtaining section 302, and the voice information file from
`
`
`
`
`
`
`
`
`
`the voice information setting section 305, and stores them as
`
`
`
`
`
`
`
`
`
`a transmission file. Once storing the transmission file is
`
`
`
`
`
`
`
`
`completed, the transmission file storage section 306 sends a
`transmission notice to the communication control section
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`307. However, the file to be sent may only be the imagefile;
`
`
`
`
`
`
`
`
`
`for example, if there is no applicable voice file or voice
`
`
`
`
`
`
`
`
`information file,