(12) United States Patent
`(10) Patent N0.:
`US 6,710,791 B1
`
`Kodama et al.
`(45) Date of Patent:
`*Mar. 23, 2004
`
`USOO6710791B1
`
`(54) PORTABLE INFORMATION GATHERING
`APPARATUS AND INFORMATION
`MANAGEMENT METHOD
`
`(75)
`
`(73)
`
`InVentorS: Yoshiyuki Kodama, NagaIlO-ken (JP);
`Yoshiteru Uchiyama, Nagano-ken (JP)
`.
`.
`,
`ASSIgHCCZ selk" EPSO“ corporatlom TOkYO (JP)
`.
`.
`.
`.
`(*) N01106:
`Thls Patent Issued on a confirmed Nos
`ecution application filed under 37 CFR
`1.53(d), and is subject to the twenty year
`patent
`term provisions of 35 U.S.C.
`154(a)(2).
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U-S-C- 154(b) by 0 days.
`
`(21) Appl. No.. 08/792,026
`(22)
`Filed:
`Jan. 31, 1997
`
`Foreign Application Priority Data
`(30)
`Feb. 1 1996
`(JP)
`............................................. 8—016903
`’
`Int. Cl.7 .................................................. G09G 5/00
`(51)
`(52) US. Cl.
`........................... 345/835; 345/839; 707/7;
`715/537
`(58) Field of Search ................................. 345/173, 169,
`345/350, 340, 156, 179, 1, 901, 903, 77,
`357, 104, 439, 835, 838, 839; 379/9323,
`9317; 707/501, 7, 100, 101, 102; 178/1803,
`18.04, 1901; 715/515, 522, 525, 526, 530,
`537
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`......... 361/679
`5,268,817 A * 12/1993 Miyagawa et al.
`5,401,917 A *
`3/1995 Yoshida et al.
`............. 345/179
`5,406,307 A *
`4/1995 Hirayama et al.
`.......... 345/179
`5,453,761 A *
`9/1995 Tanaka ....................... 345/173
`
`5,467,102 A : 11/1995 Kunoet 8.1. .............. 345/1
`
`5,634,080 A *
`5/1997 Klklnls et al.
`........... 345/156
`5,663,748 A
`9/1997 Huffman et al.
`..... 345/173
`
`5,673,406 A *
`9 1997 Hira ama ............ 345 350
`5,708,825 A *
`1/1998 Sotoilnayor
`..... 707/501
`
`5,737,599 A *
`4/1998 Rowe et al.
`............. 395/615
`
`5,745,109 A *
`4/1998 Nakano et al.
`.......... 345/340
`5,790,122 A *
`8/1998 Cecchini et al.
`............ 345/357
`
`OTHER PUBLICATIONS
`“Mastering WordPerfect 5.1 & 5.2 for Windows” Alan
`Simpson, 1993, pp. 74—76 and 250256.*
`*
`.
`.
`Cited by examiner
`Primary Examiner—Dennis-Doon Chow
`(74) Attorney, Agent, or Firm—Hogan & Hartson LLP
`(57)
`ABSTRACT
`.
`.
`.
`.
`.
`Aportable information gathering apparatus and information
`managing method in the same make it possible to capture
`information entered through an input/dismay tablet’ image
`information entered through an image input device and
`voice information entered through a voice input device, and
`other type of information for individual pageS, each corre-
`sponding to all information displayed at one time on a
`screen In response to an instruction given by a user of the
`portable information gathering apparatus, an attach-to-page
`icon setting unit attaches icons to desired pages to manage
`the page information in units of pages. Several of these icons
`are: an icon for managing the information captured in
`respective pages by time-related data such as date and time;
`an icon for specifying a particular page; an icon serving as
`a memo guide; and an icon for managing the information
`captured in the respective pages by the type of information.
`
`5,049,862 A *
`
`9/1991 Dao et al. ................... 345/179
`
`18 Claims, 17 Drawing Sheets
`
`28
`
`70
`
`NEW YEAR'S PARTY
`
`AT XXX's HOME
`
`7:00 PM
`
`
`
`101
`
`11Gb
`
`1
`
`APPLE 1016
`
`APPLE 1016
`
`1
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 1 0f 17
`
`US 6,710,791 B1
`
`ATTACH—TO-PAGE
`ICON SETTING
`PROCESSOR
`
`NEXT
`PAGE
`BUTTON
`
`PREVIOUS
`PAGE
`BUTTON
`
`CAMERA
`BUTTON
`
`VOICE
`INPUT
`BUTTON
`
`41
`
`BUTTON
`
`SIGNAL
`PROCESSOR
`
`21
`
`22
`
`34
`
`33
`
`35
`
`36
`
`cpu
`
`FLASH
`MEMORY
`
`REALTIME
`CLOCK
`
`POWER
`SWITCH
`
`30 r """"""""2,5} """"‘
`
`SERIAL
`CONTROLLER
`
`CCD
`CONTROLLER
`
`LCD
`CONTROLLER
`
`CCD
`
`CAMERA
`
`AMPLIFIER
`MICROPHONE
`
`'_______..______________________________._
`
`31
`
`3
`
`422
`
`TRANSMITTER
`
`2320
`
`425
`
`RS
`ZREEENRETNM’
`AMPLIFIER
`SPEAKER 2- PHOTODIODE
`
`28\
`FTABLEI""""""""‘
`
`_____________________1
`42
`
`2
`
`

`

`U.S. Patent
`US. Patent
`
`Mar. 23, 2004
`Mar. 23, 2004
`
`Sheet 2 of 17
`Sheet 2 0f 17
`
`US 6,710,791 B1
`
`US 6,710,791 B1
`
`
`FIG. 3
`
`3
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 3 0f 17
`
`US 6,710,791 B1
`
`101
`
`1050 102 TIMETABLE
`
`103
`
`104
`
`‘05
`
`106
`
`4
`’1
`x
`1
`’ V I“ 1-,
`7, , I"
`
`v
`
`c
`
`I,
`I/
`
` DEPARTURE
`
`xx HOUR
`ARRIVAL
`
`,
`1A
`
`,
`1w-
`I
`2’
`...4
`”g i
`
`1 05d
`
`4
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 4 0f 17
`
`US 6,710,791 B1
`
`HARDWARE
`
`DEVELOPMENT
`
`MR. TARO SUWA
`
`XX ENGINEERING SECTION
`
`1996. 01.01
`
`14:00
`
`14:30
`
`FULL—COLOR PRINTER
`
`¥
`
`99.800
`
`EXPECTED TO BE
`
`RELEASED ON
`
`FEB. 10
`
`1996. 01.01
`
`5
`
`

`

`U.S. Patent
`US. Patent
`
`Mar. 23,2004
`Mar. 23, 2004
`
`Sheet 5 of 17
`Sheet 5 0f 17
`
`US 6,710,791 B1
`US 6,710,791 B1
`
`FIG. 6A
`
`
`
`Slel7[slo]
`
`[-[_]montH (_[_Joay
`[-T Jxour []IMIN.
`
`oy[2]5]41
`
`=]
`
`6
`
`

`

`U.S. Patent
`US. Patent
`
`Mar. 23, 2004
`Mar. 23, 2004
`
`Sheet 6 of 17
`Sheet 6 0f 17
`
`US 6,710,791 B1
`US 6,710,791 B1
`
`28
`
`70
`
`[| ]montH ("|Joay
`[-TJxour (_[JMIN.
`
`
`
`103
`
`FIG. 8
`
`ot25/4)
`
`FIG. 7
`
`BGuoo
`
`
`7
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 7 0f 17
`
`US 6,710,791 B1
`
`1 1 5
`
`AUTO-ERASE SH
`
`28
`
`70
`
` EDMONTH |:|:|DAY
`
`EDHOUR C]:|MIN.
`
`73
`
`mnIII
`
`a“. 75
`
`'
`
`CONFIRMATION-
`REQUIRED
`
`‘
`
`'
`
`76
`
`117
`
`116
`
`FIG. 9
`
`TEL
`
`TEL MEMO
`
`8
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 8 0f 17
`
`US 6,710,791 B1
`
`CREATE STAMP
`
`FIG. 11B
`
`"TAKE PICTURE”
`
`FIG. 11C
`
`123
`
`"TOUCH REGISTERING LOCATION"
`
`9
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 9 0f 17
`
`US 6,710,791 B1
`
`dc
`
`az>>
`
`UN—*
`
`Address indicating lst page
`Address indicating 2nd page
`Address indicating 3rd page
`Address indicating 4th page
`
`
`
`
`
`
`
`
`
`
`lmage data
`on lst page
`
`Voice data
`
`on lst page
`
`Ink data
`
`on lst page
`Additional
`information
`on lst page
`
`lmage data
`on 2nd page
`
`Voice data
`
`on 2nd page
`
`lnk data
`
`on 2nd page
`
`Additional
`information -
`on 2nd page
`
`Image data
`on 3rd page
`
`Voice data
`
`on 3rd page
`
`Ink data
`
`on 3rd page
`
`Additional
`information
`on 3rd page
`
`
`
`
`
`
`
`
`
`
`
`
`BO- Address indicating next page
`NULL
`Address indicating previous page
`
`lst
`page
`
`.
`.
`.
`.
`Addresses Indicating input data
`
`Address indicating additional
`
`information
`
`
`
`
`
`Bi I .
`
`.
`
`.
`
`Address Indicating next page
`.3! Address indicating previous page
`2nd m _
`_
`.
`.
`page
`.3]- Addresses Indicating Input data
`
`Address indicating next page
`NULL
`IE]- Address indicating previous page
`
`
`
`
`3rd
`page
`
`ml'"l
`
`82 - .
`
`
`
`
`
`Address indicating additional
`
`information
`
`.
`
`.
`
`_
`'
`.
`_
`Addresses indicating Input data
`
`Address indicating additional
`
`information
`
`FIG. 12
`
`10
`
`10
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 10 0f 17
`
`US 6,710,791 B1
`
`NEW YEAR'S PARTY
`
`AT XXX'S HOME
`
`
`F3
`
`
`Code of alarm stamp
`Stamp position: x—coordinate
`Stamp position: y—coordinate
`Alarm
`set year
`set: month
`
`Alarm
`
`Alarm
`
`Alarm
`
`set: day
`set: hour
`
`Alarm
`
`set: minute
`
`Code of party stamp
`Stamp position: x—coordinate
`Stamp position: y—coordinate
`END
`
`F2
`
`Ink information
`
`on 4th page
`
`Additional
`
`information
`
`on 4th page
`
`
`
`11
`
`11
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 11 0f 17
`
`US 6,710,791 B1
`
`Ps2 (so, 220)
`
`(30. 220)
`
`PS1
`
`Py (o, 239)
`
`ny (319, 239)
`
`12
`
`12
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 12 0f 17
`
`US 6,710,791 B1
`
`FROM
`NORMAL MODE
`/SI
`
`TABLET
`TOUCH
`
`3211
`
`DISPLAY
`LIST OF STAMPS
`
` IN LIST
`OF SJAMP
`
`Y
`
`10'?)
`FIG. 15
`
`T9N©
`FIG. 19
`
`823
`
`52"
`
`N
`
`N
`
`OTHER
`PROCESSING
`
`END
`
`ERASE
`LIST OF STAMP
`
`DISPLAY
`SELECTED STAMP
`
`5215
`
`RECORD INFORMATION
`ON SELECTED STAMP
`
`$2
`
`N
`
`33
`
`
`¢Y 322
`
`S4
`
`N
`OTHER ‘
`PROCESSING
`
`RING)
`FIG. 17
`
`
`
`8218
`
`CHANGE PAGE NO.
`OF QUICK STAMP
`
`
`
`
`DISPLAY
`BACKGROUND
`
`
`
`
`
`
`
`8217
`
`8219
`
`BACKGROUND
`STAMP
`'
`
`S44
`
`SET
`NORMAL MODE
`
`END
`
`FIG. 16
`
`13
`
`13
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 13 0f 17
`
`US 6,710,791 B1
`
`a 831
`
`MOVE STAMP
`OR
`
`533
`
`SET REGULAR
`MODE
`
`537
`
`
`
`TOUCHED
`QUICK
`
`
`
`535
`
`
`EIEIG’BIEERD
`STAMP
`
`?
`
`
`TOUCHED
`
`USER-DEFINED
`
` Y
`STAMP
`
`
`TOUCHED
`ALARM
`
`
`
`
`TOUCHED
`BACKGROUND
`
` IS
`
`
`ERASE BUTTON
`?
`
`0?N
`' Y
`ERASE
`
`SPECIFIED ICON
`335
`
`ERASE SPECIFIED
`ICON INFORMATION
`
`STAMP
`
`
`TOUCHED
`
`ToDo
`
`
`STAMP
`
`
`'
`345
`Y
`
`DISPLAY ToDo STAMP
`
`SETTING SCREEN
`
`
`
`
`
`Y
`
`S42
`DISPLAY ALARM
`SErIING SCREEN
`343
`SET ALARM
`
`SET ToDo
`DETAILS
`
`
`
`TOUCHED
`AUTO—ERASE
`STAMP
`'
`
`349
`DISPLAY AUTO—ERASE
`SETTING SCREEN
`
`SET AUTO—ERASE
`DETAILS
`
`
`
`
`
`
`
`
`
` S44
`
`SET NORMAL
`MODE
`
`FIG. 17
`
`END
`
`14
`
`14
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 14 0f 17
`
`US 6,710,791 B1
`
`5221
`
`DISPLAY LIST OF
`RETRIEVAL MODES
`
`$222
`
`TABLET
`TOUCH
`
`Y 3223
`
`
`
`STAMP
`RETRIEVAL
`
`
`? 5224
`
`
`
`
`DISPLAY LIST
`OF STAMPS
`
`$226
`
`TOUCH
`'N LIST OF
`
`STAMPS
`9
`
`5227
`
`5223
`
`N
`
`PROCESSING
`
`END
`
`PROVIDE 9-SEGMENT
`DISPLAY SHOWING
`9 PAGES WITH
`
`SPECIFIED STAMP
`
`-
`
`
`NEXT PAGE
`BUTTON ON
`
`
`
`NEXT
`
`
`9—SEGMENT
`
`DISPLAY
`
`
`PREVIOUS
`PAGE ON
`
`
` PREVIOUS
`
`9-SEGMENT
`
`DISPLAY
`
`
`
`TABLET
`TOUCH
`
`Y
`
`3234
`
`DISPIAY TOUCHED PAGE
`FULLY 0N SCREEN
`
`END
`
`FIG. 18
`
`15
`
`15
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 15 0f 17
`
`US 6,710,791 B1
`
`$241
`
`DISPLAY ect.
`MENU
`
`FIG. 19
`
`
`SELECTED
`
`"CREATE STAMP"
`
`
`
`
`
`
`
`
`
`
`
`
`
`Y
`
`DISPLAY ”CREATE STAMP"
`SCREEN
`
`3245
`
`OTHER PROCESSING
`
`$248
`
`TAKE PICTURE
`
`REGISTER CREATED
`STAMP
`
`544
`
`SET NORMAL MODE
`
`16
`
`16
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 16 0f 17
`
`US 6,710,791 B1
`
` FIG. 20A
`
`FIG. 20B
`
`17
`
`17
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 17 0f 17
`
`US 6,710,791 B1
`
`
`
`FIG. 21
`
`PRIOR ART
`
`18
`
`18
`
`

`

`US 6,710,791 B1
`
`1
`PORTABLE INFORMATION GATHERING
`APPARATUS AND INFORMATION
`MANAGEMENT METHOD
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`invention relates to portable information
`The present
`gathering equipment such as a pen computer and, more
`particularly,
`to a portable information gather apparatus
`which can function to manage captured information on a
`page basis, and an information management method by the
`apparatus.
`2. Description of the Related Art
`In recent years, pen computers and other similar devices
`have been coming into widespread use as portable informa-
`tion gathering equipment. There are being developed pen
`computers which can not only record information written on
`a tablet-screen with an input pen, but can also capture
`images and spoken statements. There are increasing tenden-
`cies to enhance the multifunctional features of such pen
`computers for greater convenience.
`FIG. 21 is a schematic block-diagram showing conven-
`tional portable information gathering equipment constructed
`to perform the function of recording information written on
`a tablet screen with an input pen and also the function of
`capturing images and voice inputs. This portable informa-
`tion gathering equipment is constituted primarily by a CPU
`1, a ROM 2 in which processing programs and the like are
`stored, a data memory 3, a tablet 6 which combines a
`digitizer 4 and an LCD 5 into one piece, an image input
`device 7, a voice input device 8, a voice output device 9, an
`information gathering menu selector 10, and a system bus 11
`for connecting these components to the CPU 1.
`Judging from the illustrated configuration, this type of
`conventional portable information gathering equipment is
`designed to capture image and voice information in addition
`to the pen input information which is known as “ink infor-
`mation” captured through the tablet 6.
`In the conventional portable information gathering
`equipment, however, whenever entering information, a
`menu selection has to be performed through the information
`gathering menu selector 11 to properly set the mode in
`accordance with the type of information to be captured
`before entering the information. For instance, to enter an
`image and then characters, the mode must be set first for
`image input
`to capture an image,
`then the mode must
`changed to character input to take in characters.
`For the portable information gathering equipment to be
`user-friendly, all the different types of captured information,
`namely, the image information, the voice information, and
`the ink information, should be handled as a collection on a
`page basis (hereinafter referred to as “page data”). For
`example, to record the information about a certain product
`in terms of one-page information, it would be convenient to
`enter the image of the product and add the description of the
`image in terms of voice information or ink information
`entered using a pen, thus handling all the image, voice, and
`ink information as a piece of page data, that is, handling
`them in units of pages.
`is desirable that multimedia
`it
`More specifically,
`information,
`including image information, voice
`information, and ink information, captured by a user arbi-
`trarily be stored automatically in a storing means as page
`data, i.e. a single piece of collective data on a page basis
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`without the need for operation by the user so that the user is
`not even aware of the processing.
`This type of conventional portable information gathering
`equipment, however, has not been designed to automatically
`handle all of the image information, voice information, and
`ink information as page data for each page without the need
`for the performance of operations by the user of the equip-
`ment.
`
`Furthermore, this type of information gathering equip-
`ment handles a wide variety of information and the volume
`may be as large as over 100 pages. Therefore, the user is
`required to manage entered information by alarm setting or
`some other method according to the type of information
`when reading a particular piece of information from among
`the information which has been stored, or reading all related
`information, or if entered information has any time limit.
`In this type of conventional portable information gather-
`ing equipment such as a pen computer,
`it is possible to
`manage the captured information to a certain extent by
`setting an alarm or the like by writing date, time, contents of
`information, etc. according to a predetermined format.
`However, if such equipment allows information including
`pen input, image, and voice information to be entered as
`desired while it does not have any specified format, then it
`would have to have a function for managing such informa-
`tion whenever it is entered.
`
`SUMMARY OF THE INVENTION
`
`Accordingly, it is an object of the present invention to
`provide a portable information gathering apparatus and an
`information management method in the same which obviate
`the need of setting mode according to information to be
`captured whether it is image information, voice information,
`or ink information, which enable a user thereof to enter any
`type of information as he or she desires, and which permit
`efficient management of information by efficiently retrieving
`desired information among entered information, by associ-
`ating the information on different pages, or by providing
`information with alarms.
`
`To this end, according to one aspect of the present
`invention, there is provided a portable multimedia informa-
`tion gathering apparatus for gathering and displaying
`entered information and for also storing the gathered
`information, which is equipped at least with: information
`input means for entering information; page adding icon
`setting means for pasting an icon corresponding to the
`information, which has been entered for each page equiva-
`lent to one screen by the information input means,
`to a
`specified page; and data storing means wherein the infor-
`mation entered through the information input means and the
`information regarding the icon pasted to the specified page
`are written at a specified address.
`The information to be entered is at least one of: informa-
`
`tion entered through the input/display tablet for capturing
`information when the screen thereof is touched and also for
`
`image information
`displaying the captured information;
`entered through image input means, and voice information,
`or spoken statements, entered through voice input means.
`The icon pasted to the specified page is at least one of: an
`icon for managing the information captured in respective
`pages by time-related data such as date and time; an icon for
`specifying a particular page; an icon serving as a memoran-
`dum guide; and an icon for managing the information
`captured in the respective pages by the type of information.
`The information regarding the icon pasted on the specified
`page written to the data storing means is the information
`19
`
`19
`
`

`

`US 6,710,791 B1
`
`3
`indicating the type of icon and positional information indi-
`cating the position where the icon is pasted. The icon
`information for the icon for managing information captured
`in the respective pages by the time-related data such as date
`and time is the time-related information including date and
`time as well as the information indicating the type of icon
`and the positional information.
`With this arrangement, an icon corresponding to infor-
`mation captured on each page can be pasted to captured
`information, and these different types of icons are pasted on
`a page basis as desired,
`thus enabling the pages to be
`managed by the types of icons pasted on the pages. This
`makes it possible to set such functions as alarm, To Do,
`quick call, and automatic erase in units of pages, to permit
`a background stamp such as a telephone memorandum to be
`pasted on each page, and to classify page information by
`type by using the icons. These features are extremely
`convenient for managing and retrieving data which has been
`entered.
`
`invention,
`According to another aspect of the present
`there is provided an information managing method in a
`portable multimedia information gathering apparatus which
`gathers and displays entered information and also stores the
`gathered information, whereby an icon, which corresponds
`to the information captured for each page equivalent to one
`screen by information input means, can be pasted to a
`specified page, and when the icon has been pasted in a
`predetermined position on the page, the information entered
`through the information input means together with the
`information regarding the pasted icon are written at a
`specified address in data storing means, thus implementing
`information management based on the written information
`regarding the icon.
`The information to be entered is at
`
`least one of the
`
`information captured through the input/display tablet which
`captures information when the screen thereof is touched, the
`image information captured through the image input means,
`and the voice information captured through the voice input
`means.
`
`The icon pasted on a specified page is at least one of the
`following icons: an icon for managing information captured
`into the pages by time-related data such as date and time; an
`icon for specifying a particular page; an icon serving as a
`memorandum guide; and an icon for managing information
`captured into the pages by the type of information.
`The information regarding the icon pasted on the specified
`page which is written to the data storing means is the
`information indicating the type of icon and the positional
`information indicating the position where the icon is pasted.
`The information regarding the icon, if the icon is the one for
`managing the information captured into the pages by time-
`related data such as date and time, is the information related
`to time including date and time.
`Use of the method described above allows icons to be
`
`pasted according to the information captured into the pages
`and also the pages to be managed by the type of icon by
`pasting different types of icons as desired in units of pages.
`This makes it possible to set such functions as alarm, To Do,
`quick call, and automatic erase in units of pages, to permit
`a background stamp such as a telephone memorandum to be
`pasted on each page, and to classify page information by
`type by using the icons. These features are extremely
`convenient for managing and retrieving data which has been
`entered. Applying the method in accordance with the present
`invention to a pen computer or the like turns it into an
`extremely user-friendly pen computer.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a block diagram illustrative of the configuration
`of an embodiment of apparatus according to the present
`invention;
`FIG. 2 is an external view of a pen computer to which the
`present invention is applied;
`FIG. 3 is a diagram showing an example of a function
`menu selector which has been set beforehand;
`FIG. 4 is a diagram showing an example of a list of stamps
`which is displayed when a stamp is selected through the
`function menu selector;
`FIGS. 5A and 5B are diagrams showing an example of
`displayed information which has been captured into the pen
`computer;
`FIGS. 6A and 6B are diagrams illustrating the display
`operation when an alarm stamp has been selected;
`FIG. 7 is a diagram illustrating the display operation when
`a ToDo stamp has been selected;
`FIG. 8 is a diagram illustrating the display operation when
`a quick stamp has been selected;
`FIG. 9 is a diagram illustrating the display operation when
`an erase stamp has been selected;
`FIG. 10 is a diagram illustrating the display operation
`when a background stamp has been selected;
`FIGS. 11A, 11B and 11C are diagrams illustrating the
`display operation when a user-defined stamp has been
`selected;
`FIG. 12 is a diagram illustrating the contents of a flash
`memory;
`
`FIG. 13 is a diagram showing an example of a display
`screen wherein a stamp has been pasted;
`FIG. 14 is a diagram illustrating the contents of the
`memory of additional information for the page information
`to which a certain stamp has been attached;
`FIG. 15 is a diagram illustrating the positional informa-
`tion of the attached stamp;
`FIG. 16 is a flowchart illustrating a part of a processing
`procedure in the embodiment in accordance with the present
`invention;
`FIG. 17 is a flowchart illustrating a part of a processing
`procedure in the embodiment in accordance with the present
`invention;
`FIG. 18 is a flowchart illustrating a part of a processing
`procedure in the embodiment in accordance with the present
`invention;
`FIG. 19 is a flowchart illustrating a part of a processing
`procedure in the embodiment in accordance with the present
`invention;
`FIGS. 20A, 20B and 20C are diagrams illustrating a
`displaying operation for retrieving in the embodiment; and
`FIG. 21 is a block diagram showing a schematic configu-
`ration of a conventional multimedia information gathering
`apparatus.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`
`An embodiment of the present invention will now be
`described. This embodiment will use a pen computer as the
`portable information gathering equipment.
`FIG. 1 is a block diagram illustrative of the configuration
`of the entire embodiment of a pen computer to which the
`present invention has been applied. The pen computer is
`20
`
`20
`
`

`

`US 6,710,791 B1
`
`5
`constituted primarily by: a CPU 21; a ROM 22 for storing
`basic processing programs for the pen computer and a
`program and data necessary for gathering information
`according to the present invention (the processing procedure
`based on the program will be described in detail later); a
`CCD camera 23 for capturing image information; a CCD
`controller 24 for driving the CCD camera 23 and for
`converting the signals received from the CCD camera to
`digital signals; a microphone 25 for capturing voice infor-
`mation; an amplifier 26 for amplifying voice signals
`received from the microphone 25; an analog-to-digital con-
`verter 27 for converting the voice signals to digital signals;
`an input/display tablet 28 which combines a digitizer 281
`and a liquid crystal display (LCD) 282 into one unit and
`which captures information including characters and line
`drawings and which also displays the captured information;
`an LCD circuit controller 29; a digital-to-analog converter
`30 for converting the voice signals to analog signals when
`outputting voice signals; an amplifier 31; a speaker 32; a
`flash memory 33 serving as a data storing means for storing
`data captured by a user (the data to be stored in the flash
`memory and the storing areas thereof will be described in
`detail later); a RAM 34 used as a work memory when
`implementing various types of processing in the pen com-
`puter; a real time clock 35 which indicates the current time
`and provides time information and date information; a
`power switch 36; page feed buttons which are comprised of
`a next page button 37 and a previous page button 38; a
`camera button 39; a button signal processor 41 which detects
`the ON/OFF state of a button such as a voice input button 40
`and issues signals corresponding to the ON/OFF state; an
`external equipment connecting device 42 for the transfer of
`information between itself and external information process-
`ing equipment such as a personal computer; a system bus 43
`for interconnecting the individual sections and component
`units; and an attach-to-page icon setting processor 44 which
`carries out various types of processing on attach-to-page
`icons which are pasted to respective pages so as to manage
`information captured into the pages by time-related data
`such as date and time or classify the information by type or
`to indicate a background of a memorandum, etc. The attach-
`to-page icons, which are to be attached to pages, are put on
`respective pages as if they were stamped thereon; therefore,
`the icons will be hereinafter referred to as “stamps” which
`will be described in detail later.
`
`The external equipment connecting device 42 is con-
`structed primarily by: a serial controller 421; an RSZ32C
`port 422 used as an interface for connection to external
`information processing equipment such as a personal com-
`puter; an infrared receiver/transmitter 423 for performing
`optical communication with information processing equip-
`ment such as a personal computer; a light emitting diode
`(LED) 424; and a photo diode 425.
`FIG. 2 shows an external view of the pen computer in
`accordance with the present invention; components identical
`to those shown in FIG. 1 are assigned the same reference
`numerals as in FIG. 1. Among the constituent elements
`shown in FIG. 1, those that appear in the external view of
`FIG. 2 include the CCD camera 23 (more precisely a lens
`23a of the CCD camera 23), the camera button 39, the tablet
`28,
`the microphone 25,
`the speaker 32,
`the voice input
`button 40, the next page button 37, the previous page button
`38, and the power switch 36. Provided on the screen of the
`tablet 28 is a function menu selector 70. An input pen 51 is
`used to draw characters or lines on the screen of the tablet
`
`28 and to point at icons.
`As illustrated in FIG. 3, the function menu selector 70
`pre-displays icons indicating menus, including a “find” icon
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`71, an icon 72 for calling up the list of stamps, a quick call
`icon 73 for immediately calling a particular page, a key
`word-setting icon 74 for finding information, a 9-page
`display icon 75 for displaying the information of nine pages
`at the same time on the tablet screen, and an icon 76 for
`specifying other menus. These icons are shown in the form
`of patterns or the like which visually show the functions of
`the icons. When one of the icons is pointed at, the contents
`of, or information identifying, the icon are displayed on the
`screen.
`
`For instance, among the icons mentioned above, if the
`icon 72 for calling up the list of stamps is pointed at, then
`the list of stamps as shown in FIG. 4 will appear on the
`screen of the tablet 28. These stamps are used to manage
`information by each page or to provide background pictures.
`The meanings and functions of the individual, stamps will be
`described later.
`
`When the 9-page display icon 75 is pointed at after the
`power switch is turned ON, a total of nine pages of infor-
`mation are displayed at the same time in reduced sizes,
`including four pages each preceding and following a par-
`ticular page which is the page displayed when the power
`switch is turned ON and which is normally the page previ-
`ously processed. Therefore, this icon serves also as an index;
`however, no further description will be given about this icon
`because it is not directly connected to the present invention.
`In such a pen computer when characters or line pictures
`are written on the screen of the tablet 28 with the input pen
`51, the CPU 21 inside reads coordinates from the digitizer
`281 and displays the writing trace in the corresponding
`position of the LCD 282 and stores the coordinate data at a
`predetermined address of the flash memory 33.
`To input an image through the CCD camera 23, the lens
`23a of the CCD camera 23 is pointed at the image to be
`captured and the camera button 39 is first depressed halfway.
`This causes the image entered through the lens 23a to be
`projected as an animation on the screen of the tablet 28; and
`when the camera button 39 is further depressed, the image
`is locked and the locked image is captured. The image signal
`is converted to a digital signal by the CCD camera controller
`24 before it is stored at a predetermined address in the flash
`memory 33.
`To input spoken information, the voice input button 40 is
`depressed to enable audible speech to be captured through
`the microphone 25; the spoken sounds are then amplified
`through the amplifier 26 and converted to a digital signal by
`the analog-to-digital converter 27 before it is stored at a
`predetermined address in the flash memory 33.
`The operation for capturing information is performed in
`units of screens, i.e. on a page basis. The following will
`describe the processing procedure.
`If the pen computer has never been used, when the power
`switch 36 is turned ON, nothing will be displayed on the
`screen of the tablet 28; this screen will be regarded as the
`first page. However, the following will describe an example
`wherein some information has already been captured into
`the first page and additional information will be captured on
`the second and subsequent pages.
`In this example, the user of the pen computer will meet a
`person, whom he has never met before, to have a business
`discussion during which he will capture information regard-
`ing the person and the details of the discussion and also
`information about a product, which is the topic of the
`business discussion.
`
`When the user first turns ON the power switch 36, the first
`page into which information has already been captured will
`21
`
`21
`
`

`

`US 6,710,791 B1
`
`7
`be displayed. Pressing the next page button 37 causes the
`screen of the second page, into which no information has
`been captured yet, to be displayed. With the blank second
`page on the screen, in order to capture the image of the face
`of the person with whom the user is meeting, the user points
`the lens 23a of the CCD camera 23 at the person and
`depresses the camera button 39 halfway as previously
`described,
`then further, e.g. fully, depresses the camera
`button 39. This causes the then-existing face image, which
`is a stationary image, to be captured and the face image to
`be displayed on the screen of the tablet 28 at the same time,
`as shown in FIG. 5A; the image data is stored at a prede-
`termined address of the flash memory 33. The position and
`size of the area where the image is displayed may be preset
`in the display area on the screen of the tablet 28 or they may
`be set according to the situations of each case.
`FIGS. 5A and 5B show examples where an area of about
`one-quarter of the display screen at top left thereof is set as
`the image area.
`The user of the pen computer then writes with the input
`pen 51 memo-like information including the name of the
`person, the company name and the department or section to
`which he belongs as the information regarding the person.
`The ink information, such as characters or line drawings,
`entered with the input pen 51 is displayed on the screen of
`the tablet 28 as the user draws them and it is also stored at
`
`a predetermined address in the flash memory 33.
`Furthermore, to record an important part of the discussion
`in terms of spoken statements, the voice input button 40 is
`depressed to capture the such statements. The captured voice
`is subjected to digital conversion and stored as a voice signal
`at a predetermined address in the flash memory 33. Upon
`completion of capturing the voice, an icon 52 indicating the
`completion of recording appears at a predetermined point on
`the screen of the tablet 28. This icon 52 is used also for
`
`function reading. To be more specific, when the data on the
`page is read later, the voice information is read and repro-
`duced from the predetermined address in the flash memory
`33 by pointing at the icon 52 with the input pen 51.
`FIG. 5A shows an example where the information which
`has been captured according to the procedure described
`above is displayed on the screen of the tablet 28, that is, the
`information on the second page. As it is seen from. FIG. 5A,
`the visual information of the page information of one page
`unit is given by face image information 61, ink information
`62 such as characters and line drawings, and the icon 52
`whi