United States Patent
`5,615,384
`[45] Date of Patent: Mar. 25, 1997
`Allard et al.
`
`
`
`[11] Patent Number:
`
`ACAAA
`
`US005615384A
`
`[54] PERSONAL COMMUNICATOR HAVING
`IMPROVED ZOOM AND PAN FUNCTIONS
`FOR EDITING INFORMATION ON TOUCH
`SENSITIVE DISPLAY
`
`[75]
`
`Inventors: David J. Allard; Francis J. Canova,
`Jr., both of Boynton Beach; Debra A.
`G. Johnson, Fort Lauderdale; Charles
`S. Lanier, Delray Beach; James R.
`Lewis, Coconut Creek; Byron K.
`Tiller, Boca Raton; William Villafana,
`Davie; Raymond L. Yee, Coral Springs,
`all of Fla.
`
`{73] Assignee:
`
`International Business Machines
`Corporation, Armonk, N.Y.
`
`(21] Appl. No.: 521,333
`
`[22]
`
`Filed:
`
`Aug. 29, 1995
`
`Related U.S. Application Data
`
`[63] Continuation of Ser. No. 146,341, Nov. 1, 1993, abandoned.
`[SL]
`Tint. Cho inccceeecscsssscessssetsssssseaeese G06F 15/66
`[52] U.S. Ch. veces 395/800; 395/139; 395/326;
`382/298; 345/121; 345/130; 345/173
`(58] Field of Search o0......cccceccsssessssesee 395/800, 139,
`395/157, 345/156, 173; 364/DIG. 1, 521,
`518; 382/47, 8; 340/700, 731, 712
`
`{56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`12/1988 Ramage......... 4,833,625
`
`5/1986 Straton et al. oe 364/DIG.2
`7/1988 Slavinet al. ...
`
`w
`5/1989 Fisher etal. ...
`2/1990 Montagnaet al. oe 364/521
`9/1991 Tada et al. oo... essssssssetcesenee 364/521
`
`4,587,630
`4,755,811
`4,790,028
`
`4,899,292
`5,051,927
`
`.. 340/712
`6/1992 Hubeet al.
`5,119,079
`
`w 355/218
`6/1992 Baldwin........
`5,119,135
`.. 340/706
`..
`5,157,384 10/1992 Greanias et al.
`5,166,666
`11/1992 Tanaka... ssccssereerescserenseres 340/706
`5,184,314
`2/1993 Kelly et al.
`. 364/709.01
`5,187,776
`2/1993 Yanker ...........0
`«395/157
`
`5,201,034
`4/1993 Matsuura etal. .
`w. 395/155
`
`5,204,813
`4/1993 Samph et al...
`.. 364/419
`5,252,951
`10/1993 Tannenbaum etal. ...
`.. 345/156
`5,260,697
`11/1993 Barrett et al.
`.........
`.. 345/173
`5,317,140
`5/1994 Dunthor......
`w. 250/221
`
`-. 395/139
`5,341,466
`8/1994 Perlin et al.
`..
`
`oo.
`ceseesseesceesees 382/8
`5,353,356 10/1994 Waugh et al.
`11/1994 Hasebeet al. oo. scseessseeseers 358/451
`5,363,211
`
`Primary Examiner—Alyssa H. Bowler
`Assistant Examiner—Walter D. Davis
`Attorney, Agent, or Firm—George E. Grosser
`
`[57]
`
`ABSTRACT
`
`A mobile, hand-held personal communicator includes a
`casing for housing a cellular phone, modem, and data
`processing system. Graphics imagefiles are stored and can
`be selectively displayed on a touchscreen display. A zoom
`function magnifies areas of a graphics image, such as a fax
`image,that has been received andstored in the PC. When the
`zoom function is selected, the user touches the screen, and
`a magnification frame or window appears. When the user
`removeshis/herfinger, the area within the frame is magni-
`fied. The user can also drag the frame around the screen,
`while still touching the screen, to select a specific area to be
`magnified when the userreleases the screen. A pan function
`allowsthe user to shift the image within a viewing area. The
`user can pan the image by touching the display at an initial
`touch point, and moving his/her finger keeping it in contact
`with the screen, to shift the touch pointto a new touch point.
`Uponreleasing the new touch point, the image is redrawn in
`a new position corresponding to the change in position
`betweenthe initial and new touch points.
`
`9 Claims, 5 Drawing Sheets
`
`180
`
`SCREEN
`
`
`TOUCHED
`
`
`START- ZOOM
`
`
`TOOL SELECTED
`
`TO ENTER ZOOM MODE
`
`
`
`
`DISPLAY AREA IN BOX
`MOVEHIGHLIGHTED
`
`
`
`MAGNIFED TO FILL
`BOX TO NEW POSITION
`
`
`
`VIEWING AREA
`AROUND NEW TOUCH
`
`
`
`POINT
`
`MODE
`END ZOOM
`
`1
`
`APPLE 1009
`
`HIGHLIGHT BOX
`
`AROUND TOUCH
`
`POINT
`
`
`
`
`
`1
`
`APPLE 1009
`
`

`

`Sheet 1 of 5
`
`5,615,384
`
`US. Patent
`
`Mar. 25, 1997
`
`FIG. 1
`
`
`2
`
`

`

`U.S. Patent
`
`Mar. 25, 1997
`
`Sheet 2 of 5
`
`5,615,384
`
`SPEAKER
`
`MICRO-
`PHONE
`
`FIG. 3
`
`98
`
`102_
`
`SERIAL
`
`66
`
`63
`
`CONTROLLER
`
`“64
`
`14
`
`ON/OFF
`
`PORT
`ra
`DECK — PORT
`ty
`PORT
`
`in 104
`BEEPER
`
`MEMORY RO|PROCESSOR|
`
`
`Ea
`70 62 ee|60/7
`
`
`76
`BIOS
`7a}post_
`
`
`
`
`74
`
`TOUCH
`CONTROLS
`
`BATTERY
`
`GAIN
`
`5
`
`2
`
`
`Lt
`rower [ae
`
`BATTERY
`
`5B
`
`BACKLIGHT
`
`BACKLIGHT
`INVERTER
`
`DOWN
`
`40
`
`16
`
`18
`
`72\|
`
`26
`
`28
`
`DISK ROM
`MT
`oS
`
`
`TOUCH
`
`ayS{NAVIGATOR Loaic OVERLAY
`
`
`(PP PROG.
`wa
`38
`
`3
`
`

`

`U.S. Patent
`
`Mar. 25, 1997
`
`Sheet 3 of 5
`
`5,615,384
`
`FIG, 4
`
`Address Book] 128
`K
`
`Caiculator
`
` TOOLS
`
`122
`
`38
`
`120
`
`172
`
`11>
`
`140
`
`38
`
`143
`
`142
`
`58
`
`132
`
`MEMO- 7/4/93
`Subject: contacts
`
`FIRSTNOTE-1/2/493
`Subject: Meeting
`
`ITWORKS- 10/22/93
`Subject: Test Results
`
`NEWS- 12/28/93
`Subject: Retirement
`
`PROJEGCT- 7/4/93
`Subject: Vacation
`
`14 +S] sn] [Fn]
`[Sve|Laut]Lem]
`I
`44 “Tre(zo
`111 ffl] foo
`11> re oY
`
`134
`
`4
`
`

`

`5,615,384
`
`
`SCREEN
`TOUCHED
`?
`
`U.S. Patent
`Mar, 25, 1997
`Sheet 4 of 5
`FIG. 5
`
`START- ZOOM
`150
`
`
`TOOL SELECTED
`
`TO ENTER ZOOM MODE
`
`
`
`HIGHLIGHT BOX
`AROUND TOUCH
`
`
`
`
`
`DISPLAY AREA iN BOX
`MOVE HIGHLIGHTED
`
`MAGNIFED TO FILL
`
`BOX TO NEW POSITION
`VIEWING AREA
`
`AROUND NEW TOUCH
`
`POINT
`
`
`
`
`5
`
`

`

`U.S. Patent
`Mar. 25, 1997
` 172
`START- PAN
`
`
`TOOL SELECTED
`
`
`TO ENTER PAN MODE
`
`Sheet 5 of 5
`
`5,615,334
`
`FIG.7
`
`
`SCREEN
`TOUCHED
`
`TOUCH POIN
`RELEASED
`
`
`
`
`
` SHIFT DISPLAYED IMAGE
`
`
`iN SAME DIRECTION AND
`DRAW NEW BOX AND
`.TO SAME EXTENT AS
`
`
`
`“RUBBER-BAND* BETWEEN
`CHANGE IN TOUCH POINT
`
`
`
`ORIGINAL POINT AND
`BOX POSITION INDICATES
`
`
`
`NEW POINT
`
`192
`
`6
`
`

`

`5,615,384
`
`1
`PERSONAL COMMUNICATOR HAVING
`IMPROVED ZOOM AND PAN FUNCTIONS
`FOR EDITING INFORMATION ON TOUCH
`SENSITIVE DISPLAY
`
`The application is a continuation of application Ser. No.
`08/146,341 filed Nov. 1, 1993, now abandoned.
`
`RELATED APPLICATIONS
`
`The following applicationsarcall assignedto the assignee
`of this invention, and cover different inventions arising out
`of a common development project:
`(1) Application Ser. No. 07/976,127, filed Nov. 13, 1992, for
`“NAVIGATOR APPLICATION FOR PERSONAL COM-
`MUNICATOR’”, by B. A. Beatty et al.
`(2) Application Ser. No. 08/146,342,
`filed concurrently
`herewith, for “PERSONAL COMMUNICATOR HAV-
`ING IMPROVED CONTRAST CONTROL FOR
`TOUCH SENSITIVE DISPLAY”, by D. J. Allard et al.
`(3) Application Ser. No. 08/145,914,
`filed concurrently
`herewith, for “PERSONAL COMMUNICATOR HAV-
`ING COLLAPSIBLE KEYBOARD FOR EDITING
`DOCUMENTSON TOUCH SENSITIVE DISPLAY”, by
`D. J. Allard etal.
`The aboverelated application (1) deals primarily with an
`operating system extension called the “Navigator”, which
`manages screen displays and controls switching between
`different communication and data processing functions.
`Related application (2) deals with an improved contrast
`control for a touch sensitive display. Related application (3)
`deals with managing a relatively small
`touch sensitive
`display in which a simulated collapsible keyboardis used to
`edit/enter information in a data viewing area. The present
`application deals with managing the data presented on the
`screen of on a relatively small touch sensitive display and
`providing novel pan and zoom functions for viewing por-
`tions of a relatively large image.
`
`BACKGROUND OF THE INVENTION
`
`The invention was designed as part of a personal com-
`municator that combines a cellular phone, data and fax
`modem, and a data processing system, into a mobile, com-
`pact, lightweight, battery operated, handheld device having
`a variety of data processing and communication functions.
`These functions include operating the device as a standard
`cellular phone; a calendar showing a list of things to do at
`various times; an address book for organizing names,
`addresses and phone numbers; an editor allowing text entry
`and editing; a calculator; and a communication device for
`fax, E-mail and various data bases and services.
`The personal communicator is in the form of a handset
`having an earphone or speaker at one end and a microphone
`at the other end. The speaker and microphone are spaced
`apart the same distance as the distance between an ear and
`the mouth of an “average” user,
`to facilitate use of the
`communicator as a cellular phone. A small
`touchscreen
`display is located between the speaker and the microphone.
`The display has a fine pitch liquid crystal display (LCD)
`allowing it
`to be operated as a standard CGA with a
`resolution of 640x200 pixels. The touch sensitive display
`provides a graphic user interface and is used as an input/
`output device for a variety of functions or applications.
`Within the prior art, particularly that of desktop personal
`computers, it is publicly known to control pan and zoom
`functions on graphic images by positioning a cursor with the
`
`10
`
`20
`
`25
`
`40
`
`45
`
`55
`
`60
`
`65
`
`2
`aid of a cursor control device such as a keyboard, mouse,
`trackball, pen, etc. With a mouse controlled screen, a com-
`mon zoom function is to click at one corner ofthe area to be
`magnified, and drag the mouse to another corner. Upon
`release, the delineated area is magnified. Panning is done by
`moving the cursor against the edge of a screen image. U.S.
`Pat. No. 5,187,776- Yanker discloses an “IMAGE EDITOR
`ZOOM FUNCTION” for a personal computer CRT display,
`in which a small viewport window showsan imageat a (1x)
`magnification. The viewport is superimposed on a larger
`display of a magnified portion of the image centered about
`the location of the cursor in the viewport. Thecursoris also
`movable in the viewport to pan the magnified image.
`U.S. Pat. No. 4,899,292- Montagan et al, discloses a
`“SYSTEM FOR STORING AND RETRIEVING TEST
`AND ASSOCIATED GRAPHICS”. The system includes a
`computer coupled to a terminal having a resistive touch-
`screen. A wand is connected to an analog to digital converter.
`Whenthe wandis touchedto the screen, it detects a voitage
`signal
`that
`is converted into coordinates. The image is
`magnified by two centered at the point of touch.
`
`SUMMARYOF THE INVENTION
`
`Oneof the objects of the invention is to provide improved
`zoom and pan functions for a mobile, handheld personal
`communicator having a small touchscreen display actuated
`in response to being touched by a user’s finger.
`Another object of the invention is to provide a novel and
`improved meansfor selecting areas of a touchscreen graphic
`image to be panned or magnified, without requiring the use
`of a pointer device such as a trackball, pen or mouse.
`A further object of the invention is to provide a personal
`communicator in which graphic images are stored for dis-
`play on a finger operated touchscreen display, and a dis-
`played image can be zoomed by touching the display
`whereby a frame appears delineating the area to be magni-
`fied, the frame being dragged by slidingthe finger across the
`display, until the frame indicate the area the user wants
`magnified.
`Briefly, in accordance with the invention, a zoom function
`is used to magnify areas of a graphics image, such as a fax
`image,that has been received and stored in the PC. When the
`zoom function is selected, the user touches the screen, and
`a Magnification frame or window appears. When the user
`removeshis/her finger, the area within the frame is magni-
`fied. The user can also drag the frame around the screen,
`while still touching the screen, to select a specific area to be
`magnified when the user releases the screen.
`Whenthe pan function is selected, the entire screen acts
`as a window or mask over the image. The image thus
`appears as a sheet of paper behind the screen. The user can
`then press the screen anywhere at an initial touch point, and
`movehis/her finger keeping it in contact with the screen, to
`shift the touch point to a new touch point. Uponreleasing the
`new touch point, the image is redrawn in a new position
`corresponding to the change in position between the initial
`and new touch points.
`
`DRAWINGS
`
`Other objects and advantages of the invention will be
`apparent from the following description taken in connection
`with the accompanying drawings wherein:
`FIG. 1 is a plan view of a personal communicator
`embodying the invention;
`
`7
`
`

`

`5,615,384
`
`3
`FIG. 2 is a longitudinal view through the personal com-
`municator shown in FIG. 1, schematically illustrating vari-
`ous components;
`FIG.3 is ais a block diagram ofelectrical components of
`the personal communicator shown in FIG, 1;
`FIG.4 is a illustration of a sequenceof screens, beginning
`with the first screen that appears when the PC is turned on,
`showing how to get into zoom and pan functions;
`FIG. 5 is a flow chart of the zoom function:
`
`FIG. 6 is a diagram of different scrcensillustrating the
`zoom function;
`FIG. 7 is a flow chast of the pan function; and
`FIG.8 is a diagram of different screensillustrating the pan
`function.
`
`DETAILED DESCRIPTION
`
`General System Organization
`
`Referring now to the drawings, and first to FIG. 1, a
`personal communicator (PC) 10is of the type generally set
`forth above and comprises a cellular phone and a complete
`data processing system further details of which are described
`below. PC 10 includes a housing or casing 12 having
`mounted along one side thereof a plurality of external,
`manually operated push buttons including an ON/OFFpush
`button 14, an UP push button 16, and a DOWN push button
`18. PC 10 hasa size facilitating usage as a hand-held device
`and buttons 16 and 18 are Jocatedso that a right handed user
`can wrap his/her fingers around the rear of the casing and
`actuate the buttons to adjust speaker volume while the user
`is listening to the phone speaker. A pull-out antenna 20 is
`mounted at the top of the casing. A speaker 22 is mounted
`near the top of the casing and a microphone 44 (FIG.2)is
`mounted inside the bottom of the casing behind port holes
`24. A touchscreen display 26 is mounted on the middle
`portion of the casing between the speaker and the micro-
`phone, and provides an input/output (I/O) device and user
`interface for entering information into and receiving infor-
`mation, from the PC and the data processing system therein.
`The general shape of PC 10 is similar to the shape of
`commercially available portable phones except that display
`26 occupies the space normally used for a standard, elec-
`tromechanical, twelve push-buttons, phone Keyboard- Dis-
`play 26 includes a touch sensitive overlay 28 the outer
`surface of whichis flush with adjacent portionsof casing 12.
`Referring to FIG. 2, PC 10 has a plurality of miniaturized
`components mounted within casing 12. The components
`include speaker 22 and microphone 44 spaced apart a
`distance corresponding to the distance between an ear and
`the mouthof an averagesize user to facilitate use as a phone.
`PC 10 is battery operated and includes a battery 46 detach-
`ably mounted on the back wall of the casing. A cellular
`phone transceiver 48 is mounted inside casing 12 and is
`connected to antenna 20,
`the antennal being extensible
`outwardly from the casing for radio frequency (RF) com-
`munications. Display 26 is rectangularly shaped and com-
`prises a touch sensitive overlay 28 that overlies a liquid
`crystal display (LCD) 38, and a backlight 40, The LCD has
`a viewing area of 36 mm by 115 mm andis operated as an
`all points addressable display. A planar card 50 is mounted
`within the center of casing 12 and has mounted thereon
`various components including a power supply 52, a central
`processing unit (CPU) 54 that forms the system controller
`(FIG. 3), a modem 56, and other components 58 including
`a connector for connecting and supporting an optional
`
`5
`
`15
`
`20
`
`40
`
`45
`
`50
`
`55
`
`60
`
`4
`Personal Computer Memory Card International Association
`(PCMCIA)card 57 that conforms to published standards of
`the Association for a version 2.0 card support.
`Referring to FIG. 3, system controller 54 is single chip
`implementation of a PC/XT class portable computer, the
`chip being a Model VG230 processor commercially avail-
`able from Vadem, Inc., San Jose, Calif. Controller 54
`comprises a plurality of subsystems including a micropro-
`cessor 60 for executing various programs stored in the
`communicator, an LCD controller 62, a COM1 communi-
`cation port 64, and pinouts for connection to various devices
`the major ones of which are shown in FIG. 3. A bus 66
`interconnects controller 54 with a main memory 68, a read
`only memory (ROM)70, a disk ROM 72,touch controls 74,
`and modem 56.
`
`ROM70 stores a basic input/output system (BIOS) 76 and
`a poweronself test (POST) program 78, and is accessed in
`the usual manner. Disk ROM 72, however, is accessed, in a
`manner that simulates a disk drive, and stores files contain-
`ing a disk operating system (DOS) 79, the Navigator pro-
`gram 80 which is an extension to the DOS, application
`programs 81 that produce various communication and data
`processing functions in PC 10, and interrupt handlers 83 for
`handling interrupts specific to the applications. Additional
`memory and application programs can be added by the
`PCMCIAcard. When the communicator is turned on, POST
`program 78is executed. In response to a successful comple-
`tion of the test, a kernel of DOS 79 is then loaded from disk
`ROM 72 into main memory 68, for execution. DOS 79
`accesses disk ROM 72 and loads the Navigator and the
`application programs, as needed, into main memory 68 for
`execution by microprocessor 60. The Navigator controls the
`information displayed on the LCD, paints the various
`screens associated with the. application programs on the
`LCD,and spawns new programs in response to user selec-
`tion through actuation of the touchscreen interface.
`Communication functions are handled by RF deck 48 and
`modem 56. RF deck is connected to a beeper 94 that is
`sounded or actuated to alert the user to an incoming call.
`Controller 54 is also connected to beeper 94 andis able to
`selectively actuate the beeper such as when a screen button
`is selected. Deck 48 is also connected to an audio mux 96
`which controls the routing of audio signals between deck 48,
`speaker 22, microphone 44, modem 56, and audio port 98.
`Modem 56 is connected to a phone port 100 by means of
`which PC 10 can be connected by a cable (not shown) into
`a phone network or system. Deck 48 is also connected to a
`mux 102that in turn is connected further to a serial port 104
`and to COM1port 64 of controller 54.
`Battery 46 is connected to power supply 52 and supplies
`the primary powerfor operating PC 10. A backupbattery 82
`is also connectedto the power supply and supplies powerto
`volatile memory when the main battery is removed for
`replacement. Power supply 52 is connected to a backlight
`inverter 84 that, in turn, is connected to backlight 40 for
`controlling back lighting of the LCD to allow PC 10 to be
`operated under low light conditions. PC 10 preferably
`includes power managementfunctions (not shown) for con-
`serving energy usage and prolonging battery life.
`LCDcontroller 62 is connected by a data bus 88 to supply
`data to LCD 38 to control the contents of the information
`displayed by the LCD. Controller 62 drives a contrast
`control signal (CCS) onto line 90 for transmission to power
`supply 52, In response to the CCS, power supply 52, drives
`a contrast voltage signal (CVS)onto line 92 that determines
`the contrast of LCD 38,
`
`8
`
`

`

`5,615,384
`
`5
`6
`other portrait screens. PHONE button 114 is used to transfer
`Touch overlay 28is a resistive pressure sensitive overlay
`control to the phone application from other applications, to
`that produces an analog voltage signal
`indicative of an
`allow the user to place or receive a phonecall regardiess of
`overlay location being pressed. Overlay 28 is connected to
`the current application. TOOLS button 116 is used to trans-
`touch controls 74 that convert the analog signal into a digital
`fer control to an office tools screen allowing the user to
`signal representing the X, Y coordinates of the overlay
`select another application. BACK button 118 is used to
`location being pressed. The digital coordinates are then
`transfer control back to the previously selected screen(s).
`transmitted over bus 66 to controller 54 and memory 68 for
`When TOOLS button 116 of screen 108 is selected, the
`use in determining the purpose ofthe location being pressed.
`Touch controls 74 are further connected to multifunction
`Navigator then paints a Tools screen 124, where arrow 126
`represents switching from Phone screen 108 to Tools screen
`logic 86. In response to sensing that the overlay 28 has been
`124. Tools screen 124 displays a plurality of selectable
`touched, controls 74 generate an interrupt request that is
`buttons 128 that allow-a user to select an application from a
`transmitted to logic 86 and logic 86 then transmits the
`menu.of different applications such as an address book, a
`interrupt request to controller 54. UP and DOWN buttons 16
`calculator, a FAX, E-mail, data base services, and a note pad.
`and 18 are also connected to logic 86 which debounces any
`Whenthe FAX button is selected, the Navigator switches to
`signals due to the mechanical nature and generates a button
`the Fax application via arrow 129, and a FAX screen 130 is
`number identifying which button has been pressed. The
`painted on display 38.
`numbers are stored in I/O ports (not shown) which can be
`FAX screen 130 displays a plurality of selectable buttons
`polled by the system. Multifunction logic 86 is also con-
`132 that formalist of different fax files stored in the system.
`nected to RF deck 48 and audio mux 96 allowing controller
`20
`Buttons 134 are for the purpose of scrolling down and up in
`54 to communicate therewith and provide functions not done
`the list. Row 111 of the common buttons, is displayed at the
`by the system controller, such functions not being germane
`to the invention.
`bottom of screen 130. In responseto the selection of one of
`buttons 132, the Navigator switches, at 140, toa FAX VIEW
`screen 138 having a view area 142 within which the first
`page of the selected fax file, is displayed. Displayed at the
`bottom of scrcen 138, beneath view area, arc: a row 111 of
`common bottoms; two rows 144 of buttons for selecting
`functions of Save, Quit, Erase, Rename, and UnZoom; and
`a row 146 of tool buttons Draw, Eraser, Zoom, and Pan.
`By way of example, suppose a fax file named DIAGRAM
`contains an image of a flow chart similar to FIG. 5 of the
`drawingsherein. In response to selecting the button 132 with
`the name DIAGRAM therein, the Navigator and fax appli-
`cation cause the illustrated image 143 to appear in the
`viewing area 142 of FAX VIEW screen 138. The initial
`image 143 represents the full, first page of the file with no
`magnification. Quite obviously, the image, in this example,
`is so small that details thereof are not perceivable bya user.
`The zoom and pan functions described below allow the user
`to magnify the image and shift it around, as desired.
`The zoom function is initiated or started when the user
`selects the Zoom tool 146at the bottom of screen 138. With
`reference to FIGS. 5 and 6, when the Zoom toolis selected,
`a zoom mode of operation is begun at 150, Initially, a
`decision 152 is made as to whether or not, the touchscreen
`is being touched. If not, a loop is made until the user touches
`the screen at a touch point, whereupon a rectangular high-
`lighted box 168 is drawn with the touch point being within
`the borders. Box 168 delineates the area of viewing screen
`142, which is to be zoomed or magnified. Preferably, the
`shape of box 168 is similar to but smaller than the shape of
`viewing area 142. Next, a decision is made at 156 whether
`or not the touch point has been released. If not, a loop is
`entered with decision 162 and continues until the loop is
`broken the releasing the touch point, such as by removing
`the user’s finger from the display, whereupon step 158
`displays the portion of the image within box 168 tofill the
`viewing area with an enlarged image 170 having the same
`aspectratio. At that point, the zoom mode ends and a return
`is made at 160 to the normal pan view screen, which at that
`point continues to display image 170. If further magnifica-
`tion is desired, the user can repeat the process. Decision 162
`determines if the touch point has changed which would be
`caused by sliding the finger over the display to a new
`position. If a change occurs, step 164 moves the highlighted
`box to a new position around the new touch point. The loop
`is traversed as the user movesthe touch point, until the finger
`
`The screens include selectable buttons having thin solid
`lines along their left and top edges and shadowlines along
`their bottom and right edges. The buttons also bear labels or
`icons, or both. A button is selected by a touch and release
`action. That is, a button is selected when the user touches
`overlay 28 in the area overlaying the button, and then
`releases the button by releasing the overlay. When a button
`is so touched,it is highlighted. When a button is so selected,
`the system sounds beeper 94 to provide audio feedback to
`the user. All buttons are also operated by a “last contact”
`method to that a user can easily recover from touching an
`unintended button by leaving a finger on the screen and
`sliding to anothcr button or a non-button area. Asa fingcris
`slid across the overlay, new buttonsare highlighted while the
`old buttons are retumed to normal, non-highlighted condi-
`tions, thereby providing the user with visual feedback as to
`which button will be selected when the finger is removed
`from the overlay. Each application program stores one or
`more mapscorrelating the X, Y positions of the overlay with
`the different butions and applications. The Navigator scans
`the map,
`in response to receiving X, Y coordinates,
`to
`determine the function of a selected area or button, and take
`appropriate action.
`Screen 108 includes a simulated cellular phone keyboard
`120 and a data entry area 122 in which phone numbers are
`displayed as the respective digits of a phone number are
`entered by the keyboard. Screen 108 also comprises a row
`111 of selectable buttons including a HELP button 112, a
`PHONE button 114, a TOOLS button 116, and a BACK
`button 118, located along the bottom edge of the screen.
`These four selectable buttons are common to other menu
`screens and appear at the same locations on screen 110 and
`
`50
`
`35
`
`65
`
`Operation
`As indicated above, when PC 10 is tured on, DOS 79 and
`Navigator 80 are loaded into main memory 68, along with
`an application. By default, the first application to be loaded
`is a phone application that provides and controls cellular
`phone operations. Such application is executed and interacts
`with the Navigator program to create a Phone screen 108 on
`LCD 38, which screen is shown in FIG. 4. The various
`screens displayed on LCD 38 are painted by Navigator 80
`using information from the specific application program to
`which the screen is relevant.
`
`25
`
`35
`
`40
`
`45
`
`9
`
`

`

`5,615,384
`
`7
`is removed from contact with the touchscreen display.In this
`manner,the user is able to select the specific area desired to
`be magnified.
`Relative to the exemplary series of screens shown in FIG.
`6, the size of box 168 relative to the size of view area 142
`produces a magnification of two. Obviously, otherratios can
`be used.If the user touches the center of the view area, the
`box 168 is centered on the point of touch. But if the user
`touchesthe view area near an edgeof the view area, the box
`is shown in full and would not be centered at the point of
`touch. Beginning with the left screen in FIG. 6, the user
`touchesthe screen at point 166, and box 168 is drawn. The
`box initially may not fully surround, diagram 143. Then, the
`user movesthe finger until box 168 contains image 143,as
`shownin the center screen. Upon removal of the finger, the
`view area is filled with a magnified image 170.
`The pan function is initiated or started when the user
`selects the Pan tool 146 at the bottom of screen 138. With
`reference to FIGS. 7 and 8, when the Pan tool is selected, a
`pan mode of operation is begun at 172. Initially, a decision
`174 is made as to whether or not, the touchscreen is being
`touched. If not, a loop is made until the user touches the
`screen at a touch point such as at point 186, whereupon a
`small rectangular box 188 is drawn with the initial touch
`point being within the bordersof the initial box 188. A loop
`is formed with decision 180 being made in response to a
`negative decision out of 178. Decision 180 decides if the
`touch point position has changed. If so, a new box 190 is
`drawn and is connected to box 188 by a rubber-band 192. As
`the fingeris slid around the view area, box 190 is constantly
`redrawn beneath the finger, and the rubber-band is con-
`stantly redrawn to give the appearance of stretching and
`contracting. The finger is moved in the direction and to the
`same extent that the user wants to shift or move the image.
`Upon release of the screen, decision 178 passes control to
`step 184 which,in response to releasing the screen, points or
`redraws ihe image 170 in a position that is displaced from
`the initial position in the same direction and to the same
`extent as the displacement of box 190,at the time ofrelease,
`from initial box 188. The process can be repeated to shift the
`image to a newposition. It may be that portionsoftheinitial
`image are lost when shifted outside the view area, and/or
`that new image portions come into view. The combination of
`the pan function with the zoom function allows a user to
`readily view different portions of a full page of fax material
`on the smali display.
`As should be apparent to those skilled in the art, the
`programs are stored as electrical signals in the different
`memories. The term “processing means” as used in the
`claims encompasses not only memories which store the
`programmed signals but also the different components,
`including the microprocessor, which operate in responseto
`the programmed signals, to execute the different processing
`functions and operations described above. It should also be
`apparent to those skilled in the art that many changes can be
`made in the details and arrangements of steps and parts
`without departing from the scopeofthe invention as defined
`in the appended claims.
`What is claimed is:
`1. A personal communicator comprising:
`a casing having a size adapted to be heldin the hand of
`@ USeT;
`
`a touch sensitive display mounted on said casing;
`first means for displaying a graphics image on said
`display;
`:
`second means for displaying a magnification frame of
`predefined sizc and shape on said display and imposed
`
`25
`
`45
`
`60
`
`65
`
`10
`
`8
`on said graphics image in response to a user touching
`said display at a touch point with a finger, said frame
`being located beneath said touch point and delineating
`the actual portion of said graphics image to be magni-
`fied;
`third means for moving said frame around said display in
`tesponse to sliding said finger around said display and
`in contact therewith; and
`fourth means for magnifying and displaying on said
`display said portion of said graphics image delineated
`by said frame, in response to disengaging said finger
`from said display.
`2. A personal communicator in accordance with claim 1
`wherein said second meansis selectively actuated, and said
`communicator further comprises selectively actuated fifth
`means for panning said graphics image in response to said
`user touching said display with a finger at one point and
`sliding the finger to another point thereby indicating direc-
`tion and extent of panning.
`3. A personal communicator in accordance with claim 2
`wherein said fifth means comprises:
`sixth means for drawing a symbol on said display indi-
`cating wherethe userinitially touched said display and
`for continuously displaying a second symbol beneath
`the finger as the finger slides to other positions; and
`seventh meansfor redrawing said graphics image in a new
`position in responseto a release of said display at said
`second symbol.
`4. A personal communicator comprising:
`an elongated casing of a size adapted to be held in the
`hand ofa user;
`a cellular phone mounted in said casing and including a
`speaker mounted adjacent one end of said casing and a
`microphone mounted adjacent another end of said
`casing;
`a touchscreen display mounted on said casing and having
`a rectangular shape elongated in a direction similar to
`said casing, said touchscreen display forming a user
`interface for entering information into and outputting
`information from said communicator,said touchscreen
`display comprising a pressure
`sensitive overlay
`coupled with a liquid crystal display (LCD) for dis-
`playing different screens of information;
`processing means mountedin said casing, said processing
`means being connected to said touchscreen display for
`receiving information entered through said overlay and
`outputting information on said LCD, said processing
`means comprising screen management meansfor paint-
`ing different screens on said LCD, said processing
`means havinga plurality of modes of operation includ-
`ing a zoom mode;
`storage means for storing graphics imagefiles;
`said screen management means comprising
`first means for painting on said LCDa first screen having
`a plurality of selectable buttons listing said graphics
`imagefiles;
`second meansfor painting on said LCD a se