`6,073,036
`(114) Patent Number:
`United States Patent 55
`Heikkinenetal.
`[45] Date of Patent:
`*Jun. 6, 2000
`
`
`4,790,028 12/1988 Ramage ....cccccscceeseeeees 382/47
`[54] MOBILE STATION WITH TOUCH INPUT
`
`5,016,002
`5/1991 Levanto ......
`.. 340/756
`HAVING AUTOMATIC SYMBOL
`
`eseeeeeees 340/712
`5,119,079
`6/1992 Hubeet al.
`oe
`MAGNIFICATION FUNCTION
`5,201,034 4/1993 Matsuura et al.oeeee 395/155
`
`
`11/1993 ROSSE oo. eceecenesreeseeseeneeeeeeseeeees 341/22
`5,266,949
`
`8/1994 Thompsonet al. oe 455/558
`5,335,276
`
`8/1994 Perlin et al. occ 395/139
`5,341,466
`
`4/1995 Muurinen oe
`5,408,060
`eeeeeeeeeeeee 200/314
`6/1995 Allard et ab. weer 345/173
`5,422,656
`
`11/1995 Thompson oo.
`eeeee 455/566
`5,465,401
`
`10/1996 Tiller et al. oe
`eeseeeeeeeeee 379/58
`5,568,536
`
`
`3/1997 Allard et ab. wees 395/800
`5,615,384
`9/1998 Allard et ab. oceans 345/173
`5,815,142
`
`[75]
`
`Inventors: Teuvo Heikkinen, Oulu; Petri Piippo,
`Karkkila; Harri Wikberg, Helsinki;
`Miika Silfverberg, Helsinki; Panu
`Korhonen,Helsinki; Harri Kiljander,
`Espoo, all of Finland
`
`[73] Assignee: Nokia Mobile Phones Limited, Espoo,
`Finland
`
`[*] Notice:
`
`This patent issued on a continued pros-
`ecution application filed under 37 CFR
`1.53(d), and is subject to the twenty year
`patent
`term provisions of 35 U.S.C.
`154(a)(2).
`
`[21] Appl. No.: 08/847,796
`
`[22]
`
`Filed:
`
`Apr. 28, 1997
`
`Tent, Cd eeecccceeeeecessnneeeceenteeeeenneees HO4B 1/38
`PSL]
`[52] U.S. Ch.eee 455/575; 455/566; 379/354
`[58] Field of Search occ 455/575, 425,
`455/566, 567; 345/63, 147, 173, 121, 130,
`156; 379/93.19, 93.23, 354; 382/298, 451
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`Primary Examiner—Cuttis A. Kuntz
`Assistant Examiner—Binh K. Tieu
`
`Attorney, Agent, or Firm—Ohlandt, Greeley, Ruggiero &
`Perle, L.L.P.
`
`[57]
`
`ABSTRACT
`
`A wireless user station (10), such as a cellular telephone, is
`provided with a touch sensitive input device, such as a
`touch-sensitive display (20) or a touchpad (23). A display
`device displays a plurality of symbols. A first location of a
`tactile input is detected and displayed symbols in the vicinity
`of the first location of the tactile input are magnified to
`occupy a larger display area. Subsequently, a second loca-
`tion ofthe tactile input is detected, thereby identifying a new
`position of the user’s fingertip. After the second location on
`the touch sensitive screen remains constant, or after the user
`lifts the fingertip, the second location of the tactile input is
`reported and associated with the selected symbol.
`
`4,755,811
`
`7/1988 Slavin et al. oo. 340/731
`
`19 Claims, 8 Drawing Sheets
`
`040506000
`
`20a
`
`
`
`ABC...
`
` Options
`
`1
`
`APPLE 1033
`
`APPLE 1033
`
`1
`
`
`
`U.S. Patent
`
`Jun. 6, 2000
`
`Sheet 1 of 8
`
`6,073,036
`
`12
`
`4-10
`
`TRANSMITTER, 14.
`
`114A
`
`
`
`CONTROLLER,18
`
`DISPLAY,20
`
`22a
`
`22b
`
`RECEIVER, 16
`
`12
`
`TRANSMITTER,14
`
`x10
`14A
`CONTROLLER, 18
`
`
`DISPLAY,20
`
`MEMORY,24
`
`
`RECEIVER, 16
`
`FIG. 1B
`
`SPEAKER, 17
`
`
`
`
`SYMBOL MAP
`
`MICROPHONE, 19 DISPLAY/
`
`MEMORY,24
`
`2
`
`2
`
`
`
`6,073,036
`
`3
`
`
`
`U.S. Patent
`
`Jun. 6, 2000
`
`Sheet 3 of 8
`
`6,073,036
`
`fenneepenrnnpnrepennnefeereeeremeteem]
`
`INTIAL STATE
`
`5Sr B
`10: R1,
`R
`
`i bo
`
`Sa $5 |$6 S7 SB |so)
`gaRa RS| RO RT RB RO.
`
`fon-nvnnnfonnnnannfafennnnnnen4nascccnedencenecenedenceneees aoecenensiwanerenenpenennenen
`
`MAGNIFIED
`|
`|
`|
`wf (ZOOM)
`
`
`
`atteafnenewennnnenndennenenennnnnnn pnwnbaennaeeeneemnnnnrenewnnenereneneeeeeeecererereree:
`
`annennnnndnnneee hnennnfnenneenneeene
`t
`
`enennaneemereneegeemereenneeneeeeene eeneneenenmnennennangeenueseenmenemeneennapeneneeneneneenenecee
`
`4
`
`
`
`U.S. Patent
`
`Jun. 6, 2000
`
`Sheet 4 of 8
`
`6,073,036
`
`400
`
`
`
`
`ACTIVATE DETECTED
`ELEMENT ON DISPLAY
`
`
`
`IS DISPLAY
`BEING TOUCHED?
`
`404
`
` IDENTIFY ELEMENT
`
`REPORT DETECTED
`ELEMENT
`
`
`POSITION
`
`IDENTIFIED
`SYMBOL
`TO MAIN
`ROUTINE
`
`406
`
`MAGNIFY
`CHARACTERS AROUND
`ELEMENT POSITION
`
`OEOOPHONALY
`(
`)
`
`#20
`
`
`
`
`
`
`DETECT NEW ELEMENT|.NO
`POSITION
`
`HAS ELEMENT
`POSITION
`CHANGED?
`
`410
`
`
`
`
`STORE ELEMENT
`POSITION
`
`
`
`FIG. 4A
`
`418
`
`5
`
`
`
`U.S. Patent
`
`Jun. 6, 2000
`
`Sheet 5 of 8
`
`6,073,036
`
`400
`
`
`
`
`ACTIVATE DETECTED
`
`ELEMENT ON DISPLAY
`
`
`
`IS_ DISPLAY
`
`BEING TOUCHED?
`
`404
`
`406
`
`IDENTIFIED
`SYMBOL
`TO MAIN
`ROUTINE
`
`
`
`
`
`IDENTIFY ELEMENT
`REPORT DETECTED
`POSITION
`ELEMENT
`
`
`
`
`
`
`MAGNIFY
`CHARACTERS AROUND
`
`ELEMENT POSITION
`
`
`
`
`DE-—BOUNCE DELAY
`
`(OPTIONAL)
`
`
`420
`
`DETECT NEW ELEMENT|,NO
`POSITION
`
`
`
`
`HAS ELEMENT
`T
`POSITION
`STORECLEMEN
`
`FIG. 4B
`
`CHANGED?
`
`410
`
`418
`
`6
`
`
`
`U.S. Patent
`
`Jun. 6, 2000
`
`Sheet 6 of 8
`
`6,073,036
`
`
`yo|zlz[rinEiMCEICIES
`e|-|o]>|<
`
`aN—_
`FIG. SA
`
`FIG. 5B
`
`7
`
`
`
`
`
`
`
`U.S. Patent
`
`Jun. 6, 2000
`
`Sheet 7 of 8
`
`6,073,036
`
`FIG. SC
`
`20a
`
`||]>|©]g2
`[xo][ls
`
`e|-[o|>/<|4|2/2Z|HIN)
`<|o[2|4/>|*
`
`)N
`
`o|s|o|>F</s
`
`pac
`
`FIG. SD
`
`8
`
`
`
`
`
`
`U.S. Patent
`
`6,073,036
`
`Jun. 6, 2000
`
`Sheet 8 of 8
`
`040506000
`
`20a
`
`
`9
`
`
`
`6,073,036
`
`1
`MOBILE STATION WITH TOUCH INPUT
`HAVING AUTOMATIC SYMBOL
`MAGNIFICATION FUNCTION
`
`FIELD OF THE INVENTION
`
`This invention relates generally to radiotelephonesand,in
`particular,
`to radiotelephones or mobile stations, such as
`those capable of operation with a cellular network, having a
`touch sensitive input for the detection of user inputs.
`
`BACKGROUND OF THE INVENTION
`
`Portable radiotelephones for use in cellular networks are
`sometimes equipped with touch sensitive displays, or touch
`screens. The concept behind a touch screenisto facilitate the
`selection of a symbol by the simple act of pointing to, and
`touching a displayed symbol, rather than typing a particular
`keystroke on a keyboard.
`Becauseofthe relatively small size of a telephone display
`that
`is a result of the tendency to miniaturize portable
`equipment, there is generally not enough room on the face
`of a display to: (a) depictall the symbols availableto the user
`for the operation of the telephone; and/or (b) address each
`symbol by humanfingertip means.
`As an example of the former, all symbols such as
`numerals, alphanumerics, menu items, special icons, etc.
`that are related to telephone functions that may be available
`for use on a portable telephone cannot generally be allocated
`enough display pixels on the touch sensitive part of the
`display to visually distinguish and uniquely define each
`symbol. One approachof the prior art to accommodate large
`numbers of symbols on a single display was to allocate a
`fixed, small number of display pixels for each telephone
`function on a touch sensitive screen. The difficulty with this
`approachis that sometimes the numberofpixels allocated is
`insufficient to allow the easy identification of the character
`or function to be activated by touching the specific area. That
`is, with a limited number of pixels it may be difficult to
`create a sufficient numberof visual features of a symbol to
`allow its unambiguous identification.
`Another problem stems from the small amountof display
`area allocated to each symbol, legend, and/orrelated func-
`tion on a touch sensitive screen for its activation. This
`
`limitation arises because the total area necessary to accom-
`modate sufficient space to accommodatethe tip of the human
`finger, for each symbol, would typically exceed the total area
`of the display itself. When the symbols are made smaller and
`spaced closer together,
`the user’s fingertip will typically
`cover one or more adjacent symbols.
`With fingertip usage the displayed symbols or touch areas
`are required to be relatively large (e.g., 7 mmx7 mm for
`index finger use with a 2% error rate). In the past this has
`forced the use of a stylus, instead of the more naturalfinger,
`to achieve acceptable errorrates.
`Thatis, using the small area allocated in a touch sensitive
`display for a multitude of symbols cannot readily accom-
`modate the dimensions of a human fingertip, generally
`requiring the user to operate the interface using a pointing
`device, such as a stylus, pencil, pen,etc., that must be placed
`precisely over the desired character or function to be used to
`uniquely identify the desired input
`to the telephone. At
`times,
`the precision required in placing the stylus may
`challenge the coordination skills of the user, especially in a
`movingtrain, plane, or automobile where the motion of the
`vehicle makesit difficult to control the relative position of
`the stylus with respect to the touch screen. Furthermore, the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`use of a stylus may be objectionable to some users, and in
`some situations may present a safety issue.
`The foregoing problems will also arise if one were to
`employ some meansother than a touch sensitive display to
`provide a tactile, non-keypad input to a mobile telephone,
`especially if a small display device is used.
`OBJECTS OF THE INVENTION
`
`It is thus a first object of this invention to provide an
`improved method for depicting various symbols on a touch
`sensitive display of a radio telephone, thereby allowing a
`user to visually distinguish among a numberof symbols on
`the display.
`Another object of the inventionis to facilitate the manual
`designation of a symbol on a display, such as numerals,
`alphanumerics, menu itemsor other special icons related to
`telephone functions or other user generated data or instruc-
`tions.
`
`Another object of the invention is to provide an improved
`method for designating symbolsthatoffers a large input area
`on a display for designating a desired symbol,
`thereby
`facilitating fingertip control in a high vibration environment
`where the accuracy of the fingertip designation of a symbol
`is limited because of relative motion between the fingertip
`and the surface of a tactile input device such as, but not
`limited to, a touch screen.
`
`SUMMARYOF THE INVENTION
`
`The foregoing and other problems are overcome and the
`objects of the invention are realized by methods and appa-
`ratus in accordance with embodiments of this invention. A
`
`method and apparatus of this invention is disclosed for
`operating a wireless mobile or user station having a touch
`sensitive input device and a display,
`the display being
`capable of displaying a plurality of symbols. Each of the
`symbols may have a display attribute. The method com-
`prises a series of steps. A first step detects a first location of
`a tactile input on the touch sensitive input device, the tactile
`input generally being created by applying fingertip pressure
`to the face of the display for a touch screen embodiment or
`to a surface of touch pad. Next, the displayed symbolsin the
`vicinity of the first location correspondingto the tactile input
`are automatically magnified. The magnification gives more
`area on the display for the symbol(s), thereby facilitating
`manual symbol designation, as well as improving the visual
`representation of the symbol on the display. As the fingertip
`movesto designate the desired symbol on the magnified area
`of the display, a second location of the fingertip-induced
`tactile input is detected. The detection of the tactile input at
`the second location is made after the originally displayed
`character is magnified. In one embodiment, as soon as the
`tactile input remains constant for a predetermined period of
`time, the second location, whichis indicative of the selected
`symbol,
`is reported, In another embodiment
`the second
`location is reported when the fingertip is lifted from the
`touch sensitive surface. Also, concurrently with reporting
`the detected second location, the attribute of the symbol
`displayed at the second location may be changed to inform
`the user of the selection of the fingertip-designated symbol.
`In this invention the magnifying step magnifies the sym-
`bols by a factor of typically between 1.1 and 10. The time
`interval between the time a tactile input is detected at the
`second location and the time the position of the second
`location is reported (and the attribute of the display at the
`second location is changed), can be from somefraction of a
`second to several seconds. Also, the magnifying step can be
`
`10
`
`10
`
`
`
`6,073,036
`
`4
`mobile station are allocated between these devices according
`to their respective capabilities.
`A user interface includes a conventional earphone or
`speaker 17, a conventional microphone 19, a touch sensitive
`display 20, and possibly a second user input device, typi-
`cally a keypad 22, all of which are coupled to the controller
`18. The keypad 22, although optional for use in this inven-
`tion as someofits functions may be included in the display
`20, may include the conventional numeric (0-9) andrelated
`keys (#,*) 22a, and other keys 22b used for operating the
`mobile station 10. These other keys 22b may include, by
`example, a SEND key, and various menuscrolling and soft
`keys. When the keypad 22 is not included as part of the
`mobile station 10, the touch sensitive display screen 20
`provides an equivalent user interface function as the keypad
`22. The mobile station 10 also includes a battery 26 for
`poweringthe variouscircuits that are required to operate the
`mobile station.
`
`The mobile station 10 also includes various memories,
`showncollectively as the memory 24, wherein are stored a
`plurality of constants and variables that are used by the
`controller 18 during the operation of the mobilestation. For
`example, the memory 24 stores the values of various cellular
`system parameters and the number assignment module
`(NAM). An operating program for controlling the operation
`of controller 18 is also stored in the memory 24 (typically in
`a ROM device). The memory 24 may also store data,
`including user messages, that is received from the BMI 32
`prior to the display of the messages to the user. Symbols
`entered from the display 20 are buffered in the memory 24.
`A portion of the memory 24 mayalso used as a display
`screen buffer.
`It should be understood that the mobile station 10 can be
`a vehicle mounted or a handheld device. It should further be
`appreciated that the mobile station 10 can be capable of
`operating with one or more air interface standards, modu-
`lation types, and access types. By example,
`the mobile
`station may be capable of operating with any of a numberof
`air interface standards, such as IS-136, GSM and IS-95.
`Conventional AMPS mobile stations can also benefit from
`
`the teaching of this invention. It should thus be clear that the
`teaching of this inventionis notto be construedto be limited
`to any one particular type of mobile station or air interface
`standard.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`40
`
`45
`
`3
`performed more than once to accommodate situations
`wherein a particular symbol cannot be uniquely identified
`after only a single magnification step is performed.
`In
`general, if after a first magnification the fingertip is located
`on a boundary of two or more symbols, another one or more
`magnification steps can be performed.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The aboveset forth and other features of the invention are
`made more apparent in the ensuing Detailed Description of
`the Invention when read in conjunction with the attached
`Drawings, wherein:
`FIG. 1A is a block diagram of a mobile station that is
`constructed and operated in accordance with a first embodi-
`mentof this invention;
`FIG. 1B is a block diagram of a mobile station that is
`constructed and operated in accordance with a second
`embodimentof this invention;
`FIG. 2 is an elevational view of the mobile station shown
`in FIG. 1A, and which further illustrates a cellular commu-
`nication system to which the mobile station is bidirectionally
`coupled through wireless RF links;
`FIG. 3 is a detail of the display used in FIG. 1A showing
`the display in its initial state and in its magnified state, as
`described in the preferred embodiment;
`FIG. 4A showsa logic flow diagram of one embodiment
`of this invention, wherein a symbolis selected after a period
`wherein the fingertip remains stationary on the touch sen-
`sitive surface;
`FIG. 4B showsa logic flow diagram of another embodi-
`mentof this invention, wherein a symbolis selected after the
`fingertip is lifted from the touch sensitive surface; and
`in
`FIGS. 5A-5F illustrate exemplary displays useful
`explaining the operation of this invention, as well as illus-
`trating a presently preferred “fisheye” display magnification
`function.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`Reference is made to FIGS. 1A and 2 for illustrating a
`wireless user terminal or mobile station 10, such as, but not
`limited to, a cellular
`radiotelephone or
`a personal
`communicator, that is suitable for practicing this invention.
`The mobile station 10 includes an antenna 12 for transmit-
`
`ting signals to, and for receiving signals from,a base site or
`base station 30. The base station 30 is a part of a cellular
`network comprising a Base Station/Mobile Switching
`Center/Interworking function (BMI) 32 that
`includes a
`mobile switching center (MSC) 34. The MSC 34 provides a
`connection to landline trunks when the mobile station 10 is
`involved in a call.
`
`The mobile station includes a modulator (MOD) 14<A,a
`transmitter 14, a receiver 16, a demodulator (DEMOD) 16A,
`and a controller 18 that provides signals to and receives
`signals from the transmitter 14 and receiver 16, respectively.
`These signals include signalling information in accordance
`with the air interface standard of the applicable cellular
`system, and also user speech and/or user generated data.
`It is understood that the controller 18 also includes the
`
`The operating program in the memory 24 includes rou-
`tines to present messages and message-related functions to
`the user on the display 20, typically as various menu items.
`The memory 24 also includes software routines for execu-
`tion by controller 18 for implementing the method of this
`invention, described below in relation to FIG. 3 and FIGS.
`4A, 4B, and 5A-5E.
`In this embodimentof the invention, display 20 is a touch
`sensitive display where the location of, for example, a
`pressure disturbanceortactile input, such as the presence of
`a fingertip placed on the display, will send to controller 18
`the (X,Y) display coordinates identifying where display 20
`is touched. The X,Y resolution is a function of the display
`20. This determination of the position can be cyclical, at
`regular intervals, for example every 10 milliseconds. For
`example, if the display 20 is touched in the left upper most
`corner, display 20 will send two numbers, for example, 1, 1,
`circuitry required for implementing the audio and logic
`identifying the X=1, Y=1 element position of the display
`functions of the mobile station. By example, the controller
`area where the presenceofa tactile input, such asafingertip
`18 may be comprised of a digital signal processor device, a
`65
`or some other pointing means, has been sensed.
`microprocessor device, and various analog to digital
`converters, digital to analog converters, and other support
`While tactile input detection is described herein as one
`circuits. The control and signal processing functions of the
`embodimentof the invention, it is only one possible method
`
`50
`
`55
`
`60
`
`11
`
`11
`
`
`
`6,073,036
`
`5
`for detecting the presence of a fingertip on a touch sensitive
`screen. Other methods could be employed, such as sensing
`a presence/absence, and/or an amountof pressure applied by
`a fingertip or stylus.
`One suitable type of touch screen is manufactured by
`Nissha Printing Co., Ltd., Electronics Division, and is
`known as a “Fine Touch” touch screen. This particular
`analog touch screen uses a transparent resistive membrane.
`The use of digital touch screens having a transparent switch
`matrix is also within the scope of this invention.
`It should be appreciated that the position resolution of a
`tactile input on the touch sensitive screen cannot be more
`accurate than one area of resolution, or element, herein for
`example about 5 mm by 5 mm (25 millimeters square). By
`example, and assuming the analog touch screen
`embodiment, if two, four bit analog to digital converters are
`used to encode the location of the tactile input, the resulting
`position resolution is 16 by 16 resolvable locations.
`As shown in FIG. 3, for example, in a touch sensitive
`display of 50 mm by 50 mm dimensions, 100 areas of
`resolution or elements, shown as a 10 by 10 matrix,
`is
`capable of reporting the occurrenceof tactile inputs and the
`related positions referenced to the display’s X-Y coordinate
`frame. The fingertip activation (tactile input) positions from
`display 20 to controller 18 are reported, for example, as two
`numbers X,Y corresponding to the X,Y position within the
`10 by 10 matrix. The reported position of the fingertip
`activation, corresponds to the one 5 mm by 5 mm element,
`within the exemplary 50 mm by 50 mm touch sensitive
`screen (having 100 resolvable positions).
`As further shownin FIG. 3., in its initial state, each of the
`100 areasof resolution, or (sensing) elements,of the display
`showsa particular symbol, such as, for example, upper case
`ASCII characters A,B,C, .. . Z, icons, functions etc. The
`cited examples of symbols are only an example of the type
`of characters that may be displayed,in this case to substitute,
`or supplement, the character-based keypad 22. Other sym-
`bols that can be contained within an area of display resolu-
`tion are, for example, Kanji, Turkish, Arabic, Russian or
`other language specific characters, or any other
`icon,
`numeric, or legend that will fit, in abbreviated or complete
`form, on the surface defined by each pressure sensing
`element of the 10 by 10 matrix. Each displayed symbol is
`composed of a numberofpixels of display 20 and fits within
`one of the 100 pressure sensing elements present in display
`20.
`
`In FIG. 3 it is shownthat a fingertip FT is initially placed
`at the intersection of characters S2, S3, R2, and R3. S2 is
`displayed in (sensing) element row 9, column 2 (9,2), while
`S3 is displayed in element row 9, col 3 (9,3). Similarly,
`character R2 is displayed in element (10,2) while R3 is
`displayed in element (10,3). Upon sensing the presence of a
`tactile input
`in one of, or in more than one, or at
`the
`intersection of, elements (9,2), (9,3), (10,2), and (10,3),
`controller 18 refreshes and updates touch sensitive display
`20 so that element S2 is now re-displayed in a magnified
`(zoom) fashion in an area corresponding to four display
`elements (7,1), (7,2), (8,1) and (8,2). Similarly characters
`S3, R2 and R3 are also magnified, as shown in the magnified
`portion of FIG. 3, to occupy a display area, for example, four
`times their original size. It is now evident that since the
`original characters displayed in one element have been
`magnified to occupy the area of four elements each,
`the
`display area of the character itself is four times larger than
`the original, hence making it more readable and easier for
`the user to identify. Furthermore, since the area occupied by
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`the magnified characteris larger, comprising four resolvable
`sensing elements,
`it has become easier to designate by
`fingertip, since the center of the fingertip can be placed more
`readily over the symbol without accidentally activating
`another adjacent symbol.
`The controller 18 is assumedto store in the memory 24 a
`mapping between display coordinates and displayed
`symbols, and changes the mapping as a function of magni-
`fication level.
`
`The magnification ratio is not limited to a factor of 2 as
`described. The ratio can be from as little as 1.1 to, by
`example, a factor of 10. A factor of 10 is applicable when the
`display having 100 elements has to be filled with,
`for
`example, a single symbol requesting the immediate action of
`the user. An alert or visual alarm function can thus be shown
`
`in conjunction with, for example, an audible alert or alarm.
`Referring to FIG. 4A, a first method for operating the
`display 20 comprises a series of steps as described below.
`Upon power up of the mobile station 10, a software sub-
`routine starts as shown at step 400 in controller 18. Con-
`troller 18 receives periodic reports, for example, every 10
`msec, from display 20 as to the presence or absenceoftactile
`input indicative of the presence of a fingertip on the display.
`In other embodimentsofthis invention the controller 18 may
`receive a report only when the display 20 is actually
`touched. In a further embodimentof the invention it may be
`the controller’s responsibility to periodically poll the display
`20 to determine if and wherea tactile input has been sensed.
`Thus, although the invention is described in the context of
`a display that regularly outputs a signal to the controller 18,
`those skilled in the art will recognize that this is only one
`possible mode of operation.
`In step 402, if no input is detected on the touch sensitive
`display 20, the routine returns to start 400. If however, a
`tactile input is detected on display 20 by step 402,
`the
`corresponding display element(s) are identified in step 404.
`As discussed above with regard to the example shown in
`FIG. 3, step 404 may detect fingertip pressure at display
`element positions (9,2), (9,3), (10,2) and (10,3). In response
`to the results from step 404,
`in step 406,
`the symbols
`displayed in display element positions (9,2), (9,3), (10,2)
`and (10,3) are magnified (zoomed) to, for example, double
`the size of each of the four originally displayed symbols.
`This magnification causes the originally displayed four
`symbols to be redrawn, and magnified by controller 18 over
`a 16 display element area, as shown in FIG. 3. As further
`shown in the example in FIG. 3, the information originally
`displayed at location (9,2) is now magnified to cover display
`element positions (7,1), (7,2), (8,1) and (8,2). Following
`magnification step 406, the display is as shown in FIG. 3 in
`the magnified position for the fingertip location shownat FT.
`The magnification of the original display gives the user
`immediate visual feedback that the presence of the fingertip
`has been detected at a particular,first position, and a larger
`area is displayed for the fingertip to move to and designate
`a character at a second position.
`It should be realized that if the controller 18 is capable of
`unambiguously identifying a selected symbol(e.g., a symbol
`displayed in isolation), then the magnifying and subsequent
`steps need not necessarily be performed. In this case Block
`404 is modified accordingly to determine if the location of
`the fingertip can be unambiguously correlated with one
`displayed symbol. If yes, control can pass immediately to
`Block 414 to report the user-designated symbol. However,it
`may be desirable when the symbols are displayed in an
`unmagnified form to always employ the zoom function,
`
`12
`
`12
`
`
`
`6,073,036
`
`7
`is unambiguously
`even when only one selected symbol
`identified, in order to avoid the possibility that the user has
`indicated the wrong symbol.
`It is assumed that the user now repositions or slides the
`fingertip in the magnified area to a second position, until the
`desired character to be input into the mobile station 10 is
`reached. The movementof the fingertip to a second position
`is detected in step 408 wherein the position of the fingertip
`is detected within one of the 100 (10 by 10 matrix) touch
`sensitive elements of display 20. If the location of the
`fingertip is no longer changing, as sensed in the 10 by 10
`matrix,it is assumedthe fingertip is no longer moving over
`the display 20 and that the user has selected the desired
`character. Step 410 identifies a change in element position
`by comparing the previous valueofthe position of the tactile
`input with the latest value of the position of the tactile input.
`If the two positions are not the same, the method returns to
`step 408 to detect the new element on the display being
`activated. Conversely if the position has not changed, step
`410 directs the display element containing the last tactile
`input detection to be stored in memory 24, as shownin step
`418. It is understood that the search for a change in fingertip
`position, as reported by the display 20, will continue for as
`long as there is a variation between successive fingertip
`locations determinations, as indicated by the loop formed by
`step 412, 408, 410 and 418. This provision assures that if the
`user momentarily stops moving the fingertip on the display,
`and then continues moving again withoutlifting the fingertip
`off the display, a false reading is avoided.
`In this embodiment of the invention, as soon as the
`location of the tactile input on the display 20 stays constant
`for some predetermined period of time (step 412),
`for
`example, 2 seconds, it can be assumed that the user has
`completed the selection of the desired symbol and that the
`last location (stored in memory 24) indicates the desired
`symbol. After a short delay, of, for example, one to 500
`milliseconds, as shown in optional step 420,
`to provide
`“de-bouncing” of fingertip motion,
`the element position
`where the tactile input was detected last is reported, as
`shownin step 414, to a routine that identifies the manually
`designated symbol based on the last reported fingertip
`position and the current mapping betweenthe display screen
`coordinates and the displayed symbols.
`In step 412, a sequence oftactile input reports from step
`410 are compared for a time duration of, for example, 2
`seconds. The final position reported from step 412 to step
`414 is made only after a sequence of successive fingertip
`location reports are equal in value. As was indicated above,
`the de-bounce delay step 420 is optional.
`Next,
`in optional step 416,
`the symbol and associated
`display element(s), where the user input was reported in step
`414, can be made to change inattribute, as for example, to
`flash repeatedly, or change background,to inform the user of
`the selection of that symbol. The user, having visual feed-
`back as to the symbol recognized by controller 18 as an input
`from touch sensitive display 20, can then proceed to desig-
`nate the next symbol, such as a character, legend, icon or
`other screen displayed detail. Typically, the display willfirst
`be automatically returned to the unmagnified format (as in
`FIG. 3).
`FIG. 4Billustrates a further embodimentofthis invention,
`wherein the Block 412 is modified to be a Block 412a. In
`this embodiment the controller 18 detects that the user’s
`
`fingertip has been lifted from the touch sensitive display 20.
`The detection of the user lifting his or her finger is inter-
`preted as the user selecting the symbol at the corresponding
`display position.
`
`10
`
`20
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`It can be appreciated that this invention facilitates the
`selection of numbers whenoriginating a phonecall, or when
`it is desired to store a phone numberin the mobilestation 10.
`This invention is particularly advantageous when the user
`wishes to compose (or respond to) an alphanumeric text
`message, such as one to be sent from the mobile station 10
`using a Short Message Service (SMS) teleservice.
`Referring now to FIG. 1B, in a second embodiment the
`mobile station 10 is provided with a conventional LCD
`display 20, the keypad 22 is eliminated (optionally), and a
`touchpad 23 is provided. The touchpad 23 can be a known
`type of device, used with some portable computers, that
`outputs X and Y coordinate information in responseto tactile
`input from a user, either by fingertip or stylus. The touchpad
`23 can be a separate device that is interfaced with the mobile
`station 10 through a cable and/or connector, or may be
`integral with the mobile station 10 and located on a cover of
`the phone, on a pull-out or slide-out planar member, or on
`a flip-down portion of the phone’s cover or other structural
`member.
`
`In a further embodimentof the invention the display 20 of
`FIG. 1B may be a so-called “virtual display” or “virtual
`screen”, such as a headset-mounted type of monitor that
`employs a scanned array of LEDSto generate a projection
`display. One such device is available from Reflection
`Technology, Inc., and is referred to as a “Private Eye”.
`Another type of solid state miniaturized display uses a two
`dimensional array of GaAs light emitting pn junctions, and
`has dimensions of just 7 mm by 4 mm.A refractive optical
`system magnifies the array by 15 so that, when held to the
`eye, a virtual image is displayed that is equivalent to a
`conventional 45.7 cm diagonal display viewedat a distance
`of 1.5 m. With these types of displays it may be desirable to
`eliminate the conventional telephone keypad 22 altogether,
`and to use the touch sensitive pad 23 as the user input device.
`In the touchpad embodiment of FIG. 1B the controller
`receives the X-Y coordinate information from the touchpad
`23, instead of from the display 20, and in all other respects
`may operate as described above in FIGS. 3, 4A and 4B.
`Reference is now made to FIGS. 5A-5Ffor illustrating a
`further embodimentof this invention, specifically an alpha-
`numeric input method using the dynamic touchscreen 20 as
`in FIG. 1A or the touchpad 23 as in FIG. 1B. Whenthe user
`desires to enter alphanumeric characters