(12) United States Patent
`Beaton et al.
`
`I lllll llllllll Ill lllll lllll lllll lllll lllll 111111111111111111111111111111111
`US006310610Bl
`US 6,310,610 Bl
`*Oct. 30, 2001
`
`(10) Patent No.:
`(45) Date of Patent:
`
`(54)
`
`INTELLIGENT TOUCH DISPLAY
`
`FOREIGN PATENT DOCUMENTS
`
`(75)
`
`Inventors: Brian Finlay Beaton, Orleans; Colin
`Donald Smith, Ottawa; Francois
`Blouin; Guillaume Comeau, both of
`Hull; Arthur Julian Patterson
`Craddock, Greely, all of (CA)
`
`(73) Assignee: Nortel Networks Limited (CA)
`
`( *) Notice:
`
`This patent issued on a continued pros(cid:173)
`ecution application filed under 37 CFR
`1.53( d), and is subject to the twenty year
`patent term provisions of 35 U.S.C.
`154(a)(2).
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis(cid:173)
`claimer.
`
`(21) Appl. No.: 08/985,264
`
`(22) Filed:
`
`Dec. 4, 1997
`
`(51)
`
`Int. Cl.7 ....................................................... G09G 5/00
`
`(52) U.S. Cl. ............................................. 345/173; 345/358
`(58) Field of Search ..................................... 345/173, 174,
`345/358-359, 326, 179, 180, 433, 352-353,
`356, 354, 357, 355, 975, 348, 146, 339;
`178/18.01, 18.03, 18.04
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,954,967
`5,297,253
`5,512,707
`5,563,996
`5,570,109
`
`9/1990 Takahashi ............................ 345/173
`3/1994 Meisel .................................. 345/357
`4/1996 Ohshima ................................ 84/658
`10/1996 Tchao ................................... 207/521
`10/1996 Jenson .................................. 345/146
`
`0 156 593
`0 565 253
`0 605 945
`0 609 021
`0 684 543
`2 301 217
`4-060715
`4-165522
`4-199416
`4-237324
`4-278627
`5-189150
`8-307954
`9-231006
`WO 92 16900
`WO 94 28505
`
`10/1985 (EP) .
`10/1993 (EP) .
`7 /1994 (EP) .
`8/1994 (EP) .
`11/1995 (EP) .
`11/1996 (GB) .
`2/1992 (JP) .
`6/1992 (JP) .
`7 /1992 (JP) .
`8/1992 (JP) .
`10/1992 (JP) .
`7 /1993 (JP) .
`11/1996 (JP) .
`9 /1997 (JP) .
`10/1992 (WO) .
`12/1994 (WO) .
`
`OTHER PUBLICATIONS
`
`Kurtz, Max "Handbook of Applied Mathematics for Engi(cid:173)
`neers and Scientists" pp. 8.24, 12.65-12.66, 1991.*
`IBM Technical Disclosure Bulletin, "Scroll Control Box,"
`vol. 36, No. 4, Apr. 1993, pp. 399-403.
`IBM Technical Disclosure Bulletin, "Data Interpretation
`Techniques for Pen-based Computer," vol. 38, No. 9, p. 461,
`Sep. 1995.
`
`Primary Examiner-Richard Hjerpe
`Assistant Examiner-Francis Nguyen
`(74) Attorney, Agent, or Firm-Finnegan, Henderson,
`Farabow, Garrett & Dunner, L.L.P.
`
`(57)
`
`ABSTRACT
`
`The system and method consistent with the present inven(cid:173)
`tion provides a touch-responsive graphical user interface for
`electronic devices. The graphical user interface determines a
`pointer size of the object making contact with a display and
`activates a function corresponding to the pointer size. The
`graphical user interface may invoke a wide array of func(cid:173)
`tions such as a navigation tool, draw function, an erase
`function, or a drag function. The graphical user interface of
`the present invention may be especially useful in portable
`electronic devices with small displays.
`
`(List continued on next page.)
`
`31 Claims, 15 Drawing Sheets
`
`IT!.Q
`
`DETERMINE INDIVIDUAL POlNTS
`OF CONTACT MADE BY OBJECT
`
`j
`
`J]2Q_
`
`COMPUTE CENTROID OF
`DETERMINED POINTS OF CONTACT
`
`j
`
`1330
`
`I
`i
`
`COMPUTE STANDARD
`DEVIATION OF CENTROID
`
`j
`
`1340
`
`DETERMINE POINTER OF THE
`OBJECT MAKING CONTACT
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 1 of 24
`
`

`

`US 6,310,610 Bl
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5,590,257
`5,592,195
`5,600,765
`5,745,116
`5,745,716
`5,748,926
`5,760,773
`
`12/1996 Forcier ................................. 707/530
`1/1997 Misono et al. ....................... 345/146
`2/1997 Ando et al. .......................... 345/433
`4/1998 Pisutha-Arnond ................... 345/358
`4/1998 Tchao et al. ......................... 345/350
`5/1998 Fukuda et al. ....................... 345/326
`6/1998 Berman et al. ...................... 345/347
`
`7/1998
`5,777,605
`5,790,106 *
`8/1998
`8/1998
`5,796,406
`9/1998
`5,808,605
`9/1998
`5,809,257
`9/1998
`5,812,134
`5,825,352
`10/1998
`* cited by examiner
`
`Yoshinobu et al. .................. 345/173
`Hirano et al. ........................ 345/173
`Shigematsu et al. ................ 345/358
`Shieh ................................... 345/173
`Moran et al. ........................ 345/358
`Pooser et al. ........................ 345/356
`Bisset et al. ......................... 345/173
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 2 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 1 of 15
`
`US 6,310,610 Bl
`
`r l
`CJ
`8 s @
`@ @ 8
`§ 8 8
`000
`
`FIG. 1
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 3 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 2of15
`
`US 6,310,610 Bl
`
`0
`
`0
`¢>
`
`~
`
`"'
`
`~
`
`(
`
`..
`
`0
`
`¢
`
`FIG. 2
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 4 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 3of15
`
`US 6,310,610 Bl
`
`c ..-
`
`("")
`
`0
`~
`('I')
`
`0
`('I')
`('I')
`
`D
`
`)0 00
`0 0000 ~
`0000
`
`"(
`C")
`•
`(.!>
`Li:
`
`0
`..,......
`('I')
`w
`z
`0
`I
`a_
`w
`.....J w
`I-
`w
`.....J
`co
`0
`~
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 5 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 4of15
`
`US 6,310,610 Bl
`
`420 ;
`
`A NTENNA
`
`410
`
`COMMUNICATIONS MODULE
`
`424
`
`VEGA
`
`423
`;
`424
`
`>----- ,__ AID
`-
`,_ D/A
`-
`
`-
`
`-
`
`-
`
`BOCK
`
`-
`
`-
`
`D/A
`
`-
`
`-
`
`AID
`-
`
`\. 425
`
`421
`;
`GSM
`RADIO
`
`427
`!
`AUDIO
`TRANSDUCERS
`--0---SPEAKER
`428
`MICROPHONE
`,-
`429
`
`ASYNC COMMUNICATION
`(UART)
`
`,...J 430
`
`,,..;460
`ANALOG
`CONTROLLER
`
`,,..;450
`SLIDING
`KEYPAD
`
`-
`-
`-
`
`MEMORY
`FLASH
`ROM
`
`'----" 440
`
`i...;444
`
`STATIC
`RAM
`
`.. ...442
`
`FEATURE
`PROCESSOR
`
`-
`
`n
`
`,,.-1490
`
`,,...-' 480
`
`SWITCHING
`POWER
`SUPPLY
`
`BATTERY
`PACK
`
`FIG. 4
`
`TOUCH SCREEN
`PANEL
`
`.. ...474
`
`i..-"-'472
`
`LCD
`
`l
`470
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 6 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 5of15
`
`US 6,310,610 Bl
`
`MEMORY 440
`
`STATIC RAM 442
`
`FLASH ROM 444
`
`PROGRAM
`510
`
`TOUCH
`SCREEN
`PROGRAM
`520
`
`NAVIGATION
`PROGRAM
`530
`
`DRAWING
`PROGRAM
`540
`
`FIG. 5
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 7 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 6 of 15
`
`US 6,310,610 Bl
`
`430
`,_;
`
`460
`,_;
`
`474
`~
`
`FEATURE
`PROCESSOR
`
`ANALOG
`CONTROLLER
`
`TOUCH
`SCREEN PANEL
`
`FIG. 6
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 8 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 7of15
`
`US 6,310,610 Bl
`
`<( w
`er
`<(
`z
`0
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 9 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 8 of 15
`
`US 6,310,610 Bl
`
`FIG. 8
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 10 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 9of15
`
`US 6,310,610 Bl
`
`UP
`RIGHT
`LEFT
`
`DOWN
`
`WITHIN-PAGE NAVIGATION CIRCLE
`
`FIG. 9A
`
`I t ......__
`
`PREVIOUS PAGE
`
`I 1---- NEXT PAGE
`FIG. 98
`
`ADDITIONAL CONTROLS FOR
`FORWARD TO NEXT DOCUMENT,
`BACK,HOME,ETC.
`
`(
`@]
`
`FIG. 9C
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 11 of 24
`
`

`

`rs Center
`
`. ,. . ._ .. · .. _;_:;:; ;_· ·::··;·:~::; U.~-:~·.: ··.:·: ·:·.;;: ..
`
`. :·'/\:1$fa!.¢~P<?·al t¢f.YXi/
`
`Sept. 4
`
`~
`
`· ,~·~!~~,Rf~~m1
`
`. .... • ... ······' .... • ........ ,. '•' - " ..... -
`
`" ..... ·- . ~ - ···-' .......... .
`
`;:~·
`
`:!:!
`
`~h.teLcontdW
`
`fi1tf 1I
`)esig-~~ ..... ,,.r,
`,·\'.fim
`
`:~: '.·
`
`Sign Desig~
`
`~)~: .. IN
`
`FIG. 10A
`
`FIG. 108
`
`FIG. 10C
`
`d •
`\JJ.
`•
`~
`~ ......
`~ = ......
`
`0
`I")
`!"""
`~
`~=
`N c c
`'"""'
`
`'Jl =(cid:173)~
`~ .....
`'"""' c
`0 .....,
`'"""' Ul
`
`e
`
`rJ'J.
`O'I
`~
`lo-"
`
`Q °' lo-"
`
`Q
`~
`lo-"
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 12 of 24
`
`

`

`~
`
`~
`
`VIEWING WINDOWS DOES NOT MOVE (ACTIVATION AREA)
`VIEWING WINDOW MOVES SLOWLY
`,, VIE\N!NG WINDOW MOVES QUICKLY
`
`FIG. 11A
`
`NEXT PAGE IC~ VIEWING WINDOW MOVES QUICKLY TO THE PREVIOUS P
`~- VIEWING WINDOW MOVES SLOWLY TO THE PREVIOUS PA
`
`d •
`\JJ.
`•
`~
`~ ......
`~ = ......
`
`0
`I")
`!""'"
`~
`~=
`N c c
`'"""'
`
`rF.J. =(cid:173)~
`~ .....
`'"""'
`'"""' 0 .....,
`'"""' Ul
`
`Ys+.1··
`.. '5)·.· VIEWING WINDOW MOVES SLOWLY TO THE NEXT PAGE
`I~ ~VIEWING WINDOW MOVES QUICKLY TO THE NEXT PAGE
`FIG. 11 B
`
`PREVIOUS PAGE ICON
`
`e
`
`rJ'J.
`O'I
`~
`lo-"
`-..= O'I
`
`lo-"
`Q
`~
`lo-"
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 13 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 12 of 15
`
`US 6,310,610 Bl
`
`TOUCH POINT DISTRIBUTION
`
`FIG. 12
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 14 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 13 of 15
`
`US 6,310,610 Bl
`
`1310
`
`DETERMINE INDIVIDUAL POINTS
`OF CONTACT MADE BY OBJECT
`
`'
`1320
`
`COMPUTE CENTROID OF
`DETERMINED POINTS OF CONTACT
`
`1330
`
`COMPUTE STANDARD
`DEVIATION OF CENTROID
`
`I
`
`'
`1340
`
`DETERMINE POINTER OF THE
`OBJECT MAKING CONTACT
`
`FIG. 13 A
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 15 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 14 of 15
`
`US 6,310,610 Bl
`
`1310A
`
`DETECTING (AT LEAST
`150 pts/sec) AND
`DETERMINING POINTS
`OF CONTACT BY
`OBJECT
`
`1320A
`
`DETERMINE AMOUNT
`OF PRESSURE
`IMPOSED ON
`VIEWABLE AREA BY
`OBJECT
`
`1330A
`
`DETERMINE AMOUNT
`OF TIME CONTACT IS
`MADE BY OBJECT ON
`VIEWABLE AREA
`
`1340A
`
`COMPUTE POINTER
`SIZE BASED ON
`PRESSURE AND TIME
`CHARACTERISTICS OF
`THE OBJECT
`
`FINGER
`
`POINTER
`
`1360A
`
`1370A
`
`1380A
`
`ACTIVATE
`NAVIGATION TOOL
`
`ACTIVATE ERASE
`FUNCTION
`
`ACTIVATE DRAWING
`FUNCTION
`
`FIG. 13B
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 16 of 24
`
`

`

`U.S. Patent
`
`Oct. 30, 2001
`
`Sheet 15 of 15
`
`US 6,310,610 Bl
`
`PRESSURE
`
`PRESSURE
`
`PEN
`
`TIME
`
`FIG. 14A
`
`- - FINGER
`
`TIME
`
`FIG. 14B
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 17 of 24
`
`

`

`US 6,310,610 Bl
`
`1
`INTELLIGENT TOUCH DISPLAY
`
`RELATED APPLICATIONS
`
`This application is related to U.S. patent application Ser.
`No. 08/985,265, entitled NAVIGATIONAL TOOL FOR
`GRAPHICAL USER INTERFACE; and U.S. patent appli(cid:173)
`cation Ser. No. 08/985,261, entitled CONTEXTUAL GES(cid:173)
`TURE INTERFACE, both of which are filed concurrently
`herewith, and both of which are hereby incorporated by
`reference.
`
`BACKGROUND OF THE INVENTION
`
`2
`into the hierarchy before reaching the desired function. This
`is time consuming and renders the GUI cumbersome and
`ineffective.
`Therefore, it is desirable to provide navigation tools that
`5 allow small-size devices while maximizing the use of avail(cid:173)
`able screen real estate.
`It is also desirable to provide tools to navigate within a
`document at any direction at varying speeds.
`It is further desirable to provide navigation tools that can
`10 be activated without requiring specific electronic devices.
`In addition, it is further desirable to provide an improved
`GUI that simplifies GUI by recognizing various character(cid:173)
`istics of the touch input.
`
`SUMMARY OF THE INVENTION
`Systems and methods consistent with the present inven(cid:173)
`tion provide a graphical touch-responsive user interface for
`display devices.
`Specifically, a method consistent with this invention of
`providing a touch-responsive user interface comprises sev(cid:173)
`eral steps. Initially, the apparatus detects an object making
`contact with a physical viewing area, and determines a
`pointer size of the object. Thereafter, the system activates a
`function corresponding to the pointer size.
`A system consistent for this invention for providing a
`touch-responsive user interface comprises detecting means,
`determining means, and activating means. The detecting
`means detects an object making contact with a physical
`30 viewing area. The determining means determines a pointer
`size of the object, and the activating means activates a
`function corresponding to the pointer size.
`
`The present invention relates generally to graphical user
`interfaces (GUI), and more particularly to a touch- 15
`responsive user interface for graphical user interfaces.
`Until relatively recently, software-based documents have
`been primarily viewed and manipulated on desktop or laptop
`computers with relatively large displays, typically 640x480
`pixels or larger. These displays are often large enough to 20
`display a full page of standard size page or at least a
`significant portion of the page. Hence, on-screen graphical
`menus and controls displayed in window of an application
`did not greatly reduce the display area for the underlying
`document. Computers also have peripheral devices such as 25
`a keyboard or a mouse to control the display of content
`information. Thus, viewing and navigating around a single(cid:173)
`page or multi-page document have not posed much diffi(cid:173)
`culty.
`Due to increasing focus on compactness of electronic
`devices, however, the displays especially in portable elec(cid:173)
`tronic devices are becoming smaller and smaller. Popular
`electronic devices with a smaller display area include elec(cid:173)
`tronic organizers, PDA's (personal digital assistants), and
`graphical display-based telephones. Also available today are 35
`communicators that facilitate various types of communica(cid:173)
`tion such as voice, faxes, SMS (Short Messaging Services)
`messages, e-mail, and Internet-related applications. These
`products can likewise only contain a small display area.
`To enable users to navigate around a full page of content
`information, these devices typically provide hard-keys for
`arrows as shown in FIG. 1. The hard-keys, however, not only
`increase the size but also add to the cost of the devices. Also,
`hard-keys generally provide limited options for direction of
`movement, e.g., vertical or horizontal. They generally do not
`provide the freedom to move in any direction.
`Some displays of these devices also require a separate
`stylus having peripheral technology that requires transmis(cid:173)
`sion of electromagnetic pulses or light to the display. These 50
`devices often require additional controllers such as buttons
`on the body or the tip of the stylus for activation.
`Furthermore, these styli require a power source, either
`through wire or battery, and their compatibility is generally
`limited to a specific device.
`As shown in FIG. 2, other devices substitute hard-keys
`with graphical onscreen arrows or scroll bars that are typi(cid:173)
`cally used in full-size computer displays. The on-screen
`scroll bars, however, occupy valuable screen real estate and
`compound the limitations of small displays. Similar to the
`hard-keys, the onscreen arrows also generally restrict the
`navigational movement to horizontal or vertical direction.
`In other forms of on-screen GUis, e.g., pop-up menus,
`also take up valuable screen space, further reducing the
`available display area for content information. Additionally, 65
`on-screen pop-up menus typically provide available func(cid:173)
`tions in multiple layers, thus requiring a user to move deeply
`
`40
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`The accompanying drawings, which are incorporated in
`and constitute a part of this specification, illustrate the
`invention and together with the description, serve to explain
`the principles of the invention.
`In the drawings,
`FIG. 1 shows conventional hard-key arrows for naviga(cid:173)
`tion control;
`FIG. 2 shows conventional on-screen graphical naviga(cid:173)
`tion tool;
`FIGS. 3A-3B are diagrams of an exemplary mobile
`45 telephone consistent with the principles of the present inven-
`ti on;
`FIG. 4 is a block diagram showing the elements of the
`mobile telephone of FIG. 3A;
`FIG. 5 is a block diagram showing the components of the
`memory of FIG. 4;
`FIG. 6 is a block diagram of touch screen functionalities;
`FIGS. 7A-7B show an exemplary inactive and active
`graphical navigation tool, respectively;
`FIG. 8 is a sample screen showing an active navigation
`tool;
`FIGS. 9A-9C show exemplary features of the navigation
`tool;
`FIGS. lOA-lOC are sample screens showing the naviga-
`60 tion tool performing various navigation functions;
`FIGS. llA-llB show exemplary features of the naviga(cid:173)
`tion tool relating to speed of navigation;
`FIG. 12 is a diagram illustrating a touch point distribution;
`and
`FIG. 13 is a flowchart illustrating the process of deter(cid:173)
`mining the size of the object making contact with the
`viewing area.
`
`55
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 18 of 24
`
`

`

`US 6,310,610 Bl
`
`3
`FIG. 13Ais a flowchart illustrating an alternate process of
`determining the size of the object making contact with the
`viewing area;
`FIGS. 14A and 14B are graphs showing the touch char(cid:173)
`acteristic of a pen and a finger, respectively.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`Reference will now be made in detail to the present
`preferred embodiment of the invention, an example of which
`is illustrated in the accompanying drawings. Where
`appropriate, the same reference numerals refer to the same
`or similar elements. The appended claims define the scope of
`the invention; the following description does not limit that
`scope.
`The graphical navigation tool of the present invention
`may be implemented in a wide range of electronic devices
`mentioned above such as electronic organizers, PDA's, and
`graphical display-based telephones. Although the need to
`maximize the use of screen real estate is most critical in
`portable electronic devices with small displays, the present
`invention can also be implemented in full-size computers or
`electronic devices. For purposes of illustration, however, the
`present invention will be explained in detail in a mobile
`telephone environment.
`Specifically, FIG. 3A shows a mobile telephone 310 and
`FIG. 3B shows an exemplary wireline telephone preferably
`having the graphical navigation tool consistent with the
`present invention. Mobile telephone 310 includes main
`housing 210, antenna 320, keypad 330, and display 340.
`FIG. 4 shows the hardware elements in mobile telephone
`310 including antenna 410, communications module 420,
`feature processor 430, memory 440, sliding keypad 450,
`analog controller 460, display module 470, battery pack 480,
`and switching power supply 490.
`Antenna 410 transmits and receives radio frequency infor(cid:173)
`mation for mobile telephone 310. Antenna 410 preferably
`comprises a planar inverted F antenna (PIFA)-type or a short
`stub (2 to 4 cm) custom helix antenna. Antenna 410 com(cid:173)
`municates over a GSM (Global System for Mobile
`Communications) switching fabric using a conventional
`voice B-channel, data B-channel, or GSM signaling channel
`connection.
`Communications module 420 connects to antenna 410 45
`and provides the GSM radio, baseband, and audio function(cid:173)
`ality for mobile telephone 310. Communications module
`420 includes GSM radio 421, VEGA 423, BOCK 425, and
`audio transducers 427.
`GSM radio 421 converts the radio frequency information 50
`to/from the antenna into analog baseband information for
`presentation to VEGA423. VEGA423 is preferably a Texas
`Instruments VEGA device, containing analog-to-digital
`(AID)/digital-to-analog (DIA) conversion units 424. VEGA
`423 converts the analog baseband information from GSM
`radio 421 to digital information for presentation to BOCK
`425.
`BOCK 425 is preferably a Texas Instruments BOCK
`device containing a conventional ARM microprocessor and
`a conventional LEAD DSP device. BOCK 425 performs
`GSM baseband processing for generating digital audio sig(cid:173)
`nals and supporting GSM protocols. BOCK 425 supplies the
`digital audio signals to VEGA 423 for digital-to-analog
`conversion. VEGA 423 applies the analog audio signals to
`audio transducers 427. Audio transducers 427 include
`speaker 428 and microphone 429 to facilitate audio com(cid:173)
`munication by the user.
`
`30
`
`4
`Feature processor 430 provides GUI features and a Java
`Virtual Machine (JVM). Feature processor 430 communi(cid:173)
`cates with BOCK 425 using high level messaging over an
`asynchronous (UART) data link. Feature processor 430
`5 contains additional system circuitry, such as a liquid crystal
`display (LCD) controller, timers, UART and bus interfaces,
`and real time clock and system clock generators (not
`shown).
`Memory 440 stores data and program code used by
`10 feature processor 430. Memory 440 includes static RAM
`442 and flash ROM 444. Static RAM 442 is a volatile
`memory that stores data and other information used by
`feature processor 430. Flash ROM 444, on the other hand,
`is a non-volatile memory that stores the program code
`15 executed by feature processor 430.
`Sliding keypad 450 enables the user to dial a telephone
`number, access remote databases, and manipulate the GUI
`features. Sliding keypad 450 preferably includes a mylar
`resistive key matrix that generates analog resistive voltage in
`20 response to actions by the user. Sliding keypad 450 prefer(cid:173)
`ably connects to main housing 210 (FIG. 3A) of mobile
`telephone 310 through two mechanical "push pin" -type
`contacts (FIG. 4).
`Analog controller 460 is preferably a Phillips UCBL 100
`25 device that acts as an interface between feature processor
`430 and sliding keypad 450. Analog controller 460 converts
`the analog resistive voltage from sliding keypad 450 to
`digital signals for presentation to feature processor 430.
`Display module 470 preferably includes a 160x320 pixel
`LCD 472 with an analog touch screen panel 474 and an
`electroluminescent backlight. LCD 472 operates in conjunc(cid:173)
`tion with feature processor 430 to display the GUI features.
`Analog controller 460 scans touch screen overlay 474 while
`35 feature processor 430 refreshes LCD 472.
`Battery pack 480 is preferably a single lithium-ion battery
`with active protection circuitry. Switching power supply 490
`ensures highly efficient use of the lithium-ion battery power
`by converting the voltage of the lithium-ion battery into
`40 stable voltages used by the other hardware elements of
`mobile telephone 310.
`FIG. 5 is a block diagram illustrating the components of
`memory 440. Static RAM 442 stores data and other infor(cid:173)
`mation used by feature processor 430. Flash ROM 444
`contains various programs including a program 510, a touch
`screen program 520, a navigation program 530, and a
`drawing program 540. Program 520, preferably written in
`languages such as Java, C, or C++ for Macintosh, is a main
`program overseeing the operation of mobile telephone 310.
`Touch screen program 520 facilitates processing of touch
`input on touch screen panel 474 using a typical touch input
`algorithm. Navigation program 530 handles navigation of
`the content information display. Drawing program 540 is a
`graphical drawing package. Programs 520, 530, and 540
`55 may be one of any commercially available packages or a
`user-defined feature program or macro.
`The present invention provides various features through
`tactile GUI. Initially, LCD 472 displays various GUI fea(cid:173)
`tures. Referring to FIG. 6, a user touches touch screen panel
`60 474 to provide user input, for example, to navigate around
`a document or invoke a desired function. Analog controller
`460 scans touch screen panel 474 and reads the correspond(cid:173)
`ing analog voltage of touch screen panel 474. Analog
`controller 460 then converts the analog values into corre-
`65 sponding digital values representing the Cartesian
`coordinates, which are transmitted to feature processor 430
`for processing. The resolution of the touch input depends on
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 19 of 24
`
`

`

`US 6,310,610 Bl
`
`6
`5
`touch on the next page icon causes navigation program 530
`the ability of analog controller 460 to discern among mul(cid:173)
`to continue scrolling through succeeding pages of the under(cid:173)
`tiple levels of analog values, generally defined in bits.
`lying document. The previous page icon may embody simi(cid:173)
`FIGS. 7 A-7B show an exemplary graphical navigation
`lar characteristics.
`tool preferably used to navigate around documents that are
`The user may also control the speed of the navigation. As
`too large to view within a single screen of a physical display 5
`shown in FIG. llA, the speed of the navigation accelerates
`(hereinafter referred as "viewing area"). The navigation tool
`as the user touch moves from the center of the circle toward
`may be used to view any kind of document including faxes,
`the circumference of the circle, i.e., tip of the arrow. Hence,
`Web pages, or e-mail. In one embodiment consistent with
`the viewing window moves slowly when the user touches
`the present invention, an inactive navigation tool is dis(cid:173)
`10 the blunt end of the arrow located at the center of the circle
`played and accessible to the user at all times (FIG. 7A). The
`while the speed accelerates as the user moves the finger
`user may activate the navigation tool by touching and
`towards the tip of the arrow. The speed of navigation,
`holding the center of the navigation tool for a predetermined
`therefore, is determined by the distance of the touch relative
`time period, for example, one to two seconds (FIG. 7B). An
`to the center of the circle. Likewise, similar principles apply
`activated navigation tool is preferably transparent to avoid
`to previous or next page/document icons where a touch
`hindering the display of content information in the viewing 15
`closer to the outer edge of the previous or next page/
`area as shown in FIG. 8. Alternatively, the navigation star
`document icons accelerates navigation through the docu(cid:173)
`may change colors or other features of its appearance to
`ment as shown in FIG. llB.
`indicate its active status. A solid line image, for example,
`may be used in greyscale displays that do not support
`Although the exemplary transparent tool discussed above
`transparency. The present invention may be designed such 20
`is for navigation, transparent control tools may be imple(cid:173)
`that feature processor 430 ignores any touch input on the
`mented for a variety of functions. A transparent tool may, for
`navigation tool unless the navigation tool has been activated.
`example, be used for a Web browser application where the
`Instead, the touch input may be interpreted as input to access
`controls may be used for appropriate functions such as
`control buttons in the underlying document, write on the
`moving forwards or backwards through different Web pages
`underlying document, or invoke other functions related to 25
`or returning to home page. One skilled in the art may easily
`the underlying document. This will prevent against uninten(cid:173)
`vary the design or the functionality of the graphical navi-
`tional navigation in the viewing window in case the user
`gation tools described above without departing from the
`inadvertently touches touch screen panel 474. In an alter(cid:173)
`scope of the present invention.
`native embodiment, the present invention may accept stylus
`In an exemplary embodiment of a navigation tool
`input to access the underlying document while a finger or 30
`described above, a finger touch invokes navigational func(cid:173)
`non-electromagnetic touch on any part of the navigation tool
`tions based on the feature selected and the location of the
`invokes the navigation function.
`user touch. Alternatively, other objects making contact with
`touch screen panel 474 may invoke other tools or functions.
`Referring to FIGS. 9A-9C, once the navigation tool is
`activated, the user may navigate through the document by
`A pointy stylus touch, for example, may invoke a menu with
`selecting the graphical arrows, e.g., up, right, left, and down 35
`cardinal points representing multiple line widths, colors, or
`arrows (FIG. 9A), or graphical page icons, e.g., previous or
`patterns.
`next page (FIG. 9B). One skilled in the art may vary the type
`In another embodiment consistent with the present
`and number of graphical tools significantly. For example, the
`invention, tools or application programs may be stored in
`navigation tool may provide graphical representations for
`flash ROM 444 to provide related interfaces to the user. The
`forward, next document, back, or home functions (FIG. 9C). 40
`use of a finger may, for example, invoke tools or dialogues
`FIGS. lOA-lOC show exemplary screen displays while
`that are finger-touchable and large whereas the use of a sharp
`stylus may invoke a modified GUI with smaller touch
`the user is touching the navigation tool. Upon touching the
`targets. In a yet another embodiment, in a document viewing
`right arrow of the navigation tool, for example, the right
`arrow is highlighted and navigation program 530 moves the 45
`application normally navigable by a finger touch, use of a
`sharp stylus may automatically invoke a document annota-
`display to the right (FIG. lOA). Similarly, touching the down
`arrow moves the display down (FIG. lOB). Although the
`tion application for marking up the underlying document.
`four arrows are presented to guide the users, navigation
`As described above, the touch-responsive GUI of the
`program 530 supports navigational movement at any direc(cid:173)
`present invention are facilitated though various components
`tion.
`50 including touch screen panel 474, analog controller 460, and
`feature processor 430. Specifically, analog controller 460
`If the user touches an area of the navigation tool equi(cid:173)
`scans touch screen panel 474 to read the corresponding
`distant between the up and right arrows, for example,
`analog voltage of touch screen panel 474 activated by a user
`navigation program 530 will move the display towards the
`touch. Analog controller 460 then converts the analog values
`upper-right portion of the underlying document at a
`into a digital value representing the Cartesian coordinates,
`45-degree angle. Touching the arrows or any area in 55
`which is transmitted to feature processor 430 for processing
`between, moves the display in the selected direction until
`navigation program 530 reaches the edge of the page.
`according to the functionalities of the present invention.
`When a user touches touch screen panel 474, program 510
`Touching the next page icon moves the viewing window
`initiates touch screen program 520 to determine the pointer
`to the next page of the underlying document (FIG. lOC). If
`a particular document does not have a page corresponding to 60 size of the object making contact with touch screen panel
`474 based on a touch point distribution or pointer size of the
`a previous or next page icon, navigation program 530 will
`touch input. As shown in FIG. 12, touch screen program 520
`not display the respective previous or next page icons. This
`can, for example, determine whether the pointer size of the
`would apply to one-page documents, or when the user is at
`object is a finger or a sharp object.
`the beginning or end of a multi-page document. In one
`FIG. 13 is a flowchart illustrating the process of deter(cid:173)
`embodiment consistent with the present invention, a 65
`mining the size of the object making contact with the
`momentary touch of the next page icon causes navigation
`program 530 to jump to the next page while a continuous
`viewing area. Touch point program 520 first determines the
`
`Microsoft Ex. 1024
`Microsoft v. Philips - IPR2018-00025
`Page 20 of 24
`
`

`

`US 6,310,610 Bl
`
`7
`individual points of contact made by the object (step 1310).
`It computes a centroid, or other average point, of the
`determined points of contact (step 1320). Touch program
`520 then computes a standard deviation of the centroid as
`well as the variance (step 1330), and determines the pointer 5
`size based on the centroid and the standard deviation (step
`1340). These computations are preferably performed on a
`real-time basis to provide immediate system response to the
`touch input. In order to achieve optimum results and
`accuracy, analog touch controller 460 preferably generates 10
`150 points per second or more. Touch program 520 may also
`use the amount of pressure imposed on touch screen panel
`474 as a function of time to determine the size of object. As
`shown in FIG. 14A, for example, if the amount of pressure
`increases or decreases sharply at a particular instant in time, 15
`touch point program 520 may determine that the touch
`corresponds to a pen. A finger touch, on the other hand,
`resu