`
`Ex. GOOG 1101
`
`EX. GOOG 1101
`
`
`
`
`
`

`

`United States Patent [19J
`Beaton et al.
`
`111111
`
`1111111111111111111111111111111111111111111111111111111111111
`US006037937 A
`[11] Patent Number:
`[45] Date of Patent:
`
`6,037,937
`*Mar.14,2000
`
`[54] NAVIGATION TOOL FOR GRAPHICAL
`USER INTERFACE
`
`[75]
`
`Inventors: Brian Finlay Beaton, Orleans; Colin
`Donald Smith, Ottawa; Bruce Dale
`StaUde, Kanata, all of Canada
`
`5,745,716
`5,748,926
`5,760,773
`5,796,406
`5,808,605
`5,809,267
`5,812,134
`
`4/1998 Tchao et a!. ............................ 345/350
`5/1998 Fukuda eta!. ...................... 345/358 X
`6/1998 Berman et a!. ......................... 345/347
`8/1998 Shigematsu eta!. ................... 345/358
`9/1998 Shieh ...................................... 345/173
`9/1998 Moran eta!. ........................... 345/358
`9/1998 Pooser et a!. ........................... 345/356
`
`[73] Assignee: Norte) Networks Corporation, CAX
`
`FOREIGN PATENT DOCUMENTS
`
`[ *] Notice:
`
`This patent issued on a continued pros(cid:173)
`ecution application filed under 37 CFR
`1.53( d), and is subject to the twenty year
`patent term provisions of 35 U.S.C.
`154(a)(2).
`
`[21] Appl. No.: 08/985,265
`
`[22]
`
`Filed:
`
`Dec. 4, 1997
`
`[51]
`[52]
`
`[58]
`
`[56]
`
`Int. CI? ................................. G06F 3/14; G06F 3/02
`U.S. Cl. .......................... 345/339; 345/352; 345/347;
`345/341; 345/358
`Field of Search ..................................... 345/358, 352,
`345/353, 341, 354, 350, 348, 347, 356
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,954,967
`5,297,253
`5,563,996
`5,570,109
`5,590,257
`5,592,195
`5,600,765
`5,745,116
`
`9/1990 Takahashi ... ... .... ... ... ... ... ... .... .. 364/518
`3/1994 Meisel ..................................... 395/348
`10/1996 Tchao ...................................... 345/341
`10/1996 Jenson ..................................... 345/146
`12/1996 Forcier .................................... 707/530
`1!1997 Misono eta!. .......................... 345/146
`2/1997 Ando et a!. ......................... 345/326 X
`4/1998 Pisutha-Armond ..................... 345/358
`
`0 565 253 10/1993 European Pat. Off ..
`0 605 945
`7/1994 European Pat. Off ..
`0 684 543 11/1995 European Pat. Off ..
`
`OTHER PUBLICATIONS
`
`IBM Technical Disclosure Bulletin, "Scroll Control Box,"
`vol. 36, No. 4, Apr. 1993, pp. 399-403.
`
`Primary Examiner-Raymond 1. Bayerl
`Assistant Examiner-Thomas Nguyen
`Attorney, Agent, or Firm-Finnegan, Henderson, Farabow,
`Garrett & Dunner, L.L.P.
`
`[57]
`
`ABSTRACT
`
`The system and method consistent with the present inven(cid:173)
`tion provides improved graphical navigation tool for elec(cid:173)
`tronic devices. The graphical navigation tools may be used
`to navigate through documents at any direction at varying
`speeds. The navigation tool is preferably transparent to
`prevent obscuring the text of the underlying document.
`Accordingly, the navigational tool does not take up addi(cid:173)
`tional screen real estate. The GUI of the present invention
`may be especially useful in portable electronic devices with
`small displays.
`
`24 Claims, 11 Drawing Sheets
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 1 of 11
`
`6,037,937
`
`CJ§D
`CD@@
`C@D @D @_®)
`@C@Y)@
`0CDCD
`
`FIG. 1
`
`0 -
`
`-0
`
`10
`
`
`
`0
`
`~
`
`1¢1
`
`FIG. 2
`
`Ex. GOOG 1101
`
`

`

`320
`
`/310
`
`210
`
`~~ ~ @'
`
`HOLD DIAL HANGUP
`
`340
`
`<8>
`000 330
`
`d •
`\Jl
`•
`~
`~
`......
`
`~ = ......
`
`~
`
`~
`:-:
`'"""'
`~,J;;..
`N c
`c c
`
`'JJ. =-~
`~ ......
`N
`0 ......,
`'"""'
`'"""'
`
`000
`000
`000
`000
`
`[J CALL BRADY
`AI) (410) 651-3905
`DIAL
`
`QUIT
`
`<8>
`
`FIG. 3A
`
`FIG. 38
`
`Ex. GOOG 1101
`
`

`

`d •
`\Jl
`•
`
`ANTENNA
`~(\
`410
`
`420
`
`)
`
`[421
`
`GSM
`RADIO
`
`COMMUNICATIONS MODULE
`
`\423
`
`[427
`
`~424
`
`ND
`
`D/A
`
`VEGA
`
`BOCK
`
`r--
`
`r--
`
`l_425
`
`~24
`
`D/A
`
`ND
`
`AUDIO
`TRANSDUCERS
`SPEAKER
`428
`MICRO-
`PHONE
`429
`
`MEMORY
`FLASH ROM ~'~
`
`u440
`
`~'---444
`
`STATIC RAM 1'-442
`
`FIG. 4
`
`ASYNC COMMUNICATION
`(UART)
`
`\430
`
`;460
`
`t---
`
`ANALOG
`CONTROLLER
`
`~
`
`""
`
`1450
`
`SLIDING
`KEYPAD
`
`FEATURE
`PROCESSOR
`
`I
`
`[490
`
`SWITCHING
`POWER
`SUPPLY
`
`(480
`
`BATIERY
`PACK
`
`I TOUCH SCREEN PANEL ~
`I'- 474
`~
`~47
`
`I
`
`LCD
`
`'l
`
`470
`
`2
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar.14,2000
`
`Sheet 4 of 11
`
`6,037,937
`
`MEMORY 440
`
`STATIC RAM 442
`
`FLASH ROM 444
`
`PROGRAM
`510
`
`TOUCH
`SCREEN
`PROGRAM
`520
`
`NAVIGATION
`PROGRAM
`530
`
`DRAWING
`PROGRAM
`540
`
`FIG. 5
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar.14,2000
`
`Sheet 5 of 11
`
`6,037,937
`
`430
`
`460
`
`FEATURE
`PROCESSOR
`
`ANALOG
`CONTROLLER
`
`474
`
`TOUCH
`SCREEN
`PANEL
`
`FIG. 6
`
`FIG. 7A
`
`ACTIVATION
`AREA
`
`FIG. 78
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar.14,2000
`
`Sheet 6 of 11
`
`6,037,937
`
`MACROMEDI
`
`s
`
`FREE HAN
`NO CAN BE
`NT TO SIGN O~V
`
`FIG. 8
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar.14,2000
`
`Sheet 7 of 11
`
`6,037,937
`
`UP
`
`LEFT
`
`WITHIN-PAGE
`NAVIGATION
`CIRCLE
`
`FIG. 9A
`
`l=l /
`§T
`
`PREVIOUS
`PAGE
`
`~/NEXT
`@V
`PAGE
`
`FIG. 98
`
`~ ADDITIONAL CONTROLS FOR
`./
`FORWARD TO NEXT DOCUMENT,
`@]
`BACK, HOME, ETC.
`
`FIG. 9C
`
`Ex. GOOG 1101
`
`

`

`d •
`\Jl
`•
`
`RS CENTER
`
`EPICENTER GALLERY
`
`GUIDE I SEAR{!]
`
`FREEHAN
`NO CAN BE
`NT TO SIGN O~V
`
`FIG. 10A
`
`FIG. 108
`
`FIG. 10C
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar.14,2000
`
`Sheet 9 of 11
`
`6,037,937
`
`VIEWING WINDOW DOES NOT
`MOVE (ACTIVATION AREA)
`
`VIEWING WINDOW
`MOVES SLOWLY
`
`VIEWING WINDOW
`MOVES QUICKLY
`
`FIG. 11A
`
`VIEWING WINDOW MOVES
`QUICKLY TO THE PREVIOUS PAGE
`
`NEXT PAGE
`ICON
`
`VIEWING WINDOW MOVES
`SLOWLY TO THE PREVIOUS PAGE
`
`PREVIOUS PAGE
`ICON
`
`VIEWING WINDOW MOVES
`SLOWLY TO THE NEXT PAGE
`
`VIEWING WINDOW MOVES
`QUICKLY TO THE NEXT PAGE
`FIG. 118
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar.14,2000
`
`Sheet 10 of 11
`
`6,037,937
`
`TOUCH POINT
`
`DISTRIBUTION 1\
`
`c("a
`•
`""'D
`
`I
`
`~
`
`FIG. 12
`
`1310
`
`DETERMINE INDIVIDUAL POINTS
`OF CONTACT MADE BY OBJECT
`
`!
`
`1320
`
`COMPUTE CENTROID OF
`DETERMINED POINTS OF CONTACT
`
`~
`1330
`
`COMPUTE STANDARD
`DEVIATION OF CENTROID
`
`!
`1340 - -
`DETERMINE POINTER OF THE
`OBJECT MAKING CONTACT
`
`FIG. 13
`
`Ex. GOOG 1101
`
`

`

`U.S. Patent
`
`Mar.14,2000
`
`Sheet 11 of 11
`
`6,037,937
`
`PRESSURE
`
`PRESSURE
`
`PEN
`
`TIME
`
`FIG. 14A
`
`FINGER
`
`TIME
`
`FIG. 148
`
`Ex. GOOG 1101
`
`

`

`6,037,937
`
`1
`NAVIGATION TOOL FOR GRAPHICAL
`USER INTERFACE
`
`RELATED APPLICATIONS
`
`This application is related to U.S. patent application, Ser.
`No. 08/985,264 (Atty. Docket No. 3884.0116-00), entitled
`INTELLIGENT TOUCH DISPLAY; and U.S. patent
`application, Ser. No. 08/985,561 (Atty. Docket No.
`3884.0117-00), entitled CONTEXTUAL GESTURE
`INTERFACE, both of which are filed concurrently herewith,
`and both of which are hereby incorporated by reference.
`
`BACKGROUND OF THE INVENTION
`
`2
`tions in multiple layers, thus requiring a user to move deeply
`into the hierarchy before reaching the desired function. This
`is time consuming and renders the GUI cumbersome and
`ineffective.
`Therefore, it is desirable to provide navigation tools that
`allow small-size devices while maximizing the use of avail(cid:173)
`able screen real estate.
`It is also desirable to provide tools to navigate within a
`document at any direction at varying speeds.
`It is further desirable to provide navigation tools that can
`be activated without requiring specific electronic devices.
`In addition, it is further desirable to provide an improved
`GUI that simplifies GUI by recognizing various character-
`15 istics of the touch input.
`
`5
`
`10
`
`SUMMARY OF THE INVENTION
`
`Systems and methods consistent with the present inven(cid:173)
`tion provide graphical control tools for efficient navigation
`in display devices.
`Specifically, a method consistent with this invention of
`controlling display of content information in a physical
`viewing area comprises several steps. Initially, the system
`displays the content information in the physical viewing
`area. The system also displays a representation of a control
`tool over the display of content information. Thereafter, the
`system receives a user input selecting the control tool, and
`controls the display of content information according to the
`user input.
`A system consistent for this invention for controlling
`display of content information in a physical viewing area
`includes displaying means, receiving means, and controlling
`means. The displaying means displays the content informa-
`35 tion in the physical viewing area, and displays a represen(cid:173)
`tation of a control tool over the display of content informa(cid:173)
`tion. The receiving means receives a user input selecting the
`control tool. Thereafter, the controlling means controls the
`display of the content information according to the user
`40 input.
`
`30
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention relates generally to graphical user
`interfaces (GUI), and more particularly to a navigation tool
`for graphical user interfaces.
`Until relatively recently, software-based documents have
`been primarily viewed and manipulated on desktop or laptop
`computers with relatively large displays, typically 640-480 20
`pixels or larger. These displays are often large enough to
`display a full page of standard size page or at least a
`significant portion of the page. Hence, on-screen graphical
`menus and controls displayed in window of an application
`did not greatly reduce the display area for the underlying 25
`document. Computers also have peripheral devices such as
`a keyboard or a mouse to control the display of content
`information. Thus, viewing and navigating around a single(cid:173)
`page or multi-page document have not posed much diffi(cid:173)
`culty.
`Due to increasing focus on compactness of electronic
`devices, however, the displays especially in portable elec(cid:173)
`tronic devices are becoming smaller and smaller. Popular
`electronic devices with a smaller display area include elec(cid:173)
`tronic organizers, PDA's (personal digital assistants), and
`graphical display-based telephones. Also available today are
`communicators that facilitate various types of communica(cid:173)
`tion such as voice, faxes, SMS (Short Messaging Services)
`messages, e-mail, and Internet-related applications. These
`products can likewise only contain a small display area.
`To enable users to navigate around a full page of content
`information, these devices typically provide hard-keys for
`arrows as shown in FIG. 1. The hard-keys, however, not only
`increase the size but also add to the cost of the devices. Also,
`hard-keys generally provide limited options for direction of
`movement, e.g., vertical or horizontal. They generally do not
`provide the freedom to move in any direction.
`Some displays of these devices also require a separate
`stylus having peripheral technology that requires transmis(cid:173)
`sion of electromagnetic pulses or light to the display. These
`devices often require additional controllers such as buttons
`on the body or the tip of the stylus for activation.
`Furthermore, these styli require a power source, either
`through wire or battery, and their compatibility is generally
`limited to a specific device.
`As shown in FIG. 2, other devices substitute hard-keys
`with graphical on-screen arrows or scroll bars that are
`typically used in full-size computer displays. The on-screen
`scroll bars, however, occupy valuable screen real estate and
`compound the limitations of small displays. Similar to the
`hard-keys, the on-screen arrows also generally restrict the
`navigational movement to horizontal or vertical direction.
`In other forms of on-screen GUis, e.g., pop-up menus,
`also take up valuable screen space, further reducing the
`available display area for content information. Additionally,
`on-screen pop-up menus typically provide available func-
`
`45
`
`The accompanying drawings, which are incorporated in
`and constitute a part of this specification, illustrate the
`invention and together with the description, serve to explain
`the principles of the invention.
`In the drawings,
`FIG. 1 shows conventional hard-key arrows for naviga-
`50 tion control;
`FIG. 2 shows conventional on-screen graphical naviga(cid:173)
`tion tool;
`FIGS. 3A-3B are diagrams of an exemplary mobile
`telephone consistent with the principles of the present inven-
`55 tion;
`FIG. 4 is a block diagram showing the elements of the
`mobile telephone of FIG. 3A;
`FIG. 5 is a block diagram showing the components of the
`memory of FIG. 4;
`FIG. 6 is a block diagram of touch screen functionalities;
`FIGS. 7A-7B show an exemplary inactive and active
`graphical navigation tool, respectively;
`FIG. 8 is a sample screen showing an active navigation
`65 tool;
`FIGS. 9A-9C show exemplary features of the navigation
`tool;
`
`60
`
`Ex. GOOG 1101
`
`

`

`6,037,937
`
`3
`FIGS. 10A-10C are sample screens showing the naviga(cid:173)
`tion tool performing various navigation functions;
`FIGS. llA-llB show exemplary features of the naviga(cid:173)
`tion tool relating to speed of navigation;
`FIG. 12 is a diagram illustrating a touch point distribution;
`FIG. 13 is a flowchart illustrating the process of deter(cid:173)
`mining the size of the object making contact with the
`viewing area; and
`FIGS. 14A and 14B are graphs showing the touch char(cid:173)
`acteristics of a pen and a finger, respectively.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`
`Reference will now be made in detail to the present 15
`preferred embodiment of the invention, an example of which
`is illustrated in the accompanying drawings. Where
`appropriate, the same reference numerals refer to the same
`or similar elements. The appended claims define the scope of
`the invention; the following description does not limit that 20
`scope.
`The graphical navigation tool of the present invention
`may be implemented in a wide range of electronic devices
`mentioned above such as electronic organizers, PDA's, and
`graphical display-based telephones. Although the need to
`maximize the use of screen real estate is most critical in
`portable electronic devices with small displays, the present
`invention can also be implemented in full-size computers or
`electronic devices. For purposes of illustration, however, the
`present invention will be explained in detail in a mobile
`telephone environment.
`Specifically, FIG. 3A shows a mobile telephone 310 and
`FIG. 3B shows an exemplary wireline telephone preferably
`having the graphical navigation tool consistent with the
`present invention. Mobile telephone 310 includes main
`housing 210, antenna 320, keypad 330, and display 340.
`FIG. 4 shows the hardware elements in mobile telephone
`310 including antenna 410, communications module 420,
`feature processor 430, memory 440, sliding keypad 450,
`analog controller 460, display module 470, battery pack 480,
`and switching power supply 490.
`Antenna 410 transmits and receives radio frequency infor(cid:173)
`mation for mobile telephone 310. Antenna 410 preferably
`comprises a planar inverted F antenna (PIFA)-type or a short
`stub (2 to 4 em) custom helix antenna. Antenna 410 com(cid:173)
`municates over a GSM (Global System for Mobile
`Communications) switching fabric using a conventional
`voice B-channel, data B-channel, or GSM signaling channel
`connection.
`Communications module 420 connects to antenna 410
`and provides the GSM radio, baseband, and audio function(cid:173)
`ality for mobile telephone 310. Communications module
`420 includes GSM radio 421, VEGA 423, BOCK 425, and
`audio transducers 427.
`GSM radio 421 converts the radio frequency information
`to/from the antenna into analog baseband information for
`presentation to VEGA423. VEGA423 is preferably a Texas
`Instruments VEGA device, containing analog-to-digital
`(A/D)/digital-to-analog (D/A) conversion units 424. VEGA
`423 converts the analog baseband information from GSM
`radio 421 to digital information for presentation to BOCK
`425.
`BOCK 425 is preferably a Texas Instruments BOCK
`device containing a conventional ARM microprocessor and
`a conventional LEAD DSP device. BOCK 425 performs
`GSM baseband processing for generating digital audio sig-
`
`30
`
`4
`nals and supporting GSM protocols. BOCK 425 supplies the
`digital audio signals to VEGA 423 for digital-to-analog
`conversion. VEGA 423 applies the analog audio signals to
`audio transducers 427. Audio transducers 427 include
`5 speaker 428 and microphone 429 to facilitate audio com(cid:173)
`munication by the user.
`Feature processor 430 provides GUI features and a Java
`Virtual Machine (JVM). Feature processor 430 communi(cid:173)
`cates with BOCK 425 using high level messaging over an
`10 asynchronous (UART) data link. Feature processor 430
`contains additional system circuitry, such as a liquid crystal
`display (LCD) controller, timers, UART and bus interfaces,
`and real time clock and system clock generators (not
`shown).
`Memory 440 stores data and program code used by
`feature processor 430. Memory 440 includes static RAM
`442 and flash ROM 444. Static RAM 442 is a volatile
`memory that stores data and other information used by
`feature processor 430. Flash ROM 444, on the other hand,
`is a non-volatile memory that stores the program code
`executed by feature processor 430.
`Sliding keypad 450 enables the user to dial a telephone
`number, access remote databases, and manipulate the GUI
`features. Sliding keypad 450 preferably includes a mylar
`25 resistive key matrix that generates analog resistive voltage in
`response to actions by the user. Sliding keypad 450 prefer(cid:173)
`ably connects to main housing 210 (FIG. 3A) of mobile
`telephone 310 through two mechanical "push pin" -type
`contacts (FIG. 4).
`Analog controller 460 is preferably a Phillips UCBllOO
`device that acts as an interface between feature processor
`430 and sliding keypad 450. Analog controller 460 converts
`the analog resistive voltage from sliding keypad 450 to
`digital signals for presentation to feature processor 430.
`Display module 470 preferably includes a 160x320 pixel
`LCD 472 with an analog touch screen panel 474 and an
`electroluminescent backlight. LCD 472 operates in conjunc(cid:173)
`tion with feature processor 430 to display the GUI features.
`40 Analog controller 460 scans touch screen overlay 474 while
`feature processor 430 refreshes LCD 472.
`Battery pack 480 is preferably a single lithium-ion battery
`with active protection circuitry. Switching power supply 490
`ensures highly efficient use of the lithium-ion battery power
`45 by converting the voltage of the lithium-ion battery into
`stable voltages used by the other hardware elements of
`mobile telephone 310.
`FIG. 5 is a block diagram illustrating the components of
`memory 440. Static RAM 442 stores data and other infor-
`50 mation used by feature processor 430. Flash ROM 444
`contains various programs including a program 510, a touch
`screen program 520, a navigation program 530, and a
`drawing program 540. Program 520, preferably written in
`languages such as Java, C, or C++ for Macintosh, is a main
`55 program overseeing the operation of mobile telephone 310.
`Touch screen program 520 facilitates processing of touch
`input on touch screen panel 474 using a typical touch input
`algorithm. Navigation program 530 handles navigation of
`the content information display. Drawing program 540 is a
`60 graphical drawing package. Programs 520, 530, and 540
`may be one of any commercially available packages or a
`user-defined feature program or macro.
`The present invention provides various features through
`tactile GUI. Initially, LCD 472 displays various GUI fea-
`65 tures. Referring to FIG. 6, a user touches touch screen panel
`474 to provide user input, for example, to navigate around
`a document or invoke a desired function. Analog controller
`
`35
`
`Ex. GOOG 1101
`
`

`

`6,037,937
`
`5
`460 scans touch screen panel 474 and reads the correspond(cid:173)
`ing analog voltage of touch screen panel 474. Analog
`controller 460 then converts the analog values into corre(cid:173)
`sponding digital values representing the Cartesian
`coordinates, which are transmitted to feature processor 430
`for processing. The resolution of the touch input depends on
`the ability of analog controller 460 to discern among mul(cid:173)
`tiple levels of analog values, generally defined in bits.
`FIGS. 7A-7B show an exemplary graphical navigation
`tool preferably used to navigate around documents that are
`too large to view within a single screen of a physical display
`(hereinafter referred as "viewing area"). The navigation tool
`may be used to view any kind of document including faxes,
`Web pages, or e-mail. In one embodiment consistent with
`the present invention, an inactive navigation tool is dis(cid:173)
`played and accessible to the user at all times (FIG. 7A). The
`user may activate the navigation tool by touching and
`holding the center of the navigation tool for a predetermined
`time period, for example, one to two seconds (FIG. 7B). An
`activated navigation tool is preferably transparent to avoid 20
`hindering the display of content information in the viewing
`area as shown in FIG. 8. Alternatively, the navigation star
`may change colors or other features of its appearance to
`indicate its active status. A solid line image, for example,
`may be used in greyscale displays that do not support 25
`transparency.
`The present invention may be designed such that feature
`processor 430 ignores any touch input on the navigation tool
`unless the navigation tool has been activated. Instead, the
`touch input may be interpreted as input to access control 30
`buttons in the underlying document, write on the underlying
`document, or invoke other functions related to the underly(cid:173)
`ing document. This will prevent against unintentional navi(cid:173)
`gation in the viewing window in case the user inadvertently
`touches touch screen panel 474. In an alternative 35
`embodiment, the present invention may accept stylus input
`to access the underlying document while a finger or non(cid:173)
`electromagnetic touch on any part of the navigation tool
`invokes the navigation function.
`Referring to FIGS. 9A-9C, once the navigation tool is
`activated, the user may navigate through the document by
`selecting the graphical arrows, e.g., up, right, left, and down
`arrows (FIG. 9A), or graphical page icons, e.g., previous or
`next page (FIG. 9B). One skilled in the art may vary the type 45
`and number of graphical tools significantly. For example, the
`navigation tool may provide graphical representations for
`forward, next document, back, or home functions (FIG. 9C).
`FIGS. lOA-lOC show exemplary screen displays while
`the user is touching the navigation tool. Upon touching the 50
`right arrow of the navigation tool, for example, the right
`arrow is highlighted and navigation program 530 moves the
`display to the right (FIG. lOA). Similarly, touching the down
`arrow moves the display down (FIG. lOB). Although the
`four arrows are presented to guide the users, navigation 55
`program 530 supports navigational movement at any direc(cid:173)
`tion. If the user touches an area of the navigation tool
`equidistant between the up and right arrows, for example,
`navigation program 530 will move the display towards the
`upper-right portion of the underlying document at a 60
`45-degree angle. Touching the arrows or any area in
`between, moves the display in the selected direction until
`navigation program 530 reaches the edge of the page.
`Touching the next page icon moves the viewing window
`to the next page of the underlying document (FIG. lOC). If
`a particular document does not have a page corresponding to
`a previous or next page icon, navigation program 530 will
`
`6
`not display the respective previous or next page icons. This
`would apply to one-page documents, or when the user is at
`the beginning or end of a multi-page document. In one
`embodiment consistent with the present invention, a
`5 momentary touch of the next page icon causes navigation
`program 530 to jump to the next page while a continuous
`touch on the next page icon causes navigation program 530
`to continue scrolling through succeeding pages of the under(cid:173)
`lying document. The previous page icon may embody simi-
`10 lar characteristics.
`The user may also control the speed of the navigation. As
`shown in FIG. llA, the speed of the navigation accelerates
`as the user touch moves from the center of the circle toward
`the circumference of the circle, i.e., tip of the arrow. Hence,
`15 the viewing window moves slowly when the user touches
`the blunt end of the arrow located at the center of the circle
`while the speed accelerates as the user moves the finger
`towards the tip of the arrow. The speed of navigation,
`therefore, is determined by the distance of the touch relative
`to the center of the circle. Likewise, similar principles apply
`to previous or next page/document icons where a touch
`closer to the outer edge of the previous or next page/
`document icons accelerates navigation through the docu(cid:173)
`ment as shown in FIG. llB.
`Although the exemplary transparent tool discussed above
`is for navigation, transparent control tools may be imple(cid:173)
`mented for a variety of functions. A transparent tool may, for
`example, be used for a Web browser application where the
`controls may be used for appropriate functions such as
`moving forwards or backwards through different Web pages
`or returning to home page. One skilled in the art may easily
`vary the design or the functionality of the graphical navi(cid:173)
`gation tools described above without departing from the
`scope of the present invention.
`In an exemplary embodiment of a navigation tool
`described above, a finger touch invokes navigational func(cid:173)
`tions based on the feature selected and the location of the
`user touch. Alternatively, other objects making contact with
`touch screen panel 474 may invoke other tools or functions.
`A pointy stylus touch, for example, may invoke a menu with
`cardinal points representing multiple line widths, colors, or
`patterns.
`In another embodiment consistent with the present
`invention, tools or application programs may be stored in
`flash ROM 444 to provide related interfaces to the user. The
`use of a finger may, for example, invoke tools or dialogues
`that are finger-touchable and large whereas the use of a sharp
`stylus may invoke a modified GUI with smaller touch
`targets. In a yet another embodiment, in a document viewing
`application normally navigable by a finger touch, use of a
`sharp stylus may automatically invoke a document annota(cid:173)
`tion application for marking up the underlying document.
`As described above, the touch-responsive GUI of the
`present invention are facilitated though various components
`including touch screen panel474, analog controller 460, and
`feature processor 430. Specifically, analog controller 460
`scans touch screen panel 474 to read the corresponding
`analog voltage of touch screen panel 474 activated by a user
`touch. Analog controller 460 then converts the analog values
`into a digital value representing the Cartesian coordinates,
`which is transmitted to feature processor 430 for processing
`according to the functionalities of the present invention.
`When a user touches touch screen panel474, program 510
`65 initiates touch screen program 520 to determine the pointer
`size of the object making contact with touch screen panel
`474 based on a touch point distribution or pointer size of the
`
`40
`
`Ex. GOOG 1101
`
`

`

`6,037,937
`
`25
`
`7
`touch input. As shown in FIG. 12, touch screen program 520
`can, for example, determine whether the pointer size of the
`object is a finger or a sharp object.
`FIG. 13 is a flowchart illustrating the process of deter(cid:173)
`mining the size of the object making contact with the
`viewing area. Touch point program 520 first determines the
`individual points of contact made by the object (step 1310).
`It computes a centroid, or other average point, of the
`determined points of contact (step 1320). Touch program
`520 then computes a standard deviation of the centroid as 10
`well as the variance (step 1330), and determines the pointer
`size based on the centroid and the standard deviation (step
`1340). These computations are preferably performed on a
`real-time basis to provide immediate system response to the
`touch input. In order to achieve optimum results and 15
`accuracy, analog touch controller 460 preferably generates
`150 points per second or more. Touch program 520 may also
`use the amount of pressure imposed on touch screen panel
`474 as a function of time to determine the size of object. As
`shown in FIG. 14A, for example, if the amount of pressure 20
`increases or decreases sharply at a particular instant in time,
`touch point program 520 may determine that the touch
`corresponds to a pen. A finger touch, on the other hand,
`results in a gradual increase and decrease in pressure as
`illustrated by a smoother curve in FIG. 14B.
`Program 510 can also be programmed to correlate certain
`pointer size to certain objects and invoke corresponding
`functions or tools. Such GUI provides a richer, yet simplified
`interaction between the user and mobile telephone 310. If
`program 510 determines that the pointer size of the object 30
`corresponds to the size of a finger, program 510 may initiate
`a navigation tool. If the pointer size corresponds to the size
`of several fingers, program 510 may invoke a drag function
`of the navigation tool. On the other hand, if program 510
`determines that the pointer size of the object corresponds to 35
`size of a sharp point or pen, program 510 may initiate a
`drawing tool supported by drawing program 540. Similarly,
`if program 510 determines that the pointer size of the object
`corresponds to size of a pencil eraser, program 510 may
`initiate an erase function of the drawing tool. One skilled in 40
`the art may easily vary the functions or tools initiated by
`program 510. Additionally, the functions or tools may be
`commercial software packages, predetermined functions, or
`user-defined macros.
`In addition to using the pointer size to determine the 45
`desired GUI, program 510 can also incorporate other char(cid:173)
`acteristics of the user touch, e.g., gestures or movements, to
`simplify GUI and maximize screen real estate. A gesture
`recognizing interface extends the ability of the present
`invention to distinguish between different sized pointers to 50
`track gestures and movement of user input based on vector
`direction and magnitude, all in the context of active user
`application. This type of contextual gesture interface can
`infer by context, the implement, and the gesture chosen by
`the user what functions the user wishes to invoke. 55
`Accordingly, all these functions are available without menus
`or scroll bars and do not require additional screen areas to
`display the functions.
`Program 510 recognizes other characteristics of the touch
`input including the context of the input, namely the task or 60
`sub-task applications running when the GUI is invoked. If a
`user is in a document navigation application, for example,
`program 510 interprets a quick drag to the right as a next
`page function. If the underlying task is an editing
`application, program 510 may interpret the same gesture as 65
`a highlight function and highlight a portion of the document
`touched by the user. Similarly, in graphics application, a
`
`8
`quick drag to the right may invoke a drawing tool to draw
`from the starting point to the ending point of the touch
`points. In a document viewing application, the same touch
`may invoke a navigation tool to move the view of the
`5 document in the direction of the finger drag.
`All of the above functions and features described above
`focuses on providing intuitive GUis and minimize the need
`for users to memorize complicated, hierarchical menus or
`procedures. Additionally, the present invention maximize
`available screen real estate while providing a wide array of
`GUI and tools.
`It will be apparent to those skilled in the art that various
`modifications and variations can be made in the system of
`the present invention and in construction of this system
`without departing from the scope or spirit of the invention.
`Other embodiments of the invention will be apparent to
`those skilled in the art from consideration of the specifica(cid:173)
`tion and practice of the invention disclosed herein. The
`specification and examples should be considered as exem(cid:173)
`plary only, with the true scope and spirit of the invention
`indicated by the following claims.
`What is claimed is:
`1. A method of activating functions responsive to a user
`input, comprising:
`providing a manipulable