US 20050024341A1
`
`(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2005/0024341 A1
`(43) Pub. Date: Feb. 3, 2005
`
`Gillespie et al.
`
`(54)
`
`(75)
`
`TOUCH SCREEN WITH USER INTERFACE
`ENHANCEMENT
`
`Publication Classification
`
`Inventors: David W. Gillespie, Los Gatos, CA
`(US); Ray Trent, San Jose, CA (US);
`Andrew C. Hsu, Belmont, CA (US);
`Leslie R. Grate, Waltham, MA (US)
`
`Int. Cl.7 ....................................................... G09G 5/00
`(51)
`(52) U.S.Cl.
`.............................................................. 345/173
`
`(57)
`
`ABSTRACT
`
`Correspondence Address:
`SIERRA PATENT GROUP, LTD.
`P 0 BOX 6149
`STATELINE, NV 89449 (US)
`
`(73)
`
`Assignee: Synaptics, Inc.
`
`(21)
`
`Appl. No.:
`
`10/125,067
`
`(22)
`
`Filed:
`
`Apr. 17, 2002
`
`Related US. Application Data
`
`(60)
`
`Provisional application No. 60/291,694, filed on May
`16, 2001.
`
`The present invention is a graphical user interface in a
`computing device having a processor running an operating
`system and a display. The graphical user interface comprises
`a touch screen and a driver coupling the touch screen to the
`operating system. The driver can display a plurality of icons
`on the touch screen, or a plurality of screen images having
`at least one icon, with each of the icons associated with
`operations on the display and/or the touch screen. Other
`embodiments include the touch screen having unactivated
`and activated states, as well as the presence of an application
`programming interface that enables an application to display
`at least one image on the touch screen.
`
`426
`
`
`
`412
`
`414
`
`APPLE 1030
`
`APPLE 1030
`
`1
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 1 0f 14
`
`US 2005/0024341 A1
`
`/
`
`Figure 2
`
`2
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 2 0f 14
`
`US 2005/0024341 A1
`
`300
`
`412
`
`
`414
`
`428
`
`426
`
`3
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 3 0f 14
`
`US 2005/0024341 A1
`
`005
`
`
`
`Figure 5
`
`4
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 4 0f 14
`
`US 2005/0024341 A1
`
`600
`
` W 6
`
`34
`
`63
`
`Figure 6B
`
`5
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 5 0f 14
`
`US 2005/0024341 A1
`
`702
`
`700
`
`710 \
`
`700
`
`720 \
`
`700
`
`722
`
`Figure 7A
`
`Figure 7B
`
`Figure 7C
`
`730
`
`732 \\®
`
`700
`
`744
`
`740 E 742
`i
`i
`746 \
`i
`
`Figure 71)
`
`Figure 7E
`
`6
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 6 0f 14
`
`US 2005/0024341 A1
`
`800
`
`Slide 6: 2001 sales data
`
`Used Fred’s estimates for UK rgn
`
`Remember to highlight new prods!
`
`an
`
`806 u-———-—.-u-_«——.
`
`Dept intro
`
`(1:11
`
`Projections
`for 2002 sales
`
`D
`
`Figure 8A
`
`820
`
`822
`
`826
`
`824
`
`Figure 8B
`
`7
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 7 0f 14
`
`US 2005/0024341 A1
`
`
`
`Paying too much for jewelry?
`
`www.hot-jewe|ry.com
`
`842
`
`
`
`862
`
`864
`
`870
`
`8
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 8 0f 14
`
`US 2005/0024341 A1
`
`Painter.
`
`the format from a selected
`
`or text and applies it to the
`
`To copy the
`or text you click.
`to more than one item, doublee
`
`click the icon, click each item you
`want to format,
`then press ESC.
`
`More Help
`g Find and Replace
`
`908
`
`Fgm£9
`
`902
`
`1000
`
`1004
`
`i002
`
`9
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 9 0f 14
`
`US 2005/0024341 A1
`
`
`
`Figure 1013
`
`1 062
`
`Figure 10C
`
`10
`
`10
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 10 0f 14
`
`US 2005/0024341 A1
`
`1100
`
`
`
`
`------------
`
` 1102
`1204
`
`1202
`
`1200
`
`11
`
`11
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 11 0f 14
`
`US 2005/0024341 A1
`
`1302
`
`Figure 13A
`
`1320
`
`Magnification Level
`
`Figure 13B
`
`12
`
`12
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 12 0f 14
`
`US 2005/0024341 A1
`
`printf("Hello");
`
`Figure 13C
`
`getUserName();
`
`
`1408
`
`1402
`
`Figure 14
`
`13
`
`13
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 13 0f 14
`
`US 2005/0024341 A1
`
`Stop! This computer is registered to
`
`John Q. Doe
`
`
`
`Enter your PIN
`and touch Begin
`to proceed.
`
`-EJ
`
`1500
`
`1520
`
`1522
`
`1524
`
`1528
`
`1530
`
`
`
`
`
`Display &
`Peripheral
`Central
`Hard Disk
`
`
`
`Controllerl Processor
`
`
`
`
`Other
`
`Devices
`
`1526
`
`Figure 153
`
`14
`
`14
`
`

`

`Patent Application Publication Feb. 3, 2005 Sheet 14 of 14
`
`US 2005/0024341 A1
`
`1610
`
`1612
`
`1614
`
`Peripheral
`Controller
`
`Driver
`
`Hardware
`
`Touch
`Screen
`
`l 622
`
`Pointing
`Device
`
`Driver
`
`1 620
`
`1 624
`
`1600
`\
`
`1 602
`
`
`
`F‘_—_-|I—_—__'_"——lF__
`
`1604
`
`1606
`
`Software
`Application
`
`Software
`Application
`
`
`
`Software
`Application
`
`1640
`
`1642
`
`1644
`
`Figure 16
`
`15
`
`15
`
`

`

`US 2005/0024341 A1
`
`Feb. 3, 2005
`
`TOUCH SCREEN WITH USER INTERFACE
`ENHANCEMENT
`
`PRIORITY TO RELATED APPLICATIONS
`
`[0001] The present application claims priority to US.
`Patent Provisional Application Ser. No. 60/291,694, entitled
`“Touch Screen with User Interface Enhancement”, filed on
`May 16, 2001, which is incorporated herein in its entirety.
`
`BACKGROUND
`
`[0002] The present invention relates to computer interface
`devices, and more particularly, to a computer touch pad with
`integrated display device, and enhancements to the portable
`computer user interface employing same.
`
`[0003] Touch pads are widely used in computer applica-
`tions, particularly as pointing devices in portable computers.
`In typical usage,
`the touch pad is a featureless, finger
`sensitive surface in a rectangular opening of the palm rest of
`the computer. The touch pad serves solely as an input device
`for the computer. The touch pad functions primarily as a
`cursor pointing device, but some touch pads offer additional
`functions.
`
`[0004] For example, US. Pat. No. 5,543,591 to Gillespie
`et al. discloses a typical prior art touch pad sensor in which
`finger tapping gestures in designated regions of the touch
`surface invoke special commands on the computer. US. Pat.
`No. 5,943,052 to Allen et al. discloses a touch pad in which
`finger motions in designated regions invoke a scrolling
`command. These tap regions and scrolling regions have
`proven useful to expert users but confusing to novice users
`as the regions are invisible to the eye but different
`in
`behavior. Marking the regions with screen-printed icons on
`the opaque sensor surface can help, but it can also lead to
`greater confusion if the regions are software configurable.
`
`[0005] A further disadvantage of prior art touch pads is
`that they use up a significant fraction of the surface area of
`the computer for a single dedicated input function. Other
`pointing devices such as isometric joysticks (see, e.g., US.
`Pat. No. 5,521,596 to Selker et al) and force sensing keys
`(see, e.g., US. Pat. No. 4,680,577 to Straayer et al) have
`been proposed as compact alternatives, but these devices are
`not as expressive or as easy to use as touch pads.
`
`[0006] Touch screens are also well known in the art. One
`example of a touch screen is disclosed in US. Pat. No.
`4,806,709 to Blair. In typical use, the main display screen of
`a computer is overlaid with or implemented as a touch
`sensitive input device. This eliminates the need to dedicate
`separate parts of the surface of the computer for input and
`output. If the touch screen serves as the main pointing device
`of the computer, pointing is accomplished by a direct
`mapping from finger position to selection of a point on the
`screen beneath the finger. This direct mapping makes touch
`screens easy to understand and use. However, touch screens
`are impractical for everyday use as the main display of a
`computer because the user’s arm tires from being continu-
`ously held up to touch the screen. If the touch screen is laid
`flat to avoid arm wear, the arm tends to rest on the touch-
`sensing surface and, with many touch sensing technologies,
`this disrupts the ability to sense the finger. Touch screens the
`size of a main computer display may also be prohibitively
`bulky or expensive for use in applications that do not require
`them.
`
`[0007] Atransparent touch pad suitable for placement over
`a display such as an LCD screen has been developed and is
`disclosed and claimed in co-pending US. patent application
`Ser. No. 09/415,481, filed Oct. 8, 1999, assigned to the same
`assignee as the present invention. This application discloses
`a touch screen having the small size and low cost of a
`conventional touch pad for portable computers and notes
`that
`the touch pad and display could be included in a
`personal computer to enhance the user interface in various
`ways, but it does not disclose details of the software imple-
`mentation, nor how such a device can simultaneously func-
`tion as the pointing device of the computer, nor how this
`arrangement enhances the user interface.
`
`SUMMARY
`
`[0008] The drawbacks and disadvantages of the prior art
`are overcome by the touch screen with user interface
`enhancement.
`
`[0009] The present invention is a graphical user interface
`in a computing device having a processor running an oper-
`ating system and a display. The graphical user interface
`comprises a touch screen and a driver coupling the touch
`screen to the operating system. The driver can display a
`plurality of icons on the touch screen, or a plurality of screen
`images having at least one icon, with each of the icons
`associated with operations on the display and/or the touch
`screen. Other embodiments include the touch screen having
`unactivated and activated states, as well as the presence of
`an application programming interface that enables an appli-
`cation to display at least one image on the touch screen.
`
`BRIEF DESCRIPTION OF THE DRAWING
`FIGURES
`
`[0010] Referring now to the figures, wherein like elements
`are numbered alike:
`
`[0011] FIG. 1 is a diagram showing a notebook computer
`system with main display, keyboard, and touch screen;
`
`a diagram showing an illustrative
`[0012] FIG. 2 is
`embodiment of a touch screen in greater detail;
`
`[0013] FIG. 3 is a diagram illustrating an example default
`image for use when the touch screen is operating as a
`conventional touch pad;
`
`[0014] FIG. 4 is a diagram illustrating an example of a
`first “iconic” usage mode of the touch screen;
`
`[0015] FIG. 5 is a diagram illustrating the touch screen
`image of FIG. 4 modified to indicate the activated state of
`the touch screen using a dashed line around each icon that
`is touch-sensitive in the activated state;
`
`[0016] FIG. 6A is a diagram illustrating a portion of the
`keyboard featuring several keys; FIG. 6B is a diagram
`illustrating one possible arrangement of a special
`touch
`sensitive region or second touch sensor could be provided
`that activates the touch screen when touched;
`
`[0017] FIG. 7A is a diagram illustrating small icons that
`may be smaller than a finger and may be completely
`obscured by the finger when the finger touches them;
`
`[0018] FIG. 7B through 7E illustrate several mechanisms
`to eliminate the problem of obscuring small icons;
`
`16
`
`16
`
`

`

`US 2005/0024341 A1
`
`Feb. 3, 2005
`
`[0019] FIGS. 8A through 8D are diagrams illustrating
`use of a small control panel on the touch screen associated
`with an application, reserving the entire main display for
`visual data associated with the application;
`
`[0020] FIG. 9 is a diagram showing an example use of the
`touch screen to display subsidiary help text;
`
`[0021] FIG. 10A is a diagram illustrating employment of
`the touch screen to display a find/replace dialog on the touch
`screen, leaving the main display free to display a document
`unobstructed;
`
`[0022] FIG. 10B is a diagram illustrating use of the touch
`screen to act as a joystick emulator while displaying the
`control layout established by the game, leaving the main
`display free to display game graphics unobstructed;
`
`[0023] FIG. 10C is a diagram illustrating an example in
`which a touch screen image includes icons drawn from a
`typical toolbar,
`leaving the main display free to display
`document or an image unobstructed;
`
`[0024] FIG. 11 is a diagram illustrating a pop-up image
`including various icons representing commonly used tools
`and software applications on the computer;
`
`[0025] FIG. 12 is a diagram illustrating a pop-up calcu-
`lator application that operates entirely within the touch
`screen;
`
`[0026] FIGS. 13A and 13B are diagrams illustrating
`different features of a magnifier as a pop-up image on a
`touch screen, leaving the main display undisturbed;
`
`[0027] FIG. 13C is a diagram illustrating a debugger
`implemented as a pop-up application on a touch screen,
`providing a secondary debugging display with no extra cost
`or bulk;
`
`[0028] FIG. 14 is a diagram illustrating an example of an
`ideographic handwriting entry system on a touch screen in
`which a handwriting entry area responds to finger touch to
`enter an ideographic character;
`
`[0029] FIG. 15A is a diagram illustrating use of a touch
`screen as a user interface device for a computer security
`interlock;
`
`[0030] FIG. 15B is a diagram illustrating an exemplary
`hardware architecture for implementing the computer secu-
`rity interlock of FIG. 15A; and
`
`[0031] FIG. 16 is a diagram illustrating an exemplary
`software architecture for a touch screen.
`
`DETAILED DESCRIPTION
`
`[0032] Those of ordinary skill in the art will realize that
`the following description of the present invention is illus-
`trative only and not in any way limiting. Other embodiments
`of the invention will readily suggest themselves to such
`skilled persons.
`
`[0033] FIG. 1 illustrates a notebook computer system 100
`with main display 102 and keyboard 104. Touch screen 106
`is mounted in palm rest 110. The touch screen is typically
`equipped with left and right “mouse” buttons 108. Touch
`screen 106 is integrated into computer system 100 in a
`similar way as a touch pad would be in a prior art computer.
`Touch screen 106 will usually be located in the palm rest as
`
`shown in FIG. 1, but other locations are equally applicable,
`such as above the keyboard, adjacent to the keyboard or
`main display, or located in a separate enclosure connected by
`cable or wireless link to the computer. Although touch
`screen 106 usually replaces the conventional touch pad of a
`computer, touch screen 106 could be introduced in addition
`to the other user interface devices of the computer.
`
`[0034] FIG. 2 illustrates an illustrative embodiment of
`touch screen 106 in greater detail. Touch screen assembly
`200 consists of touch sensor 202, display 204, and backlight
`206 stacked or laminated together. Touch screens can be
`built in a variety of alternative ways as are well known in the
`art. For example, touch sensor 202 can be an active sensor
`employing capacitive, resistive, inductive, or other methods,
`or it can be a passive surface on which touch sensing is
`accomplished by optical, acoustic, or other methods.
`Capacitive touch sensors are ideally suited for use in the
`present invention due to their sensitivity, low cost, rugged-
`ness, and suitability to small sensing areas. However, any
`touch screen technology would serve for the present inven-
`tion.
`
`[0035] Similarly, display 204 can be a liquid crystal dis-
`play (LCD), organic light emitting diode (OLED) display,
`electroluminescent display, or any other type of small dis-
`play suitable for mounting in a portable computer. LCD
`displays are ideally suited for use in the present invention
`due to their low cost and availability, but other types of
`displays may be employed. Display 204 may be color or
`monochrome, and need not have the same resolution, color
`capabilities, or other qualities as the main display of the
`computer.
`
`[0036] The touch screen assembly may include a backlight
`206 to enhance readability in all lighting conditions. In
`alternative embodiments, backlight 206 may be replaced by
`a frontlight, passive reflector, or other light source, or it may
`be omitted altogether.
`
`[0037] Touch screen assembly 200 may include additional
`layers or components to assist the mounting or mechanical
`properties of the touch screen or to integrate the touch screen
`with other components of the computer system. The touch
`screen may also include hardened, antireflective, textured, or
`other surface layers. The inclusion, omission, or nature of
`these additional layers and components is immaterial to the
`present invention.
`
`[0038] Touch sensor 202 is connected to touch sensing
`controller 208. The nature of controller 208 depends on the
`design of touch sensor 202 and its details are immaterial to
`the present invention. Likewise, display 204 is connected to
`a suitable display controller 210, and backlight 206,
`if
`present, is connected to backlight controller 212. Each of
`controllers 208, 210, and 212 communicate with host com-
`puter 214. In an illustrative embodiment, controllers 208,
`210, and 212 are connected to a central
`touch screen
`controller 216 that connects to host computer 214 by a single
`interface 218. Interface 218 may be a mouse interface such
`as PS/2, or a general purpose peripheral interface such as the
`Universal Serial Bus (USB). USB has the advantage of high
`bandwidth and wide availability. Any of controllers 208,
`210, 212, and 216 may be implemented as chips or discrete
`components, combined onto fewer chips or one chip, inte-
`grated with assembly 200, or combined with other functions
`of host computer 214. Host computer 214 may be embodied
`
`17
`
`17
`
`

`

`US 2005/0024341 A1
`
`Feb. 3, 2005
`
`in the central processing unit of computer system 100, a
`peripheral processor such as a USB host controller, or a
`combination thereof.
`
`corner tap region that simulates a right mouse button click,
`and an icon 308 which represents a logo for the computer
`vendor.
`
`In an alternative illustrative embodiment, control-
`[0039]
`lers 208, 210, and 212 may connect to host computer 214
`through different
`interfaces. For example,
`touch screen
`controller 208 could connect as a conventional touch pad
`using a PS/2 interface, while display controller 210 and
`backlight controller 212 connect by USB or by a specialized
`display interface.
`
`[0040] Because touch screen 106 of FIG. 1 replaces a
`conventional touch pad, touch screen 106 usually serves as
`a conventional pointing device for the computer. For this
`reason, the touch screen must be able to interface to the
`computer as a conventional mouse. This is a further reason
`for interface 218 to be either a mouse interface such as PS/2,
`or a general interface such as USB that includes support for
`conventional mice. Interface 218 may also provide for an
`alternate or extended interface protocol
`that allows for
`additional information about finger activity to be commu-
`nicated to computer 214, and for computer 214 to control
`display 204 and backlight 206. This additional finger activity
`information may include the absolute location of the finger
`on the sensor surface. When appropriate driver software is
`loaded onto computer 214, the driver software can enable
`the alternate or extended interface protocol to support the
`user interface enhancements of the present invention. When
`other driver software, such as a conventional mouse or touch
`pad driver, is loaded instead, interface 218 can revert to
`mouse or touch pad compatibility using touch sensor 202 as
`a conventional touch pad, and controller 210 or 216 can
`operate the display autonomously, such as by furnishing a
`suitable default display image for display 204.
`
`[0041] When the touch screen is used as a conventional
`touch pad, finger motions on the touch sensor (e.g., in a
`cursor positioning region, which could identify a starting
`position) will typically cause corresponding motions of a
`cursor on the main display, and clicks of “mouse” buttons
`(or action control icons) 108 will typically cause special
`actions, such as selections on the main display. Tapping
`gestures may be interpreted as “mouse” clicks or other
`special actions, as disclosed in US. Pat. No. 5,543,591.
`Other gestures may also be recognized, such as scrolling
`motions as disclosed in US. Pat. No. 5,943,052. The default
`display image may include graphical icons to indicate spe-
`cial tapping or scrolling regions on the touch sensor surface
`or the default screen image may be a blank screen with only
`a manufacturer’s logo.
`
`In one embodiment, the cursor positioning region
`[0042]
`is denoted by the absence of icons for actions, other than
`cursor positioning. However, there are many different ways
`of identifying the cursor positioning region on the touch
`screen, such examples include, but are not limited to, a box
`could enclose the cursor positioning region, a shaded region
`or icon could cover the entire cursor positioning region, or
`an icon could be centered in an otherwise blank area, thus
`labeling the blank area as a cursor positioning region.
`
`[0044] Alternatively, computer system 100 of FIG. 1 can
`include a secondary pointing device, such as an isometric
`joystick located in keyboard 104 or an external mouse,
`which relieves touch screen 106 from the responsibility of
`functioning as primary pointing device in addition to its role
`as an enhanced user interface device.
`
`touch pad with default screen
`[0045] A conventional
`image is just one of several general modes of usage that are
`envisioned for the touch screen of the present invention.
`Subsequent drawing figures illustrate several other usage
`modes that employ the touch screen as a fully interactive
`input/output device to enhance the user interface of the
`:a
`computer system. These general usage modes include
`“iconic,”“auxiliary,” and “pop-up
`touch screen modes,
`each with a variety of possible applications. The same touch
`screen can operate in each of these various modes, or other
`modes, at different
`times. The different modes can also
`appear on the screen at the same time; for example, icons can
`appear in an auxiliary or pop-up image, or an auxiliary or
`pop-up image could be overlaid in a window on the iconic
`mode image instead of fully replacing that image.
`
`[0046] FIG. 4 illustrates an example of a first “iconic”
`usage mode of the touch screen. In the iconic mode, the
`screen displays an image that includes a number of small
`icons such as pictures or buttons. The touch sensor operates
`as a touch pad pointing device in iconic mode, in which
`finger motions and taps on the sensor are generally inter-
`preted the same as when the touch screen operates as a
`conventional touch pad. The screen image in iconic mode
`may include elements in common with the default image of
`FIG. 3, as the two modes operate similarly. Iconic mode will
`generally display additional icons relating to software that is
`running on the computer and other aspects of the operation
`of the computer.
`
`image 400
`In the example image of FIG. 4,
`[0047]
`includes scroll arrow icons 402 and 404 and a touch region,
`such as illustrated by corner tap icon 406 in common with
`FIG. 3. Logo 308 has been omitted from image 400 in this
`example to reduce clutter.
`In an alternate embodiment,
`non-critical graphics from the default
`image could be
`retained as a background image on which icons overlap; in
`yet another embodiment, a different image such as static or
`dynamic “wallpaper” may serve as a background image.
`
`In example image 400, additional icons have been
`[0048]
`added to represent various system status indicators and
`functions. Icon 410 defines a second touch region or a corner
`tapping region to activate the “back” function of web
`browsers and other software. As the user enables and dis-
`
`ables special tap regions and changes their assigned func-
`tions, such as by using a software control panel, the tap
`region icons such as icons 406 and 410 can appear, disap-
`pear, move, and change in shape to reflect
`the current
`settings.
`
`[0043] FIG. 3 illustrates an example default image for use
`when the touch screen is operating as a conventional touch
`pad. FIG. 3 depicts the image on the touch screen display as
`seen by the user. Image 300 includes arrow icons 302 and
`304 indicating scrolling regions, an icon 306 indicating a
`
`Icon 412 is a continuous display of the time and
`[0049]
`date. This icon would normally have no effect on the
`interpretation of finger taps within its region. Instead, a
`finger tap within its boundaries would be interpreted as a
`simulated mouse button click, just as if the tap occurred
`
`18
`
`18
`
`

`

`US 2005/0024341 A1
`
`Feb. 3, 2005
`
`away from any icon. If every icon responded specially to
`finger taps, the main function of tapping to simulate a mouse
`click would become too inconvenient
`to use. A visual
`
`convention may be used to indicate which icons represent
`tap-sensitive regions; in the example of FIG. 4, dashed lines
`426 and 428 are used to indicate these regions.
`
`Icon group 414 includes the traditional set of status
`[0050]
`icons that appear on modern portable computers, such as
`numeric keypad lock, caps lock, scroll
`lock, hard disk
`activity, battery life, and system power. By locating these
`system icons on the touch screen display,
`the system
`designer eliminates the need for the special dedicated LED
`or LCD status displays that are typically used in prior art
`computers.
`
`In some prior art portable computers, the dedicated
`[0051]
`system status displays are situated so that they are visible
`even when the cover of the computer is closed over the main
`display. The touch screen of the present invention could
`similarly be situated so that all or part of the screen image
`is visible when the cover is closed, for example, by causing
`the touch screen to protrude from under the cover or by
`cutting a notch in the cover over the location of the touch
`screen. This arrangement would allow the user to monitor
`battery recharging and other quiescent activities of the
`computer system while the computer is not in use.
`
`Icon 416 is an e-mail notification status icon; icon
`[0052]
`416 may, for example, change to a new shape or animated
`image to indicate that e-mail has arrived. Icon 418 similarly
`notifies the user of imminent appointments. These icons
`suggest a natural action that could be taken when the user
`taps on the icons, such as opening the associated e-mail
`reading or appointment scheduling software. Because these
`icons are located nearer the center of the touch sensing area
`and could easily be tapped by accident, icons 416 and 418
`may be made sensitive to finger taps only when they have
`been activated by some separate means such as pressing a
`special function key on keyboard 104.
`
`Icons 420 and 422 represent commands to select
`[0053]
`pop-up applications on the touch screen. Icon 420 selects an
`application launcher.
`Icon 422 selects a calculator or
`numeric keypad. Like icons 416 and 418, icons 420 and 422
`may be made sensitive to finger taps only when the touch
`screen is in the activated state.
`
`Icon 424 represents the volume control for the
`[0054]
`sound system and speakers of the computer.
`Icon 424
`includes a visual slider and “thumb.” The position of the
`thumb on the slider reflects the current volume setting.
`When the touch screen is in the activated state, finger
`motions within the volume control region can move the
`thumb to a different location on the slider to adjust the
`volume level. When the touch screen is not in the activated
`
`state, icon 424 is a visual display only and has no special
`interpretation when touched. Similar slider controls may be
`provided to adjust other system parameters such as the
`sound balance among several sound sources, the brightness
`and contrast of the main screen or touch screen, or the power
`management strategy.
`
`[0055] The icons depicted in FIG. 4 are illustrative of the
`types of icons that can be provided on the iconic mode
`screen. FIG. 4 does not necessarily represent
`the ideal
`selection or placement of icons. Human-factors testing may
`
`be used to decide on the number, types, and placement of
`icons in the default iconic screen. Also, it may be advanta-
`geous to allow the user to select which icons are present and
`to rearrange the icons, possibly using a software control
`panel. Because the number of candidate icons likely exceeds
`available space,
`it may be desirable to provide multiple
`iconic screen layouts selectable by some means such as in
`the software control panel or by tapping on an icon on the
`touch screen.
`
`It will be obvious to one skilled in the art that many
`[0056]
`other images, logos, status indicators, command buttons,
`controls, and other types of icons can share the touch screen
`display in the iconic usage mode. These icons can be purely
`display indicators, or they can indicate control regions that
`respond specially to finger motions and/or finger taps, either
`at all times or only when the touch screen has been activated
`in a special way. Some icons may be built-in by the system
`designer, such as the system status icons or the logo of the
`computer manufacturer. Other icons may be created and
`maintained by application software running on the com-
`puter, such as an e-mail notification icon.
`
`[0057] The activated state of the touch screen may be
`indicated by a visual convention. FIG. 5 illustrates the touch
`screen image of FIG. 4 modified to indicate the activated
`state of the touch screen using a dashed line around each
`icon that is touch-sensitive in the activated state. In image
`500, dashed lines 516, 518, 520, and 522 have surrounded
`certain icons to indicate that finger taps in the regions near
`these icons will be interpreted as special commands to the
`icons. Similarly, dashed outline 524 indicates that finger
`motions in the volume control region will adjust the setting
`of the control. Outline 512 for the time and date icon has
`
`become dashed to indicate that a tap on this icon will
`activate a special function such as setting the time or
`accessing a world clock. Outline 514 for the system status
`icons remains solid to indicate that, in the example of FIG.
`5,
`these icons have no special
`tapping functions in the
`activated state. Dashed lines 526 and 528 remain to indicate
`
`that the corner tap regions continue to have their special tap
`interpretations when the touch screen is in the activated
`state. Many other visual conventions would serve equally
`well
`to indicate touch-sensitive icons, such as solid or
`colored lines, colored or inverted backgrounds, changes in
`brightness or coloration of the activated icons, changes in
`shape or animation of the activated icons, or other well-
`known conventions for highlighting a portion of an image.
`
`[0058] The example of FIGS. 4 and 5 illustrates the same
`set of icons in the activated and unactivated state. However,
`activation of the touch screen could also create additional
`
`icons that are not present, for example to reduce clutter,
`when the touch screen is not in the activated state. Existing
`icons could also be removed or rearranged, although to
`avoid confusion, this could be done only to replace icons not
`useful in the activated state, such as icon 414 of FIG. 4, with
`other icons that are most useful when activated, such as
`icons similar to icons 420 and 422.
`
`[0059] There are many possible alternative mechanisms
`for the user to signal the activation of touch screen icons. In
`the simplest case, the icons are either always inactive or
`always active for tapping or motion commands. The corner
`tapping and scrolling region icons 302, 304, and 306 of
`FIGS. 3 and 402, 404, 406, and 410 of FIG. 4 are examples
`
`19
`
`19
`
`

`

`US 2005/0024341 A1
`
`Feb. 3, 2005
`
`of icons that are active at all times. The logo icon 308 of
`FIG. 3 and system status icons 414 of FIG. 4 are examples
`of icons that are inactive at all times. For simple touch screen
`images like that of FIG. 3, all icons may fall into these
`simple categories and no overt activation mechanism is
`needed. For more elaborate touch screen images like that of
`FIGS. 4 and 5, an overt activation mechanism is necessary
`for those icons that must respond to touch but cannot
`reasonably be made touch sensitive at all times.
`
`In an illustrative embodiment, a key on the main
`[0060]
`keyboard 104 of computer system 100 of FIG. 1 is desig-
`nated as the touch screen activation key. FIG. 6A illustrates
`a portion 600 of the keyboard featuring several keys. Most
`keys of the keyboard, such as letter “Z” key 602, have
`preassigned functions that do not overlap well with touch
`screen activation. Even the existing shifting keys such as
`shift key 604, control key 606, and Alt key 614 are not
`suitable because they are often pressed in conjunction with
`mouse clicks in application software for features such as
`extending selections; hence, it is desirable for the user to be
`able to tap normally on the touch sensor to simulate a mouse
`click while these shifting keys are pressed.
`
`[0061] Function or “Fn” key 608 is common on the
`keyboards of portable computers. This key, when held down,
`changes the interpretations of various other keys to perform
`special control functions. For example,
`in one portable
`computer, the arrow keys change to screen brightness con-
`trols, certain letter keys change to become a numeric key-
`pad, and various other keys change to control the external
`video and various other functions. The alternate “Fn” func-
`
`tions of the various keys are often indicated by blue writing
`next to the white writing ind