I 1111111111111111 11111 111111111111111 IIIII IIIII IIIII 111111111111111 11111111
`US007768501Bl
`
`c12) United States Patent
`Maddalozzo, Jr. et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,768,501 Bl
`Aug. 3, 2010
`
`(54) METHOD AND SYSTEM FOR TOUCH
`SCREEN KEYBOARD AND DISPLAY SPACE
`SHARING
`
`(75)
`
`Inventors: John Maddalozzo, Jr., Austin, TX (US);
`Gerald Francis McBrearty, Austin, TX
`(US); Johnny Meng-Han Shieh, Austin,
`TX (US)
`
`(73) Assignee: International Business Machines
`Corporation, Armonk, NY (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by O days.
`
`(21) Appl. No.: 09/071,189
`
`(22) Filed:
`
`May 1, 1998
`
`(51)
`
`Int. Cl.
`G06F 3/041
`(2006.01)
`(52) U.S. Cl. ....................... 345/173; 345/163; 345/168;
`345/174
`( 58) Field of Classification Search . ... ... .. 345/156-168,
`345/173-179; 361/680, 682,683; 178/18.01-18.07,
`178/19.01-19.05
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,267,327 A *
`5,491,495 A *
`5,543,588 A *
`5,581,243 A
`5,675,362 A *
`5,748,185 A *
`
`11/1993 Hirayama .................... 382/13
`2/1996 Ward et al. .................. 345/173
`8/1996 Bisset ......................... 178/18
`12/1996 Ouellette et al ............. 345/173
`10/1997 Clough et al ................ 345/173
`5/1998 Stephan et al. .............. 345/173
`
`5,767,457 A *
`5,777,605 A *
`5,778,404 A *
`5,838,302 A *
`5,956,020 A *
`6,029,214 A *
`
`6/1998
`7/1998
`7/1998
`11/1998
`9/1999
`2/2000
`
`............ 178/18
`Gerpheide et al.
`Yoshinobu et al. .......... 345/173
`Capps et al.
`................ 345/179
`Kuriyarna et al. ........... 345/173
`D' Amico et al. ............ 345/173
`Dorfman et al ............... 710/73
`
`* cited by examiner
`
`Primary Examiner-Vijay Shankar
`(74) Attorney, Agent, or Firm-Dillon & Yudell LLP
`
`(57)
`
`ABSTRACT
`
`A method and system in a portable computer having a display
`screen for increasing portable computer compactness. Data is
`displayed initially within the display screen. The display
`screen is then partitioned into a touch-sensitive input area and
`a display area, wherein data input at the touch-sensitive input
`area may be simultaneously displayed in the display area, in
`response to a particular user input. A test is performed to
`detect if a user's hands are positioned at the touch-sensitive
`input area. A touch-sensitive pad is thereafter graphically
`displayed at the touch-sensitive input area within the display
`screen, in response to detecting a user's hands positioned at
`the touch-sensitive area, wherein a user may enter data that
`may be simultaneously displayed in the display area. An
`additional test may then be performed to detect if the user's
`hands are no longer positioned at the touch-sensitive input
`area. The touch-sensitive pad is subsequently concealed from
`view, in response to detecting if the user's hands are no longer
`positioned at the touch-sensitive input area. The touch-sensi(cid:173)
`tive pad graphically displayed at the touch-sensitive input
`area within the display screen may be composed of a touch(cid:173)
`sensitive keyboard, which may in and of itself be an ergo(cid:173)
`nomic keyboard.
`
`21 Claims, 8 Drawing Sheets
`
`82
`
`1 2~8 g
`Dear Sir,
`This letter is in reply to your pre 10us etter. At
`this time we are interested in purchasing o iers from
`your fnm. Our purchasing department w?\ in cohtact
`with you about this issue in the near futur\_.12
`8 5~ :-\'
`87
`
`90
`
`8 9~~ '
`Vc'01
`
`83
`
`84
`
`86
`
`92
`
`88
`
`Page 1 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`U.S. Patent
`
`Aug. 3, 2010
`
`Sheet 1 of 8
`
`US 7,768,501 Bl
`
`46
`
`Pig. 1
`
`Page 2 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`"'""' = "'""'
`tit =
`
`00
`0--,
`~
`-....l
`d r.,;_
`
`0 ....
`N
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`QO
`
`N
`~
`~
`~
`
`~
`
`0 ....
`
`0
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`Ptd, 2
`
`48
`
`23
`
`26
`
`50
`
`32
`
`PROGRAM
`CONTROL IJ51
`
`MAIN MEMORY
`
`USER INTtRFACE
`
`ADAPTER
`
`40
`
`ADAPTER
`DISPLAY
`
`37
`
`COMMUNICATIONS
`
`ADAPTER
`
`ADAPTER
`
`1/0
`
`RAM
`
`ROM
`
`CPU
`
`49
`
`NETWORK
`
`39
`
`DISK/TAPE
`
`DRIVE
`
`34
`
`36
`
`31
`
`2c
`
`Page 3 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`U.S. Patent
`
`Aug. 3, 2010
`
`Sheet 3 of 8
`
`US 7,768,501 Bl
`
`Page 4 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`"'""' = "'""'
`tit =
`
`00
`0--,
`~
`-....l
`
`d r.,;_
`
`tFziJ. 4
`
`96
`
`83" )
`
`0 ....
`.....
`rJJ =(cid:173)
`
`QO
`
`.i;...
`
`('D
`('D
`
`N
`~
`~
`~
`
`~
`
`0 ....
`
`0
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`88
`
`92
`
`86
`
`84
`
`90
`
`with you about this issue in the near future.
`your fum. Our purchasing departtnent will be in contact
`this time we are interested in purchasing copiers from
`This letter is in reply to your previous letter. At
`
`Dear Sir,
`
`94
`
`B2
`
`80~
`
`Page 5 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`"'""' = "'""'
`tit =
`
`00
`0--,
`~
`-....l
`
`d r.,;_
`
`0 ....
`Ul
`.....
`rJJ =- ('D
`
`('D
`
`QO
`
`0
`
`N
`~
`
`~
`
`0 ....
`> = ~
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`88
`
`92
`
`86
`
`84
`
`96
`
`9o
`
`94
`
`y~ur fnm. Our p~~hasin~ department wu;· contact
`
`.
`
`thi
`
`tFzif 5
`
`83
`
`87
`
`85~ :'\
`
`Vc'a1
`89) r-0
`~ :'\'
`
`with you about this issue m the near futur
`
`s time we are tntereste m pure asmg o 1ers ott1
`.
`This letter is in reply to your pre mus etter. At
`
`fr
`
`·
`
`.
`
`h
`
`d'
`
`1 2~8 9
`
`Dear Sir~
`
`82
`
`80~
`
`Page 6 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`"'""' = "'""'
`tit =
`
`00
`0--,
`~
`-....l
`
`d r.,;_
`
`Pig. 6
`
`I
`
`95J
`
`0 ....
`O'I
`.....
`=- ('D
`
`('D
`
`rJJ
`
`QO
`
`N
`~~
`~
`~
`
`0 ....
`
`0
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`88
`
`92
`
`86
`
`90
`102
`
`84
`
`. :ms letter i~ in reply ~o your pre~o!18 l~tter. At 1 o 6
`
`your firm. Our purchasing department will be in contact
`this time we are mterested m purchasin~op1ers from
`
`108
`
`82
`
`80~
`
`8 3"\ )
`
`y
`
`Page 7 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`"'""' = "'""'
`tit =
`
`00
`0--,
`~
`-....l
`d r.,;_
`
`(Fig. 7
`
`I
`
`95J
`
`8 3" )
`
`0 ....
`-....J
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`QO
`
`N
`~
`~
`~
`
`~
`
`0 ....
`
`0
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`88
`
`92
`
`86
`
`90
`102
`
`84
`
`copiers from your firm. Our purchasing department w111
`this time we are interested in purchasing ten industria
`
`This letter is in reply to your previous letter.
`
`82
`
`80~
`
`Page 8 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`U.S. Patent
`
`Aug. 3, 2010
`
`Sheet 8 of 8
`
`US 7,768,501 Bl
`
`START
`
`142
`
`,-[140
`
`144
`
`146
`
`USER PLACES HANDS
`AT TOUCH SCREEN
`DISPLAY
`
`ER'S HAN
`ETECTED
`
`YES
`
`NO
`
`"M"
`PICKED
`
`150
`
`152
`
`1 54
`
`156
`
`INITIATE
`PARTITIONING OF
`TOUCH SCREEN
`DISPLAY (INITIATE
`KEYBOARD MODE)
`
`PARTITION INTO
`TOUCH SENSITIVE
`INPUT AREA
`
`DISPLAY
`MINI-WINDOW
`AND PORTION OF
`DOCUMENT WITHIN
`MINI-WINDOW
`
`MANI PU LA TE WITH
`THE MOUSE IN
`THE MOUSE MODE
`
`NO
`
`NO
`
`UK"
`PICKED
`
`DISPLAY MODE
`
`166
`
`END
`
`168
`
`Pig. 8
`
`Page 9 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`US 7,768,501 Bl
`
`1
`METHOD AND SYSTEM FOR TOUCH
`SCREEN KEYBOARD AND DISPLAY SPACE
`SHARING
`
`BACKGROUND OF THE INVENTION
`
`5
`
`2
`It is another object of the invention to provide improved
`information-retrieval methods and systems utilized in asso(cid:173)
`ciation with graphical user interface devices.
`It is yet another object of the invention to provide an
`improved portable computer system.
`It is still another object of the invention to provide an
`improved touch screen display unit which may be utilized in
`association with information-retrieval systems.
`It is another object of the invention to provide a method and
`10 system for promoting increased compactness in portable
`computer systems by reducing the space consumed by dis(cid:173)
`play screens and user input devices utilized in such computer
`systems.
`The above and other objects are achieved as is now
`15 described. A method and system are disclosed in a portable
`computer having a display screen for increasing portable
`computer compactness. Data is displayed initially within the
`display screen. The display screen is then partitioned into a
`touch-sensitive input area and a display area, wherein data
`20 input at the touch-sensitive input area may be simultaneously
`displayed in the display area, in response to a particular user
`input. A test is performed to detect if a user's hands are
`positioned at the touch-sensitive input area. A touch-sensitive
`pad is thereafter graphically displayed at the touch-sensitive
`25 input area within the display screen, in response to detecting
`a user's hands positioned at the touch-sensitive area, wherein
`a user may enter data that may be simultaneously displayed in
`the display area.
`An additional test may then be performed to detect if the
`user's hands are no longer positioned at the touch-sensitive
`input area. The touch-sensitive pad is subsequently concealed
`from view, in response to detecting if the user's hands are no
`longer positioned at the touch-sensitive input area. The touch-
`35 sensitive pad graphically displayed at the touch-sensitive
`input area within the display screen may be composed of a
`touch-sensitive keyboard. Thus, when the user places his or
`her hands at the touch-sensitive area, the transparent key-
`board appears at the touch-sensitive area within the display
`screen. When the user removes his or hands from the touch(cid:173)
`sensitive input area, the transparent keyboard vanishes. The
`keyboard itself can be an ergonomic keyboard.
`In addition, physical characteristics associated with the
`user can be analyzed, while the user is entering a particular
`sequence of data utilizing the touch-sensitive keyboard, in
`order to customize the keyboard according a user's physical
`characteristics and personal preferences. In response to ana(cid:173)
`lyzing such physical characteristics, the touch-sensitive key(cid:173)
`board is configured so that the sensitivity of the touch-sensi(cid:173)
`tive keyboard may be raised or lowered according to the
`physical characteristics associated with the user.
`
`30
`
`1. Technical Field
`The present invention relates to improved information(cid:173)
`retrieval methods and systems. In particular, the present
`invention relates to improved information-retrieval methods
`and systems utilized in association with graphical user inter(cid:173)
`face devices. More particularly, the present invention relates
`to portable computers. Still more particularly, the present
`invention relates to touch screen display units. Finally, the
`present invention relates to methods and systems for saving
`space in touch screen keyboard and display devices.
`2. Description of the Related Art
`Touch screen technology is increasingly being utilized in
`all facets of data-processing applications. A touch screen is a
`display device that allows users to interact with a computer
`system by touching a particular area on the surface of the
`screen. Touch screen technology can be implemented by any
`of a number of sensing techniques. For example, touch screen
`technology can be implemented by detecting disturbances on
`a screen utilizing the fingers of a user. Typically, this distur(cid:173)
`bance is detected through the interruption or detection of heat
`on the screen.
`Touch screen technology is increasingly being imple(cid:173)
`mented in association with portable computers, such as so(cid:173)
`called "laptop" and "notebook" computers, and small calcu(cid:173)
`lator-size mini-portable computers. As devices become
`increasingly portable in nature, the need for smaller and more
`compact devices increases. Thus, a problem encountered in
`decreasing the size of portable computers is the need to save
`space.
`For example, in most portable computers, two distinct
`spaces are required. First, a panel for the display itself and
`second, a section for the computer keyboard. These two
`requirements add weight, cost and size to the design of a 40
`laptop computer. Touch screen technology offers an alterna(cid:173)
`tive approach around such spatial restrictions. However, to
`date, touch screen technology usually includes keyboards that
`occupy distinct areas away from the display, or when com(cid:173)
`bined with the display, are uncomfortable and awkward for 45
`the user to access. An example of a touch screen display can
`be found in U.S. Pat. No. 5,581,243, "Method and Apparatus
`for Displaying Simulated Keyboards on Touch-Sensitive Dis(cid:173)
`lays," by Ouellette et al. Such touch screen keyboards, while
`advancing the art, do not permit users to access touch screen 50
`keyboards in an efficient and versatile manner. The challenge
`lies in crafting a portable computer in which compactness and
`versatility is improved by incorporating touch screen key(cid:173)
`boards that are easy to access and use.
`Based on the foregoing, it can be appreciated that a need 55
`exists to reduce the size of portable computers, which in turn
`saves on the parts necessary to construct such computers. As
`a result of reducing the size of the touch screen, portable
`computers which utilize touch screen technology may be
`constructed with greater compactness and efficiency. It is 60
`believed that the disclosure provided addresses and solves
`these problems in a unique and previously unknown manner.
`
`SUMMARY OF THE INVENTION
`
`It is therefore an object of the invention to provide an
`improved information-retrieval method and system.
`
`65
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The novel features believed characteristic of the invention
`are set forth in the appended claims. The invention itself,
`however, as well as a preferred mode of use, further objects,
`and advantages thereof, will best be understood by reference
`to the following detailed description of an illustrative
`embodiment when read in conjunction with the accompany(cid:173)
`ing drawings, wherein:
`FIG. 1 illustrates a pictorial representation of a computer
`system, which may be utilized to implement a preferred
`embodiment of the present invention;
`FIG. 2 depicts a representative hardware environment of a
`computer system, which may be utilized to implement a
`preferred embodiment of the present invention;
`
`Page 10 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`US 7,768,501 Bl
`
`3
`FIG. 3 illustrates a pictorial representation of a prior art
`touch screen display;
`FIG. 4 depicts a pictorial representation of a document
`displayed within a graphical user interface window in a nor(cid:173)
`mal display mode, in accordance with a preferred embodi(cid:173)
`ment of the present invention;
`FIG. 5 illustrates a pictorial representation of a document
`displayed within a graphical user interface window in a
`mouse mode, in accordance with a preferred embodiment of
`the present invention;
`FIG. 6 depicts a pictorial representation of graphical user
`interface window displayed in a keyboard mode, in accor(cid:173)
`dance with a preferred embodiment of the present invention;
`FIG. 7 illustrates a pictorial representation of graphical
`user interface window displayed in a keyboard mode follow(cid:173)
`ing manipulation of a document by a graphical keyboard, in
`accordance with a preferred embodiment of the present
`invention; and
`FIG. 8 depicts a high-level flowchart of operations illus(cid:173)
`trating a method for keyboard and display space saving in a
`touch screen display, in accordance with a preferred embodi(cid:173)
`ment of the present invention.
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENT
`
`With reference now to the figures and in particular with
`reference to FIG. 1, there is depicted a pictorial representation
`of a computer system, which may be utilized to implement a
`preferred embodiment of the present invention. A computer 30
`system 20 is depicted that includes a system unit 21, speakers
`46 and a video display 23. Computer system 20 includes a
`graphical user interface that resides within a machine-read(cid:173)
`able media to direct the operation of computer system 20.
`Those skilled in the art will appreciate that the method and 35
`system of the present invention apply equally to any computer
`system, regardless of whether the computer system is a com(cid:173)
`plicated multi-user computing apparatus or a single-user
`workstation. A touch screen display is utilized in a 5 associa(cid:173)
`tion with computer system 20 to convey visual data and 40
`receive input from a user. Such a touch screen may also utilize
`or incorporate graphical user interface features.
`FIG. 2 depicts a representative hardware environment of
`computer system 20 of FIG. 1, which may be utilized to
`implement a preferred embodiment of the present invention. 45
`System unit 21 of FIG. 1 includes a Central Processing Unit
`("CPU") 31, such as a conventional microprocessor, and a
`number of other units interconnected via system bus 32.
`Computer system 20 includes random-access memory
`("RAM") 34, read-only memory ("ROM") 36, display 50
`adapter 37 for connecting system bus 32 to video display 23,
`and I/O adapter 39 for connecting peripheral devices such as
`disk and tape drives 33 to system bus 32.
`Disk and tape drives 33 are electro-mechanical devices that
`read from and write to disks. The main components of a disk 55
`drive include a spindle on which the disk is mounted, a drive
`motor that spins the disk when the drive is in operation, one or
`more read/write heads that perform the actual reading and
`writing, a second motor that positions the read/write heads
`over the disk, and controller circuitry that synchronizes read/ 60
`write activities and transfers information to and from com(cid:173)
`puter system 20. A disk itself is typically a round, flat piece of
`flexible plastic ( e.g., floppy disk) or inflexible metal ( e.g. hard
`disk) coated with a magnetic material that can be electrically
`influenced to hold information recorded in digital (i.e., 65
`binary) form. A disk is, in most computers, the primary
`method for storing data on a permanent or semipermanent
`
`4
`basis. Because the magnetic coating of the disk must be
`protected from damage and contamination, a floppy (e.g.,
`5.25 inch) disk or micro-floppy (e.g., 3.5 inch) disk is encased
`in a protective plastic jacket. A hard disk, which is very finely
`5 machined, is typically enclosed in a rigid case and can be
`exposed only in a dust free environment.
`Video display 23 is the visual output of computer system
`20. Video display 23 can be, for example, a CRT-based video
`display well-known in the art of computer hardware. "CRT"
`10 is an acronym for cathode-ray tube. Video display 23 can also
`be implemented as an LCD-based or a gas plasma-based
`flat-panel display. "LCD" is an acronym for liquid crystal
`display. Those skilled in the art can thus appreciate that com(cid:173)
`puter system 20 may be modified to be implemented as a
`15 notebook-based computer. Computer system 20 may further
`include user interface adapter 40 for connecting keyboard 26,
`mouse 28, speaker 46 (i.e., analogous to speakers 46 of FIG.
`1), microphone 48, and/or other user interface devices to
`system bus 32. Communications adapter 49 connects com-
`20 puter system 20 to a computer network. Although computer
`system 20 is shown to contain only a single CPU and a single
`system bus, it should be understood that the present invention
`applies equally to computer systems that posses multiple
`CPUs and to computer systems that include multiple buses
`25 that each perform different functions in different ways.
`Computer system 20 also includes a graphical user inter(cid:173)
`face that resides within a machine-readable media to direct
`the operation of computer system 20. Any suitable machine(cid:173)
`readable media may retain the graphical user interface, such
`as RAM 34, ROM 36, a magnetic diskette, magnetic tape, or
`optical disk (the last three being located in disk and tape
`drives 33). Any suitable operating system and associated
`graphical user interface ( e.g., Microsoft Windows) may direct
`CPU 31. For example, the AIX operating system and AIX(cid:173)
`windows windowing system (i.e., graphical user interface)
`can direct CPU 31. The AIX operating system is IBM's
`implementation of the UNIX operating system. UNIX is a
`trademark of UNIX Systems Laboratories, Inc. Other tech(cid:173)
`nologies can also be utilized in association with CPU 31, such
`as touch-screen technology or human voice control. Those
`skilled in the art will appreciate that the hardware depicted in
`FIG. 2 may vary for specific applications. For example, other
`peripheral devices such as optical disk media, audio adapters,
`or chip-programming devices, such as PAL or EPROM pro(cid:173)
`gramming devices well-known in the art of computer hard(cid:173)
`ware, and the like may be utilized in addition to or in place of
`the hardware already depicted.
`Main memory 50 is connected to system bus 32, and
`includes a control program 51 that resides within main
`memory 50 and contains instructions that when executed on
`CPU 31, carry out the operations depicted in the logic flow
`chart described herein. The computer program product can
`also be referred to as a program product. It is important that,
`while the present invention has been ( and will continue to be)
`described in the context of a fully functional computer sys(cid:173)
`tem, those skilled in the art will appreciate that the present
`invention is capable of being distributed as a program product
`in a variety of forms, and that the present invention applies
`equally regardless of the particular type of signal-bearing
`media utilized to actually carry out the distribution.
`Examples of signal-bearing media include: recordable(cid:173)
`type media, such as floppy disks, hard disk drives and CD
`RO Ms, and transmission-type media such as digital and ana(cid:173)
`log communication links. Examples of transmission-type
`media include devices such as modems. A modem is a type of
`communications device that enables a computer to transmit
`information over a standard telephone line. Because a com-
`
`Page 11 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`US 7,768,501 Bl
`
`5
`puter is digital (i.e., works with discrete electrical signals
`representative of binary 1 and binary 0) and a telephone line
`is analog (i.e., carries a signal that can have any of a large
`number of variations), modems can be utilized to convert
`digital to analog and vice-versa. The term "media" as utilized
`herein is a collective word for the physical material such as
`paper, disk, CD-ROM, tape and so forth, utilized for storing
`computer-based information.
`FIG. 3 illustrates a pictorial representation of a prior art
`touch screen display 60. Touch screen display 60 is a special(cid:173)
`ized computer display screen which is designed to recognize
`the location of a touch on the surface of the screen (i.e. touch
`screen 63). By touching the screen, the user can make a
`selection or move a cursor. Although not depicted in FIG. 3,
`those skilled in the art will appreciate that touch screen dis(cid:173)
`play 60 may be implemented on a portable computer display,
`such as the compact computer system illustrated in FIG. 1.
`Display 23 of FIG. 1 may be implemented as a compact
`version of touch screen display 60 of FIG. 3.
`Touch screen 63 can be composed of one or more sensing 20
`line grids, which determine the location of a touch by match(cid:173)
`ing vertical and horizontal contacts. In addition, touch screen
`display may include an electrically charged surface and sen(cid:173)
`sors around the outer edge of the screen to detect the amount
`of electrical disruption and pinpoint exactly where contact
`has been made. Touch screen display 60 may also include
`infrared light-emitting diodes (LEDs) around the outer edges
`of the screen (i.e., touch screen 63), which together with
`additional sensors, create an invisible infrared grid, which the
`user's finger interrupts, in front of the screen.
`Thus, according to the diagram depicted in FIG. 3, a user's
`hand 64 (i.e., particularly a user's fingers) may touch keys 62,
`which are graphically displayed within touch screen display
`60. By touching a particular key, such as a key 62, a series of
`data-processing operations associated with that particular
`key, may be initiated. Touch screen 63 of touch screen 60 is
`thus a display device that allows the user to interact with a
`computer system, such as computer system 20 of FIG. 1 and
`FIG. 2, by touching a particular area on the screen.
`A preferred embodiment of the present invention calls for 40
`a merging of the display area and a keyboard input area
`utilizing touch screen technology. When a user places his or
`her hands over the bottom half or two-thirds of the touch
`screen display, a translucent keyboard appears under his or
`her fingers. The current text or graphics displayed on the
`touch screen display will still be seen through the keyboard.
`However, when the user touches the graphically displayed
`keyboard "keys," keyboard input is reflected onto the screen
`immediately behind the graphically displayed keyboard. The
`keyboard is in essence a "phantom" keyboard, in that it
`appears and vanishes depending upon the presence of a par(cid:173)
`ticular user input, such as the placement of a user's hands near
`a designated area on the touch screen. When graphically
`displayed on the screen, the keyboard is transparent, such that
`any data ( e.g., words or paragraphs) that were displayed prior
`to the display of the transparent keyboard are visible "below"
`the keyboard.
`In FIG. 4, FIG. 5, and FIG. 6, and FIG. 7 similar parts are
`indicated by identical reference numerals. FIG. 4 depicts a
`pictorial representation of a document 82 displayed within a
`graphical user interface window 83 in a normal display mode,
`in accordance with a preferred embodiment of the present
`invention. The graphical user interface, as described previ(cid:173)
`ously, is a type of computer interface composed of a visual
`metaphor reflecting a real-world scene, often of a desktop.
`Within this scene are icons, representing actual objects, that
`the user may access and manipulate with a pointing device.
`
`6
`Window 80 includes specific icons, which when manipulated
`by a user via a pointing device, allow a user to scroll up or
`down through document 82. For example, down-arrow 96
`allows a user to scroll downward, while up-arrow 94 allows a
`5 user to scroll upward through document 82.
`Down-arrow 96, up-arrow 94 and slider 90 are positioned
`within scroll bar 92. Scroll bar 92 is a graphical device that
`can be utilized to change a user's viewpoint of a list or data
`file. The user alters the view by sliding slider 90 up or down
`10 in the scroll area or by pressing one of the scroll arrows (i.e.,
`down-arrow 96 or up-arrow 94), which causes the view to
`scroll up or down in the window area adjacent the scroll bar.
`Window 83 additionally includes an area in which "M," "K,"
`and "D" graphical buttons are located (i.e., "M" button 84,
`15 "K" button 86, and "D" button 88). As described herein, "M"
`button 84 initiates a "mouse" mode, "K" button 86 initiates a
`"keyboard" mode, and "D" button 88 initiates a normal dis(cid:173)
`play mode. As indicated by the shaded "D" button 88, win-
`dow 83 of FIG. 4 depicts a normal display mode.
`FIG. 5 illustrates a pictorial representation of document 82
`displayed within graphical user interface window 83 in a
`mouse mode, in accordance with a preferred embodiment of
`the present invention. Cursor 120 indicates a position within
`document 82 from which a user might begin editing docu-
`25 ment 82. The document displayed within window 83 of FIG.
`5 is in a "mouse mode, as indicated by shaded "M" button 84.
`The user may manipulate document 82 utilizing a pointing
`device such as a mouse. Because the graphical user interface
`depicted in FIG. 5 is implemented on a touch screen device,
`30 the user merely places an index finger at mouse pointer 89 and
`moves the mouse pointer toward the desired area. Movement
`of the mouse pointer is indicated in FIG. 5 by curved lines 85.
`Circle 87 indicates the actual position of a user's finger on the
`touch screen as the mouse pointer 89 is moved toward cursor
`35 120.
`FIG. 6 depicts a pictorial representation of graphical user
`interface window 83 displayed in a keyboard mode, in accor(cid:173)
`dance with a preferred embodiment of the present invention.
`FIG. 7 illustrates a pictorial representation of graphical user
`interface window 83 displayed in a keyboard mode following
`manipulation of document 82 by keyboard 110, in accordance
`with a preferred embodiment of the present invention. The
`keyboard mode is indicated by shaded "K" button 86. When
`the graphical user interface window functions in a keyboard
`45 mode, a user can place his or her hands over the bottom half
`or two-thirds of display screen 80. Touch screen technology
`detects the presence of the user's hands in these areas.
`Thus, when the user's hands are detected, a translucent or
`transparent keyboard 110 appears beneath the user's fingers
`50 on display screen 80. Although not depicted specifically in
`FIG. 6 and FIG. 7, current text, graphics or other documents
`will still be visible through the keyboard. When the user
`touches the "keys" displayed on display screen 83, keyboard
`input is reflected onto the screen beneath transparent key-
`55 board 110. Those skilled in the art will appreciate that the
`circles depicted on transparent keyboard 110 indicate where a
`user's hands can be positioned to manipulate the keyboard.
`For example, a user's left hand fingers may be positioned
`respectively on the "A", "S", "D," "F" and space bar keys,
`60 while a user's right hand fingers may be positioned at the";",
`"L", "K", "J" and space bar keys. The larger circles depicted
`in FIG. 6 and FIG. 7 indicate the position of the user's palms.
`If the user takes his or hands away from the touchscreen
`surface, the projected keyboard fades away. If a user's hand is
`65 then detected at the touchscreen, the keyboard reappears. This
`hide and display action continues until an option other than a
`"K" mode is selected.
`
`Page 12 of 16
`
`GOOGLE EXHIBIT 1006
`
`

`

`US 7,768,501 Bl
`
`20
`
`7
`Because the user must also be able to see the actions asso(cid:173)
`ciated with his or her typing, a smaller "active" window 100,
`is displayed to indicate the actual typing taking place. Win(cid:173)
`dow 100 includes a scroll bar 102 having a slider 106, an
`up-arrow 108, and a down-arrow 104 which functions similar
`to the scroll bar, slider and arrows associated with window 83.
`Of course, as indicated previously, the actions performed via
`keyboard input are simultaneously displayed in the actual text
`that resides "beneath" keyboard 110, as well as in window
`100. Window 100 thus includes a document 101, which is in
`essence, a smaller version of document 82.
`If the user has the need to manipulate document 82 with a
`mouse, the user simply moves his or her hands away from the
`keyboard, and touches "M" button 84. A mouse pointer ( e.g.,
`mouse pointer 89) subsequently appears and transparent key(cid:173)
`board 110 vanishes from sight. The user can then manipulate
`the mouse or other pointing device for object selection or
`drag. When the user finishes this task, the user then touches
`"K" button 86, which places the display screen back into the
`keyboard mode. When the user's hands approach display
`screen 80 (i.e., when the presence of the user's hands at the
`display screen are detected), the keyboard materializes on the
`display screen beneath his or her fingers. Various levels of
`translucency can be specified by the user.
`Thus, transparent keyboard 110 is that part of computer 25
`system 20 of FIG. 1 that resembles a typewriter keyboard and
`which enables a user to control particular aspects of the com(cid:173)
`puter. Transparent keyboard 110 can include a standard set of
`printable characters presented in a QWERTY pattern typical
`of most typewriters. In addition, transparent keyboard 110 30
`may include a calculator-like numeric keypad at one side and
`additional specialized keys. Some of these keys, such as the
`"Control," "Alt," and "Shift" keys may be utilized to change
`the meaning of another key. Other special keys and combina(cid:173)
`tions of keys can be utilized to control program operations or 35
`to move either displayed text or cursors.
`FIG. 8 depicts a high-level flowchart of operations 140
`illustrating a method for keyboard and display space saving in
`a touch screen display, in accordance with a preferred
`embodiment of the present invention. It can be appreciated by 40
`those skilled in the art that FIG. 8 presents a self-consistent
`sequence of steps leading to a desired result. The steps are
`those requiring physical manipulation of physical quantities.
`Usually, although not necessarily, these qua