Ulllted States Patent [19]
`Wolfe et al.
`
`US006037930A
`[11] Patent Number:
`[45] Date of Patent:
`
`6,037,930
`Mar. 14, 2000
`
`[54] MULTIMODAL TOUCH SENSITIVE
`PERIPHERAL DEVICE
`
`[75] Inventors; Andrew Lawrence Wolfe, Monroev?le,
`
`3,364,473
`3,422,418
`3,648,270
`4,088,904
`
`1/1968 ReitZ et al. ........................... .. 340/720
`1/1969 Simoneau
`.. 340/720
`3/1972 MetZ et al. ............................ .. 340/711
`5/1978 Green ................................ .. 340/365 C
`
`
`
`Pa‘; Gary Md
`
`Barrett, Annapolis,
`
`4,158,759
`
`
`
`Yeschick .. 6/1979 Mason ..
`
`340/711
`
`[73] Assignee; The Whitaker Corporation,
`Wilmington DeL
`,
`
`[21] Appl- No" 08/839,081
`[22]
`Filed:
`Apr. 23, 1997
`
`Related US. Application Data
`
`.
`.
`.
`.
`[63] Continuation of application No. 08/539,269, Oct. 11, 1995,
`abandoned’
`is a Continuation of application I\IO~
`08/261,211, Jun. 15, 1994, abandoned, which is a continu
`ation of application No. 07/614,022, Nov. 16, 1990, aban
`doned, which is a continuation of application No. 07/430,
`961, Nov. 1, 1989, abandoned, which is a continuation of
`application No. 07/056,309, Jun. 1, 1987, abandoned, which
`is a continuation of application No. 06/904,752, Sep. 5,
`1986, abandoned, which is a continuation of application No.
`06/675,658, Nov. 28, 1984, abandoned.
`
`[
`l
`Int. Cl.7 ..................................................... .. G09G 5/00
`51
`[52] us. Cl. .................. .. 345/174; 178/1805; 178/2001
`[58] Field of Search ................................... .. 345/156, 157,
`345/159, 160, 161, 163, 173, 174; 178/18,
`18_01_20_04; 341/20, 26, 33; 273/148 B
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`1/1980 HoltZ ................................. .. 340/365 C
`4,186,392
`5/1983 Slater .................................... .. 340/712
`4,386,232
`7/1983 Irie et a1.
`. 340/365 (3
`4,394,087
`4,486,741 12/1984 NoZaWa et al. .
`.. 340/365 S
`4,649,499
`3/1987 Sutton et al.
`340/712
`4,763,117
`8/1988 Blattner et al. ....................... .. 340/712
`
`FOREIGN PATENT DOCUMENTS
`
`0142132 12/1978 Japan ................................... .. 340/709
`
`0208641 11/1984 Japan ............... ..
`.. 340/712
`_
`_
`2139762 11/1984 UIllted Kingdom ................. .. 340/709
`
`Primary Examiner—Richard A, Hjerpe
`~
`-
`1V1
`'
`ff Ltd
`2885mm ‘Examiner F. ' gahlyar & W. t
`tmrney’ gem’ Or Wm anner
`1 CO ’
`'
`
`[57]
`
`ABSTRACT
`
`P
`g
`The resent invention is to a touch sensitive intelli ent
`multimedal peripheral device Which includes a touch S99
`Sitive pad to Which the Operator applies a light pressure in 9
`prescribed manner to effect cursor movement on a CRT
`screen or to input data to a computer. The device has four
`operating modes Which are de?ned as the mouse mode, the
`absolute mode, the joystick mode and the numeric keypad
`mode.
`
`Re. 28,238 11/1974 Koster ................................... .. 340/720
`
`15 Claims, 15 Drawing Sheets
`
`MOUSE
`MODE
`
`I
`
`A
`
`ABSOLUTE
`MODE
`x1
`
`x2
`A
`
`JOYSTICK
`MODE
`
`__ _’
`
`\1/
`/ v \
`A
`
`x
`
`x1
`
`B
`
`1X1
`
`/
`,/’/
`x1’
`\\‘\
`
`“ “(2
`
`B
`
`x
`
`NUMERIC KEYPAD MODE
`
`B
`
`9 a 1 +
`6 5 4 -
`a 2 1 x
`O . = +
`
`9 8 H31
`s 5 4%
`a 2 1
`FOTO'ITI
`
`TCL EXHIBIT 1023
`Page 1 of 22
`
`

`
`TCL EXHIBIT 1023
`Page 2 of 22
`
`

`
`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 2 0f 15
`
`6,03 7,930
`
`F4
`
`TOUCH
`SENSOR
`ONIOFF
`DETECTOR
`
`/\1
`
`TOUCH
`SENSOR
`
`/\2
`
`3
`
`SWITCH
`NETWORK
`
`SCALING
`CIRCUIT
`
`MODE/ACTION
`SWITCHES
`
`ND
`
`LEDS
`
`F 8
`
`MICROCONTROLLER
`
`/‘ 9
`
`/—1 0
`
`INTERFACE
`
`COMPUTER
`
`F I G 2
`
`SW3
`LED 1
`
`SW2
`LED 2
`
`SW0
`LED 4
`
`SW1
`LED 3
`{
`TOP
`
`SW4
`
`sw5
`
`BOTI'OM
`
`BOTTOM
`
`TOP
`%
`
`FIG. 3
`
`Y_
`
`X+
`
`Y+
`
`_)S'
`
`TCL EXHIBIT 1023
`Page 3 of 22
`
`

`
`U.S. Patent
`
`Mar. 14,2000
`
`Sheet 3 0f 15
`
`6,037,930
`
`VREF
`
`Q1
`
`BOTTOM LAYER
`
`P
`
`Y COORDINATE
`SIGNAL
`
`VREF
`
`MULTIPLEXER
`
`+5
`:5
`XIY
`
`XIY COORDINATE
`SIGNAL
`
`VREF
`
`Q2
`
`TOP LAYER
`_______1_—
`p
`
`Q4_€
`:-
`
`X COORDINATE
`
`SIGNAL
`FIG. 6
`
`TCL EXHIBIT 1023
`Page 4 of 22
`
`

`
`U.S. Patent
`
`Mar. 14,2000
`
`Sheet 4 0f 15
`
`6 037 930
`’
`’
`
`VREF
`
`Q1
`ON
`
`R1 g
`
`R2
`P
`»——~W'
`
`-
`
`R3 E
`
`Q4
`OFF
`
`ON
`
`T
`
`FIG. 7A
`
`VREF
`
`Q2
`0N
`
`x COORDINATE
`SIGNAL
`
`R7
`
`R4 2’
`
`R5
`P
`"———~\M
`
`‘
`
`,
`
`R6
`
`Q3
`OFF
`
`Y COORDINATE
`SIGNAL
`
`‘
`
`Ra
`
`?? ON
`
`_
`
`FIG. 7B
`
`TCL EXHIBIT 1023
`Page 5 of 22
`
`

`
`U.S. Patent
`
`Mar. 14,2000
`
`Sheet 5 0f 15
`
`6,037,930
`
`(
`
`BEGIN
`
`X
`
`1
`
`INITIALIZATION V2
`
`FLASH LIGHTS
`TO SIGNAL
`USER - OK
`
`3
`
`READ MODE
`FROM HOST
`COMPUTER
`
`5
`
`o]
`
`1
`
`2,7 |
`
`3
`
`4,6
`
`5
`
`RESET
`LIGHTS
`
`\6
`
`MOUSE
`MODES
`
`JOYSTICK
`MODES
`
`L7
`
`L’ 8
`
`Y
`
`\
`
`9
`F
`
`1
`/' °
`
`SET & LOCK
`LIGHTS
`
`ABSOLUTE
`MODES
`
`11
`/
`
`KEYPAD
`MODES
`
`FIG. 8
`
`TCL EXHIBIT 1023
`Page 6 of 22
`
`

`
`U.S. Patent
`
`Mar. 14,2000
`
`Sheet 6 0f 15
`
`6,037,930
`
`RESET
`LIGHTS
`
`I
`
`FIG. 9
`
`SET & LOCK
`LIGHTS
`
`-
`
`FIG. 10
`
`r1
`
`CLEAR LIGHT
`CONTROL
`FLAG
`
`U
`
`READ NOW
`LIGHT SETTINGS
`FROM HOST
`
`SET LIGHTS
`TO NEW
`SETTINGS
`
`/“ 3
`SET LIGHT
`CONTROL
`FLAG
`
`U
`
`TCL EXHIBIT 1023
`Page 7 of 22
`
`

`
`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 7 0f 15
`
`6,037,930
`
`MOUSE
`MODES
`
`__—>
`
`LIGHT
`CONTROL
`FLAG
`
`SET
`
`CLEAR
`
`SET LIGHTS
`TO
`REPRESENT
`MODE
`
`OBTAIN VALID
`FILTERED
`COORDINATE FROM
`PAD
`
`3
`
`SCALE x a Y
`COORDINATES
`
`__/
`
`SEND HOST
`"PAD RELEASED
`MESSAGE
`
`PAD IS
`BEING TOUCHED
`NOW
`
`FIG. 11A
`
`TCL EXHIBIT 1023
`Page 8 of 22
`
`

`
`U.S. Patent
`
`Mar. 14,2000
`
`Sheet 8 0f 15
`
`6,037,930
`
`FIG. 11B
`
`COMPUTE VECTOR
`FROM LAST POINT
`TO CURRENT POINT J
`
`FORMAT
`OUTPUT DATA
`
`10
`/
`
`/\_11
`
`SEND HOST
`MOVEMENT VECTOR
`SWITCH POSITIONS
`TOUCH STATES
`
`12
`SAVE CURRENT
`POINT AS
`LAST POINT j
`
`U
`
`TCL EXHIBIT 1023
`Page 9 of 22
`
`

`
`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 9 0f 15
`
`6,037,930
`
`ABSOLUTE
`MODES
`
`CLEAR
`
`SET LIGHTS
`TO
`REPRESENT
`MODE
`
`LIGHT
`CONTROL
`FLAG
`
`SET
`
`II
`
`OBTAIN VALID
`FILTERED
`COORDINATE FROM
`PAD
`
`3
`
`4
`_ /
`
`5
`
`N0
`
`SCALE X & Y
`COORDINATES
`
`ANY CHANGE
`IN PADS OR
`SWITCHES
`
`SEND HOST
`‘PAD RELEASED
`MESSAGE
`
`FORMAT &
`SEND POSITION
`SWITCH SETTINGS
`& TOUCH DATA
`
`9
`SAVE CURRENT
`DATA AS OLD \/
`DATA
`
`FIG. 12
`
`TCL EXHIBIT 1023
`Page 10 of 22
`
`

`
`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 10 0f 15
`
`6,037,930
`
`JOYSTICK
`MODE
`
`—»
`
`LIGHT
`CONTROL
`FLAG
`
`SET
`
`1
`
`CLEAR
`
`2
`
`SET LIGHTS
`To
`REPRESENT
`MODE
`
`OBTAIN VALID
`FILTERED
`COORDINATE FROM
`PAD
`
`3
`
`4
`COMPUTE X & Y J
`DIRECTIONS
`(+1, 0' -1)
`
`PAD BEING
`TOUCHED
`OR SWITCH
`
`FIG. 13
`
`THE SAME FOR
`20 CYCLES
`
`SEND HOST
`"PAD RELEASED
`MESSAGE
`
`I f8
`
`SPEED = 5
`
`SPEED = 1
`
`L—-—» [~11
`
`SEND SWITCH.
`TOUCH INFO &
`VECTOR OF SPEED
`& DIRECTION
`I.‘
`
`TCL EXHIBIT 1023
`Page 11 of 22
`
`

`
`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 11 0f 15
`
`6,037,930
`
`JOYSTICK
`MODE
`
`FIG. 14
`
`LIGHT
`CONTROL
`FLAG
`
`SET
`
`CLEAR
`
`SET LIGHTS
`TO
`REPRESENT
`MODE
`
`READ F ILTERED
`VALID
`COORDINATE
`
`TRANSLATE
`COORDINATE TO
`PAD NUMBER
`(0-15)
`
`PAD NO.
`JUST
`PRESSED
`7
`
`PAD NO.
`HELD FOR
`120 CYCLES
`SINCE LAST
`SENT
`I?
`
`TOUCH
`PAD RELEASED
`OR SWITCHES
`CHANGED
`
`f8
`
`SEND PAD NUMBER,
`CHARACTER.
`SCAN CODE, SWITCH
`INFOI TOUCH INFO
`
`i7
`
`TCL EXHIBIT 1023
`Page 12 of 22
`
`

`
`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 12 0f 15
`
`6,037,930
`
`OBTAIN
`FILTERED
`VALID
`COORDINATE
`
`/\1
`___>U
`
`READ IN VALID
`X AND Y
`POINTS
`
`ANY EVIDENCE
`THAT PAD WAS RELEASE I
`DURING THE READING
`OF THE LAST
`N POINTS
`
`FIG. 15
`
`f4
`
`DEBOUNCE
`SWITCHES
`
`COMPUTE A
`WEIGHTED
`AVERAGE OF
`LAST N POINTS
`
`RETURN
`AVERAGES
`
`U
`
`TCL EXHIBIT 1023
`Page 13 of 22
`
`

`
`U.S. Patent
`
`Mar. 14,2000
`
`Sheet 13 0f 15
`
`6,037,930
`
`READ VALID /\1
`X AND Y ____
`COORDINATES
`
`TURN X
`CIRCUITS ON
`
`READ X
`VALUE
`
`TURN Y
`CIRCUITS ON
`
`2
`\/
`
`3
`
`4
`\/
`
`5
`
`YES
`
`PAD
`TOUCHED OR
`RELEASE DURING
`CYCLE
`?
`
`NO
`
`[-7
`
`SCALE POINTS
`TO 0 - 999
`RANGE
`
`U
`
`TCL EXHIBIT 1023
`Page 14 of 22
`
`

`
`U.S. Patent
`
`Mar. 14, 2000
`
`Sheet 14 0f 15
`
`6,037,930
`
`B.
`F
`A W
`
`TCL EXHIBIT 1023
`Page 15 of 22
`
`

`
`U.S. Patent
`
`Mar. 14,2000
`
`Sheet 15 0f 15
`
`6,037,930
`
`JUMPERS
`
`a:
`
`I.“
`z
`
`E o
`3 o
`5
`E
`E
`Z
`
`FIG. 17B
`
`’
`
`TCL EXHIBIT 1023
`Page 16 of 22
`
`

`
`1
`MULTIMODAL TOUCH SENSITIVE
`PERIPHERAL DEVICE
`
`6,037,930
`
`This application is a continuation of application Ser. No.
`08/539,269, ?led Oct. 11, 1995, noW abandoned Which is a
`continuation of Ser. No. 08/261,211, ?led Jun. 15, 1994 noW
`abandoned Which is a continuation of Ser. No. 07/614,022,
`?led Nov. 16, 1990, noW abandoned Which is a continuation
`of Ser. No. 07/430,961, ?led Nov. 1, 1989, noW abandoned
`Which is a continuation of Ser. No. 07/056,039, ?led Jun. 1,
`1987, noW abandoned Which is a continuation of Ser. No.
`06/904,752, ?led Sep. 5, 1986, noW abandoned Which is a
`continuation of Ser. No. 06/675,658, ?led Nov. 28, 1984,
`noW abandoned.
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates generally to the ?eld of
`computer peripheral devices, and more particularly, is
`directed to a multimodal touch sensitive peripheral device
`Which greatly enhances the input of information to a com
`puter system.
`In most general purpose computer systems, commands
`and data are entered to the system from a typeWriter-style
`keyboard and information from the system is displayed on a
`CRT display or printed on a hard copy printing device.
`Where a CRT is used, the computer normally indicates the
`current entry position by displaying a distinctive marker
`called a cursor on the CRT screen. The cursor can take
`several forms but is usually a ?ashing underscore or rect
`angle or a graphical type pointer such as an arroW. The
`position of the cursor may be controlled in a number of
`Ways. In some systems, a particular keystroke, or combina
`tion of keystrokes, is interpreted by the computer as a
`command to move the cursor to the right, left, up or doWn.
`Depending on the system and/or the particular softWare
`being used at a time, the cursor may be moved in increments
`of a single character position, an entire Word, line, paragraph
`or piXel. In many systems, special keys are devoted to these
`functions and are usually located in a keypad separated from
`the typeWriter keyboard.
`Some systems also provide for a pointing device Which
`can be used by the computer to control the location of the
`cursor on the CRT. Such devices include joysticks, paddles,
`light pens, touch panels, touch screens, and so-called
`“mice”. All of these devices enable an operator to “point” at
`a position on the display Where the cursor is to be located.
`Joysticks and paddles are perhaps the most Widely knoWn
`pointing devices due to their Widespread use in video games.
`These devices are usually of the analog or ?Xed aXis type.
`The simplest analog joysticks comprise a pair of potenti
`ometers mechanically coupled to a movable handle at right
`angles to each other. The potentiometers form part of a
`voltage divider netWork and as the handle is moved, provide
`analog signals representing the X and Y coordinate position
`of the handle. The analog signals may then be converted to
`digital form for processing by a computer to locate the
`cursor. The cursor, or other point of interest, can therefore be
`positioned on the CRT by manipulating the handle. The ?Xed
`aXis type of joystick is usually comprised of a handle
`mechanically coupled to a plurality of sWitches. Each sWitch
`represents an aXis and direction of movement of the handle.
`In a 8-aXis joystick, 4 sWitches are provided Which indicate
`handle movement along a vertical, a horiZontal and tWo
`diagonal aXes. Paddles are similar in operation to joysticks
`but usually provide a signal indicating movement of the
`paddle along one aXis.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`One eXample of a joystick knoWn in the prior art is
`disclosed in Us. Pat. No. 4,313,277 issued in the name of
`Eder. The Eder joystick provides a pair of 4 bit logic signals
`Which represent the X and Y coordinate aXis of the joystick
`handle. These signals are continuously scanned by an inte
`gral microcomputer along With the logic output signals from
`a small keyboard. The scanned signals are serially transmit
`ted as a modulated infrared light signal to a receiver attached
`to a computer for demodulation and processing.
`Light pens, touch screens and touch panels represent
`another signi?cant class of pointing devices. Light pen and
`touch screens are used to detect a desired location on a CRT
`screen. Light pens usually have a sWitch in their side or tip
`Which the user activates When the pen is positioned at the
`desired location on the CRT screen. The light pen then
`monitors When the piXel under the pen comes on. Based on
`a reference time signal, the location of the piXel on the CRT
`screen can then be determined. Touch screens use some form
`of overlay on the CRT screen Which When touched, indicate
`to the host computer a corresponding location on the screen.
`Touch panels provide coordinate signals Which represent a
`contact point on the panel. The coordinate signals are
`usually based on some form of grid scheme.
`Mice represent another type of pointing device Which
`have gained popularity With the proliferation of personal
`computers. Mice sit on a ?at surface, such as a table top, and
`are moved about by hand. In some mouse systems, the
`movement of Wheels or bearings Within the mouse are
`coupled to sWitches to provide X and Y motion signals. In
`other systems, the movement of Wheels is used to rotate an
`aperture Which interrupts a beam of light to provide the X
`and Y coordinate signals. US. Pat. No. 4,369,439 issued in
`the name of Broos, U.S. Pat. No. 3,541,541 to Engelbart and
`US. Pat. No. 3,541,521 to Koster disclose mice of the ?rst
`category while US. Pat. No. 4,464,652 issued in the name
`of Lapson discloses a mouse of the latter category.
`Another type of mouse uses a glass grid. A light source
`inside the mouse is directed toWard the grid to determine
`hoW many grid lines the mouse crosses as it is moved. This
`information is then provided to the host computer.
`While the above described pointing devices aid in the
`positioning of the cursor on the CRT screen, they are
`de?cient in a number of areas. For eXample, analog joysticks
`are dif?cult to control, have loW resolution and cannot be
`used to draW straight lines on a CRT screen. Fixed aXis
`joysticks cannot be used to draW complex curves, are sloW
`to move from one place to another and their speed cannot be
`controlled. Mechanical mice require a large amount of table
`space to operate, are dif?cult to maintain in proper
`orientation, are limited in speed and also cannot be used to
`draW straight lines. Moreover, mice can readily fall off the
`operating surface and be damaged and are prone to collec
`tion of dirt and dust in their mechanism. Optical mice share
`many of the same dif?ciencies as mechanical mice and also
`require a special operating surface. Light pens require spe
`cial CRT monitors, are loW in accuracy and are fragile in
`construction. Touch screens obstruct the user’s vieW of the
`CRT screen and require signi?cant arm motions, and con
`comitant time delay, for their use. Thus, extended use of
`touch screens is very fatiguing. Touch screens are also
`dif?cult to use due to parallaX problems. The above
`described devices are also de?cient in that they all require a
`signi?cant amount of supervision control and interrogation
`by the host computer. Thus valuable computer time is used
`to perform relatively mundane operations. Accordingly,
`there is a great need in the art for a pointing device Which
`overcomes the above described de?ciencies in prior art
`
`TCL EXHIBIT 1023
`Page 17 of 22
`
`

`
`3
`devices While at the same time being effective, convenient
`and easy to use. Moreover, different applications require the
`use of different devices. Thus, there is also a need in the art
`for a universal or multimodal device to meet the needs and
`desires of various users and various computer programs.
`
`SUMMARY OF THE INVENTION
`
`It is the overall object of the present invention to provide
`a pressure sensitive multimodal peripheral device Which
`greatly enhances the input and control of information to a
`computer system.
`It is a speci?c object of the present invention to provide
`a multimodal device Which provides a plurality of useful
`operating modes.
`It is another speci?c object of the present invention to
`provide a multimodal device Which is easier to operate and
`to interface to a host computer.
`It is another speci?c object of the present invention to
`provide an intelligent multimodal device Which does not
`signi?cantly add to the overhead of the host computer.
`It is still another object of the present invention to provide
`a multimodal device Which is relatively small in physical
`siZe and is convenient to operate.
`Manipulation of on screen data, Whether graphic, Words,
`or numbers, is a dif?cult process. To move or reorient data,
`Which can be easily done visually, is currently an aWkWard
`process of typing arroW keys or confusing control
`sequences. The present invention alloWs the user to link his
`visual abilities to complex screen movements.
`The device comprises a touch sensitive pad to Which the
`operator applies a light pressure in a prescribed manner to
`effect cursor movement on a CRT screen or to input data to
`the computer. The device has four operating modes Which
`may be automatically selected by the host computer or
`manually selected by the user.
`The four modes are de?ned as the mouse mode, the
`absolute mode, the joystick mode and the numeric keypad
`mode. In the mouse mode, the motion of the user’s ?nger as
`it is traced across the touch sensitive pad is continuously
`measured and is provided to the host computer as a change
`in X and Y, and can be termed an analog input since it
`integrates a series of discrete items of information Which are
`otherWise input of a digital form coordinates, i.e., AX and
`AY. This information may be processed by the host com
`puter to perform a number of useful functions. The key
`feature of the mouse mode is the ability to readily push the
`cursor to a speci?c location on the screen. The mouse mode
`does not require the user to look at the device of the
`invention to exactly locate his ?nger. The device requires
`only that the touch sensitive pad be touched someWhere and
`the user’s ?nger be slid in the appropriate direction. Thus the
`user need only realiZe that he Wants to make a change to, for
`example, a number in the upper right hand portion of the
`screen, and the cursor moved to that exact place by the user
`sliding his ?nger in the relative direction. Should the user
`reach the edge of the pad before the desired point is reached,
`he need only lift his ?nger from the pad and begin a neW
`trace until the point is reached.
`In the absolute mode, the CRT cursor may be made to
`“jump” from one position on the screen to another merely by
`the user touching the corresponding point on the touch
`sensitive pad. In this mode, each coordinate point on the pad
`is mapped to a point on the CRT screen. The key feature of
`the absolute mode is the ability to move the cursor instantly
`to the location desired. It also affords the ability to trace an
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6,037,930
`
`4
`object, draW, or re-map the touch sensitive pad into function
`sWitches for speci?c softWare.
`In the joystick mode, the touch sensitive pad is divided
`into a plurality of areas Which represent cursor direction.
`Cursor movement is controllable in terms of direction, speed
`of movement and total travel distance, again input in an
`analog form. To move the cursor in any of eight directions,
`(up, doWn, right, left, and the four diagonal directions), the
`operator touches the pad at a corresponding place. Location
`of the user’s ?nger touch determines the direction of cursor
`movement. Speed of cursor movement is also variable. Total
`travel of the cursor is determined by the time duration of
`touch. The joystick mode enables directionally stable con
`trol of cursor movement. Such directional stability is a
`necessity for graphics applications. No other device in the
`prior art has the combined capability of draWing a straight
`line as Well as complex curves and shapes. This is also useful
`in operating softWare that has a column-roW format such as
`spreadsheets, accounting, ?nancial and other business soft
`Ware.
`In the numeric key pad mode, the touch sensitive pad in
`divided into a plurality of sections, each section representing
`a numeric value or numeric key function. The pad may also
`have sections Which represent cursor back space and car
`riage return. Thus, the device of the present invention may
`be used to represent the keypad found in calculator-style
`keyboards. To enter numeric data or numeric functions to the
`host computer, the user need only touch the area of the pad
`assigned to the particular number or function.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a representative illustration of the various modes
`of operation of the multimodal device of the present inven
`tion.
`FIG. 2 is a block diagram of the multimodal device of the
`present invention.
`FIG. 3 is a top vieW of the multimodal device of FIG. 2.
`FIG. 4 is a schematic diagram shoWing one set of driver
`transistors for the touch sensitive pad used in the multimodal
`device of the invention.
`FIG. 5 is a schematic diagram shoWing all of the driver
`transistors for the touch sensitive pad used in the multimodal
`device of the invention.
`FIG. 6 is a mechanical diagram of the top layer member
`used to form the touch sensitive pad used in the multimodal
`device of the invention.
`FIGS. 7A and 7B are equivalent circuit diagrams for the
`electrically resistive portions of the touch sensitive pad used
`in the multimodal device of the invention.
`FIG. 8 is a How chart of the main control routine for the
`microcontroller used in the multimodal device of the inven
`tion.
`FIGS. 9—16 are How charts of subroutines used in the
`main control routine illustrated in FIG. 8.
`FIG. 17A is a schematic diagram of the multimodal device
`of FIG. 2.
`FIG. 17B is a schematic diagram of one example of an
`interface for connecting the multimodal device of the inven
`tion to a host computer.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`
`The present invention is directed to a touch sensitive
`multimodal peripheral device Which greatly enhances the
`
`TCL EXHIBIT 1023
`Page 18 of 22
`
`

`
`6,037,930
`
`5
`use of a computer. The device comprises a touch sensitive
`pad to Which the operator applies a light pressure in a
`prescribed manner to effect cursor movement on a CRT
`screen or to input data to the computer. The device includes
`four operating modes Which may be automatically selected
`by the host computer or manually selected by the user. These
`modes are illustrated in FIG. 1 of the draWings and include
`a mouse mode, an absolute mode, a joystick mode and a
`numeric keypad mode, of Which absolute and numeric
`keypad are most usefully termed “digital” modes and mouse
`and joystick are most usefully termed “analog” modes. For
`purposes of explanation With respect to the mouse, absolute
`and joystick modes, the touch sensitive pad is represented by
`blocks A in FIG. 1 and the actual CRT screen in represented
`by blocks B in FIG. 1. The CRT cursor is indicated by an
`“X” in blocks B. With respect to the numeric keypad mode,
`the touch sensitive pad is divided into a plurality of sections,
`each section representing a numeric value or numeric key
`function. By touching a particular section, the associated
`numeric value or key function can be input to the host
`computer.
`In the mouse mode, the motion of the user’s ?nger as it
`is traced across the touch sensitive pad is continuously
`measured and is provided to the host computer as a change
`in X and Y coordinates, i.e., AX and AY. This information
`may be processed by the host computer to perform a number
`of useful functions.
`In the absolute mode, the CRT cursor may be made to
`“jump” from one position on the screen to another merely by
`the user touching the corresponding point on the touch
`sensitive pad. In this mode, each coordinate point on the pad
`corresponds to a point on the CRT screen. As shoWn in FIG.
`1 for example, in order to move cursor X to point X1, the
`user need only touch corresponding point X1 on the pad to
`effect the move. The cursor can be moved to point X2 in a
`likeWise manner or by tracing the ?nger to point X2.
`In the joystick mode, the touch sensitive pad is divided
`into a plurality of areas Which represent cursor direction.
`This mode emulates the function of a conventional 8-axis
`joystick.
`In the numeric key pad mode, the touch sensitive pad is
`divided into a plurality of sections, each section representing
`a numeric value or numeric key function. Thus, the device
`of the present invention may be used to represent the keypad
`found in calculator-style keyboards. To enter numeric data
`or numeric functions to the host computer, the user need
`only touch the area of the pad assigned to the particular
`number or function.
`With reference to the block diagram shoWn in FIG. 2, the
`construction of the multimodal touch sensitive device of the
`present invention Will be generally described. Each of the
`elements shoWn in FIG. 1 are discussed in more detail beloW
`With reference to the remaining ?gures.
`As shoWn in FIG. 2, the device comprises touch sensor
`pad 1 Which includes a pair of close spaced parallel layers
`Which have superposed electrically resistive and conductive
`portions. Sensor pad 1 forms a tWo dimensional plane
`Which, in combination With sWitch netWork 2, provides
`analog X and Y coordinate signals representative of a
`position of applied pressure in the tWo dimensional plane.
`Sensor pad 1 Will be described in more detail beloW With
`reference to FIGS. 3—7. Sensor pad 1 is coupled to sWitch
`netWork 2 Which provides the appropriate bias voltages
`required by the pad. As Will be described beloW With
`reference to FIG. 5, the X andY coordinate signals are taken
`from the inactive surface of the pad at its connection point
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`to the emitter electrodes of the PNP driver transistors. The
`coordinate signals from sWitch netWork 2 are supplied to
`scaling circuit 3. Circuit 3 is used to scale the coordinate
`signals to a predetermined range and also includes calibra
`tion provisions for calibrating the coordinate signals. The
`scaled signals are then provided to A/D convertor 6 Which
`converts the signals to digital form for processing by micro
`controller 8. SWitch netWork 2 is also coupled to microcon
`troller 8. Mode/Action sWitches 5 comprise a plurality of
`membrane sWitches Which are a part of sensor pad 1. These
`sWitches provide for manual selection of the operating
`modes of the multimodal device and also provide manual
`sWitch closure indications for use by the softWare resident in
`the host computer. The sWitches that perform the latter
`function are often called action sWitches. The host computer
`softWare often interprets their closure to indicate that it is
`time to perform some function. For example, in a telecom
`munications program, one could move the cursor over a
`phone number and the number could automatically be dialed
`When the action sWitch is pressed. Touch sensor on/off
`detector 4 is likeWise coupled betWeen sWitch netWork 2 and
`microcontroller 8. Detector 4 provides a logic signal to
`microcontroller 8 Which indicates Whether the pad is being
`touched at a particular moment. LEDs 7 are coupled to
`microcontroller 8 and give a visual indication to the user of
`the status of the multimodal device. Interface 9 is coupled to
`microcontroller 8 and is provided as a means for interfacing
`the multimodal device to computer 10. Computer 10 is a
`host computer Which takes its input from the multimodal
`device. Computer 10, hoWever, is not part of the present
`invention and may be virtually any digital computer knoWn
`in the prior art including so-called personal or home com
`puters. Computer 10 may be driven by an interrupt system,
`as is knoWn in the art, Which alerts the computer that data is
`available from the multimodal device and in the present
`invention such an interrupt system Would be contained in
`microcontroller 8 and connected to host computer 10
`through interface 9.
`FIG. 3 is a top vieW and partial mechanical diagram of the
`multimodal device of the present invention shoWing mode/
`action sWitches 5 of FIG. 2 (mode sWitches SWO—SW3 and
`action sWitches SW4 and SW5) and and LEDs 7 of FIG. 2
`(LED1—LED4). Also shoWn in FIG. 3 are the electrically
`resistive layers of sensor pad 1 and their associated elec
`trodes.
`Touch sensor pad 1 comprises tWo close spaced parallel
`layers. The opposed surfaces of the layers have superposed
`electrically resistive portions With a respective pair of elec
`trodes. As shoWn in FIGS. 4 and 6, the electrode pairs for
`each layer are positioned at right angles With respect to each
`other. The resistive portion of each layer is biased With a
`voltage supplied by the driver circuit shoWn in FIG. 5 at a
`predetermined frequency. The driver circuit comprises a ?rst
`pair of transistors, Q1—Q4, and a second pair of transistors,
`Q2—Q3. The base electrodes of transistors Q1 and Q4 are
`connected together. Transistor Q1 is an NPN type and
`transistor Q 4 is a PNP type. Thus, When one transistor
`conducts, the other is cut off. The base electrodes of NPN
`transistor Q2 and PNP transistor Q3 are similarly connected
`together. Each transistor pair (Q1—Q4 and Q2—Q3) is con
`trolled by multiplexer 12 shoWn in FIG. 5 such that When
`one transistor in each pair is cut off the other transistor in
`each pair is turned on. This function is controlled by input
`line x/y of multiplexer 12. When line x/y is at a logic LO for
`example, transistors Q1 and Q3 conducts and transistors Q2
`and Q4 are cut off. This condition reverses When input line
`x/y is at a logic HI. Multiplexer 12 is controlled by micro
`controller 8.
`
`TCL EXHIBIT 1023
`Page 19 of 22
`
`

`
`10
`
`15
`
`25
`
`35
`
`7
`As shown in FIG. 4, transistors Q1 and Q3 are connected
`to the electrodes associated with the bottom layer of sensor
`pad 1. Transistors Q2 and Q4 are likewise connected to the
`electrodes associated with the top layer of sensor pad 1.
`Sensor pad 1 provides X and Y coordinate signals in the
`following manner. The X coordinate signal is produced by
`multiplexer 12 causing transistors Q1 and Q3 to conduct
`while transistors Q2 and Q 4 are cut off. This causes a How
`of current through the resistive portion of the bottom layer
`thereby forming a voltage gradient in the X or horiZontal
`direction. When pressure is applied to sensor pad 1 to point
`P for example, the resistive portions of each layer comes into
`electrical contact at point P. Thus the equivalent circuit
`shown in FIG. 7A is formed with resistor R1 representing the
`resistance between the emitter electrode of transistor Q1 and
`point P; resistor R2 representing the resistance between point
`P and the emitter electrode of transistor Q 4; and resistor R3
`representing the resistance between point P and the emitter
`electrode of transistor Q3. The X coordinate signal is taken
`from the emitter electrode of transistor Q 4. The Y coordinate
`signal is produced by multiplexer 12 causing transistors Q2
`and Q4 to conduct while transistors Q1 and Q3 are cut off.
`This causes a How of current through the resistive portion of
`the top layer forming a voltage gradient in Y or vertical
`direction. The equivalent circuit shown in FIG. 7B is formed
`with resistor R4 representing the resistance between the
`emitter of transistor Q2 and point P; resistor R5 representing
`the resistance between point P and the emitter electrode of
`transistor Q3; and R6 representing the resistance between
`point P and the emitter electrode of transistor Q4. The Y
`coordinate signal is taken from the emitter electrode of
`transistor Q3.
`As shown in FIGS. 7A and 7B, the X and Y coordinate
`signals are tapped from a voltage divider network for
`processing by the device. The network comprises resistors
`R2, R7 and resistors R5, R8, respectively. The resistance
`value for R7 and R8 is selected such that substantially all of
`the voltage is dropped across them.
`The X and Y coordinate signals are coupled to multiplexer
`12 which provides one or the other coordinate signal at
`output terminal X/Y depending on the logic state of input
`terminal x/y. Multiplexer 12 also includes input terminals for
`receiving a positive and a negative voltage. These voltages
`are used to drive transistors Q1—Q4 into hard saturation and
`deep cut off during their co