United States Patent [191
`Dunthorn
`
`[11]
`[45]
`
`Patent Number:
`Date of Patent:
`
`4,914,624
`Apr. 3, 1990
`
`[54]
`VIRTUAL BUTTON FOR TOUCH SCREEN
`[76] Inventor:
`David I. Dunthorn, 908 W. Outer Dr.,
`Oak Ridge, Tenn. 37830
`Appl. No: 191,227
`Filed:
`May 6, 1988
`
`[21]
`[22]
`[51]
`[52]
`
`[58]
`[56]
`
`Int. Cl.‘ .............................................. .. G06F 3/02
`US. Cl. ............................... .. 364/900; 364/927.2;
`364/927.“
`Field of Search ...................... .. 364/200, 300, 900
`References Cited
`U.S. PATENT DOCUMENTS
`
`Primary Examiner—-Raulfe B. Zache
`Attorney, Agent. or Firm—David B. Harrison
`[5 7]
`ABSTRACT
`Apparatus and methods for creating a virtual push but
`ton comprise a touch sensitive orthogonal data ?eld
`input device useably connected within a computing
`system in which a touch action at'the device generates
`a stream of data related to the location within the ?eld
`of the touch action. The orthogonal data ?eld input
`device includes circuitry for sensing the onset of a ?rst
`touch action and for monitoring the continuity thereof;
`and the computing system including a second condition
`sensor for sensing the occurrence of a second predeter
`mined condition; and, computing circuitry for generat
`ing the virtual push button upon the concurrence of
`touch action continuation and the occurrence of the
`second predetermined condition.
`
`22 Claims, 2 Drawing Sheets
`
`3,911,215 10/1975 Hurst et al. ......................... .. 178/18
`
`4,220,815 9/1980 Gibson et al. . . . .
`. . . . .. 173/18
`4,763,356 8/1988 Day, Jr. et al. ............... .. 364/900 X
`
`OTHER PUBLICATIONS
`Elographics, Prelim. IntelliTauch Systems Manual,
`3/19/88, pp. 1-10.
`
`CRT DISPLAY SCIIEEN 16
`
`J
`TEUBIIB SCREEN
`
`DISPLAY
`ADAPT ER _‘
`
`TOUCH SCREEN
`CONTROLLER g2
`
`cnMPuTER g
`
`E
`
`CIHPUTING
`SYSTEM ll]
`
`KEYBOARD 30
`
`Samsung USP 7,973,773
` Exhibit 1016 Page 1
`
`

`
`US. Patent Apr. 3, 1990
`
`Sheet 1 0f 2
`
`®'\26 WM
`
`29
`
`g
`28
`
`CRT DISPLAY SCREEN 16
`
`/
`J
`TUUCH SCREEN
`18
`
`j
`
`DISPLAY
`ADAPTER
`
`TUUCH SCREEN
`CDNTREILLER g1
`
`COMPUTER _1_2_
`
`TIMER
`13
`—
`
`CUMPUTING
`SYSTEM 10
`
`111 1 I
`III/II
`IIIIIII
`
`KEYBOARD 30
`
` Exhibit 1016 Page 2
`
`

`
`US. Patent Apr. 3, 1990
`
`Sheet 2 of 2
`
`4,914,624
`
`= _
`DY Y Yp-"V'WS
`
`66
`
`6B
`
`11x 2 + BY 3 >
`BUTTON THRESHOLD
`
`Y
`
`g7"
`
`I
`REPURT
`VIRTUAL BUTTUN
`(72
`SET UNTDUCH '
`DEACTIVATIUN TRUE
`
`REPURT UNTIJUCH
`
`SET BUTTONS TRUE
`SET NEV SEQUENCE TRUE
`SET UNTDUCH DEACTIVATIDN
`FALSE
`
`56;
`
`(-62
`
`x PREVIIIIUS=X ’
`Y PREVIUUS=Y
`
`76
`574
`' W
`SET NEV SEQUENCE FALSE
`
`SET BUTTDNS FALSE
`
`REPORT X,Y CUDRDINATES
`
`F IG.—2
`
` Exhibit 1016 Page 3
`
`

`
`1
`
`VIRTUAL BUTTON FOR TOUCH SCREEN
`
`5
`
`15
`
`4,914,624
`2
`desirable or necessary to
`shortcomings. Often it is very
`display the visual feedback following a user's ?nger
`around the screen while not forcing the user actually to
`make a selection upon the untouch action. To achieve
`this feedback using normal touch screen operations, it is
`necessary to completely divorce the untouch action
`from selection. An operation with visual feedback is
`?rst used to determine a position on the screen, but the
`untouch action does not cause a selection to be made. A
`second touch/untouch operation is then required on the
`screen (or via some other input device such as a key
`board) in order to perform the select operation.
`This prior requirement for two disparate actions to
`perform a single selection is not satisfactory. In order to
`accomodate the occasional time when an untouch ac
`tion will not lead to a selection, the user’s attention must
`be diverted for each such action. This inconvenience
`becomes intolerable when dealing with a sequence of
`operations as typically encountered in computer aided
`design (CAD) or even in simple word processing appli
`cations. One way of further illustration of this inconve
`nience is by drawing an analogy to a hypothetical type
`writer which would require the typist to strike a ribbon
`advance key a?er each symbol key was struck merely
`to advance the ribbon during typing, a function auto
`matically performed by conventional typewriters.
`This inconvenience in prior art touch screens and
`touch pads has not been experienced with other posi
`tion-reporting devices, such as computer "mice", “tab
`lets”, and "light pens" which usually have actual push
`buttons present on the mechanism manipulated by the
`user’s hand during operations. Thus, the user is able to
`manipulate a computer mouse to a desired orthogonal
`(x,y) position typically followed by a visual cursor on
`the display screen and then to actuate one of a plurality
`of switches on the mouse to initiate or discontinue a
`particular function.
`Thus, a hitherto unsolved need has arisen for the
`provision of a “virtual“ push button for use with touch
`screens or touch pads which functionally extends the
`capability thereof to match the capability of other posi
`tion reporting devices, such as computer “mice", “tab
`lets”, and “light pens”
`which usually have electrome
`chanical push-buttons present on the mechanism for
`manipulation by the user’s hand during positioning op
`erations thereof.
`
`30
`
`FIELD OF THE INVENTION
`The present invention relates to methods and appara
`tus for data input to computing systems. More particu
`larly, the present invention relates to methods and appa
`ratus for creating a virtual button for a touch screen or
`touch pad orthogonal ?eld data input device.
`BACKGROUND OF THE INVENTION
`Touch pads and touch screens responsive to localized
`depression pressure have been provided to create “but
`tons" for data input. A touch pad device is disclosed, for
`example, in U.S. Pat. No. 3,911,215; and, a transparent
`touch screen for overlayment and use with the display
`screen of a cathode ray tube is disclosed, for example, in
`U.S. Pat. No. 4,220,815, the disclosures of both of these
`patents being incorporated by reference herein.
`These previously patented devices rely upon resistive
`sheets in which uniform orthogonal electric ?elds are
`generated in two-phase sequences, for example, as
`shown in FIG. 1 of the referenced ‘215 patent. When a
`low resistance or ground connection is established at a
`particular location on the sheet, as by depressing an
`overlying, low resistance ground or signal return sheet
`into electrical contact with the resistive sheet, precise
`orthogonal ?eld (x,y) pinpointing of the location of
`depression may be provided. Thus, these prior devices
`have been utilized in conjunction with a computer sys
`tem to create “push-buttons”. When a particular push
`button (i.e. location on the resistive sheet) is depressed
`(“touch”), or released (“untouch”), the computer has
`sensed this occurrence and has responded operationally
`thereto.
`Other technologies have been applied to create touch
`screens and pads, including surface accoustic wave
`technology; and these screens and pads have also been
`used with computing systems to create push-buttons.
`The surface accoustic wave touch screen has the addi
`tional capability of reporting a z-coordinate indicative
`of the velocity or force with which the screen or pad is
`touched.
`Traditionally, touch screens or pads have accom
`plished push-button sensing either through a touch, i.e.
`when the operator's ?nger or a stylus ?rst touches the
`screen; or, an untouch, i.e. when the finger or stylus is
`released from the screen or pad. “Touch” is the most
`natural action for a novice user, because it represents
`activation on contact, just like the expected action of a
`conventional electromechanical push-button. For ex
`ample, with a touch screen implementation, a user
`touches a labelled, delineated area on the screen, and a
`resulting action and
`acknowledgement are then given
`e.g. by an appropriate display on the screen.
`For “untouc ”, the touch step (which of course must
`come ?rst) usually initiates some form of visual feed
`back, such as highlighting a screen area or option. The
`area or option is
`not actually selected to perform its
`action until the user “untouches” the screen. Thus, with
`“untouch" the slightly more experienced user can move
`?nger or stylus on the screen to be sure the proper area
`is highlighted and then untouch or remove ?nger or
`stylus in order to complete the selection process. This
`approach enables a far greater precision in selection of
`65
`desired areas on the screen.
`The “touch" and “untouch” procedures are satisfac
`tory for many applications, but they have signi?cant
`
`35
`
`45
`
`50
`
`55
`
`SUMMARY OF THE INVENTION WITH
`OBJECTS
`A general object of the present invention is to over
`come functional and operational drawbacks hitherto
`associated with touch screen and touch pad devices by
`enabling generation of virtual push-buttons as a func
`tion of activation of at least one actual push button
`location on the screen or pad.
`A speci?c object of the present invention is to extend
`the functionality of touch screen and touch pad orthog
`onal data input devices to match that achievable with
`computer mice, tablets, light pens and the like.
`In accordance with the principles of the present in
`vention, methods and apparatus are provided for creat
`ing a virtual push button at a touch sensitive orthogonal ’
`?eld data input device used in conjunction with a com
`puting system in which a touch action at the device
`generates a stream of data related to the location within
`the ?eld of the touch action. The methods and the appa
`ratus embodying the methods carry out the steps of:
`
` Exhibit 1016 Page 4
`
`

`
`4,914,624
`3
`sensing the onset of a ?rst touch action at the device
`and monitoring the continuity thereof,
`sensing the occurrence of a second predetermined
`condition,
`generating the virtual push button upon the concur
`rence of touch action continuation and the occurrence
`of the second predetermined condition.
`In one speci?c aspect
`of the present invention, the
`step of sensing the occurrence of the second predeter
`mined condition comprises the step of sensing the onset
`of a second touch action at the device.
`In a second speci?c aspect of the present invention,
`the step of sensing the onset of a second touch action at
`the device includes the step of determining the direction
`of the second touch action relative to the location of the
`?rst touch action and wherein the virtual push-button is
`generated as a function of the determined direction.
`In a third aspect of the present invention, the step of
`sensing the onset of a second touch action at the device
`includes the step of determining the distance of the
`second touch action from the location of the ?rst touch
`action and wherein the virtual push-button is generated
`as a function of the determined distance.
`In a fourth aspect of the present invention, the step of
`sensing the occurrence of a second predetermined con
`25
`dition comprises sensing an increase in applied pressure
`at the touch point.
`In a fifth aspect of the present invention, the step of
`sensing the occurrence of the second predetermined
`condition comprises the step of detecting elapse of
`a
`predetermined time interval and detecting an untouch
`action at the ?rst location after the predetermined time
`interval has elapsed.
`In a sixth aspect of the present invention, the step of
`sensing the occurrence of the second predetermined
`35
`condition comprises the steps of detecting elapse of
`predetermined time intervals and comprising the fur
`ther step of generating and displaying to the user a
`sequence of virtual push-buttons with a virtual push
`button being displayed during each time interval, the
`step of generating the virtual push button occurring
`upon detection of an untouch action at the ?rst location
`during display of a preferred one of the virtual push
`buttons.
`In a seventh aspect of the present invention, the step
`of sensing the occurrence of the second predetermined
`condition comprises the step of sensing an input at an
`other input device functionally connected to the com
`puting system.
`In an eighth aspect of the present invention, the step
`of sensing an input at another input device comprises
`the step of sensing actuation of a key of a keyboard
`device.
`These and other objects, advantages and features of
`the present invention will be more fully understood and
`55
`appreciated upon considering the following detailed
`description of preferred embodiments, presented in
`conjunction with the accompanying drawings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`In the Drawings:
`FIG. 1 is a somewhat diagrammatic block diagram of
`a computing system including a touch screen input/dis
`play device con?gured in accordance with the princi
`ples of the present invention.
`65
`FIG. 2 is a ?owchart of one preferred embodiment of
`the method of the present invention when used in con
`junction with the hardware depicted in FIG. 1.
`
`4
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`It has been discovered that when a resistive touch
`screen or pad of the type described in the referenced
`patents is touched in two places at
`the same time, the
`screen returns a single position which is located about
`half way between the two positions actually being
`touched. This untouched, single position is therefore
`available to serve as a virtual push-button.
`- Also, when the screen or pad is being touched at one
`point and is then touched at another point as well, there
`is a sudden large change or discontinuity in reported
`position which is more sudden and discontinuous than
`would be produced by a rapid motion of the ?nger or
`stylus across the screen or pad. Since there will have
`been no detected untouch action prior to this large,
`discontinuous change, its occurrence is distinct, and
`may therefore be distinguished as abnormal from other
`functional activities at the touch screen/touch pad.
`Therefore, the
`action of touching the screen/pad at a
`second touch point without ?rst releasing a still active
`?rst touch point thereby creates one form of virtual
`push-button.
`Although the present invention has been developed
`for use with a glass resistive touch screen, the present
`techniques may be effectively used with any touch
`screen or touch pad which, when touched at two differ
`ent points, will return a position signi?cantly different
`than for either point actually being touched.
`The activation of a virtual push-button not only indi
`cates that a sudden change in position has take place,
`but also reports the direction and amount of that
`change. Thus, it is easily possible to determine whether
`the second touch was above, below, to the right or to
`the le? of the original touch. Finer distinctions of angu
`lar direction are also easily implemented In particular,
`the implementation of four distinct virtual buttons, one
`for each of the cardinal directions, is both easy to imple
`ment and easy for an operator to remember. It is also
`possible to differentiate different virtual push-buttons in
`the same direction on the basis of the distance between
`the two simultaneous touch actions giving rise to the
`virtual push-button.
`In addition to enabling abortion of a normal touch
`untouch operation, virtual push-buttons have other
`very useful properties. On a touch screen or touch pad,
`actual or assigned push buttons are usually defined by
`establishment of a function (push-button) at a particular
`location. Usually, visual information is provided to
`identify each actual button’s position and function.
`Touch screens in particular are often implemented so
`that the visual information presented at the screen
`changes as required in order to match rede?ned button
`positions. However, within any given visual presenta
`tion, the location of each actual push button will be
`fixed.
`A virtual push-button, on the other hand, is not ?xed
`to any position on the screen, but is de?ned at a position
`related to the positions of two simultaneous touch ac
`tions giving rise to the virtual push-button. Thus, virtual
`push buttons may coexist with and even overlie actual
`push button locations without creating any interference
`with the functions normally assigned to those actual
`push button locations. This feature is particularly useful
`for buttons which are to retain common and useful
`functions regardless of the information presented on the
`screen; functions such as “cancel" and “con?rm".
`
`60
`
`30
`
`40
`
`45
`
` Exhibit 1016 Page 5
`
`

`
`20
`
`4,914,624
`6
`5
`?nger and thumb of a user. The touch screen 18 is con
`Since the virtual push button is by de?nition an ab
`nected electrically to a touch screen controller 20,
`normal or discontinuous action which readily lends
`which is in turn connected to the bus of the computer
`itself to "abort" or “cancel” operations, it is apparent
`12.
`that this principle may be extended to other characteris
`The touch screen controller 20 provides the drive
`tics and usages of the touch screen or touch pad. For
`voltages for the x and y axes of the screen 18, converts
`example, timing may be considered as a control factor
`the analog signals from the touch screen 18 into digital
`in the operation of touch screens and touch pads. Nor
`mally, once a selection has been accurately located by
`coordinates for x and y, and transmits these coordinates
`using visual feedback, the time interval until the accom
`to the host processor 12. Since the touch screen 18 has
`analog outputs, the coordinate (x,y) resolution is deter
`panying untouch selection action occurs is quite short.
`mined by the resolution of the analog to digital con
`It is therefore possible to de?ne an abnormally long
`time interval as another variety of virtual push-button.
`verter within the controller 20. With a twelve bit analog
`For example, if an abnormally long time interval
`to digital converter, the screen resolution may be up to
`4000 by 4000 points across the active surface of the
`elapses between the touch action and the untouch ac
`tion, this inordinate delay may be used to cancel the
`touch screen 18. The controller 20 may be polled or
`interrupt driven, and it generates coordinate pairs as
`selection function of the untouch action. If accompa
`nied by visual feedback, this timeout feature may be
`long as the touch screen 18 is being touched.
`Preferably, the touch screen 18 may be an Elograph
`come a powerful tool in its own right. For example, the
`ics ACCUTOUCH (tm) E274 or E270 Touch Screen,
`visual feedback may be made to cycle through a se
`quence of patterns representing different functional
`and the controller 20 may be an Elographics E271-l4l
`controller which is con?gured as a half slot expansion
`push-buttons whereby the user may select a particular
`function or activate a particular push-button by per
`card for insertion into a bus connector of the personal
`computer 12. In order to work with the software ap
`forming an untouch action while the desired visual
`pattern is currently being presented. In such an opera
`pearing at the end of this document, the E27l-l4l con
`troller may be switch-con?gured for 12 bit resolution,
`tion the patterns are not restricted to
`the touch point,
`25
`with streaming output to the host 12. Other con?gura
`but they may be visual signals displayed elsewhere on
`tions are easily implemented with minor changes in the
`the screen, off the screen, .or they may be other forms of
`signals such as aural or voice synthesized signals pres
`software. Other touch screens, such as the Elographics
`DURATOUCH (tm) resistive membrane touch screen
`ented to the user via appropriate transducer devices.
`Any signal capable of being perceived by the user may
`or the Elographics SAW (surface accoustic wave)
`Touch Screen and other Elographics controllers, in
`be employed with touch-untouch action time intervals
`cluding the E27l-l40 and E27l-60 work equally well
`in order to provide a successive series of virtual push
`with the present invention.
`buttons in accordance with the principles of the present
`As illustrated in FIG. 1, the user's fore?nger may
`invention;
`touch the screen 18 at a location 22. When this happens,
`Furthermore, with touch screens which sense the
`the touch screen controller derives orthogonal (x,y)
`level of touch pressure
`at, as well as the orthogonal
`coordinate information to fix the location 22 and sends
`coordinates of, the touch location, the amount of sensed
`an interrupt to the computer 12. The host processor
`touch pressure may be used in a manner analogous to
`time in the immediately preceding example, with an
`within the computer 12 then enters a series of service
`increase in pressure being considered an abnormal con
`routines. These routines sense the location 22 as an
`actual button and generate predetermined graphical
`dition producing a virtual push-button. Further in
`creases in pressure, especially if accompanied by feed
`information at the screen 16 in response to the touch
`action at the location 22. If the user’s fore?nger is re
`back to the user from the computing system, such as a
`moved from the location 22, then an untouch action is
`visual display, can be sensed as other distinct virtual
`push-buttons. Untouch action then provides selection of
`detected, and this action may be used to select the func
`45
`tion or operation called to the screen when the location
`‘ the virtual button de?ned and determined by the great
`22 was originally touched.
`est pressure sensed.
`Now, if before the fore?nger is removed from the
`Virtual push-buttons give touch screens and touch
`location 22,
`the user's thumb is brought into contact
`pads the same range of capabilities as other position
`reporting devices, such as computer "mice”, “tablets",
`with the screen 16 at a location 24, a virtual push button
`and “light pens” which usually have electromechanical
`at location 26 is thereby established, and this virtual
`push-button may operate e.g. to cancel the function
`push-buttons present on the mechanism which are avail
`able for manipulation by the user’s hand during posi
`originally called up by depressing the location 22 of the
`tioning operations.
`touch screen 18, or it may have some other preestab
`With reference to FIG. 1, a computing system 10
`lished function or consequence.
`Alternatively, the touch action of the fore?nger at
`includes a computer 12 such as an IBM Personal Com
`puter (tm). In addition to conventional CPU, memory,
`the location 22 may set a timer which, after a predeter
`mass storage and input/output interface elements con
`mined normal untouch period, causes a menu 28 to
`begin to cycle through program options or choices for
`nected by internal data and control buses, the computer
`example. When a particular option or choice of interest
`12 may include an internal timer established by software
`or hardwrre for marking predetermined time intervals.
`to the user becomes displayed in the menu area 28, an
`A display adaptor 14 is connected to
`the bus of the
`untouch action at the location 22 then causes that par
`ticular choice to be selected. In this way the virtual
`computer 12 and is also connected to drive a suitable
`push button is established in the temporal, as opposed to
`display device such as a CRT monitor having a display
`the spatial, domain.
`screen 16. A transparent touch screen 18, for example of 65
`the type described in the referenced US. Pat. No.
`Alternatively, the screen 16 may be of the surface
`4,220,815, overlies the display screen 16 and senses
`accoustic wave type, and increased pressure of the
`touch and untouch actions performed e.g. by the index
`touch action of the fore?nger at the location 22 may be
`
`35
`
`55
`
` Exhibit 1016 Page 6
`
`

`
`. 5
`
`20
`
`25
`
`30
`
`35
`
`4,914,624
`7
`sensed as a second predetermined condition, thereby
`establishing the presence of a virtual push-button.
`The computing system 10 may conventionally be
`provided with a keyboard 30 for text entry. One or
`more of the keys of the keyboard may be used in con
`junction with the touch screen 18 to provide a virtual
`push-button. For example, during continuation of the
`touch action at the location 22, a key of the keyboard 30
`may be depressed, resulting in the generation of a vir
`tual button which may produce a predetermined display
`upon the screen 16.
`FIG. 2 sets forth a flow chart illustrative of a single
`virtual push-button in accordance with the principles of
`the present invention. The virtual push-button is estab
`lished by detection of I'large" distance changes in but
`ton location. The FIG. 2 ?owchart also includes un
`touch detection. The ?ow sequence charted in FIG. 2
`may be activated either by an interrupt from the touch
`screen controller 20 or by a timer established within the
`computer 12. If a timer is used, the time interval for
`activation is set according to the response time desired
`and other requirements of the system.
`For example, typical touch controllers are capable of
`producing streaming data output of touch coordinates
`at a rate around I00 points per second. Individual de
`signs may easily vary from less than one fourth to over
`four times the 100 points per second rate. The routine
`described by the FIG. 2 ?owchart is typically activated
`by an interrupt each time a coordinate pair is generated.
`Thus, interrupts are continuous at the controller-deter
`mined rate during a touch operation, and cease com
`pletely when no touch is sensed. If the controller is not
`designed to detect and report the end of a touch opera
`tion (untouch action), it is helpful to activate the routine
`via a timer interrupt as well as the controller interrupt,
`in order to detect when the controller has ceased to
`generate interrupts. Timer interrupt frequency is se
`lected to enable reliable untouch detection, which typi
`cally requires about 50 milliseconds and may easily vary
`from less than one fourth to four times that interval,
`40
`depending upon the speci?c controller, screen and op
`erating environment.
`The FIG. 2 control program is entered at an entry
`point 40 and a logical node 42 determines whether a
`touch is
`in progress. Initially, the flag to “deactivate
`untouch" is false and the “new sequence” and "buttons”
`?ags are true. If a touch is not in progress, as tested at
`the node 42, a logical node 44 determines if the present
`sequence is a "new sequence” in order to check to see
`whether the associated untouch can be reported yet. If
`a new sequence is detected at the node 44, the program
`is exited at an exit point 46. If not, a logical node 48
`determines whether untouoh has been detected. If not,
`the program is exited at the exit point 46. If so, accord
`ing to the “deactivate untouch" flag as established at a
`logical node 50, the untouch is either reported at an
`output block 52 or discarded. In either case, a process
`block 54 sets the “deactivate untouch” flag to false, and
`the “buttons” and "new seuence" ?ags to true. Setting
`the “new sequence” ?ag true prevents the same un
`
`8
`touch from being reported more than once as well as
`insuring the proper initiation of the next sequence of
`touch reportings. The program is then exited at an exit
`point 56.
`If a touch is in progress as tested at the node 42, the
`x, y coordinate values are obtained from the touch
`screen controller 20 at a process block 58. A check is
`made at a logical node 60 to see if a “new sequence” is
`starting. If so, there is no check made for virtual button
`operation, since there is no reference position from
`which to measure changes in distance; and the program
`execution proceeds directly to a process block 62 in
`which the x value is stored in an x previous value regis
`ter and the y value is stored in a y-previous-value regis
`ter.
`If the present sequence is not a “new sequence" as
`determined at the logical node 60, then a "buttons" ?ag
`is checked at a logical node 64. If the “buttons” flag is
`not enabled, then program execution proceeds directly
`to the process block 62. If the “buttons” flag is set, then
`a process block 66 calculates a Delta x as equal to the
`present x value minus the previous x value, and a Delta
`y as equal to the present y value minus the previous y
`value. The process block 66 also calculates the sum of
`Delta x square and Delta y square.
`While the FIG. 2 program calculates circular dis
`tance (Dx2+Dy2), in some situations it may be prefera
`ble to use other values such as absolute distance |Dx|
`or |Dy| for the comparison made at the process block
`66.
`A logical node 68 determines whether the sum of
`Delta x square and Delta y square is larger than a prede
`termined virtual button threshold value. If not, the pro
`gram execution proceeds directly to the process block
`62. If so, then a virtual button is reported at a process
`block 70, and the “deactivate untouch” flag may be set
`true and the “buttons" ?ag (tested at node 64) is set false
`at a block 72. The “buttons” ?ag is set false at this block
`72 to prevent multiple activations of the same button.
`The “deactivate untouch” flag is usually set true at the
`block 72 since it is usually desirable to ignore untouch
`action when a virtual button is ?rst activated, but this ,
`decision is a matter of program convention and is not
`necessary.
`n
`In any case, when program execution leaves either
`the process block 62‘or the process block 72, the “new
`sequence” flag (tested at node 60) is set false and the x
`and y coordinates are reported at a process block 74,
`and the program is then exited at an exit node 76; to be
`repeated at the next interrupt or time out from the inter
`nal timer.
`_
`Here follows a program listing in Microsoft Macro
`Assembler language for the Intel 8086/8088 series mi
`croprocessors. In addition to the program ?ow charted
`in FIG. 2, the following program also implements un
`touch detection and the distinction of virtual buttions in
`the four cardinal directions. One of the buttons is inten
`tionally made more difficult to activate so that it cannot
`be pressed by accident.
`
`45
`
`50
`
`55
`
`60
`
`65
`
` Exhibit 1016 Page 7
`
`

`
`4,914,624
`
`10
`9
`TOUCH SCREEN INTBRHUPT HITH VIRTUAL BUTTON ANALYSIS
`Hritton in Microsoft Macro Assembler
`'
`for the Intel 8086/8088 series
`microprocessors
`
`This is on example 0!‘ e reduction to practice.
`The "Virtual Button" technique is directly applicable
`other computers, other languages, and may be used in
`places other then an interrupt routine.
`
`Screen is the interrupt service routine for the
`Biographies touch screen using a model 141 controller
`in en IBM-PC or similer computer. It responds to
`screen interrupts, reeds the data es required, and
`does "Virtual Button" lnelysis, since the interrupt
`is one place certain to see each successive reading.
`this routine requires 12-bit, streaming input from
`the model H41 controller.
`
`a.‘ - -e we ‘a nee .0 see -. - - -- ‘Q ‘I Q. ~- eee <.
`
`SEGMENT BYTE PUBLIC 'CODE'
`ASSUME CS: __TEXT, OS: NOTHING, SS: NOTHING, ES: NOTHING
`
`SCREEN
`
`PROC
`STI
`PUSH
`PUSH
`PUSH
`HOV
`HOV
`
`FAR
`
`DX
`AX
`DS
`AX,CS
`DSJX
`
`;Set up DS=C8 for easy memory eeceas
`
`HOV
`
`DX,ELOPORT ;l’crt address of NH
`
`IN
`TEST
`JZ
`SUB
`IN
`HOV
`INC
`:8:
`IN
`
`888
`883
`S118
`HOV
`HOV
`HOV
`JMP
`
`:Get statue from NH
`AL,DX
`ALH‘OH :‘i'eot X deta
`GE'LY
`:Get Y if‘ not
`01,2
`38:01: to first port
`AL,DX
`3Get X high byte, 12 bit mode
`AHJL
`DX
`
`:‘I‘iming units for very fast machines
`Hub! Had I low byte
`
`AX. 1
`Mt, 1
`:Shitt right 4 to normel word
`AX,1
`T£HPX,AX
`:Scve it for later
`AL,NTCOUNT
`?imer count
`USE_FLAG,AL
`;Reeet use {log
`SCREXIT :And exit the interrupt
`
`; The model 11H generates two interrupts, one for X, then
`; another for l'.
`The X_ only stores data in TERPX ‘above
`5 and resets timer and event flags .
`Y follows through
`; with the complete analysis.
`
`SUB
`IN
`nov
`INC
`NOP
`
`nx,2
`AL,DX
`AH,AL
`DX
`
`5Baok to port
`met I high byte
`
` Exhibit 1016 Page 8
`
`

`
`11
`
`4,9 14, 4
`62
`
`12
`
`:And ‘1 low byte
`
`NO?
`AL,DX
`IN
`AX, 1
`SHE
`ML, 1
`883
`AK, 1
`SEE
`gSnift right U , normal word
`AX, 1
`SHR
`;Save regs for temporary store
`BX
`PUSH
`____
`will
`PUSH
`Calculate t e change in ‘1 (delta 1) and the change
`in X (delta X) from the previous values
`
`HOV
`SUB
`HOV
`Nov
`HOV
`SUB
`HOV
`HOV
`CM?
`HOV
`HOV
`JNE
`CHP
`JNE
`
`BXJX
`BXJ-ASTY
`LASTLAX
`AX,TEHPX
`CLAX
`CLLASTX
`LAS'I‘X,AX
`AL,NTCOUNT
`UNHAI‘I‘, 1
`UNHAI‘I', 1
`USE_FLAG,AL
`SCRBXO
`UOANCEL,O
`SCREXO
`
`Button Analysis
`
`38X : delta ‘I
`;Save new 1!
`
`;CI : delta X
`;Get a. matching pair
`;<-----..----: Tiller count
`;<----------: ‘rest acting on stream
`:THESE ARE 1
`Baiting for untouch
`:TIBD‘
`Reset use flag
`}
`:<