United States Patent (19)
`Wagner et al.
`
`54 SYSTEM AND METHOD FOR BUILDING,
`TESTING AND INTEGRATING A
`GRAPHICAL TOUCH USER INTERFACE
`
`75 Inventors: S.A.W.s. Gary R.
`9,
`y,
`73 Assignee: NCR Corporation, Dayton, Ohio
`
`*
`
`Notice:
`
`This patent issued on a continued pros-
`ecution application filed under 37 CFR
`1.53(d), and is subject to the twenty year
`patent term provisions of 35 U.S.C.
`154(a)(2).
`Appl. No.: 08/742,004
`21
`22 Filed:
`Oct. 31, 1996
`51
`Int. Cl." ........................................................ G06F 3/00
`52 U.S. Cl. ........................... 345/333; 345/335; 34.5/173
`58 Field of Search ..................................... 345/333,334,
`345/335, 348,339,967
`
`56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`5,121,477 6/1992 Koopman et al. ...................... 345/333
`
`US006002395A
`Patent Number:
`11
`(45) Date of Patent:
`
`6,002,395
`*Dec. 14, 1999
`
`5,163,130 11/1992 Hullot ..................................... 345/334
`5,297.250 3/1994 Leroy et al. ............................ 345/333
`5,347,629 9/1994 Barrett et al. ........................... 345/334
`5,533,184 7/1996 Malcolm ................................. 345/333
`5,600,778 2/1997 Swanson et al. ....................... 345/333
`5,642,511
`6/1997 Chow et al. ......
`... 345/967 X
`5,652,850 7/1997 Hollander ................................ 345/333
`5,764,226 6/1998 Consolatti et al. ..................... 345/333
`Primary Examiner Raymond J. Bayerl
`ASSistant Examiner-Chadwick A. Jackson
`Attorney, Agent, or Firm Needle & Rosenberg
`57
`ABSTRACT
`A builder, tester, and runtime integration System and method
`for a graphical touch user interface, Such as a point-of-Sale
`(POS) touch screen interface system, is provided. The sys
`tem and method may include a builder tool to create and
`modify the graphical interface, a tester tool for testing the
`interface, and a run-time System for actually executing the
`interface logic. Because the interface may be designed,
`modified and tested external to the run-time System, the
`design and maintenance may be performed by a non
`programmer without ever changing run-time code.
`Additionally, the builder and tester may be designed to be
`general-purpose, and the run-time code may be reusable.
`
`21 Claims, 30 Drawing Sheets
`
`211
`
`
`
`211-5
`
`NICE CONTROLEXAMPLES (TOUCH): PIZZA, PIZZAS, SPECIALPEPPERONI, ASA MEAL, GUEST CHECK #1
`
`203
`
`OTHER
`
`GUEST CHECK #1
`
`21-
`
`SPECIAL
`PEPPERON
`
`PIZZAS
`
`SPECIAL
`DELUX
`
`KIDS MEAL
`
`MISC
`
`211-3
`
`211-4
`
`SPECIAL
`WEGGIE
`
`
`
`212
`
`APPETIZERS
`
`GSSEE P BREADSTICKS
`
`213
`
`213-1P
`
`LARGE SODA
`
`R&R
`D
`COFFEE
`
`BEER
`
`ASA MEAL
`
`WIMEAL
`
`PITCHER SODA
`
`202
`
`221 - 1 TO221-8
`
`BALANCE
`
`END
`ORDER
`
`MODIFY
`TEM
`
`INDIVIDUAL
`TOTAL
`
`CHANGE
`QUANTITY
`
`204
`
`CANCEL
`ORDER
`
`REMOVE
`|TEM
`
`N---/
`241- TO241-6
`
`Lightspeed Ex. 1022.1
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 1 of 30
`
`6,002,395
`
`F.G. 1A
`
`22
`
`
`
`26
`
`PRINTER
`
`
`
`
`
`
`
`VIDEO
`DISPLAY
`
`14
`
`COMPUTER
`
`
`
`16
`
`18
`
`HARD DISK
`
`
`
`FLOPPY DISK
`
`20
`
`F.G. 1B
`
`28
`/
`
`
`
`PRINTER
`
`
`
`
`
`CASH
`DRAWER
`
`38
`
`40
`
`
`
`TOUCH SCREEN
`
`32
`
`
`
`COMPUTER
`
`DISKDRIVE
`
`36
`
`34
`
`Lightspeed Ex. 1022.2
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 2 of 30
`
`6,002,395
`
`903
`
`
`
`|# XOEHO ISHT|0HEHLO
`
`| || Z.
`
`9 - || ?
`
`VZ º?IH
`
`WCTOS HE HO ||d
`
`ZOZ
`
`
`
`SHEZII EddV
`
`SCIWTWS
`
`
`
`
`
`SETdWWXE TOHINOO EOIN
`
`
`
`
`
`
`
`Lightspeed Ex. 1022.3
`
`

`

`U.S. Patent
`
`
`
`Dec. 14, 1999
`
`Sheet 3 of 30
`
`6,002,395
`
`XOEHO
`
`EHOLSNOd[\OO
`
`
`
`
`
`|# XOEHO ISE (15)
`
`
`
`ETNOJ EKONWTWE
`
`
`
`{{Z "OIH
`
`Lightspeed Ex. 1022.4
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 4 of 30
`
`6,002,395
`
`
`
`
`
`
`
`HEMVHG HSVO (HOQOL) SETdWWXE TOHINOO BOIN
`
`
`
`
`
`
`
`ESOTO
`
`
`
`HENWWHO HSV/O
`
`|
`
`OZ "?IH
`
`Lightspeed Ex. 1022.5
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 5 of 30
`
`6,002,395
`
`
`
`
`
`[×]
`
`TEONWO >
`
`SNO|NO
`()
`
`
`
`
`
`INOHEddBd (HOQOL) SETdWWXE TOHINOO EOIN
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SI? NQOvº
`
`
`
`CIZ “?IH
`
`Lightspeed Ex. 1022.6
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 6 of 30
`
`6,002,395
`
`1301
`
`1301-2
`
`PIZZA
`STYLE:WS BORDER
`WINDOW: O O 218 240
`COLOR: 254 167 146
`FONT: 24
`TEXT: 15
`JUSTFY: C
`LNE: O
`EXPLICIT ENABLE:y
`ENABLE:n
`
`FIG. 3A
`
`1302
`
`1302-3
`
`1303
`
`PIZZAS
`TITLE: 16
`STYLE: MFBS CTRIANGLE
`WINDOW: 1 105 101 64
`ENTRY:y
`FUNCTION1:80 PIZZA1
`DESTROY1:37 8 4 5 - 1
`CREATE1: 1 14-1
`ENABLE: y
`SPECIAL PEPPERON
`TITLE: 17
`STYLE: MFBS LOVAL | MFSB RTRIANGLE
`WINDOW. 117 34 101 64
`ENTRY:y
`FUNCTION1: 75 SPECIAL PEPPERONI1
`DESTROY1: 8-1
`CREATE: 7 -1
`FUNCTION2:76 SPECIAL PEPPERONI2
`DESTROY2:37 8 4 5 -1
`CREATE2: 11 12-1
`ENABLE: y
`ASA MEAL
`TITLE: 24
`STYLE: GPBS TXTBELOW
`ICON: SALAD C
`WINDOW: 1. 414 101 64
`ENTRY:y
`FUNCTION1; 92 SALAD AS A MEAL
`DESTROY1: 8-1
`CREATE1: 7 -1
`ENABLE: y
`GUEST CHECK #1
`BLACKBOARD; 23 RECEPT1
`TITLE: 31
`STYLE: WS CAPTION | RECEIPT WNDSTYLE | RCS ITEMSEPERATORS
`RCS SCROLLITEM | RCS NOTIFY | RCS ITEMSELECTOR
`WINDOW: 437 O 199 214
`COLOR: 255 255 255
`ENABLE:y
`NICE CONTROLEXAMPLES (LANGUAGE): PIZZA, PIZZAS, SPECIALPEPPERON, ASA MEAL, GUEST CHECK #1
`
`1304
`
`1304-3
`
`1305
`
`Lightspeed Ex. 1022.7
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 7 of 30
`
`6,002,395
`
`1311
`
`1311-6
`1311-7
`1311-8
`1311-9
`
`1312
`
`1313
`
`FG. 3B
`
`CASH, SO.82, $1.00, $5.00, $10.00, $20.00
`BLACKBOARD: 4 CASH FLEX BAR
`TITLE: 138
`STYLE: FBS TOGGLE FBS MUTEX I FBS 6BUTTON
`WINDOW: 8 48 437 64
`COLOR: 137 209 140
`ENTRY:y
`FUNCTION1:39 FLEX CASH SPECIAL1
`DESTROY1: 21 22 23 -1
`CREATE1:27 - 1
`FUNCTION2:58 FLEXTENDER CASH26
`DESTROY2:21 22 23 -1
`CREATE2: 27 - 1
`FUNCTION3:58 FLEXTENDER CASH26
`DESTROY3: 21 22 23 -1
`CREATE3:27-1
`FUNCTION4:58 FLEXTENDER CASH26
`DESTROY4: 21 22 23 -1
`CREATE4: 27 - 1
`FUNCTION5:58 FLEX TENDER CASH26
`DESTROY5: 21 22 23 -1
`CREATE5:27-1
`FUNCTION6:58 FLEXTENDER CASH26
`DESTROY6: 21 22 23 -1
`CREATE6:27-1
`ENABLE: y
`
`KEYPAD
`STYLE: KPS NUMCALCCE
`WINDOW: 84 233 269 240
`LENGTH: 7
`MAPPING: y
`ENTRY:y
`FUNCTION1: 125 TENDER KEYPAD1
`EXPLICIT ENABLE: y
`ENABLE: y
`
`EDIT BOX
`STYLE. EBS NUMERIC EBS CARETOFF EBS RIGHT | EBS MONETARY
`WINDOW. 26O 209 91 25
`COLOR. 255 255 255
`FONT:24
`FOCUS:y
`LENGTH: 7
`EXPLICIT ENABLE: y
`ENABLE:y
`
`NICE CONTROLEXAMPLES (LANGUAGE): CASH, SO.82, $1.00, $5.00, $10.00, $2000, KEYPAD, EDITBOX
`
`Lightspeed Ex. 1022.8
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 8 of 30
`
`6,002,395
`
`FIG. 3C
`
`CASH DRAWER
`STYLE: ECDS RIGHTTOLEFT | ECDS ACTIVE IECDS SOUNDON
`WINDOW: 111 64 404 304
`ENABLE:n
`
`PEPPERON
`BLACKBOARD: 41 TOPPINGO
`TITLE: 75
`STYLE: MFBS LPLAIN IMFBS CPLAIN IMFBS RPLAIN
`ICON: PEPPZAC
`WINDOW: 8 219 142 71
`ENTRY:y
`FUNCTION1: 132TOPPING LEFT1
`FUNCTION2: 131 TOPPING CENTER2
`FUNCTION3: 133TOPPING RIGHT3
`ENABLE:y
`
`NICE CONTROLEXAMPLES (TOUCH): CASH DRAWER, PEPPERON
`
`Lightspeed Ex. 1022.9
`
`

`

`U.S. Patent
`
`
`
`
`
`
`
`
`
`
`
`(WALISAOWSY—HCHOON):S‘(GONV1V9LdIS03Y)‘P“(SMNINQ):Z‘(HSHLO-WZZId)‘©“JIdWVX3YSLSSL-
`
`
`
`YAHLO TWAYON
`
`Dec. 14, 1999
`
`Sheet 9 of 30
`
`6,
`
`002,395
`n
`
`AAIGOWGN4
`
`
`
`Wa!Y3A0HO
`
`ALILNVAD
`
`WLOL
`
`JDNVHO
`TWACIAIONI
`
`JAOWSY
`
`WALI
`
`TAONVO
`
`YAqHO
`
`lL
`
`SYNIWC
`
`Val039!
` YaosWNIdSnAVa0s
`TIVWS
`
`
`
`WAWSOI
`
`XN14d
`
`WldadS
`
`AIDA
`
`SHAZILaddV
`
`ASSSHO/M
`
`SVZZld
`
`LVZZId
`
`SSHOIMGNYVS
`ILLSHOVdS
`INOWAddad
`
`Wl0ddS
`
`
`
`#MOAHOLSAND
`
`LO
`
`vO
`
`OlV
`
`Lightspeed Ex. 1022.10
`
`SMOWSAVSYE||qyauaonYvo
`
`
`
`Tvi0adSSWZZIdL-L12
`-Y3aSSNVEL
`
`G¢ZE-SNASKOS
`
`Lightspeed Ex. 1022.10
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 10 of 30
`6,002,395
`FIG. SA-1
`
`102
`
`NO
`2 / GETINTERFACE
`FILE NAME
`
`YES
`
`SET INTERFACE
`PREFIXNAME TO
`"autopos1"
`
`04
`
`OPEN
`SUPPORTING FILES
`
`106
`
`114
`
`OPEN INTERFACE
`AND SUPPORTING FILES
`
`READ
`SUPPORTING FILES
`
`1
`08
`
`116
`
`READ INTERFACE
`AND SUPPORTING FILES
`
`118
`
`DISPLAY FIRST
`CRT SCREEN
`
`iSCreenNum = 1
`iChildNum = 1
`iNumScreens = 0
`CHOOSE CHILD
`
`
`
`110
`
`120
`
`CLEAR
`CRT SCREEN
`
`NO
`
`YES
`
`124
`
`REMOVE
`OBJECTS OWNED BY
`ScreenNum
`
`&Edo (e)
`
`YES
`
`REMOVE
`CRT OBJECTS
`
`128
`
`Lightspeed Ex. 1022.11
`
`

`

`U.S. Patent
`
`Dee. 14, 1999
`
`Sheet 11 of 30
`
`6,002,395
`
`
`
` SAASAAééYANV1dLeoONYAAOLYVLS
`
`ONZeL
`
`Or8E1
`
`C7VS(OVOe!
`
`|=WNNPIIYO!|+SUB@JOSWNNI=WNNUeaIS!
`
`CH)9El
`
`
`
`wnNusaS|AdGANMOSLOSrdO
`
`AW1dSI0
`
`Lightspeed Ex. 1022.12
`
`Lightspeed Ex. 1022.12
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 12 0f 30
`
`6,002,395
`
`160
`162
`7s in/ iChildNum = 0
`
`A
`
`.
`
`GET
`ScreenNum
`ChildNum
`164
`
`
`
`NITIALIZE
`bBlackboard,
`bloon
`y
`bELL
`166
`
`(D)
`
`(G)
`
`144
`
`146
`
`YES
`
`YES
`
`SCREEN
`MODIFICATION
`NO
`
`
`
`
`
`CHILD
`MODIFICATION
`
`NO
`
`NO
`
`SAVE
`INTERFACE
`
`148
`
`
`
`YES
`
`CHANGE
`INTERFACE NAME
`
`NO
`
`150
`
`
`
`GET NEW
`INTERFACE NAME
`
`WRITE INTERFACE
`TO DISK
`
`154
`
`FIG. SB
`
`
`
`MORE
`BUILDERWORK
`
`
`
`YES
`
`(a)
`
`156
`
`NO
`
`Lightspeed Ex. 1022.13
`
`

`

`U.S. Patent
`(D)
`
`Dec. 14, 1999
`
`170
`
`Sheet 13 0f 30
`6,002,395
`F.G. 5C
`
`168
`
`172
`
`MOVE
`TO TOP LEFT
`
`YES
`
`NO
`
`174
`MOVE
`for
`iGHid-YES
`
`NO
`
`178
`
`176
`
`MOVE
`foBOTfLEG)-YES
`
`NO
`
`
`
`182
`
`180
`
`MOVE
`to Boffo RIGHTo-YES
`
`NO
`
`184
`YES
`SE)
`NO
`
`188
`
`& E>
`
`YES
`
`NO
`
`192
`
`YES
`
`SEld
`NO
`
`196
`
`& Ed
`
`YES
`
`NO
`
`ALIGNiScreenNum
`OR ChildNum
`OBJECT(S) TOTOPLEFT
`
`ALIGNiSCreenNum
`OR iChildNUm
`OBJECT(S) TO TOP RIGHT
`
`162
`GE)
`
`ALIGN iSCreenN
`SCreenNU
`OR iChildNum
`OBJECT(S) TOBOTTOM LEFT
`
`LGN iSCreenN
`A
`SCreenNum
`OR iChildNum
`OBJECT(S) TOBOTTOMRIGHT
`
`186
`
`190
`
`194
`
`198
`
`Y = -1
`X = 0
`
`Y - 1
`
`X = 0
`
`Y = 0
`
`X = -1
`
`Y = 0
`
`X = 1
`
`(E)
`
`GE)
`
`(F)
`
`(F)
`
`(F)
`
`(F)
`
`Lightspeed Ex. 1022.14
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 14 of 30
`
`6,002,395
`
`F.G. SD
`
`
`
`REMOVE
`CRTOBJECTS/-200
`
`
`
`DISPLAY
`CRT OBJECTS
`
`202
`
`CHANGE
`CRT CONTENTS
`
`
`
`
`
`MORE
`RIGHT
`MOUSE
`CLICKS
`
`
`
`SAME
`CRT
`OBJECTS
`MoyED
`
`
`
`
`
`
`
`
`
`APPLY DELTA X
`AND DELTAYTO
`SCreenNum OR iChildNum
`CRT object(s)
`
`
`
`
`
`REMOVE
`CRT OBJECTS
`
`
`
`DISPLAY
`CRT OBJECTS
`
`210
`
`212
`
`YES
`
`FG, 5E
`
`
`
`
`
`
`
`
`
`
`
`
`
`Lightspeed Ex. 1022.15
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 15 0f 30
`
`6,002,395
`
`218
`
`220
`
`226
`
`
`
`236
`
`238
`
`YES (D)
`
`NO
`
`
`
`222
`
`YES UPDATE CHILD
`STRUCTURES
`
`228
`
`
`
`F.G. 5F
`
`
`
`224
`
`UPDATE
`SCREEN
`INDEX
`ARRAY
`
`NO
`
`NO
`
`NO
`
`NO
`
`230
`GET
`YES (NE INTERCHANGE
`iScreenNum2
`CHILDREN
`iChildNum2
`
`
`
`
`
`GE)
`
`YES (D
`
`Lightspeed Ex. 1022.16
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 16 of 30
`
`6,002,395
`
`FIG SG
`
`FOUND
`CHOSEN OBJECT
`
`chClass = chCommoniPtri)
`
`
`
`
`
`END OF
`CHOOSEMENU
`
`Ptre Ptr-
`
`246
`
`
`
`
`
`
`
`
`
`
`
`
`
`Lightspeed Ex. 1022.17
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 17 Of 30
`
`6,002,395
`
`USE chClass TO DISPLAY
`OBJECT'S COMMON DIALOG
`
`250
`
`F.G. SH
`
`
`
`
`
`
`
`COLOR SHASE
`
`YES
`
`256
`
`SELECT
`NEW COLOR
`
`MODIFY DIALOG FIELDS
`
`258
`
`DIALOG
`NFORMATION OK
`
`260
`
`YES
`UPDATE CHILD OBJECTFIELDS-262
`
`264 Sisld NO
`
`
`
`266
`
`INITIALIZE bloon bblackboard bELL
`
`270
`
`272
`
`268
`
`CHILD
`
`NO
`
`st- CRTOBJECTS
`
`YES
`
`REMOVE
`
`DISPLAY
`
`CRTOBJECTS
`
`Lightspeed Ex. 1022.18
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 18 of 30
`
`6,002,395
`
`GD
`
`F.G. 5
`
`274
`
`276
`
`bBlackbOard= TRUE
`p
`
`YES
`
`
`
`BLACKBOARD
`MODIFY
`
`
`
`278
`
`YES
`
`DISPLAYBLACKBOARD DIALOG
`
`
`
`
`
`
`
`
`
`
`
`REMOVE
`BLACKBOARD NAME
`?
`
`YES
`
`SET
`BLACKBOARD
`NAME TO NULL
`
`284
`
`BLACKBOARD CHANGED
`
`CHOOSE NEW
`BLACKBOARD NAME
`
`
`
`292
`
`DIALOG
`
`INFORMATIONOK)YES BLAIBARD
`
`UPDATE
`
`(K)
`
`290
`
`Lightspeed Ex. 1022.19
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 19 of 30
`
`6,002,395
`
`294
`
`296
`
`298
`
`5icon=TRUE -NO
`YES
`
`(D
`
`NO
`
`(D
`
`YES
`
`DISPLAY CONDIALOG
`
`
`
`FIG. 5,
`
`
`
`YES
`
`SET ICON
`NAME TONULL
`
`300
`
`302
`
`306
`
`NO
`
`REMOVE
`CONNAME
`NO
`
`ICON
`NAME CHANGED
`
`
`
`
`
`CHOOSE NEW CONNAME
`
`
`
`
`
`310
`
`DIALOG
`INFORMATION OK
`
`YES
`
`314
`
`316
`
`318
`
`32
`
`
`
`
`
`ICON CHANGED
`
`
`
`NO
`
`YES UPDATE
`ICON
`NAME
`
`REMOVE
`CRT
`OBJECTS
`
`DISPLAY
`CRT
`OBJECTS
`
`Lightspeed Ex. 1022.20
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 20 of 30
`
`6,002,395
`
`32
`
`SGd ()
`
`32
`
`2 SD NO (M)
`
`
`YES
`
`324
`
`DISPLAYELL DIALOG
`
`MODIFY
`CURRENTELL
`
`
`
`ENTER
`CURRENTELL
`CHANGES
`
`UPDATE
`ELL N9
`
`
`
`READ ELL
`DIALOG INFO
`
`SAVE CURRENT
`ELL CHANGES
`
`
`
`340
`
`
`
`ADVANCE/EEERs
`ONDIALOG
`
`
`
`ELL POINTER
`
`NO
`
`
`
`DIALOG
`INFORMATION OK
`
`346
`YES UPDATE CHILD
`ELL INFO
`
`(M)
`
`
`
`
`
`
`
`
`
`
`
`Lightspeed Ex. 1022.21
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 21 of 30
`
`6,002,395
`
`(M)
`
`F.G. SL
`
`348
`
`COMMON WORK
`
`YES
`
`O
`
`NO
`
`350
`BLACKBOARD WORK YES
`
`GD
`
`NO
`
`352
`
`3.Ed YES
`
`NO
`
`(K)
`
`354
`
`Sc YES O
`
`N9
`
`-GA)
`
`NO
`
`356
`
`
`
`MORE
`NEW CHILDREN
`
`YES
`
`358
`
`ChildNum=iChildNum+1
`
`360
`
`CHOOSE
`CHILD
`
`Lightspeed Ex. 1022.22
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 22 of 30
`
`6,002,395
`
`F.G. 6A
`
`START
`
`500
`
`OPEN INTERFACE
`AND SUPPORTING FILES
`
`502
`
`READ INTERFACE
`AND SUPPORTING FILES
`
`504
`
`DISPLAY FIRST
`CRT SCREEN
`
`506
`
`PUSHLOGICAL BUTTON(S)
`INFORMATION ON STACK
`
`508
`
`PUSH LOGICAL BUTTON'S
`INFORMATION OFF STACK
`
`510
`
`DISPLAY
`TESTER WINDOW
`
`512
`
`
`
`gold GE)
`
`QUIT PROGRAM
`
`YES
`
`NO
`
`SUSPEND PROGRAM
`
`NO
`
`YES
`
`RESTART PROGRAM
`
`Lightspeed Ex. 1022.23
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 23 of 30
`
`6,002,395
`
`FIG. 6B
`
`520-REMOVE TESTER WINDOW
`
`POPLOGICAL BUTTON'S
`INFORMATION OFF STACK
`
`522
`
`YES
`
`suggests YES
`SAME CRT SCREEN
`
`NO
`
`GB)
`
`END
`
`524
`
`NORMAL BRANCHING
`
`NO
`
`528
`
`530
`
`532
`
`536
`
`DESTROY NORMALSCREEN(S)
`
`DESTROY AUXILIARY SCREEN(S)
`
`
`
`CREATENORMALSCREEN(S)
`34
`5
`
`CREATE AUXILIARY SCREEN(S)
`538
`
`540 SS-> YES
`
`(A)
`
`NO
`
`DESTROY CRT SCREEN(S) A-542
`
`RESTORE PREVIOUSA 544
`CRTSCREEN(S)
`
`Lightspeed Ex. 1022.24
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 24 of 30
`
`6,002,395
`
`FG. 7A
`
`600
`
`RUNTIME INITIALIZATION WORK
`
`602
`
`OPEN INTERFACE
`
`604
`
`READ INTERFACE
`
`606
`
`NITIALIZE
`iScreenSACtivei
`iACtivePtr
`iStartNeWSCreeni
`
`
`
`608
`
`SET SPECIAL BOOLEAN
`VARABLES TO TRUE
`
`
`
`614
`
`i> iActivePtr
`
`YES
`
`66
`
`DOSPECIAL BOOLEAN
`VARIABLES LOGIC WORK
`
`Lightspeed Ex. 1022.25
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`(C)
`
`Sheet 25 0f 30
`6,002,395
`FIG 7B
`
`
`
`n = iScreensActivei
`iStart = Screenlndexn)
`iEnd = iScreenindexn+1)
`INITIALIZE
`
`618
`
`FETCHCHILD-620
`
`
`
`622
`
`CHILD
`EXISTING RECEPT TYPE
`
`YES
`
`NO
`
`DECODE TITLE
`626-CREATE CHILD
`STORE CHILD
`
`sHow WINDow-624
`SHOW WINDOW
`
`
`
`(E) NO
`
`
`
`628
`
`
`
`
`
`IS
`DECODED
`TITLE NULL
`?
`
`YES
`
`630
`
`HIDE WINDOW
`
`Lightspeed Ex. 1022.26
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 26 of 30
`
`6,002,395
`
`FIG 7C
`
`632
`
`CHILD
`HAS COLOR
`p
`
`NO
`
`
`
`634
`
`GET COLOR HANDLE
`STORE CHILD
`
`636
`
`CHILD
`HASICON
`
`NO
`
`638
`
`GETICON HANDLE
`STORE CHILD
`
`642
`
`PROCESSTEXT
`STORE CHILD
`
`
`
`644
`
`
`
`CHILD
`NUMERIC KEYPAD
`
`NO
`
`
`
`646
`
`SETUP
`KEYBOARD MAPPING
`
`Lightspeed Ex. 1022.27
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 27 Of 30
`
`6,002,395
`
`CHILD
`HAS FOCUS
`
`NO
`
`4 648
`
`6 50
`
`GIVE CHILD FOCUS
`ECHO BOX WORK
`
`FIG 7D
`
`5ExplicitEnable = TRUE)-N9
`
`ENABLE CHILD
`USINGbEnable VALUE
`
`CHILD
`FLEX BAR
`
`NO
`
`SET FLEX BAR STATE
`
`664
`
`BLACKBOARD
`AND MISC WORK
`
`NO
`
`j= + 1-666
`
`CHILD
`TR-STATE
`t
`
`652
`
`654
`
`656
`
`658
`
`660
`
`662
`CHILD TR-STATE WORK
`
`
`
`668
`
`NO (b)
`
`Lightspeed Ex. 1022.28
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 28 Of 30
`6,002,395
`FIG 7E
`
`670 Sid YES
`
`NO
`
`674 - INITIALIZE
`
`2
`67
`
`SCROLLING
`WORK
`
`676
`
`i = i + 1
`
`678
`
`is ActivePrsNO
`
`YES
`
`Lightspeed Ex. 1022.29
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 29 of 30
`
`6,002,395
`
`680
`
`n = iScreenSActivei)
`Start = Screenindexin
`iEnd = iScreenindexn+1)
`j= Start
`
`
`
`FIG. 7F
`
`682
`
`FETCH CHILD
`
`684
`
`CHILD'S
`bEntry Function = 'V'
`
`NO
`
`O
`
`YES
`IS
`686 <gades
`NO
`690
`YES
`
`IS
`
`688
`
`INTIALIZE
`KEYBOARD WARIABLES
`-
`k = EntryFunctions 1
`
`692
`
`69
`
`INITIALIZE iPr
`k = EntryFunctions Ptr
`(M)
`
`700
`
`YES
`
`NO
`
`6
`
`INITIALIZE Pitr
`k = EntryFunctions iPtri)
`
`702
`
`NO
`
`RIGHT BUTION TOUCH
`NO
`
`(D
`
`YES
`
`(M)
`
`YES
`
`696
`
`704
`
`INITIALIZE iPtr
`k = EntryFunctions iPtri)
`
`706
`708 SEd)-N O
`
`YES
`
`(M)
`
`Lightspeed Ex. 1022.30
`
`

`

`U.S. Patent
`
`Dec. 14, 1999
`
`Sheet 30 0f 30
`6,002,395
`FIG 7G
`
`710
`
`BUSINESS
`MODEL LOGIC
`
`
`
`
`
`712
`
`DESTROY
`SCREENS
`
`
`
`
`
`714
`
`DESTROY SCREEN'S
`CHILDREN AND OBJECTS
`
`
`
`716
`
`CREATE
`SCREENS
`
`YES
`
`Lightspeed Ex. 1022.31
`
`

`

`1
`SYSTEMAND METHOD FOR BUILDING,
`TESTING AND INTEGRATING A
`GRAPHICAL TOUCH USER INTERFACE
`
`6,002,395
`
`2
`Therefore, there is a need in the art for a System and
`method for building, testing, and integrating a touch Screen
`interface into a runtime System that is simpler and leSS
`expensive than the Systems and methods of the prior art, is
`more accurate in progressing from design to
`implementation, and enables non-programming perSonnel to
`do a significant amount of the design and maintenance work
`SUMMARY OF THE INVENTION
`A builder, tester, and runtime integration System and
`method for a graphical touch user interface, Such as a
`point-of-sale (POS) touch screen interface System, is pro
`vided. The system and method may include a builder tool to
`create and modify the graphical interface, a tester tool for
`testing the interface, and a run-time System for actually
`executing the interface logic. Because the interface may be
`designed, modified and tested external to the run-time
`System, the design and maintenance may be performed by a
`non-programmer without ever changing run-time code.
`Additionally, the builder and tester may be designed to be
`general-purpose, and the run-time code may be reusable.
`In one embodiment, the System and method of building or
`creating a graphical user interface, Such as a POS touch
`Screen interface, may include the Steps of choosing, one-at
`a-time, the “controls that comprise a Screen, accessing the
`menu items and their dialogs to gather the controls
`information, and Saving the dialog information into the
`object’s “slots”. This process may be repeated for each new
`SCCC.
`The System and method of testing the interface may begin
`by "painting the Screens that comprise the first touch Screen
`display. The button controls appearing on the first display
`may have their logic information placed in a Stack Structure
`so that the first button's data is on the top of the stack. Then,
`the stack is popped and its pertinent logical information is
`placed in a popup window for a few Seconds. Branching then
`may take place and the new Screens button information is
`placed on top of the Stack, and the top item is again popped
`off the Stack. The process continues until all the branching
`has been done. If a particular Screen is ever redisplayed, then
`none of its button data is placed on the Stack a Second time.
`If a Screen number does not exist, then its number is placed
`in a popup window along with an error message. The tester
`can be paused at anytime by pressing the Space bar, and can
`be terminated by pressing a digit key.
`The System and method of integrating the external graphi
`cal interface into a runtime System may begin by performing
`a Series of Steps that read and Store the interface into
`program Structures in memory. Next, the first Screen is
`painted by creating each of the child windows (objects)
`using the slot information to formulate the necessary win
`dow commands to cause the object to be displayed properly.
`Next, a Series of Steps that map the interface function code
`to the execution of the right function within the run-time
`code may be performed. Finally, the Series of Steps that
`manage the displaying of the new touch Screen display may
`be performed.
`The present invention provides a highly efficient System
`and method for designing, testing and integrating a graphical
`interface as human interface engineers and Software pro
`grammerS may work in parallel to create the actual external
`interface. Additionally, most of the interface design, imple
`mentation and maintenance can be performed using menus
`and dialogs within the familiar Windows environment.
`BRIEF DESCRIPTION OF THE FIGURES OF
`THE DRAWINGS
`FIGS. 1A and 1B are block diagrams depicting examples
`of a developmental system and a POS touch screen system
`in accordance with the teachings of the present invention.
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The present invention generally relates to the building,
`testing and integration of graphical user interfaces (GUI),
`and in one embodiment to a touch Screen point-of-Sale
`(POS) whereby the interface design, testing and mainte
`nance is performed external to run-time code.
`2. Description of the Prior Art
`In recent years, touch Screen devices have become com
`monplace in a variety of environments. For example, touch
`15
`screen devices have been used in point-of-sale (POS)
`applications, Such as in a retail establishment Within certain
`types of retail establishments, such as “fast food”
`restaurants, touch Screen POS devices have become quite
`advantageous, given their ease of use and flexibility.
`Superior interfaces are required in many cases to ensure
`that different types of users of touch Screen interfaces can
`easily use these Systems. Accordingly, many of these inter
`faces are now designed by human interface (HI) engineers,
`who have training in the design of human-machine
`interfaces, but who may or may not have an in-depth
`knowledge of computer programming. Moreover, it may be
`most efficient for a human interface engineer to not waste
`time performing Software programming work, but rather to
`concentrate on his or her area of expertise. The problem then
`arises as to how an interface designed by an HI engineer is
`best communicated to customers (for approval of the design)
`and to Software engineers (for implementation and testing of
`the design).
`One current approach is to have HI engineers use a
`presentation tool like Microsoft(R) PowerPoint(R) to design
`and layout the graphical user interface. The problem, of
`course, is that this work does not contribute directly to the
`final interface. In other words, after an HI engineer has
`designed the interface, a programmer must then take the
`design and then Start programming. Furthermore, as with
`any design, there will be design iterations that further
`compound the problem.
`A better approach would be to have an HI engineer use a
`builder tool that utilizes graphical windows, menus, dialogs,
`and a common Set of control names to build the user
`interface. A non-programmer could build the interface accu
`rately and quickly. Then, this actual external interface would
`be read by a skeleton, runtime System that has just a few
`dummy entry level logic functions defined. The result would
`be that an HI engineer could very quickly get feedback on
`how the interface will actually look-and-feel. Also, the HI
`engineer would have the capability to correct errors and
`make improvements as the interface is fine-tuned.
`Further, by providing a general-purpose tester program,
`the HI engineer would have another way to judge the design
`by watching the System run as if a human were pressing the
`various buttons. Should they have inadvertently forgotten to
`build a particular Screen, then the tester would inform them
`of the mistake, and the HI engineer could again use the
`builder tool to correct the problem. Software programming
`perSonnel could also use the tester to ensure that the proper
`entry level logic functions are specified. Finally, when touch
`Screen Systems are deployed, it may be desirable to modify
`the user interface without changing the run-time interface.
`For example, a user might want to change a button's
`position, an object's color, Some object text, etc.
`
`65
`
`45
`
`50
`
`55
`
`60
`
`25
`
`35
`
`40
`
`Lightspeed Ex. 1022.32
`
`

`

`3
`FIGS. 2A-2D depict sample graphical displays illustrat
`ing various objects integrated in a POS touch Screen System.
`FIGS. 3A-3C illustrate pertinent external interface lan
`guage slots names and their respective data for the objects
`shown in FIGS 2A-2D.
`FIG. 4 is a visual depiction of a tester in accordance with
`the teachings of the present invention.
`FIGS. 5A-1, 5A-2, and 5B-5L are flowcharts depicting
`the Steps that may be performed when building an external
`touch Screen interface.
`FIGS. 6A and 6B are flowcharts depicting the steps that
`may be performed when testing a touch Screen interface.
`FIGS. 7A-7G are flowcharts depicting the steps that may
`be performed when integrating a touch Screen interface into
`a runtime System.
`DETAILED DESCRIPTION OF THE
`INVENTION
`Referring now to FIG. 1A, system 10 is used for building,
`maintaining, and testing the external interface. It includes
`computer 12, video display (CRT) 14, memory 16, hard disk
`18, floppy disk 20, keyboard 22, mouse 24, and printer 26.
`In a preferred embodiment, system 10 may be a typical
`PC developmental system, whereby computer 12 is an
`Intel(R 486 or Pentium(R) computer, video display 14 is a
`VGA monitor, memory 16 comprises at least 16 MB of
`RAM, hard disk 18 has a capacity of at least 1.2 GB of
`Secondary Storage, floppy disk 20 is a 3.5" flex disks,
`keyboard 22 is a common PC keyboard, mouse 24 is a
`MicroSoft(R) mouse or equivalent, and printer 26 is a laser or
`inkjet printer that is PC compatible.
`Referring now to FIG. 1B, system 28 may be a point-of
`sale (POS) system and may include computer 30, touch
`screen 32, memory 34, disk 36, printer 38, and cash drawer
`40.
`In a preferred embodiment, system 28 may be a touch
`Screen System used in a retail environment. Computer 30
`may be an NCR 7450 PC or it can be an ordinary Intel-based
`PC, touch screen 32 is a 10-inch NCR Touch Screen or one
`from another vender, memory 34 may comprise 16 MB of
`RAM or more, disk 36 is optional and may be at least 500
`MB of secondary storage, printer 38 may be an NCR printer
`or equivalent, and cash drawer 40 may be an NCR cash
`drawer or equivalent.
`The present invention will now be described in further
`detail below. In describing the present invention, reference
`is made to a Sample application for the teachings of the
`invention-namely, a point-of-Sale touch Screen retail Sys
`tem for use within a restaurant, Such as a pizzeria. Of course,
`it will be readily recognized that this is merely a Sample
`application, and the present invention can be used in various
`other environments as well.
`1. The Builder Tool
`In FIGS.5A-1, 5A-2, and 5B through 5L, the builder tool
`(hereinafter also referred to as <B>) is used to build a new
`or modify an existing touch Screen interface. The builder
`tool <B> may be implemented by programming computer
`12 using any one of a variety of programming languages,
`Such as Microsoft C/C++.
`AS mentioned previously, the interface is external to the
`run-time System (described later) and in one embodiment
`may consist of two ASCII files. The first file is the main file
`and makes use of a frame-like language with Screen frames
`and child frames. Each Screen is begun with a “begin
`Screen:i' line, where the value of “i' is the screen number.
`Likewise, each Screen ends when the “end Screen:i' entry
`is found.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6,002,395
`
`4
`In a preferred embodiment, within each Screen frame are
`one or more “NICE” object frames given in sequential order.
`The NCR NICE(R) (Natural Interface for Computing
`Environments) product is a library of run-time application
`building blocks for use with PC and PC-compatible work
`stations running in a Microsoft(R) WindowsTM environment.
`NICE Client is a version of NICE for systems operating
`under Microsoft Windows 3.1 (and up). NICE Client pro
`vides the application developer with a set of user interface
`objects appropriate for use in a touch Screen environment.
`The first line is an object definition that begins with a
`“begin child:i type” line and is terminated with an “end
`child:i type” entry. Here “i' is the relative child number
`within a screen and “type' is the NICE classification. For
`example, in FIG. 3A, the “Pizza' control is associated with
`a NICE “Reply Box”, and the type for this control is
`“ncReplyBox'. Between the begin-end pair of object state
`ments are slots that give the child information. Each Slot
`occupies one line and is denoted by a slot name followed by
`a colon. For example, in FIG. 3A, the first slot for the
`“Pizza” control is the “style” slot
`The second interface file is directly related to the first
`interface file. For example, in FIG. 3A notice the “text:15”
`slot information for "Pizza'. The number 15 indicates the
`15th entry in the second file will contain the text “Pizza'.
`Thus, the number 15 is a pointer into the second file.
`FIG. 2A depicts an actual user interface that is displayed
`on video display 14 based upon the NICE controls of FIGS.
`3A-3C (as well as other controls not shown). Control 211-5
`in FIG. 2A corresponds to the Pizza control shown in FIG.
`3A. Because of the "text:15” slot for the Pizza control in
`FIG. 3A, the 15th entry in the second file (e.g., the text
`“Pizza") is displayed for control 211-5 in FIG. 2A. In short,
`the Second file contains one line for each text unit needed by
`the controls defined in the first file.
`Importantly, each touch Screen display 32, Such as those
`depicted in FIGS. 2A-2D, may have multiple screens as just
`defined. For example, in FIG. 2A, the display has four
`Screens. The first screen 201 includes the "Pizza” 211,
`“Appetizer” 212, “Salads” 213, and “Other” 214 units. A
`unit is defined to consist of one or more NICE objects. For
`example, the “Pizza” unit 211 has four button objects 211-1,
`211-2, 211-3 and 211-4, and these controls are placed on top
`of another background object 211-5. This latter object 211-5
`is the one with a background color which contains the text
`“PZZa.
`The second screen 202 is the "Drinks' unit 221, which
`contains eight buttons 221-1 through 221-8. The third screen
`203 is the one having the title “Guest Check #1 and the
`“balance' line. The fourth Screen 204 is in the lower
`right-hand corner of FIG. 2A where the six buttons 241-1
`through 24.1-6 are placed on a colored background.
`The choice of multiple Screens for a touch Screen display
`32 is useful when the pressing of a button control causes
`only part of the display to be destroyed. For example, again
`referring to the touch Screen example in FIG. 2A, if the
`“Beer' button is pressed, then only the “Drinks' screen 202
`needs to be deleted and replaced. The other three Screens
`201, 203 and 204 remain. Thus, when the touch screen
`display 32 is built using the builder tool <B>, a decision may
`be made whether to have one or more Screens.
`Referring again to the flowcharts in FIGS. 5A-1, 5A-2,
`and 5B through 5L, the builder tool may be executed by a
`Suitably programmed computer 12, as will be recognized by
`a person of ordinary skill in the art. The builder tools <B>
`execution starts at block 100 and then continues to block 102
`
`Lightspeed Ex. 1022.33
`
`

`

`6,002,395
`
`15
`
`35
`
`40
`
`25
`
`S
`where an inquiry is made to decide whether to create a new
`interface or open an existing one. The implementation for
`the inquiry may be performed by constructing a Windows
`File Menu with “New' and “Open” selections, as is standard
`Windows programming practice.
`If in block 102 the result of the inquiry is “yes”, meaning
`a new interface is to be created, then blocks 104 through 110
`are sequentially executed. First, in block 104, the new
`interface prefix name is Set to “autopoS1’, So later the two
`interface files, described previously, saved to disk 18 will be
`named “autopos1.frm” and “autopos1.dat' respectively. The
`"..frm' suffix is chosen because it denotes the file is an
`Artificial Intelligence, frame-like language where various
`slots are used to store information. The “...dat' Suffix indi
`cates the file contains data, and in this case the data will be
`text data.
`Continuing, block 106 is executed next and the two
`Supporting files are opened. The first contains the ELL
`function names, and the Second contains the “blackboard'
`names. The use of these files is explained later. The execu
`tion of block 108 follows and causes the Supporting files to
`be read and the respective record contents are Stored in two
`arrayS. Finally, the preliminary work in creating a new
`interface is finished with the execution of block 110. In this
`block the variables “iScreenNum”, “iChildNum', and
`“iNumScreens' are initialized. That is, the first Screen in the
`interface and first child number in that Screen are initialized
`to one and the total number of Screens is Set to Zero.
`In block 110 the choice for a new object is also made. The
`Selection is made by choosing from, for example, a Choose
`Menu for one of the ten NICE objects. The choices are
`“Reply”, “GPB