United States Patent c191
`Kaplow et al.
`
`[11)
`
`[45]
`
`4,202,041
`May 6, 1980
`
`[75]
`
`[56)
`
`[54) DYNAMICALLY VARIABLE KEYBOARD
`TERMINAL
`Inventors: Roy Kaplow, Newton; Mldlael JC.
`Molnar, Cambridge, both of Mass.
`(73] Assignee: Maasachaetts lmtitate of
`Technolol[y, Cambridge, fdass.
`[21] Appl. No.: 814.723
`Jul. 11, 1977
`[22] Filed:
`Int. a,2 ......................... GOff 3/14; Cl06F 13/06
`[51)
`[52] U.S. a ..................................................... 364/900
`[58) Field of Searm ... 364/200 MS File, 900 MS File;
`197 /98; 340/365 R, 324 A
`Relerencea <ltecl
`U.S. PATENT DOCUMENTS
`Shashoua et al. .................... 364/900
`6/1967
`Hart et al. .. .......................... 364/200
`10/1970
`Hamada et al. .. ............... 340/324 A
`12/1971
`5/1973
`Bouchard et al. .. ............. 340/324 A
`Barkan et al . .................... 340/365 C
`9/1973
`Reynolds et al. ........... 340/365 R X
`9/1973
`lcbida ................................... 364/900
`10/1973
`Hacon .......................... 340/365 R X
`11/1973
`Harris ........................... 340/365 S X
`12/1973
`Saito et al. ........................... 364/900
`S/1974
`Watson et aL ....................... 364/200
`1/1975
`Saich ........................... j.to/324 A X
`6/1977
`Sukoniclt et al. .................... 364/900
`1/1978
`OTHER PUBLICATIONS
`Crook, Ken, "CRT Touch Panels Provide Maximum
`
`3,325,786
`3,534,396
`3,631,457
`3,731,299
`3,757,322
`3,760,360
`3,766,528
`3,773,989
`3,777,222
`3,811,113
`3,8S9,63S
`4,028,695
`4,070,710
`
`Flexibility in Computer Interaction", Control Engi(cid:173)
`neering, Jul. 1976, pp. 33-34.
`
`Primary Examiner-Gareth D. Shaw
`Assistant Examiner-Thomas M. Heckler
`Attorney. Agent, or Finn-Arthur A. Smith, Jr.; Robert
`F. O'Connell
`
`ABSTRACT
`[57]
`A dynamically variable keyboard terminal system hav(cid:173)
`ing a keyboard display unit using an input panel with a
`plurality of touch-sensitive locations thereon and a key(cid:173)
`board display unit in alignment therewith. The system
`includes means for generating and storing information
`concerning symbols which make up one or more work(cid:173)
`ing symbol sets, such symbol information being author
`def"mable in accordance with a predetermined terminal
`def"mition language. The symbol information can in(cid:173)
`clude a symbol configuration for display, if applicable,
`or a symbol action to be performed, if applicable. Means
`are also provided for generating and storing informa(cid:173)
`tion concerning a plurality of keyboard images which
`use such symbols and which keyboard information can
`be used for displaying a keyboard image at the keyboard
`display unit. A user can selectively touch one or more
`such locations so as to obtain symbol information for
`display on the keyboard display unit or on a monitor
`display unit, if applicable, or for performing an action
`associated with such symbol, if applicable.
`
`12 Claims, 18 Drawing Fipres
`
`RE FRESH ·MEMO RY
`CONTROLLER
`
`•
`
`16
`
`15
`
`17
`
`PROCESSOR
`ANO
`INTERNAL
`---MEMORY
`
`II
`
`INTERRUPT
`
`KEYBOARD LOGIC
`CIRCUITRY AND
`INTERFACE
`
`TWO 256K • 81 T
`SOLID ·STATE
`MEMORIES
`
`YIOEO SIGNAL
`GENERATOR
`
`12
`
`13
`
`KEYBOARD OVERLAY
`
`CLICK, 8UZ2 1 ETC
`
`Valve Exhibit 1059
`Valve v. Immersion
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 1 of 16
`
`4,202,041
`
`21
`
`DISK-._
`
`16
`
`15
`
`17
`
`PROCESSOR
`AND
`INTERNAL
`___ ......,. MEMORY
`
`II
`
`INTERRUPT
`
`KEYBOARD LOGIC
`Cl RCU ITRY A ND
`INTERFACE
`
`TWO 256K-BIT
`SOLi D -STATE
`MEMORIES
`
`14
`
`VIDEO SIGNAL
`GENERATOR
`
`12
`
`13
`
`KEYBOARD OVERLAY
`
`19
`
`CLICK, BUZZ,ETC
`
`FIG. I
`
`

`

`SEARCH
`SCAN
`SIGNAL
`(FROMFIG.2A)
`
`23
`
`RECORD
`SCAN
`SIGNAL
`FROM FIG.2A
`
`SCAN CLOCK TRIGGER
`
`27
`
`24 __
`
`SCAN
`CLOCK
`(PERIOD= Tc)
`
`,
`
`~-
`>..,
`
`I
`
`25
`_L
`COLUMN I
`I COUNTER
`
`I
`
`COLUMN
`I ~ DECODER
`I
`I
`, - _J
`
`26
`)_
`
`ROW
`~ COUNTER
`
`FIG.2
`
`I
`. I
`I
`
`I
`I I
`I :
`1_1.-:_
`
`28
`
`ROW
`
`DE COD ER
`
`Yo
`
`Y1
`
`Yn
`
`CROSS-POINT
`PULSE
`(WHEN CLOSED)
`
`( TO FIG. 2A) ..
`
`Xo
`
`29 f
`
`x ,~
`
`Xo
`--t
`X1
`
`Xm
`
`IT ~ I
`~
`
`'
`
`r+-~
`
`Xm
`
`~~ r+-
`J
`
`18
`
`'" ..... "
`
`TO ROW, COLUMN
`CROSS- POI NT
`IDENTIFICATION
`REGISTER
`(TO FIG.2A)
`
`it
`
`~
`en .
`
`~
`~
`f""t'
`
`(l) a
`I
`9' -'° 00
`
`0
`
`Cl)
`:::r
`('1)
`
`('1) -N
`a, -0-,
`
`...
`.i::.
`N
`0
`N
`...
`~
`
`~
`
`I
`_j
`w I RE OVERLAY KEYBOARD
`- - - - - - - - - - - - - - - - - J
`- - - - - - - - - - - - - - - - - - -
`
`

`

`CROSS-POINT
`PULSE
`(FROM FlG.2)
`
`31
`
`I
`
`TRIGGER
`Cl RCUIT
`
`,.,cuRRENT CROSS-POINT
`COUNTER (A)
`
`35
`
`RECORD STATUS
`SIGNAL GEN.
`
`30
`
`RECORD
`ENABLE
`SIGNAL
`
`RECORD
`SCAN
`SIGNAL
`
`ROW, COLUMN
`CROSS- POINT
`IDENTIFICATION
`REGISTER
`
`CROSS-POINTS
`LIST TO
`PROCESSOR
`
`I
`ROW, COLUMN
`CROSS- POINTS
`( FROM FIG.2)
`
`t C: -
`
`32)
`COMPARATOR
`FOR (A) a (Bl
`COUNTS
`
`33
`)
`SEARCH SCAN
`
`COMPLETION
`OF SCAN
`NOTED BY
`PRO CESSOR
`
`END OF CHORD
`SIGNAL TO
`PROCESSOR
`
`FIG,2A
`
`OUTPUT WHEN
`A= 0
`B> 0
`
`I
`I
`I
`r-'
`I OUTPUT-
`I WHEN
`• A <B
`I
`I -, SIGNAL GEN. 1 - - - - - - - -
`I
`I
`I
`I
`I_ -----7 t
`. CHORD MODE
`I SIGNAL GEN,
`l
`38
`
`l~R;;;SFER (~ T;(B~~• PREVIOUS
`I AND RESET (Al
`CROSS- PO INT
`L TO ZERO
`COUNTER (B)
`·------
`--L-- ______ ;
`
`I
`
`OUTPUT WHEN
`A>B
`
`RECORD S-CAN
`SIGNAL
`(TO FIG.2)
`
`RESET
`SIGNAL l--34
`GEN.
`
`I NT ERR U PT
`SIGNAL
`GEN.
`
`INTERRUPT
`SIGNAL TO
`PROCESSOR
`
`SEARCH
`
`SCAN
`SIGNAL
`( TO FIG. 2)
`
`36
`I-,/
`
`RECORD
`SCAN FINISH
`
`CHORD MODE
`SW ITCH 37
`
`CHORD
`MODE ON
`
`....
`0...CHORD
`MODE OFF
`
`OUT PUT WHEN
`
`A= 0
`
`C .
`(I) .
`
`~
`~
`r-t-
`(1)
`:::s
`rot-
`
`~
`~
`
`?--\0
`
`00
`0
`
`V'.l
`::,-(cid:173)
`('ti
`....
`('ti
`w
`0 .....,
`_.
`O"I
`
`.i::...
`...
`N
`0
`N
`...
`~ ......
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 4 of 16
`
`4,202,041
`
`INITIATE COUNTERS ( PREVIOUS COUNT~ CURRENT COUNT
`BEGIN
`''SEARCH" SCAN . - - - - . CURRENT COUNT¢ 0);
`OF DYNAMIC KEYBOARD
`
`NO
`
`YES
`
`ADD I TO CROSSPOINT
`COUNTER
`
`(/')
`
`z
`<t u
`:c u
`a:
`<t
`w
`-;(/')
`
`_______ ...,. NO
`
`AD VANCE SCAN TO
`NEXT CROSSPOINT
`
`II
`II
`( START RECORD
`SCAN)
`
`NO
`
`FlG.3
`
`

`

`U.S. Patent May. 6, 1980
`
`Sheet 5 of 16
`
`4,202,041
`
`11 RECORD 1
`START
`'
`SCAN OF KEYBOARD
`FR ESH CROSSPOINT
`BUFFER LIST
`
`NO
`
`A D D CROSS PO I NT
`IDENTIFICATION TO
`CURRENT BUFFER LIST
`IN PROCESSOR MEMORY
`
`ADVANCE SCAN
`TO NEXT CROSSPOINT
`
`NO
`
`YES
`
`z
`<t u
`..
`
`V)
`
`C
`0::
`
`0 u w
`
`0::
`
`11
`
`11
`
`SCAN
`RECORD
`SIGNAL
`FI NISH ED j PROCESSOR SAVES
`CURRENT CROSSPOINT LIST
`AND REPLACES WITH SYMBOL
`IF COMPLETE
`
`11
`SEARCH
`( START
`SCAN)
`
`11
`
`FIG.3A
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 6 of 16
`
`4,202,041
`
`NO
`
`YES
`
`NO
`
`NO
`
`YES
`
`YES
`
`SIGNAL
`END OF CHORD
`
`FIG.3B
`
`

`

`KEYBOARD
`I MAGE
`SET
`
`KEYBOARD
`IMAGE
`DIRECTORY
`
`~~
`
`KEYBOARD
`
`PAGES
`
`~ //1
`I I r
`[7
`DESCRIPTION ll ,,,./
`
`_//1
`
`WORKING
`SYMBOL SET
`
`SYMBOL
`CODES
`
`SYMBOL~-
`DESCRIPTION
`'
`
`~ ....
`
`-
`
`I
`
`-
`
`53---
`
`·~VBOARD
`KEYBOARD
`PAGE
`IDENTITY
`
`' ,I
`50
`
`✓/I
`'
`
`j
`
`LOCAL
`SUB-ROUTINES
`
`/
`
`/
`/'
`
`al
`
`I II
`
`I
`
`I
`
`/
`
`COMPLETE
`DATA BASE
`
`/
`
`52 _,,,;r I SYMBOL SET) •
`
`( FOR USE WITH U v /
`/ ~ FIG.4
`
`~ . Cl.)
`
`•
`~
`~
`~
`
`('D a
`I
`.. a-. -\,0
`
`00
`0
`
`Cl}
`::,(cid:173)
`('D
`
`('D -....J
`0 ..., -°'
`
`~ ...
`N
`0
`tv
`'"
`
`~ ......
`
`

`

`U.S. Patent
`
`May, 6, 1980
`
`Sheet 8 of 16
`
`4,202,041
`
`CROSS-POINTS
`L 1ST FROM
`KEYBOARD LOGIC
`
`"SYMBOL SEGMENT
`COMPLETE'' SIGNAL
`FROM KEYBOARD
`LOGIC
`
`KEYBOARD
`AUDIO
`DEVICE
`
`CURRENT CROss:POINT
`LIST BUFFER
`
`I CHORD MODE I- --
`
`PREVIOUS CROSS-PO INT
`L 1ST BUFFER
`
`...
`
`1
`
`,- -
`
`SEARCH CURRENT KEYBOARD
`DISCRIPTION FOR IDENTIFICATION
`OF KEY CORRESPONDING TO
`NEW CROSS-POINT
`
`GENERATE
`''CLICK" ON
`KEYBOARD
`AUDIO
`
`YES
`
`LEGITIMATE KEY
`
`YES.,__ __ _
`
`NO
`
`'--+-l IGNORE
`
`I
`I
`I
`I
`I
`I TABLE
`
`SHIFT
`NOTATION
`SYMBOL
`CROSS-POINT
`co DE
`
`CURR ENT KEYBOARD IMAGE
`DESCRIPTION
`
`WO R KlNG
`SYMBOL SET
`SYMBOL
`SYMBOL
`DESCRl PT ION
`COE
`
`61
`" 1 - - -4 - - - - - - - - - - -<
`
`IKEY FOUND I
`1---+--lf-----1 y
`I L_
`
`(Kl
`
`60
`
`--,
`
`I
`I
`SYMBOL CODE ____ 1
`I
`CORRESPONDING TO
`I_
`CLOSED KEY (CROSS-POINT)
`
`FIG.5
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 9 of 16
`
`4,202,041
`
`TO KEYBOARD DI SPLAY
`
`+
`
`VIDEO REFRESH
`MEMORY FOR
`KEYBOARD
`
`I 1NPUT SYMBOL BUFFER: LIST OF SYMBOL CODE
`
`( SAVE FOR EDITING OF INPUT LINE
`MW SUBSEQUENT TRANSMISSION)
`
`ADD SYMBOL
`CODE TO INPUT
`8 UFFER j THIS
`'
`LINES INPUT
`STREAM
`
`DRAW SYMBOL IN
`11
`11 I NP UT L I N E
`SPACE ON
`KEYBOARD AND
`EXECUTE OTHER
`PRE-TRANSMIT
`SYMBOL ACTIONS
`
`1
`
`i
`'
`I
`l
`I
`- _L_l 1
`I
`r - __ J
`I ,-,-.lJ
`I
`I
`I
`I
`
`WORKING SYMBOL SET
`
`EXEMPLARY
`SYMBOL DESCRIPTION
`
`SYMBOL VISUAL DESCRIPTION
`CODE
`
`I
`I
`I
`I_
`L __
`
`O,PTIONAL SYMBOL
`CODE SEQUENCE
`
`DELAYED ACTION?NO
`
`TRANSMIT? YES
`
`TERMINATE LINE ( NO
`IMMEDIATE SEND ? NO
`
`NON - DELAYED ACTIONS
`
`FIG.6
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 10 of 16
`
`4,202,041
`
`TO MONITOR
`DISPLAY
`
`DATA LINE TO
`COMPUTER OR
`SECONDARY STORAGE
`
`I INPUT
`
`BUFFER
`j
`
`VIDEO REFRESH
`MEMORY FOR
`MONITOR
`
`I
`I
`L __ -- -,
`I L -
`'
`
`I.
`)
`
`TRANSMIT TO
`HOST COMPUTER
`
`\\
`II
`INITIAL PART CHECK TO SEE EXECUTE DELAYED
`OF PROCESS
`IF SYMBOL IS
`ACTIONS FOR ALL
`11TERMINATE''
`EXACTLY AS
`SYMBOLS
`IN
`IN AGS, 5 Si 6
`TYPE
`INPUT BUFFER
`OPTIONALLY DISPLAY
`----------..--,.--1-.------.-1 ALL SYMBOLS
`IN
`INPUT LINE ON
`MONITOR DISPLAY UNIT
`
`IF YES
`
`REPRESENT! NG
`ALL SYMBOLS
`IN INPUT
`BUFFER
`
`DESCRIPTION OF SYMBOLS
`SYMBOL LIST
`
`r - - -
`
`I
`_____ I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`L
`
`SYMBOL
`CODE
`
`VISUAL DESCRIPTION
`
`TERMINATE LINE? YES
`
`FIG.7
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 11 of 16
`
`4,202,041
`
`L __ J
`
`r - PAGE IDENTITY ~I
`I
`- - - - - t
`I
`I
`I
`LOCATE KEYBOARD
`PAGE
`IN KEYBOARD
`10
`DIRECTORY INFORMA(cid:173)
`TION STORE
`L ___ _
`
`KEYBOARD
`DI RECTORY
`INFORMATION
`
`_KEYBOARD
`
`IDENTITY
`CODE
`
`LOCATE KEYBOARD
`DESCRIPTION DATA
`PAGE IN DATA
`PAGE STORE
`I
`_ __ j
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I_
`
`I
`,.., ____ ,, ~1t- -
`
`KEYBOARD
`DESCRIPTION
`DATA PAGES
`t DISK
`MEMORY}
`
`G)
`
`READ KEYB-OARD
`DESCRIPTION DATA
`INTO PROCESSOR
`MEMORY
`
`1 1
`
`CR-OSS(cid:173)
`POINTS
`ENTRY
`LIST
`(PROCES.
`MEM,)
`
`r GLOBAL DES CR IP,
`I
`t;::::==:=======:J
`CROSS-POINT ENTRY
`:..
`cRossPOINT IDENTITY a SHIFT#
`
`/ / SYMBOL CODE/SHIFT LEVEL
`I 1 - - - - - - - - - t v ,
`LABEL#
`I • - - - . . - - - - '
`OUTLINE#
`..
`t_ - _ . ,~ -~ '- MARKER#
`\
`
`---
`
`LOOKUP
`SU BRO UTI NES
`FOR EACH
`I
`
`I
`
`LABEL
`STRING
`END
`
`OUTLINE
`VECTOR
`SET
`END
`
`0
`
`DIS PLAY
`KEYBOARD
`
`DI SPLAY GLOBAL 8
`STEP THROUGH
`CROSS-POINT ENTRY
`LIST 8 DISPLAY
`MARKERS, OUTLINES,
`LABELS ON KEY -
`BOARD DISPLAY
`AT COMPUTED
`KEY LOCATION
`
`I •
`KEYBOARD D
`
`DISPLAY
`UNIT
`
`@ ,--
`•
`
`MARKER
`MATRIX
`END
`
`I
`I
`I
`I
`I
`I
`I
`I
`L _1_
`
`\
`
`\
`
`\
`
`\
`
`FIG.8
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 12 of 16
`
`4,202,041
`
`EXAMINE NEXT
`CROSS POINT ENTRY
`
`PERFORM
`•• INTERPRET
`
`DESCRIPTION"
`
`FrG.9
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 13 of 16
`
`4,202,041
`
`ENTER NEXT
`GRAPHICAL
`OPERATOR FROM
`DESCRIPTION AND
`CL ASS I FY AS TO
`TYPE
`
`DRAW
`MATRIX
`
`DRAW
`VECTOR
`
`DISPLAY
`STRING
`
`EXECUTE
`SUBROUTINE
`
`FIG.10
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 14 of 16
`
`4,202,041
`
`OUTLINE
`
`LABEL
`
`KEYBOARD
`GLOBAL
`
`CHOOSE A DEMONSTRATION J
`
`8 STANDARD TYPEWRITER
`8 ASSORTED FIGURES
`8 SPIRALS
`8 DRAWING ON THE DISPLAY
`8 DRAWING ON THE KEYBOARD
`C"l CHANGE Q
`Q CLEAR
`(:"'i PRINT
`L.:.J KEYBOARD L:J HELP
`L..:.) DI SPLAY L:..) DIS PLAY
`
`KEYBOARD
`GLOBAL
`
`FIG.II
`
`OUTLINE
`
`CONSTRUCT A Fl GURE ON THE KEYBOARD DIS PLAY
`
`LA8E.L
`
`f l
`CLR
`t..__J
`
`Q CHANGE Q
`LJ KEYBOARD ~ t-1 ELP
`
`Q PRINT
`(:"") CLEAR
`l:.J DISPLAY LJ DI SPLAY
`
`FIG.12
`
`

`

`U.S. Patent May, 6, 1980
`
`Sheet 15 of 16
`
`4,202,041
`
`CONSTRUCT A FIGURE ON THE KE.YBOARD DISPLAY
`
`0 ____ ...----T _ _
`,,
`,,
`LINE
`
`@
`® 1 . . - _ . - - . : ; : . - - -
`
`,,
`LINE
`
`II
`
`,--..,
`LINE
`"--"
`
`r---"I
`ERAS
`"--"
`
`r:-) CHANGE
`r:-)
`L'.J KEYBOARD~ HELP
`
`f':l PRINT
`r;--"l CLEAR
`L:.) DISPLAY L:) DISPLAY
`
`FIG.13
`
`MARKER
`
`(7
`r.)CHANGE
`L.:J KEYBOARD L.) HELP
`
`f.1 CLEAR
`l:) DISPLAY
`
`PRINT
`DISPLAY
`
`FIG.14
`
`

`

`
`
`U .s. Patent May, 6, 1980
`U.S. Patent May, 6, 1980
`
`Sheet 16 of 16
`Sheet 16 of 16
`
`4,202,041
`4,202,041
`
`Fr SH
`FISH
`
`FIG.15
`FIG.IS
`
`

`

`1
`
`4,202,041
`
`DYNAMICALLY VARIABLE KEYBOARD
`TERMINAL
`
`The Government has rights in this invention pursuant 5
`to Grant No. J-43163 and IPA-0010 awarded by the
`National Science Foundation.
`
`2
`formity of notation. This is distracting and confusing,
`particularly in an educational situation.
`Another interesting type of eumpJe, which stresses
`the differences between display and input capabilities,
`can be drawn from a report on a sophisticated, multi(cid:173)
`c:haracter cathode-ray-type display terminal developed
`at the MIT Electronic Systems Laboratory, for use in
`libraries, as part of the lntrex project (See "A Transla-
`tor Program for Displaying a Computer Stored Set of
`BACKGROUND OF THE INVENTION
`The field of this invention is the computer terminal 10 Special Characters,. ~obyuki, Goto. Report ESL-R-
`and particularly interactive terminals which permit
`429,_M~husetts ~nstitute ofTechn~logy, 1970). The
`access and operation with respect to computers by op-
`partial line of text illustrated below_ (m quotes). ~epre-
`erators using as an input device a keyboard or some
`aents a si~cant s~ beyond the display capa_bility of
`such form or data entry arrangement by which the
`run-of:the-mill terminals. The use of superscnpts and
`operator can utilize the power available in the computer 15 subac:npts ~ ofn~tandard symbols allows the ~hem-
`ical formula m the line to be represented on the display
`• ted
`and software
`d ·ts
`tandard fi
`an 1 associa memory
`•
`• •
`m tts s
`In the prior art, one standard form or computer termi-
`orm:
`nal has been a keyboard device of the teletype or type(cid:173)
`" ... in the system Mn;rFe1-A+yfor < 1 •• .''
`writer variety and at present such devices have the
`ability to provide the USASCII, 1968 STANDARD 20 Th same report also explains how that information
`SET OF NINETY-FIVE PRINTABLE CHARAC
`e
`.
`TERS OR SOME PORTION THEREOF. This stan-
`must be mput:
`dard set or symbols which has evolved from the type(cid:173)
`writer with its keyboard and symbol set is generally 25
`associated with a video display output device which
`The augmented display capability illustrated above is
`will dis~lay the selected characters in the form of the
`text ~hich has been entered b>: ~ k e~ and con-
`certainly valuable in its own right. However, there is
`teas advantage in its use if the user must train himself to
`ventio~ arrangements for editing, era&lD8 and other
`30 use the different inputting code. It is perhaps surprising
`operati~ns:
`.
`.
`.
`that the latter form is as readable as it is; for a person
`The limitations of pnor art terminals may not have
`unfamiliar with chemistry it might even be more read-
`seemed unduly restrictive in many quarters. Indeed. the
`able than the correctly displayed form. As an input
`concept of defining the ~vailable _symbols, or gro"8 of
`scheme for "chemists", however, spelling out the non-
`symbols, to have special meanmp under partieular
`circumstances has allowed the standard symbol set to 35 standard symbols and format in English-like text is at
`do more than double duty. Thus f and•• and .P. have
`best a tedious and error-prone option.
`all three been used to mean "raise to the power''; "0
`Other schemes, deviating further and further from
`backspace -•• may mean "the Greek letter theta,.; "INT
`the standard symbol set, have been developed in re-
`(a,b,f,x)" may mean
`sponse to the continuing pressure for more and different
`40 input symbols. One approach has been to replace the
`entire standard keyboard symbol set with another one,
`SJ!eCially designed for a specific application. Perhaps
`the best known example of this is an APL terminal, for
`.
`..
`...
`.
`..
`which the keyboard labels and type head are changed to
`and A.SUB.1+J may mean Ai+F· In an envtron- 45 provide a unique set of symbols, the symbol set of the
`ment in which much of the serious communication from
`APL programming language.
`the ~ to the com~uter is still r_eferred to as ''writing
`Another approach has been to add more keys to the
`code such _conventions ar! ~ily -~ted._ And Yet.
`keyboard. Some of the extra keys may be assigned high-
`th~re are ugns ~t the s1tuabon II increasingly less
`level operational meaning. such as log x or eX. Bxperi-
`sattsfactory, particularly as the user group becomes less so mental terminals for engineering applications have been
`dominated by computer professionals and demi-profes-
`developed with as many as 250 ''side" keys, alongside
`sionals.
`the standard keyboard. Nonetheless, such keyboards are
`The alternate forms in which the standard symbol
`still mainly limited to the fixed, though larger, symbol
`sets are used are not desirable in instructional applica-
`set which happens to be included. An additional disad-
`tioos, for example. because they require the student to 55 vantage to this, as a general approach, is simply the
`learn extraneous codes and because they stress nota-
`distraction that is presented by so formidable an array or
`tional schemes different from those used in the pub-
`keys, only a fraction of which are likely to be used in
`lished literature and in textbooks. They are also trouble-
`any brief period of time.
`some because the particular notational conventions are
`A related scheme has been to assign multiple sets of
`relatively arbitrary; the convention adopted by one 60 meanings to a fixed number of keys. This is an extension
`author is likely to be different from his colleagues' and
`of the ordinary "shift,. concept which allows each key
`different from the conventions used in algorithmic pro-
`to represent two different symbols, depending on
`gramming languages, such u Fortran, Cobol, MAD,
`whether the shift key is depressed or not. With multiple
`etc. These languages, in turn, use conventions which meanings, the user may be required to type a special
`differ from one to another. Thus, the use of a limited, 65 code-e.g., LEVEL III-to indicate that he desires to
`standard symbol set which does not include the natural
`switch to a particular set of meanings for the keys. It is
`symbols (or positioning) of particular applications para-
`difficult to provide many readable labels per key, how-
`doxically tends to lead to a diversity rather than a uni-
`ever, so the basic limitation remains.
`
`" ... in the system Mn-.Ub x•Fe•sub 3-x•o•sub 4+
`•gamma•• for x •teas than* I .. .''
`
`"
`
`i,
`I Axr:
`0
`
`

`

`4,202,041
`
`3
`An interesting attempt to provide a multiplicity of
`key labels for a multi-"shift"-level scheme has been
`made by the Philco Houston Operations of the Philco(cid:173)
`Ford Corporation (See "New Conception Man(cid:173)
`Machine Interface Devices", Developments Section, S
`Computer Designs, 1, 32; 1968). Their device consists of
`a "keyboard" comprising two 4 X 4 arrays of push but(cid:173)
`tons and an (exchangeable) ten page "book" of plastic
`sheet label overlays which have holes through which
`the buttons can protrude. The book spline locks into the to
`center of the keyboard (between the two separate 4 X 4
`button arrays). When the book is open to a given pair of
`pages the overlays provide the key labels. Mechanical
`cams attached to the book spline and to the individual
`pages actuates switches under the keyboard to identify 15
`the particular book pad pages selected.
`In a related but more recent approach, workers at the
`Bell Telephone Laboratories have assembled a device
`in which key labels are projected onto a mechanical-key
`keyboard, while the use of partially-silvered mirrors 20
`allows the user to see his hand, the keys, and the label(cid:173)
`s-all simultaneously (See Knowlton, K., "Virtual
`Pushbuttons as a Means of Person-Machine Interac(cid:173)
`tion", Proceedings Computer Graphics, Patterson Rec(cid:173)
`ognition and Data Structure Conference, Beverly Hills, 25
`California, 1975).
`Another approach, used with CRT and other display
`terminals, has been to generate "soft-copy" input areas,
`in which special symbols or function names are dis(cid:173)
`played on a reserved section of the CRT screen and 30
`selected by pointing at them with a light-pen, or by
`moving a cursor controlled by a "joy-stick" or track(cid:173)
`ball, or by interrrupting a light beam in an array, or by
`a similar technique. The previously mentioned MIT
`Project-Intrex BRISC terminal extended this idea by 35
`displaying variable labels to identify a set of extra push
`buttons mounted along the bottom of the CRT screen.
`"Soft-copy" input modes implemented on the display in
`these various ways can provide an additional richness
`and flexibility in the input symbol set but are generally 40
`slower to work with than separate keyboards and are
`often not suitable for routine input. They are thus used
`in combination with a standard keyboard, which results
`in the user having to transfer back and forth from one
`type of input device to the other-he must drop the 45
`light pen to type, for example, and vice versa. More
`importantly, information input in that fashion is usually
`handled in a manner that is not homogeneous with nor(cid:173)
`mal input. That is, with normal input-handling software
`a key push is simply converted into a binary-bit code, SO
`unique to the particular symbol selected, and appen(cid:173)
`ded-if appropriate-to a symbol string input-stream.
`Such a string is subsequently amenable to the usual
`range of "character-handling" operations, and, of
`course, it can be edited later or redisplayed. In contrast, 55
`with input schemes like those referred to immediately
`above, the controlling software routine is likely to be a
`specialized one, designed in conjunction with the spe(cid:173)
`cific picture then being displayed, and directly carrying
`out one branch of the program or another, depending 60
`on the numerical x and y values of the screen locations
`selected.
`A second basic type of flaw in the usual input device
`is the lack of format flexibility. The user faces a fixed
`key layout, usually with fixed key labels; an instrument 65
`based on a design for putting English prose onto paper.
`As it happens, the standard symbol layout used is not
`ideally suited even for that purpose, but that quantita-
`
`4
`tive detail is almost irrelevant in the present context. It
`is important, however, to consider the possibility that
`touch-input communications to a computer may be
`qualitatively enhanced through the use of formats more
`cJosely related to the specific interactions underway.
`A third major flaw regarding standard, all-purpose
`keyboards, is that they are detrimental to the efficiency
`of human-computer interaction, especially in consider(cid:173)
`ation of errors and the fallibility of human memory. In
`particular, it is not desirable to always present the user
`with a single keyboard through which he can send any
`communication which is, which has ever been, and
`which might ever be a valid statement. Such keyboard
`does not provide any clues as to what input is meaning(cid:173)
`ful at a given instant in a given process, although the
`computer software is often very "knowledgeable" and
`sensitive about which is legitimate and what is not. At
`the same time, the very generality of the standard key(cid:173)
`board provides unlimited opportunity for error.
`
`SUMMARY OF THE PRESENT INVENTION
`The enhanced-input-terminal of the present invention
`provides major new degrees of freedom for touch-type
`input, especially for on-line use of interactive computer
`programs and comprises an integrated system of hard(cid:173)
`ware and software. The primary input device is a
`"cross-wire", touch-sensitive panel, overlaying a video
`display. A companion video unit is used for output
`display. These are supported by page-refresh memories,
`written (and read) directly by a suitable data processor
`which implements the software logic. The enhanced(cid:173)
`input-terminal a11ows the user to define an essentially
`infinite set of symbols and an unlimited variety of "key(cid:173)
`board" formats. In spite of the generality of input con(cid:173)
`tent and format, the integrated system develops a stand(cid:173)
`ard-type of binary-bit-coded input stream, in which the
`individual "symbols" are uniquely and canonically rep(cid:173)
`resented. As symbol-strings or as a text-file, the input
`stream is amenable to all of the usual "character-han(cid:173)
`dling" operations, such as substring, concatenate, etc.,
`and to operations of editing, transmission and output.
`The system includes a unique ''Terminal Definition
`Language" and compiler, with which one can define
`new symbols, keys, chords (as later defined), and key(cid:173)
`board images, all comprising an author working set of
`keyboard images and symbols for a user process. Thus,
`a specific set of symbols, keyboard formats, key labels,
`and local operational functions can be defined for each
`task and suitably stored to form a data base for the
`system.
`The objects and features of the invention are thus to
`provide a terminal having available:
`( 1) Working symbol set stored in the data base, the
`symbols making up such set being author-definable
`and the set being virtually unlimited in terms of the
`number of symbols or the visual aspect of the sym(cid:173)
`bols;
`(2) Keyboard descriptions permitting such keyboards
`to be displayed or drawn on a keyboard display
`unit, the keyboard formats being essentially com(cid:173)
`pletely arbitrary, allowing an essentially unlimited
`number of possibilities ranging from the standard
`key layout of a typewriter, to drawings, templates,
`or any format oriented to inputting a problem and
`working through to a solution by interaction with
`the computer;
`(3) Descriptions for using keyboards, the keyboards
`being dynamically switchable, keeping pace with
`
`

`

`4,202,041
`
`6
`5
`into the refresh memories, a video readout and signal
`the user's path through an interaction, being specif-
`generator 17, and a "keyboard" overlay 18 with associ-
`ically relevant to the then current aspect of the
`ated keyboard logic control and processor interface
`task;
`circuitry 19 are also included in the system. Controller
`(4) Sub-routines for use in the presentation of symbols
`from the symbol set. Input actions by the user gen- 5 16 and its interface circuitry is of a standard type and
`crate a binary coded input stream which is explic-
`includes ten registers directly accessible by the proces-
`itly meaningful in terms of the defined symbol set,
`sor and through which information is transferred to and
`and which has the attributes of an ordinary text
`from the refresh memories. The keyboard overlay 18 as
`used in conjunction with the video display 13 makes up
`that one applies to text files (or symbol strings)
`such as:
`10 the keyboard display unit which operates as the termi-
`(a) character-handling operations (e.g., length,
`nal's primary input device for use by the terminal opera-
`concatenate)
`tor.
`(b) string comparison (e.g., for response analysis
`The processor II provides its computational power,
`and interpretations)
`program and library storage media, and storage access
`(c) editing
`15 mechanisms in a conventional manner. A PDP 11/40
`(d) outputting (including display format control).
`data storage media, for example, can include 28 k bytes
`Input is canonicalized, so that a given symbol is al-
`of internal core memory and 4.8 megabytes of fast disk
`ways repr~ted identically_ intef!lally, regardless of
`storage 21. The processor II is programmed in assem-
`the keyh?ard-1mage used to ,~put it and regardl~ ?f
`bly language and utilizes a standard disk-operating sys-
`the _ locations ?f the keys on. ~1fferent keyboards within 20 tem for disk storage 21. With the core memory and disk
`a give~ workin~ set. In add1t1on, to the extent allo:Wed
`storage, the processor II provides most of the opera-
`~Y logical consistency, s~bols may a19;0 be can_onical-
`tional logical power, the character and display process-
`1zed among tw? or more ~1fferent wo_rki~g sets; indeed,
`ing, and the computational requirements of the termi-
`the A:SCII set 1s automatically canomcalized among all
`nal. Details of the operation of particular processors are
`25 normally well known to the art and that of the PDP
`working sets.
`DESCRIPTION OF THE ORA WINGS
`11/40, for example, can be found in the operating in-
`FIG. I is a system block diagram of a preferred em-
`st~ction manual and other information normally sup-
`bodiment of the invention;
`plied to ~rs thereof by the manufacturer. Accord-
`FIGS. 2 and 2A show in general a block diagram of 30 mgly, detail~ thereof are not_ further need~ here._
`the keyboard logic depicted in FIG. I;
`~e user mterface comp~ the two video dtSplay
`FIGS. 3, 3A and 3B contain an overall flow chart
`~onttors 12 ~d 13. The m?nttor ~2 serves th~ no~
`showing the operation of the keyboard logic of FIG. l
`display. functions of a te_rmmal w~tle ~e momtor 13 1s
`in identifying cross-point closures at a keyboard display
`the basts for the dynamically variable mput keyboard.
`unit.
`35 Each monitor is backed up by a 256 K-bit solid-state
`FIG. 4 is a diagrammatic representation of the overall
`"re~resh" memory 14, ~5 and video-dri".er ci_rcuit~ 17
`data base used in the invention·
`which develops a continuous black/white video mput
`FIG. 5 is a diagrammatic rep~esentation of the opera-
`signal from t~e bit _Pat~ms in the respective ~emories
`14, 15: The ~1deo ctrcwt~y 17 gene~tes a 454 hne sc~,
`tion required to identify a symbol and to acquire a sym-
`bol description associated with a cross-point selected by 40 each hne bemg resolved mto 576 potnts. The memones
`14, 15 are double-ported, and the memory access speeds
`a user;
`are sufficiently fast that read-and-write access by the
`FIGS. 6 and 7 are diagrammatic representations of
`processor II does not interfere with the read activity of
`the operations required to display the visual appearance
`video circuitry. Thus, the displayed image on either
`and to perform an action associated with symbols se(cid:173)
`monitor can be modified essentially at processor speeds.
`lected by a user and to transmit their symbol codes 45
`In addition, bit-map images stored on the fast disk mem-
`when required;
`ory can be transferred directly to the video refresh
`FIG. 8 is a diagrammatic representation of the opera(cid:173)
`memory using conventional computer interface hard(cid:173)
`tion required to change a keyboard image at the key(cid:173)
`ware. In this fashion, the system can switch keyboard
`board display unit;
`images in the order of 0. 3 seconds.
`FIGS. 9 and 10 are flow charts depicting the steps so
`The memory word addressing unit associated with
`required in a portion of the operation shown in FIG. 8;
`the display hardware allows an automatic multiple(cid:173)
`FIGS. 11-15 depict exemplary keyboards useful in
`word incrementing mode, which is extremely conve(cid:173)
`explaining a typical operation which could occur in
`nient for displaying dot matrix characteris one at a time.
`using the system of the invention.
`55 The memory control 16 allows software controlled
`DETAILED DESCRIPTION OF THE
`switching among the memories for the computer read(cid:173)
`PREFERRED EMBODIMENT
`write operations.
`While only the two aforementioned video displays
`The system shown in FIG. 1 consists of three main
`are implemented in the current configuration, up to
`parts: a processor II which in a particular embodiment
`sixteen 256 K-bit memories can be addressed without
`may be, for example, a processor made and sold by 60
`requiring the normal memory extender unit, for exam-
`Digital Equipment Corporation of Maynard, Massa(cid:173)
`ple, on the PDP 11/40 itself, and without using any of
`chusetts under the model designation PDP 11/40. Such
`the computer's own memory address space. Thus, the
`processor has appropriate data storage media, includ(cid:173)
`ing, for example, a peripheral disk storage memory unit
`exemplary system can be augmented to support as many
`21 as well as processor internal core memory. Two 65
`as eight separate user-stations, each having the desired
`video monitor display units ll,13 with solid-state, page(cid:173)
`two video displays. Alternately, extra memories might
`refresh memories 14,15 and a memory control circuit 16
`be used to provide grey scales or multi-color signals, or
`for controlling the insertion of data from the data base
`various combinations for a smaller number of users.
`
`

`

`4,202,041
`
`7
`The third major component of the hardware is the
`keyboard display unit which includes overlay panel 18,
`with