READING RB SKETCH BY HUNCH
`
`by
`
`James kichard ‘laggart
`
`Submitted in Partial Fulfillment of the Requirements
`
`for the Degree of
`
`Master of Science
`
`at the
`
`Massachusetts Institute of Technology
`
`May,
`
`1973
`
`Signature of Author
`
`
`Department of ElectricNEngineeEngineering,
`
`__
`lI, 1973
`
`-
`Thesis supervisor“tpeadeatIc)ny -
`Certified by_ ee weenneeyee
`wet?
`
`penn
`Aecepted bg__
`Chairman, DepartmentalCommittee on Graduate Students
`
`~~
`
`ates mt BRARLES
`
`Valve Exhibit 1067
`Valve Exhibit 1067
`Valve v. Immersion
`Valve v. Immersion
`
`
`
`by
`
`James kichard ‘laggart
`
`Submitted in Partial Fulfillment of the Requirements
`
`for the Degree of
`
`Master of Science
`
`at the
`
`Massachusetts Institute of Technology
`
`May,
`
`1973
`
`Signature of Author
`
`
`Department of ElectricNEngineeEngineering,
`
`__
`lI, 1973
`
`-
`Thesis supervisor“tpeadeatIc)ny -
`Certified by_ ee weenneeyee
`wet?
`
`penn
`Aecepted bg__
`Chairman, DepartmentalCommittee on Graduate Students
`
`~~
`
`ates mt BRARLES
`
`Valve Exhibit 1067
`Valve Exhibit 1067
`Valve v. Immersion
`Valve v. Immersion
`
`
`
`READI*‘G A SKETCH BY 4UNCH
`
`oy
`
`James Rizaacd Taggart
`
`Submitted to the Lepartment of Electrical Engineering on
`May ll, 1973 in partial fulfillment of the requirements for
`the Degree of Master of Science.
`
`RBSTRACT
`
`This thesis describes the development of a computer system,
`HUNCH,
`intended to provide a simple means for a person to
`communicate with a computer his ideas through the medium of
`sketching.
`The emphasis is not on developing a computer
`system which produces finished quality drakuings from
`sketched input, but rather on having the computer understand
`what is meant by the sketch.
`An overview of
`the intended
`goals of such a system is described, along with a comparison
`to other systems of sketch recognition.
`A history of the
`development of HUNCH is given to siow the reader the
`evolution of the ideas invoked by HUNCH as it currently
`stands.
`A description of how dUNCi performs a data
`reduction pass to simplify and structure the sketch is
`given. Finally, a proposal for. a graphical compiler is made
`to permit development of a system which watld be able to
`understand sketches of a predefined class.
`
`Thesis Supervisor: Nicholas P. Negroponte
`Title: Bssociate Frofessor of Architecture
`2
`
`
`
`ACKNOWLEDGEMENTS
`
`It is traditional to thark everybody in the world in this
`section. Special reference must be made to Vick Negroponte for
`providing the inspiration, anc to the NSF and Project MAC for
`providing the means for keeping body and soul together while the
`Froject was carried out. There are also nameless millions who
`have contributed over the years to the development of HUNCH,
`especially Doris Ju and Cindy C*Connell
`sho had to work under my
`direction, and Mike “%illzr and Chris Herot.s who have been bricks
`throughout.
`Leon Groisser should be tharked for providing a
`needed boot at a crucial moment. Finally, a nod to Archie the
`Frchitect, for whom this systen has heen developed.
`
`
`
`TABLE OF CONTENTS
`
`ABSTRACT
`
`ACKNOWLEDGEMENTS
`
`TABLE OF CONTENTS
`
`ORGANIZATION
`
`OVERVIEW; Why am I here?
`
`SECTION I:
`GOALS
`OTHER SYSTEMS
`LETLOWN
`WHAT HUNCH DOES AND DOES NO?
`HARDWARE USEE BY HUKCh
`
`bow did !i get here?
`PAST:
`SECTION II:
`READING AND REDISPLAYING--DRAW/SHOW
`SQUIGGLES
`EXTRACTING LINE sEGMENTS--STRAIT/STRAIN
`RATE/PRESSURE
`INTERSECTIONS--iNSECT
`HORIZONTALIZING ANE VERTICALIZING--LEVEL
`CURVES
`EDITING
`
`SECTION IiI:
`
`PRESENT; Where am I now?
`
`FUTURE: Where dc we go from here?
`SECTION IV:
`DESCRIPTION OF A SKETCH
`EXISTING KNOKN SETS
`LINE SET ATTRIBUTES
`CTHER KNOWN SETS
`FEATURES AND SET ATTRIBUTES
`RECURSE
`
`REFERENCES
`
`APPENDIX I:
`
`EXISTING DATA STRUCTUFE
`
`APPENDIX II:
`
`ThE GRID FACILITIES
`
`APPENDIX III:
`
`LISTLAGS CF STRAL
`
`
`
`ORGANIZATION
`
`The body of this paper
`
`is divided into four parts: overview,
`
`past, present, and future. The first part of
`
`the paper is a look
`
`at the goals for developing an interactive sketching systen.
`
`Other
`
`implimentations of computer systems are described, and how
`
`they matched up to the defiired goals is discussed.
`
`The second
`
`part
`
`is a history of events from its inception to HUNCH in its
`
`present state.
`
`Sucn a description seems
`
`important for two
`
`reasons. First, while the way HKUNC: works can be understood
`
`without the history attached, why HUNCH exists as it is, and the
`
`motivation for the fourth part of the paper needs as part of its
`
`explanation how the ideas were derived and which ideas were
`
`discarded.
`
`HUNCH in its present
`
`form appears to be a regression
`
`from earlier successes.
`
`The motivation for this change of state
`
`is best explained by describing the sequence of events which led
`
`to the current state. Second, some of what has been learned about
`
`sketch recognition through the development of HUNCH is represen-
`
`ted only by certain elements which are NOT included, perhaps
`
`despite earlier versions with these features.
`
`The knowledge
`
`gained by failures and chanjes of attitude over time is almost as
`
`great as that which currently is known. This history of HUNCH,
`
`then,
`
`is an attempt to apprise the reader of this knowledge.
`
`The third part of the paper is a description of how the system
`
`as it currently exists is run.
`
`An extensive description of how
`
`5
`
`
`
`straight line data is extracted from the raw sketched input is
`
`included. This descripticn not only shows how the more
`
`complicated portions of HUNZH work, but also indicates the
`
`philosophy of how operations are performed .in a HUNCH-like way.
`
`From this outlook, one can se2 how other functions, which might
`
`be added to the HUNCH systen later, would be implemented.
`
`The last section of the paper may be looked an as conclusions
`
`and indications for future work.
`
`As it exists now, HUNCH falls
`
`short of its stated goals by quite a distance.
`
`Its biggest
`
`shortcoming is in its inability to derive a higher level
`
`description from a sketch.
`
`The last section provides the frame
`
`of mind which one night need in order to begin solving this
`
`inability.
`The solution proposed is neither complete nor
`rigorous, and in that sense, it seeas strangely inconclusive.
`
`The only explanation 1 can offer is that the solution proposed
`
`seems to solve all the challenges I. can think of, although
`
`sometimes the thinking required seems unnecessarily baroque.
`
`There must be simpler solutions, but finding them can only come
`
`with further experience.
`
`
`
`I.
`
`OVERVIE’
`
`Why am I here?
`
`GOALS
`
`There have been many computer systems developed which purport to
`
`let the user sketch using a computer.
`
`The user is placed before
`
`a console display of some sort, handed something which looks like
`
`a pen, perhaps is given some instruction in how to use the
`
`system, and is told to fraw.
`
`it is reasonable to wonder
`
`in
`
`abstract,
`
`if one tac such a system, what would he want it to do.
`
`There seem to be two answers:
`
`first, the system should help in
`
`the construction and storage of graj;hical images. Second,
`
`the
`
`system should act as an aide in the development of the
`
`information the sketch is meant to convey.
`
`Under
`
`the first goal, when pictures can be constructed out of
`
`elements, saved, wodified, and recalled at a later time,
`
`the
`
`designer has a useful time-savirg tool for handling pictorial
`
`data. Repetitive elements need only be described once to the
`
`systen. Representations of the complete structure can then be
`
`evoked with only a single stroke of the pen. Thus,
`
`instead of
`
`the designer can describe the
`laboring hours over a drawing,
`whole drawing to the computer using many previously defined
`
`elements, and the computer can construct the complete, finished
`
`work. Similarly,
`
`if two drawings are the same, with only minor
`
`7
`
`
`
`variations,
`
`the designer can construct one of them and store it
`
`away-
`
`He can then modify a copy of the saved image to match it
`
`to the second intended drawing, saving himself the trouble of
`
`constructing the drawing essentially twice.
`
`More complicated than the first goal of a sketching system, one
`
`might ask a system to perform some more abstract operations on
`
`the sketch.
`
`A person uses sketches for two purposes:
`
`to convey
`
`information to other people which is difficult to transmit
`
`verbally, and to act as a sort of physical memory,
`
`in a sense,
`
`conveying informaticen to himself.
`
`‘(nce the sketch has been
`
`commited to paper,
`
`the user can modify it to change the
`
`information it contains. This act can be prompted either hy the
`
`ebb and flow of the dialogue with the observer, or by a change in
`
`the sketcher*s own :idea Erought about by the feed-back loop
`
`running between brain, hand, paper, and eye.
`
`In either case,
`
`the
`
`sketch is important because of
`
`the intended meanings it contains.
`
`In a similar way, it is useful for a computer system being used
`
`as a sketching tool to be able to attach some meaning to the
`
`objects being sketched.
`
`The result of such a dialogiae is that the information contained
`
`in the interaction is greater than the amount of
`
`information
`
`which could be contained in the sketch alone, or which the user
`
`could carry around :in his head. Thus, one would like a computer
`
`system for sketching to be alert enough to be able to affect a
`
`8
`
`
`
`dialogue with the user.
`
`It would need to be knowledgeable enough
`
`about the subject matter being sketched to be able to ask
`
`reasonable (intelligent?) questions, and perhaps offer some
`
`information of its own.
`
`In short,
`
`the computer should be able to
`
`enter into a dialogue with the user,
`
`in much the same way as a
`
`person observing the sketch being created might interact.
`
`Such a
`
`provocative system would tend tc maximize the amount of
`
`information generated in a sketching session.
`
`OTHER SYSTEMS
`
`Computer systems which have been developed to date tend to. be
`
`divided into two classes, reflecting to a certain extent the two
`
`goals for a computer sketching system.
`
`The first class,
`
`historically,
`
`is that which uses some syecialized set of
`
`functins, keys, or symbols in the process of sketching.
`
`Input
`
`was accomplished by invoking a function (key, e.g.}, which told
`
`the computer what the user was
`
`intending to do, with the ultimate
`
`goal of permitting ‘the computer to store, retrieve, and assist in
`
`modifying a drawi1g.
`
`In rasponse to the user‘s request,
`
`the
`
`system performed some output which accomplished the action
`
`specified.
`
`The second sort of computer system seen uses a
`
`limited set of known symbols, and attempts to map the user's
`
`sketched ikons, usually irawn on a data tablet,
`
`into these
`
`symbols.
`
`In this case,
`
`the comjuter does not know in advance
`
`exactly what the user is going to do. What the user intended
`
`S
`
`
`
`must be inferred from the match of the sketched item to the knoun
`
`symbols, and the computer is usually expected to take some action
`
`as a result of the recognition of
`
`the symbol. Because of this
`
`level of guessing, such systems are not infallible, but this
`
`objection is matched by a comparable improvement
`
`in the ease of
`
`input (In the first class of sketch handling programs, it should
`
`be noted,
`
`the computer is incapable of making a mistake; only the
`
`user).
`
`The most notable example of the first sort of program is Ivan
`
`Sutherland"*s SKETCHFAD (Sutherland, 1963), particularly since it
`
`was the first atteapt at communicating a visual
`
`image between
`
`user and computer
`
`in an interactive manner.
`
`In its stated goals,
`
`however, SKETCHPAD was to be a system unlike drawing with pencil
`
`and paper, because interacting with a computer was seen to be a
`
`totally different kind of experience. Using primitives common to
`
`all line drawings (line segments, foints,. and arcs of circles),
`
`the user creates symbols, structures, and composites of these
`
`images. To increase the power of
`
`the interaction, certain
`
`functions could be applied to previously defined images. Thus,
`
`when the user laid down two lines, he could indicate to the
`
`system that he wanted them to be parallel or perpendicular, and
`
`the system performed the requisite steps to make them exactly
`
`that way. Thus,
`
`the us2r could be inaccurate in his original
`
`layout and yet get a highly specific output of his final
`
`image.
`
`Furthermore, since the user was not drawing symbols for the
`
`bo
`
`
`
`system to recognize,
`
`the kinds of. graphic images the system could
`
`accept waS unrestricted.
`
`For such freedom,
`
`the user pays a penalty. however.
`
`The system
`
`of light pen on display and function keys utilized by SKETCHPAD
`
`bears no relation to the normal means of communicating an idea
`
`graphically.
`
`The fairly demanding system of input required by
`
`such a system would tend to interfere with the creative thinking
`
`process.
`
`The user is concentrating so hard on getting the
`
`drawing into the machine that is difficult to think about what he
`
`is drawing.
`
`IEM is marketing a new system similar to SKETCHPAD
`
`which uses a tablet instead of a light pen (Saderholm, 1973).
`
`While this hardware is an improvement over the old
`
`setpoint-rubber- band-line, it still relys on a set of function
`
`buttons on the tablet to relay commands to the system. The
`
`degree of explicitness required in such systems quickly generates
`
`tedium sufficiert to offset any prefererce over the less
`
`complicated job of digitizing tie data.
`
`Any sense of natural
`
`graphical communication is lost. Furthermore, since the computer
`
`is operating continuously in “slave™ mode, it can add no
`
`information to the cialogue. Thus, an important potential
`
`is
`
`lost.
`
`In the second class of computer systems,
`
`the computer does add
`
`information of its own to the dialoge in its interpretation of
`
`the sketched symbols of the user. Although this approach is
`
`Ll
`
`
`
`primarily found in character recognition programs
`
`-, perhaps the
`
`most notable example is the GRAIL system developed by the Rand
`
`Corporation (Ellis, et. al., 1969). Besides recognizing the
`
`alphabet and decimal digits, it could also handle a set of flow
`
`charting symbols (rectargles, triangles, and'so forth), and lines
`
`connecting these symbols. Using the Rand data tablet,
`
`the user
`
`drew his flow chart and labeled it.
`
`As each symbol was drawn,
`
`the system identified it, and the rough display of the user‘s
`
`line was replaced by the machine‘s representation of the symbol,
`
`appropriately scaled and positioned. Because of the level of
`
`inference making,
`
`the program was capable of making mistakes.
`
`In
`
`order to allow for errors ind to permit
`
`the user to change his
`
`mind, one of the symbols recognized by the system was the
`
`scribbling out motion normally used by people to cross out an
`
`errer or a misplaced line. The symktol was called a “squiggle,”
`
`and caused any line or symbol which appeared beneath it to
`
`disappear. Cnce the flow chart was completed,
`
`the user could
`
`ascribe specific functions to the symbols of the flow chart and
`Lia
`%
`
`It provided a
`
`see what happened when the flow chart was
`
`“run.
`
`neat way of seeing informati‘on which might otherwise have been
`
`too difficult to visualize.
`
`Systems in the GRAIL class are quite attractive, since they
`
`provide a sort of interaction which is very natural and familiar
`
`to the experience of the probable user. Drawing with a pen on
`
`paper
`
`is an experience common to most people, and GRAIL’s
`
`12
`
`
`
`On the other hand,
`replication of this experierce is not bad.
`these pattern recognition systems can only handle a limited class
`
`of inputs. Given an unrecognizable symbol,
`
`the system is lost.
`
`This problem is partially overcome in many character recognition
`
`systems by having a "learning" mode, where the system samples the
`
`individual user's representation of the symbols it knows,
`
`thereby
`
`adapting its models for the symilols to the habits of the
`
`individual user. There are two limitations usually imposed,
`
`however. First,
`
`the user"s representation usually can not
`
`deviate beyond some accepted boundary conditions.
`
`For example, a
`
`character recognition program normally accepts either script or
`
`printed characters, but not both. Thus, a user who mixes his
`
`characters would inevitbly be mis-understood. Second, it is
`
`usually impossible for the aser to define symbols of his own.
`
`Thus, if a mathematician wished to user a character recognition
`
`system for the alphabet, he might be hampered by the inability of
`
`the program to accert Greek symbols; similarly, a Russian
`
`translator would have to start all over. Furthermore, as the
`
`number of symbols recogrized by the system is increased,
`
`typically,
`
`the frequency of error increases at a much faster
`
`rate.
`
`‘This phenomenon occurs because the system can not use
`
`clues about
`
`the interaction between elements in a sketch.
`
`If a
`
`character recognition systen had difficulty distinguishing
`
`between U*s and 3's,
`
`for example,
`
`it would be useful to look to
`
`see if the preceeding character was a Q (in English, anyway}.
`
`13
`
`
`
`LETLOWN
`
`It would be nice to be able to claim to have developed the
`
`alert, provocative,
`
`interactive system mentioned in the earlier
`
`section.
`
`The systex which has been developed, HUNCH, falls short
`
`of this goal, however. Provocative it is, although not
`
`in the
`
`manner described above.
`
`It is also moderately interactive.
`
`It
`
`does not, however, carry on anything which can be called a
`
`dialogue.
`
`. «yet.
`
`lLialecgue implies purpose and a developing
`
`context, and although HUNTH does know a few tricks, once it has
`
`performed, all it can do is walk off stage.
`
`The name HUMCH is detived from the methods it uses to achieve
`its ultimate goal.
`It uses guesses about
`implied intensions to
`
`determine what
`
`the sketcher PROHABLY meant.
`
`In that sense, it is
`
`similar to the character recognition systems discussed.
`
`It does
`
`not have a set of patterns it is trying to match, however.
`
`Rather,
`
`it atteupts to extract froa the stream of input data the
`
`primitives which make it uf: line segments, arcs of curves, end
`
`points of lines. Once the data has been so compressed and
`
`structured,
`
`these components can be combined to form objects of a
`
`higher order. This second step is not as well understood, since
`
`it requires extensive knowledge about
`
`the subject matter being
`
`sketched to. accomplish this joal.
`
`14
`
`
`
`Because its knowledge is somewhat mcre limited than a human's,
`
`it would appear that HUNCH is at a disadvantage when it comes to
`
`reading a sketch.
`
`iif it was Limited to those cues available to
`
`auman, that claim would probably be true. However, because of
`
`the way the data is collected, FUNC: can use some information
`
`unavailable to the human on.tooker.
`
`Inherent
`
`in the way the data
`
`is sampled is the sequence in which the sketch was drawn,
`
`the
`
`pressure on the pen at a particular time, and the rate at which
`
`the user was drawinc.
`
`These cues provide additicnal
`
`information which the program can
`
`use to make decisions.
`
`In general,
`
`for example,
`
`the faster the
`
`user draws,
`
`the less accurate he is. Furthermore, if he is
`
`drawing rapidly, it can usually be inferred that he is not
`
`interested in the fiine detail of his line, but rather in the
`
`gjrosser features of what he is drawing. When it encounters a
`
`rapidly drawn line,
`
`then, FUNCH is prepared to make bigger
`
`assumptions and to permit greater inaccuracies before declaring
`
`one line segment ended and a second one begun. Similarly, .if the
`
`user is drawing slowly with great deliberation,
`
`then nearly every
`
`bend or tweak in the line is preserved.
`
`Pressure can also be sensed to provide cues to the intentions of
`
`the user. Here,
`
`the role variations of pressure in a sketch is
`
`not se clear.
`
`‘It ajpears, however that pressure variations occur
`
`in quanta? a user typically draws in. no more than three or four
`
`L5
`
`
`
`pressure ranges.
`
`An initial inference is that pressure behaves
`
`something like inverse rate--that is,
`
`the harder a person pushes,
`
`the greater the detail implied.
`
`The cue to look for, however, is
`
`the quantum pressure change, not
`
`the small. variations across a
`
`line.
`
`WHAT HUNCH DOES ANT FOES NCT
`
`Sketching can be considered to Le a kind of graphical
`
`language.
`
`A person can read a sketch if he knows the rules for making a
`
`sketch, and if he knows the symbols used in the sketch--syntax
`
`and semantics.
`
`In order to carry on a useful: dialogue, you have
`
`to have both.
`
`The EUNCh system does a reasonable job at
`
`providing a large portion of
`
`the syntax.
`
`It is one of the goals
`
`of this paper to outline a means of supplying some of the
`
`semantics.
`
`The sketch, as received by 1UNCH,
`
`is one long serial stream of
`
`data.
`
`The system tries to apply some structure to this data,
`
`to
`
`make the search for neaning more manageable.
`
`Ina sense,
`
`the
`
`solution developed so far still leaves the computer doing what it
`
`does well--number crunching..
`
`In the process of discovering the
`
`structure of the sketch, massive amounts of data are reduced to a
`collection of points and relations between points.
`It performs
`
`these operations with uncanny accuracy, using only local
`
`information about the dynamics of the line.
`
`16
`
`
`
`The relations formed are ooiy phose based on information
`
`explicit in the data, such as the aforementioned rate and
`
`pressure, continuity of line, and sequence. Attempts to apply
`
`further relations to the data failed for various reasons
`
`described later, and in fact, ajpear inevitably doomed without
`
`the application of some semantic guides.
`
`Some of those relations
`
`attempted include latching of tuo known points, and
`
`horizontalizing and verticalizing lines in a sketch which appear
`
`nearly so- These functions failed largely because HUNCH was
`
`unable to judge those situations in which those relations might
`
`or might not apply. Thus,
`
`it applied them indiscriminately to
`
`any set of lines which fell within its guide-lines.
`
`The result
`
`ineviitably was a severe distortion of
`
`the original sketch.
`
`Parenthetically. it should se noted that if input was limited to
`
`those sketches where the rule was always appropriate, HUNCH
`
`solved the problems with distinction. Thus,
`
`the problem was not
`
`in the rule, but
`
`in when to apply the rule. Given any rule,
`
`there is always a condition wrere, applied indiscriminately,
`
`the
`
`rule will fail (including this one). Thus,
`
`some means is needed
`
`to guide the system about when i particular relation might be
`
`appropriate.
`
`The final output of
`
`the system is not
`
`intended to be a “working
`
`drawing" with all extraneous lines eliminated, ali corners
`
`squared, and all lines straight and parallel. Rather it is
`
`intended that the output be the description of the sketch which
`
`L7
`
`
`
`might correspond in some way to the vertal description a human
`
`might make of the sxetch having observed it.
`
`The structure of
`
`this description would be hierarchical, having at its top the
`
`major features of the sketca, at its interim levels the elements
`
`which combine to make up :these features, and at the bottom the
`
`individual line segments of which the sketch is made.
`
`The
`
`interaction of the various elements in the description provides
`
`contextural
`
`information. This information can be used to augment
`
`the rules about relations between lines which got us into trouble
`
`before to provide the guide-lines about when a particular rule
`
`might be applied.
`
`‘Furthermore, it helps to avoid the
`
`difficulties systems of the GRAIL class get
`
`into when called upon
`
`to recognize a large number of different elements. The context
`
`limits the number of plausible elements which may be used,
`
`preventing the system from drifting too far afield.
`
`Unfortunatiely, this desirable description has not been
`
`implemeated in any. form..
`
`In order to derive such a description,
`
`the system needs to know what the elements are for which it
`
`should be searching (wired in to most systems). While in most
`
`types of sketches the number of these elemetns is not large, it
`
`has never been clear how ons. would specify these elements for the
`
`system.
`
`The description offered in the last section of this
`
`paper is a first attempt at making such a specification possible.
`
`1&
`
`
`
`HARDWARE USED BY HUNCH
`
`HUNCH runs on the Architecture Yiachine, a family of Interdata
`
`mini-computers, running under a disk resident operating systen.
`
`The original sketch is read from a Sylvania data tablet, and can
`
`be stored on a variety cf mass storage devices. The Sylvania
`
`tablet is (was) the Cadillac of data tablets, offering resolution
`
`to three thousandths of an inch, constant rate data sampling, and
`
`a clear tablet. This clear tablet means that it can either be
`
`drawn on as with otier tablets, or it can be placed in front of
`
`the display and used in a manner similar to a light pen. Both of
`
`these modes are used by various parts of HUNCH.
`
`The tablet
`
`samples data at a constant rate (two hundred times per second),
`
`sending off to the computer twelve Lits of x- and twelve of
`
`y-coordinate data at each sample.
`
`The tablet can also sense a
`
`limited capability for a z-dimension (three bits), such that it
`
`can tell if the pen is touching,
`
`is in the near field (about one
`
`half inch),
`
`is in tte far field (up to four inches}, or is away
`
`from the tablet. This feature suggested a logical extension, and
`
`the pen was modified to be able to sense pressure--how hard the
`
`penman is pressing on the paper.
`
`A load cell fa sort of
`
`transverse strain gauge) has been built into the shaft of the
`
`pen,
`
`taking the thrust fron the top of the ball point pen
`
`cartridge.
`
`It ican neastre pressure from a fraction of an ounce
`
`up t9 a pound. This load is converted into a digital signal
`
`which is sent as a six-bit number
`
`to the computer. Each pressure
`
`13
`
`
`
`sample is associated with the point read when the sample was
`
`taken.
`
`The display is not crucial to HUNCH, although it is useful for
`
`demonstration and debugging purposes. Because of the amount of
`
`data developed by the data tablet and the complicated pictures
`
`possible, it would be impossible to maintain a flicker free image
`
`on a refreshing display. After ten seconds of drawing,
`
`the
`
`screen would have two thousand vectors on it.
`
`HUNCH uses an ARDS
`
`storage tube, which effectively avoids this difficulty. Although
`
`it is difficult to dynamically modify the image on a storage
`
`tube,
`
`there is very little need to do so in a sketching
`
`environment.
`
`The difficulty. of erasing is not unlike that the
`
`user experiences when drawing with fen on paper, anyway. Rather
`than erasing,
`the user just gets a clean sheet of paper.
`The
`
`ARDS has a limited dynamic mode, called write-through. which
`
`permits the dynamic alteration of a limited number of lines.
`
`This feature is adequate fcr those rare occasions when a picture
`
`must be modified.
`
`While the sketch is being initially stored, it can be displayed
`
`im an exact mimic of the criginal. The ARDS is a relatively slow
`
`display, however, and the time taken to display the image reduces
`
`‘the sample rate which can be oktained from the tablet. Thus,
`
`the
`
`resulting stored sketch is less detailed.
`
`To overcome this
`
`difficulty, display while drawing can be suppressed.
`
`The stored
`
`20°
`
`
`
`sketch can, of course, be r: played to the display. During times
`
`when the pen is not touching the tablet, a real time clock is
`
`sampled,
`
`so that the lengtn of pauses in drawing can also be
`
`stored.
`
`The replayed sketch,
`
`then, can be an exact replica of
`
`the dynamic development of che original.
`
`The addition of pressure sensitivity demanded an additional
`
`feature for the data display--some method for showing variations
`
`in pressure.
`
`The ARDS was altered such that its focus could be
`
`modified under computer control. Thus, while the tube normally
`
`displayed a thin, sharp line, by defocusing the beam slightly.
`
`the width of that line could he increased up to an eighth of an
`
`inch. This feature is integrated with the load cell in the
`
`pressure pen such that the width of the line varies as a function
`
`of the toriginal or redisplayed) pressure (Figure 1). This line
`
`variation greatly enhances the visual effect of the display,
`
`since it provide; a better feel for pressure than the line output
`
`of a ball point pen can provide.
`
`Figure l.
`
`21
`
`
`
`II.
`
`PAST
`
`How Did :I Get kere?
`
`READING AND REDISPLAYING--DRAW/ SHOW
`
`In the spring of 1397),
`
`the Architecture Machine Group obtained
`
`its Sylvania tablet, and embarked on an experiment
`
`to discover
`
`about reading a sketch by computer.
`
`The tablet was an ideal
`
`device for this experiment, hav.ing the natural feel of ren on
`
`paper, while at the same tine providing a fast, accurate,
`
`time-dependent sampling of the sketch as it was created.
`
`The
`
`first programs written, naturally enough, were programs to read
`
`and save the data from a sketch, DRAW, and to redisplay the
`
`stored data, SH3iW.
`
`DRAW seases the z-position of the pen, only
`
`recording data when the pen touches the tablet. The maximum
`
`distance the pen is away from the tablet (near or far field) is
`
`recorded as a flag in the stream of data whenever the fen leaves
`
`the tablet.
`
`The distinction of the z-fields is not used by any
`
`part of the program to date. It is thought, however,
`
`that the
`
`degree of pen lift: may be useful for providing some clues into
`
`‘logical separation of
`
`the sketch into sub-sections, divided by
`
`higher. lifts of the pen.
`
`Where the pen went: while it was not
`
`in contact with the tablet
`
`could be read from the data, and there was some discussion at the
`
`22
`
`
`
`time about whether. or not this information should be saved.
`
`Since at the time, we did not know what information was going to
`
`become important, we wer:
`
`leery of discarding any obtainable
`
`information.
`
`The use to which pen-ctp information could have been
`
`put was unclear at the time (it is still so), however, and space
`
`limitations for storacge «cf data were relatively severe at the
`
`time.
`
`It was decided,
`
`therefore,
`
`to discard this data.
`
`DRAW begins by sensing the position of the pen in the z-field.
`
`When the pen touches down, DRAW records a far-field pen-lift flag
`
`and the x- and y-cosriinate: of the first point. With the recent
`
`addition of pressur.: sensitivity,
`
`the value of pressure is also
`
`saved.
`
`It then continues to read successive points and
`
`pressures, storing them away and toptionally) displaying them on
`
`the storage tube. when the pen is lifted from the tablet, DRAW
`
`waits for the pen to be replaced and saves the pen-up flag
`
`recording the farthest field reached by the pen and the time the
`
`pen was lifted.
`
`[RsW continues to read and save data in this
`
`Manner until the pen is lifted away from the tablet field and
`
`then signals that the drawing is complete.
`
`Once the data has been saved,
`
`ist can be redisplayed by a call
`
`to
`
`the program SHOW.
`
`When this program was run for the first time,
`
`it caused the sort of sererndipidous discovery which occasionally
`
`provides direction for research. Altaough it seems obvious in
`
`hind-sight,
`
`the effect the time based sampling of data would have
`
`23
`
`
`
`on the data itself had not occurred to anyone.
`
`Since the tablet
`
`samples data at a constant frequency (200 times per second),
`
`the
`
`distance the pen covers between samples is a direct function of
`
`how fast the pen is moving. Obviously--now--the faster the pen
`
`is going,
`
`the greater the iistance it will cover in a two
`
`hundredth of a second--the farther apart
`
`the recorded points will
`
`be.
`
`Ihe effect of this fact, of course,
`
`is that SHOW not only
`
`redisplays the original sketch, but also it replays the sketch at
`
`exactly the same rate it was originally drawn.
`
`Inherent
`
`in the
`
`way the data is stored is the data is stored is the RATE at which
`
`the line was drawn.
`
`This fact provided the ground on which HUNCH is built.
`
`It may
`
`be assumed that the speed at which a person draws reflects in
`
`some way his degree of purposefulness, his detailed interest in
`
`exactly what he is sketching.
`
`‘Fore specifically, it is usually
`
`true that if a person is frawing quickly, he is not as interested
`
`in detail as he is when drawing slowly.
`
`In a quick sketch,
`
`the
`
`person is usually interested in the general
`
`impression his lines
`
`make, rather than in the exact reproduction of those lines.
`
`Conversely, a slowly drawn sketch may often be painstakingly
`
`detailed.
`
`In this case,
`
`the position of each line hecomes
`
`important, and the sketcher wants his drawing to be seen exactly
`
`as drawn.
`
`24
`
`
`
`SQUIGGLES
`
`One special purpose kind of line is detected “on the fly.” As
`
`mentioned in the descrirtion of the GRAIL system,
`
`the scribbling
`
`out motion has a special meaning.
`
`If drawn over previously drawn
`
`lines, it means that the earlier lines are to be crossed out.
`
`If
`
`filling an open area,
`
`the scribble implies that the area is to be
`
`shaded.
`
`‘In either :case,
`
`the exact configuration of the line is
`
`not as important as the area it covers. Thus, it is not critical
`
`to submit the line to exact analysis.
`
`In order to extract these
`
`scribbles from the raw data, a “squiggle” recognizer was devised
`
`as a part of [TRAW. Coincidentally, Rand's use of a squiggle,
`
`even the to the name itself, was not discovered until after the
`
`one in HUNCH hac been developed.
`
`A squiggle is characterized by several things. First of all, it
`
`has many changes of dirzction.
`
`It is usually drawn at a fairly
`
`high rate of speed, however, so it is not confused with the
`
`curving line of a driveway,
`
`for example. Finally these changes
`
`of direction form a sawtooth pattern (they are neither too spread
`
`out nor t
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
