United States Patent
`
`[19]
`
`[11] Patent Number:
`
`5,021,976
`
`Wexelblat et a1.
`Jun. 4, 1991
`[45] Date of Patent:
`
`[54] METHOD AND SYSTEM FOR GENERATING
`DYNAMIC, INTERACTIVE VISUAL
`REPRESENTATIONS OF INFORMATION
`STRUCTURES WITHIN A COMPUTER
`
`[75]
`
`Inventors:
`
`Alan D. Wexelhlat; Kim M. Fairchild,
`both of Austin, Tex.
`
`[73] Assignee:
`
`Microelectronics and Computer
`Technology Corporation, Austin, Tex.
`
`[21] Appl. No.: 271,091
`
`[22] Filed:
`
`Nov. 14,1933
`
`Int. Cl.5 .............................................. GO6F 3/153
`[51]
`[52] U.S.Cl. ................................. 364/521; 364/518;
`364/146; 340/74?
`[53] Field of Search ............... 364/518,521.522,146,
`364/141; 340/721, 723, 74?. 3'50, 793,800 '
`
`[56]
`
`References filed
`U.S. PATENT DOCUMENTS
`
`4.752.893
`4.172.882
`4.813.013
`4.814.155
`4.323.233
`
`...........
`6/1988 Guttag et al.
`9/1988 Mica]
`........
`3/1989 Dunn ..............
`3/1989 Johnson et a].
`.
`4/1939 Diehmetal.
`
`
`
`364/521 X
`364/521 X
`364/52i X
`364/521 X
`....... 364/518
`
`OTHER PUBLICATIONS
`
`Fairchild, S. Poltrock and G. P. Furnas. Lawrence
`Erlbaum Associates, 1987.
`“Picture Generation Using Semantic Nets". R. D. Gius-
`tini, M. D. Levine, and A. S. Malowany, Computer
`Graphics and Image Processing, vol. 1. pp. 1-29. Aca-
`demic Press. Inc., 1978.
`“Human Factors in Data Access", '1‘. K. Landauer, S.
`T. Dumais, L. M. Gomez and G. W. Fumas, Bell Sys-
`tem Technical Journal, vol. 61. No. 9, Nov. 1982.
`“Generalized Fisheye Views", G. W. Furnas, Human
`Factors in Computing Systems, Apr. 1986.
`“The Alternate Reality Kit". Randall B. Smith. 1986
`IEEE Computer Socieg/ Workshop on Visuai Languages,
`Jun. 1986.
`
`Primary Examiner—Heather R. HerndOn
`Attorney. Agent, or Firm—Johnson & Gibbs
`[57]
`ABSTRACT
`
`A method and system for generating dynamic. interac-
`tive visual
`representations of information structures
`within a computer which enable humans to efficiently
`process vast amounts of information. The boundaries of
`the information system containing the information to be
`processed are established and a set of mathematical
`relationships is provided which indicates the degree of
`correlation between parameters of interest to a user and
`segments of information contained within the bound-
`aries. A visual display is generated for the user which
`has a plurality of different iconic representations and
`visual features corresporiding to the parameters defined
`by the mathematical relationships. The iconic represen-
`tations and visual features of the visual diSpiay change
`with the movement of the mathematical relationships
`within the boundaries of the information system accord-
`ing to the degree of correlation between the parameters
`of interest and the segment of informatiOn through
`which the mathematical relationships are passing.
`
`Steamer: An Interactive Inspectable Simulation—Based
`Training System, by James D. Hollan. Edwin I... Hut-
`chins and Louis M. Weitzman. dated 1984.
`“Direct Manipulation Interfaces". by Edwin L. Hut-
`chins, James D. Hollan and Donald A. Norman,
`Human—Computer
`Interaction.
`I985,
`voi.
`I,
`pp.
`311—338.
`in
`“Graphic Interfaces for Simulation", Advances
`Man—Machine Systems Research, vol. 3. pp. 129—163
`(JAI Press, Inc. 1987}.
`“SemNet: Three—Dimensional Graphic Representa-
`tiOns of Large Knowledge Bases". Cognitive Science and
`Its Applications for Human Computer Interaction. Kim
`
`
`30 Claims, 3 Drawing Sheets
`
`54
`
`
`
`
`DISPLAY
`INTERFACE AND
`
`CONTROL
`
`MS 1020
`
`MS 1020
`
`1
`
`

`

`US. Patent
`
`June 4, 1991
`
`Sheet 1 of 3
`
`5,021,976
`
`I3
`
`18
`
`
`
`
`
`I4
`
`ior Designer
`
`I?
`
`
`Skyscraper Design
`
`
`Cust I
`I5
`
`Space
`
`:ber
`
`FIG.
`
`I
`
`
`
`2
`
`

`

`U.S. Patent
`
`June 4, 1991
`
`Sheet 2 of 3
`
`5,021,976
`
`
`
`
`
`FIG. 5
`
`3
`
`

`

`US. Patent
`
`June 4, 1991
`
`Sheet 3 of 3
`
`5,021,976
`
`54
`
`
`
`INTERFACE AND
`
`CONTROL
`
`DISPLAY
`
`
` CENTRAL
`
`PROCESSING
`UN IT
`
`
`
`FIG. 6
`
`4
`
`

`

`1
`
`5,021,976
`
`5
`
`15
`
`20
`
`25
`
`30
`
`35
`
`2
`A second kind of icon employed in the past is also in
`the static category, although it incorporates a certain
`amount of dynamism. This dynamism reflects a certain
`activity in the information or in the condition of the
`system represented by the icon. For example, one icon
`employed in a windowing system appears when a win-
`dow is closed as a shrunken representation of the win-
`dow. Ifsome new text appears in the window while it is
`closed down. the text is also represented in the icon, but
`0 in a proportionally smaller size. That is. some additional
`information appears in the icon when changes occur in
`the system, but that information is treated as a change
`going on in the background, to which the operator is
`paying little attention.
`A related type of icon, which is basically static but
`which incorporates a certain amount of dynamism, is
`the animated icon. For example,
`in one application
`program, a trash can grows in size as the user discards
`files without emptying the trashcan‘s contents. While
`only three actual sizes of icons are used—a normal slim
`size, an intermediate size. and a very full size—they are
`displayed sequentially to connote growth to the user.
`Each of the individual graphical images comprising the
`animated icon is static in that it is fixed in size; the com-
`posite appears to move only as the result of an animated
`illusion.
`Highly sophisticated animated icons have also been
`used to provide graphical interfaces for computer simu-
`lations of complex physical systems. By way of illustra—
`tion. numerous editorially selectable animated icons are
`used in the interactive,
`inspeetable, simulation-based
`instructional steam power plant system described in
`“Graphic Interfaces for Simulation,“ Advances in Man-
`Machine Systems Research, Vol. 3, pp. 129463 (JAI
`Press. Inc. 198?). Each of these icons is directly con-
`nected to respond to an associated variable within the
`mathematical simulation algorithm to indicate the exis-
`tence of particular conditions within the simulated
`physical system.
`is
`tool because it
`The icon is an extremely useful
`designed to trigger within the mind ofthe human opera«
`tor concepts that quickly communicate the contents or
`operation of the information system. Most
`icons are
`either static or animated and are connected so that they
`respoud directly to either an information system condi-
`tion or a variable within a simulation algorithm. If an
`icon could instead be coupled to a means for moving
`through and inspecting the contents of an information
`system and graphically depicting the results of that
`inspection, it could be used to great advantage in com
`municating information about large cyberspacas of in-
`formation. The present invention proposes such an au‘
`tomatically generated icon system for enabling a user to
`interface with data contained within an informatiOn
`system.
`
`METHOD AND SYSTEM FOR GENERATING
`DYNAMIC. INTERACTIVE VISUAL
`REPRESENTATIONS OF INFORMATION
`STRUCTURES WITHIN A COMPUTER
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`The invention relates to human interface with infor-
`mation systems. and, more particularly, to a computer
`system for inspecting and modifying data contained
`within an information system.
`2. History of the Prior Art
`Two results of the rapid increase in available facilities
`with huge capacities for data storage and with enhanced
`speeds for manipulating data are the accumulation of
`vast complexes of information spaces and the intercon-
`nection of networks of such spaces. The term "semantic
`network“ is used herein for a set of information orga-
`nized along a particular cOnceptual line, while the term
`“cyberspace" is used herein for a large pool of complex
`information organized along virtually every conceptual
`line that can be thought of. One of the problems associ-
`ated with such huge cyberspaces is how to determine
`optimum ways for human operators to interact with
`meaningful subsets of information contained within a
`cyberspace. Although humans are wonderful cognitive
`processors of information, they rapidly become satu-
`rated and ineffectual when confronted with too much
`information at one time.
`In the management and use of large cyberspaces of
`information.
`the principal human problem is how to
`enable people to navigate through the information
`space focusing on specific information without losing
`their awareness of information at the global level. More
`particularly, there is the problem of how to enable a
`user to interact efficiently with the information con-
`tained in a large information system. A user must be
`able to rapidly inspect, select, and modify data within a
`system if this data is to be widely and effectively used.
`While highly skilled programmers and other such
`users may be able to meaningfully interpret lines ofcode
`on a display screen, most users cannot. For this reason.
`application programs have often incorporated various
`types of graphical displays to communicate information
`about the internal condition and operation of the infor-
`mation system. Such graphical display interface systems
`enable the user to make program selections and provide
`other input related to the information processing activ-
`ity. One particularly useful tool for interfacing with an
`information system has been the graphical symbols
`called “icons."
`An icon is a small pictorial representation of some
`larger set of informatiOn. Icons have been used for
`many years as a way to graphically indicate certain
`information about the contents of a sysrern or state of
`operation. Generally. icons fit into one of two catego-
`ries:static or dynamic. A static icon is simply a picture
`of something that indicates a condition within a com-
`puter system. For example. it could be an image of a
`window within a windowing system. which has been
`closed down and put into the background of the display.
`Alternatively.
`it could be a picture of an unopened
`document in a word processor type of information sys-
`tem. Overall. it is a simple, static picture on a display
`screen, coanoting certain encoded information in the
`mind of a human operator.
`
`45
`
`50
`
`55
`
`SUMMARY OF THE INVENTION
`
`invention includes the
`The system of the present
`definition of mathematical relationships that are mov-
`able within an information space. The system also de-
`fines a means for relating to that information space in
`accordance with a set of criteria delineating a plurality
`of parameters that are of potential interest to a user. An
`automatic icon is defined by associating certain graphi-
`cal primitives with certain mathematical relationships
`so that as an embodiment of the relationships are moved
`through an information space.
`the appearance of the
`icon automatically changes as a result ofthe correlation
`
`65
`
`5
`
`

`

`3
`between the mathematical relationships and the con-
`tents of the information space.
`The present
`invention also includes a method for
`moving through and examining the contents ofa knowl-
`edge space by means of a set of criteria. A graphical
`icon is automatically generated in response to the rela-
`tionship between the criteria and the observer’s location
`within the knowledge space. This icon visually repre-
`sents what is being observed within the space.
`One aspect of the present invention includes a system
`for enabling a user to interact with information cort-
`tained within an information system The boundaries of
`the system containing the information are established
`and a set of mathematical relationships is provided to
`define a plurality of parameters of potential interest to
`the user. The mathematical relationships are capable of
`indicating a degree of correlation between the parame-
`ters defined by the relationships and the segments of
`information contained with the boundaries of the infor-
`mation system. A visual display is generated that has a
`variety of visual features each of which can assume
`various conditions over a range of different possible
`conditions. The parameters defined by the mathemati-
`cal relationships are associated with corresponding fea-
`tures of the display. An embodiment of the set of mathe-
`matical relationships is moved within the boundaries of
`the information system and interacts with the informa-
`tion to produce a visual display. The features of this
`display indicate the degree of correlation between the
`associated parameters of interest to the user and the
`segment of information through which the embodiment
`is passing.
`The visual display can also enable the user to interact
`with the features of the display and to provide input
`about a desired change in the degree of correlation
`between a selected parameter and an associated seg-
`ment ofinformation. The system includes the means for
`changing the content of the segment of information
`through which the embodiment of the mathematical
`relationships is passing and establishing the degree of
`correlation indicated by the user‘s interaction with the
`display.
`In a different aspect, the present invention includes a
`system for interacting with an information system that
`uses memory to store data and that connects a processor
`to the memory to access selected segments of the data.
`A display is connected to the processor. Generated on
`the display is an icon that has a plurality of features.
`each of which is capable of changing in appearance
`over a range of different possible appearances. A set of
`mathematical relationships contained in the processor
`defines a plurality of parameters of potential interest to
`the user. The mathematical relationships are capable of
`indicating a degree of correlation between the parame-
`ters defined by the mathematical relationships and the
`segments of the data. The system is responsive to indi-
`cate the degree of correlation beIWeen the parameters
`defined by the relationships and the data, and it auto-
`matically generates a particular corresponding appear-
`ance in an associated feature of the icon. The features of
`the icon represent the relationships between the param-
`eters of interest to the user and the segment of data to
`which the mathematical relationships have aCCESS at
`each sequential period of time.
`BRIEF DESCRIPTION OF THE DRAWING
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`4s
`
`50
`
`55
`
`60
`
`65
`
`For a more complete understanding of the present
`invention and for
`further objects and advantages
`
`5,021,9?6
`
`4
`thereof, refer to the following description, in conjunc-
`tion with the accompanying drawings:
`FIG. it is a diagram that depicts multiple views of an
`illustrative knowledge space;
`FIG. 2'ts a diagram that illustrates an artificial reality
`containing nodes and waypoints;
`FIG 3'is a diagram showing the use of an automatic
`icon within a waypoint of the artificial reality illustrated
`in FIG. 2;
`FIG. 4 is a schematic illustration of a base waypoint
`for automatic icons;
`FIG. 5 is a schematic illustration of a specialized
`waypoint allowing selection of automatic icon proper-
`ties; and
`FIG. 6 is a block diagram of an interface system con-
`structed in accordance with the criteria of the present
`invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`Artificial Reality Models
`
`A helpful technique for the management and use of
`cyberspaces of information is the creation of dynamic
`visual representations of subsets of cyberspace. These
`subsets are represented as a collection of automatic
`icons that map abstract semantic entities from that sub-
`set into graphical forms that people can edit and manip»
`ulate. These representations may take the form of com-
`puter-maintained worlds, or artificial realities. governed
`in accordance with specific rules to define the subsets of
`cyberspace presented, the form of the presentation of
`the information. and the operator actions required to _
`manipulate the artificial reality artifacts that make up
`the worlds. That is, the artificial realities create a hu-
`man-understandable structure out of and within cybers-
`paces of information and provide a conceptual frame-
`work for communication with the cyberspace by a sin-
`gle individual or by groups of individuals.
`Three conceptual underpinnings support the creation
`of artificial
`realities: multiple views of information.
`hierarchical object structures, and semantic navigation.
`These three concepts are fundamental to the present
`system, which allows the user to interact with data
`contained in such information systems.
`FIG. 1 is a conceptual illustration ofa central knowl-
`edge base 11 for a skyscraper design that comprises a
`plurality of views 12-13 of the base. These multiple
`views are used to create the artificial reality so that
`many different individuals can use it. The design space
`in FIG. 1 is that ofa skyscraper. and the different views
`are those of the customer 15.
`the architect 14.
`the
`plumber 16, and others. The view associated with the
`customer 15 is a traditional view of the skyscraper. and
`the customer's view is accorded controls for moving
`the viewpoint
`in a three-dimensional space. for per-
`forming relatively superficial changes within the space,
`and for communicating certain comments on the design
`to the architect.
`In contrast.
`the view provided within the design
`space 1 for the plumber 16 is a floor-by-floor view that
`emphasizes where the piping is going within the sky-
`scraper and that includes artificial reality artifacts that
`might interfere with the paths chosen for the piping.
`The plumber's view 16 may be given controls that allow
`the editing of plumbing topologies and that can access
`city plumbing codes, other design specifications, and
`other information related to this particular view. By
`
`6
`
`

`

`5,021,9'l6
`
`5
`way of further illustration, the architect 14 is the leader
`of the design team and requires many views of the cyb»
`erspace ll—for example, views for the management of
`the project, views for cornmunication with other design
`team members, and views related to the performance of
`each of the contractors working within the space.
`The different views created within the artificial real-
`ity for each of the team members together compose a
`specialized metaphorical
`representation of the sky—
`scraper itself. Each view encodes design information
`into the form of artificial reality artifacts that have been
`given appropriate graphical representations and that
`interact with the other artifacts and with the user as if
`they were real.
`traditional
`To extend the artificial reality concept,
`mathematical relationships and software techniques can
`be used to define a complete environment for the design
`of any particular system. One problem with using tradi-
`tional software techniques to create such environments
`is that if the problem statement changes even slightly,
`many of the multiple views will have to be modified,
`frequently at great cost. It would be more useful
`to
`provide an artificial reality environment that is extend-
`able. growing to accommodate new and various design
`tasks and including tools and facilities for creating and
`manipulating artificial realities themselves. This envi-
`ronment should support reuse of the artificial reality
`artifacts and contain libraries, Specialized editors, and
`general artificial reality tools to support finding and
`modifying artifacts for any design task that may arise.
`All artificial realities rely on a hierarchy of objects.
`These objects are created by means of a corresponding
`hierarchy of editors, each of which is specialized to
`produce the type of object needed. The relationships
`between these editors exactly parallel those of the ob-
`jects they create. So if an object is a SpecializatiOn of
`another object,
`its corresponding editOr would be a
`specialization of the editor for the other object. In order
`to make it easier to create and manipulate these objects,
`specialized object editors can be used to change arti-
`facts into the form most suited to the user's tasks. To
`support this editor infrastructure, various artificial real-
`ity objects are placed in the cyberspace and used to
`create the initial environment.
`Since all artifacts share a basic artificial reality struc~
`ture, only base editors are needed for the general con-
`struction of artifacts; however, it is typically desirable
`to provide specialized editors that are associated with
`each of the more important artifact classes. For exam-
`ple, in the artificial reality illustrated in FIG. 1, editors
`would be provided for writing notes from the customer
`to the architect and for changing both the artifacts
`within the world as well as the different viewpoints of
`the architect.
`To minimize the amount of effort needed to set up
`each successive artificial real ity. artifacts fmm previous
`projects are stored in a way that allows easy search and
`selection of artifacts to create another reality. Artifacts.
`artifact groups, and their associated editors may all be
`reused across various projects; however. since no two
`projects are completely identical, the artifacts may also
`be changed between projects. For example, in the sky-
`scraper illustration of FIG. 1, the architect may experi-
`ence over enthusiastic customers and thus desire to limit
`the customers‘ supply of paper upon which to provide
`comments.
`To achieve effective reuse of artificial reality arti-
`facts, four basic steps are required:
`
`6
`(1} Finding existing artifacts
`match the new specifications;
`(2} Determining how close the existing artifacts are to
`those needed for the new application;
`(3) Using the most specific editor to make the artifact
`precisely what is needed for the new application;
`and
`
`that approximately
`
`10
`
`is
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`(4) Connecting and/or introducing the modified arti-
`fact to others within the new environment.
`Artificial reality worlds are specialized to accomplish
`a particular task with a subset of cyberspace present in
`each world. Subsets may be pre-chosen to support the
`tasks to be accomplished, such as the design of the sky-
`scraper illustrated in FIG. 1. However, even with pre-
`chosen artificial reality worlds, it
`is still necessary to
`search for things that are not available within the cur»
`rent world. To allow this, artificial reality environments
`support switching from One world to another, move-
`ment through a single world. and the creatioa of infor-
`mation displays as the user moves through cyberspace.
`The methods of movement
`through cyberspace are
`referred to herein as semantic navigation.
`The general problem in constructing visual views of
`information for a user is how to assign positions to ele-
`ments so that the organization or structure of the infor-
`mation is maximally apparent to the user. A general
`solution to this problem probably does not exist, and
`experience has shown that the optimum method ofposi-
`tion assignment for one set of information may be to-
`tally inappropriate for another set of information.
`Constructing spatial representations of the intercon-
`nection of elements is a powerful method of coping
`with the size and complexity ofcyberspaces ofinforma»
`tion. However, a new set of problems is raised by such
`representations, which is associated with navigating in a
`virtual space. These problems of navigation can be
`grouped into two categories: recognizing locations and
`controlling locations. One solution to the problem of
`allowing a user to know where the current viewpoint is
`in an artificial reality world is navigation aids similar to
`those found in the real world. Artificial realities can
`provide tools similar to maps that show the current
`position of an observer in the x-y and x~z planes. In
`addition, structures similar to landmarks may be either
`provided specially or just occur naturally when ele-
`ments form groups that, with experience, become rec-
`ognizable by the operator.
`Alternatively. artificial landmarks can also be pro-
`vided. For example. a simple ruler-like device could
`show the viewer how far from the center of the knowl-
`edge weh his viewpoint is currently located and in what
`direction therefrom. One of the most important goals of
`an interface between the user and the artificial reality
`world is to make the user experience a real, N-dimen-
`sional space. That is, to allow the user to see and manip-
`ulate the information itself on a display screen rather
`than to simply do things which, in turn, manipulate an
`unseen system. The movements that control the real
`world should map directly into the virtual world of the
`artificial reality.
`Another technique of semantic navigation within an
`artificial reality is that ofabsolute movement or telepor-
`tation. Artificial realities may fulfill the desire to tele-
`port instantly from One location to another by supplying
`an abstract map of the cyberspace-and allowing the user
`to point to locations on the map for changing the view—
`point within the cyberspace. Once an observer has Spent
`the cognitive and computational energy to travel
`to
`
`7
`
`

`

`5,021,9'l6
`
`7
`some place within a cyberspace. the chance of needing
`to go back to that same location increases dramatically.
`Thus. teleportation allows the user to pick up a recent
`cyberspace contact location from a two-dimensional list
`or menu and instantly move back there. For example,
`the user could move from his current location within a
`cyberspace to a library, make a copy of an artificial
`reality artifact, and then teleport back to the original
`location and add that artifact element to the current
`world.
`_
`Browsing within a cyberspace typically involves
`examining elements ofthe artificial reality world one at
`a time, observing which elements are related to the one
`under examination, and then following a selective rela-
`tionship to examine other elements. Hyperspace move—
`ment is particularly useful in large cyberspaces in which
`the relationship among elements is not easily recognized
`and in situations in which a user is searching for an
`element similar or related to one that has been found.
`Extremely large cyberspaces of information may
`include hundreds or even thousands of artificial reality
`worlds. Navigating between these worlds is certainly
`possible with various known movement methods, but
`even move desirable is a concept that provides easy
`access to the core worlds in which a designer has the
`most interest and the ability to extend the core set.
`For unexplored areas a technique is also needed for
`creating dynamic signposts that guide exploration. As
`the number of artifacts goes up, the energy associated
`with moving between the most commonly used artifacts
`must also remain reasonable in the context. One naviga-
`tional technique that has the possibility of being ex-
`tremely powerful within the realm of semantic naviga-
`tion is that of “muscle memory.“ This technique in-
`volves the development of automatic reactions within
`an operator in response to a graphical stimulus. Such a
`technique requires very little cognitive attention during
`routine movements and during navigation in new terri-
`tory. and it supports the formation of stimulus-response
`associations within the operator.
`Various video disk-based games appear to develop
`such muscle memory responses to visual stimuli, while
`other basic screen-oriented text editors that bind com-
`mands to arbitrary keystroke sequences work in a simi-
`lar fashion. It appears that the user forms an association
`between visual stimuli and muscle responses despite the
`fact that certain factors work against the user in this
`development. In the video games, the designers deliber-
`ately give very few advance clues as to which reaction
`the user must
`take. In the editing systems,
`the key-
`strokes are largely non~mnemonic and are subject to
`arbitrary change at a user‘s whims. The fact that muscle
`responses continue to develop even in the face of such
`adversity indicates that these responses include a pow—
`erful technique for semantic navigation within cybers-
`paces.
`
`Use of Icons Within Information Spaces
`FIG. 2 shows a set of artificial reality worlds that are
`depicted by circles 21 located within a cyberspace of
`information. One can navigate among the worlds 2]
`either by pointing to a particular world and teleporting
`there or by following the interconnection between the
`worlds. An inherent problem is that as the number of
`worlds within the artificial reality increases, so does the
`amount of cognitive energy required either to search
`the abstract map of the world or to follow the links
`between the worlds. Also shown in the cyberspace of
`
`IO
`
`15
`
`20
`
`25
`
`30
`
`35
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`FIG. 2 is a plurality of squares 22 labeled “WP.“ This
`plurality of squares represents a plurality of additional
`worlds that are referred to as "waypoints" worlds
`herein. These waypoints 22 enable the user to establish
`passways and connectivities between similar artificial
`reality worlds 21. The intention is to allow navigation
`to proceed at a higher level. Instead of moving from
`world to world, one can move between groups of
`worlds that have been clustered by users in accordance
`with individual criteria.
`In effect.
`the waypoints 22
`provide shortcuts along frequently travelled routes
`between worlds 2!.
`FIG. 3 shows a sample of one of the waypoint worlds
`22, which includes a central icon 31 containing land-
`marks such as a bird‘s-eye view of the graph 32 to:—
`establishing the user‘s current position and contacts
`within the world. Around the border of the waypoint
`world of FIG. 3 are branch points that are represented
`by ic0ns 33—39. These points provide possible directions
`for the user to move from within the waypoint. These
`branch points lead to artificial reality worlds 35,39 and
`to other waypoint worlds 33 and 34. represented by the
`abbreviation “wp”. Their purpose is to present to the
`user a stimulus that contains information required for
`making a branch selection in order to move from within
`the waypoint. Selecting a particular branch moves the
`user tOward the desired world. Once the user acquires
`proficiency with the waypoint movement within
`worlds. sequences of selections allow him to move to
`any one of a large number of worlds. The number of
`worlds is determined by the number ofbranch points on
`a waypoint and the length of the selection sequence
`between the branchpoints.
`
`Possibility Space and the Creations and Editing of
`Automatic Icons
`
`In waypoint navigation, the user is effectively editing
`an N-tuple that represents his position within cybers-
`pace. If the user‘s position is considered to be an artific-
`ial reality artifact that is edited by navigation, the con-
`cept can be extended to the editing of other similar
`artifacts. That is, the user has available a definition ofan
`N-tuple that represents the position in a cyberspace. If
`each axis of the space represents a vector of values of
`interest, a possibility space can be constructed. Navigat-
`ing through_the possibility space can produce an N-
`tuple for use as a parameter of an artificial reality arti-
`fact. As shown in FIG. 3, each of the branch points
`gives information about which world the user will be in
`or what contacts the user will have if the branch is
`taken. This is done by taking a subset of the semantic
`information about that world and encoding it into ap-
`pearance features of an iconic display as shown in FIG.
`3. For example, one can give all
`icons representing
`waypoint worlds 3 square shape. Worlds where Newto-
`nian physics apply might be represented by red icons,
`and worlds created by the user might have larger icons
`to illustrate their relatively greater importance. An
`inherent problem is that the total amount of information
`the icon stands for is more than can be effectively en-
`coded using common techniques, i.e., shape. size, posi-
`tion,
`texture, annotation, animation, etc. Thus, small
`segments of data must be taken that comprise subsets of
`the total information available in the system. In addi-
`tion,
`the required subset of information changes, de-
`pending on the reason for the user‘s movement or
`search. This reason is a function of the relationships
`defined to be moved through the information space.
`
`8
`
`

`

`9
`Therefore, the N-space navigation concept can be ap-
`plied directly to the problem of defining the mappings
`of semantic information into graphical shapes and ap-
`pearances used in automatic icon systems.
`The problem inherent in defining semantic informa-
`tion by iconic shapes is tripartite. First, it is necessary to
`determine which subset of information is most impor-
`tant to the current search. For example, the mathemati-
`cal relationships forming the basic parameters for defin-
`ing the subsets of information and assigning priorities to
`them may be structured in accordance with a “degree of
`interest function,“ as discussed in “Generalized Fisheye
`Views,“ Proceedings of CH! '86 Human Factors In Coni-
`putt'ng Systems (ACM New York 1986).
`A degree ofinterest may. in turn, be composed oftwo
`parts: first, the importance of the object itself. which is
`determined by the combination of the content of the
`object‘s semantic information and the view desired (for
`example. piping information is a high priority