RPX Exhibit 1005
`RPX v. AIT
`
`

`
`and to transport and dynamically load interfaces that capture
`enough application semantics to maintain sessions and give
`adequate feedback.
`
`tools do not meet these re-
`interface development
`Traditional
`Interface builders are easy to use, but focus on low
`quirements.
`level details and do not provide adequate support for exploring
`designs and building customizable Uis [Szekely et al. 1993].
`Furthermore,
`they do not support
`interface transport, which
`requires a framework in which a U1 ciient can provide an inter-
`face from a description sent over the network.
`
`Currently, UI tools do not explicitly provide such a framework.
`Tools supporting automatic generation of Uis from high-level
`specifications provide enabling technology for interface trans-
`port. However, most of those tools focus on menus and dialog
`boxes
`(e.g., Mickey [Olsen 1989], Chisei
`[Singh and M.
`1989],
`ITS [Wiecha et al.
`I990}, DON [Kim and Foley 1993],
`TRIDENT [Vanderdonckt and Bodart 1993]},
`thus limiting the
`range of possible Uis. Also, most of the tools supporting
`automatic generation typically translate the specification into
`21 tom requiring compilation (e.g., MIKE [Olsen I986}, ADEPT
`[Johnson et al.
`i993]) and are thus not suitable for dynamic
`generation at run time.
`
`HUMANOID [Szekciy et al. 1992] and time [Foley et al. I99 1,
`Sukaviriya et al.
`l993]) use more sophisticated models. which
`are interpreted at run time and support a wider range of interac-
`tion styles. MASTERMIND [Neches et al. 1993, Szekely et ai.
`1995} is an effort to build a comprehensive model-based envi-
`ronment by integrating the HUMANOID and UIDE models and
`building on their strengths; it is still in the design stage, but is
`expected to provide both compiled and interpreted run-time Ul
`support.
`
`MUSE, presented in this paper, is based on the TACTICS model
`[Kovacevic l992a, Kovacevie l992b, Kovacevic I994], which
`is also derived from UIDE. TACTICS uses an application con-
`ceptual rnodel comparable to I-IUMANOID and UIDE, but it also
`has an explicit U] model and transformations for mapping the
`application conceptual model into the U] model, as well as for
`transforming Uis
`into a desired took and feel. TACTICS sup-
`ports dynamic generation because it does not require compila-
`tion of U1 structures.
`In addition, it instantiates the run-time UI
`structure which then executes without interpreting the applica-
`tion model, thus minimizing the run-time overhead compared to
`the interpreted environments such as HUMANOID and UID E.
`
`standards
`Standard Generalized Markup Language (SGML)
`[Newcomb er al. [991] provide a framework for interface trans-
`port based on a document paradigm. An SGML specification,
`instead of describing formatting features directly, describes
`document structure which a display engine can map to presenta-
`tions. SGML standards have enabled development of HTML
`{Hyper Text Markup Language) display engines, or browsers,
`such as Mosaic and Netscape. Their growing popularity is due
`to their case of use, simplicity, and effectiveness in presenting
`information [Laufmann 1994].
`
`Capabilities of user interface tools depend on their underlying
`model, how expressive it is and how much application seman-
`
`tics it captures. For instance, Web browsers also use a model,
`which is what enabled their success, but theirs is a very simple
`model, that of a hypertext document. Thus, these browsers are
`limited to document browsing and forms-based display and in-
`put. Graphical (direct) manipulation Uis are not supported,
`because the browsers do not have the notion of objects to be
`manipulated with semantics behind the manipulations.
`
`These HTML browsers also lack the notion of a session and a
`context history.
`If the order of interaction steps matters,
`the
`interaction must be broken into a sequence of documents. Simi-
`larly, a U1 in which what
`is available/enabledfpresented to a
`user depends on a previous context is not directly supported but
`requires additional helper applications acting as filters and ses-
`sion managers. Such a U1 has to be captured in a set of docu-
`ments. either predefined or dynamically generated.
`For in-
`stance, updating a list of items based on the most recent selec-
`tion requires fetching a new page and using either helper appli-
`cations that can modify pages or separate, predefined pages for
`each possible selection.
`
`HTML browsers share some similarities with early User Inter-
`face Management Systems (UlMSs}, which were limited in
`terms of the feedback they could provide because of their lack of
`knowledge about application semantics [Hayes et al. 1985,
`Tanner and Buxton 1983]. Similarly, HTML browsers are lim-
`ited to knowledge about document structure and hypertext links.
`Because in the case of HTML browsers the separation is not
`only conceptual but also physical,
`the internal
`interface may
`become a real bottleneck, either toward the server providing
`HTML pages that contain feedback for user selections, or to-
`ward hclper applications managing semantics needed for pro-
`viding the feedback.
`
`An additional limitation of HTML browsers is that they do not
`ailow dynamic changes to page contents;
`i.c., a whole page
`must be replaced for any change. Some proposed extensions to
`I~l’I'ML (e.g.,
`the push and pull mechanism by Netscape
`[Netscape l995]} allow limited changes by helper applications.
`Hotlava [Sun I995] goes a step further, by allowing integrated
`helper applications {“applets") that can also be shipped around
`as documents. Hotlava is “programmable“ and can support
`graphical
`interaction and allow packaging morelfunctionality
`in a U] (not just browsing). However, it does not directly sup-
`port dynamic creation of Uls, which is one of the Fundamental
`requirements we have. On the other hand, the Java language can
`provide a delivery platform for such a framework and we are
`exploring using Java in MUSE.
`
`’
`S L O O P
`SLOOP (the System for Learning Object-Oriented Paradigms) IS
`an intelligent
`tutoring system. Currently, SLOOP provides
`training for requirements modeling. Future SLOOP modules will
`address analysis and design modeling.
`
`SLOOP provides an intelligent coaching environment for prac-
`ticing object-oriented modeling skills.
`Students perfonrt a
`series of activities, or steps,
`to develop a requirements model
`(implementation independent) of a proposed system, starting
`from a problem set that consists of a high-level orientation
`diagram and use case scenarios describing structural and dy-
`
`I09
`
`

`
`narnie information and exceptions {Hurley and Hughes 1994}.
`At each step, SLOOP evaluates student responses and provides
`feedback and different kinds of help.
`-
`
`cation context corresponding to the activity practiced. The U]
`server parses the client requests and propagates necessary in-
`formation to the coach.
`
`Discussion of the HTML code generation and the relationship
`between MUSE and the HTML browser is beyond the scope of
`this paper.
`In the rest of the paper, we focus on how MUSE
`handles UI specifications and generates Uls, and what underly-
`ing model it uses.
`
`MUSE - Ul DEVELOPMENT AND MANAGEMENT
`
`MUSE is based on the TACTICS model, which supports the U1
`development process shown in Figure 2. The bolded compo-
`nents in the figure pertain to activities performed at run time by
`MUSE and are discussed below in more detail. An interactive
`
`authoring tool can be used to create an initial specification, as
`well as to modify (transform) the application and its lll (while
`preserving its
`functionality [Foley et al.
`1991, Kovaccvic
`l992b}), with changes being propagated back to the specifica-
`tion. The authoring tool and links that are shaded are beyond
`the scope of this paper.
`
`.
`2.‘
`=
`.
`.
`.
`.
`2.
`2' Interactive Author1nglCustom1zation Tool i
`in .....~...w.... wvawvvw ..
`..wm.»...,...
`.W.. ._.. ..._,....w..._._......
`
`High—Level
`Specification
`
`Interface -
`
`Conceptual
`Mdel
`
`SLOOP Architecture
`
`The early SLOOP prototype developed in 1994 was a mono-
`iithic application, built in Kappa [lntcllicorp I993]. Based on
`experience with this prototype and with a related intelligent
`tutoring system, LEAP [Bloom ct al.
`I995], which is in a de-
`ployment phase, we have decided to redesign SLOOP into a
`clientlserver architecture with a long term goal of developing a
`shell for intelligent tutoring systems.
`
`In developing SLOOP, one of our goals was to leverage on ex-
`isting, already available components; hence our decision to use
`hyper-text
`transfer protocols
`(HTTP) and HTML browsers.
`However, because of limitations of the current HTML-based
`technology, we developed MUSE as an additional component
`for handling interactions not supported by HTML browsers.
`
`Figure 1 shows a simplified view of the SLOOP architecture.
`We have abstracted details not pertinent
`to the focus of this
`paper. For instance, the coach component of the SLDOP server
`encapsulates representations of expert,
`instructional and stu-
`dent knowledge, as well as a session manager that keeps track
`of multiple active sessions and provides necessary information
`for generating UI requests sent to the LII server. The UI server
`prepares messages for the UI client, specifying what UI to load,
`or how to change it. Depending on its interaction requirements,
`the U1 can be loaded as an HTML page handled by the HTML
`browser, or as a specification handled by MUSE.
`In both cases,
`the specification is assembled dynamically, using a predefined
`template and the current context data (c.g., information specific
`to a student and a lesson being practiced).
`
`I-ITTML Browser
`
`(Netscape)
`
`
`
`SLOOP Server
`
`Figure 1 - SLOOP Architecture
`
`The Web server manages the actual communications between
`the SLOOP server and UI clients, and it consists of an HTTP
`demon and additional communication programs. The U1 client
`consists of a commercially available HTML browser (currently
`Netscape), MUSE, and a client communication program [CCP)
`that links the two.
`
`
`
`generate ” '
`
`Figure 2 — UI development in MUSE.
`
`A U1 [see Figure 2) is generated from a high-level specification
`that describes application semantics and, optionally, details of
`a desired look and feel. The high-level specification is first
`parsed into an application conceptual model. Details of the
`desired look and feel are associated with the corresponding
`components of the application model as Ul hints. The applica-
`tion rnodel defines application information requirements and a
`U1 is generated to meet these requirements. The resulting Ul
`also tries to achieve the desired look and feel, when possible,
`by following the UI hints given. The generation process is
`described in more details using an example based on the SLOOP
`practice activity lderttrfiz Static Associations.
`It is a step in the
`methodology for building an 00 model involving identifying
`static relationships among entities. The real activity and its U1
`are more complex, but we have simplified the example in order
`to better focus on salient points of the generation process. The
`specification for the activity is shown in Figure 3, and the re-
`sulting UI in Figure 4. We now discuss how this graphical, di-
`rect manipulation interface is produced from the given specifi-
`Cation.
`
`to the SLOOP
`Student requests are passed from the U1 client
`server, together with the relevant context. The context can be
`as simple as a token identifying the type of request (e.g.,
`re-
`quests originating from a browser), or as complex as the appli-
`
`High-level Specification
`'The specification typically consists of four parts describing the
`initial context (after “:init" keyword), data types (after “:data“
`keyword), data model (after “:objects“ keyword), and control
`model (after “:aetions” keyword). The initial context is speci-
`
`I10
`
`

`
`zapplication-header StaticAssoeiations
`:ui (:po ((Dia1ogBox :namc "Identify Static Associations" :template Graphical znlenubar t)))
`:init
`(zobjects ((noun-phrase (name consortium)
`(name banks) ...))
`:data ((:data-type cardinality :enum (0 Many)))
`zohjects ((:object noun-phrase :attributes ((:attr name :type :string)) :ui (zpo (nil)))
`(zohject actor zui (zpo (Actor))
`:attributes ((:attr name) (:attr cardinality :type cardinality)
`(:attr external-properties :set-min 0 :set-max nil
`:type (mbject-instance external-property))
`(:attr noun-phrase ttype (zobjeet-instance noun-phrase))))
`(zobject object-class ...)
`{zobject external-property :attributes ((:attr name)
`(:attr origin ttype (zobject-instance (actor object-class)))
`(:attr target :type (:object-instance (actor object-class))))
`_
`_
`_
`nu (ma ((F:.xternal-Property zinclude {origin target)
`:n1ap-pro perti es ((origin origin-obj) (target target-obj ))))))))
`zactions {(:action CoachAssistanoe :rnssg Mssg2Coach :parameters 0 zxform ((:confirm :implicit)
`:ui (zitec (:select ((MenuItern :narne "Coach Assistance" zsubmenu Coach)))))
`(:action CoachHint ...)
`(:action identify-actor :rnssg sar-create-actor zxform (treusable (:visible always))
`:pararneters ((:par noun-phrase :type (zobject-instance noun-phrase)
`:ui (:itee ((ChoiceSe1ection:filter(:attributes name)
`:prefix I :name "Noun Tenn"))))
`(:par cardinality :ty pe cardinality»
`:prec o ii iii ti 0 ns ((exist noun~phrase)))
`(:action identify-object—class ...)
`(: ac ti 0 ll identi fy-external-property trns s g sar—create-external-prop
`zpararneters ((:par name ztype tstring)
`(:par origin ztype (:object-instance (actor object-class))
`: ui (: i tee ((MouseButton))))
`(zpar target ztype (zobject-instance (actor object-ciass))
`:ui (:itee ((MouseButton :action Re1ease)))))
`:precondi tions ((exist (object-class actor) 2))
`zpostconditions 0
`txform (:reusable (:visihle always)))))
`
`Figure 3 — A high-level specification.
`fied in terms of instances to be created at start up time. The data
`model concerns the application objects,
`their attributes and
`relationships.
`The control model specifies application ac-
`tions,
`their parameters, pre- and postconditions, and relation-
`ships to other actions (task structure) and objects.
`
`The specification provides four types of information: applica-
`tion semantics,
`initial context, U1 hints, and design transfor-
`mations. Only the first one is necessary,
`the other three are
`optional. Application semantics are defined in terms of the
`data and control models. The initial context can be defined
`
`only at the application level, while Ul hints and design trans-
`formations ean be defined at different
`levels. associated with
`the application component as a whole, or with the lower level
`components such as objects, actions and parameters.
`
`The high-level specification is directly translated into an ap-
`plication model, and then into a Ul, as is described in the fol-
`
`the high-level
`It is in this sense that
`lowing subsections.
`specification serves as an external, persistent representation of
`the application and its UI that can be stored and communicated
`between the server and U1 clients, and as such it facilitates in-
`terface transport. The specification fully defines the applica»
`tion model, but not the UI model.
`It contains only those Ul
`details that we do not want MUSE to decide for us.
`
`Because the main role of the specification is to serve as an ex-
`tental representation, we were concerned more with its content
`than with its form.
`It is not intended to be directly created and
`modified by UI designers, but through an interactive authoring
`tool, which is why we did not try to make it more user friendly.
`
`Without going into all the details of the specification syntax,
`let us just say that each concept has a set of properties that can
`be defined, and each property is specified as a keywordlvalue
`pair “keyword value" where value can be a single token or a
`
`lll
`
`

`
`in the case of the
`and how to present the items.
`parameter noun-p}tr'ttse of the action irt‘erttt')fv—
`actor,
`the interaction technique knows
`(From
`the parameter type) that choices are instances
`of object class rtotttt—p;‘tt'a.s'e and the hint speci-
`fies that these instances be presented using the
`value of their attribute nature.
`For the action
`
`specify
`identmr-exttztnal-_oropert_it, Ul hints
`that both the origin and target parameters
`should be selected using a mouse {Monsc»Battor.-
`interaction]: to prevent ambiguities as to which
`atti-ibute’s value is selected, hints specify that a
`button release selects the target, and a button
`pfis (a default button operation)
`selects the
`origin.
`
`Each practice activity in SLOOP can have a
`number of requests for different kinds of coach
`assistance and help. For simplicity,
`the exam-
`ple has only two coach actions, Coach/lasts-
`tartce and Coacltrltnt. which will both be placed
`in a submcnu Coach in the main rnenubar.
`
`Application Conceptual Model
`The Ul is not generated directly from the specification, but from
`the application conceptual model (or aplication model, AM)
`instantiated based on the specification content. The AM pro-
`vides internal representation of the information contained in
`the specification and is used by the generation, transformation,
`and consistency checking rules [Kovacevic 1992b].
`
`
`
`ns rtium
`
`
`
`
`
`"teach-
`
`Tdenti F5}.-ExtPro;':i' " .-
`
`O-
`
`
`
`-emwwtcmsteaegzs-.ns-e;a.imweae-an:w,tu
`
`Application |
`Object
`Application '
`
`Action
`
` Attribute
`
`Type
` I.
`Action
`
`Parameter
`
`
`Data
`
`
`
`Preconditions
`
`
`
`Postconditons
`
`
`
`Action '
`
`Action
`
`Figure 5 —- Application conceptual model.
`
`Figure 5 shows the seven entity types of the AM and their rela-
`tions. Objects can be connected to other objects through in-
`heritance and part/whole relations. Parttwhole relations be-
`tween actions can be used to define composite tasks. More
`details on the AM can be found in [Kovaeevic 1992b]. Some of
`the relations shown are not necessary for the generation proc-
`ess, but provide information needed by the conceptual
`trans-
`formations [Foley i987], which is why not all relations are
`defined in the specification in Figure 3.
`
`Whereas the AM could be created and modified interactively,
`‘MUSE currently instantiates it solely from the specification.
`The initial context is provided by the coach component at run
`time, while the rest of the specification is defined at design
`time.
`
`112
`
`Figure 4 — UI generated for the specification in Figure 3
`(5.!)
`
`and are
`list of tokens. Keywords typically begin wit
`shown bold in the examples.
`For instance,
`the top level of
`specification corresponds to the application concept with key-
`words (properties)
`:app|ieation-header
`(for application
`name),
`:ui
`(U1 hints).
`:init
`(initialization
`information),
`:data (user-defined data types), zobjects
`(data model defini-
`tion}, and :actions
`(control model definition). The proper-
`ties can be specified in any order.
`
`the application
`The example shown in Figure 3 specifies that
`Staticzlssociatioris has one user-defined data type (cardinality),
`data model comprising of 4 object classes {norm-phrase, actor,
`object-class, and exterrta!—properry), and control model with S
`actions (Coachrlssistance, CoachHt'rtt, identtjjz-actor, ld€l'i!tfy—
`object-class, identify’-externaI-property). Note that each object
`in turn also represents a user-defined type and can be used when
`defining object attributes and action parameters. For instance,
`the actor object has attribute cardinality of (user-defined data)
`type cardinality, as well as attribute external’-properties of
`{user-defined object) type external-property. The object exter-
`nal-properry also has attributes of object type, origin and tar-
`get, which point
`to an instance of either object-class or actor
`object class. Each external-property must have one and only
`one value for attributes origin and target, which is the default
`for any attribute and parameter. On the other hand, each object-
`class or actor object class can have any number of relations,
`indicated by “:set-max nil" (no upper limit on number of val-
`ues).
`
`UI hints are defined at several levels in our example: at the ap-
`plication level they specify that the application will use for its
`main window a dialog box template Graphical, which will be
`named Izientifi: Static Associations and will have a top level
`menu bar. Ul hints for object classes specify what presentation
`objects to use and how to configure them.
`For actions,
`the
`hints specify what window to use,
`if needed,
`and for action
`parameters what interaction techniques to use and how to con-
`figure them. For instance,
`the Choiceselection technique al-
`lows specifying where to get items forming the list of choices
`
`

`
`in TACT1CS, a U1 is no longer only a passive
`Consequently,
`transducer between the user and the application's
`functional
`core; it also performs three major tasks: communication, buff-
`ering, and controlling and maintaining information flow.
`
`Moving this boundary increases reusability and flexibility of a
`U1. A UI tool can provide more support and can change at U! in
`more ways without requiring changes in the application spe-
`cific part;
`it allows more control over the behavior — the feel
`aspect of look-and-feel. While a U1 has greater functionality
`and designers can exercise more control,
`this is not additional
`burden for designers because of the increased reuse. The bound-
`ary is moved by extracting functions which are not application-
`specific out of an application and placing them into a U1.
`These functions are generalized to become reusable building
`blocks of an application, or more specifically, its UI.
`
`Figure it illustrates the old and the new boundaries. The old
`boundary represents a traditional view of U15. We have moved
`this boundary and the UI now consists of a number of functional
`components performing different U1 tasks.
`The communi-
`cation, buflering, and control components are necessary parts
`of a U1, because they perform mandatory U1 tasks, but a U1 can
`also include anciflary components for performing optional
`tasks, such as error-recovery and help. These tasks belong to
`the U1 because they do not contribute to the application's func-
`tionality but to the quality of its Ul. They are optional because
`a U1 can function without
`them,
`though a user will probably
`perceive such a U] as of lesser quality. More details on each
`type of component, and on individual primitives, can be found
`in [Kovaccvic l992b, Kovacevic 1993]. Here we only review
`major components and then continue with our example to illus-
`trate how the Ul model is composed and how it functions.
`
`Ul Primitives
`
`The three components (control, buffering and communication)
`perform a variety of tasks and each is supported by a number of
`U1 primitives, or building blocks.
`Figure 8 shows a partial
`
`Application
`
`Functional
`
`Core
`
`User Interface
`_
`
`
`
`Semantic
`Action
`Routines
`
`Figure 7 — Moving the boundary between the application and
`the U1 functionality
`
`113
`
`
`
`identify-externahproperty
`
`
`Pre
`Pst
`condition
`conditions
`
`
`
`
`
`
`built-in type
`string
`
`instance of
`user-defined class
`
`object-class
`
`l Application
`
`Action
`
`Parameter
`
`Data
`Type
`
`Apphcatmn
`Object
`
`Figure 6 — Partial conceptual model
`property action.
`
`for r'denrifiJ-e.xrerooI-
`
`Figure 6 shows a partial instantiation of the AM corresponding
`to the specification in Figure 3.
`It models the action identify-
`exrernal-properry, showing that it has 3 parameters and what
`their
`types are. Parameter name is of built-in type .'srn'ng,
`while the other two have the type representing an instance of
`object classes actor or object-class. Note that Figure 6 does not
`show any U1-specific details — these are resolved when mapping
`the AM into a concrete U1 structure. The AM reflects the infor-
`
`mation requirements of the application and does not depend on
`specific U1 choices (although U1 hints are stored internally for
`later use during the generation stage}. For instance, Figure 6
`illustrates that the action idermfy-exrernaI-property requires a
`string, which will be interpreted as a relation name, and two
`instances of either actor or object-class. However,
`it does not
`say how these values are to be provided — using some default or
`global values, or by a student and, if so, how. That
`is deter-
`mined in the U! model, described next.
`
`UI Model
`TACTICS views a user interface as a collection of primitives
`that are structured based on the application and on the desired
`dialogue style. The TACTICS U1 model
`identifies Ul compo-
`nents,
`their properties, and structuring principles for assem-
`bling components into a coherent
`interface. The behavior and
`the look-and-feel of a U1 are explained in terms of properties of
`UI components and the way they are structured. The look and
`feel can be changed by applying transformations which modify
`the structure and reconfigure primitives in a principled way
`[Kovacevic 1992b]. Uls are structured around a blackboard ar-
`chitecture, which is a base for integration of different modali-
`ties [Kovacevic 1994].
`
`The TACTICS model moves the boundary between a U1 and the
`application-specific part by identifying a layer of components
`that maintain and utilize the U1-related context of an appli-
`cation.
`In that sense it goes beyond the traditional U1 tools and
`models that focus only on input and output communication
`tasks. TACTICS identifies both a set of buffering components
`that store the U1 context and a set of supporting control com-
`ponents that maintain and utilize the contextual
`information.
`
`

`
`action, such as default and global
`parameters values.
`it serves as a
`local blackboard where control
`primitives pertaining to the some
`action keep information needed
`for their synchronization, eg.
`to
`cnfot'cc specific sequencing, con-
`straints.
`
`ob_;'ecr—co:rre.rl—proror_vpr2
`The
`primitives keep global proper-
`ties for each application object
`class,
`including
`default
`and
`global attribute values. The ob-
`ject-context-:n.s'ronce primitives
`keep attribute values
`for object
`instances which are part of the U]
`context
`ti.e., are not
`internal
`to
`the functional core).
`
`
`
`
`
`Context Context Context
`
`Infoation
`Flow
`Control
`
`Buffering Component
`
`Communication Cornponont
`
`Objct Action ha}
`
`Control Componams
`
`Interaction
`
`Technique
`
`Figure 8 -— Ui Modeling Primitives
`
`hierarchy of Ul modeling primitives. Each primitive is defined
`by a set of properties which control
`its behavior, methods
`which deliver that behavior, and messages it can receive from
`or send to other primitives. Primitives are configured to serve a
`specific role by setting their properties depending on applica-
`tion semantics and the desired dialogue style. Designers can
`modify these properties and change the look and feel of a {II by
`applying design transforrnations [Foley et al. 1991, Kovacevic
`1992b, Kovaeevic i993].
`
`components. Two types of U1 primi-
`Communication
`tives perform communication tasks: interaction techniques map
`external
`inputs into internal
`forms, and presentation objects
`map a given internal
`information form into desired presenta-
`tion properties. such as color, size, shape, or tone pitch.
`
`Buffering components. Different aspects of the overall Ul
`context are kept
`in three types of U1 primitives:
`action-
`context, object-context, and global-context.
`
`During the activation of an action, the action-context primitive
`keeps all (known) pieces of information needed to perform the
`action. decoupling the order in which information is specified
`from the order in which it
`is passed to the application func-
`tional part. For instance, when a user enters a parameter value,
`the value is not necessarily passed to the functional core right
`away, but it may be stored in the U1 until all values are col-
`lected. This way,
`the application's functional part does not
`depend on the order in which values are entered — it is shielded
`from a specific syntax which can be changed without having to
`change the functional core. Between the activations,
`the ac-
`tion-context instance keeps the information which is reusable
`across multiple activations of the action and specific to that
`
`The action and the object related
`buffering primitives
`implicitly
`describe some aspects of the U1
`state;
`the remaining context
`in-
`
`J formation is maintained in the
`
`gt’obm'—contexr primitives. They
`serve as a public blackboard ac-
`cessible to all, which is not
`the
`and objcct~context primitives.
`case with the action-context
`The postings
`include state information for maintaining the
`sequencing and information flow control,
`and properties of
`sets, e.g. currently selected objects and clipboard objects.
`
`Control components. Their role spans three major tasks,
`with specialized U1 primitives for each:
`information flow con-
`trol, sequencing control, and event propagation.
`
`in-
`primitives maintain
`{ll-‘C3
`Information-flow-control
`formation flow in a U1, integrating functional parts of the U1
`into asingle structure. These primitives act as intermediaries
`in obtaining information required by an action. They know
`where to look for each piece of information —- whether to get it
`from a communication primitive or from a buffering compo-
`nent, and which one.
`
`The sequencing control primitives maintain and monitor the
`relevant Ul context. They update the context whenever some-
`thing potentially affecting IFC primitives happens, and they
`constantly evaluate the context
`to enablefdisable those primi-
`tives. The two tasks are performed by specialized primitives:
`postconditions for updating the context and preconditions for
`evaluating it.
`
`The event-propagation primitives propagate events of interest,
`performing relation detection and enforcement.
`
`We now show how these primitives are used to build a Ul struc-
`ture.
`
`Ul Generation
`
`The UI generation process is driven primarily by the informa-
`
`I14
`
`

`
`Interaction
`
`
`
`InputBroker
`
`Interaction
`
`
`
`l St
`
`Stagfi
`
`2nd
`
`stage
`
`3rd
`U’)-1-D3D0G
`
`
`
`rec. Select
`1 P
`Action Event:
`Select
`OSI.
`
`Select
`
`
`
` IgllllllllllllIIIIIIIIIIIIllllllllllll
`
`
`
`P P
`
`rec. name
`Par. Value:
`name
`
`Post. name
`
`Prec. ori :_ in
`
`
`
`
`Technique:
`_, MouseButton
`
`;
`Eton release
`
`
`
`Par. Value:
`Earet
`
`--
`
`. Action Event:
`Cancel
`
`
`
` lllllllllll
`
`Action Event:
`Confirm
`
` Ilgllilllllllll
`
`Interaction
`
`
`
`Technique:
`
`Button
`" button
`
`no
`
`filll
`
`
`
`Figure 9 — UI generation steps creating a partial Ul structure for action fdenttfit-externa!-property fi'om its
`conceptual model.
`
`tion requirements of an application, and secondarily by the
`requirements of a selected look and feel. The secondary re-
`quirements are satisfied only if all necessary interaction com-
`ponents arc available and compatible with the information
`requirements.
`
`The U] generation consists of three steps. The first step in-
`voives transformations that transition from application con-
`ceptual model to a [ll model that preserves the original applica-
`tion
`semantics while incorporating additional U1 domain
`knowledge. These transformations map each application mod-
`eling primitive (Figures 5,6) onto one or more U1 primitives
`{Figure 8}, configured accordingly, and build an initial Ul struc-
`ture that reflects all the information needs of the original appli-
`cation. The initial Ul structure consists only of buffering and
`control primitives.
`For instance, each Application Action
`primitive is mapped onto one Action Context buffering primi-
`tive and three Action Event primitives: Select Action (makes
`the action active), Canoe! Action (deactivates the action}, and
`Confirm Action (confirms action parameter values and invokes
`the semantic action routine in the functional core that uses the
`parameters}.
`
`Figure 9 shows the UI structure generated for that part of the AM
`shown in Figure 6. The AM components are shown with the
`dark background. The Data Type components from Figure 6 are
`not shown here, but this and all other related information that
`can be inferred from the model
`is used when configuring Ul
`primitives. Thick lines link the AM components
`to the U1
`components instantiated and configured in the first generation
`step. The Ul components are configured to preserve the seman-
`tics captured in the AM. For instance,
`the action‘s semantic
`preconditions (precondition “exist ((object-class actor) 2)“ in
`Figure 3, meaning that the action is enabled only if there are at
`least 2 instances of classes Actor or Object-Cioss) are incorpo-
`rated into preconditions of the Select Action component. Simi-
`larly, postconditions,
`if any, are mapped into postconditions
`of the Confirm Action component. The generation rules also
`include syntactic constraints on sequencing into the pre— and
`postconditions generated for Information Ftow Control Primi-
`tives.
`For
`instance, preconditions of the Confirnr Action
`primitive will require that
`the action be selected and that all
`parameters have their values.
`In turn, each Parameter Value
`primitive‘s postconditions will assert that the parameter has a
`value, and Select Action primitive’s postconditions will assert
`
`ll5
`
`

`
`In the case of prefix syntax, e