CHI’92
`
`May3-7, 1992
`
`COUPLING APPLICATION DESIGN AND
`USER INTERFACE DESIGN
`
`de Baar
`Dennis J.MJ.
`of Technology,
`of Technical Mathematics
`and Informatics,
`Delft University
`Faculty
`email: winfddb@duticai.
`tudelft.nl
`Binnenwatersloot
`3,2611
`BJ Delft, The Netherlands,
`
`James D. Foley
`of Technology
`of Computing,
`Georgia
`Institute
`College
`emaik foley@cc.gatech.edu
`Atlanta, GA 30332-0280,
`
`Kevin E. Mullet
`Technology
`Group, SunSoft
`Interface
`Human
`2550 Garcia Ave. MS 1-07, Mountain
`View, CA 94043-1100,
`
`Inc.,
`email; mullet@ sun.com
`
`ABSTRACT
`
`Building an interactive application involves the design of
`both a data model and a graphical user interface (GUI) to
`present that model
`to the user. These two design activities
`are typically approached as separate tasks and are frequently
`undertaken by different
`individuals or groups. Our approach
`eliminates redundant specification work by generating an
`interface directly from the data model
`itself. An inference
`engine using style rules for selecting and placing GUI
`controls (i.e., widgets) is integrated with an interface design
`tool
`to generate a user interface definition. This approach
`allows a single data model to be mapped onto multiple GUI’s
`by substituting the appropriate rule set and thus ~presents a
`step toward a GUI-independent run-time layout facility.
`
`KEYWORDS: User Interface Software, Automatic User
`Interface Design, Data Models
`
`INTRODUCTION
`
`An early step in the design of an interactive application is
`the definition of the application’s data model. In an object-
`oriented design, the data model consists of an object class
`hierarchy in which each object has an associated set of
`attributes and methods. Single or multiple inheritance is
`typically used to avoid repetitive specification of shared
`methods and attributes. The attributes and methods of an
`object are either internal or external.
`Internal attributes and
`
`is granted
`fee all or part of this material
`Permission to copy without
`provided that
`the copies are not made or distributed
`for direct com-
`mercial advantage,
`the ACM copyright notice and the title of the publica-
`tion and its date appear, and notice is given that copying is by permis-
`sion of the Association for Computing Machinery. To copy otherwise,
`or to republish,
`requires a fee and/or specific permission.
`
`01992
`
`ACM 0-89791-513-5/92/0005-0259
`
`1.50
`
`methods are meant for use within the application and are not
`exposed in the user interface, External
`attributes
`and
`methods are represented in the user interface as standard
`interaction
`objects such as buttons, settings, or sliders
`(hereafter referred to as “controls”) or as data manipulated
`directly by the user.
`
`The design of the user interface normally takes place only
`after the application data model has been completed. The
`external attributes and methods of the data model must be
`mapped onto a set of controls
`in the target GUI. These
`controls must
`then be divided
`among one or more
`application windows and arranged to fit within the available
`space while maintaining the organizational characteristics
`required for clear communication
`and aesthetic quality.
`Guidelines
`for control
`selection
`and arrangement
`are
`provided by GUI-specific
`style guides such as the OPEN
`LOOK GUI Application
`Style Guidelines
`[13],
`the OSF/
`Motif Style Guide [1 1],
`the Apple Human Interface
`Guidelines
`[1] and the CUA Style Guide [7], These
`documents provide high-level
`rules and principles that help
`to maximize
`consistency
`across applications
`in the
`respective GUI’S.
`
`software tools provide
`A number of commercially-available
`a GUI interface to the GUI design task. GUI builders such as
`Devguide [14], Interface Builder
`[9], and Interface Architect
`[6] present
`the designer with standard user interface
`components that can be dragged onto the work surface and
`arranged using direct manipulation. Each of these tools can
`be used to generate source code or executable
`for a
`particular configuration of GUI components, but all rely on
`the skill and knowledge of the user interface designer to
`create the desired configuration
`themselves by manually
`selecting the appropriate
`controls and specifying
`their
`locations within the parent window, This limitation means
`that the interface designer is ~quired to repeatedly perform
`three tasks that are at least potentially unnecessary:
`
`259
`
`PayPal Inc. v. IOENGINE, LLC
`IPR2019-00886 (US 8,539,047)
`Exhibit 2004
`
`

`

`v [HI ’92
`
`May3-7, 1992
`
`(1)
`
`(2)
`
`(3)
`
`Access details of the data model, either from
`documentation or from (the designer’s) memory.
`Access and apply GUI-specific control selection rules that
`determine how each element of the data model is mapped
`onto a particular control
`in the target GUI.
`Access and apply GUI-specitic layout rules governing the
`placement of each control
`in the target GUI:
`
`for example, the designer must
`To create a property window,
`know the attributes (properties) of the object represented by
`that property window, decide on a particular GUI component
`to use for each attribute, and arrange the components within a
`window according to the layout conventions established by
`the target GUI.
`
`generation of window and menu layouts from
`Automatic
`information already present in the application data model can
`relieve the application designer of unnecessary work while
`providing an opportunity to automatically apply style rules to
`the interface
`design. A number
`of existing
`systems
`demonstrate the feasibility of automating one or more of the
`design tasks identified
`above. Mickey
`[10] generates a
`Macintosh user interface (menus and dialog boxes)
`from
`interface descriptions embedded in Pascal. ITS [15] and Jade
`[16] use a set of style rules created by a style expert and a
`specification of the dialog content to generate dialog boxes.
`Jade also includes graphical editing capabilities, which allow
`the designer
`to refine
`the user
`interface
`generated
`automatically
`by the system. Chisel
`[12]
`includes a set of
`“hints” which assist in placing dialog box elements. DON [8]
`combines a User
`Interface Design Environment
`(UIDE)
`specification [4] with a set of layout rules to design dialog
`boxes that reflect visual design goals such as weight and
`balance.
`
`This paper describes an approach to automating the interface
`design process that addresses all
`three of
`the potentially
`redundant
`design tasks identified
`above. Our system
`integrates D2M2edit
`[2], an interactive tool for creating data
`models, Devguide [14], an interactive user interface design
`tool, and a set of design conventions embodied in the OPEN
`LOOK GUI Application
`Style Guidelines
`[13]. We will
`describe the relation between the application data model and
`the user interface, mechanisms used to select a set of controls
`to represent a data model and arranging them within a target
`window,
`and a design environment
`that
`integrates
`the
`semantic
`data modelling
`and user
`interface
`design
`capabilities. A companion video tape segment is included in
`the conference video tape.
`
`DATA OBJECTS AND INTERACTION OBJECTS
`
`application, every
`In the data model of an object-oriented
`data object has associated attributes
`and actions. Together,
`these constitute the semantic layer of the application.
`In a user
`interface management system (UIMS) such as UIDE [4],
`the
`actions are associated with ure- and ~ost-conditions
`describing the state of
`the application be~ore and after the
`
`260
`
`action is invoked. This information, when available, can be
`a valuable aid in determining the most appropriate means of
`presenting the application semantics to a user. Interaction
`objects are controls such as buttons, sliders, or menus that
`can be manipulated directly by the user. Every interaction
`
`Attributes
`
`Figure 1 The relation between application data and
`interaction objects.
`
`in the user interface is associated with an action or
`object
`attribute in the application data model (Figure 1). In Figure
`2,
`for example,
`the attribute
`‘volume
`Input’
`in the
`application data model
`is linked to a slider that can be used
`to change its value.
`
`Attribute:
`
`integer
`-
`
`volumeInput:
`range[O..lO]
`recision[low]
`ength[2]
`f
`label[Volume]
`
`Figure 2 An attribute and its descriptors from the
`application data modei aiong with its
`representation as an interaction object in
`the user interface,
`
`includes information that
`the data model
`In this example,
`specifies the number of digits (length),
`the allowable values
`(range), and the associated label string (label). All of these
`characteristics are inherent attributes of the data object. The
`data model may also include supporting information,
`or
`that can be used by the modeler
`to clarify
`the
`metadata,
`intended semantics of the data object.
`In this example, a
`piece of metadata (precision) provides a rough estimate of
`the accuracy users will expect
`in manipulating
`the data
`object. Metadata describes the way a data object will be
`used instead of the qualities inherent in the data object itself.
`
`is
`Metadata provides guidance in control selection that
`essential if the flexibility of the graphical user interface is to
`be fully exploited. Sophisticated control selection schemes
`can be devised if extensive metadata is available, but this
`approach tends to defeat
`the goal of
`isolating
`the data
`modelling task from the interface design task. Metadata
`specified as part of
`the application
`data model should
`therefore address only characteristics that are inherent in the
`intended usage or structure of the data objec~ it should not
`
`PayPal Inc. v. IOENGINE, LLC
`IPR2019-00886 (US 8,539,047)
`Exhibit 2004
`
`

`

`~ (H1’92
`
`May3-7, 1992
`
`simply provide hooks on which to hang interface design
`decisions.
`
`and interface
`Dependencies between the data modelling
`design environments
`can be reduced by making interface
`related metadata an optional part of the data modelling task.
`If
`the metadata is omitted
`during data modelling,
`an’
`appropriate control can still be selected on the basis of the
`default for that characteristic. The resulting selections will be
`satisfactory, but may not be optimal.
`
`Data Model
`
`Actions
`Attribute
`n-
`
`Inference Engine
`
`uInteraction
`
`Library
`(Toolkit)
`
`rlUser
`
`Interface
`Description
`(GIL Fde)
`
`MAPPING DATA OBJECTS INTO INTERACTION
`OBJECTS
`
`data
`In the current system, objects from the application
`model are used as input to an automatic control selection and
`layout
`facility. An inference engine uses the actions and
`attributes of the data objects to generate a set of interaction
`objects
`in a direct manipulation
`interface
`design
`environment, which instantiates the interaction objects using
`a GUI toolkit and produces a textual specification describing
`the GUI components and their locations within one or more
`windows (see Figure 3).
`
`Following the selection of specific interaction objects, a set
`of layout rules determines the location of each object. The
`layout
`rules implement a set of high-level graphic design
`conventions addressing the size, spacing, and alignment of
`individual controls.
`
`The inference engine consists of a control selection compo-
`nent and a layout component. Both components me imple-
`mented as linear lists of simple if-then rules that can be easily
`modified and extended to accommodate local design conven-
`tions or different GUI standards. The inference engine is de-
`signed to be independent of the particular interaction library,
`or toolkit,
`that implements the standard GUI components in
`the target environment. The engine produces general conclu-
`sions that can be used to select and lay out appropriate con-
`trols for any target GUI toolkit
`for which an appropriate set of
`mapping rules has been defined. The following sections pro-
`vide an overview of the two rule sets needed to generate an
`interface design.
`
`CONTROL SELECTION RULES
`
`The control selection rules make use of the OPEN LOOK GUI
`components shown in Figure 4, Additional widget
`types can
`be added to the rule base easily enough,
`though to make
`iterative design practical,
`the new widgets must be available
`in the design environment as well. On the following pages
`we present
`the control selection rules in a decision-table
`format.
`In Tables 1-4 below, a blank space means that the
`value is unchanged from the previous
`row in the same
`column. The value, any, means that the outcome for that row
`in the decision table is the same regardless of the value taken
`by the parameter in question.
`
`261
`
`Figure 3 information fiow in the automatic controi
`seiection and layout process.
`
`command‘“”0” m
`MenuButtolc(imiiv)
`
`Exclusive Settirrgx
`
`Alternative 1 Alternative 2
`
`Non-exclusive
`
`Set4ings: _
`
`-[
`
`Check BOX ~ Choice
`
`Settings Menu: ~ Choice
`
`El
`
`‘wO’’ingL’stL!
`
`Multi-Line
`
`Textfield:
`
`.
`
`Single Line Textfield:
`
`Numeric Field
`
`3
`—m
`
`Siider w/ Typa-in
`
`Field: O
`
`0 II
`
`~,,,,,,’n’oo
`
`Siide~ O II
`
`~,,11,
`
`,’”oo
`
`Gauge: ~
`
`100
`
`Read-only Message: Vaiue
`
`Figure 4 OPENLOOKGUI components used by
`the system.
`
`real,
`integer,
`types (boolean,
`attribute
`Five different
`enumerated, and string) are covered by the control selection
`rules. The control selection rule for individual
`boolean
`attributes is trivial: check boxes are used for all booleans.
`The OPEN LOOK GUI does provide an alternative control
`-
`the nonexclusive setting - but the more intuitive syntax and
`semantics of the check box make it clearly preferable for
`isolated boolean attributes.
`
`PayPal Inc. v. IOENGINE, LLC
`IPR2019-00886 (US 8,539,047)
`Exhibit 2004
`
`

`

`May3-7, 1992
`
`Cmttent
`
`Limits
`
`Range
`
`Precision Widget
`
`editable
`
`unknown
`
`any
`
`known
`
`small
`
`read-only
`
`any
`
`large
`
`~Y
`
`any
`
`low
`high
`
`any
`
`~Y
`
`Textfield
`
`Slider
`Textfield
`
`Texffield
`
`Read-only Message
`
`Table 2
`
`Control selection for rea/ attributes.
`
`The selectionrules for real data objects (Table 2), specify a
`slider
`for real values with high precision
`only when the
`range is within -1 to +1. Otherwise, a standard textfield is
`used (the OPEN LOOK GUI does not define a numeric field
`for real numbers).
`
`kerns
`
`Set Size
`
`Label
`
`Min Max Widget
`
`few
`
`static
`
`short
`
`med
`
`long
`
`dynamic
`
`any
`
`many
`
`any
`
`~Yll
`
`1
`01
`o
`
`1
`01
`O
`
`1
`01
`o
`
`1
`01
`O
`
`01
`o
`
`1
`
`1+
`
`1
`
`1+
`
`1
`
`1+
`
`1+
`
`1
`
`setting*
`Exclusive
`Variation on exclusive
`1+ . Nonexclusive
`setting
`
`setting*
`Exclusive
`Variation on exclusive
`Check boxes
`
`setting
`
`setting
`
`settings menu*
`Exclnsive
`Variation on excl settings menu
`Nonexclusive
`settings menu
`
`settings menu
`Exclusive
`Variation on excl settings menu
`Nonexclusive
`settings menu
`
`list
`scrolling
`Abbwviated
`Variaticxr on excl scrolting list
`Nonexclusive
`scrolling
`list
`
`The selection rules for numeric data (integers and real
`numbers) depend on three attributes of the data object and
`one piece of metadatix
`. Content
`- whether the value of the attribute can be edited
`directly by the user (writable)
`or not (read-on/y).
`0 Limits - whether the upper and lower bounds on potential
`values can be determined in advance (known) or not
`(unknown).
`0Range - the relative size, either large or small of the range
`of possible values as determined by the limits. Attributes
`with unknown limits are always considered to have a large
`range.
`
`* Precision - whether the user is more interested in an
`approximate value that is meaningful only in relation to
`the range of possible values (low) or in a precise value that
`is meaningful
`in its own right (high).
`
`these variables to allow for
`to parametrize
`It is important
`easy customizing on a project-or enterprise-wide basis. The
`specific values assumed in the current rule set are presented
`along with the decision table for each type of attribute in the
`discussion below.
`
`Content
`
`Limits
`
`Range
`
`Precision Widget
`
`editable
`
`nnknown
`
`known
`
`read-ordy
`
`unknown
`
`known
`
`any
`
`smalt
`
`large
`
`any
`
`any
`
`~Y
`
`low
`high
`
`low
`high
`
`~Y
`
`low
`high
`
`Numeric
`
`field
`
`Slider
`Numeric
`
`field
`
`Slider
`Slider w/tyPs-in
`
`field
`
`Read-only message
`
`Gauge
`Read-only message
`
`*Two-item enumerates whose labels contain opposing terms (e.g., On/Off,
`Tree/False, etc.) should use check boxes.
`
`Tab!e 1 Control selection for integer attributes.
`
`Table 3 Control selection for enurneratedattributes.
`
`In the integer control selection rules (Table 1), the precision
`characteristic is used to determine whether a relative or an
`absolute control
`is used. Relative controls such as sliders or
`(for
`read-only data) gauges are preferred whenever
`the
`precision is low or the range is known and large. We assume
`a range with more than 10 possible values to be large.
`
`The precision characteristic figures less prominently in the
`decision table for real attributes, Because real numbers
`represent continuous variables,
`inherently discrete controls
`such as sliders (whose possible values are limited to the
`number of pixels
`spanned by the slidable
`area) are
`appropriate only when the range is extremely small. A slider
`representing the range 0..10 in hundredth-unit
`increments,
`for example, would span the entire width of
`the typical
`workstation display.
`
`The control selection rules for enumerated attributes (Table
`3) are based on several characteristics:
`l Items - the total number of items in the enumerated set of
`choices. The current rule set distinguishes only between
`enumerations with 9 or fewer choices ~ew) and those with
`10 or more (muny),
`
`l Set Size- whether the number of choice items is fixed
`(static) or allowed to vary at run-time (dynam”c).
`
`- the length of the label for the longest choice item in
`l Label
`the enumeration. Characters are the unit of measure in the
`current rule set, which distinguishes between three label
`lengths: short (0-10 chars), medium (11-50), and long
`(5 l+).
`Ideally,
`the unit of measure should be pixels or
`points, especially if a proportioned font is used for labels.
`
`. Min/Max
`- the minimum and maximum number of
`simultaneous choices that is possible at any time in the
`enumemtion. If the choices are exclusive,
`than exactly one
`item is chosen at all times. A variation
`on exclusive
`enumeration allows zero or one item to be chosen, while a
`
`262
`
`PayPal Inc. v. IOENGINE, LLC
`IPR2019-00886 (US 8,539,047)
`Exhibit 2004
`
`

`

`CHI’92
`
`May3-7, 1992
`
`non-exclusive enumeration allows any number of choices,
`including zero.
`
`The amount of space consumed in the parent window is an
`important criterion for selecting the interaction object for an
`enumerated attribute. The total space occupied by the
`widget
`for an enumeration is a function of the number of
`items and the length of the character strings serving as the
`item Iabeis.
`
`whenever possible, it is desirable to make ail of the choices
`visible at ail times. When the number of choices is relatively
`small and does not vary at run-time,
`a setting of
`the
`appropriate type is preferred. The settings are placed on a
`menu when the number of choices can vary dynamically. A
`menu might also be used - even for a small number of
`choices - when vertical space is at a premium in the parent
`window. When the number of items is large, a seroiiing list
`is aiways used.
`
`Check boxes, for example, are normaily prefemed over non-
`exclusive settings in the current rule set, both because their
`semantics are somewhat mom intuitive and because they are
`less confusable with OPEN LOOK exclusive settings. Check
`boxes, however, do not group well with their labels when
`arranged with more than one item per row. The rule set thus
`prefers nonexclusive settings when the item labels are short
`enough to permit
`three or more settings in the same row.
`When the label is too long to fit within the available qwe, a
`settings menu displays the complete label on demand.
`
`One problem encountered in the development of the control
`selection rules is that any set of attributes
`that can be
`modeled
`as a non-exclusive
`enumerate
`can also be
`expressed as a series of apparently unrelated booleans and
`might be represented
`as such in the data modelling
`environment, Communication is improved by presenting the
`booleans as a set of options in a non-exclusive enumerate.
`The problem is to identify boolean attributes that are truly
`independent of all other attributes
`so that
`they can be
`collected in a single group with a generic, “Options,”
`label.
`
`The selection rules for string attributes (Table 4) are fairly
`straightforward. A single or multi-line
`textfield is chosen
`based on the size of the string represented by the field. The
`length characteristic is the maximum length,
`in characters,
`of the text string. The current
`rule set distinguishes only
`between short (1-50 chars) and long (50+) strings.
`
`Content
`
`Length
`
`Widget
`
`editable
`
`tead-only
`
`short
`long
`
`short
`long
`
`field
`Single line text
`Multi-line
`textfield
`
`Read-only message
`Read-only multi-line
`
`texttield
`
`Table 4 Control selection for string attributes.
`
`data model are realized as
`Actions from the application
`commands appearing in menus in an OPEN LOOK base
`window. The individual menu commands can be of one of
`three types. If
`the action is invoked direetiy by the menu
`item, a comnd
`item is used. If
`the action causes a pop-up
`window to be displayed, a window item is used. If the action
`causes a submenu to be dispiayed, a menu item is used.
`
`The control selection rules produce a set of standard GUI
`components that provides an interfaee to the associated data
`objeet, but these controls must still be arranged within the
`application window before they can be used. The following
`section describes how this task has been automated.
`
`LAYOUT RULES
`
`The output of the control selection rules is fed directly into
`the layout portion of the inference engine. The layout rules
`take a set of interaction objects and a parent window as
`input and produce a spatial arrangement of
`the controls
`the window as output. In the current implementation,
`withh
`interaction objects whose parent is a base window (actions)
`are arranged within a control area at the top of the window.
`Those whose parent
`is a pop-up window (attributes) are
`arranged in an OPEN LOOK command or property window.
`
`The current rule set creates layouts that are consistent with
`the OPENLOOK Application Style Guidelines [13]. Figure 5
`shows a layout generated automatically by the rule set. The
`example window contains two strings, an integer,
`two
`enumerates, and a boolean. (The Apply and Reset buttons
`appear in every OPEN LOOK property window and are
`present by default.)
`
`(9
`
`POPUP Window
`
`Authon
`
`Einstein
`
`6ook : E=MC2
`
`Cove~
`
`Hard Soft
`
`Bookid
`
`~~
`
`Autograph
`
`w
`
`Setting @ School Book
`
`@i!DQ!m
`
`Figure 5 An OPENLOOKcompiiant propetty window
`generated by the system,
`
`A few simpie spacing guidelines help ensure a consistent
`and aesthetically pleasing result. A virtual
`layout grid is
`used to align objects and provide adequate visuai separation
`
`263
`
`PayPal Inc. v. IOENGINE, LLC
`IPR2019-00886 (US 8,539,047)
`Exhibit 2004
`
`

`

`(H1’92
`
`May3-7, 1992
`
`and grouping within the window. The grid is made up of
`regular uni~swhose size depends on the scale of the window,
`which is usually determined by the size of the font used in
`the window’s controls. With the default 12-point font, a 10-
`point grid unit is used. Five basic constraints are sufficient
`to
`satisfy the OPEN LOOK layout requirement.%
`l 1 grid unit vertical space between controls.
`l 1 grid unit Eelween the colon and the left of the control.
`
`e 2 grid unit between groups of controls.
`
`l 2 grid units of space between the edges of the pane and the
`outermost extents of the labels and controls within it.
`
`The current implementation uses a glossaty, shown in Table
`5, of common command names to place each command into
`one of the standard menus. Matching is based on textual
`comparison of the command label. A wild card(*)
`is used to
`match any string
`appearing with the critical
`word.
`Commands that cannot be assigned to one of the standard
`menus are collected in a fifth menu labeled “Other.”
`
`After commands are grouped appropriately in the top-level
`menus, they can be added to the control area of the parent
`window. The system attempts to optimize the use of the
`control area by adjusting the size of the window or the
`.
`
`. Execution buttons centered at the bottom of the window.
`
`File
`
`~tew
`
`Edit
`
`:
`
`I&3&ties”
`
`Controls are arranged in a single column with their boldface
`labels right-aligned
`to the left of the controls themselves
`(see Figure 5). The spacing between controls depends on the
`baseline of the text contained in the control or its label. The
`system automatically
`adjusts the size of
`the window to
`match the amount of space required by the set of controls.
`
`A similar approach is used to arrange the menu commands
`in a series of menus at the top of a base window. Commands
`are assigned to a standard menu location if possible. The
`OPEN LOOK GUI defines four standard menus whose
`contents are standardized along the following
`high-level
`principles:
`l File - contains commands that operate on entire filesystem-
`level objects without directly altering their contents.
`s View - contains commands that control
`the presentation of
`information within the window without affecting the
`actual data itself
`in any way.
`* Edit
`- contains standard, generic commands that can be
`used to alter the data itself, regardless of the manner in
`which the data is displayed.
`
`- contains commands that alter the
`l Properties
`the
`characteristics, or properties, of the application,
`window,
`the document, or the currently selected data.
`
`* Size
`* Yo
`Hide *
`Show *
`
`Find
`
`.
`
`New
`Load
`Save
`Save as
`Browse
`Revert
`Import
`Export
`Print
`Print Optns
`
`Undo
`Again
`cut
`copy
`Paste
`Delete
`Select All
`Clear
`
`Selection
`Objeet
`Font
`Line
`Character
`Paragraph
`Number
`Page
`Document
`Application
`
`Table 5 Menu Glossary.
`
`layout of the menus to take full advantage of the available
`space. If the window is too small to accommodate all of the
`menus, its size is increased as necessary. When the window
`has extra space in the control area, submenus and then
`individual
`items are promoted from the shortest remaining
`menu to the top level until
`the available space is filled.
`
`INTEGRATING THE DATA MODELLING AND USER
`INTERFACE DESIGN ENVIRONMENTS
`is an irtter-
`The OPENLOOK Developer’s Guide (Devguide)
`active user interface design and specification tool horn Sun
`
`Sun Microsystems Developers Guide.
`
`C*MB -1.1
`
`O@#nwt”dLlws Mlwbws
`
`@
`@3@E3 @E)@!@
`EM=in=l @@o-DTe*_
`~mm
`“.SS.” m n
`
`Ewli+nt
`CWxWO
`Psitiori
`(d 1990, s“” ukrmwstc!m,
`
`Sim OOONOOO
`Imc
`
`Poi.lec %
`
`Imlm
`
`;–,
`
`! Q.......=
`
`\
`
`to
`
`and Attributes
`Actions
`are dragged
`from D2M2edit
`an
`Devguide,
`which
`invokes
`inference
`engine
`that automatically
`standard GUI
`selects
`and arranges
`objectsfrom the OPENLOOK GUI.
`6 Devguide and D2M2-Edit.
`
`H9UW
`
`a
`
`<v
`
`‘Lk=R
`
`Om
`
`-
`
`264
`
`D2m2edit
`
`- Object hierarchy esUtor.
`
`.PP,ICA,,II..blml
`
`“Iumrh,
`
`D. flmlI19n
`
`wm. n.!.
`“N.*
`
`“..”...”.!.
`
`“ur”bw
`
`I,-.
`
`.
`
`.
`
`.cIu& lm.
`
`~
`
`MeW,a
`
`“.
`
`rub,.ow
`‘,
`
`3
`
`L
`
`“
`
`-
`
`-@?D@!D@D
`
`@?Ei!9
`t.*..m*1..*mw.-n! B.w ,,,,
`,
`,, P.
`
`,
`
`PayPal Inc. v. IOENGINE, LLC
`IPR2019-00886 (US 8,539,047)
`Exhibit 2004
`
`

`

`v [HI ’92
`
`May3-7, 1992
`
`Microsystems [14]. Devguide’s base window contains a
`graphical palette (see Figure 6) from which standard GUI
`objects can be instantiated and added to a user interface by
`&agging them onto the screen, The designer uses a combi-
`nation of direct manipulation and property sheets to specify
`the attributes of each interaction object. Devguide produces
`a textual description of the resulting user interface in a for-
`mat (Guide Interface Language, or GIL) that serves as input
`to one of several toolkit-specific code generators that create
`user interface source modules for the application.
`
`is the Delft University of Technology’s Direct
`D2M2-Edit
`[2]. It featuresan interactive
`Manipulation Manager Editor
`graphics editor that uses semantic data diagrams to visualize
`the application data model. The data model describes all of
`the data objects used by the application,
`the relations
`between objects (both part-whole
`relations
`and class
`hierarchy relations),
`the available actions on objects,
`the
`pre- and postconditions on the actions, and the attributes, or
`properties, of the data objects. Every data object has an
`associated property sheet in which the attributes, actions and
`pre- and postconditions can be viewed and modified.
`
`To create a user interface directly from the data model, the
`designer selects one or more actions and attributes from the
`D2M2-Edit window and drags them onto one of the windows
`being defined within the Devguide environment or onto the
`base window of Devguide itself
`(see Figure 6). Devguide
`has been experimentally modified to invoke the inference
`engine directly.
`It applies the control selection and layout
`rules to the application data objects and adds an appropriate
`configuration
`of OPEN LOOK controls to the destination
`window,
`If
`the data is dropped onto the Devguide base
`window, a new pop-up window is created to contain the
`automatically generated controls,
`
`Devguide can then be used in the traditional manner to
`modify or refine the automatically generated interface. New
`controls can be added, either from within Devguide or by
`dragging additional objects from the application data model
`in D2M2-Edit onto the target window. To support iterative or
`incremental
`design, controls
`can be added to existing
`interfaces represented as GIL specifications and provided as
`input to the layout component of the design tool. Together
`these tools provide an easy way to generate a “quick and
`dirty” prototype of the application’s
`user interface while
`eliminating much of the work in the final
`implementation.
`
`CONCLUSIONS AND FUTURE WORK
`
`Automatic generation of the user interface directly from an
`application
`data model
`is a logical
`next step in the
`continuing evolution of user interface design tools. The
`ability to formally link the semantics of the application data
`objects to the stylistic conventions and interaction objects of
`a particular GUI environment brings traditional
`software
`engineering
`and user interface
`design one step closer
`together. Automating
`control
`selection and layout can
`increase the quality and consistency of the resulting user
`
`interfaces even in the absence of significant design expertise
`in the target GUI.
`
`layout
`the automatic
`of
`application
`important
`Another
`capability may be seen in object-oriented systems that allow
`instance-specific or user-specific attributes to be associated
`with objects. Objects whose set of attributes can be extended
`or modified by users or by other objects require a run-time
`window layout
`facility since the exact set of attributes and
`values cannot be known until after the design stage. End-
`user customization can also be simplified greatly if users can
`rely on the system to provide an adequate layout as soon as a
`desirable subset of the available features has been identified.
`
`The automatically generated interface designs do not have
`to be perfect to be valuable in any of the above applications.
`The current
`rule set generates usable solutions
`that are
`consistent with the OPEN LOOK GUI style guidelines. This
`nxearch effort demonstrates the feasibility of integrating the
`modelling and design environments
`and establishes the
`foundation
`for a simple run-time control
`selection and
`layout system. There is of course much room for further
`development and extension of the present work.
`
`The most promising areas for future investigation involve
`more extensive use of the semantics available in the form of
`pre- and postconditions in the application data model. This
`information,
`perhaps augmented by additional metadata
`supplied by the application
`designer, could be used to
`support a number of additional design capabilitie~
`
`the
`of controls - in the current implementation,
`l Prioritizing
`order in which controls appear in their parent window
`depends on both the order in which they are selected in the
`data modelling environment and the order in which they
`are introduced into the interface design environment.
`Future versions should be able to identify critical actions
`and attributes - based either on the inherent semantics of
`the data model or on additional metadata provided by the
`application designer - and prioritize layouts accordingly.
`
`- a great deal of information about
`of controls
`l Grouping
`the relatedness of controls could be derived by examining
`the objects they affect and the state of
`the application
`before and/or after they are invoked (as reflected in their
`pre- and post-conditions). The automatic control selection
`and layout technology should be able to identify important
`functional groups and to visually distinguish them within
`the window using additional space, hierarchical
`labeling,
`or appropriate rules and borders.
`l Hierarchical menus - more sophisticated structuring of the
`command hierarchy and resulting menu system should be
`possible based on an analysis of the semantics of the menu
`items as reflected in their pre- and postconditions.
`- all of the attributes
`to windows
`of controls
`l Allocation
`selected in the data modelling environment are currently
`assigned to a