`
`
`
`'r
`‘
`Wséh;a.
`1‘
`
`'
`
`lies?
`
`‘
`J
`a»:
`u '.4»vIsgfi’3€542 wt 11:,
`.1191”
`1
`IE
`* \;'r;r “47"
`\.'
`a-
`I. ,1“, j.‘_‘
`v
`3“:
`,c
`.‘v.
`.‘Iiv
`'
`:2: ,
`:7‘:“‘"-f’* ”l“‘k."
`
`BEJ:C‘n3f‘] .
`
`a {IE—3A“?
`
`VRST2 000
`
`Proceedings of the ACM Symposium on
`Virtual Reality Software and
`Technology
`
`Oct. 22nd ~ 25th , 2000
`Seoul Metropolitan Museum, Seoul, Korea
`
`"
`
`
`
`Korea Information Science Society
`
`Korea Association for Virtual Reality
`
`Edited by Kwang Yun Wohn
`Korea Advanced Institute of Science & Technology, Korea
`
`
`
`Sponsored by ACM SIG—CHI, SIG-GRAPH, Ministry of Information & Communication,
`Institute of Information Technology Assessment (IITA),
`Korea Science Engineering Foundation (KOSEF), Korea Research Foundation (KRF).
`Emmagmorg
`www.mic. gokr
`
`wwwkgseflrekr
`wwwkrfonkr
`
`“ ‘
`
`MEDIATEK, EX. 1022, Page 1
`
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 1
`IPR2018-00101
`
`

`

`
`General Library System
`KGIF w
`University of Wisconsin- Madison
`UDNBrsflyofmmsLigbfag .
`728 State Street
`215 ~_ Randau Ave
`ad
`Madison, WI 53705-1494
`Madison W, my, "“9
`U.S.A.
`
`- r
`
`The Association for Computing Machinery
`1515 Broadway
`New York, New York 10036
`
`Copyright 2000 by the Association for Computing Machinery, Inc. (ACM).
`Permission to make digital' or hard copies of this work for personal or classroom use is granted without fee
`provided that copies are not made or distributed for profit or commercial advantage and that copies bear this
`notice and the full citation on the first page. Copyright for components of. this work owned by others than
`ACM must be honored. Abstracting with credit is permitted.
`
`To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission
`and/or a fee. Request permission to republish from: Publications Dept. ACM, Inc.
`Fax +1 (212) 869-0481 or E-mail permissions@acm.org.
`
`For other copying of articles that carry a code at the bottom of the first or last page, copying is permitted
`provided that the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222
`.4
`Rosewood Drive, Danvers, MA 01923.
`.
`. .
`
`Notice to Past Authors of ACM-Published Articles
`
`ACM intends to create a complete electronic archive of all articles and/or other material previously published
`by ACM. If you have ’written a. work that was previously published by ACM in any journal or conference
`proceedings prior to 1978, or any SIG Newsletter at any tine, and you do NOT want this work to appear in th
`ACM Digital Library, please infor
`permissions@acm.org, stating the title of the work, the author(s), and where and when published.
`
`
`
`ACM ISBN: 1-581
`
`-316-2
`
`Additional copies may be ordered prepaid from
`
`ACM Order Department
`P.O.BOX 11405
`Church Street Station
`New York, NY 10286-1405
`
`’
`
`Phone: 1—800—342—6626
`(USA and Canada)
`+1 -212-626-0500
`(All other countries)
`Fax: +1-212-944-1318
`
`Email: acmhelp@acm.org
`
`ACM Order Number: 429009
`
`Printed in the U.S.A.
`
`IPR2018-00101
`
`
`MEDIATEK, EX. 1022, Page 2
`
`MEDIATEK, Ex. 1022, Page 2
`IPR2018-00101
`
`

`

`
`
`KURT F. WENDT LIBRARY
`COLLEGE OFW6
`
`1 90m
`
`UW-fiADEON. WI 53705
`
`IPR2018-00101
`
`MEDIATEK, EX. 1022, Page 3
`
`MEDIATEK, Ex. 1022, Page 3
`IPR2018-00101
`
`

`

`A Framework for the Structured Design of VR/AR Content
`
`C. Geiger, V. Paelke.
`C. Reimann, W. Rosenbach
`C-LAB
`Fuerstenallee 11
`33102 Paderborn
`+49 +5251 60 60 47
`
`ABSTRACT
`
`{chris,vox,reimann,bobka}@c-lab.de
`designers to have fine control over their creation.
`
`We describe a framework that allows to easily design and
`implement virtual and augmented reality worlds. Based ona
`structured design approach for interactive animated 3D conten
`we want to supply designers and content experts of complex
`virtual environments with a component based to olset for the
`structured design of the visual and abstract components of 3D
`applications.
`
`Keywords
`‘Virtual and Augmented Reality, Design Framework, Interactive
`3D Animation.
`
`1. INTRODUCTION
`Recent advances
`in 3D computer graphics have allowed
`application developers to integrate animated 3D graphics in their
`work without being burdened by low level details. While
`technical advances have laid the foundation for wide spread use
`of interactive 3D [24], there is still a significant lack of design
`experience and ded icated tools that would allow a seamless
`integration of interactive 3D components in current multimedia
`applications. Most often the development of 3D content requires
`an ad—hoc realisation that is implemented from scratch resulting in
`unacceptable development costs.
`
`To address these problems we are concerned with the structured
`design of interactive animated 3D content. An important point is
`the applicability of our approach in a wide range of application
`areas (multimedia / 3D / VR / AR) and hardware/software
`configurations ranging from mobile devices over standard PCs to
`graphics super computers and parallel processing architectures.
`Moreover, we want to support content expert users (i.e. designers)
`and not only technical experts (i.e. 3D programmers): Current
`application areas include interactive 3D illustrations of technical
`systems, animated 3D user interfaces and the visualization of
`simulation results. It is our aim to provide content experts with a
`methodology and a corresponding toolset that supports a high
`level design view of the application and at the same time allows
`
`Based on previous experiences in tool support for interactive 3D
`worlds, e.g.[l9,5], we have developed i4D, a framework for the
`structured design of animated VR/AR content based on reusable
`elements (components) that can be used in a large number of
`applications. Our approach extends a design methodology with
`high level tools for the design of visual 3D components. The
`concepts and tools have been validated in a number of relevant
`application scenarios.
`
`2. DESIGN OF INTERACTIVE VR / AR
`CONTENT
`
`A previous study of 3D development projects in the education
`domain identified a number of ke -problems in the design and
`development of interactive 3D content [2,3]
`
`I
`
`.
`
`I
`
`The lack of an appropriate conceptual model hinders the
`design process and renders the communication between
`stakeholders in the design process difficult. Conceptual
`design tools adopted from conventional media design (e.g.
`storyboards)
`fail
`to address the special requirements of
`viewer interaction and dynamic content.
`
`The lack of a structured design process makes it difficult for
`designers and developers to bridge the large conceptual gap
`between communicative intent and technical implementation.
`The lack of structure also complicates the deve opment of an
`appropriate tool-set.
`
`Existing implementations of visual presentation techniques
`are often application specific and non -portable, hindering
`their reuse, combination and modification.
`
`
`
`I4D Design and Development
`
`Figure 1: Overview Framework
`
`Pemrission to make digital or hard copies of all or part of this work for personal
`or classroom use is granted without fee provided that copies are not made or
`distributed for profit or commercial advantage and that copies bear this notice
`and the full citation on the first page. To copy otherwise, to republish, to post
`on servers or to redistribute to lists, requires prior specific pemrission and/or a
`fee.
`
`VRST 2000 Seoul, Korea
`Copyright 2000 ACM 1-581 13-316-2/00/0010. . .$5 .00
`
`We believe that these problems are not specific to the education
`context and apply to other 3D content as well. The cost proble
`due to the lack of specialized tools and non —portable and non—
`reusable implementations is probably even more critical outside
`the education domain.
`In typical AR/VR scenarios high
`development costs can not be amortized over large numbers of
`distributed instances as in multimedia applications and games.
`
`
`
`MEDIATEK, Ex. 1022, Page 4
`
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 4
`IPR2018-00101
`
`

`

`With i4D we address these problems by providing a conceptua
`model, a structured design approach and support for efficient
`implementation through tools that cover
`the whole design
`process from idea to implementation (see figure 1)
`
`2.1 Design Approach
`The i4D design approach consists of the conceptual model of
`actors, an iterative design process and a componen -based 3D
`animation system:
`.
`
`Conceptual models are used by designers both to reason about
`design problem and to communicate with other stakeholders in
`the design process. Conceptual models in multimedia design are
`commonly borrowed from the visual design domains of print and
`film (books, pages, cards, slides). These models are familiar to al
`stakeholders and are therefore well suited if the content fits the
`constraints of the model (e.g. static input and output). However,
`these models fail to address the dynamic aspects of interactive
`animated 3D applications introduced by dynamic content and
`viewer interaction.
`i4D therefore employs a model based on
`actors. The conceptual model of actors is well suited to dynamic
`design problems in an interdisciplinary context:
`
`An anthropomorphic view of 3D components as actors that act
`out roles is well suited to communicate about the design because
`it provides a common ground of tech niques and metaphors from
`the domains of film and theatre. At the same time, the technical
`concept of actors as parallel entities that achieve goals through
`coordinated actions is powerful enough to handle the dynamic
`aspects of interaction, and thus allows a direct mapping to an i4D
`based implementation.
`
`The design process for interactive 3D content needs to address
`interactivity,
`bridge
`the
`large
`conceptual
`gap between
`communicative intent and technical
`implementation,
`support
`extensive user involvement and handle fuzzy requirements.
`
`14D is based on the approach illustrated in Figure 2 that combines
`scenarios, hierarchical decomposition and iterative prototyping.
`
`Scenario Description
`(Plot, Story)
`
`Scenario Analysis
`(Treatment. Storyboard)
`-
`
`Decomposition Into Actors
`(Scenes.Roles)
`
`.
`
`-
`
`I
`
`,
`
`.
`
`Specification of Actor Behaviour
`(Lines, Commands )
`
`.
`
`.
`
`i;
`
`_,
`
`’
`
`i“
`
`Fig. 2: Design approac
`
`Evaluation
`(Rehearsal )
`
`While the interactive behavior of interactive illustrations is
`
`difficult to capture with sequential and formal specificationsi
`can be described readily as acol lection of representative
`interaction scenarios. In the design process the scenarios are
`progressively refined to define co municative goals of actors and
`
`actions to achieve them, which can then be mapped to ani4D
`based implementation:
`
`in a
`state the communicative intent
`Scenario descriptions
`narrative form. The scenarios
`should provide representative
`examples of the 3D content, how users interact with it and what
`results from this interaction.
`
`Scenario analysis refines the scenario sketches into structured
`situation descriptions that detail interaction and animation content.
`The designer uses these descriptions to identify relevant actors
`and actions.
`
`Decomposition analyses the role of each relevant actor ina
`scenario. The resulting actions
`and actors haveadirect
`correspondence to the technical imple mentation: Each actor
`controls the corresponding geometry and behavior in the 3D
`environment.
`
`Behavior specification formalizes actor behavior. Two cases can
`be distinguished: Behavior that is specific to an actor and has not
`been encountered before needs to be refined into a new action.
`For common (reoccurring) behavior it is sufficient to specif
`which matching standard action from the library should be used.
`In the following section we describe our component based library
`that allows to implement and manage such visual actors and their
`actions.
`
`3. A COMPONENT-BASED 3D LIBRARY
`For the development of our 3D animation library for virtual and
`augmented environments we have set a number of requirements:
`
`I
`
`I
`
`I
`
`I
`
`actor based metaphor supporting a high level design view,
`
`extendible by components (even third-party),
`
`from operating systems
`independence
`graphics library,
`
`and underlying
`
`efiicient and easy design by providing a programming API,
`a scripting interface and visual tools.
`
`3.1 Actor Based Metaphor
`Following an actor based metaphor that matches the design
`process, an i4D application consists of a number of actors placed
`on stages. Actors include conceptual objects like lights and
`cameras, visual 3D objects and even software elements without a
`visual representation. Actors perform actions (e.g. animations,
`sending messages,
`sound) by continuously modifying their
`attributes. Conceptually a stage is viewed by a number of cameras
`and displayed on monitors which themselves are actor objects.
`The assignment of monitors, cameras. stages and actors can be
`altered interactively during runtime. Similar to scene graphs,
`actors can be hierarchically structured. As a central difference to
`traditional
`scene
`graph APIs,
`attributes
`(e.g.
`color,
`transformation) are integral parts of i4D actors. To ensure
`localit lwhen executing complex animations, a special type of
`component called High Level Attribute (HLA) was introduced
`into i4D that allows to encapsulate the dependencies of attributes
`in a local component. For example, a virtual spider walk cycle
`can 'be animated by constructing a HLA WalkPosition.
`Continuously setting the X/Y/Z-value of this attribute results in
`
`4
`
`1 Often a problem in scene graph based approaches like Java3D,
`Performer, Open Intentor,
`
`76
`
`MEDIATEK, Ex. 1022, Page 5
`
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 5
`IPR2018-00101
`
`

`

`setting the orientation of each leg including the relevant joints and
`links of upper and lower parts.
`
`Components for continuously modifying attributes of actors are
`called 14D actions. Actions can be started and stopped and can be
`executed in parallel even if two parallel actions modify the same
`attribute of an actor. [40 Reactions are special components
`having a functionality lying between actions and HLAs. While
`actions are used for time consuming functions that modif
`attribute values and HLAs realize a non time consuming complex
`mapping from an attribute value to a number of different attribute
`values, reactions are used to observe the modification ofan
`attribute value and react
`to this in a specified way.
`14D
`interactions are used for mapping input device parameters to an
`actor's attribute values.
`'
`
`The combination of actions, reaction and HLAs makes it often
`possible to create complex behaviors by composing a smal
`number of predefined elements. For example, a keyframing
`animation Cthec3fAbs (Change 3D vector absolutely)
`that
`animates the WalkPosirion attribute from the start position to a
`new end position with a given speed or time will animate the
`virtual spider. Once this action has been realized, it can be
`assigned to all attributes of type 3D float vector (e.g. color, size,
`position). A library of such simple components is provided in i4D
`including a LookAt reaction that makes an object always look at
`another object or a FollowActor reaction that always follows
`other actors. Because actors are not
`limited to 3D geometr
`objects, but also in lude conceptual objects like cameras or lights
`such actions allow to build complex camera movements or
`moving spotlights using the same building blocks. Through the
`HLA concept these behaviors can be extended to create intelligen
`virtual cameras based on cinematographic knowledge or smart
`spot lights that always highlight important scene features.
`
`For the interactive illustration of spatial hierarchies of objects (e.g.
`a car engine, PC motherboard), a component ExplosionView has
`been designed. This component
`is initialized with an object
`hierarchy and a list of vectors indicating the direction and scaling
`values
`for an exploded representation. Collision detection
`between exploded parts can be switched on. This component is
`realized as HLA that takes the degree of explosion (0-1) as input
`value and arranges the object parts accordingly. Anexplosion
`animation is realized simply by attaching a CthecIfAbs action
`that interpolates between 0 and 1. Adding a LookAt action allows
`to rotate important objects parts toward the camera.
`
`New components can be easily designed and implemented using a
`standard template. An assistant
`for creating components is
`currently under development.
`
`3.2 Architecture
`
`Figure 3 illustrates the architecture of the i4D system. The central
`unit of this
`system is
`the management component
`that
`is
`responsible for the organization and execution of the different i4D
`components. Main features of this unit include
`
`.
`
`Loading and registering
`
`14D components can be easily loaded into the system while
`the registration is performed automatically. Loading and
`unloading of elements is possible using binary or scripted
`components and it is possible to load components even at
`run—time using a mechanism similar to the “factory" design
`pattern known from component based programming [21]
`
`'
`
`Communication managemen
`
`Communication between components is realized usinga
`dedicate message interface that allows each registered entity
`to process
`structured messages based on strings. If a
`component receives a message request it checks if it can
`process the message, i.e., it has services that fulfill parts of
`the request. Parts of the message that cannot be served are
`forwarded to sub-components or other components in the
`i4D-scene that can handle them. Broadcast and multicast
`messages are also supported. If a component cannot process
`a message request and does not know an appropriate
`component for forwarding it ignores the request or raises an
`exception.
`
`I
`
`Execution of Components
`
`For an efficient and flexible management of the execution of
`components we provide a virtual time component that is
`mapped to system time. This allows stretching and squashing
`of time and realizes effects like time slow and fast motion.
`By stretching the virtual time a user defined frame rate can
`be ensured independent from the scene's complexity and
`allows to implement a screen shot action that renders frames
`into bitmaps and movies. Components have been designed
`to act as independent as possible, e.g. there is no explicit
`"connect"-mechanism that gives an actor access to another
`actor's attributes. Each component can access all public
`attributes from other components if access is granted.
`
`
`I
`Mistral Tools
`Extenslon
`
`Ewurnmmlm
`Interface
`
`,.
`
`
`
`
`
`Fig. 3: i4D Structure
`
`3.3 Extendibility
`The i4D system was designed to work seamlessly with a number
`of different
`systems
`(e.g.
`simulation systems, AI based
`approaches like multi agent systems [22] even across networks).
`Thus it was important to develop a framework that allows to
`manage different components in a flexible way and to provide a
`structured and open interface
`for
`external
`(third party)
`components. For our projects we are currently building complex
`components for animated textures, import of different 3D models,
`physical simulation [6] and augmented reality toolkits [7]. For an
`easy reuse in other projects i4D worlds can be saved in a X ML
`compliant format and can be loaded from any i4D application.
`For
`the integration in existing (Microsoft) applications we
`developed a prototype of an ActiveX wrapper for i4D and
`
`77
`
`
`
`MEDIATEK, EX. 1022, Page 6
`
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 6
`IPR2018-00101
`
`

`

`successfully extended a Macromedia Director Application with
`animated 3D content (see example 1).
`
`3.4 Independence
`Due to the large number of combinations of operating systems
`(Irix, Solaris, Linux. Windows 98/NT/2000)
`and graphics
`libraries (OpenGL, Optimizer, Inventor, Performer, Direct3D,
`XSG...) we decided to build i4D independent from a given
`combination and designed i4D to consist of two major parts (see
`fig 1), the main part (80%) consists of a platform / graphics
`library independent management system called I-layer. The other
`layer IO-layer encapsulates platform and library specific code. As
`the IO-layer also consists of components for operating system and
`graphics library which can be loaded and unloaded, the adaption
`to new configurations requires only the modification of the
`corresponding IO-components. The current i4D pro otype runs on
`Win NT and Irix using Open GL Optimizer. A component for
`pure OpenGL has been completed recently and a Performer pot
`for Linux / Irix and a Direct3D component are under development.
`
`3.5 Scripting vs. Programming vs. Tools
`Many scene graph API’s offer an efficient but less intuitive C++
`API. For the design of interactive virtual worlds we found the
`technique of rapid prototyping extremely useful [19]. To suppor
`this, a script binding (e.g. for Tcl/I'k, Python, Perl) is extremely
`useful to allow an interactive design of the virtual environment
`without compiling. Thus, we developed a scripting interface for
`i4D that allows to integrate i4D in arbitrary scripting languages.
`So far, we use Tcl/Tk and completely integrated our
`i4D
`prototype in this
`language. Based on this
`integration th
`development of visual tools for the design of i4D worlds is also
`very efficient.
`I4D-Tools under development include a gesture
`editor for i4D avatars and an editor for i4D scenes (see Figure 4).
`The gesture editor is based on VRML / EAI and allows the direct
`manipulation of ke -frame animations of an avatar's body parts
`that follows a given bone structure similar to the H —ANIM
`standard of VRML avatars. The resulting interpolation values are
`then transformed to an i4D action. The scene editor is based on
`Tcl/Tk and allows a visual development of all i4D elements b
`providing special editor forms for each component
`type and
`combining them in configuration form (see figure 4) . The editor
`also follows a component based design approach. Sub-editors for
`individual i4D components can be loaded and unloaded at run
`time similar to the i4D elements. A component developer only has
`to provide his special component and a corresponding sub —editor
`and the designer can use and visually manipulate the component
`instantly. If no sub—editor is available the most appropriate editor
`in the component's inheritance hierarchy will be used.
`
`4. EXAMPLES
`We have implemented a number of demos that validate special
`features of i4D. It is beyond the scope of this paper to present a
`detailed description of the actor—based development process and
`the
`results.
`In the
`following section we
`focus on the
`implementation aspects and only sketch design-related aspects.
`
`4.1 Multimedia Example
`One of the first “real-life” applications illustrated the control of a
`pathfinder robot
`[4]
`that can be remote controlled over the
`
`
`
`
`
`
`
`
`
`Fig. 4: i4D Tools
`
`Internet. With the simulated robot users can perform the same
`actions as with the real robot. The illustration combines an
`introductory video with an interactive 3D illustration. After the
`pathfinder robot
`is assembled from its components users can
`control the simulated pathfinder robot and experiment with it in a
`simple environment. The application was conceptually designed
`by defining a number of scenarios:
`
`0
`0
`
`o
`
`0
`
`a non interactive presentation of the pathfinder construction
`control of the robot and selection of camera perspectives
`
`interactive exploration of the environment
`
`solving a simple task (uncaging a flying fish)
`
`Following our design approach, a number of actors (e.g. user,
`robot, fish, spider) were defined that have goals (e.g. explore
`environment,
`free fish, avoid collisions) which resulted in
`different behaviors by performing a number of actions.
`
`78
`
`MEDIATEK, Ex. 1022, Page 7
`
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 7
`IPR2018-00101
`
`

`

`that encapsulates an i4D
`developed an ActiveX component
`illustration. Messages from/to Director sent to/from this ActiveX
`component can be simply transformed to/from i4D messages.
`This follows a similar approach as used in the script binding.
`
`4.2 Physical Simulation and Soft Computing
`As a complex example that validates the reuse of e xisting
`components we have recently integrated the mathengine physical
`simulation library [7] in i4D. This component can be loaded into
`i4D at runtime and allows to apply physical attributes like mass,
`force, torque to i4d objects and animate position and orientation
`according to the physical properties of the elements. Access to
`the mathengine is possible via the i4D scripting interface or the
`C++-programming interface. Using the mathengine we are
`currently developing a smart virtual 3D environment for the
`evolution of “i4D-creatures”. This example is based on the well
`known work of Karl Sims who developed his block creatures tha
`evolved in a physical environment using a genetic algorithm [23].
`The results of the evolution (i.e. the walk cycle of acreatu re)
`were rendered as
`films and presented at
`the SIGGRAPH
`conference in 1994. For our example we are rebuilding this
`experiment in our interactive 3D graphics environment using
`special components for physical simulation and the definition of
`the internal creature structure.
`
`
`
`
`update queue
`
`Torque
`Effector
`
`Torque
`Effector
`
`
`
`Figure 5: i4D Multimedia Application
`
`The resulting application features a 3D video sequence that shows
`the construction of a virtual pathfinder. After completion the
`interactive i4D application is started that allows to controla
`virtual pathfinder in a simple 3D environment. The interface is the
`same as for the real pathfinder. The user can solve a simple task
`in this environment: If the pathfinder presses a secret button
`hidden behind a big block, a creature (a flying fish) is uncaged
`from its cage and will follow the pathfinder using a follow—actor
`action. The world is also inhabited by a virtual spider that moves
`randomly around. A simple collision detection was realized b
`constraining the position attribute of the pathfinder, fish and
`spider to a given interval. This feature is inherent to numerical
`attributes of i4D actors. The current prototype of this application
`also features sound and music.
`
`
`
`Fig. 6: i4D Creature World
`
`The control via web form was illustrated in a multimedia
`application using the Macromedia Director software package. We
`1
`
`Angle
`Sensor
`
`Angle
`Sensor
`
`Fig. 7: Brain and Body Structure of an i4D Creatures
`
`A creature basically consists of a body component, that describes
`its visual appearance including body parts, joints, sensor and
`effectors (figure 2). A brain component defines how sensor inpu
`values are processed and forwarded to the effectors. This
`component features a neural net
`like structure that connects
`sensor and effector attributes (see Figure 7). A simple behavior is
`illustrated in Figure 6. The resulting walk cycle makes the
`
`
`
`79
`
`'MEDIATEK, Ex. 1022, Page 8
`
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 8
`IPR2018-00101
`
`

`

`creature move forward by sending the values of ale g to its
`counterpart which uses them in the next processing cycle. The
`walk angle of the left (forward) leg is inverted and sent to the
`right ( backward) leg. The definition of a creature's structure and
`behavior can be described using a special data structure that can
`be easily manipulated (by the user or a genetic~algorithm)
`allowing to design many different creatures.
`In the fina
`application a number of different creatures will evolve and
`interact (predator and prey, mating, etc). For highlighting and
`presenting interesting aspects of the simulation “intelligent
`cameras” and “smart lights” will be developed that always follow
`a creature or switch the camera perspective if an interesting
`situation occurs (e.g. a fight between two creatures happens).
`
`4.3 Interactive Augmented Reality
`Illustrations
`The most recent application integrates the augmented realit
`toolkit ARToolkit from Hitlab [7], University of Washington in
`i4D. The real scene is augmented with a number of virtual 3D
`objects. The composition of rea 1 and virtual scene elements is
`realized by the AR toolkit. that recognizes pre -def1ned patterns
`and their position / orientation. The integration in i4D allows the
`efficient design of complex animated 3D content in real scenes.
`In combination with existing components we aim at providing a
`high level
`support
`for
`the design of augmented realit
`illustrations that can be used for interactive explanations of rea
`artefacts
`(e.g.
`in museums),
`technical devices
`(e.g. for
`maintenance) and wayfinding (e.g. navigational aids).
`
`4.3.1 Interactive 3D Illustrations
`User guidance in interactive 3D illustrations is providedb
`augmenting 3D simulations with interactive illustration tech -
`niques inspired by traditional
`(print)
`illustration techniques.
`Traditional illustration techniques like annotations, meta-objects
`and object
`substitution modify or augment
`the graphical
`representation of the illustrated subject. They focus attention,
`point out relevant detail and provide additional information —
`guiding
`a viewer
`through
`graphical
`content.
`Illustration
`techniques thus provide a rich set of guidance techniques that can
`be added to arbitrary graphical content. Adapting illustration
`techniques to 3D virtual environments makes it possible to
`provide users with visual guidance in a fa iliar way while
`exploiting
`the
`additional
`possibilities
`of
`animation and
`interactivity
`in
`virtual
`environments. Because
`illustration
`techniques are generally context
`independent
`their interactive
`counterparts can be implemented as reusable components that can
`be applied to arbitrary content,
`thereby reducing developmen
`cost.
`
`4.3.2 Interactive Illustration Techniques:
`The i4D system provides a library if illustration techniques as
`components that derive and update a graphical representation and
`its parameters depending on the initial configuration speci fied b
`the designer, interaction by the user and changes in associated
`virtual environment objects. Three categories of interactive
`illustration techniques are supported in i4D:
`
`.
`
`.
`
`Interactive illustration techniques that introduce additional
`(meta) objects
`
`Interactive illustration techniques
`objects.
`
`that modify existing
`
`'
`
`Interactive illustration techniques that modify conceptual
`objects
`’
`
`4.3.3 Example of interactive AR illustration
`Figure 8 shows a simple example of illustration techniques
`currently under development. This application will illustrate the
`features of a new laptop and explains the handling of the device
`(e.g. features of new card). The application is designed as a
`highly interactive illustration and provides a virtual 3D presenter
`as part of the user interface that explains important features of the
`product based on the users interactions (e.g. opening the laptop,
`showing interest in a feature, etc). Based on different scenarios
`(Eyecatcher / presentation a a fair, Guided Tour on a Give-Awa
`CD,
`Interactive Manual)
`the behavior
`specification of the
`identified actors results in a number of illustration techniques that
`are used in this application.
`
`Examples of the first illustration category (Interactive illustration
`techniques that
`introduce additional (meta -) objects) include
`arrows and pointers that show important features of the product,
`e.g. the slot for inserting PCM/CIA cards. Examples of the second
`categor
`(Interactive illustration techniques that modifi’ existing
`objects are visible in the highlighting of important regions the
`virtual pr’esenter points to. The explosion animation describedin
`section 3.1 is another illustration technique and explains the
`different parts of the internal modem card. Illustration techniques
`in i4D used in this example are not limited to simple presentation
`techniques, as the example of a virtual animated presenter in
`figure 2 illustrates. The presenter is a fully animated 3D character
`that can use gestures, animation and audio to present objects and
`properties. The presentation an mations are composed from basic
`gestures that can be defined in the special editor shown in figure 4.
`
`5. Related work
`A number of relevant projects have influenced this work although
`we do not know of a similar approach that covers all aspects of
`our framework and the intended application of i4D.
`
`Because the development of interactive 3D content is still a ver
`diverse and immature area with few established rules in the sense
`of design principles or techniques i4D is based on a design
`approach that integrates techniques from multimedia and user
`interface design,
`that support ex ploratory and interdisciplinary
`design. This is in contrast to intelli-media applications for more
`established areas of communication design (e.g.WIP/PPP[1]) that
`have experimented with automated design techniques.
`
`The conceptual model of actors can be compared to the actor
`metaphor in multimedia authoring systems (eg. Macromedia
`Director[9]) but extends this in the in the sense of Laurel’s [8]
`computers
`as
`theatre
`to
`characterize
`all
`components as
`independent actors that collaborate to create a performance. This
`mean