`
`
`
`edSbEe TY teh
`ic wang ost
`.
`“att ae
`foeam) voa
`'
`Fmbs |reedMeaa ath
`r manP*oth, 2 gee
`owt
`SS
`ae, Syed ort wideNh
`
`wee
`
`*)
`
`tJTORN“$ pian
`
`VRST2000
`
`Proceedings of the ACM Symposium on
`Virtual Reality Software and
`Technology
`
`Oct. 22" ~ 25", 2000
`Seoul Metropolitan Museum, Seoul, Korea
`
`”
`
`Korea Information Science Society
`
`Korea Association for Virtual Reality
`
`Edited by Kwang Yun Wohn
`Korea Advanced Institute of Science & Technology, Korea
`
`
`
`Sponsored by ACM SIG-CHI, SIG-GRAPH,Ministry of Information & Communication,
`Institute of Information Technology Assessment(IITA),
`
`Korea Science Engineering Foundation (KOSEF), Korea Research Foundation(SRE).
`
`www.acm.org
`www.mic.go.kr
`
`www.kosef.re.kr
`www.krf.or.kr
`
`MEDIATEK,Ex. 1022, Page 1
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 1
`IPR2018-00101
`
`

`

`
`General Library System
`University of Wisconsin - Madison
`728 State Street
`Madison, WI 53706-1494
`U.S.A.
`
`|
`
`KurtF.Wendt Library
`Unversity of
`215 N..Rengensin “ Madison
`Madison wi sogeeUe
`
`a
`
`The Association for Computing Machinery
`1515 Broadway
`New York, New York 10036
`
`.
`
`Copyright 2000 by the Assoqiation for Computing Machinery, Inc. (ACM).
`Permission to makedigital’ or hard copies of this work for personal or classroom use is granted without fee
`provided that copies are not madeor distributed for profit or commercial advantage and that copies bear this
`notice and the full citation on the first page. Copyright for components of this work ownedby others than
`ACM mustbe honored. Abstracting with credit is permitted.
`
`-
`
`To copy otherwise, to republish, to post on servers orto redistribute to lists, requires prior specific permission
`and/or a fee. Request permission to republish from: Publications Dept. ACM,Inc.
`Fax +1 (212) 869-0481 or E-mail permissions @acm.org.
`
`For other copying of articles that carry a code at the bottom ofthe first or last page, copying is permitted
`provided that the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222
`Rosewood Drive, Danvers, MA 01923.
`Lt
`
`Notice to Past Authors of ACM-Published Articles
`ACMintends to create a complete electronic archive of all articles and/or other material previously published
`by ACM.If you have-written a. work that was previously published by ACM in any journal or conference
`proceedingsprior to 1978, or any SIG Newsletter at any tine, and you do NOT want this work to appear in th
`ACMDigital Library, please infor
`permissions @acm.org,stating the title of the work, the author(s), and where and when published.
`
`ACM ISBN: 1-581
`
`-316-2
`
`Additional copies may be ordered prepaid from
`
`ACM Order Department
`P.O.BOX 11405
`ChurchStreet Station
`New York, NY 10286-1405
`
`~
`
`Phone: 1-800-342-6626
`(U.S.A and Canada)
`+1-212-626-0500
`(All other countries)
`Fax: +1-212-944-1318
`E-mail: acmhelp @ acm.org
`
`ACM Order Number: 429009
`
`Printed in the U.S.A.
`
`| |
`
`|
`
`
`
`MEDIATEK,Ex. 1022, Page 2
`IPR2018-00101
`
`
`MEDIATEK, Ex. 1022, Page 2
`IPR2018-00101
`
`

`

`
`
`KURT F. WENDT LIBRARY
`COLLEGE OF
`G
`
`1 7008
`
`UW-MADISON, WI 53706
`
`MEDIATEK,Ex. 1022, Page 3
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 3
`IPR2018-00101
`
`

`

`A Frameworkfor the Structured Design of VR/AR Content
`
`C. Geiger, V. Paelke,
`C. Reimann, W. Rosenbach
`C-LAB
`Fuerstenallee 11
`33102 Paderborn
`+49 +5251 60 60 47
`{chris,vox,reimann,bobka}@ c-lab.de
`designers to have fine control overtheir creation.
`Based on previous experiences in tool support for interactive 3D
`worlds, e.g.[19,5], we have developed i4D, a framework for the
`structured design of animated VR/AR content based on reusable
`elements (components) that can be used in a large numberof
`applications. Our approach extends a design methodology with
`high level tools for the design of visual 3D components. The
`concepts and tools have been validated in a numberof relevant
`application scenarios.
`
`ABSTRACT
`We describe a framework that allows to easily design and
`implement virtual and augmented reality worlds. Based ona
`structured design approach for interactive animated 3D conten
`we want to supply designers and content experts of complex
`virtual environments with a component based to olset for the
`structured design of the visual and abstract components of 3D
`applications.
`
`Keywords
`- Virtual and Augmented Reality, Design Framework, Interactive
`3D Animation.
`
`1. INTRODUCTION
`Recent advances
`in 3D computer graphics have allowed
`application developers to integrate animated 3D graphics in their
`work without being burdened by low level details. While
`technical advances havelaid the foundation for wide spread use
`of interactive 3D [24], there is still a significant lack of design
`experience and ded icated tools that would allow a seamless
`integration of interactive 3D components in current multimedia
`applications. Most often the development of 3D content requires
`an ad-hocrealisation that is implemented from scratch resulting in
`unacceptable developmentcosts.
`To address these problems we are concerned with the structured
`design of interactive animated 3D content. An importantpointis
`the applicability of our approach in a wide range of application
`areas (multimedia / 3D / VR / AR) and hardware/software
`configurations ranging from mobile devices over standard PCs to
`graphics super computers and parallel processing architectures.
`Moreover, we wantto support content expert users(i.e, designers)
`and not only technical experts (ie. 3D programmers). Current
`application areas include interactive 3D illustrations of technical
`systems, animated 3D user interfaces and the visualization of
`simulation results. It is our aim to provide content experts with a
`methodology and a corresponding toolset that supports a high
`level design view of the application and at the same time allows
`
`2. DESIGN OF INTERACTIVE VR/ AR
`CONTENT
`A previous study of 3D developmentprojects in the education
`domain identified a number of ke -problems in the design and
`developmentofinteractive 3D content[2,3]
`"
`The lack of an appropriate conceptual model hinders the
`design process and renders the communication between
`stakeholders in the design process difficult. Conceptual
`design tools adopted from conventional media design(e.g.
`storyboards)
`fail
`to address the special requirements of
`viewer interaction and dynamic content.
`The lack ofa structured design process makesit difficult for
`designers and developers to bridge the large conceptual gap
`between communicative intent and technical implementation.
`Thelack of structure also complicates the deve opment of an
`appropriate tool-set.
`Existing implementations of visual presentation tech niques
`are often application specific and non -portable, hindering
`their reuse, combination and modification.
`
`*
`
`»
`
`
`
`14D Design and Development
`
`Figure 1: Overview Framework
`
`Permission to makedigital or hard copiesofall or part of this work for personal
`or classroom use is granted without fee provided that copies are not made or
`distributed for profit or commercial advantage and that copies bear this notice
`and the full citation on the first page. To copy otherwise, to republish, to post
`on servers or to redistribute to lists, requires prior specific permission and/or a
`fee.
`
`VRST2000 Seoul, Korea
`Copyright 2000 ACM 1-58113-316-2/00/0010...$5.00
`
`Webelieve that these problemsare not specific to the education
`context and apply to other 3D content as well. The cost-proble
`due to the lack of specialized tools and non -portable and non -
`reusable implementations is probably evert more critical outside
`the education domain.
`In typical AR/VR scenarios high
`development costs can not be amortized over large numbers of
`distributed instances as in multimedia applications and games.
`
`
`
`75
`
`MEDIATEK,Ex. 1022, Page 4
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 4
`IPR2018-00101
`
`

`

`With i4D we address these problems by providing a conceptua
`model, a structured design approach and support for efficient
`implementation through tools that cover
`the whole design
`process from idea to implementation (see figure 1)
`
`actions to achieve them, which can then be mapped to an i4D
`based implementation:
`ina
`state the communicative intent
`Scenario descriptions
`narrative form. The scenarios
`should provide representative
`examples of the 3D content, how users interact with it and what
`results from this interaction.
`
`Scenario analysis refines the scenario sketches into structured
`situation descriptions that detail interaction and animation content.
`The designer uses these descriptions to identify relevant actors
`and actions.
`
`2.1 Design Approach
`The i4D design approach consists of the conceptual model of
`actors, an iterative design process and a componen -based 3D
`animation system:
`=.
`Conceptual models are used by designers both to reason about
`design problem and to communicate with other stakeholders in
`Decomposition analyses the role of each relevant actor ina
`the design process. Conceptual models in multimedia design are
`
`scenario. The resulting actions and actors haveadirect
`commonly borrowed from the visual design domains ofprint and
`correspondence to the technical imple mentation: Each actor
`film (books, pages, cards, slides). These models are familiar to al
`stakeholders and are therefore well suited if the content fits the
`controls the corresponding geometry and behavior in the 3D
`environment.
`constraints of the model (e.g. static input and output). Ho wever,
`these models fail to address the dynamic aspects of interactive
`animated 3D applications introduced by dynamic content and
`viewer interaction.
`i4D therefore employs a model based on
`actors. The conceptual model of actors is well suited to dynamic
`design problems in an interdisciplinary context:
`An anthropomorphic view of 3D components as actors that act
`out roles is well suited to communicate about the design because
`it provides a common groundof tech niques and metaphors from
`the domainsof film and theatre. At the same time, the technical
`concept of actors as parallel entities that achieve goals through
`coordinated actions is powerful enough to handle the dynamic
`aspects of interaction, and thus allows a direct mapping to an i4D
`based implementation.
`The design process for interactive 3D content needs to address
`interactivity,
`bridge
`the
`large
`conceptual
`gap between
`communicative intent and technical
`implementation,
`support
`extensive user involvement and handle fuzzy requirements.
`14D is based on the approachillustrated in Figure 2 that combines
`scenarios, hierarchical decomposition and iterative prototyping.
`
`Behavior specification formalizes actor behavior. Two cases can
`be distinguished: Behavior that is specific to an actor and has not
`been encountered before needs to be refined into a new action.
`For common (reoccurring) behavior it
`is sufficient to specif
`which matching standard action from the library should be used.
`In the folowing section we describe our componentbasedlibrary
`that allows to implement and manage such visual actors and their
`actions.
`
`ge&
`
`=
`
`3. A COMPONENT-BASED 3D LIBRARY
`For the development of our 3D animation library for virtual and
`augmented environments we have set a numberof requirements:
`"
`actor based metaphor supporting a high level design view,
`=
`extendible by components (even third-party),
`"
`independence
`from operating systems
`and underlying
`graphics library,
`efficient and easy design by providing a programming API,
`a scripting interface and visualtools.
`3.1 Actor Based Metaphor
`Following an actor based metaphor that matches the design
`process, an i4D application consists of a numberof actors placed
`on stages. Actors include conceptual objects like lights and
`cameras, visual 3D objects and even software elements without a
`visual representation. Actors perform actions (e.g. animations,
`sending messages,
`sound) by continuously modifying their
`attributes. Conceptually a stage is viewed by a number of cameras
`and displayed on monitors which themselves are actor objects.
`The assignment of monitors, cameras, stages and actors can be
`altered interactively during runtime. Similar to scene graphs,
`actors can be hierarchically structured. As a central difference to
`traditional
`scene
`graph APIs,
`attributes
`(eg.
`color,
`transformation) are integral parts of i4D actors. To ensure
`localit
`' when executing complex animations, a special type of
`component called High Level Attribute (HLA) was introduced
`into i4D that allows to encapsulate the dependencies ofattributes
`in a local component. For example, a virtual spider walk cycle
`can ‘be
`animated by constructing a HLA WalkPosition.
`Continuously setting the X/Y/Z-value of this attribute results in
`
`' Often a problem in scene graph based approacheslike Java3D,
`Performer, Open Intentor,
`
`76
`
`MEDIATEK,Ex. 1022, Page 5
`IPR2018-00101
`
`Fig. 2: Design approac
`
`Scenario Description
`(Plot, Story)
`
`Scenario Analysis
`(Treatment, Storyboard )
`
`Specification of Actor Behaviour
`(Lines, Commands }
`
`Evaluation
`(Rehearsal)
`
`While the interactive behavior of interactive illustrationsis
`difficult to capture with sequential and formal specificationsi
`can be described readily as acol lection of representative
`interaction scenarios. In the design process the scenarios are
`progressively refined to define co municative goals of actors and
`
`MEDIATEK, Ex. 1022, Page 5
`IPR2018-00101
`
`

`

`setting the orientation of each leg including the relevantjoints and
`links of upper and lowerparts.
`Components for continuously modifying attributes of actors are
`called /4D actions. Actions can bestarted and stopped and can be
`executed in parallel even if two parallel actions modify the same
`attribute of an actor. 14D Reactions are special components
`having a functionality lying between actions and HLAs. While
`actions are used for time consuming functions that modif
`attribute values and HLAsrealize a non time consuming complex
`mapping from an attribute value to a numberofdifferent attribute
`values, reactions are used to observe the modification of an
`attribute value and react
`to this in a specified way.
`14D
`interactions are used for mapping input device parameters to an
`actor's attribute values.
`
`The combination of actions, reaction and HLAs makesit often
`possible to create complex behaviors by composing a smal
`number of predefined elements. For example, a keyframing
`animation ChgVec3fAbs
`(Change 3D vector absolutely)
`that
`animates the WalkPosition attribute from the start position to a
`new end position with a given speed or time will animate the
`virtual spider. Once this action has been realized, it can be
`assignedto all attributes of type 3D float vector(e.g. color, size,
`position). A library of such simple componentsis provided in i4D
`including a LookAt reaction that makes an object always look at
`another object or a FollowActor reaction that always follows
`other actors. Because actors are not
`limited to 3D geometr
`objects, but also in lude conceptual objects like camerasorlights
`such actions allow to build complex camera movements or
`moving spotlights using the same building blocks. Through the
`HLA concept these behaviors can be extendedto create intelligen
`virtual cameras based on cinematographic knowledge or smart
`spot lights that always highlight importantscene features.
`Forthe interactiveillustration of spatial hierarchies of objects(e.g.
`a car engine, PC motherboard), a component ExplosionView has
`been designed. This component
`is initialized with an object
`hierarchy anda list of vectors indicating the direction and scaling
`values
`for an exploded representation. Collision detection
`between exploded parts can be switched on. This componentis
`realized as HLA that takes the degree of explosion (0-1) as input
`value and arranges the object parts accordingly. An explosion
`animation is realized simply by attaching a ChgVec1fAbs action
`that interpolates between 0 and 1. Adding a LookAtaction allows
`to rotate important objects parts toward the camera.
`New components can beeasily designed and implemented using a
`standard template. An assistant
`for creating components is
`currently under development.
`
`3.2 Architecture
`Figure3 illustrates the architecture of the 14D system. The central
`unit of this
`system is
`the management component
`that
`is
`responsible for the organization and executionofthe different i4D
`components. Main featuresof this unit include
`
`*
`
`Loading and registering
`14D components can be easily loaded into the system while
`the registration is performed automatically. Loading and
`unloading of elements is possible using binary or scripted
`components and it is possible to load components even at
`run-time using a mechanism similar to the "factory" design
`pattern known from componentbased programming [21]
`
`"Communication managemen
`Communication between components is realized using a
`dedicate message interface that allows each registered entity
`to process
`structured messages based on strings. Ifa
`component receives a message request it checks if it can
`process the message,i.e., it has services that fulfill parts of
`the request. Parts of the message that cannotbe served are
`forwarded to sub-components or other. components in the
`i4D-scene that can handle them. Broadcast and multicast
`messagesare also supported. If a component cannotprocess
`a message request and does not know an appropriate
`component for forwarding it ignores the requestor raises an
`exception.
`Execution of Components
`For an efficient and flexible managementof the execution of
`components we provide a virtual time component thatis
`mapped to system time. This allows stretching and squashing
`of time and realizes effects like time slow and fast motion.
`By stretching the virtual time a user defined frame rate can
`be ensured independent from the scene's complexity and
`allows to implement a screen shotaction that renders frames
`into bitmaps and movies. Components have been designed
`to act as independent as possible, e.g. there is no explicit
`"connect"-mechanism that gives an actor access to another
`actor's attributes. Each component can accessall public
`attributes from other componentsif access is granted.
`
`*
`
`|
`Visual Tools
`
`Extension
`Bro:
`
`
`
`interface
`
`Sis] sense
`
`Fig. 3: i4D Structure
`
`3.3 Extendibility
`The i4D system was designed to work seamlessly with a number
`of different
`systems
`(e.g.
`simulation systems, AI based
`approaches like multi agent systems [22] even across networks).
`Thus it was important to develop a framework that allowsto
`manage different components in a flexible way and to provide a
`structured and open interface
`for
`external
`(third party)
`components. For our projects we are currently building complex
`components for animated textures, import of different 3D models,
`physical simulation [6] and augmentedreality toolkits [7]. For an
`easy reuse in other projects i4D worlds can be saved ina X ML
`compliant format and can be loaded from any i4D application.
`For
`the integration in existing (Microsoft) applications we
`developed a prototype of an ActiveX wrapper for i4D and
`
`
`
`77
`
`MEDIATEK,Ex. 1022, Page 6
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 6
`IPR2018-00101
`
`

`

`successfully extended a Macromedia Director Application with
`animated 3D content (see example1).
`
`
`
`
`
`
`3.4 Independence
`Due to the large number of combinations of operating systems
`
`(Irix, Solaris, Linux, Windows 98/NT/2000) and_graphics
`libraries (OpenGL, Optimizer, Inventor, Performer, Direct3D,
`XSG...) we decided to build i4D independent from a given
`combination and designed i4D to consist of two majorparts (see
`fig 1), the main part (80%) consists of a platform / graphics
`library independent management system called J-layer. The other
`layer J0-layer encapsulates platform andlibrary specific code. As
`the /0-layer also consists of components for operating system and
`graphics library which can be loaded and unloaded, the adaption
`to new configurations requires only the modification of the
`corresponding J0-components. The current i4D pro otype runs on
`Win NT and Irix using Open GL Optimizer. A componentfor
`pure OpenGL has been completed recently and a Performer por
`for Linux/ Irix and a Direct3D component are under development.
`3.5 Scripting vs. Programmingvs. Tools
`Manyscene graph API’s offer an efficient but less intuitive C++
`API. For the design of interactive virtual worlds we found the
`technique of rapid prototyping extremely useful [19]. To suppor
`this, a script binding (e.g. for Tcl/Tk, Python,Perl) is extremely
`useful to allow an interactive design of the virtual environment
`without compiling. Thus, we developed a scripting interface for
`i4D that allows to integrate i4D in arbitrary scripting languages.
`So far, we use Tcl/Tk and completely integrated our
`i4D
`prototype in this
`language. Based on this
`integration th
`developmentof visual tools for the design of 14D worlds is also
`very efficient.
`I4D-Tools under development include a gesture
`editor for i4D avatars and aneditor for i4D scenes (see Figure 4).
`The gesture editor is based on VRML / EAI and allows the direct
`manipulation of ke -frame animations of an avatar's body parts
`that follows a given bone structure similar to the H -ANIM
`standard of VRML avatars. The resulting interpolation values are
`then transformed to an i4D action. The scene editor is based on
`Tcl/Tk and allows a visual developmentof all i4D elements b
`providing special editor forms for each component
`type and
`combining them in configuration form (see figure 4). The editor
`also follows a componentbased design approach. Sub-editors for
`individual i4D components can be loaded and unloaded at run
`time similar to the i4D elements. A componentdeveloperonly has
`to provide his special component and a corresponding sub-editor
`and the designer can use and visually manipulate the component
`instantly. If no sub-editor is available the most appropriate editor
`in the component's inheritance hierarchywill be used.
`
`
`
`Fig. 4: i4D Tools
`
`Internet. With the simulated robot users can perform the same
`actions as with the real robot. The illustration combines an
`introductory video with an interactive 3D illustration. After the
`pathfinder robot
`is assembled from its components users can
`control the simulated pathfinder robot and experiment with it in a
`simple environment. The application was conceptually designed
`by defining a numberofscenarios:
`e
`anoninteractive presentation of the pathfinder construction
`e
`control of the robot and selection of camera perspectives
`
`4. EXAMPLES
`Wehave implemented a number of demos that validate special
`e_interactive exploration of the environment
`features of i4D. It is beyond the scope of this paper to present a
`e
`solving a simple task (uncaging a flying fish)
`detailed description of the actor-based development process and
`the
`results.
`In the
`following section we
`focus on the
`Following our design approach, a numberof actors (e.g. user,
`implementation aspects and only sketch design-related aspects.
`robot, fish, spider) were defined that have goals (e.g. explore
`environment,
`free fish, avoid collisions) which resulted in
`different behaviors by performing a numberofactions.
`
`4.1 Multimedia Example
`Oneofthefirst “real-life” applicationsillustrated the controlof a
`pathfinder robot
`[4]
`that can be remote controlled over the
`
`78
`
`MEDIATEK,Ex. 1022, Page 7
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 7
`IPR2018-00101
`
`

`

`
`
`Figure 5: i4D Multimedia Application
`
`that encapsulates an i4D
`developed an ActiveX component
`illustration. Messages from/to Director sent to/from this ActiveX
`component can be simply transformed to/from i4D messages.
`This follows a similar approachas usedin the script binding.
`4.2 Physical Simulation and Soft Computing
`As a complex example that validates the reuse ofexisting
`components we haverecently integrated the mathengine physical
`simulation library [7] in i4D. This componentcan be loaded into
`i4Dat runtime and allowsto apply physical attributes like mass,
`force, torque to i4d objects and animate position and orientation
`according to the physical properties of the elements. Access to
`the mathengine is possible via the i4D scripting interface or the
`C++-programming interface. Using the mathengine we are
`currently developing a smart virtual 3D environment for the
`evolution of “i4D-creatures”. This example is based on the well
`known work of Karl Sims who developed his block creatures tha
`evolvedin a physical environmentusing a genetic algorithm [23].
`The results of the evolution (i.e. the walk cycle of a creatu re)
`were rendered as
`films and presented at
`the SIGGRAPH
`conference in 1994. For our example we are rebuilding this
`experiment in our interactive 3D graphics environment using
`special components for physical simulation andthe definition of
`the internal creature structure.
`
`The resulting application features a 3D video sequence that shows
`the construction of a virtual pathfinder. After completion the
`interactive i4D application is started that allows to control a
`virtual pathfinder in a simple 3D environment. Theinterface is the
`same as for the real pathfinder. The user can solve a simple task
`in this environment: If the pathfinder presses a secret button
`hidden behindabig block, a creature (a flying fish) is uncaged
`from its cage and will follow the pathfinder using a follow-actor
`action. The world is also inhabited by a virtual spider that moves
`randomly around. A simple collision detection was realized b
`constraining the position attribute of the pathfinder, fish and
`spider to a given interval. This feature is inherent to numerical
`attributes of i4D actors. The currentprototype of this application
`also features sound and music.
`
`
`
`
`Fig. 6: i4D Creature World
`
`The control via web form was illustrated in a multimedia
`application using the Macromedia Director software package. We
`4
`
`
`Effector
`
`cen di:
`
`‘update queue
`
`Torque
`Effector
`
`Torque
`
`Angle
`Sensor
`
`Angle
`Sensor
`
`Fig. 7: Brain and Body Structure of an i4D Creatures
`
`A creature basically consists of a body component,that describes
`its visual appearance including body parts, joints, sensor and
`effectors (figure 2). A brain componentdefines how sensor inpu
`values are processed and forwarded to the effectors. This
`component features a neural net
`like structure that connects
`sensor and effector attributes (see Figure 7). A simple behavioris
`illustrated in Figure 6. The resulting walk cycle makes the
`
`
`
`79
`
`MEDIATEK,Ex. 1022, Page 8
`IPR2018-00101
`
`MEDIATEK, Ex. 1022, Page 8
`IPR2018-00101
`
`

`

`creature move forward by sending the values of ale g to its
`counterpart which uses them in the next processing cycle. The
`walk angle of the left (forward) leg is inverted and sent to the
`right ( backward) leg. The definition of a creature’s structure and
`behavior can be described using a special data structure that can
`be easily manipulated (by the user or a genetic -algorithm)
`allowing to design many different creatures.
`In the fina
`application a number of different creatures will evolve and
`interact (predator and prey, mating, etc). For highlighting and
`presenting interesting aspects of the simulation “intelligent
`cameras”and “smart lights” will be developed that always follow
`a creature or switch the camera perspective if an interesting
`situation ogcurs(e.g. a fight between two creatures happens).
`
`4.3 Interactive Augmented Reality
`Illustrations
`The most recent application integrates the augmented realit
`toolkit ARToolkit from Hitlab [7], University of Washington in
`i4D. The real scene is augmented with a numberof virtual 3D
`objects. The composition of rea 1 and virtual scene elementsis
`realized by the AR toolkit, that recognizes pre-defined patterns
`and their position / orientation. The integration in i4D allows the
`efficient design of complex animated 3D content in real scenes.
`In combination with existing components we aim at providing a
`high level
`support
`for
`the design of augmented realit
`illustrations that can be used for interactive explanations of rea
`artefacts
`(e.g.
`in museums),
`technical devices
`(e.g. for
`maintenance) and wayfinding (e.g. navigational aids).
`
`a
`
`=
`
`Interactive illustration techniques that modify conceptual
`objects
`.
`
`4.3.3 Example of interactive AR illustration
`Figure 8 shows a simple example of illustration techniques
`currently under development. This application will illustrate the
`features of a new laptop and explains the handling of the device
`(e.g. features of new card). The application is designed asa
`highly interactive illustration and provides a virtual 3D presenter
`as part of the user interface that explains important features of the
`product based on the users interactions (e.g. opening the laptop,
`showing interest in a feature, etc). Based on different scenarios
`(Eyecatcher / presentation a a fair, Guided Tour on a Give-Awa
`CD,
`Interactive Manual)
`the behavior
`specification of the
`identified actors resultsin a number ofillustration techniquesthat
`are used in this application.
`Examplesofthe first illustration category (Interactive illustration
`techniques that
`introduce additional (meta -) objects) include
`arrows and pointers that show important features of the product,
`e.g. the slot for inserting PCM/CIA cards. Examples of the second
`categor
`(Interactive illustration techniques that modify existing
`objects are visible in the highlighting of important regions the
`virtual presenter points to. The explosion animation described in
`section 3.1 is another illustration technique and explains the
`different parts of the internal modem card. Illustration techniques
`in i4D usedin this example are not limited to simple presentation
`techniques, as the example of a virtual animated presenter in
`figure 2 illustrates. The presenteris a fully animated 3D character
`that can use gestures, animation and audio to present objects and
`properties. The presentation an mations are composed from basic
`gestures that can be defined in the special editor shownin figure 4.
`
`4.3.1 Interactive 3D Illustrations
`User guidance in interactive 3D illustrations is provided b
`augmenting 3D simulations with interactive illustration tech -
`5. Related work
`niques inspired by traditional
`(print)
`illustration techniques.
`A numberofrelevant projects have influenced this work although
`Traditional illustration tech niques like annotations, meta-objects
`we do not know ofa similar approach that covers all aspects of
`and object
`substitution modify or augment
`the graphical
`our framework and the intended application of i4D.
`representation of the illustrated subject. They focus attention,
`point out relevant detail and provide additional information —
`Because the developmentof interactive 3D contentis still a ver
`guiding
`a viewer
`through
`graphical
`content.
`[lustration
`diverse and immature area with few established rules in the sense
`techniquesthus providearich set of guidance techniques that can
`of design principles or techniques i4D is based on a design
`be added to arbitrary graphical content. Adapting illustration
`approach that integrates techniques from multimedia and user
`techniques to 3D virtual environments makes it possible to
`interface design,
`that support ex ploratory and interdisciplinary
`provide users with visual guidance in a fa iliar way while
`design. This is in contrast to intelli-media applications for more
`exploiting
`the
`additional
`possibilities
`of
`animation and
`established areas of communication design (e.g.WIP/PPP[1]) that
`interactivity
`in
`virtual
`environments. Because
`illustration
`have experimented with automated design techniques.
`techniques are generally context
`independent
`their interactive
`The conceptual model of actors can be compared to the actor
`counterparts can be implemented as reusable componentsthat can
`metaphor in multimedia authoring systems (eg. Macromedia
`be applied to arbitrary content,
`thereby reducing developmen
`cost.
`Director[9]) but extends this in the in the sense of Laurel’s [8]
`computers
`as
`theatre
`to
`characterize
`all
`componentsas
`independentactors that collaborate to create a performance. This
`meansthat actors are not just graphical objects that appear on the
`screen or “stage”as in many multi-media tools, but also cover the
`deeper technical aspects of object be haviour and collaboration.
`This allows to cover the complex behaviour of interactive,
`animated 3D objects and also allows the smooth integration into a
`conventional multimedia authoring context, based on the actor
`metaphor. Theiterative design process is similar to prototyping
`processes employed in user interface and multimedia design.
`However, the scenario and actor based conceptallowsto structure
`and or ganize development, so that components
`can be reused
`across applications. This
`is
`a majorben efit compared to
`
`4.3.2 Interactive Illustration Techniques:
`The i4D system provides a library if illustration techniques as
`components that derive and update a graphical representation and
`its