`
`DECLARATION OF SCOTT DELMAN
`
`
`
`
`I, Scott Delman, am over twenty-one (21) years of age. I have never been convicted of a felony,
`and I am fully competent to make this Declaration. I declare the following to be true to the best of my
`knowledge, information, and belief:
`
`1. I am Director of Publications for the Association for Computing Machinery (“ACM”).
`
`2. Neither I nor ACM itself is being compensated for this Declaration.
`
`3. Among my responsibilities as Director of Publications, I act as a custodian of records
`for ACM’s publications.
`
`4. I make this Declaration based on my personal knowledge, information contained in the
`business records of ACM, or confirmation with other responsible ACM personnel with
`such knowledge.
`
`5. As part of its ordinary course of business, ACM publishes technical papers, including
`ACM’s conference proceedings, journals, and other full-text publications. ACM
`publications are distributed to ACM’s individual and institutional subscribers, ACM
`members, and/or ACM conference attendees. Additionally, ACM publications are
`available for public download through ACM’s Digital Library (as described further
`herein), which has existed since July 1997. Prior to the creation of the Digital Library in
`July 1997, members of the public could have obtained access to ACM publications by
`contacting ACM directly.
`
`6. As part of its ordinary course of business, since July 1997, ACM has maintained a Digital
`Library where ACM publications are available to the public, either for free or purchase,
`depending on the article. The Digital Library includes keyword search functionality
`permitting users to search for articles using keyword search terms and to filter search
`results by whether the term appears in the title, full-text, index terms, and/or abstract of the
`article. In addition, users may search by author name, and may filter search results by the
`particular type of journal, type of proceeding, and/or the date of publication.
`
`7. In addition to providing access to ACM publications, the Digital Library also indexes
`papers by other publishers. When a non-ACM publication is indexed in the Digital
`Library, the public may access “metadata” concerning the publication through the Digital
`Library but cannot access the full text of the paper; instead, a member of the public who
`wishes to obtain such a paper would have to contact the publisher of that paper directly.
`
`8. “An Open Software Architecture for Virtual Reality Interaction” by Gerhard Reitmayr
`and Dieter Schmalstieg on March 25, 2002, in Proceedings of the ACM Symposium on
`Virtual Reality Software and Technology (VRST '01). Association for Computing
`Machinery, New York, NY, USA, 47-54. DOI:https://doi.org/10.1145/505008.505018.
`Exhibit 1 is a true and correct copy of this paper.
`
`
`
`
`1601 Broadway, 10th Floor
`New York, NY 10019-7434
`
`Tel: +1-212-869-7440
`Fax: +1-212-944-1318
`
`acmhelp@acm.org
`www.acm.org
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
` 1
`
`META 1021
`META V. THALES
`
`

`

`
`
`9. The conference started on November 15, 2001, and the paper would have been available to
`the conference attendees on this date.
`
`10. I hereby declare that all statements made herein of my own knowledge are true and that
`all statements made on information and belief, are believed to be true, and further that
`these statements were made with the knowledge that willful false statements and the
`like are punishable by fine or imprisonment, or both, under 18 U.S.C. § 1001.
`
`
`
`
` declare under penalty of perjury that the foregoing statements are true and correct.
`
`
`
`
`
` I
`
`
`
`
`Executed on: __________________
`
`
`
`
`
`
`
`
`
`
`
`________________________________
`Scott Delman, Director of Publications
`
`1601 Broadway, 10th Floor
`New York, NY 10019-7434
`
`
`
`Tel: +1-212-869-7440
`Fax: +1-212-944-1318
`
`acmhelp@acm.org
`www.acm.org
`
`January 28, 2022
`
` 2
`
`META 1021
`META V. THALES
`
`

`

`
`
`
`
`
`
`
`
`Exhibit 1
`Exhibit |
`
`
`
`TA 1021
`META V. THALES
`
`META 1021
`META V. THALES
`
`

`

`An Open Software Architecture for Virtual Reality
`Interaction
`
`Gerhard Reitmayr
`Vienna University of Technology
`Favoritenstraße 9-11/188/2
`A1040 Vienna, Austria
`reitmayr@ims.tuwien.ac.at
`
`Dieter Schmalstieg
`Vienna University of Technology
`Favoritenstraße 9-11/188/2
`A1040 Vienna, Austria
`schmalstieg@ims.tuwien.ac.at
`
`ABSTRACT
`This article describes OpenTracker, an open software archi-
`tecture that provides a framework for the different tasks in-
`volved in tracking input devices and processing multi-modal
`input data in virtual environments and augmented reality
`application. The OpenTracker framework eases the develop-
`ment and maintenance of hardware setups in a more flexible
`manner than what is typically offered by virtual reality deve-
`lopment packages. This goal is achieved by using an object-
`oriented design based on XML, taking full advantage of this
`new technology by allowing to use standard XML tools for
`development, configuration and documentation. The Open-
`Tracker engine is based on a data flow concept for multi-
`modal events. A multi-threaded execution model takes care
`of tunable performance. Transparent network access allows
`easy development of decoupled simulation models. Finally,
`the application developer’s interface features both a time-
`based and an event based model, that can be used simulta-
`neously, to serve a large range of applications. OpenTracker
`is a first attempt towards a “write once, input anywhere“ ap-
`proach to virtual reality application development. To sup-
`port these claims, integration into an existing augmented
`reality system is demonstrated. We also show how a proto-
`type tracking equipment for mobile augmented reality can
`be assembled from consumer input devices with the aid of
`OpenTracker. Once development is sufficiently mature, it is
`planned to make Open-Tracker available to the public under
`an open source software license.
`
`Keywords
`Tracking, Mobile Augmented Reality, Virtual Reality, XML
`
`INTRODUCTION
`1.
`Tracking is an indispensable part of any Virtual Reality
`(VR) and Augmented Reality (AR) application. While the
`need for quality of tracking, in particular for high perfor-
`
`Permission to make digital or hard copies of all or part of this work for
`personal or classroom use is granted without fee provided that copies are
`not made or distributed for pro£t or commercial advantage and that copies
`bear this notice and the full citation on the £rst page. To copy otherwise, to
`republish, to post on servers or to redistribute to lists, requires prior speci£c
`permission and/or a fee.
`VRST’01, November 15-17, 2001, Banff, Alberta, Canada.
`Copyright 2001 ACM 1-58113-427-4/01/0011 ...$5.00.
`
`mance and fidelity, have led to a large body of past and
`current research, little attention is typically paid to softwa-
`re engineering aspects of tracking software. Some current
`systems have a modular approach that allows to substitute
`one type of tracking device for another. Typically, this is
`the approach taken by commercial VR products that offer
`turn-key support for many popular tracking and input de-
`vices, but at the cost of a limited amount of extensibility
`and configuration options. In particular, they make it hard
`to combine existing features in novel ways.
`In contrast, research systems may offer features not found
`in commercial systems, such as prediction or sensor fusion,
`but are usually limited to their particular research domain
`and not intended for the end user. In such systems, repla-
`cing a piece of hardware or changing its configuration usually
`leads to rewriting a significant portion of the tracker soft-
`ware.
`In the middle(-ware), there is a lack of tools that allow for
`a high degree of customization, yet are easy to use and to
`extend. One notable exception is the MR toolkit [21] of the
`University of Alberta, which still serves as a starting point
`for many VR research projects despite its aged architecture
`and lack of active development. What is needed is a system
`that allows mixing and matching of different features, as well
`as simple creation and maintainance of possibly complex
`tracker configurations.
`In this article, we describe a tracking software system cal-
`led OpenTracker with the following characteristics:
`• An object-oriented approach to an extensive set of sen-
`sor access, filtering, fusion, and state transformation
`operations
`• Behavior specification by constructing graphs of tracking
`objects (similar in spirit to scene graphs or event cas-
`cades) from user defined tracker configuration files
`• Distributed simulation by network transfer of events
`at any point in the graph structure
`• Decoupled simulation by transparent multi-threading
`and networking
`• A software engineering approach based on XML [4],
`which allows to use many generic tools such as [2, 11,
`10] for development, documentation, integration and
`configuration
`• An application independent library to be integrated
`into software projects
`
`47
`
`META 1021
`META V. THALES
`
`

`

`Through its scripting capability (tracker configuration fi-
`les) as well as easy integration of new tracking features,
`OpenTracker encourages exploratory construction of com-
`plex tracking setups. It is equally useful for end users, which
`can fully exploit their hardware without any custom pro-
`gramming, as well as developers, who can easily build test
`environments. The modular approach gives instant access
`to wide range of tracking related functionality for any app-
`lication. Through the release under the LGPL Open Source
`license [7], OpenTracker is available to a larger audience.
`
`2. RELATED WORK
`Ideas implemented in OpenTracker were drawn from se-
`veral areas:
`Device abstraction is a standard requirement for 2D gra-
`phical user interfaces, (e. g. GKS [12]), and sometimes in-
`corporated into 3D applications [9]. There is a number of
`libraries such as VRPN [15], MRToolkit [21] implementing
`device abstraction for input devices typically found in VR
`and AR systems. Their main goal is to provide a fixed in-
`terface to the application for different devices and provide
`simple services for relaying the data over the network bet-
`ween several hosts. However, these libraries mostly lack any
`further means to process the data. Device abstraction is also
`an important goal of OpenTracker. However, it goes beyond
`pure abstraction using a static interface in that the data can
`be re-combined in novel ways.
`Many interactive systems employ sophisticated event hand-
`ling schemes. State changes to attributes of scene objects
`are either propagated through functional dependencies (e.
`g. routes in VRML [5], engines in Open Inventor [22]), or
`may be handled by user supplied callback functions (e. g.
`script nodes in VRML [5]). These approaches inspire the
`architecture of OpenTracker, although none of them deals
`specifically with tracker configurations.
`Finally, an important requirement for virtual environments
`is support for distributed simulation, partly to support si-
`multaneous users, partly to better exploit available hardwa-
`re. Decoupled simulation was first introduced in MR [21],
`and later used in almost any major VR software system.
`Decoupled simulation can either be implemented by multi-
`threading and/or symmetric multiprocessing on one host, or
`by configuring a small set of hosts to work as an ensemble.
`The latter approach may be inferior performance-wise be-
`cause of network lag, but it is inexpensive and flexible, and
`thus favored by many researchers - for example, Rekimoto’s
`“hyperdragging“ system [19] uses a distributed architecture
`very much like our own.
`
`3. DATA FLOW OF TRACKING DATA
`In a typical VR or AR application tracking data passes
`through a series of steps. It is generated by tracking hard-
`ware, read by device drivers, transformed to fit the require-
`ments of the application and send over network connections
`to other hosts. Different setups and applications may requi-
`re different subsets and combinations of the steps described
`but the individual steps are common among a wide range of
`applications. Examples of such invariant steps are geometric
`transformations, Kalman filters and data fusion of two data
`sources.
`The main concept behind OpenTracker is to break up
`the whole data manipulation into these individual steps and
`
`build a data flow network of the transformations. To descri-
`be the details of this concept, we will need some theoretical
`definitions which are discussed in section 3.1. Details of an
`actual implementation are described in section 3.2.
`
`3.1 Data Flow Concept
`Each transformation is represented by a node in a data
`flow graph. Nodes are connected by directed edges to des-
`cribe the direction of flow. The originating node of a directed
`edge is called the child whereas the receiving node is called
`the parent. To allow more than simple linear graphs, we
`introduce the following concepts.
`Multiple Input Ports and References
`Each node has one or more input ports and a single out-
`put port. A port is a distinguished connection point for an
`edge, i.e. the node can distinguish between events passing
`through different node ports. The output port of one node
`is connected to any of the input ports of another node. This
`establishes the flow by defining directed edges in the graph.
`A node receiving a new data event via one of it’s inputs
`computes a new update for itself and sends the new data
`event out via its output port.
`Multiple input ports are desirable because computations
`typically have more than one parameter. Dynamic trans-
`formations, for example, are parameterized by the value of
`another node and thus use the data value received by a child
`to be transformed differently from the data of the parame-
`terizing child. Merge nodes may select part of the data of an
`event based on the input port the event used. This allows
`more complex computational structures.
`Additionally, an input port can be connected to several
`output ports. This enables several children nodes connec-
`ted to the same input port of a node. Upon receiving an
`event, the parent node can only distinguish between the in-
`put ports, not between the actual children.
`Conversely, an output port can also be connected to other
`nodes by using references within the graph. This establishes
`new edges between a nodes output port and other nodes
`input ports. However this is transparent to the child node.
`It cannot selectively send events to only one parent, but all
`events are distributed equally to all parents.
`Edge types
`The basic mechanism behind the data flow concept is
`event passing. Data events are passed from the children no-
`des upward to their parents. However, not all computations
`fit well into this model: Algorithms that operate on a vec-
`tor of tracker measurements or that require or compute the
`tracker state at an arbitrary point in time require different
`types of input or output interfaces. Examples are smoothing
`algorithms that take a history of events into account, or pre-
`diction algorithms that compute an expected measurement
`for a given point in time.
`Therefore, we also distinguish between different edge ty-
`pes. Edges are typed by typing the ports of the nodes they
`connect. We establish the rule that only two ports of the sa-
`me type can be connected and this type is then the type of
`the edge. There are three edge types: event, which is imple-
`mented by event passing, event queue and time dependent.
`The latter two are implemented as interfaces that are polled
`by the parent node, because the data returned is paramete-
`rized. In the case of the event queue interface, it is possible
`to query the number of stored events and retrieve them by
`index. The time dependent interface can be queried by spe-
`
`48
`
`META 1021
`META V. THALES
`
`

`

`cifying a point in time, for which the appropriate data value
`is returned.
`
`a) linear graph
`
`b) multiple ports
`
`c) reference nodes
`
`d) putting it all together
`
`Figure 1: Visualizations of a data flow graphs as used
`in OpenTracker
`
`Figure 1 gives some examples of data flow graphs that
`can be build with OpenTracker. Part a) shows a simple li-
`near graph applying a geometrical transformation to a data
`source, b) shows a node with several input ports, combining
`the received data. Part c) is a graph using a reference node
`to get a copy of the output of a node and d) combines these
`features in a more complicated graph.
`3.2 Implementation Speci£c Details
`In an actual implementation we distinguish source nodes,
`which are leaves in the graph and receive their data values
`from external sources, filter nodes, which are intermediate
`nodes and modify the values received from other nodes, and
`sink nodes, which propagate their data values received from
`other nodes to external outputs.
`Source Nodes
`Most source nodes encapsulate a device driver that di-
`rectly accesses a particular tracking device, such as a Pol-
`hemus or Ascension tracker connected to a serial interface.
`Other nodes objects form bridges to complex self-contained
`systems, such as the video tracking library from ARTool-
`kit [13]. A third type of source node emulates a tracker via
`the keyboard, access network data (see section 5) or simply
`responds with constant values (useful for development and
`debugging).
`Some source nodes have a multi-threaded execution model
`to implement an efficient decoupled simulation model [21] (e.
`g., when blocking I/O must be used).
`Filter Nodes
`Filter nodes receive values from one or more child nodes.
`Upon receiving an update from one or more of their children,
`they compute their own state based on the collected data.
`A non-exhaustive list of filters includes:
`• Transformation filters perform geometric transforma-
`tions of their children’s values. These include pre- and
`
`post-transformations and may be static or depend on
`data values received from other children. The latter
`allows to modify the filtered state relative to another
`tracker state.
`• Prediction filters allow to partially compensate for lag
`in the measuring and processing tracker data.
`• Noise and smoothing filters are handy to deal with
`inherent inaccuracies of trackers.
`• Undistortion filter are necessary e.g. to linearize dis-
`tortions in the magnetic field of a magnetic tracking
`device.
`• Permutation filters are necessary to match data repre-
`sentations from different hardware or software plat-
`forms, such as equivalent, but incompatible quaternion
`representations.
`• Merge filters assemble new data values using different
`parts of the data values of several children. Sample uses
`include the combination of orientation from an inerti-
`al tracker with position information from an acoustic
`tracker, or adding a button device to a closed tracking
`solution such as Polhemus Ultratrak.
`• Conversion filters are able to translate one data type
`into another. For example, 2D positions from a desktop
`pointing device can be translated into 3D positions by
`adding a constant third value.
`• Clamp filter are special nonlinear transformation fil-
`ters that cut off values at user-specified extrema, for
`example to deliberately limit interaction to a valid ran-
`ge.
`• Store-and-forward filters are useful if transient loss of
`tracking can be expected, for example if occlusion oc-
`curs in optical tracking. The last measured value is
`simply repeated to provide at least a reasonable and
`valid state.
`• Confidence filters select data values from different child-
`ren based on some measure of confidence in the accu-
`racy of the data.
`
`Sink Nodes
`Sink nodes are similar to source nodes but distribute da-
`ta rather than receive it. They include output to network
`multicast groups, debugging output to a user interface or
`thread-safe shared memory output to integrate OpenTracker
`as a library into other applications.
`3.3 Time
`Time is reflected in several ways in the architecture of
`OpenTracker. The type system for edges supplies us with
`different ways of dealing with Time, either having an event
`based approach, with or without queuing of events, or by
`specifying functions of tracking data as continuous functions
`of time.
`For the event based nodes, each event is time stamped
`by the individual device driver or node that generated it.
`Thus nodes can react on the temporal aspects of tracking
`data. For example, a simple prediction node incorporates the
`time difference between single events to correctly update its
`output.
`
`49
`
`META 1021
`META V. THALES
`
`

`

`More complex aspects such as a prediction for a changing
`prediction interval is satisfied by the different edge types.
`An application that wants to get a calculated value for an
`arbitrary point in time can query the state at that time from
`a node supporting time dependent output. How this value is
`calculated depends on the node’s implementation.
`OpenTracker does not implement any clock synchroniza-
`tion of different hosts working together in a network. There
`are already well established means to solve this problem such
`as the NNTP protocol.
`
`4. SOFTWARE ARCHITECTURE OF THE
`LIBRARY
`The intent of OpenTracker is to provide an auxiliary li-
`brary that is to be integrated into VR or AR applications.
`Therefore it is kept very lightweight and customizable. The
`library is designed as a class hierarchy of tracker objects,
`implemented in C++. It is build around a small set of core
`classes that implement the basic node interfaces, a parser
`that builds the runtime structure from a configuration file
`and the main loop driving the event model. Any other func-
`tionality is implemented by a set of module classes that can
`be easily extended or modified.
`The module classes create and manage the nodes repre-
`senting the functionality of the module. In the main loop
`of the library each module is called to provide new events
`and after an event is processed to handle results of the da-
`ta flow. For example, the implementation of a network sink
`node stores any event data that it received during event pro-
`pagation. Afterwards the network module checks each net-
`work sink node for updated data values, constructs a new
`network packet and sends it to the configured destination.
`Modules may be implemented multi-threaded to avoid stal-
`ling the main thread during longer computations or polling
`a device with blocking I/O.
`There are also nodes that perform without an underlying
`module. Examples are filter nodes that implement geometric
`transformations on incoming events and pass the transfor-
`med events to their parents.
`There is no fixed interface to the integrating applicati-
`on to maximize flexibility. Application programmers have
`to either use on of the supplied nodes (such as a generic
`call back node) or supply their own module implementing
`sink nodes as interfaces to their application. Moreover, the
`use of the library main loop is not mandatory. The proces-
`sing can be integrated with the applications main loop to
`avoid additional threads and synchronize the tracking data
`processing more closely with the application. These design
`decisions ensures that the library can adapted to the special
`needs of every application.
`The primary type of event used in the current implemen-
`tation is tailored toward tracking applications. It encodes a
`position in space, an orientation, button states, a time stamp
`and a confidence value to describe the quality of the data.
`Although this restriction to a fixed data type appears as an
`limitation, it can easily be extended or generalized because
`nothing in the supporting system relies on the type of the
`event data.
`Figure 2 shows a class diagram of the core classes. The
`class Context implements the main loop and keeps refe-
`rence of all modules and the data flow data structure. It
`employs an object of class ConfigurationParser to parse the
`
`Figure 2: Architecture of the OpenTracker library
`
`configuration files. Actual node implementations are derived
`from Node, for example the Transformation or the TestSour-
`ce class. WrapperNode and RefNode are special nodes that
`implement the port and reference functionality. State is the
`default event type.
`
`5. DISTRIBUTED TRACKING
`There are several reasons why is is desirable to share
`tracker data over a network:
`• Using the tracker data at multiple host computers for a
`distributed virtual environment (local or remote): In-
`put in the form of tracker data becomes readily availa-
`ble through transparent network access via OpenTracker.
`The scene database still has be to kept consistent through
`a proprietary application protocol, but the task is much
`simplified.
`• With the same approach, multi-processing based on
`inexpensive PCs becomes possible with little configu-
`ration effort. This is useful to achieve some degree of
`load balancing. In particular, computationally expen-
`sive functions such as filtering or undistortion can be
`assigned to either sender or receiver, depending on the
`computational budget.
`• Network support makes it easy to span multiple opera-
`ting systems, in particular if a specific tracking device
`or service is only available at one particular host (e.g.,
`an SGI O2 has fast video hardware but a slow CPU,
`whereas for a PC the opposite may be true).
`
`Magnetic tracker
`
`Optical tracker
`
`Rendering hosts
`
`User 1
`
`User 2 …
`
`Figure 3: Distributing tracking data send to different
`rendering hosts
`
`50
`
`META 1021
`META V. THALES
`
`

`

`OpenTracker allows multiple senders and receivers of tracker
`data to communicate asynchronously through the use of
`IP multicasting (Figure 3). This approach effectively im-
`plements decoupled simulation in a distributed over several
`hosts, since each of the senders and receivers can operate
`independently. It is even possible for a single host to opera-
`te as a sender and receiver at the same time, by picking up
`data, then modifying it and re-sending it to the network on
`another network channel.
`While there is a preferred network protocol for Open-
`Tracker, support for additional formats can be easily im-
`plemented. In the following, we give some examples as to
`how a networked setup can be used:
`• A tracker server (typically a cheap PC with lots of
`serial I/O boards running Linux) samples an Ascension
`Flock of Birds at highest rate and sends the resulting
`data stream via multicast to several clients using this
`data to animate a collaborative virtual environment.
`• The Polhemus Ultratrak uses a proprietary network
`format and IP unicast packages. Unfortunately, its clo-
`sed architecture does not support input devices with
`buttons such as a stylus or 3D-mouse. Therefore, we
`added a tracker object to the client that is able to
`decode the Ultratrak protocol. A button source reads
`button values from a standard parallel interface, and
`a merge filter combines these two sources to emulate a
`complete VR input device.
`• A combination of vision tracking and magnetic tracking
`– see section 7 for details.
`
`6. SOFTWARE ENGINEERING WITH XML
`XML, the eXtensible Markup Language, is the emerging
`standard primarily aimed at web-based applications and
`software systems [4]. XML is a markup definition langua-
`ge that allows to define hierarchical markup languages with
`so-called document type definitions (DTD). With the ap-
`propriate DTD, standard XML tools can be used to con-
`veniently edit, type check, parse, and transform any XML
`file.
`Thus, providing a simple DTD for describing the data flow
`graphs of tracker nodes opens access to software libraries and
`tools that simplify several steps of the development cycle:
`• A visual DTD editor can be used to design and main-
`tain the DTD.
`• An XML parser [2] enforces content format on the
`tracker configuration file while building the correspon-
`ding structure in memory, thus automatically perfor-
`ming many of the consistency checks that have other-
`wise to be hand-coded.
`• The same parser implements an API to manipulate the
`data structure at runtime and still keep it consistent
`with the DTD. Such a runtime structure can easily be
`written out to a valid configuration file again.
`• A convenient XML editor such as [11, 10] with a gra-
`phical user interface allows the end user to design the
`tracker configuration without having to master the
`syntax. It also enforces the correct content format, re-
`ducing syntax and semantic errors made by users.
`
`• Integration with high-level software engineering tools
`that create code or configuration files from specificati-
`ons is simplified by the use of XML. Even automatic
`reverse engineering of complex configurations is easier
`relying on a defined structure than from pure source
`code.
`• Using the extendible style language (XSL) [1, 6], auto-
`matic textual and even graphical documentation can
`be created from a tracker configuration file, for exam-
`ple by using the free graph drawing utility dot [3] (see
`Figure 1).
`
`Markup languages are generally used to annotate textual
`documents with structural information. Thus a general XML
`document consists of text grouped and structured with tags.
`Markup languages defined in XML consist of elements, es-
`sentially expressed as tags, and a structural model (the con-
`tent model) of the possible ways these elements may be ne-
`sted. Moreover, elements are annotated by name-value pairs
`called attributes.
`OpenTracker maps elements to nodes and attributes to
`members of these nodes. We are not using any textual con-
`tent but purely rely on the content model provided by the
`DTD. An open source XML parser [2] builds a tree of ele-
`ments representing the given configuration file. OpenTracker
`walks the tree and creates a new node for each element based
`on the elements name. The string values of the attributes are
`parsed according to the objects class and the corresponding
`members are set. Attributes typically describe such data as
`the parameters of a transformation. The parent - child rela-
`tionship of the data flow graph is directly mapped onto the
`parent - child relationship of XML elements.
`The content model enforces interface and semantic cons-
`traints on the specified graph. As described in section 3.1
`edges and the corresponding node ports are typed and the-
`refore restrict the possible combinations in the construction
`of the graph. These constraints are expressed in the DTD
`and are checked by an XML parser or enforced by an XML
`editor. Also restrictions on the number of children are des-
`cribed in the DTD. Source nodes typically do not have any
`children as they rely on data from external sources to com-
`pute their own data. A number of filter nodes get the value of
`a single child node, transform it and pass it on. In contrast,
`confidence filters use any number of children to compute
`their data value.
`The reference structure is created by using unique ID at-
`tributes on elements and referencing these IDs in reference
`elements. Again XML enforces the uniqueness of these IDs
`and the parser library simplifies the search for the referenced
`elements.
`While children of nodes with only one input port are di-
`rectly mapped to children elements in the XML file, children
`of different input ports need to be addressed differently. This
`is handled using wrapper elements. Any group of children
`that is connected to a specific input port is wrapped by an
`additional XML element. This element in turn is the direct
`child of the node of interest. These elements are closely rela-
`ted to the node’s element and are typically the only possible
`children elements. They are mapped to special wrapper no-
`des that can be distinguished by the node implementation.
`Otherwise they are transparent to the actual data proces-
`sing.
`
`51
`
`META 1021
`META V. THALES
`
`

`

`<?xml version="1.0" encoding="UTF-8"?>
`<!DOCTYPE OpenTracker SYSTEM "opentracker.dtd">
`<OpenTracker>
`<configuration>
`<ARToolKitConfig camera-parameter="camera_para.dat"/>
`</configuration>
`<ConsoleSink comment="Pip">
`<StbSink station="0">
`scale="0.001 0.001 0.001">
`<EventTransform
`DEF="Camera"
`<ARToolKitSource tag-file="pip.tag" />
`</EventTransform>
`</StbSink>
`</ConsoleSink>
`<ConsoleSink comment="Pen">
`<StbSink station="1">
`<EventDynamicTransform>
`<TransformBase>
`<Ref USE="Camera"/>
`</TransformBase>
`<EventTransform scale="-2.1 -2 0" translation="0.14 0.1 -0.01">
`<WacomGraphireSource device="1"/>
`</EventTransform>
`</EventDynamicTransform>
`</StbSink>
`</ConsoleSink>
`<ConsoleSink comment="Viewpoint">
`<StbSink station="2">
`<EventTransform rotation="1 0 0 0">
`<TestSource frequency="25"/>
`</EventTransform>
`</StbSink>
`</ConsoleSink>
`</OpenTracker>
`
`ARToolKitSource
`pip.tag
`
`WacomGraphireSource
`
`EventTransform
`
`EventTransform
`
`Ref
`
`TestSource
`
`Data
`
`Base
`
`DynamicTransformation
`
`StbSink
`
`EventTransform
`
`StbSink
`
`StbSink
`
`ConsoleSink
`Pen
`
`ConsoleSink
`Pip
`
`ConsoleSink
`Viewpoint
`
`Figure 4: Example configuration file and resulting graph. (This figure is reproduced in color on page 196.)
`
`Figure 4 gives an example of such a configuration file,
`using all of the features described before. The interesting
`constructs are highlighted and cross linked with the corre-
`sponding nodes in the resulting data flow graph.
`
`7. APPLICATIONS