Arbeitspapiere der GMD
`
`No. 1104
`
`Silvia Hollfelder, Florian Schmidt, Matthias Hemmje, Karl Aberer, Arnd Steinmetz
`
`Transparent Integration of Continuous Media Support into a Multimedia DBMS
`
`
`
`December 1997
`
`GMD – German National Research
`Center for Information Technology
`
`PAGE 1 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`The “Arbeitspapiere der GMD – GMD Technical Reports” primarily comprise preliminary
`publications, specific partial results and complementary material. In the interest of a subse-
`quent final publication the “Arbeitspapiere/Technical Reports” should not be copied. Critical
`comments would be appreciated by the authors.
`No part of this publication may be reproduced or further processed in any form or by any
`means without the prior permission of GMD. All rights reserved.
`
`E Copyright 1997
`
`Addresses of the authors:
`
`Silvia Hollfelder, Matthias Hemmje, Karl Aberer, Arnd Steinmetz
`GMD-IPSI
`Dolivostraße 15
`D-64293 Darmstadt
`Phone: +49/6151/869-953/ 869-844/ 869-935/869–862
`Fax:
`+49/6151/869-966
`Email:
`{hollfeld, hemmje, aberer, arnd.steinmetz}@darmstadt.gmd.de
`WWW:
`http://www.darmstadt.gmd.de/{~hollfeld,~hemmje,~aberer, ~steinmet)
`
`Address all correspondence to:
`
`Silvia Hollfelder
`
`2
`
`PAGE 2 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`Abstract*
`
`Multimedia Database Management Systems (MM-DBMS) have to efficiently provide the specific
`functionalities required by time-dependent multimedia data types. During presentation playout,
`this requires support for continuous data delivery and media-oriented optimization of presenta-
`tion quality according to user requirements. In this work, we show how we integrated this type of
`support transparently with other MM-DBMS functions like media editing and querying. This is
`achieved by extending an existing data model for continuous media, in particular by means of
`stream abstractions for editing and presentation, and by extending the system architecture by
`means of supporting stream processing components, e.g., for intra-media synchronization by
`adaptation. This approach has been implemented as an extension of existing commercial object-
`relational DBMS technology.
`
`This work has partly been funded by the European Union within the framework of the ESPRIT Long
`*
`Term Research Project HERMES, N. 9141 [http://www.ced.tuc.gr/hermes/hermes.html].
`
`3
`
`PAGE 3 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`1 Introduction
`In commercial DBMS there is a clear trend to provide specialized functionality, which has usual-
`ly been provided as separate software packages and required substantial development efforts for
`the integration with database applications, as integrated database extensions. Examples are
`particularly found in the management of nonstandard or multimedia data types, e.g., text
`retrieval, image handling, statistical functions. This trend is a key factor for the rapid spreading of
`object-relational database technology among all major database vendors and is the reaction to the
`requirements of new applications exploiting distributed multimedia technology, like digital
`video production [S96][SH94], education and training or digital libraries [BAN95]. For static
`data and media, this approach is relatively straightforward as it does not heavily affect the
`fundamental processing paradigms used in conventional DBMS. For handling continuous
`media, like, e.g., video and audio, additional characteristics and requirements related to the
`presentational aspects of these media need to be considered. Currently, there are two “schools” to
`that respect (for a nice discussion see also [O96]):
`1. Media presentation is not a problem that is to be considered by the DBMS. The DBMS
`provides a storage framework for the media, including meta data required for search and
`manipulation. Presentation is supported by external systems, like video servers, or within
`distributed multimedia applications.
`2. The DBMS has to support presentational aspects, since this is a fundamental aspect of acces-
`sing the media data managed by the DBMS. Presentational requirements like synchronization
`are to be considered as a new type of constraints that need to be consistently supported by the
`DBMS [MS96].
`From our perspective there are several arguments in favor of the second school:
`1. Presentations of composite multimedia objects require the coordinated scheduling of the
`available resources based on the characteristics of and relationships between the involved
`media. Since this information is stored in the database the DBMS itself most efficiently per-
`forms the scheduling. For example, when presenting a video with text inserts the DBMS needs
`to support a buffering scheme for the video that exploits the stream semantics and expected user
`interactions. In addition, it has to schedule the delivery of the text inserts such that they can be
`presented timely.
`2. In case of concurring requests, the DBMS is the place that can optimize the database usage
`and usage of available different media, for example by performing quality adaptations
`[TKCWP97].
`3. Different abstractions are used in the different stages of processing of media data. For exam-
`ple, a MPEG video stream is edited at the frame granularity, retrieved at the granularity of
`scenes or shots, and distributed over a network for presentation at the granularity group-of-pic-
`tures. The different data structures representing these abstractions, which are also relevant for
`presentation purposes, need to be kept consistent during updates.
`4. Exploiting application-specific characteristics of multimedia objects for distribution and
`presentation of the media increases the necessity of integrated presentation support in the
`DBMS. Examples are quality adaptations for parts of presentations resulting from a retrieval
`request parametrized by their relevance, buffering strategies taking into account possible future
`interactions based on application semantics, e.g., favorite hyperlinks, or resource scheduling
`taking user profiles or usage statistics into account.
`In this paper, we show the feasibility of the integration of presentational support for multimedia
`data with a standard commercial object-relational DBMS (Informix Universal Server IUS).
`Basic support in the spirit of school 1 is provided by the IUS by means of the Video Foundation
`
`4
`
`PAGE 4 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`Blade [VFUG97]. We actually take advantage of the mechanisms provided by this database
`extension to simplify the physical integration of the media server and obtain additional
`abstraction and querying capabilities. The additional presentational support that we provide
`addresses the continuous data transport required during the presentation of continuous media
`streams, client-side buffering mechanisms exploiting stream and interaction semantics, and cli-
`ent-side quality adaptation mechanisms to overcome fluctuations in the network and server
`resources. Most of the concepts were already developed while we extended the object-oriented
`DBMS VODAK [V95] to a (proprietary) multimedia DBMS [RKN96]. This paper shows that
`the concepts for presentational support developed in this earlier proprietary work can be
`smoothly integrated into standard platforms, taking advantage of the available functionality of
`these systems. Thus there is no principal difficulty in a modular extension of modern, extensible
`DBMS with presentational support for multimedia data.
`2 General Concepts
`In the following, we go into some details of the basic concepts supporting the storage and
`presentation that have been integrated as multimedia database extensions for presentational
`support for multimedia objects.
`2.1 Continuous Long Fields Data Abstractions
`We provide a data type Continuous Long Field (CLF) as generic, abstract representation for any
`kind of continuous media, like audio, video or animations. This datatype supports operations for
`editing and presentation, like “insert”, “request”, or “delete”. By modelling the generic concept
`CLF, we combine the concepts of streams and long fields. While streams are only “consumed”,
`e.g., read by presentation entities, the concept of CLFs allows consumption as well as any access
`to the data. Structural meta data, like the format and other recording parameters, are stored with
`every CLF object. Multiple media encoded in a single stream (like, e.g., with MPEG-encoded
`audio and video) are modelled as separate CLF objects, to enable the individual retrieval and
`manipulation of each of the media parts. For editing purposes, CLF objects are segmented into a
`sequence of manipulation units, called Continuous Object Data Units (CODU). In consequence,
`a CLF is a sequence of CODUs. The granularity of these units is determined individually for each
`type of media, e.g. Motion-JPEG frames, MPEG-1 frames or audio samples. At this level of
`granularity, it is possible to manipulate the data stream, e.g., by inserting, deleting or appending
`CODUs of a Continuous Long Field.
`For continuous presentation our approach supports a client-pull mechanisms, i.e., the client
`continuously requests chunks of data which are provided in our system by the server in a best-ef-
`fort manner. Since requests at the granularity level of CODUs are not very effective and presenta-
`tion engines, like, e.g., MPEG players or other video and audio devices, handle larger units of
`data, a second abstraction is introduced to support presentation. An atomic unit requested during
`synchronous presentation is called a Continuous Object Presentation Unit (COPU). One COPU
`may consist of several CODUs as illustrated in Figure 1. By this generic concept, the buffer man-
`ager is able to handle any kind of continuous media in the same way by means of generic COPU
`requests.
`The CLF data type that provides the above abstractions has been implemented as part of our
`database extension for basic continuous media support. It provides, in particular, storage support
`for the raw media data and associated meta data, like, e.g., content descriptions, physical
`characteristics or presentational characteristics [HL97]. By means of the explicit representation
`of meta data entities, we offer the opportunity to support queries on meta data, too. This means it
`is, e.g., possible to send queries about the content, the structure or technical parameters of a video
`stream. Currently, the content-oriented meta data has to be managed manually in the database,
`because we do not support any automatic content-oriented analysis or retrieval mechanisms in
`the current implementation. More details on the implementational aspects will be provided in
`Section 3.
`
`5
`
`PAGE 5 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`COPU
`
`COPU
`
`Media Type
`
`Audio
`
`M-JPEG
`
`MPEG
`
`Block of
`samples
`
`Frame
`
`Group of
`Pictures
`
`CODU
`
`CODU CODU CODU CODU CODU CODU CODU CODU CODU
`
`Sample
`
`Frame
`
`Frame
`
`Continuous Long Field
`
`Figure 1: Modelling of Continuous Long Fields
`
`2.2 Client Buffer Mechanism Least/Most Relevant for Presentation
`An intelligent buffering mechanism for continuous media streams has not only to optimize trans-
`port of data from server to client but also to take into account the presentational requirements of
`the media stream. Since the size of continuous objects it too large to store them in the client buffer
`as a whole, single COPUs are requested continuously by the buffer manager. We use an
`algorithm called Least/Most Relevant for Presentation (L/MRP) [MKK95] for preloading and
`replacing COPUs in the client buffer. Its goal is to support a synchronized presentation of the
`media stream.
`The algorithm considers the current presentation state and likely interactions when deciding
`which COPUs need to be preloaded and replaced in the client buffer. For that purpose, it assigns a
`relevance value to each COPU, such that COPUs with least relevance are replaced first and those
`with highest relevance value are preloaded first. Dynamic relevance values are related to the
`current presentation state. For example the COPUs following the presentation point in
`presentation direction have the highest value, decreasing with the distance from the presentation
`point. If, for example, a change of the presentation direction is likely in the application, the
`COPUs preceding the presentation point, that are typically already loaded in the buffer, obtain
`relatively high relevance, too. Static relevance values are assigned independent of the current
`presentation point, for example, to working points within the streams that are more likely to be
`referenced. Therefore, these are preloaded into the buffer more likely and are replaced from the
`buffer later than other COPUs. This results in a higher availability of these COPUs. The functions
`assigning relevance values can be modelled according to the typical needs of the application.
`2.3 Buffer-Triggered Adaptation
`Since we do not assume the availability of reservation mechanisms for stream delivery, bottle-
`necks on server or network may occur during presentation and lead to delay of presentation.
`Therefore, we use a client-based adaptation mechanism to dynamically change the presentation
`quality in order to reduce the data volume that needs to be delivered from the server to the client.
`In this way, intra-media synchronization can be maintained by reducing disk utilization, memory
`consumption or required network bandwidth.
`Our smooth adaptation technique assigns different presentation qualities to different intervals of
`client buffer fill levels. The rationale of this approach is that with high buffer utilization the
`danger of getting delays in the presentation is low and that only moderate adaptations may be
`necessary. But, as with low buffer utilizations the danger of loosing intra-media synchronization
`increases, more effective adaptation techniques have to be invoked by the client in order to fill up
`the buffer more quickly with inferior quality data. The buffer fill level intervals used in the
`adaptation specification overlap. Thus if the buffer fill level is close to the border of an interval,
`frequent quality switches are avoided by using hysteresis. The quality adaptations are modelled
`separately for each type of media [AFK95] and can be effected along the temporal dimension
`(e.g., frame dropping) or along the spatial dimension (e.g., reduction of resolution). In case of
`dropping, additional constraints like the minimum distance between dropped frames and the
`
`6
`
`PAGE 6 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`maximum distance inbetween dropped frames can be introduced to, e.g., reduce jitter. A detailed
`description of the adaptation framework is given in [HKR97].
`In Figure 2, the architecture of the adaptation feedback and its interaction with the buffering
`mechanism, described in Section 2.2, is illustrated. The presentation process starts with a filled
`client buffer. The presentation engine continuously requests COPUs for their synchronized
`presentation and handles user interactions. While the COPUs are presented, the preloader
`asynchronously prefetches COPUs from the server in the client buffer. If prefetching from the
`server is faster than or as fast as the presentation of COPUs, the buffer utilization should never
`decrease. If server CPU, server disk or network bandwith have temporary bottlenecks, the cli-
`ent’s buffer utilization starts to decrease. In this case the presentation quality is gracefully
`degraded. The adaptation control module permanently controls the buffer size, and, based on the
`adaptation specification, determines which quality has to be requested. In case of dropping the
`request of COPUs, logically empty COPUs are inserted into the buffer to increase buffer utiliza-
`tion.
`
`Presentation
`
`Adaptation
`Specification
`
`ÉÉÉÉÉÉ
`User Interactions
`ÉÉÉÉÉÉ
`
` COPUs
`
`Current Interval
`
`
`ÉÉÉÉÉÉÉÉÉÉÉÉ
`COPU reference
`COPU request
`
`ÉÉÉÉÉÉÉÉÉÉÉÉ
`ÉÉÉÉ
`Adaptation
`ÉÉÉÉ
`Control
`ÉÉÉÉ
`Buffer Utilization
`
`Buffer
`Preloader Empty
`COPU
`
`Adaptation
`Quality
`
`Continuous Object Manager (COM)
`Request to server
`
`COPU from server
`
`Figure 2: Adaptation Feedback Mechanism
`
`Our adaptation mechanism is triggered by the client that means that the clients react on bottle-
`necks detected in their own buffer. The reason for that is that our adaptation mechanism is closely
`related with the preloading and replacement strategy. The drawback is that the mechanism is not
`able to consider the knowledge about global resource consumption as the server is able to in dis-
`tributed system architectures. The adaptation mechanism has been implemented as part of a
`continuous media presentation environment of the database client. It constitutes a further
`component in our database extension for basic continuous media support.
`The drawback of the generic modelling approach introduced in chapter 2.1 is that in some cases
`media specific properties have to be considered and, as consequence, the generic operations
`mentioned above are only suitable for the basic operations on continuous media. For example, in
`video adaptation mechanisms, frame dropping is often used1. This does not match very easily the
`video compression format MPEG because the dependencies between the single frames have to
`be considered while the generic buffer manager only handles independent, atomic units, which
`will, e.g., be dropped by an adaption mechanism in case of bottlenecks. This implies, at least in
`the MPEG scenario, a demand for more specific preloading strategies in case of adaptation to
`bottlenecks. Therefore, the whole stream has to be parsed when the object is stored into the data-
`base, and the positions of the single frames in the stream have to be determined. In addition to
`that, insert and update operations have to guarantee consistent meta information [HL97].
`
`–––––––––––––––––––––––––––––––––––––––––
`1.
`In our client-pull system, dropping COPUs means that they are not requested by the client.
`
`7
`
`PAGE 7 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`3 Integration of Multimedia Data Support
`3.1 Object-relational DBMS
`As the name suggests object-relational DBMS combine features of relational as well as object-
`oriented DBMS. The drawback of relational DBMS is that the datatypes are limited to the ones
`supplied by the system. The system is able to manage these predefined datatypes very efficiently,
`but data which does not belong to a system datatype must be stored as an uninterpreted string of
`bytes (a so-called binary large object, BLOB). The DBMS has no semantic knowledge of the data
`stored within a BLOB and, therefore, no interpretation of the data is possible.
`Object-oriented DBMS on the other hand are specifically designed to support the modelling of
`new and potentially complex datatypes which are modelled as object types. Functions supporting
`an object type can be defined within the OO-DBMS providing the system with knowledge for the
`interpretation of data. Since all data is stored as instances of object types, querying is often com-
`plicated.
`Object-relational DBMS combine the positive aspects of both system alternatives. They store
`data in tables and provide the powerful querying of relational DBMS, but the data types used in
`the tables are not limited to a predefined set. New types can be created and used in the database
`just like any predefined type. Functions can be defined to support content-based operations on
`new datatypes or special purpose functionality to the database.
`The combination of efficient retrieval and extensibility made an OR-DBMS the system of choice
`for the Multimedia DBMS proposed in this paper. The object-relational DBMS Informix Univer-
`sal Server (IUS) that we decided to use as a platform enables the integration of so-called Data-
`Blades. DataBlades are libraries of datatypes and corresponding functions that are loaded into
`the DB kernel at runtime. They provide flexible extensions by new datatypes and their corre-
`sponding functions which are dynamically integrated into the DBMS’s data model. These func-
`tions may be used in SQL-commands as well as in the application programmers interface. High
`performance is achieved by means of the tight integration of any kind of new types into the
`DBMS kernel which operates on a parallel architecture. Furthermore, the Informix Universal
`Server supports object-oriented APIs, like C++ or Java, as an interface to the application pro-
`grammer.
`3.2 Introduction to the Video Foundation DataBlade
`The so-called “Video Foundation DataBlade” (VFDB) implements the base schemata that Infor-
`mix Universal Server (IUS) applications use to handle continuous media streams. As it is de-
`signed in an open and extendable way, it is used as the foundation for the development of the
`“Continuous Long Field DataBlade” (CLFDB) described in this paper. The VFDB defines a
`schema that enables applications to control DB-external storage managers which are responsible
`for the storage of continuous media streams. By means of the abstract functions and datatypes
`defined by the VFDB data model and schema as well as by means of its potential extensions (as
`e.g. defined by the CLF DataBlade schema), all stream-related meta data, whether they are re-
`lated to physical properties, the content or the usage and applications of the stream, can be manip-
`ulated through this common media and platform-independent access and management layer.
`The VFDB application scenario’s software architecture consists of three main components: The
`IUS, an external storage manager and the application program. The IUS stores all meta data. It
`manages and controls the access to the external storage managers and devices. The external stor-
`age managers handle the storage of the various media streams. Whenever an application wants to
`access continuous objects, it can access them indirectly through the IUS VFDB schema and its
`access functions. If the access is time-critical, a direct connection to the responsible storage man-
`ager (through an appropriate client-server connection) can be mediated through the IUS and the
`VFDBs meta data as well. If the application requests only meta data, the IUS will serve the re-
`quest on the basis of the VFDB schema.
`
`8
`
`PAGE 8 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`To enable a format independent representation of times and periods of time, the VFDB defines
`the so-called “MedPtr”, “MedTC” and “MedChunk” data types which are responsible for repre-
`senting exact times in the timing format of the corresponding medium, for representing exact
`times according to the SMPTE timecode standard and for representing a period of time within a
`media stream.
`The VFDB registers all external storage managers within a so-called “MedVSIRegister” table. It
`also stores representations of objects stored externally as values of the so-called ”MedLoc” data-
`type.
`By means of a so-called “virtual storage interface” (VSI), the VFDB provides access functions
`like open, close, read, write, seek, and tell which enable applications to manipulate contents han-
`dled by the storage managers. The VFDB predefines the VSI for three simple external storage
`managers: one stores data as files on the client, one stores files on the server, and the third on
`media that are not electronically manageable.
`Besides providing these virtual access and management functions forming the general purpose
`VSI, the VFDB defines the above mentioned meta data schema, displayed in figure 3, which con-
`tains tables for technical and physical meta information (stored in so-called “VidMDGeneral“,
`“VidMedia”, “VidPVideo“, “VidPAudio”, “VidPOther”, “VidVCodec”, “VidACodec”, “Vid-
`Format”, “VidStandard”, “VidPVideo”, “VidPAudio”, and “VidPOther” tables) as well as for
`structure- and content-oriented meta information (stored in so-called “VidChunkMD”, “Vid-
`Strata”, “VidVidDesc”, “VidImgDesc”, “VidAudDesc”, and “VidTextDesc” tables). Further in-
`formation can be found in [VFUG97].
`
`VidMDGeneral
`
`(1,*)
`
`has copies
`
`(1,1)
`
`(0,*)
`
`(0,1)
`
`(0,1)
`
`(0,1)
`
`VidPVideo
`
`VidPAudio
`
`VidPOther
`
`VidMedia
`
`(0,*)
`
`contained_in
`
`(1,1)
`
`VidChunkMD
`
`(0,*)
`
`(0,*)
`
`(1,1)
`
`media
`specific
`
`(0,*)
`
`describes
`
`describes
`
`describes
`
`describes
`
`(1,1)
`
`(1,1)
`
`(1,1)
`
`(1,1)
`
`VidTextDesc
`
`VidImgDesc
`
`VidAudDesc
`
`VidVidDesc
`
`(0,*)
`
`(0,*)
`
`(0,*)
`
`member of
`
`member of
`
`member of
`
`(1,*)
`
`(1,*)
`
`(1,*)
`
`VidStrata
`
`Figure 3: Database Schema of the VFDB
`
`(0,*)
`
`member of
`
`(1,*)
`
`3.3 System Architecture
`In the following, we describe the system architecture that integrates creation, editing, continuous
`presentation support, and content-based annotation of Continuous Long Fields in the OR-DBMS
`Informix Universal Server. Our system is based on a client/server architecture where the server is
`responsible for the storage of continuous and discrete data. The client is responsible for request-
`ing data from the server, for presenting them and for handling interactions with the user. The ar-
`
`9
`
`PAGE 9 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`chitecture is composed of four parts, the IUS, the VF DataBlade, the external storage manager
`and the CLF DataBlade. The CLF DataBlade in turn consists of three parts, the DBMS function-
`ality extending the OR-DBMS, the presentation support module located on the client, and the
`continuous transport module connecting the presentation support module and the external stor-
`age manager during a presentation. Figure 4 displays the system components and the different
`connections that exist between them. Additionally, an application is connected to outline the in-
`terfaces to the system.
`The “raw” multimedia data streams are stored in external storage servers, not in the OR-DBMS
`itself. This approach enables the use of specialized storage systems for multimedia objects. It is
`capable of meeting the data delivery demands posed by presentations of continuous media ob-
`jects. For example, an average compressed MPEG video requires the delivery of about 1.4 MB
`per second. We chose the Experimental Object Store (EOS) Storage Server from AT&T as exter-
`nal storage manager because it can handle arbitrarily large objects and it provides random access
`to them [BP94].
`
`Application
`
`(e.g. video on demand system)
`
`CLF DataBlade
`Continuous Object Management client side
`
`CLF DataBlade
`Continuous Transport Module client side
`
`Client
`
`discrete operators (SQL)
`
`(e.g. get, append)
`
`control
`
`data transport
`
`continuous data
`
`CLF DataBlade
`DBMS functionality
`
`Video Foundation DataBlade
`
`IUS
`
`Server
`
`CLF DataBlade
`Continuous Transport Module server side
`
`external storage manager
`
`Legend:
`
`Module
`
`Application
`
`Connection
`
`Figure 4: System Architecture
`
`The OR-DBMS manages storage and retrieval of all meta data related to the represented multi-
`media objects. This is achieved by extending the OR-DBMS with the VF DataBlade and CLF
`DataBlade. All discrete operations not subject to time constraints on the Continuous Long Fields
`are performed by means of the DBMS functionality of the CLF DataBlade. For non-discrete op-
`erations (e.g., time-critical operations like presentation) the OR-DBMS grants direct access to
`the object stored on the external storage manager to the Continuous Object Management module
`on the client.
`In addition to the supported DBMS functionality the CLF DataBlade offers continuous presenta-
`tion support on the client side. This is implemented by the Continuous Object Management
`(COM) module on the client and the Continuous Transport Module. The COM implements the
`L/MRP client buffer management described in section 2.2 and the adaptation techniques pres-
`ented in section 2.3 to support continuous presentation and fast reaction on user interactions dur-
`ing a presentation.
`
`10
`
`PAGE 10 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`The Transport Module connects the COM with the external storage manager. It implements fast
`data access, fast delivery over the network, and transport of data requests from the client to the
`server and vice versa. The Transport Module sets up a continuous data connection to the external
`storage manager utilizing an appropriate network. The data connection and a corresponding con-
`trol connection work in parallel to the other components of the DBMS and the application pro-
`gram. One data connection can consist of several logical channels to support the presentation of
`composed media. Using this connection, time-dependent data are transferred from the server to
`client buffers. The control connection is responsible for transferring user commands as well as
`system control commands.
`The application accesses a Continuous Long Field through the SQL interface of the IUS. By this
`means all editing functionality is performed on the objects. For presentation, the applications ini-
`tializes the CLF DataBlade COM module with the required meta data of the object to be pres-
`ented. The Transport module is initialized and the application is able to use the COM mecha-
`nisms to request data, to set quality of service parameters or communicate user interactions to the
`system.
`3.4 The Continuous Long Field DataBlade
`The CLF DataBlade: DBMS functionality
`The advantage of using the VF DataBlade is that it offers adequate support for meta data manage-
`ment The drawback is that the functionality for continuous media support is not adequate. One
`problem is that discrete operations on multimedia objects integrated into the Universal Server
`allow only very “simple” operations on byte granularity that resemble file access. Furthermore,
`the VF DataBlade supports only a limited number of functions which may be integrated into
`SQL, like new and copy. For the other class of realized functions, like open or close, only an in-
`tegration into other functions of the DBMS is possible, but the integration into SQL-Functions is
`not supported.
`The CLF DataBlade DBMS functionality integrates Continuous Long Fields into the Universal
`Server. Datatypes representing Continuous Long Fields are defined together with tables to store
`the CLF specific meta data. Functionality for creation and deletion as well as editing on CODU
`granularity is provided for Continuous Long Fields. A function for the creation of minor quality
`replicates of a Continuous Long Field is also provided. Additionally, the CLF DataBlade imple-
`ments the virtual storage interface for the external storage manager EOS. The interface to the
`external storage manager is a requirement of the VF DataBlade to enable the IUS to uniformly
`access all objects stored externally. However, as this part of the implementation has no relevance
`for the issues discussed in this paper, it will not be explained in further detail.
`In addition to the meta data tables provided by the VF DataBlade, special meta information is
`needed to support editing and presentation of Continuous Long Fields. The following list de-
`scribes this special information in detail:
`– name of the object
`– maximum size of the CODUs
`– indicator whether the CODU size is variable or not
`– original presentation speed in CODUs per second
`– type indicator distinguishing between video, audio and other media types
`– a set of keywords describing the object
`– the number of CODUs in the object
`– a list of replicates of minor quality used for adaptation
`In order to represent the meta data for the Continuous Long Fields, the database schema of the VF
`DataBlade was extended. Figure 5 displays the combined database schema of the VF and CLF
`DataBlade. In this figure, only the physical information tables of the VF DataBlade are shown
`since they are used to store meta data of Continuous Long Fields. The content-based information
`tables of the VF DataBlade are demonstrated in figure 3.
`
`11
`
`PAGE 11 of 17
`
`PETITIONERS' EXHIBIT 1010
`
`

`

`Two tables are defined as extensions to the schema of the VF DataBlade. The table CLF_codu-
`strings holds the meta data the user has to supply to enable the system to handle, edit, and present
`the Continuous Long Field correctly. The table CLFSYS_codustrings holds information that is
`generated and maintained by the system, e.g., the entry number_of_codus that will be updated by
`any insert or append operation.
`
`CLF_codustrings
`
`(1,1)
`
`system data
`
`(1,1)
`
`CLFSYS_codustrings
`
`(1,1)
`
`(1,1)
`
`general data
`
`physical data
`
`(1,1)
`VidMDGeneral
`
`(1,*)
`
`has copies
`
`(1,1)
`
`(1,1)
`
`VidMedia
`
`(1,1)
`
`media
`specific
`
`(0,1)
`
`(0,1)
`
`(0,1)
`
`VidPVideo
`
`VidPAudio
`
`VidPOther
`
`Tables from the Continuous Long Field DataBlades
`
`Tables from the Video Foundation DataBlade
`
`Figure 5: Database Schema of VF and CLF DataBlade
`
`create table CLF_codustrings of type CODUSTRING_Type
`(
`
`PRIMARY KEY (object_name),
`FOREIGN KEY (VideoID) references VidMDGeneral of the VF DataBlade
`FOREIGN KEY (MediaID)
`references VidMedia of the VF DataBlade
`
`);
`
`create table CLFSYS_codustrings of type CODUSTRING_SYSType
`(
`
`FOREIGN KEY (object_name) references CLF_codustrings
`
`);
`The row types describing the tables are ”CO