(12) United States Patent
`(10) Patent N0.:
`US 6,225,993 B1
`
`Lindblad et al.
`(45) Date of Patent:
`*May 1, 2001
`
`U5006225993B1
`
`(54) VIDEO ON DEMAND APPLET METHOD
`AND APPARATUS FOR INCLUSION OF
`MOTION VIDEO IN MULTIMEDIA
`DOCUMENTS
`
`(75)
`
`Inventors: Christopher Lindblad, Stanford, CA
`(US); Stephan E. Cachat, Taninges
`(FR)
`
`(73) Assignee: Sun Microsystems, 1119-, Palto Alto,
`CA (US)
`
`(*) Notice:
`
`This patent issued on a continued pros-
`ecution application filed under 37 CFR
`1. 53(d), and is subject to the twenty year
`patent
`term provisions of 35 U.S.C.
`15400(2).
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. NO’: 08/636’118
`(22)
`Filed:
`Apr. 22, 1996
`
`Int. Cl.7 ..................................................... H04N 7/173
`(51)
`(52) us. Cl.
`.............................. 345/327; 709/219; 348/7;
`348/10
`370/352 389
`(58) Field of Search
`370/485;487:345/327302328 525’ 526?
`348/6 7’ 10 12. 4553’1 4 ’1 4 2’ 5 1f
`395/200 32’ 2’00 3’3 2’00 47_'2(’)0 49’ 2'0’0 61?
`.
`’
`i
`’ 709,217 218 ’219 216
`’
`’
`’
`
`(56)
`
`References Cited
`US. PATENT DOCUMENTS
`*
`
`5,461,415
`5,491,800 *
`
`............................... 348/7
`10/1995 Wolf etal.
`2/1996 Goldsmith et al.
`.................. 395/800
`
`OTHER PUBLICATIONS
`
`Wong et al., “Synchronization in Specification—based Mul-
`timedia Presentations,” Software—Practice and Experience,
`vol. 26, No. 1, Jan. 1996, pp. 71—81.
`
`(L15t continued on next page.)
`
`Primary Examiner—Chris Grant
`(74) Attorney, Agent, or Firm—Conley, Rose & Tayon; B.
`Noel Kivlin
`
`(57)
`ABSTRACT
`A
`.d
`t'
`f
`t
`h' h
`t
`t
`compu er process W ic reques s s reams 0 mo ion V1 eo
`titles and decodesand displays the motionvideo signals of
`the stream for display in a computer display deVice is
`constructed in the form of an applet of a multimedia docu-
`ment viewer such as a WOfld Wide Web browser.
`Accordingly, a deSignerof multimedia documents. such as
`HTML pages can eaSily incorporate motion Video titles into
`such HTML pages by specifying a few parameters of a
`desired title or a desired portion of a title to be requested
`from a video server. The applet builds bit stream control
`signals from the specification of the title or the portion of the
`title. The bit stream control signals request transmission of
`the rifle or 4:: Portion of tale title: fro? a bit stream Steffi
`sue
`as a V1 60 server a“
`are 1“ a 0m aPPrOPHate 0r
`processing by the bit stream server. The applet transmits the
`bit stream control signals to the bit stream server .to thereby
`request that the bit stream server initiate transmiSSion of a bit
`stream representing the requested title or the requested
`portion of the title. The applet also .builds decoder control
`Signals from the speCification of the title or the portion of the
`title. The decoder control signals direct a bit stream decoder
`to receive the requested bit stream from the bit stream server
`and to decode a motion video signal from the bit stream. The
`applet transmits the decoder control signals to the decoder to
`cause the decoder to receive the bit stream and to decode the
`motion video signal from the bit stream.
`
`(L15t continued on next page.)
`FOREIGN PATENT DOCUMENTS
`
`0 680 213
`
`11/1995 (EP) .
`
`Memory M
`
`45 Claims, 3 Drawing Sheets
`.
`.
`Microfiche Appendix Included
`(1 Microfiche, 98 Pages)
`
`
`Computer
`
`System
`m
`
`
`“deg“ @
`“33m? +
`+ Deggge'
`
`
`
`
`
`
`
`
`
`
`
`
`@
`Multimedia
`_
`Document
`
`Viewer
`+
`A let
`&
`
`
`.
`& ->
`Prefisor
`Lag
`T
`1&6
`l
`
`
`
`
`
`
`
`
`
` t
`
`t
`
`r
`
`1 o
`
`120
`
`%—v—’
`130
`
`Petitioner Apple Inc. - Exhibit 1009, p. l
`
`Petitioner Apple Inc. - Exhibit 1009, p. 1
`
`

`

`US 6,225,993 B1
` Page 2
`
`US. PATENT DOCUMENTS
`
`Web,” 4th International World Wide Web Conference, Dec.
`
`........................ 348/13
`5,682,511 * 10/1997 Sposato etal.
`
`~~~~~~ 395/761
`577427768 *
`4/1998 Gennam etaL
`5,805,153 *
`9/1998 Nielsen ................................ 345/327
`OTHER PUBLICATIONS
`
`1995’ pp‘ 144'
`European Search Report, Application No. 97 30 2676,
`mailed Jul. 12’ 1999.
`
`Chen et al., “Real Time Video and Audio in the World Wide
`
`* cited by examiner
`
`Petitioner Apple Inc. - Exhibit 1009, p. 2
`
`Petitioner Apple Inc. - Exhibit 1009, p. 2
`
`

`

`US. Patent
`
`May 1, 2001
`
`Sheet 1 0f 3
`
`US 6,225,993 B1
`
` Na
`
`u‘lowioEos.
`
`5389.;
`
`.Bsano
`
`E2w>m
`
`aw
`
`Eiom82>
`
`omlm
`
`.0380
`
`fl
`
`£85235.
`
`EQEDOOD
`
`335:3:
`
`«CQEDOOD
`
`.mbS
`
`Now
`
`Petitioner Apple Inc. - Exhibit 1009, p. 3
`
`Petitioner Apple Inc. - Exhibit 1009, p. 3
`
`
`
`
`
`
`

`

`US. Patent
`
`May 1, 2001
`
`Sheet 2 0f 3
`
`US 6,225,993 B1
`
`%52mm82>
`
`H0.53gm
`
`hmnoomo
`
`fl
`
`$8523:
`
`E95500
`
`5305
`
`Na
`
`63%
`
`£85222
`
`- 8
`
`—a.~.—
`6.92:.
`
`
`
` 3mm...$394%EmEzooo
`
`Petitioner Apple Inc. - Exhibit 1009, p. 4
`
`Petitioner Apple Inc. - Exhibit 1009, p. 4
`
`

`

`US. Patent
`
`May 1, 2001
`
`Sheet 3 0f 3
`
`US 6,225,993 B1
`
`.2680
`
`8:98
`
`5330
`
`<
`
`Now
`
`vow
`
`mom
`
`mom
`
`05
`
`N5
`
`Sum
`
`9am
`
`mwm
`
`omm
`
`%%
`
`SN
`
`m.9“.
`
` %IEI
`
`Petitioner Apple Inc. - Exhibit 1009, p. 5
`
`Petitioner Apple Inc. - Exhibit 1009, p. 5
`
`
`
`

`

`US 6,225,993 B1
`
`1
`VIDEO ON DEMAND APPLET METHOD
`AND APPARATUS FOR INCLUSION OF
`MOTION VIDEO IN MULTIMEDIA
`DOCUMENTS
`
`CROSS REFERENCE TO MICROFICHE
`APPENDIX
`
`is a
`Appendix A, which is a part of this disclosure,
`microfiche appendix consisting of one (1) sheet of micro-
`fiche having a total of ninety-eight (98) frames. Microfiche
`Appendix A is a list of computer programs and related data
`in one embodiment of the present
`invention, which is
`described more completely below.
`
`SPECIFICATION
`
`A portion of the disclosure of this patent document
`contains material which is subject to copyright protection.
`The copyright owner has no objection to the facsimile
`reproduction by anyone of the patent document or the patent
`disclosure, as it appears in the Patent and Trademark Office
`patent file or records, but otherwise reserves all copyright
`rights whatsoever.
`
`FIELD OF THE INVENTION
`
`The present invention relates to computer graphical dis-
`play of motion video and, in particular, to a method and
`apparatus for facilitating inclusion of motion video in mul-
`timedia computer displays.
`
`BACKGROUND OF THE INVENTION
`
`Video servers, including networked video servers, trans-
`mit “bit streams” to a video client. Such bit streams, which
`are sometimes referred to as “streams,” generally represent
`video and/or audio signals which represent titles in a library
`of multimedia sources. Examples of titles of such a library
`typically include recordings of motion pictures. In general,
`a video server receives from a video client a request for a
`particular title and transmits a stream of the particular title
`to the video client. An example of a video client is a set top
`box which is generally known and which decodes the stream
`received from the video server and transmits the decoded
`
`signal to a connected television. The requesting of a par-
`ticular title, receiving the stream of the particular title, and
`decoding the stream for display on a television are collec-
`tively and generally referred to as video on demand.
`Examples of such video on demand servers are described
`in US. Pat. No. 5,862,450 by Kallol Mandal and Steven
`Kleiman and entitled “Method and Apparatus for Delivering
`Simultaneous Constant Bit Rate Compressed Video Streams
`at Arbitrary Bit Rates with Constrained Drift and Jitter”
`(hereinafter the ’450 Patent) and in US. patent application
`Ser. No. 08/572,648, filed Dec. 14, 1995 by Kallol Mandal
`and Steven Kleiman and entitled “Method and Apparatus for
`Distributing Network Bandwidth on a Video Server for
`Transmission of Bit Streams Across Multiple Network Inter-
`faces Connected to a Single Internet Protocol (IP) Networ ”
`(hereinafter the ’648 Application). Both the ”639 Patent and
`the ’648 Application are incorporated herein in their entirety
`by reference.
`The popularity of the Internet global network is growing
`extremely rapidly, and perhaps the most popular protocol of
`the Internet is the Hyper Text Transfer Protocol (HTTP) of
`the World Wide Web. According to the HTTP protocol of the
`World Wide Web, documents, which are generally referred
`to as “pages,” incorporate text, graphical images, sound, and
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`motion video which, when viewed, form a multimedia
`presentation to user. Such pages are typically viewed using
`a World Wide Web browser, which is a computer process
`capable of retrieving HTTP pages and presenting the con-
`tents of such pages to a user of a computer system through
`output devices such as a computer video display device and
`a computer audio circuit coupled to one or more audio
`speakers. An example of a World Wide Web browser is the
`Netscape browser available from Netscape Communications
`Corporation of Mountain View, Calif.
`To display motion video, conventional browsers typically
`(i) transfer to the computer system in which the browser
`executes an entire data file which includes data representing
`a title and (ii) subsequently initiate execution of a player
`computer process which displays the title to the user on a
`computer display device. The player computer process is
`separate from the browser and therefore displays the motion
`video of the title outside of the page displayed by the
`browser. In addition, transferring the entire data file prior to
`displaying the motion video of the title delays substantially
`the display of the motion video since such data files are
`typically quite large, e.g., typically 1.8 gigabytes of data to
`represent a two-hour, VHS-quality motion picture.
`Currently, no browser is capable of seamlessly integrating
`motion video streams into a page of the World Wide Web.
`SUMMARY OF THE INVENTION
`
`invention, a computer
`In accordance with the present
`process which requests streams of motion video titles and
`decodes and displays the motion video signals of the stream
`for display in a computer display device is constructed in the
`form of an applet of a multimedia document viewer such as
`a World Wide Web browser. Accordingly, a designer of
`multimedia documents such as HTML pages can easily
`incorporate motion video titles into such HTML pages by
`specifying a few parameters of a desired title or a desired
`portion of a title to be requested from a video server. The
`specification of the parameters is in the general form of a
`well-known parameter specification format dictated by the
`particular interface of the computer instruction language in
`which the applet is written.
`The applet builds bit stream control signals from the
`specification of the title or the portion of the title. The bit
`stream control signals request transmission of the title or the
`portion of the title from a bit stream server such as a video
`server and are in a form appropriate for processing by the bit
`stream server. The applet transmits the bit stream control
`signals to the bit stream server to thereby request that the bit
`stream server initiate transmission of a bit stream represent-
`ing the requested title or the requested portion of the title.
`The applet also builds decoder control signals from the
`specification of the title or the portion of the title. The
`decoder control signals direct a bit stream decoder to receive
`the requested bit stream from the bit stream server and to
`decode a motion video signal from the bit stream. The applet
`transmits the decoder control signals to the decoder to cause
`the decoder to receive the bit stream and to decode the
`
`motion video signal from the bit stream.
`By using an applet of a multimedia document viewer to
`request and control receipt by a decoder of a motion video
`bit stream and to control decoding of the motion video bit
`stream by the decoder, a designer of a multimedia document
`can easily and conveniently include motion video images in
`multimedia documents. In addition, since the applet trans-
`mits bit stream control signals to a video server, the motion
`video signals which can be incorporated into a multimedia
`
`Petitioner Apple Inc. - Exhibit 1009, p. 6
`
`Petitioner Apple Inc. - Exhibit 1009, p. 6
`
`

`

`US 6,225,993 B1
`
`3
`document are any such motion video signals stored in such
`a video server. Such video servers will likely include a large
`number and wide variety of motion video signals, thereby
`providing a wealth of motion video content for inclusion in
`multimedia documents.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a block diagram of a computer system which is
`connected to a video server through a network and which
`includes a multimedia document viewer which in turn
`
`10
`
`processes an applet to include motion video images in a
`representation of a multimedia document in accordance with
`the presenting invention.
`FIG. 2 is a block diagram showing the multimedia docu-
`ment viewer, applet, and video server of FIG. 1 in greater
`detail.
`
`FIG. 3 is a block diagram of an applet tag of FIG. 2 in
`greater detail.
`FIG. 4 is a block diagram of the applet of FIG. 2 in greater
`detail.
`
`DETAILED DESCRIPTION
`
`In accordance with the present invention, a multimedia
`document 206 (FIG. 2) includes an applet tag 214 which
`causes a multimedia document viewer 202 to execute an
`
`applet 212. Execution of applet 212 requests transmission of
`a bit stream of a particular title from a video server 250 and
`controls receipt and decoding of the bit stream by a decoder
`204. Decoder 204, in response to control signals received
`from applet 212, decodes the received bit stream to produce
`a motion video image and displays the motion video image
`as an integral part of the representation of multimedia
`document 206. To include a motion video image as an
`integral part of a multimedia document, a designer of the
`multimedia document simply includes in the multimedia
`document an applet tag, e.g., applet tag 214, which specifies
`(i) applet 212, (ii) video server 250 as the source of a bit
`stream, and (iii) the particular bit stream to request from
`video server 250. A brief description of the operating envi-
`ronment of multimedia document viewer 202 and applet 212
`facilitates appreciation of the present invention.
`FIG. 1 is a block diagram of a computer system 100 which
`is generally of the architecture of most computer systems
`available today. Computer system 100 includes a processor
`102 which fetches computer instructions from a memory
`104 through a bus 106 and executes those computer instruc-
`tions.
`In executing computer instructions fetched from
`memory 104, processor 102 can retrieve data from or write
`data to memory 104, display information on one or more
`computer display devices 130, or receive command signals
`from one or more user-input devices 120. Processor 102 can
`be, for example, any of the SPARC processors available
`from Sun Microsystems, Inc. of Mountain View, Calif.
`Memory 104 can include any type of computer memory
`including, without limitation, randomly accessible memory
`(RAM), read-only memory (ROM), and storage devices
`which include magnetic and optical storage media such as
`magnetic or optical disks. Computer 100 can be,
`for
`example, any of the SPARCstation workstation computer
`systems available from Sun Microsystems, Inc. of Mountain
`View, Calif.
`Sun, Sun Microsystems, the Sun Logo, Java and Hot Java
`are trademarks or
`registered trademarks of Sun
`Microsystems, Inc. in the United States and other countries.
`All SPARC trademarks are used under license and are
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`trademarks of SPARC International, Inc. in the United States
`and other countries. Products bearing SPARC trademarks
`are based upon an architecture developed by Sun
`Microsystems, Inc.
`Computer display devices 130 can include generally any
`computer display device such as a printer, a cathode ray tube
`(CRT),
`light-emitting diode (LED) display, or a liquid
`crystal display (LCD). User input devices 120 can include
`generally any user input device such as a keyboard, a
`keypad, an electronic mouse, a trackball, a digitizing tablet,
`thumbwheels, a light-sensitive pen, a touch-sensitive pad, or
`voice-recognition circuitry.
`Computer system 100 also includes network access cir-
`cuitry 140 which is coupled to processor 102 and memory
`104 through bus 106 and which is coupled to a network 150.
`In accordance with control signals received from processor
`102 through bus 106, network access circuitry 140 coordi-
`nates transfer of data through network 150 between network
`access circuitry 140 and similar network access circuitry
`(not shown) in computer 100B or other computer systems
`coupled to computer system 100 through network 150. The
`transfer of data through network 150 is conventional. Since
`a video stream representing a VHS-quality motion picture
`encoded in MPEG-1 format has a bit rate of approximately
`1.5 Mbit/second to 2 Mbit/second, a useful minimum thresh-
`old is that network access circuitry 140 is capable of
`receiving data at a rate of at least 2 Mbit/second. Higher
`quality motion video images have bit rates as high as 8
`Mbit/second or higher. Therefore, in one embodiment, net-
`work access circuitry 140 is capable of receiving data at a
`rate of at least 8 Mbit/second. Network access circuitry 140
`can be generally any circuitry which is used to transfer data
`between a computer system and network such as computer
`system 100 and network 150 and can be, for example, an
`Ethernet controller chip.
`Anumber of computer processes execute in processor 102
`from memory 104, including a multimedia document viewer
`202 and a decoder 204. Multimedia document viewer 202 is
`
`a computer process which reads a multimedia document 206
`and displays the multimedia information specified in mul-
`timedia document 206 in one or more of computer display
`devices 130. In one embodiment, multimedia document 206
`is a document in HTML format and multimedia document
`viewer 202 is an IITML viewer such as the Netscape World
`Wide Web browser available from Netscape Communica-
`tions Corporation of Mountain View, Calif. Multimedia
`document viewer 202 and multimedia document 206 are
`
`shown in greater detail in FIG. 2.
`Multimedia document viewer 202 retrieves data and tags
`from a multimedia document such as multimedia document
`
`206. A tag is data which is not itself substantive content of
`a multimedia document but instead provides format infor-
`mation and can include specification of substantive content
`which is to be included in the multimedia document and
`
`which is located in memory 104 outside of multimedia
`document 206. For example, a tag can specify a file stored
`in memory 104 as containing a graphical image which is to
`be included as substantive content of multimedia document
`
`206. The data and tags of multimedia document 206 collec-
`tively define the composition, including substantive content
`and formatting, of multimedia document 206; and multime-
`dia document viewer 202 displays such substantive content
`in one or more of computer display devices 130 (FIG. 1) in
`accordance with the data and tags of multimedia document
`206. In one embodiment, multimedia document 206 is an
`HTML document, and the data and tags of multimedia
`document 206 comport with the HTML language. Multime-
`
`Petitioner Apple Inc. - Exhibit 1009, p. 7
`
`Petitioner Apple Inc. - Exhibit 1009, p. 7
`
`

`

`US 6,225,993 B1
`
`5
`dia document 206 includes an applet tag 214 (FIG. 2) which
`specifies an applet 212 and a number of operational char-
`acteristics of applet 212 as described more completely
`below.
`
`Multimedia document viewer 202 includes an applet
`interpreter 210 which retrieves from applet 212 computer
`instructions and translates such computer instructions into
`computer instructions of a form appropriate for execution by
`processor 102 (FIG. 1) and submits the translated computer
`instructions to processor 102 for execution.
`In one
`embodiment, applet interpreter 210 (FIG. 2) translates and
`submits for execution a single computer instruction of applet
`212 prior to translation and submission for execution of a
`subsequent computer instruction of applet 212. Applet inter-
`preter 210 can be, for example, the Java applet interpreter or
`the Hot Java World Wide Web browser available from Sun
`
`Microsystems, Inc. and, in such an embodiment, applet 212
`comports with the Java computer instruction language inter-
`preted by the Java applet interpreter. As described more
`completely below, applet 212 is a novel applet which, when
`executed by processor 102 (FIG. 1) through applet inter-
`preter 210 (FIG. 2), requests a title from a video server 250
`and causes the received bit stream representing the requested
`title to be decoded in a decoder 204 and displayed in a
`computer display device as an integral part of a multimedia
`display of multimedia document 206.
`In executing the computer instructions of applet 212,
`applet interpreter 210 transmits, through network 150 (FIG.
`1), control signals to an applications programming interface
`(API) 252 (FIG. 2) of a video server 250 which executes
`within a computer system 160 (FIG. 1).
`Illustrative
`examples of video server 250 of computer system 160 are
`described in the ’450 Patent and the ”648 Application. API
`252 (FIG. 2) of video server 250 implements a remote
`procedure calling (RPC) protocol in which API 252 controls
`video server 250 in response to control signals received by
`API 252. For example, in response to control signals which
`request a title and which are transmitted to AP1252 by applet
`interpreter 210, API 252 causes a bit pump 254 of video
`server 250 to initiate transmission through network 150
`(FIG. 1) to decoder 204 (FIG. 2) of a bit stream representing
`the requested title. In addition, API 252 can transmit to
`applet interpreter 210 status information regarding a title
`stored within video server 250 or regarding a bit stream
`transmitted by bit pump 254 in response to control signals
`requesting such status information.
`Decoder 204 is a computer process executing within
`processor 102 (FIG. 1) from memory 104. Decoder 204
`receives data representing a motion video display encoded in
`a particular format. In one embodiment, decoder 204 is the
`MPEG Expert (MPX) decoder available from Applied
`Vision and decodes motion video signals according to the
`MPEG-1 encoding format. Applet interpreter 210 transmits
`to decoder 204 control signals which control the decoding
`by decoder 204 of the bit stream received from bit pump 254
`of video server 250. Specifically, applet
`interpreter 210
`transmits to decoder 204 control signals directing decoder
`204 to start or stop decoding the bit stream received from bit
`pump 254 or specifying characteristics of the bit stream
`received from bit pump 254 such as the bit rate, encoding
`format, and the coordinates of a particular location within
`one or more of computer display devices 130 (FIG. 1) in
`which to display the decoded motion video images.
`In
`addition, applet 212 determines which communications port
`through network access circuitry 140 (FIG. 1) the bit stream
`is to be received and transmits to decoder 204 (FIG. 2)
`control signals identifying the selected communications
`
`6
`port. Applet 212 can therefore determine which communi—
`cations ports are used by other applications and can avoid
`conflicts resulting from access of decoder 204 of a commu-
`nications port by selecting a communications port which is
`not used by another computer process of computer system
`100 (FIG. 1).
`Applet tag 214 is shown in greater detail in FIG. 3. Applet
`tag 214 includes a number of fields which collectively define
`a bit stream to be received and decoded for display by
`decoder 204 (FIG. 2). A field is a collection of data which
`collectively define a item of information. Applet tag 214
`includes (i) an applet identifier field 302, (ii) a width field
`304, (iii) a height field 306, (iv) a server identifier field 308,
`and (v) an encoding format field 310. Applet tag 214 can also
`include any of the following optional fields: (vi) a title field
`312, (vii) an image field 314, (viii) a play/pause field 316,
`(ix) a start field 318, and (x) a duration field 320.
`Applet identifier field 302 specifies applet 212 as the
`applet to be retrieved and executed by applet interpreter 210.
`Width field 304 and height field 306 specify the width and
`height, respectively, in display coordinate space of a com-
`puter display device, i.e., specify the size of the viewport in
`which the decoded motion video image is displayed. Server
`identifier field 308 specifies video server 250 (FIG. 2) as the
`source of the desired bit stream. Encoding format field 310
`(FIG. 3) specifies the particular encoding format, e.g.,
`MPEGleS encoding format, of the bit stream received by
`decoder 204 (FIG. 2). Title field 312 (FIG. 3) specifies the
`particular title to be retrieved from server 250 (FIG. 2).
`Alternatively, title field 312 can specify the address of a
`multicast bit stream.
`
`Image field 314 (FIG. 3), if included, specifies a still video
`image to be displayed in the space specified by width field
`304 and height field 306 if the title specified by title field 312
`is unavailable. Play/pause field 316, if included, specifies
`whether the motion video image received from video server
`250 (FIG. 2) is initially in a play state or in a paused state.
`Start field 318 (FIG. 3), if included, specifies an offset into
`the title of a portion of the title, i.e., the point within the title
`at which the bit stream should begin. For example, start field
`318 can specify that the requested bit stream begin at 3
`minutes and 10 seconds into the title. Duration field 320, if
`included specifies the duration of a desired portion of the
`title. For example, duration field 320 can specify that a
`30-minute portion of the title is requested.
`In one
`embodiment, start field 318 and duration field 320 are
`specified in terms of an integer number of nanoseconds.
`Thus, by specifying the few fields described above and
`shown in FIG. 3, a designer of multimedia document 206
`can include as an integral part of multimedia document 206
`a motion video image retrieved from video server 250. The
`following is an illustrative example of applet tag 214 in
`HTML format.
`
`<applet code=“SunMediaCenterPlayer.class” width=704
`height=520>
`param name=port value—“1973”>
`<param name=format value=“MPEG1$YS”>
`<param name=host value=“sqas-6”>
`<param name=img value“/images/bkgx.gif”>
`</applet>
`includes computer instructions
`Applet 212 (FIG. 2)
`which, when executed, request a title from video server 250
`and control decoding and display of the decoded motion
`video signals by decoder 204 and is shown in greater detail
`in FIG. 4. The computer instructions of applet 212 are
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Petitioner Apple Inc. - Exhibit 1009, p. 8
`
`Petitioner Apple Inc. - Exhibit 1009, p. 8
`
`

`

`US 6,225,993 B1
`
`7
`organized into various levels, each of which defines a
`respective component of the behavior of applet 212. Applet
`212 includes a player level 402, an API level 404, a decoder
`level 406, and a detailed decoder level 408.
`Player level 402 includes computer instructions which,
`when executed,
`implement a graphical user interface in
`which a user can control the bit stream received from video
`
`server 250 (FIG. 2) and the display of the decoded motion
`video signals of the bit stream by physical manipulation of
`one or more of user input devices 120 (FIG. 1). In one
`embodiment, the computer instructions of player level 402
`(FIG. 4), when executed, cause graphical and/or textual
`representation of control mechanisms to be displayed in one
`or more of computer display devices 130 (FIG. 1). Such
`control mechanisms are known and conventional and
`include, without
`limitation, virtual buttons, pull-down
`menus, virtual radio buttons, virtual check boxes, and slid-
`ing scroll bars. In a conventional manner, a user activates
`one or more of such control mechanisms by physical
`manipulation of one or more of user input devices 120 (FIG.
`1) and such physical manipulation results in receipt by
`player level 402 (FIG. 4) of applet 212 of signals and/or data
`representing such activation.
`API
`level 404 includes computer instructions which,
`when executed, implement the RPC protocol of API 252
`(FIG. 2) of video server 250 and invoke RPC calls to API
`252 to control the bit stream transmitted by bit pump 254 in
`accordance with interaction of a user with the graphical user
`interface implemented by player level 402 (FIG. 4).
`Decoder level 406 and detailed decoder level 408 collec-
`
`tively control operation of decoder 204 (FIG. 2), generally
`controlling the decoding of the bit stream received from
`video server 250 by decoder 204 and the display in a
`computer display device of the decoded motion video
`image. Decoder level 406 includes computer instructions
`and data structures which are not specific to any particular
`decoder, while detailed decoder level 408 includes computer
`instructions and data structures which are specific to decoder
`204. It is generally preferred that detailed decoder level 408
`is as small and simple as possible such that the majority of
`computer instructions of decoder levels 406 and 408 are
`included in decoder level 406. Accordingly, adapting applet
`212 (FIG. 2) to operate in conjunction with a decoder other
`than decoder 204 requires modification of only detailed
`decoder level 408 and, therefore, as little modification as
`possible.
`Microfiche Appendix A is a computer source code listing
`of a preferred embodiment of applet 212. The modules of
`Microfiche Appendix A are written in the Java applet com-
`puter instruction language developed by Sun Microsystems,
`Inc. of Mountain View, Calif. The computer instructions of
`the Java applet computer instruction language are object-
`oriented, and each of the modules of Microfiche Appendix
`A represents a respective class of objects. Player level 402
`(FIG. 4),
`in this embodiment,
`includes classes
`SunMediaCenterPlayer, Player, and PositionSlider as
`defined in the computer source code listing of Microfiche
`Appendix A. API level 404, in this embodiment, includes
`classes MsmPlayer, MsmSession, MsmAccessRight,
`MsmPersistence, MsmPlaylist, MsmToString, MsmItem,
`MsmTitleItem, MsteadAirItem, MsmException,
`XdrBlock, and PortMapper as defined in the computer
`source code listing of Microfiche Appendix A. Decoder level
`406,
`in this embodiment,
`includes classes Decoder and
`DecoderImpl as defined in the computer source code listing
`of Microfiche Appendix A. Detailed decoder level 408, in
`this embodiment,
`includes class MprecoderImpl as
`defined in the computer source code listing of Microfiche
`Appendix A.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`invention
`In the preferred embodiment of the present
`defined by Microfiche Appendix A, a module “loop”
`includes computer instructions of the C computer instruction
`language and defines a loop computer process which
`executes independently of multimedia document viewer 202
`(FIG. 2). The loop computer process cooperates with mul-
`timedia document viewer 202 and decoder 204 to request
`and receive from video server 250 bit streams representing
`multicast motion video signals.
`The above description is illustrative only and is not
`limiting. The present invention is therefore defined solely
`and completely by the appended claims together with their
`full scope of equivalents.
`What is claimed is:
`
`1. An applet, capable of executing within a computer
`system for requesting and controlling decoding of a bit
`stream specified in a multimedia document stored in a
`memory of the computer system, the applet comprising:
`an API module; and
`a decoder module operatively coupled to the API module;
`wherein the API module is configured to build, from a
`specification of the bit stream in the multimedia
`document, bit stream control signals which request
`transmission of the bit stream from a bit stream server
`
`and which are in a form appropriate for processing by
`the bit stream server;
`wherein the API module is further configured to transmit
`the bit stream control signals to the bit stream server to
`thereby request
`the bit stream from the bit stream
`server;
`
`wherein the decoder module is configured to build, from
`the specification of the bit stream in the multimedia
`document, decoder control signals which direct a
`decoder to receive the bit stream from the bit stream
`
`server and which are in a form appropriate for process-
`ing by the decoder; and
`wherein the decoder module is further configured to
`transmit the decoder control signals to the decoder to
`thereby cause the decoder to receive and decode the bit
`stream.
`
`2. The applet of claim 1,
`wherein the decoder module is further configured to build,
`from the specification of the bit stream in the multi-
`media document, display control signals which direct
`the decoder to perform one or more display control
`actions for controlling a display of motion video
`decoded by the decoder in at least one of one or more
`display devices coupled to the computer system,
`wherein the dis