(12) United States Patent
`Carmel et al.
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 6,389,473 B1
`May 14, 2002
`
`US006389473B1
`
`(54) NETWORK MEDIA STREAMING
`
`5,404,446 A
`
`4/1995 Bowater a a1. .......... .. 345/537
`
`(75) Inventors: Sharon Carmel; Tzur Daboosh, both
`of Giv’atayim; Eli Reifman, Rishon le
`Zion; Naftali Shani; Ziv Eliraz, both
`of Tel Aviv; Dror Ginsberg, Karkur;
`Edan Ayal, Kfar Saba, all of (IL)
`
`(73) Assignee: Geo Interactive Media Group Ltd.,
`Givatayim (IL)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`USC 154(b) by 0 days.
`
`(21) Appl. No.: 09/275,703
`(22) Filed:
`Mar. 24, 1999
`(30)
`Foreign Application Priority Data
`
`Mar. 24, 1998
`
`(IL) .............................................. .. 123819
`
`(51) Int. Cl.7 .............................................. .. G06F 13/00
`(52) US. Cl. ..................................................... .. 709/231
`(58) Field of Search ..................... .. 707/500.1; 709/200,
`709/231, 236, 246, 247; 382/236, 239;
`375/24012
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,267,334 A 11/1993 Normille et a1. .......... .. 382/236
`
`5,841,432 A 11/1998 Carmel et a1. ......... .. 707/5001
`
`Primary Examiner—Robert B. Harrell
`(74) Attorney, Agent, or Firm—Ladas & Parry
`
`(57)
`
`ABSTRACT
`
`A method for real-time broadcasting from a transmitting
`
`computer to one or more client computers over a network,
`including providing at the transmitting computer a data
`stream having a given data rate, and dividing the stream into
`a sequence of slices, each slice having a predetermined data
`siZe associated therewith. The slices are encoded in a
`corresponding sequence of ?les, each ?le having a respec
`tive index, and the sequence is uploaded to a server at an
`upload rate generally equal to the data rate of the stream,
`such that the one or more client computers can download the
`
`sequence over the network from the server at a download
`rate generally equal to the data rate.
`
`41 Claims, 11 Drawing Sheets
`
`Micro?che Appendix Included
`(2 Micro?che, 138 Pages)
`
`STANDARD
`NETWORK
`SERVER
`
`28
`
`PAGE 1 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14, 2002
`
`Sheet 1 0f 11
`
`US 6,389,473 B1
`
`26
`
`24w
`
`BROADCAST
`
`- SERVER
`
`REALTIME ENCODE
`
`ZO/
`
`FIG. 1 PRIOR ART
`
`PAGE 2 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`U.S. Patent
`
`May 14, 2002
`May 14, 2002
`
`Sheet 2 0f 11
`Sheet 2 of 11
`
`US 6,389,473 B1
`US 6,389,473 B1
`
`2
`
`FIG.
`
`PAGE 3 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14, 2002
`
`Sheet 3 0f 11
`
`US 6,389,473 B1
`
`SUCE
`1
`
`SUCE
`2
`
`SUCE
`3
`
`SLICE
`4
`
`K42
`/
`40
`
`K44
`
`T46
`
`\48
`
`ME
`
`50 /
`
`FIG. 3B
`
`SUCE N M 52
`
`MESSAGE
`
`54
`_/
`
`FIG. 3C
`
`56
`g
`
`J J+1 J+2 J+Z>
`
`.. .
`
`N
`
`PAGE 4 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`2002491yaM
`
`Sheet 4 of 11
`
`US 6,389,473 B1
`
`PAGE 5 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14, 2002
`
`Sheet 5 0f 11
`
`US 6,389,473 B1
`
`ow
`
`N@\| N x2:
`
`
`
`ww) m x2: v
`
`‘lcqh w x2: I‘
`
`IL
`
`w .01
`
`PAGE 6 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14, 2002
`
`Sheet 6 0f 11
`
`US 6,389,473 B1
`
`FIG. 5
`
`CONNECT
`TO SERVER
`
`I
`
`INPUT
`BROACAST
`DATA
`
`~
`
`ENCODE
`
`80
`NJ
`
`SLICE
`
`82
`
`II
`
`FTP TO SERVER 84
`
`UPDATE
`INDEX
`FILE
`
`'\
`86
`
`II
`
`CHECK
`LINK
`FUNCTION
`
`—\
`88
`
`PAGE 7 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 7 0f 11
`
`US 6,389,473 B1
`
`FIG. 6A
`
`CONNECT
`TO SERVER
`
`I
`HTTP FROM
`SERVER
`
`I
`
`READ INDEX
`FILE
`I
`SELECT SLICE
`
`II
`DECODE
`I
`OUTPUT
`DATA
`
`CONNECT
`NEW LINK
`l
`
`CONTINUE
`
`PAGE 8 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14, 2002
`
`Sheet 8 0f 11
`
`US 6,389,473 B1
`
`CONTINUE
`
`FIG
`
`CONNECT
`TO SERVER
`
`HTTP FROM
`SERVER
`I
`READ HEADER
`
`I
`CHOOSE
`LEVEL
`
`I
`
`READ SLICE
`I
`DECODE
`I
`OUTPUT
`DATA
`I
`DETERMINE
`LINK RATE
`
`CHOOSE
`LOWER
`AUDIO/ VIDEO
`LEVEL
`
`CHOOSE
`HIGHER
`AUDIO/VIDEO
`LEVEL
`
`PAGE 9 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14, 2002
`
`Sheet 9 0f 11
`
`US 6,389,473 B1
`
`FIG. 7
`
`INPUT DATA
`
`80
`
`82
`
`SET COMLRESSQIQO ..__ CONTROL
`RATTD
`__
`FROM 88
`+
`COMPRESS DATA
`
`/92
`1
`SET SLICE DURATION _____gggbTARgg
`PREgARE
`SL|C+EI —’ 1:1“
`
`TO FTP 84
`
`PAGE 10 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14, 2002
`
`Sheet 10 0f 11
`
`US 6,389,473 B1
`
`FIG. 8
`
`84
`
`FROM
`SLICE 82
`
`94 FTP OPEN ___L|NK TIME-OUT FROM CHECK 88
`\ LINK J
`=
`
`V
`FTP SEND ‘
`94
`\ SLICE I
`
`NEXT
`LINK
`
`U
`
`I=I+1
`
`J=J+1
`
`YES
`
`NO
`
`v
`
`TO UPDATE
`INDEX 86
`
`PAGE 11 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 11 0f 11
`
`US 6,389,473 B1
`
`88
`\
`
`FROM
`UPDATE 86
`
`CHECK SLICE
`TIME TSL
`
`FIG. 9
`
`TSL > TMAX
`
`RE-INITIALIZE ‘
`LINK
`
`INCREASE
`COMPRESSION
`
`TMIN<TSL< TMAX
`
`TO SET
`COMPRESSION 9O
`
`DECREASE
`SLICE DURATION
`
`_" TO SET
`——-DURATION
`92
`
`INCREASE
`1
`' SUCE DURATION l
`
`L- _ _ _ _ _ _ _ __l
`
`NO
`
`NEXT SLICE
`
`TO ENCODE
`
`PAGE 12 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`US 6,389,473 B1
`
`1
`NETWORK MEDIA STREAMING
`
`A computer printout is attached hereto as an appendix in
`micro?che form and is incorporated herein by reference. The
`printout comprises executable program ?les in hexadecimal
`format. This appendix includes 2 micro?ches, containing a
`total of 138 frames.
`
`FIELD OF THE INVENTION
`
`The present invention relates generally to netWork data
`communications, and speci?cally to real-time multimedia
`broadcasting over a netWork.
`
`BACKGROUND OF THE INVENTION
`
`In netWork broadcasting, data are transmitted over a
`netWork in real time from a single transmitting computer to
`a plurality of clients simultaneously. The netWork may be a
`LAN, a WAN, an intranet or a public netWork such as the
`Internet. NetWork broadcasting is most commonly used to
`stream multimedia data, typically comprising images and
`sound.
`FIG. 1 is a schematic illustration shoWing a real-time
`broadcasting system 20, as is knoWn in the art. One or more
`input devices 22 (for example, a video camera and/or
`microphone) are used to generate a multimedia data stream
`representing an entertainment or informational program to
`be transmitted to a plurality of clients 30 via a netWork 28.
`Because of bandWidth limitations of the netWork, the data
`stream from host 22 must ?rst be compressed by a real-time
`encoder 24 and then routed to appropriate clients 30 by a
`broadcast server 26 (since not all clients on the netWork are
`necessarily intended to receive the broadcast).
`Encoder 24 and server 26 typically comprise high-cost,
`dedicated computer systems, such as a Sun Station
`(produced by Sun Microsystems) or a WindoWs NT server,
`running suitable RealSystem 5.0 softWare (produced by
`RealNetWorks Inc., Seattle, Wash.). These dedicated sys
`tems are required in order to ensure that the data stream is
`distributed and received by clients 30 in real time. Similarly,
`host 22 must typically be connected directly to encoder 24
`by a high-speed data link or LAN, and not via the Internet
`or other narroWband netWork. Therefore, real-time broad
`casting is normally possible only for hosts having a suitable,
`dedicated encoder and broadcast server and cannot be
`offered by Internet service providers (ISPs) to their general
`clientele.
`
`SUMMARY OF THE INVENTION
`
`It is an object of some aspects of the present invention to
`provide substantially continuous, high-bandWidth data
`streaming over a netWork using common, existing server
`and netWork infrastructure.
`It is a further object of some aspects of the present
`invention to provide data broadcasting capability, particu
`larly for multimedia data, Without the need for a dedicated
`broadcast computer system.
`It is a further object of some aspects of the present
`invention to provide apparatus and methods for data broad
`casting at reduced cost by comparison With systems knoWn
`in the art.
`It is still another object of some aspects of the present
`invention to enable a personal computer to remotely broad
`cast a multimedia program through an Internet service
`provider (ISP) using common, universally-supported Inter
`net communication protocols.
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`2
`In preferred embodiments of the present invention, a
`transmitting computer generates a data stream and broad
`casts the data stream via a netWork server to a plurality of
`clients. The data stream is divided into a sequence of
`segments or slices of the data, preferably time slices,
`Wherein the data are preferably compressed. Each slice is
`preferably assigned a respective slice index. The transmit
`ting computer uploads the sequence of slices to the server
`substantially in real time, preferably using an Internet
`protocol, most preferably the File Transfer Protocol (FTP),
`as is knoWn in the art. The clients doWnload the data stream
`from the server, preferably using an Internet protocol, as
`Well, most preferably the Hypertext Transfer Protocol
`(HTTP), or alternatively, using other protocols, such as UDP
`or RTP, Which are similarly knoWn in the art. The clients use
`the slice indices of the frames to maintain proper synchro
`niZation of the playback. The division of the data stream into
`slices and the inclusion of the slice indices in the data stream
`to be used by the clients in maintaining synchroniZation
`alloWs the broadcast to go on substantially in real time
`Without the use of special-purpose hardWare.
`Preferably, each segment or slice is contained in a
`separate, respective ?le. Alternatively, the segments or slices
`may all be contained in a single indexed ?le, Which is
`streamed to the client in a series of packets, each covering
`a range of one or more indices. HTTP version 1.1 supports
`this sort of ?le streaming. Other protocols may also be used
`for this purpose.
`In some preferred embodiments of the present invention,
`the data stream comprises multimedia data captured or
`generated by the transmitting computer. The term “multi
`media” as used in the context of the present patent applica
`tion and in the claims refers to images or sound or to data
`representative of images or of sound or a combination
`thereof. Multimedia image data may include still images,
`video, graphics, animation or any combination thereof,
`including text displayed in conjunction thereWith. It Will be
`appreciated, hoWever, that the principles of the present
`invention may similarly be applied to streaming of other
`data types.
`Preferably, the transmitting computer compresses the
`frames in the data stream, most preferably using methods of
`image and audio compression such as those described in
`US. patent application Ser. No. 08/919,027, Which is
`assigned to the assignee of the present patent application and
`incorporated herein by reference. Alternatively, any suitable
`methods of compression knoWn in the art may be used. The
`compressed data are conveyed to the server and thence to the
`clients, Which decompress the data.
`In some preferred embodiments of the present invention,
`the transmitting computer and the clients monitor the
`uploading and doWnloading of data to and from the server,
`respectively, in order to determine the amount of time
`required to convey each slice and to verify that the slices are
`conveyed at a suf?cient rate. When the data stream com
`prises multimedia data, the data rate should be generally
`equal to or faster than the rate at Which the data are generated
`at the transmitting computer.
`In some of these preferred embodiments, the transmitting
`computer and/or the clients each open a plurality of FTP or
`HTTP links, respectively, With the netWork server. The slices
`are transferred over different ones of the links in alternation.
`Although typically none of the plurality of links has suf?
`cient bandWidth on its oWn to convey the entire data stream
`in real time, the combined bandWidths of the plurality of
`links are generally sufficient for this purpose. Preferably,
`
`PAGE 13 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`US 6,389,473 B1
`
`3
`each of the links is monitored to determine its speci?c data
`transfer rate. If the transfer rate of any of the links is below
`a predetermined minimum, that link is preferably closed,
`and a neW link is opened in its place.
`In other preferred embodiments, the slices are provided by
`the server at multiple resolution or quality levels. Each such
`level has a different degree of data compression, and thus
`corresponds to a different data bandWidth requirement. The
`client or the server monitors the data transfer rate of a data
`link opened therebetWeen and selects the level that is
`appropriate to the link bandWidth. If the monitored data
`transfer rate changes during transmission, the quality level is
`preferably reselected accordingly.
`Preferably, the transmitting computer monitors the band
`Width of the data stream that it is uploading to the server, and
`compares the data stream bandWidth to a knoWn or esti
`mated bandWidth of the link or links betWeen the transmit
`ting computer and the server. The transmitting computer
`preferably compresses the data stream at a compression ratio
`that is adjusted so as to match the data stream bandWidth to
`the available link bandWidth, using methods described, for
`eXample, in the above-mentioned US. patent application
`Ser. No. 08/919,027.
`There is therefore provided, in accordance With a pre
`ferred embodiment of the present invention, a method for
`real-time broadcasting from a transmitting computer to one
`or more client computers over a netWork, including:
`providing at the transmitting computer a data stream
`having a given data rate;
`dividing the stream into a sequence of slices, each slice
`having a predetermined data siZe associated thereWith;
`encoding the slices in a corresponding sequence of ?les,
`each ?le having a respective index; and
`uploading the sequence to a server at an upload rate
`generally equal to the data rate of the stream, such that
`the one or more client computers can doWnload the
`sequence over the netWork from the server at a doWn
`load rate generally equal to the data rate.
`Preferably, dividing the stream into the sequence of slices
`includes dividing the stream into a sequence of time slices,
`each having a predetermined duration associated thereWith.
`Preferably, uploading the sequence includes comparing
`the upload rate to the data rate and adjusting the upload rate
`responsive to the comparison. Further preferably, encoding
`the stream includes compressing data in the stream at a
`desired compression ratio, and adjusting the upload rate
`includes changing the compression ratio. Alternatively or
`additionally, adjusting the upload rate includes adjusting the
`siZe of one or more of the slices.
`Preferably, uploading the sequence includes opening a
`plurality of ?le transfer links betWeen the transmitting
`computer and the server, each link characteriZed by a
`respective link data rate, and uploading different ?les in the
`sequence over different ones of the plurality of links. Further
`preferably, opening the plurality of links includes opening
`links such that the data rates of the links taken together are
`sufficient to upload the sequence at the upload rate generally
`equal to the data rate.
`Preferably, uploading the sequence includes uploading a
`sequence using an Internet Protocol, most preferably using
`FTP.
`Preferably, the method includes doWnloading the
`sequence using an Internet protocol, most preferably HTTP,
`or alternatively, UDP or RTP, over the netWork from the
`server to the one or more client computers. Preferably, the
`one or more client computers decode the sequence and play
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`4
`back the data stream responsive to the indices of the ?les, at
`a replay rate generally equal to the data rate.
`Preferably, uploading the sequence includes uploading
`and updating an indeX ?le containing the indeX of the ?le in
`the sequence that Was most recently uploaded, and the one
`or more client computers read the indeX ?le to play back the
`sequence. In a preferred embodiment, doWnloading the
`sequence includes selecting a ?le in the sequence earlier
`than the ?le Whose indeX is contained in the indeX ?le and
`doWnloading at least a portion of the sequence of ?les
`beginning With the selected ?le.
`Preferably, the one or more client computers include a
`plurality of client computers, and doWnloading the sequence
`includes doWnloading to the plurality of client computers
`substantially simultaneously.
`Preferably, doWnloading the sequence includes opening a
`plurality of doWnload links betWeen one of the client com
`puters and the server, each link characteriZed by a respective
`link data rate, and doWnloading different ?les in the
`sequence over different ones of the plurality of links. Most
`preferably, opening the plurality of links includes opening
`links such that the data rates of the links taken together are
`suf?cient to doWnload the sequence at the doWnload rate
`generally equal to the data rate.
`Preferably, opening the plurality of links includes moni
`toring the data rates of the links and opening a neW link in
`place of one of the links having a data rate loWer than a
`predetermined level.
`In one preferred embodiment, opening the neW link
`includes retransmitting at least one of the ?les in the
`sequence, Wherein the at least one of the ?les Was incom
`pletely transmitted over the one of the links having the data
`rate loWer than the predetermined level.
`In another preferred embodiment, opening the neW link
`includes dropping at least one ?le out of the sequence,
`Wherein the at least one of the ?les Was incompletely
`transmitted over the one or more of the links having the data
`rate loWer than the predetermined level.
`In still another preferred embodiment, encoding the slices
`includes encoding slices at a plurality of different quality
`levels, such that the ?les corresponding to a given one of the
`slices have a different, respective data siZe for each of the
`quality levels. Preferably, doWnloading the sequence
`includes determining a data bandWidth of the netWork
`betWeen the server and the client computer and selecting one
`of the quality levels responsive to the determined band
`Width.
`Preferably, the data stream includes multimedia data.
`There is further provided, in accordance With a preferred
`embodiment of the present invention, apparatus for real-time
`broadcasting of a data stream having a given data rate over
`a netWork, including:
`a transmitting computer, Which divides the stream into a
`sequence of slices, each slice having a predetermined
`data siZe associated thereWith, and encodes the slices in
`a corresponding sequence of ?les, each ?le having a
`respective indeX, and
`Which uploads the sequence to a server at an upload rate
`generally equal to the data rate, such that one or more
`client computers can doWnload the sequence over the
`netWork from the server at a doWnload rate generally
`equal to the data rate.
`Preferably, the transmitting computer compares the
`upload rate to the data rate and adjusts the upload rate
`responsive to the comparison. Most preferably, the trans
`mitting computer compresses the data at a compression ratio
`Which is varied responsive to the comparison. Additionally
`
`PAGE 14 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`US 6,389,473 B1
`
`5
`or alternatively, the transmitting computer adjusts the siZe of
`one or more of the slices responsive to the comparison.
`Preferably, the transmitting computer opens a plurality of
`links betWeen the transmitting computer and the server, each
`link characterized by a respective data rate, and transmits
`different ones of the sequence of ?les over different ones of
`the plurality of links. Most preferably, the transmitting
`computer opens the plurality of links such that the data rates
`of the links taken together are sufficient to upload the
`sequence at the upload rate generally equal to the data rate.
`Further preferably, the transmitting computer monitors the
`data rates of the links and opens a neW link in place of one
`of the links Whose data rate is loWer than a predetermined
`level.
`In a preferred embodiment, the slices are encoded at a
`plurality of different quality levels, such that the ?les cor
`responding to a given one of the slices have a different,
`respective data siZe for each of the quality levels.
`Preferably, the transmitting computer uploads the
`sequence using an Internet upload protocol, most preferably
`FTP.
`Preferably, the one or more client computers decode the
`sequence and play back the data stream responsive to the
`indices thereof, at a data replay rate generally equal to the
`data rate. Preferably, the one or more client computers
`doWnload the encode sequence using an Internet doWnload
`protocol, most preferably HTTP or alternatively, UDP or
`RTP.
`Preferably, the one or more client computers include a
`plurality of client computers, Which doWnload the sequence
`substantially simultaneously.
`Preferably, the netWork includes the Internet.
`Further preferably, the data stream includes multimedia
`data, and the predetermined data siZe of each of the slices
`corresponds to a time duration of the slice.
`The present invention Will be more fully understood from
`the folloWing detailed description of the preferred embodi
`ments thereof, taken together With the draWings in Which:
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a schematic illustration of a computer broadcast
`netWork, as is knoWn in the art;
`FIG. 2 is a schematic illustration of a computer broadcast
`netWork, in accordance With a preferred embodiment of the
`present invention;
`FIG. 3A is a block diagram that schematically illustrates
`a data structure of a broadcast sequence, in accordance With
`a preferred embodiment of the present invention;
`FIG. 3B is a block diagram that schematically illustrates
`an indeX ?le associated With the data structure of FIG. 3B,
`in accordance With a preferred embodiment of the present
`invention;
`FIG. 3C is a schematic illustration of a user interface
`graphic, for use in conjunction With the data structure of
`FIG. 3A, in accordance With a preferred embodiment of the
`present invention;
`FIG. 3D is a block diagram that schematically illustrates
`a data structure of a broadcast sequence, in accordance With
`another preferred embodiment of the present invention;
`FIG. 4 is a block diagram that schematically illustrates a
`netWork connection betWeen a transmitting computer and a
`netWork server, for use in broadcasting of a data sequence,
`in accordance With a preferred embodiment of the present
`invention;
`FIG. 5 is a How chart that schematically illustrates a
`method of uploading broadcast data from a transmitting
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`computer to a server, in accordance With a preferred embodi
`ment of the present invention;
`FIG. 6A is a How chart that schematically illustrates a
`method of doWnloading broadcast data from a server to a
`client, in accordance With a preferred embodiment of the
`present invention;
`FIG. 6B is a How chart that schematically illustrates a
`method of doWnloading broadcast data from a server to a
`client, in accordance With another preferred embodiment of
`the present invention;
`FIG. 7 is a How chart that schematically illustrates a
`method for preparing data ?les for transmission, in accor
`dance With a preferred embodiment of the present invention;
`FIG. 8 is a How chart that schematically illustrates a
`method of ?le transfer, in accordance With a preferred
`embodiment of the present invention; and
`FIG. 9 is a How chart that schematically illustrates a
`method for monitoring netWork links, in accordance With a
`preferred embodiment of the present invention.
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`Reference is noW made to FIG. 2, Which is a schematic
`illustration of a computer system 32 for remote broadcasting
`of a multimedia sequence over a netWork 28, in accordance
`With a preferred embodiment of the present invention.
`System 32 comprises a transmitting computer 34, Which
`generates the sequence, a plurality of clients 30, and a
`netWork server 36, all of Which communicate over netWork
`28, preferably using the Well-knoWn Internet Protocol (IP).
`Computer 34 preferably receives audiovisual input from
`input devices 22, although data inputs of other types may be
`generated at or by computer 34 using any suitable means
`knoWn in the art.
`NetWork 28 preferably comprises the Internet, although it
`may equally comprise a LAN, WAN, intranet or other
`computer netWork as is knoWn in the art. Computer 34 and
`clients 30 preferably comprise conventional personal com
`puters or Workstations. Server 36 may comprise any suitable
`type of computer or computer system, for eXample, a Sun
`Microsystems UltraSPARC station or a WindoWs NT server,
`as are commonly used by Internet Service Providers (ISPs)
`. In any case, it is noted that transmitting computer 34 can
`be remotely located relative to server 36, and that the server
`need not be equipped With any special-purpose hardWare or
`softWare for real-time data broadcasting, unlike broadcast
`systems knoWn in the art, such as the real-time encoder and
`broadcast server shoWn in FIG. 1.
`After preparing the multimedia sequence, computer 34
`uploads the sequence over netWork 28, preferably using the
`Internet File Transfer Protocol (FTP). Alternatively, other
`Internet protocols may be used, such as the TCP/IP, UDP or
`RT(X) protocols, Which are knoWn in the art. Preferably, the
`data in the sequence are compressed, although compression
`is not essential to implementation of the present invention.
`The sequence is preferably generated and compressed in real
`time, and could comprise, for eXample, an intervieW pro
`gram or an entertainment or sports event, although a prere
`corded sequence may similarly be broadcast in this manner.
`Computer 34 is preferably equipped With suitable softWare
`for preparing and compressing the multimedia sequence. For
`eXample, for audio data, the computer may typically run
`GSM 6.10 standard audio compression softWare, operating
`at a sample rate of 8 kHZ, With 16 bits/sample. Some useful
`techniques for preparing, compressing and transmitting mul
`timedia sequences are described in US. Pat. No. 5,841,432
`
`PAGE 15 of 21
`
`PETITIONERS' EXHIBIT 1003
`
`

`

`US 6,389,473 B1
`
`7
`and in the above-mentioned U.S. patent application Ser. No.
`08/919,027, both of Which are incorporated herein by ref
`erence.
`Clients 30 connect to server 36 and receive the multime
`dia sequence, substantially in real time. Clients 30 prefer
`ably doWnload the sequence using the Hypertext Transfer
`Protocol (HTTP), although other Internet protocols may also
`be used, such as UDP or RTP, as noted hereinabove With
`reference to uploading by computer 34. Since FTP and
`HTTP are supported by substantially all netWork servers,
`server 36 need not include any special-purpose broadcasting
`hardWare or softWare, as noted above. Similarly, because
`HTTP is supported by substantially all modern Web
`broWsers, clients 30 Will typically need only add a Java
`applet or plug-in to their existing Web broWsers, as
`described further hereinbeloW, in order to receive and play
`back the broadcast.
`FIG. 3A is a block diagram that schematically illustrates
`the structure of a stream of broadcast data 40 produced by
`computer 34, typically corresponding to a multimedia data
`sequence, in accordance With a preferred embodiment of the
`present invention. Data stream 40 comprises a series of data
`slices 42, 44, 46, 48, etc. Each slice contains a segment of
`video and/or audio data, corresponding to a respective,
`successive time interval labeled T1, T2, T3, etc. The data are
`preferably compressed, as described further hereinbeloW.
`Computer 34 stores each slice as a corresponding ?le,
`having a running slice index 1, 2, 3 .
`.
`. N. Preferably, each
`?le also includes one or more time stamps, indicating a real
`time at Which the data in the ?le Were recorded or an elapsed
`time relative to the beginning of stream 40. The ?les are
`uploaded to server 36, such that While any given slice (other
`than ?rst slice 42) is being created, one or more preceding
`slices are in the process of being uploaded.
`Computer 34 monitors the time codes as ?le 40 is
`transmitted, and clients 30 similarly monitor the time codes
`as the ?le is received, in order to ensure that the transmission
`or reception is “keeping up” With the input of the data to the
`computer. In the event that a lag is detected, steps are taken
`to increase the data transmission or reception rate, as
`described further hereinbeloW. For example, as shoWn in
`FIG. 3A, time intervals T1, T2, T3, etc., are not all equal, but
`rather are adjusted by computer 34 in response to the
`transmission rate. Alternatively or additionally, the compres
`sion level of the data is varied, as is likeWise described
`beloW, so as to adjust the data streaming rate to the available
`bandWidth over one or more channels betWeen computer 34
`and server 36, and/or betWeen server 36 and client 30.
`Computer 34 continues to upload ?les 42, 44, 46, etc.,
`until data stream 40 is ?nished or terminated by a user of
`computer 34. All of the ?les in the data stream may be saved
`on server 36 for any desired period of time, as long as the
`server has suf?cient free memory that is accessible to
`computer 34. Typically, hoWever, the memory available on
`server 36 is limited, and ?les 42, 44, 46, etc., Will be stored
`on the server and erased therefrom in a “?rst-in-?rst-out”
`sequence.
`FIG. 3B is a block diagram that schematically illustrates
`an index ?le 50, Which is created by computer 34, and is
`uploaded to server 36, in accordance With a preferred
`embodiment of the present invention. The index ?le com
`prises a slice ID 52, indicating the index of the ?le in data
`stream 40 that Was most recently uploaded by computer 34.
`Each time a neW ?le 42, 44, 46, etc., is uploaded, ID 52 in
`?le 50 on server 36 is updated. Preferably, ID 52 holds the
`?le name of the neW ?le, Wherein the name typically
`
`10
`
`15
`
`25
`
`35
`
`55
`
`65
`
`8
`comprises a string folloWed by the index of the ?le. When
`one of computers 30 connects to server 36 and begins to
`doWnload the data stream, it ?rst reads the index ?le in order
`to identify at What point in stream 40 to begin and to start
`receiving the data stream substantially in real time, prefer
`ably With only a minimal lag, as it is transmitted from
`computer 34. Alternatively, a user of one of computers 30
`may choose to begin doWnloading data stream 40 from an
`earlier point in time than that indicated by ID 52. Further
`alternatively, stream 40 may be multicast to clients 30, as is
`knoWn in the art, typically Without the use of an index ?le.
`Index ?le 50 may further include a message 54, Which is
`read by computers 30 When they connect to server 36 to
`doWnload data stream 40 or, alternatively or additionally, at
`any time the message is updated by computer 34. The
`message contains parameters relating generally to the data
`stream and/or instructions to computers 30, for example,
`“transmission paused.” FIG. 3C is a schematic representa
`tion of a user interface graphic “slider” 55, available to users
`of computers 30, in accordance With a preferred embodiment
`of the present invention. Slider 55, Which is preferably
`displayed on the screens of computers 30, includes a bar 56
`and a movable indicator 58. The symbols J, J+1, J+2, .
`.
`. N
`in the ?gure are the indices of the slices of stream 40 that are
`stored on server 36, Wherein N is the index of the most
`recent slice, and J is the index of the earliest stored slice. J
`may indicate the ?rst slice in the sequence, if all of the ?les
`are stored on server 36, or it may be the earliest ?le not yet
`erased. (The indices are marked in the ?gure on bar 56 for
`clarity, and need not actually be shoWn on the computer
`screen.)
`When one of computers 30 reads index ?le 50 and begins
`to doWnload stream 40, indicator 58 preferably marks the
`most recent slice, as shoWn in FIG. 3C. This is the point at
`Which the doWnload Will begin, unless the user of the
`computer chooses otherWise. If the user Wishes to begin the
`doWnload at an earlier point, he may move indicator 58 to
`the left along bar 56 to that point, preferably using a mouse
`or other pointing device, as is knoWn in the art. Indicator 58
`may be moved back and forth along bar 56 to jump back and
`forth along stream 40.
`FIG. 3D is a block diagram that schematically illustrates
`a ?le format of a multi-level data stream 41, in accordan