`Carmel et al.
`
`USOO6389473B1
`(10) Patent No.:
`US 6,389,473 B1
`(45) Date of Patent:
`May 14, 2002
`
`(54) NETWORK MEDIA STREAMING
`(75) Inventors: Sharon Carmel; Tzur Daboosh, both
`of Givatayim; Eli Reifman, Rishon le
`Zion; Naftali Shani; Ziv Eliraz, both
`of Tel Aviv; Dror Ginsberg, Karkur,
`Edan Ayal, Kfar Saba, all of (IL)
`(73) Assignee: Geo Interactive Media Group Ltd.,
`Givatayim (IL)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`(21) Appl. No.: 09/275,703
`(22) Filed:
`Mar. 24, 1999
`(30)
`Foreign Application Priority Data
`Mar. 24, 1998
`(IL) ................................................ 1238.19
`(51) Int. Cl. ................................................ G06F 13/00
`(52) U.S. Cl. ....................................................... 709/231
`(58) Field of Search ....................... 707/500.1; 709/200,
`709/231, 236, 246, 247; 382/236, 239;
`375/24012
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,267,334 A 11/1993 Normille et al. ............ 382/236
`
`4/1995 Bowater et al. ............ 34.5/537
`5,404,446 A
`5.841,432 A 11/1998 Carmel et al. ........... 707/500.1
`
`Primary Examiner Robert B. Harrell
`(74) Attorney, Agent, or Firm-Ladas & Parry
`(57)
`ABSTRACT
`
`A method for real-time broadcasting from a transmitting
`computer to one or more client computers over a network,
`including providing at the transmitting computer a data
`Stream having a given data rate, and dividing the Stream into
`a Sequence of Slices, each Slice having a predetermined data
`Size associated therewith. The Slices are encoded in a
`corresponding Sequence of files, each file having a respec
`tive index, and the Sequence is uploaded to a Server at an
`upload rate generally equal to the data rate of the Stream,
`Such that the one or more client computers can download the
`Sequence over the network from the Server at a download
`rate generally equal to the data rate.
`
`41 Claims, 11 Drawing Sheets
`
`Microfiche Appendix Included
`(2 Microfiche, 138 Pages)
`
`STANDARD
`NETWORK
`SERVER
`
`28
`Y
`
`2
`
`2. '
`
`Rall
`
`
`
`
`
`
`
`Petitioners' Exhibit 1003
`Page 0001
`
`
`
`U.S. Patent
`U.S. Patent
`
`
`
`May14, 2002
`May 14, 2002
`
`Sheet 1 of 11
`Sheet 1 of 11
`
`US 6,389,473 B1
`US 6,389,473 B1
`
`S.
`26
`
`N,
`24
`
`
`
`REALTIME
`
`ENCODE
`
`Petitioners’ Exhibit 1003
`Page 0002
`
`PRIORART
`
`0 1
`
`wn
`
`s
`FIG.
`
`\
`
`22
`
`wo”
`
`— A
`
`eO
`
`SS ge
`S
`QS
`S. 5 ?o
`So
`
`Petitioners' Exhibit 1003
`Page 0002
`
`
`
`U.S. Patent
`U.S. Patent
`
`May14, 2002
`May 14, 2002
`
`Sheet 2 of 11
`Sheet 2 of 11
`
`US 6,389,473 B1
`US 6,389,473 B1
`
`Ni
`
`&
`28
`
`CN
`2
`
`S.
`FIG.
`
`
`
`STANDARD
`
`NETWORK
`
`SERVER
`
`Petitioners’ Exhibit 1003
`Page 0003
`
`Petitioners' Exhibit 1003
`Page 0003
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 3 of 11
`
`US 6,389,473 B1
`
`F-11-I-72-73--74-
`
`FG, 3A
`
`SLCE
`
`SLCE SCE SLCE
`2
`3
`4.
`
`42
`
`44
`
`46
`
`48
`
`1
`
`40
`
`TME
`
`FG. 3B
`
`
`
`
`
`SLCEN
`
`MESSAGE
`
`52
`
`54
`
`F.G. 3C
`
`
`
`Petitioners' Exhibit 1003
`Page 0004
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 4 of 11
`
`US 6,389,473 B1
`
`
`
`! 7
`
`| # TEAGT
`(I)
`
`Petitioners' Exhibit 1003
`Page 0005
`
`
`
`U.S. Patent
`U.S. Patent
`
`May14, 2002
`May 14, 2002
`
`Sheet 5 of 11
`Sheet S of 11
`
`US 6,389,473 B1
`US 6,389,473 B1
`
`r
`
`4 F
`
`IG.
`
`xa
`
`Na 5
`e PS
`o>
`So St. e
`= &tu
`?o
`Ln
`
`co
`Co
`rm
`r
`
`Q
`N
`O on
`oo
`co
`CO
`(Cd
`
`wo
`CO
`co
`d
`
`ol o
`CO O
`ol]
`fe
`(O
`N.
`
`_
`w
`Me
`Na
`2.
`=
`=_d
`-
`
`LO
`To)
`+
`~~
`N
`co
`sh-
`n
`N
`<<
`x
`x
`x
`wa
`N
`Na.
`N.
`s
`Na
`2. 2 2 2 2
`2
`2
`2
`=
`2
`—_!
`wood
`—
`—!
`4
`-
`-
`-
`-
`-
`
`
`
`
`
`Petitioners’ Exhibit 1003
`Page 0006
`
`Petitioners' Exhibit 1003
`Page 0006
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 6 of 11
`
`US 6,389,473 B1
`
`F.G. 5
`
`
`
`
`
`
`
`
`
`CONNECT
`TO SERVER
`
`INPUT
`BROACAST
`DATA
`
`ENCODE
`
`80
`
`82
`
`FTP TO SERVER 84
`
`UPDATE
`NDEX
`FILE
`
`86
`
`CHECK
`LINK
`FUNCTION
`
`
`
`88
`
`
`
`
`
`
`
`
`
`
`
`
`
`Petitioners' Exhibit 1003
`Page 0007
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 7 of 11
`
`US 6,389,473 B1
`
`F.G. 6A
`
`CONNECT
`TO SERVER
`
`
`
`
`
`HTTP FROM
`SERVER
`
`READ INDEX
`FILE
`
`SELECT SLICE
`
`DECODE
`
`OUTPUT
`DATA
`
`
`
`
`
`
`
`
`
`CONTINUE
`
`
`
`
`
`
`
`
`
`CONNECT
`NEW LINK
`
`Petitioners' Exhibit 1003
`Page 0008
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 8 of 11
`
`US 6,389,473 B1
`
`F.G. 6B
`
`CONNECT
`TO SERVER
`
`HTTP FROM
`SERVER
`
`READ HEADER
`
`CHOOSE
`LEVEL
`
`DECODE
`
`OUTPUT
`DATA
`
`DETERMINE
`LINK RATE
`
`
`
`
`
`
`
`
`
`
`
`CHOOSE
`LOWER
`AUDIO/VIDEO
`LEVEL
`
`
`
`CHOOSE
`HIGHER
`AUDIO/WIDEO
`LEVEL
`
`
`
`RATE
`HIGHER THAN
`NEED
`2
`
`
`
`
`
`
`
`
`
`
`
`CONTINUE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Petitioners' Exhibit 1003
`Page 0009
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 9 of 11
`
`US 6,389,473 B1
`
`FIG. 7
`
`INPUT DATA
`
`80
`
`82
`
`
`
`
`
`
`
`
`
`90
`SET COMPRESSION
`RATIO
`
`
`
`
`
`CONTROL
`FROM 88
`
`COMPRESS DATA
`
`
`
`
`
`
`
`7-92
`SET SLICE DURATION
`
`
`
`CONTROL
`FROM 88
`
`PREPARE
`SLICE I
`
`TO FTP 84
`
`Petitioners' Exhibit 1003
`Page 0010
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 10 of 11
`
`US 6,389,473 B1
`
`F.G. 8
`
`FROM
`SLICE 82
`
`84
`N
`
`
`
`94
`
`FTP OPEN
`LINK J
`
`
`
`LINK TIME-OUT FROM CHECK 88
`
`FTP SEND
`SLICE I
`
`TO UPDATE
`INDEX 86
`
`Petitioners' Exhibit 1003
`Page 0011
`
`
`
`U.S. Patent
`
`May 14, 2002
`
`Sheet 11 of 11
`
`US 6,389,473 B1
`
`s
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F.G. 9
`
`FROM
`UPDATE 86
`
`CHECK SLICE
`TIME TSL
`
`RE-NITIALIZE
`LINK
`
`NO
`
`OPEN 94.
`
`INCREASE
`COMPRESSION
`
`COMPRESSION 90
`
`DECREASE
`SLICE DURATION
`
`TO SET
`DURATION 92
`
`YES
`
`INCREASET
`SLICE DURATION
`- - - - - - - - -
`
`NO
`
`NEXT SLCE
`
`
`
`
`
`
`
`TO
`
`ODE
`
`Petitioners' Exhibit 1003
`Page 0012
`
`
`
`1
`NETWORK MEDIA STREAMING
`
`A computer printout is attached hereto as an appendix in
`microfiche form and is incorporated herein by reference. The
`printout comprises executable program files in hexadecimal
`format. This appendix includes 2 microfiches, containing a
`total of 138 frames.
`
`FIELD OF THE INVENTION
`The present invention relates generally to network data
`communications, and Specifically to real-time multimedia
`broadcasting over a network.
`
`BACKGROUND OF THE INVENTION
`In network broadcasting, data are transmitted over a
`network in real time from a Single transmitting computer to
`a plurality of clients simultaneously. The network may be a
`LAN, a WAN, an intranet or a public network Such as the
`Internet. Network broadcasting is most commonly used to
`Stream multimedia data, typically comprising images and
`Sound.
`FIG. 1 is a Schematic illustration showing a real-time
`broadcasting System 20, as is known in the art. One or more
`input devices 22 (for example, a video camera and/or
`microphone) are used to generate a multimedia data stream
`representing an entertainment or informational program to
`be transmitted to a plurality of clients 30 via a network 28.
`Because of bandwidth limitations of the network, the data
`Stream from host 22 must first be compressed by a real-time
`encoder 24 and then routed to appropriate clients 30 by a
`broadcast Server 26 (since not all clients on the network are
`necessarily intended to receive the broadcast).
`Encoder 24 and Server 26 typically comprise high-cost,
`dedicated computer Systems, Such as a Sun Station
`(produced by Sun Microsystems) or a Windows NT server,
`running suitable RealSystem 5.0 software (produced by
`RealNetworks Inc., Seattle, Wash.). These dedicated sys
`tems are required in order to ensure that the data Stream is
`distributed and received by clients 30 in real time. Similarly,
`host 22 must typically be connected directly to encoder 24
`by a high-Speed data link or LAN, and not via the Internet
`or other narrowband network. Therefore, real-time broad
`casting is normally possible only for hosts having a Suitable,
`dedicated encoder and broadcast Server and cannot be
`offered by Internet service providers (ISPs) to their general
`clientele.
`
`SUMMARY OF THE INVENTION
`It is an object of Some aspects of the present invention to
`provide Substantially continuous, high-bandwidth data
`Streaming over a network using common, existing Server
`and network infrastructure.
`It is a further object of Some aspects of the present
`invention to provide data broadcasting capability, particu
`larly for multimedia data, without the need for a dedicated
`broadcast computer System.
`It is a further object of Some aspects of the present
`invention to provide apparatus and methods for data broad
`casting at reduced cost by comparison with Systems known
`in the art.
`It is still another object of Some aspects of the present
`invention to enable a personal computer to remotely broad
`cast a multimedia program through an Internet Service
`provider (ISP) using common, universally-Supported Inter
`net communication protocols.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,389,473 B1
`
`2
`In preferred embodiments of the present invention, a
`transmitting computer generates a data Stream and broad
`casts the data Stream via a network Server to a plurality of
`clients. The data Stream is divided into a Sequence of
`Segments or Slices of the data, preferably time Slices,
`wherein the data are preferably compressed. Each Slice is
`preferably assigned a respective Slice index. The transmit
`ting computer uploads the Sequence of Slices to the Server
`Substantially in real time, preferably using an Internet
`protocol, most preferably the File Transfer Protocol (FTP),
`as is known in the art. The clients download the data Stream
`from the Server, preferably using an Internet protocol, as
`well, most preferably the Hypertext Transfer Protocol
`(HTTP), or alternatively, using other protocols, such as UDP
`or RTP, which are similarly known in the art. The clients use
`the Slice indices of the frames to maintain proper Synchro
`nization of the playback. The division of the data stream into
`Slices and the inclusion of the slice indices in the data Stream
`to be used by the clients in maintaining Synchronization
`allows the broadcast to go on Substantially in real time
`without the use of Special-purpose hardware.
`Preferably, each Segment or Slice is contained in a
`Separate, respective file. Alternatively, the Segments or Slices
`may all be contained in a single indexed file, which is
`Streamed to the client in a Series of packets, each covering
`a range of one or more indices. HTTP version 1.1 supports
`this Sort of file Streaming. Other protocols may also be used
`for this purpose.
`In Some preferred embodiments of the present invention,
`the data Stream comprises multimedia data captured or
`generated by the transmitting computer. The term "multi
`media' as used in the context of the present patent applica
`tion and in the claims refers to images or Sound or to data
`representative of images or of Sound or a combination
`thereof. Multimedia image data may include Still images,
`Video, graphics, animation or any combination thereof,
`including text displayed in conjunction therewith. It will be
`appreciated, however, that the principles of the present
`invention may similarly be applied to Streaming of other
`data types.
`Preferably, the transmitting computer compresses the
`frames in the data Stream, most preferably using methods of
`image and audio compression Such as those described in
`U.S. patent application Ser. No. 08/919,027, which is
`assigned to the assignee of the present patent application and
`incorporated herein by reference. Alternatively, any Suitable
`methods of compression known in the art may be used. The
`compressed data are conveyed to the Server and thence to the
`clients, which decompress the data.
`In Some preferred embodiments of the present invention,
`the transmitting computer and the clients monitor the
`uploading and downloading of data to and from the Server,
`respectively, in order to determine the amount of time
`required to convey each Slice and to Verify that the slices are
`conveyed at a Sufficient rate. When the data Stream com
`prises multimedia data, the data rate should be generally
`equal to or faster than the rate at which the data are generated
`at the transmitting computer.
`In Some of these preferred embodiments, the transmitting
`computer and/or the clients each open a plurality of FTP or
`HTTP links, respectively, with the network server. The slices
`are transferred over different ones of the links in alternation.
`Although typically none of the plurality of links has Suffi
`cient bandwidth on its own to convey the entire data Stream
`in real time, the combined bandwidths of the plurality of
`links are generally Sufficient for this purpose. Preferably,
`
`Petitioners' Exhibit 1003
`Page 0013
`
`
`
`3
`each of the linkS is monitored to determine its Specific data
`transfer rate. If the transfer rate of any of the links is below
`a predetermined minimum, that link is preferably closed,
`and a new link is opened in its place.
`In other preferred embodiments, the slices are provided by
`the Server at multiple resolution or quality levels. Each Such
`level has a different degree of data compression, and thus
`corresponds to a different data bandwidth requirement. The
`client or the Server monitors the data transfer rate of a data
`link opened therebetween and selects the level that is
`appropriate to the link bandwidth. If the monitored data
`transfer rate changes during transmission, the quality level is
`preferably reselected accordingly.
`Preferably, the transmitting computer monitors the band
`width of the data Stream that it is uploading to the Server, and
`compares the data Stream bandwidth to a known or esti
`mated bandwidth of the link or links between the transmit
`ting computer and the Server. The transmitting computer
`preferably compresses the data Stream at a compression ratio
`that is adjusted So as to match the data Stream bandwidth to
`the available link bandwidth, using methods described, for
`example, in the above-mentioned U.S. patent application
`Ser. No. 08/919,027.
`There is therefore provided, in accordance with a pre
`ferred embodiment of the present invention, a method for
`real-time broadcasting from a transmitting computer to one
`or more client computers over a network, including:
`providing at the transmitting computer a data Stream
`having a given data rate;
`dividing the Stream into a sequence of Slices, each Slice
`having a predetermined data Size associated therewith;
`encoding the slices in a corresponding sequence of files,
`each file having a respective index; and
`uploading the Sequence to a Server at an upload rate
`generally equal to the data rate of the Stream, Such that
`the one or more client computers can download the
`Sequence over the network from the Server at a down
`load rate generally equal to the data rate.
`Preferably, dividing the Stream into the Sequence of Slices
`includes dividing the Stream into a sequence of time Slices,
`each having a predetermined duration associated therewith.
`Preferably, uploading the Sequence includes comparing
`the upload rate to the data rate and adjusting the upload rate
`responsive to the comparison. Further preferably, encoding
`the Stream includes compressing data in the Stream at a
`desired compression ratio, and adjusting the upload rate
`includes changing the compression ratio. Alternatively or
`additionally, adjusting the upload rate includes adjusting the
`Size of one or more of the slices.
`Preferably, uploading the Sequence includes opening a
`plurality of file transfer links between the transmitting
`computer and the Server, each link characterized by a
`respective link data rate, and uploading different files in the
`sequence over different ones of the plurality of links. Further
`preferably, opening the plurality of links includes opening
`linkS Such that the data rates of the links taken together are
`Sufficient to upload the Sequence at the upload rate generally
`equal to the data rate.
`Preferably, uploading the Sequence includes uploading a
`Sequence using an Internet Protocol, most preferably using
`FTP
`Preferably, the method includes downloading the
`Sequence using an Internet protocol, most preferably HTTP,
`or alternatively, UDP or RTP, over the network from the
`server to the one or more client computers. Preferably, the
`one or more client computers decode the Sequence and play
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,389,473 B1
`
`4
`back the data Stream responsive to the indices of the files, at
`a replay rate generally equal to the data rate.
`Preferably, uploading the Sequence includes uploading
`and updating an indeX file containing the index of the file in
`the Sequence that was most recently uploaded, and the one
`or more client computers read the indeX file to play back the
`Sequence. In a preferred embodiment, downloading the
`Sequence includes Selecting a file in the Sequence earlier
`than the file whose index is contained in the index file and
`downloading at least a portion of the Sequence of files
`beginning with the Selected file.
`Preferably, the one or more client computers include a
`plurality of client computers, and downloading the Sequence
`includes downloading to the plurality of client computers
`Substantially simultaneously.
`Preferably, downloading the Sequence includes opening a
`plurality of download links between one of the client com
`puters and the Server, each link characterized by a respective
`link data rate, and downloading different files in the
`sequence over different ones of the plurality of links. Most
`preferably, opening the plurality of links includes opening
`linkS Such that the data rates of the links taken together are
`Sufficient to download the Sequence at the download rate
`generally equal to the data rate.
`Preferably, opening the plurality of links includes moni
`toring the data rates of the links and opening a new link in
`place of one of the links having a data rate lower than a
`predetermined level.
`In one preferred embodiment, opening the new link
`includes retransmitting at least one of the files in the
`Sequence, wherein the at least one of the files was incom
`pletely transmitted over the one of the links having the data
`rate lower than the predetermined level.
`In another preferred embodiment, opening the new link
`includes dropping at least one file out of the Sequence,
`wherein the at least one of the files was incompletely
`transmitted over the one or more of the links having the data
`rate lower than the predetermined level.
`In Still another preferred embodiment, encoding the Slices
`includes encoding Slices at a plurality of different quality
`levels, Such that the files corresponding to a given one of the
`Slices have a different, respective data Size for each of the
`quality levels. Preferably, downloading the Sequence
`includes determining a data bandwidth of the network
`between the Server and the client computer and Selecting one
`of the quality levels responsive to the determined band
`width.
`Preferably, the data stream includes multimedia data.
`There is further provided, in accordance with a preferred
`embodiment of the present invention, apparatus for real-time
`broadcasting of a data Stream having a given data rate over
`a network, including:
`a transmitting computer, which divides the Stream into a
`Sequence of Slices, each Slice having a predetermined
`data Size associated there with, and encodes the slices in
`a corresponding Sequence of files, each file having a
`respective index, and
`which uploads the Sequence to a Server at an upload rate
`generally equal to the data rate, Such that one or more
`client computers can download the Sequence over the
`network from the Server at a download rate generally
`equal to the data rate.
`Preferably, the transmitting computer compares the
`upload rate to the data rate and adjusts the upload rate
`responsive to the comparison. Most preferably, the trans
`mitting computer compresses the data at a compression ratio
`which is varied responsive to the comparison. Additionally
`
`Petitioners' Exhibit 1003
`Page 0014
`
`
`
`S
`or alternatively, the transmitting computer adjusts the size of
`one or more of the Slices responsive to the comparison.
`Preferably, the transmitting computer opens a plurality of
`links between the transmitting computer and the Server, each
`link characterized by a respective data rate, and transmits
`different ones of the sequence of files over different ones of
`the plurality of links. Most preferably, the transmitting
`computer opens the plurality of linkS Such that the data rates
`of the links taken together are Sufficient to upload the
`Sequence at the upload rate generally equal to the data rate.
`Further preferably, the transmitting computer monitors the
`data rates of the links and opens a new link in place of one
`of the links whose data rate is lower than a predetermined
`level.
`In a preferred embodiment, the Slices are encoded at a
`plurality of different quality levels, such that the files cor
`responding to a given one of the Slices have a different,
`respective data Size for each of the quality levels.
`Preferably, the transmitting computer uploads the
`Sequence using an Internet upload protocol, most preferably
`FTP
`Preferably, the one or more client computers decode the
`Sequence and play back the data Stream responsive to the
`indices thereof, at a data replay rate generally equal to the
`data rate. Preferably, the one or more client computers
`download the encode Sequence using an Internet download
`protocol, most preferably HTTP or alternatively, UDP or
`RTP
`Preferably, the one or more client computers include a
`plurality of client computers, which download the Sequence
`Substantially simultaneously.
`Preferably, the network includes the Internet.
`Further preferably, the data stream includes multimedia
`data, and the predetermined data Size of each of the Slices
`corresponds to a time duration of the Slice.
`The present invention will be more fully understood from
`the following detailed description of the preferred embodi
`ments thereof, taken together with the drawings in which:
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a Schematic illustration of a computer broadcast
`network, as is known in the art;
`FIG. 2 is a Schematic illustration of a computer broadcast
`network, in accordance with a preferred embodiment of the
`present invention;
`FIG. 3A is a block diagram that schematically illustrates
`a data Structure of a broadcast Sequence, in accordance with
`a preferred embodiment of the present invention;
`FIG. 3B is a block diagram that schematically illustrates
`an index file associated with the data structure of FIG. 3B,
`in accordance with a preferred embodiment of the present
`invention;
`FIG. 3C is a schematic illustration of a user interface
`graphic, for use in conjunction with the data Structure of
`FIG. 3A, in accordance with a preferred embodiment of the
`present invention;
`FIG. 3D is a block diagram that schematically illustrates
`a data Structure of a broadcast Sequence, in accordance with
`another preferred embodiment of the present invention;
`FIG. 4 is a block diagram that Schematically illustrates a
`network connection between a transmitting computer and a
`network Server, for use in broadcasting of a data Sequence,
`in accordance with a preferred embodiment of the present
`invention;
`FIG. 5 is a flow chart that schematically illustrates a
`method of uploading broadcast data from a transmitting
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,389,473 B1
`
`6
`computer to a Server, in accordance with a preferred embodi
`ment of the present invention;
`FIG. 6A is a flow chart that schematically illustrates a
`method of downloading broadcast data from a Server to a
`client, in accordance with a preferred embodiment of the
`present invention;
`FIG. 6B is a flow chart that schematically illustrates a
`method of downloading broadcast data from a Server to a
`client, in accordance with another preferred embodiment of
`the present invention;
`FIG. 7 is a flow chart that schematically illustrates a
`method for preparing data files for transmission, in accor
`dance with a preferred embodiment of the present invention;
`FIG. 8 is a flow chart that schematically illustrates a
`method of file transfer, in accordance with a preferred
`embodiment of the present invention; and
`FIG. 9 is a flow chart that schematically illustrates a
`method for monitoring network links, in accordance with a
`preferred embodiment of the present invention.
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`Reference is now made to FIG. 2, which is a schematic
`illustration of a computer System32 for remote broadcasting
`of a multimedia Sequence over a network 28, in accordance
`with a preferred embodiment of the present invention.
`System 32 comprises a transmitting computer 34, which
`generates the Sequence, a plurality of clients 30, and a
`network server 36, all of which communicate over network
`28, preferably using the well-known Internet Protocol (IP).
`Computer 34 preferably receives audiovisual input from
`input devices 22, although data inputs of other types may be
`generated at or by computer 34 using any Suitable means
`known in the art.
`Network 28 preferably comprises the Internet, although it
`may equally comprise a LAN, WAN, intranet or other
`computer network as is known in the art. Computer 34 and
`clients 30 preferably comprise conventional personal com
`puters or WorkStations. Server 36 may comprise any Suitable
`type of computer or computer System, for example, a Sun
`Microsystems UltraSPARC station or a Windows NT server,
`as are commonly used by Internet Service Providers (ISPs)
`. In any case, it is noted that transmitting computer 34 can
`be remotely located relative to server 36, and that the server
`need not be equipped with any special-purpose hardware or
`Software for real-time data broadcasting, unlike broadcast
`Systems known in the art, Such as the real-time encoder and
`broadcast server shown in FIG. 1.
`After preparing the multimedia Sequence, computer 34
`uploads the Sequence over network 28, preferably using the
`Internet File Transfer Protocol (FTP). Alternatively, other
`Internet protocols may be used, such as the TCP/IP, UDP or
`RT(x) protocols, which are known in the art. Preferably, the
`data in the Sequence are compressed, although compression
`is not essential to implementation of the present invention.
`The Sequence is preferably generated and compressed in real
`time, and could comprise, for example, an interview pro
`gram or an entertainment or Sports event, although a prere
`corded Sequence may similarly be broadcast in this manner.
`Computer 34 is preferably equipped with suitable software
`for preparing and compressing the multimedia Sequence. For
`example, for audio data, the computer may typically run
`GSM 6.10 Standard audio compression Software, operating
`at a sample rate of 8 kHz, with 16 bits/sample. Some useful
`techniques for preparing, compressing and transmitting mul
`timedia sequences are described in U.S. Pat. No. 5,841,432
`
`Petitioners' Exhibit 1003
`Page 0015
`
`
`
`US 6,389,473 B1
`
`15
`
`25
`
`35
`
`40
`
`7
`and in the above-mentioned U.S. patent application Ser. No.
`08/919,027, both of which are incorporated herein by ref
`CCCC.
`Clients 30 connect to server 36 and receive the multime
`dia Sequence, Substantially in real time. Clients 30 prefer
`ably download the Sequence using the Hypertext Transfer
`Protocol (HTTP), although other Internet protocols may also
`be used, such as UDP or RTP, as noted hereinabove with
`reference to uploading by computer 34. Since FTP and
`HTTP are supported by substantially all network servers,
`Server 36 need not include any special-purpose broadcasting
`hardware or Software, as noted above. Similarly, because
`HTTP is supported by Substantially all modern Web
`browsers, clients 30 will typically need only add a Java
`applet or plug-in to their existing Web browsers, as
`described further hereinbelow, in order to receive and play
`back the broadcast.
`FIG. 3A is a block diagram that schematically illustrates
`the structure of a stream of broadcast data 40 produced by
`computer 34, typically corresponding to a multimedia data
`Sequence, in accordance with a preferred embodiment of the
`present invention. Data Stream 40 comprises a Series of data
`Slices 42, 44, 46, 48, etc. Each slice contains a Segment of
`Video and/or audio data, corresponding to a respective,
`Successive time interval labeled T, T, T, etc. The data are
`preferably compressed, as described further hereinbelow.
`Computer 34 Stores each Slice as a corresponding file,
`having a running Slice index 1, 2,3 . . . N. Preferably, each
`file also includes one or more time Stamps, indicating a real
`time at which the data in the file were recorded or an elapsed
`time relative to the beginning of stream 40. The files are
`uploaded to server 36, Such that while any given Slice (other
`than first slice 42) is being created, one or more preceding
`Slices are in the process of being uploaded.
`Computer 34 monitors the time codes as file 40 is
`transmitted, and clients 30 similarly monitor the time codes
`as the file is received, in order to ensure that the transmission
`or reception is "keeping up' with the input of the data to the
`computer. In the event that a lag is detected, Steps are taken
`to increase the data transmission or reception rate, as
`described further hereinbelow. For example, as shown in
`FIG. 3A, time intervals T, T, T, etc., are not all equal, but
`rather are adjusted by computer 34 in response to the
`transmission rate. Alternatively or additionally, the compres
`Sion level of the data is varied, as is likewise described
`below, So as to adjust the data Streaming rate to the available
`bandwidth over one or more channels between computer 34
`and server 36, and/or between server 36 and client 30.
`Computer 34 continues to upload files 42, 44, 46, etc.,
`until data stream 40 is finished or terminated by a user of
`computer 34. All of the files in the data stream may be saved
`on Server 36 for any desired period of time, as long as the
`Server has Sufficient free memory that is accessible to
`computer 34. Typically, however, the memory available on
`server 36 is limited, and files 42, 44, 46, etc., will be stored
`on the server and erased therefrom in a “first-in-first-out”
`Sequence.
`FIG. 3B is a block diagram that schematically illustrates
`an index file 50, which is created by computer 34, and is
`uploaded to Server 36, in accordance with a preferred
`embodiment of the present invention. The index file com
`prises a slice ID 52, indicating the index of the file in data
`stream 40 that was most recently uploaded by computer 34.
`Each time a new file 42, 44, 46, etc., is uploaded, ID 52 in
`file 50 on server 36 is updated. Preferably, ID 52 holds the
`file name of the new file, wherein the name typically
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`comprises a string followed by the index of the file. When
`one of computers 30 connects to server 36 and begins to
`download the data stream, it first reads the index file in order
`to identify at what point in stream 40 to begin and to start
`receiving the data Stream Substantially in real time, prefer
`ably with only a minimal lag, as it is transmitted from
`computer 34. Alternatively, a user of one of computers 30
`may choose to begin downloading data Stream 40 from an
`earlier point in time than that indicated by ID 52. Further
`alternatively, stream 40 may be multicast to clients 30, as is
`known in the art, typically without the use of an indeX file.
`Index file 50 may further include a message 54, which is
`read by computers 30 when they connect to server 36 to
`download data stream 40 or, alternatively or additionally, at
`any time the message is updated by computer 34. The
`message contains parameters relating generally to the data
`Stream and/or instructions to computers 30, for example,
`“transmission paused.” FIG. 3C is a Schematic representa
`tion of a user interface graphic "slider' 55, available to users
`of computers 30, in accordance with a preferred embodiment
`of the present invention. Slider 55, which is preferably
`displayed on the screens of computers 30, includes a bar 56
`and a movable indicator 58. The symbols J, J-1, J+2, ... N
`in the figure are the indices of the slices of stream 40 that are
`stored on server 36, wherein N is the index of the most
`recent Slice, and J is the index of the earliest Stored slice. J
`may indicate the first Slice in the Sequence, if all of the files
`are stored on server 36, or it may be the earliest file not yet
`erased. (The indices are marked in the figure