(12) United States Patent
`Carmel et al.
`
`IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
`US00638()473BI
`US 6I389I473 Bl
`May 14,2002
`
`(io) Patent js(o.:
`(45) Date of Patent:
`
`(S4) NETWORK MFDIA STRFAMING
`
`(75)
`
`Inventors: Sharon Cannel; Tzur Dahoosh, both
`of Giv*atayim, Eli Reihnan, Rtshon le
`Zion, Naftali Shani, Ziv L'liraz, both
`of Tel Aviv; Dror Cinshcrg, Karkxtr;
`Fdan Ayal, Kfar Saba, all of (II.)
`
`(73) Assignee: Geo Interactive Media ('roup Ltd.,
`(iivatayim (II.)
`
`( v ) Notice:
`
`Subtect to any disclaimer, Ihe term of this
`patent is extended or adjusted under 3S
`U S C I S4(b) by 0 days.
`
`(Z2)
`
`(Zl) Appl. No: 09/275,703
`Mar. 24, 1999
`Filed:
`Foreign Application Priority Data
`
`(3(l)
`
`Mai 24, 1998 (II,)................
`Int. Cl.'.............. G06F 13/00
`
`I23819
`
`(Sl)
`(52) U.S. Ck ................................................... 709/231
`Irield of Search ...................... 707i500.1, 709i200,
`(58)
`709/231, 236, 246, 247; 382 236, 239,
`375/240 12
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`lti199I3 Normill«et al............. 382/23(i
`
`5,267,334 A
`
`5,404,446 A
`5.841,432 A
`
`...... 341/537
`4 1995 nowatei et al
`11 1998 C umel et al............ 7074500.1
`
`Pri man Examiner—Robert B. Harrell
`(74) Attorney, Agent, or Firm—hades ye Parry
`
`(57)
`
`ABSTRACT
`
`A method for real-time broadcasting from a Iransmining
`computer to one or more client computers over a networl,
`the transmitting computer a data
`inclutling providing at
`stream having a given data rate, anil dividing the stream mto
`a sequence of. shces, each slice having a predetermmed data
`size associated therewith. The slices are encoded in a
`corresponding sequence of liles, each lile having a respec-
`uve index, and the sequence is uploaded to a server at an
`upload rate generally equal to the data rate of Ihe stream,
`such that the one or more client mtmputers can download the
`sequence over the network from the server at a download
`rale generallv equal Io the data rate.
`
`41 Claims, ll Drawing Sheets
`
`Microfiche Appcnrlix Inclurlcd
`(2 Microfiche, 138 Pages)
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 1
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 1 of 11
`
`US 6,389,473 81
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 2
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 2 of 11
`
`US 6,389,473 81
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 3
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 3 of 11
`
`US 6,389,473 81
`
`Tgq
`
`FIG. 3A
`
`40
`
`TIME
`
`FIG. 3B
`
`52
`
`50~
`
`FIG. 3C
`
`55
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 4
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 4 of 11
`
`US 6,389,473 81
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 5
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 5 of 11
`
`US 6,389,473 81
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 6
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 6 of 11
`
`US 6,389,473 81
`
`FlG. 5
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 7
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 7 of 11
`
`US 6,389,473 81
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 8
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 8 of 11
`
`US 6,389,473 81
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 9
`
`

`

`U.S. Patent
`
`Maay 14,2002
`
`Sheet 9 of 11
`
`US 6,389,473 81
`
`FIG. 7
`
`INPUT DATA
`
`80
`
`82
`
`TO FTP 84
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 10
`
`

`

`U.S. Patent
`
`May 14,2002
`
`Sheet 10 of 11
`
`US 6,389,473 81
`
`FIG. 8
`
`FROM
`SLICE 82
`
`TO UPDATE
`INDEX 86
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 11
`
`

`

`U.S. Patent
`
`ofay 14,2002
`
`Sheet 11 of 11
`
`US 6,389,473 81
`
`88
`
`FROM
`UPDATE 86
`
`FIG. 9
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 12
`
`

`

`NETWORK MEDIA STREAMING
`
`A computer pmntout is a«ached hereto as an appenihx in
`microliche form and is incorporated herein by reference. The
`printout comprises executable program files in hexadecimal
`format This appenrhx includes 2 microfiches, icnntaining a
`total of 138 frames.
`
`FIELD OF THE INVENTION
`
`The present invention relates generally to network data
`communications, and specificall to real-time mulhmctlia
`broadcasting over a network
`
`BACKGROUND OF THE INVENTION
`
`rlata are transmitted over a
`In network broadcasting,
`netivnrk in real time from a single transmitting computer to
`a plurality of clients simultaneously. Thc network may bc a
`LAN, a WAN, an intranct or a public network such as the
`Internet. Network broadcasung is most commonly usetf to
`stream multimedia data, typically comprising rma es aml
`sound.
`is a schematic illustration showing a real-time
`I'IG I
`broadcasting system 20, as is known in thc art. Onc or more
`input dcviccs 22 (for cxamplc, a video camera anil or
`microphone) are used to generate a multimedia data stream
`representing an entertainmenl or inlormauonal program Io
`be transmitted to a plurality of clients 30 via a network 2g
`Because of bandkvidth limitations of the network, the data
`stream from host 22 must first fie compressed by a real-time
`encoder 24 and then routed to appropriate clients 30 by a
`broadcast scrvcr 26 (since not all clients on thc nctivorl're
`necessarily intentfed to receive the broailcast).
`L'needer 24 and server 26 typically comprise high-cost,
`dedicated cnmputer systems, such as a Sun Station
`(produced by Sun Microsystcms) or a Windows NT scrvcr,
`running suitable RcalSystcm 5.0 softivarc (produced by
`RealNetworks Inc., Sea«le, Wash.). These dedicated sys-
`tems are required in ortfer to ensure that Ihe data stream is
`distributed ami received by chents 30 in real time. Simrlarly,
`host 22 must typically be connected rlirectly to enicnder 24
`by a high-speed data link or I.AN, anil not via the Internet
`or other narrowband network. Thcrcforc, real-time broad-
`casting is normally possible only for hosts having a suitable,
`dedicated encoder and broadcast server and cannot be
`olfered by Internet service prov«lers (ISPs) to their general
`chentele.
`
`SUMMARY OF THE INVENTION
`It is an object of some aspects of the present invention Io
`provide substantially continunus, high-bandwidth data
`over a network using cnmmon, existing server
`streamin
`and network infrastructure.
`is a further obtect of some aspects of the present
`It
`invennon to provide data broadcas«ng capability, particu-
`larly for multimedia data, without the neeii for a dedicated
`broadcast computer system
`is a further object of some aspects of the present
`It
`invention to provide apparatus and methods for data broad-
`casting at reduced cost by comparison ivith systems known
`in the art.
`It is still another obtect of some aspects of the present
`invennon to enable a personal computer Io remotely broatf-
`cast a multimedia program through an Internet service
`provider (ISV) using commnn, universally-supported Inter-
`net communication protocols
`
`In preferred embodiments of the present
`invention, a
`transmitting computer generates a data stream and broad-
`casts the data stream via a netivork server to a plurality of
`'I'he data stream is divided into a sequence of
`clients
`segments or slices of the data, prcfcrably time slices,
`whcrcin thc data arc preferably comprcsscd. Each slice is
`preferablv assigned a respec«ve shee index. The transmit-
`ting computer uploads Ihe sequence of. slices Io the server
`substantially in real
`time, preferably using an Internet
`iii prntocol, most preferably the I'ile 'I'ransfer Vrotocnl (I "I'V),
`as is known in the art The clients download the data stream
`from thc server, preferably using an Intcrnct protocol, as
`well, most prcfcrably thc Hypcrtcxt Transfer Protocol
`(HTTP), or alternatively, using other protocoLs, such as UDP
`is or RTP, which are similarly known in the art. The clients use
`the slice indices nf the frames to maintain proper synchni-
`nization of the playback I'he division of the data stream into
`shces and the inclusion nf the slice indices in the data stream
`to be used by thc clients in maintaining synchronization
`zo allows thc broadcast
`to go on substantially in real
`time
`without the use of special-purpose hardware.
`Preferably, each segment or slice is contained in a
`separate, respective file. Alternatmely, the segments nr slices
`may all bc contained in a single indexed fil, which is
`-'treamed to thc client in a scrics of packets, each covering
`a range of one or more indices. HTTP version 1.1 supports
`this sort of life streaming. Other protocols may also be used
`fiir Ibis purpose.
`In some preferred embodiments of the present invention,
`ie the data stream comprises multimctlia elate captured or
`gcncratcd liy thc transmitting computer. Thc term 'muhi-
`meiha's used in the context of the present patent applica-
`uon aml in Ihe claims refers to images or sound or to data
`representative of images or of sound or a cnmbination
`35 thereof. Multimedia image data may include still
`images,
`virieo, graphics, animation or any cnmfiination thereof,
`includin text displayed in conjunction thcrcwith. It will bc
`thc principles of thc prcscnt
`appreciated, however,
`that
`invention may similarly be applied to streaming of other
`an iiata types.
`the transmitting computer cnmpresses the
`Preferably,
`frames in the data stream, most prcfcrably using methods of
`image and audio compression such as those dcscribcd in
`U.S. patent application Ser. No. Ogr919,027, which is
`assignetf Io the assi mee of the present patent applicanon and
`incorporated herein by reference. Alternauvely, any suitable
`methods of compression known in the art may be used. The
`compressed data are conveyed to the server and thence to the
`clients, ivhich dccomprcss thc data.
`In some prelerreti embodiments of the present invention,
`the transmitting computer and the clients monitor the
`uploading and doivnloading of data to anil from the server,
`in order to determine the amnunt of time
`respectively,
`required to convey each slice and to verify that the slices are
`convcycd at a sufficicnt rate. When thc rlata stream com-
`pnscs multimedia data, thc data rate should bc
`cncrally
`equal to or fasier than Ihe rate at which the data are generated
`at the transmitting computer.
`In some of these preferred embodiments, the transmitting
`computer and/or thc clients each open a plurality of FTP or
`HTTP linLs, rcspcctivcly, with thc network scrvcr. The slices
`are translerred over difierent ones of Ihe links in alternation.
`Although Ivpically none of the plurality of links has suifi-
`ss cient bandw«1th on its own to convey the en«re data stream
`in real time, the combined bandwidths of the plurality of
`links are generally sufficient for this purpose. Preferably,
`
`sii
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 13
`
`

`

`US 6,389,473 Bl
`
`each of. the hnl s is monitored Lo determine its speciiic data
`transfer rate. If the transfer rate nf any of the links is below
`a predetermined minimum, that link is preferably closed,
`and a new link is opened in its place
`In other prcferrcd embodiments, thc slices arc provirlcrl by
`the scrvcr at multiple resolution or quality lcvcls. Each such
`level has a diilerent degree of data compression, and thus
`corresponds to a different data bandwidth requirement. The
`chen& or the server monitors the da&a transier rate of a data
`link opened therehetween anti selects the level
`that
`is
`appropriate to the link bandwidth. If the monitored data
`transfer rate changes during transmission, thc quality icvcl is
`preferably resclccted accorrlingly.
`Preferably, thc transmitting computer monitors the band-
`width of the data stream that u is uploading lo the server, aml
`compares lhe dale stream bandwidth to a I nown or esu-
`mated bandwidth of the link or links between the transmit-
`I'he transmitting computer
`ting computer and the server
`preferably compresses the data stream at a compression rat&o
`that is adjusted so as to match thc data stream bandwirlth to
`the available link banrlwidth, using methods dcscribcrl, for
`in the above-menuoned U.S. patent apphcahon
`example,
`Ser. No. Og 919,I&z7.
`There Ls therefore provided, in secor&lance with a pre-
`ferred embodiment of the present invention, a method for
`real-time broadcasting from a transmitting computer to one
`or morc client computers over a network, inclurlin:
`thc transmitting computer a data stream
`providing at
`having a given data rate;
`dividing thc stream into a scqucncc of slices, each slice
`having a predetermined &tata size associated therewith,
`encoding Lhe slices in a correspomling sequence of tiles,
`each Iile having a respective mdex; and
`uploading the sequence Lo a server at an upload rate
`enerally equal to the data rate of the stream, such that
`the one or more client computers can download the
`sequence over the network from the server at a down-
`load rate generally equal to the data rate
`Preferably, dividing thc stream into thc scqucncc of slices
`includes dividing thc stream into a scqucncc of time slices,
`each having a predetermined iiurauon associated therewith.
`Preferably, uplomiing the sequence includes compann ~
`the upload rale to the data rate and adiusting the uploml rate
`responsive to the comparison. I'urther preferably, encod&ng
`the stream includes compressing data in the stream at a
`desired compression ratio, and adjusting thc uploarl rate
`includes changing thc compression ratio. Altcrnativcly or
`addiuonally, adiusung lhe upload rate includes adjusnng the
`swe of one or more of the slices.
`Preferably, uploading the sequence mcludes opening a
`plurality of lile transfer links between the transmitt&ng
`computer and the server, each link characterized by a
`respcctivc link data rate, anil uploadin difi'crcnt file in the
`scqucncc over diffcrcnt ones of thc plurality of links Further
`preferably, opening the plurality of 1&nks includes openin ~
`1&nks such that the data rates of &he links taken to ether are
`sulhcient Lo upload the sequence a& Lhe uploatl rate generally
`equal to the data rate
`Preferably, uploading the sequence includes uploadmg a
`scqucncc using an Intcrnct Protocol, most prcfcrably usin
`FTP.
`the method includes downloading the
`Preferably,
`sequence using an Interne& prolocol, most preferably HTTP,
`or alternahvely, UDP or RTP, over &he network from the
`server to the one or more client computers. Preferably, the
`one or more client computers decode the sequence and play
`
`lhe dale stream responsive Lo the imlices ol lhe liles, at
`bacl
`a replay rale generally equal Lo the data ra&e.
`Preferably, uploading thc sequence includes uploading
`and updating an index file containing thc index of the fil in
`the sequence that ivas most recently uploarled, and the one
`or morc client computers read thc index file to play hack thc
`In a prcfcrrcd embodiment, rlownloading thc
`scqucncc.
`scqucncc includes selcctmg a fik in thc scqucncc carlicr
`than the Iik whose in&lax is contained in the index lile and
`least a porlion of Lhe sequence of tiles
`iiownloading al
`beginning with the selected file
`Preferably, the one or more client computers include a
`plurality of client computers, and rlownloading thc scqucncc
`includes downloading to thc plurality of client computers
`,s substantially simultaneously
`Preferably, downloading the sequence includes opening a
`plurality ol download links between one of lhe client com-
`puters and the server, each link charactenzed by a respective
`link data rate, and downloading different
`files in the
`zp st'qucncc over differcnt ones of thc plurality of links. Most
`preferably, opening thc plurality of links includes opening
`links such that thc data rates of thc links taken together arc
`sufiicienl lo download the sequence al Lhe download rate
`generally equal Lo Lhe dale rale.
`Preferably, opening the plurahty of links includes moni-
`toring the data rates of the hnks and opening, a new link in
`place of onc of the links having a data rate lower than a
`prcrlctcrmincd lcvcl.
`In onc prcfcrrcd cmbodimcnt, opening thc ncw link
`least one of the tiles in the
`incluties relransmining at
`sequence, wherein Lhe at least one of the tiles was incom-
`pletely transmitted over the one of the links having the data
`rate inwer than the predetermined level.
`In another prcfcrrcd embodiment, opening the ncw link
`least onc tile out of lhc scqucncc,
`includes dropping at
`least onc of thc tiks was incomplctcly
`whcrcin the at
`transmitted over the one or more of the hnks having lhe data
`rale lower than the pretletermined leveL
`In still another preferred embodiment, encoding, the slices
`ap includes encoding slices at a plurality of difi'erent quality
`lcvcls, such that thc files corrcsponrling to a given onc of thc
`slices have a differcnt, rcspcctivc data size for each of thc
`quality lcvcls. Prcfcrably, downloading thc scqucncc
`incluties determining a data banrlwidlh of Lhe networl
`between the server and Lhe client computer and selecnng one
`of the quality levels responsive to the rleterminerl band-
`width.
`Preferably, the data stream includes multimedia data
`Thcrc is further provided, in accordance v ith a prcfcrrcd
`sp embodiment of thc prcscnt invention, apparatus for real-time
`broaiicasting of a data stream having a given data rate over
`a networl, including.
`a transmitting computer, which divirlcs thc stream into a
`sequence of slices, each shee having a predelermmed
`dale mze associated therewith, ami encodes &he slices in
`sequence of files, each hie having a
`a correspondin
`respective index, and
`which uploads the sequence Lo a server al an uploml rate
`generally equal to the data rate, such that one or more
`client computers can download the sequence over the
`network from thc server at a download rate
`cncrally
`equal to thc data rate.
`lhe transmitung computer compares the
`Preferably,
`upload rale lo the data rate anil adiusts lhe upload rate
`lhe trans-
`responsive lo Lhe comparison. Most preferably,
`mitting computer compresses the data at a compression ratio
`which is varied responsive to the comparison Additionally
`
`zs
`
`ap
`
`ps
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 14
`
`

`

`US 6,389,47:) Bl
`
`is
`
`35
`
`or alternatively, the transmitting computer adjusts the size of
`one or more of the slices responsive tn the comparison.
`Preferably, the transmitting computer opens a plurahty of
`links bet&veen the transmitting computer andi the server, each
`link charactcrizcd by a rcspcctivc data rate, and transmits
`difl'crcnt ones of thc scqucncc of filcs over rliffcrcnt ones of
`the plurality of hnks. Most prelerably,
`the transminin ~
`computer opens the plurality of hnks such that the data rates
`of the links taken together are sufficient
`to upload the
`sequence at the upload rate generally equal to the data rate
`Irurther preferably, the transmitting, cnmputer monitors the
`data rates of the links anil opens a ncw link in place of one
`of thc links whose data rate is lower than a prcrlctcrmincrl
`level.
`the slices are encoded at a
`In a prelerred embodiment,
`plurality of different quality levels, such that the files cor-
`responding to a given one of the slices have a ditferent,
`respective data size for each of the quality levels
`thc transmitting computer uploads thc
`Prcfcrably,
`scqucncc using an Intcrnct upload protocol, most prcfcrably zo
`FTP.
`Preferably, the one or more client computers decode the
`sequence and play back the data stream responsive to the
`indices thereof, at a data replay rate generally equal to the
`the one or more client computers
`data rate Preferably,
`download thc cncodc scqucncc using an Intcrnct downloarl
`protocol, most prcfi:rably HTTP or altcrnativcly, UDP or
`RTP.
`Preferably, the one or more client computers include a
`plurality of chent computers, which doivnload the sequence
`substantiallv simultaneously
`Preferably, the network includes the Internet.
`Further prcfi:rably, thc data stream inclurlcs mulhmcrlia
`data, aml thc prcrlctcrmincrl data size of each of the slices
`corresponds to a ume duration of the shee.
`The present invention w&11 be more fully understood from
`the following detailed descnpuon of the preferred embodi-
`ments thereof, taken together with the drawings in which
`BRIEI'L'S('RIP'11ON Ol'I IL DRAWINGS
`I'IG I is a schematic illustratinn of a computer broadcast
`netivork, as is known in the art;
`FIG. 2 is a schematic illustration of a computer broadcast
`network, in accordance with a prcfcrred cmbodimcnt of the
`present invenuon,
`FIG. 3A is a block ihagram that schemaucally illustrates
`a data structure of a broadcast sequence, in accordance with
`a preferred embodiment of the present invention;
`I'IG 313 is a block diagram that schematically illustrates
`an index hie assomated with the data structure of I I(l 333,
`in accordance with a prcfcrrcd cmbodimcnt of thc prcscnt
`invention;
`FIG. 3C is a schematic illustration of a user interface
`graphic, for use in o&a)unct&on with the &late structure of,-&
`FIG. 3A, in ac&or&lance with a preferred emboilrment of the
`present invenuon,
`FIG. 3D is a block diagram that schemaucally illustrates
`a data structure of a broadcast sequence, in accordance with
`another preferred embodiment of the present invention;
`FIG. 4 is a block diagram that schemancally illustrates a
`network connecuon between a transmitung computer anil a
`netivork server, for use in broadcasting of a data sequence,
`in accordance with a preferred embnrliment of the present
`invention;
`that schematically illustrates a
`I'IG 5 is a flow chart
`method of uploading broadcast data frnm a transmitting
`
`n
`
`es
`
`computer to a server, in accordance with a pret erred emboih-
`ment of the present invention;
`I IG 6A is a tloiv chart that schematically illustrates a
`method of downloading broadcast data from a server to a
`in accordance with a prcfcrrcrl embodiment of thc
`client,
`prcscnt invention;
`FIG. 6B is a flow chart that schematically illustrates a
`method ol downloading brorulcast data from a server to a
`client, in secor&lance with another preierred embodiment of
`the present invention;
`that schematically illustrates a
`I IG 7 is a flow chart
`method for preparing data hles for transmission, in accor-
`rlance with a prcfcrred cmborlimcnt of thc present invention;
`that schematically illustrates a
`FIG. 8 is a flow chart
`in accordance with a prcfcrrcd
`method of file transfer,
`embodiment of the present invenuon; and
`that schematically illustrates a
`FIG. 9 is a flow chart
`method lor monitoring network hnks, in accordance with a
`preferred embodiment of the present invention.
`
`DETAILED DESCRIPTION OF PREFERRED
`L'M I IOI)I MENTS
`Rcfcrcncc is now made to FIG. 2, which is a schematic
`illustration of a computer system 32 for remote broadcasting
`of a multimedia sequence over a network 28, in accordance
`with a preferred embodiment of the present
`invention
`System 32 comprises a transmitting computer 34, which
`generates the sequence, a plurahty of clients 30, and a
`network server 36, all of which communicate over network
`28, preferably using thc well-known Intcrnct Protocol (IP).
`Ci&mputer 34 preferably receives audiovisual input from
`input devices 22, although data inputs of other types may be
`generated at or by computer 34 using any suitable means
`knnwn in the art.
`Network 28 preferably cnmprises the Internet, although it
`intranet or other
`may equally comprise a LAN, WAN,
`computer network as is knoivn in thc art. Computer 34 and
`clients 3U preferably compnse conventional personal com-
`puters or workstauons. Server 36 may compnse any suitable
`type ol computer or computer system, for example, a Sun
`Micrnsystems UltraS VAR('tation or a Windows NT server,
`as are commonly used by Internet Service Vroviders (ISVs)
`. In any case, it is noted that transmitting computer 34 can
`bc rcmotcly located rclativc to scrvcr 36, and that thc scrvcr
`need not bc equipped with any special-purpose hardware or
`software for real-time data broadcasting, unlike brea&least
`systems known in the art, such as the real-lime encoder and
`brnadcast server shmvn in I'IG I
`After preparing the multimedia sequence, compuier 34
`uploads the sequence over network 28, preferably using the
`Internet I'ile 'I'ransfer Protocol (I'TV). Alternatively, other
`Internet protocols may be used, such as the I'('V IV, UDV or
`RT(x) protocols, which arc known in thc art. Prcfi:rably, thc
`rlata in the sequence are comprcsscd, although compression
`is not essential to implementation of the present invention.
`The sequence is preferably generated anil compressed in real
`ume, and coul&I comprise, for example, an interview pro-
`gram or an entertainment or sports event, although a prere-
`corded sequence may similarly be broadcast in this manner
`Computer 34 is preferably cquippcrl with suitable software
`for preparing and compressing thc multimedia scquencc. For
`example, for audio data, the computer may typically run
`GSM 6.10 standard audio compression sot)ware, operating
`at a sample rate of 8 kHz, with 16 bits&sample. Some useful
`techniques for preparing, compressmg and transmitting mul-
`timedia sequences are described m U S Vat. No. 5,841,432
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 15
`
`

`

`ic
`
`and in the above-menhoned U.S. patent application Ser No.
`()S,'919,027, both of which are incorporated herein by ref-
`erence
`Clients 30 connect to scrvcr 36 and rcccivc thc multime-
`dia sequence, substanually in real ume. Clients 30 prefer-
`ably download the sequence using the Hypertext Transfer
`Protocol (HTTPj, ahhough other Internet protocols may aLso
`be used, such as UDP or Kl'P, as noted hereinabove with
`reference to uploading by computer 34. Since I'TP and
`HTTP are supported by substantially all network servers,
`scrvcr 36 nccrl not include any special-purpose broadcastin
`hardware or software, as noted above. Similarly, because
`HTTP is supported by substannally all modern Web
`browsers, clients 30 will
`typically neeil only add a Java
`applet or plug-in to their existing Web browsers, as
`described further hereinbelow, in orrler to receive and play
`hack thc broadcast.
`FIG. 3A is a block ihagram that schemaucally illustrates
`the suucture of a stream of broadcast data 40 produced by
`computer 34, typically correspnnrling to a multimedia data 30
`sequence, in accordance with a preferred embodiment of the
`present invention. Data stream 40 comprises a series of data
`slices 42, 44, 46, 48, ctc. Each slice contains a scgmcnt of
`video andior audio clat), corrcspondin
`rcspcctivc,
`to a
`successive time interval labeled T„Ta, Ta, etc. The data are
`preferably compressed, as described further hereinbelow.
`Computer 34 stores each slice as a corresponding file,
`having a running slice index I, 2,3 .. N Preferably, each
`tile also includes one or more time stamps, indicatmg a real
`time at which the rlata in thc file werc rccordcrl or an elapscrl
`time rclativc to thc beginning of stream 40. Thc file arc
`uploaded to server 36, such that while any given slice (other
`than lira) slice 42j is being created, one or more precedin ~
`slices are in the process of bemg uploaded.
`Computer 34 monitors thc time codes as file 40 is
`transmitted, and clients 30 similarly monitor thc time codes
`as the lile is received, in order to ensure that the transmission
`or recephon rs "keeping up'ith the input of. )he data to the
`computer. In the event that a lag is detected, steps are taken
`to increase the data transmission or reception rate, as
`described further hereinbelow I'or example, as shown m
`FIG. 3A, time intervals T„T, T„ctc., arc not all equal, but
`rather are adjusted by computer 34 in rcsponsc to the
`transmission rate. Alternauvely or addiuonally, the compres-
`sion level of the data is varied, as ts likewise descnbed
`below, so as to ad)ust the data streaming rate to the available
`bandwidth over one or more channels between computer 34
`and server 36, and/or between server 36 and client 30
`Computer 34 continues to upload file 42, 44, 46, ctc, sc
`unnl data stream 40 is linished or terminated by a user of
`computer 34. All of the liles in the data stream may be saved
`on server 36 for any desired perind of time, as long as the
`server has sufficient
`is accessilile to
`free memory that
`computer 34 Typically, however, the memory available on
`scrvcr 36 is limited, and files 42, 44, 46, ctc., will be storer)
`on thc scrvcr and crascd thcrcfrom in a
`"first-in-first-out'equence.
`I'IG 313 is a block diagram that schematically illustrates
`an index file 50, which is created by computer 34, and is
`in accordance with a prefcrrcrl
`uploadcd to scrvcr 36,
`embodiment of the prcscnt invention. Thc index fil com-
`pnses a shee ID 52, indicaung the imlex of the lile in data
`stream 4U that was most recently uploaded by computer 34.
`Each nme a new lile 42, 44, 46, etc., is uploaded, ID 52 in ss
`tile 50 on server 36 is updated Preferably, 0) 52 holds the
`tile name of the neiv file, wherein the name typically
`
`33
`
`comprises a sinn ~ followed by the index of the file. When
`one of computers 30 connects to server 36 and begins to
`rlownload the data stream, it first rcarls thc index hlc in order
`to identify at what point in stream 40 to begin and to start
`receiving the data stream substantially in real time, prefer-
`is transmitted from
`ably with only a minimal
`lag, as it
`computer 34. Alternatively, a user of one of computers 30
`may choose to be nn downloading data stream 4U from an
`carlicr point in time than that indicated by ID 52. Further
`alternanvely, stream 40 may be multicast to clients 3U, as is
`known in the art, typically without the use of an index lile.
`Index file 50 may further inclurlc a mcssagc 54, which is
`rerul by computers 30 when they connect to server 36 to
`ilownload data abeam 40 or, ahernauvely or addiuonally, at
`any time the message is updated by computer 34 The
`message contains parameters relatmg generally to the data
`stream and,'or instructions to computers 30, for example,
`"transmission paused." FIG. 3C is a schematic rcprcscnta-
`tion of a user intcrfacc graphic "slider" 55, available to users
`of computers 30, in accordance with a preferred embodiment
`of the present
`invenhon. Slider 55, which is preferably
`displayed on the screens of computers 30, mcludes a bar 56
`anti a movable indicator 58 The symbnls.l, 1+1,.1+2,
`N
`in the figure are the indices of the slices of stream 40 that are
`stored on scrvcr 36, whcrcin N is thc index of thc most
`rcccnt slice, and J is thc inc)ex of thc carlicst stored slice. J
`may intlicate the lirst slice in the sequence, iT. all of the )iles
`are stored on server 36, or it may be the earliest lile not yet
`erased (The indices are marked in the hgure on bar 56 for
`clarity, and need not actually be shown on the computer
`screen )
`When one of computers 30 reads index tile 50 anil liegins
`to download stream 40, indicator 58 preferably marks the
`most recent slice, as shov n in FIG 3C. This is thc point at
`which the dovvnload v ill begin, unless thc user of thc
`computer chooses othcrwisc. If thc user wishes to bc in thc
`ilownload at an earlier point, he may move indicator 58 to
`the left along bar 56 to that point, preferably using a mouse
`or other pointing device, as is known in the art
`lnrlicator 58
`may be moved hack and forth along bar 56 to jump back and
`forth along stream 40.
`I IG 3D is a block diagram that schematically illustrates
`a tile format of a multi-level data stream 41, in accordance
`with another prcfcrred cmborlimcnt of thc prcscnt invention.
`Thc data stream is divided into aurlio slices 45, 47 and video
`slices 49, 51, and may also inclurlc other rlata formats, such
`as a text shee. 53 anil!or a URL slice 55. Each slice is
`preferablv itlentilied by a level itlentiiier 57, a presentation
`time stamp (P'IS) index 59 and, as appropriate, a size
`identifier 61 The function of these identifiers and indices is
`rlcscrilicd further hcrcinbclow. A hcadcr 43 includes clara
`such as the title, author, copyright anil formats of thc data in
`the stream; the duration of thc multimedia scqucncc rcprc-
`sented by the stream, and a descnption of the available
`stream levels and associated data sizes.
`Each time slice in stream 41 includes multimedia data at
`multiple quality levels. There are two such levels in the
`example shown in FIG. 3D, itlentilied as level ¹1 and level
`¹2, but a larger number of levels may also be used. Typically,
`the audio and video data in level ¹1, contained in slices 45
`and 49, are morc highly comprcsscrl rclativc to thc data in
`slices 47 and 51 of level ¹'2. In conscqucncc, thc level ¹I
`slices have smaller data volume than the level ¹2 slices and
`can therefore be transmitted over a lower-bandwidth data
`link, wide maintaining the required shee uming indicated
`by time stamps 59. The loiver data-rate transmission gener-
`ally comes at the expense of inferior sounrl anrLior image
`
`Amazon / WAG Acquisition
`Exhibit 1005
`Page 16
`
`

`

`quality. Size idenulier 61 descrd&es the size of those slices in
`stream 41 that have a fixe&I size associated therewith,
`wherein typically the size (nr the corresponding resolution)
`of the level ¹I video slices is smaller than that of the level
`¹'Z slices.
`Each of. chen&a 3U chooses or is assigned the quality level
`appropriate to the bandwidth of its hnl on network 28 to
`server 36. A method for selecting and, as required, varying
`the level is described hereinbelow ivith reference to I'IG 68
`FIG. 4 i