`US008122141B2
`
`c12) United States Patent
`Price
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 8,122,141 B2
`*Feb.21,2012
`
`(54) STREAMING MEDIA BUFFERING SYSTEM
`
`(75)
`
`Inventor: Harold Edward Price, Bethel Park, PA
`(US)
`
`(73) Assignee: WAG Acquisition, LLC, Flanders, NJ
`(US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by O days.
`
`This patent is subject to a terminal dis(cid:173)
`claimer.
`
`(21) Appl. No.: 12/800,152
`
`(22) Filed:
`
`May 10, 2010
`
`(65)
`
`Prior Publication Data
`
`US 2010/0235536 Al
`
`Sep. 16, 2010
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 10/893,814, filed on
`Jul. 19, 2004, now Pat. No. 7,716,358, which is a
`continuation-in-part of application No. 09/819,337,
`filed on Mar. 28, 2001, now Pat. No. 6,766,376.
`
`(60) Provisional application No. 60/231,997, filed on Sep.
`12, 2000.
`
`(51)
`
`Int. Cl.
`G06F 15116
`(2006.01)
`(52) U.S. Cl. ....................................................... 709/231
`( 58) Field of Classification Search . ... ... ... ... .. ... 709/230,
`709/231
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,526,353 A
`6/1996 Henley et al. ................ 370/60.1
`5,793,980 A
`8/1998 Glaser et al .............. 395/200.61
`5,822,537 A
`10/1998 Katseff et al. ............ 395/200.61
`
`5,922,048 A
`5,923,655 A
`5,928,330 A *
`5,999,525 A
`6,002,720 A
`6,014,693 A
`6,014,694 A
`6,014,706 A
`6,029,194 A
`6,047,317 A
`6,057,832 A
`6,061,731 A
`6,061,732 A
`6,065,050 A
`6,085,221 A
`6,292,834 Bl
`
`7/1999 Emura .......................... 709/219
`7/1999 Veschi et al. .................. 370/394
`7/1999 Goetz et al. ................... 709/231
`12/1999 Krishnaswamy et al.
`.... 370/352
`12/1999 Yurt et al.
`..................... 375/240
`1/2000 Ito et al ......................... 709/219
`1/2000 Aharoni et al ................ 709/219
`1/2000 Cannon et al. ................ 709/231
`2/2000 Tilt ............................... 709/219
`4/2000 Bisdikian et al. ............. 709/217
`5/2000 Lev et al. ...................... 345/327
`5/2000 Blakeslee ..................... 709/231
`5/2000 Korst et al.
`................... 709/231
`5/2000 DeMoney ..................... 709/219
`7 /2000 Graf .............................. 709/202
`9/2001 Ravi et al.
`(Continued)
`
`OTHER PUBLICATIONS
`
`Salehi et al., "Supporting Stored Video: Reducing Rate Variability
`and End-to-End Resource Requirements through Optimal Smooth(cid:173)
`ing," IEEE/ACM Transactions on Networking, vol. 6, Issue 4, p.
`397-410, Aug. 1998.
`
`(Continued)
`
`Joseph Avellino
`Primary Examiner -
`Assistant Examiner - Marshall McLeod
`(74) Attorney, Agent, or Firm - Ernest D. Buff; Ernest D.
`Buff & Assoc. LLC; Gordon E. Fish
`
`(57)
`
`ABSTRACT
`
`Streaming media, such as audio or video files, is sent via the
`Internet. The media are immediately played on a user's com(cid:173)
`puter. Audio/video data is transmitted from the server more
`rapidly than it is played out by the user system. The audio/
`video data in the user buffer accumulates; and interruptions in
`playback as well as temporary modem delays are avoided.
`
`28 Claims, 3 Drawing Sheets
`
`12
`
`~ _ } , 8
`~
`22 (~°', ~ 20
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 1
`
`
`
`US 8,122,141 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`7/2003 Eyer et al.
`6,588,015 Bl
`7/2003 Sahai et al.
`6,594,699 Bl
`6,598,228 B2
`7 /2003 Hejna
`6,637,031 Bl*
`10/2003 Chou .............................. 725/87
`6,665,751 Bl
`12/2003 Chen et al.
`3/2004 Bommaiah et al.
`6,708,213 Bl
`6,728,753 Bl*
`4/2004 Parasnis et al. ............... 709/203
`6,829,368 B2
`12/2004 Meyer et al.
`6,889,257 Bl
`5/2005 Patel
`6,907,481 B2
`6/2005 Kovacevic
`7,054,500 Bl
`5/2006 Lillevold
`7,111,058 Bl
`9/2006 Nguyen et al.
`7,170,856 Bl
`1/2007 Ho et al.
`
`3/2008 Hannaway
`7,346,698 B2
`7,373,413 Bl
`5/2008 Nguyen et al.
`2001/0047377 Al* 11/2001 Sincaglia et al.
`
`................. 709/1
`
`OTHER PUBLICATIONS
`
`in an
`"Smoothing variable-bit-rate video
`Rexford et al.,
`internetwork," IEEE/ ACM Transactions on Networking, vol. 7, Issue
`2, p. 202-215, Apr. 1999.
`Bianchi, "Buffer sizing for high speed video information retrieval on
`ATM networks," Globecom-NewYork-, 1997, vol. 2, pp. 1057-
`1061.
`
`* cited by examiner
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 2
`
`
`
`U.S. Patent
`
`Feb.21,2012
`
`Sheet 1 of 3
`
`US 8,122,141 B2
`
`Fig. 1
`
`12
`
`= '
`_,
`=,
`
`1c26
`
`/14
`,.
`
`◄
`
`24a
`
`I
`
`I
`
`I
`
`24n
`
`18
`
`.------, ,-----,
`
`I
`I
`I _ _ _ _ _ _ 1
`
`I
`I
`
`I
`
`18
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 3
`
`
`
`U.S. Patent
`
`Feb.21,2012
`
`Sheet 2 of 3
`
`US 8,122,141 B2
`
`Fig. 2
`
`28
`
`12
`
`_,
`I = ,
`
`-
`
`~ -
`~ I
`
`_,
`=• =,
`
`-
`
`I
`I
`
`30
`
`,-----, ,-----,
`
`I
`I
`I
`, ______ 11
`
`I
`
`18
`
`22
`
`1-=-( =====--~!===..l.,\ ~ 20 22
`
`{-'=( ===='---~~~,~ 20
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 4
`
`
`
`U.S. Patent
`
`Feb.21,2012
`
`Sheet 3 of 3
`
`US 8,122,141 B2
`
`Fig. 3
`
`..
`-
`
`-~
`
`32
`
`, ,
`
`34
`
`1~
`
`36
`
`1 '
`
`38
`
`1 '
`
`40
`
`1'
`
`42
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 5
`
`
`
`US 8,122,141 B2
`
`1
`STREAMING MEDIA BUFFERING SYSTEM
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`2
`results in unsatisfactory viewing or listening, if viewing or
`listening is possible at all. The connection available to most
`Internet users is by dial-up modem, which has a maximum
`receive data rate of 56,000 bits per second. Most audio and
`5 video available on the Internet has been compressed to be
`listenable or viewable within the 56,000 bits per second
`modem bandwidth. Requirements for achieving adequate
`audio and video over the Internet generally consume a con-
`siderable portion of the listener's available bandwidth.
`Internet connection quality can vary rapidly overtime, with
`two primary factors responsible for degradation of the instan(cid:173)
`taneous bandwidth actually available to the user. These fac(cid:173)
`tors are the quality of the user's modem connection over
`telephone lines, which can have periods of interference caus-
`15 ing reduced available bandwidth, and momentary Internet
`congestion at various points along the route over which the
`user's data flows. Each of these factors can cause delays and
`interruptions in the transmission of data to the user. Internet
`data communications devices such as routers are designed to
`20 drop data "packets" if they get overloaded. For material that is
`not time sensitive, these dropped packets will usually be
`resent, and the user will eventually be presented with the
`material. However, since streaming media is time sensitive,
`dropped packets can have a significant impact on the receipt
`25 and playback of an audio or video stream. These degradations
`in the receipt oflnternet data are very common, and prevent
`most users from being able to listen to or view streaming
`media without interruption unless some special provisions
`have been incorporated into the user's computer software to
`30 accommodate data transmission interruptions.
`These interruptions are commonly referred to as "drop(cid:173)
`outs", meaning that the data flow to the user has been inter(cid:173)
`rupted (i.e., the audio "drops out"). Dropouts can be
`extremely annoying-for example, while listening to music.
`35 The current state-of-the-art solution to the problem uses a
`pre-buffering technique to store up enough audio or video
`data in the user's computer so that it can play the audio or
`video with a minimum of dropouts. This process requires the
`user to wait until enough of the media file is buffered in
`40 memory before listening or viewing can begin. The media
`data is delivered by a server computer which has available to
`it the source of the media data, such as by a connection to a
`radio station. When the user connects to the server via the
`Internet, audio/video output at the user's system is delayed
`45 while the user's buffer is filled to a predetermined level.
`Typical pre-buffering wait times range from 10 to 20 seconds
`or more, determined by the vendor providing the audio or
`video media. Even with this pre-buffering process, interrup-
`tions in playback still occur.
`In this process, the user has a software application on the
`computer commonly called a "media player". Using the fea(cid:173)
`tures built into the media player, the user starts the audio or
`video stream, typically by clicking on a "start" button, and
`waits 10-20 seconds or so before the material starts playing.
`55 During this time data is being received from the source and
`filling the media player's buffer. The audio or video data is
`delivered from the source at the rate it is to be played out. If,
`for example, the user is listening to an audio stream encoded
`to be played-out at 24,000 bits per second, the source sends
`60 the audio data at the rate of 24,000 bits per second. Provided
`that the user waits 10 seconds, and the receipt of the buffering
`data has not been interrupted, there is enough media data
`stored in the buffer to play for 10 seconds.
`Gaps in the receipt of audio/video data, due to Internet
`65 slowdowns, cause the buffer to deplete. Because transmission
`of audio/video media data to the user takes place at the rate it
`is played out, the user's buffer level can never be increased or
`
`This application is a continuation of U.S. patent applica(cid:173)
`tion Ser. No. 10/893,814, filed Jul. 19, 2004 (published on
`Dec. 9, 2004 as U.S. patent publication number 2004/
`0249969 Al, and now U.S. Pat. No. 7,716,358), which was a
`continuation-in-part of U.S. patent application Ser. No. 10
`09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No. 6,766,
`376), which claimed the benefit under 35 U.S.C. §119(e) of
`U.S. provisional patent application Ser. No. 60/231,997, filed
`Sep. 12, 2000; claims the benefit, under 35 U.S.C. §120, of
`the respective filing dates of said applications, as well as
`benefit of the filing date of copending U.S. patent application
`Ser. No. 10/825,869, filed Apr. 16, 2004 (published on Dec.
`23, 2004 as U.S. patent publication number 2004/260828
`Al), which was a continuation of said U.S. patent application
`Ser. No. 09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No.
`6,766,376), which claimed the benefit under 35 U.S.C. §119
`(e) of said U.S. provisional patent application Ser. No.
`60/231,997, filed Sep. 12, 2000; and hereby incorporates by
`reference the entire disclosure of each of said prior applica-
`tions.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`The present invention relates to multimedia computer com(cid:173)
`munication systems; and more particularly, to a buffering
`system for streaming media, such as audio/video, on the
`Internet.
`2. Description of the Related Art
`Prior to the development oflnternet streaming media tech(cid:173)
`nologies, audio and video were formatted into files, which
`users needed to download to their computer before the files
`could be heard or viewed. Real time, continuous media, as
`from a radio station, was not suitable for this arrangement in
`that a file of finite size must be created so it could be down(cid:173)
`loaded. The advent of streaming media technologies allowed
`users to listen or view the files as they were being down(cid:173)
`loaded, and allowed users to "tune-in" to a continuous media
`broadcast, or "stream", such as from a radio station. There are
`two fundamental types of streaming media: (i) material that
`originates from a source having a real-time nature, such as a
`radio or TV broadcast, and (ii) material that originates from a
`non-real-time source such as from a disk file. An example of
`non-real-time material might be a piece of music stored as a
`disk file, or a portion of a broadcast that originally was real- 50
`time, perhaps yesterday's TV evening news, and was
`recorded into a disk file. For purposes of clarity within this
`document, streaming media of type (i) will be referred to as
`"broadcast" media, and streaming media of type (ii) will be
`referred to as "file based" media. Broadcast streaming media
`has as its source a system or arrangement that by definition
`can only be transmitted to users as fast as the material is
`generated; for example, a disk jockey speaking into a micro(cid:173)
`phone. Broadcast streaming media is the focus of this patent
`application.
`Since audio and video media must play out over a period of
`time it is more appropriate to think of bandwidth require(cid:173)
`ments than file size. The bandwidth requirement of an audio
`or video media refers to the data rate in bits per second that
`must be transmitted and received in order to listen or view the
`material uninterrupted. Transmitting the audio or video mate(cid:173)
`rial over a connection slower than the bandwidth requirement
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 6
`
`
`
`US 8,122,141 B2
`
`3
`replenished while it is playing. Thus, gaps in the receipt of
`audio/video media data inexorably cause the buffer level to
`decrease from its initial level. In time, extended or repeated
`occurrences of these gaps empty the user's buffer. The audio/
`video material stops playing, and the buffer must be refilled to
`its original predetermined level before playing of the media
`resumes.
`By way of illustration in a 10 second pre-buffering sce(cid:173)
`nario, if the data reception stopped the instant that the media
`started playing, it would play for exactly 10 seconds. Once it
`starts playing, the media data plays out of the buffer as new
`media data replenishes the buffer. The incoming data rate
`equals the rate at which the data is played out of the user's
`buffer, assuming the receipt of data across the Internet is
`unimpeded. If there are no interruptions in the receipt of the
`media data for the duration of the time the user listens to or
`watches the material, the buffer level remains constant and
`there will still be 10 seconds of data stored in the media
`player's buffer when the user stops the player. On the other
`hand, if the media player encounters interruptions totaling 6
`seconds while playing the material, there would only be 4
`seconds of media data remaining in the buffer when the user
`stopped it. If data reception interruptions at any time during
`the playing exceed 10 seconds, the user's media player buffer
`becomes exhausted. There is no media data to play, and the
`audio or video stops-a dropout has occurred. At this point a
`software mechanism in the media player stops attempting to
`play any more of the material, and starts the buffering process
`again. The media player remains silent until the buffer refills,
`at which time the media player will once again start playing
`the material.
`There are two fundamental types of streaming media: (i)
`material that originates from a source having a real-time
`nature, such as a radio or TV broadcast, and (ii) material that 35
`originates from a non-real-time source such as from a disk
`file. An example of non-real-time material might be a piece of
`music stored as a disk file, or a portion of a broadcast that
`originally was real-time, perhaps yesterday's TV evening
`news, and was recorded into a disk file. For purposes of clarity 40
`within this document, streaming media of type (i) will be
`referred to as "broadcast" media, and streaming media of type
`(ii) will be referred to as "file based" media.
`Both streaming media types are handled similarly in con(cid:173)
`ventional systems, and both are handled similarly by the 45
`streaming media buffering system of the present invention.
`The two streaming media types are readily distinguished.
`Broadcast streaming media has as its source a system or
`arrangement that by definition can only be transmitted to
`users as fast as the material is generated; for example, a disk 50
`jockey speaking into a microphone. File based media, on the
`other hand, can be transmitted to users at any data rate, since
`there is no inherent time element to a file residing on a com(cid:173)
`puter disk. With conventional Internet streaming media sys(cid:173)
`tems for streaming media of either type, media data is trans- 55
`mitted from the server to the user at the rate at which it will be
`played out, regardless of the data rate capabilities of the
`connection between the server and the user.
`Conventional streaming media systems may incorporate
`buffering systems for progranimatic purposes. For example,
`the system may buffer media data at the server for the purpose
`of packet assembly/disassembly. Media data may also be
`buffered at the server to permit programming conveniences
`such as dealing with chunks of data of a specific size. Such
`server buffering of media data is not used by conventional
`streaming media systems to mitigate long term Internet per(cid:173)
`formance degradation as described hereinafter.
`
`4
`The sending of audio or video files via a network is known
`in the art. U.S. Pat. No. 6,029,194 to Tilt describes a media
`server for the distribution of audio/video over networks, in
`which retrieved media frames are transferred to a FIFO
`5 buffer. A clock rate for a local clock is adjusted according to
`the fullness of the buffer. The media frames from the buffer
`are sent in the form of data packets over the networks in
`response to interrupts generated by the local clock. In this
`manner, the timing for the media frames is controlled by the
`10 user to assure a continuous stream of video during editing.
`U.S. Pat. No. 6,014,706 to Cannon, et al. discloses an appa(cid:173)
`ratus and method for displaying streamed digital video data
`on a client computer. The client computer is configured to
`receive the streamed digital video data from a server com-
`15 puter via a computer network. The streamed digital video data
`is transmitted from the server computer to the client computer
`as a stream of video frames. U.S. Pat. No. 6,002,720, to Yurt,
`et al. discloses a system of distributing video and/or audio
`information wherein digital signal processing is employed to
`20 achieve high rates of data compression. U.S. Pat. No. 5,923,
`655, to Veschi et al. discloses a system and method for com(cid:173)
`municating audio/video data in a packet-based computer net(cid:173)
`work wherein transmission of data packets through the
`computer network requires variable periods of transmission
`25 time. U.S. Pat. No. 5,922,048 to Emura discloses a video
`server apparatus having a stream control section which deter(cid:173)
`mines a keyframe readout interval and a keyframe playback
`interval that satisfy a playback speed designated by a terminal
`apparatus. Finally, U.S. Pat. No. 6,014,694 to Aharoni, et al.
`30 discloses a system and method for adaptively transporting
`video over networks, including the Internet, wherein the
`available bandwidth varies with time.
`There remains a need in the art for a method and system
`that afford immediate and uninterrupted listening/viewing of
`streaming media by the user.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides a system and method for
`sending streaming media, such as audio or video files, via the
`Internet. Immediate playing of the media on a user's com(cid:173)
`puter is afforded while reducing interruptions in playback due
`to Internet congestion and temporary modem delays due to
`noisy lines. Nearly instantaneous playback is achieved, while
`maintaining protection against playback
`interruption.
`Delayed starts, heretofore required to provide protection
`against interruption, are avoided. Data loss due to interrup(cid:173)
`tions in the receipt of media data by the media player can be
`recovered while the player continues to play out the audio or
`video material. If the interruptions are so severe as to deplete
`the user's buffer and stop the play out, the media player will
`begin to play out again as soon as the media player begins to
`receive media data without waiting to first build up the buffer.
`Generally stated, the invention provides a system for dis-
`tributing via the Internet streaming media composed of a
`plurality of time-sequenced data elements. The system has a
`server connected to the Internet for transmitting the data
`elements. Associated with the server are a buffer manager and
`a FIFO buffer for storing at least one of the data elements for
`60 transmission. The buffer manager comprises means for:
`receiving the media data; supplying media data in order to the
`FIFO buffer; supplying the FIFO buffer with a predetermined
`numberof data elements; maintaining a pointer into the buffer
`for each user computer indicating the last media data element
`65 that has been sent to that user, thus indicating the next element
`or elements to be sent; and, once the FIFO buffer is full,
`deleting the oldest data elements in the buffer as new data
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 7
`
`
`
`US 8,122,141 B2
`
`5
`
`35
`
`5
`elements are received, said means arranged to maintain the
`pre-determined number of data elements in the FIFO buffer.
`At least one user computer is connected to the server via the
`Internet or other data communications medium.
`This invention presumes the existence of a data communi(cid:173)
`cations transport mechanism, such as the TCP protocol, for
`the reliable delivery of data in an ordered sequence from the
`source of the media data to the server, or from the server to the
`media player software of the user computer. Thus, the deliv(cid:173)
`ery of data in the proper sequence is outside the scope ofthis 10
`invention.
`The user computer is associated with a media player soft(cid:173)
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined number of media data
`elements which are received sequentially by the media player,
`playing the data out sequentially as audio and/or video, and
`deleting media data elements from the buffer as they are
`played out. As data is played out, the next sequential data
`elements are received from the server in such a fashion as to
`approximately maintain the predetermined number of data
`elements in the user's buffer.
`There are two types of encoding schemes used for audio
`and video material-"Variable Bit Rate"-VBR, and "Con(cid:173)
`stant Bit Rate"-CBR. CBR encoding represents the encoded
`media with a constant bit rate per second, regardless of the 25
`complexity of the material being encoded, for example, if an
`audio source is encoded at 20 kilobits per second at a Constant
`Bit Rate, the media data being produced from the encoding is
`at 20 kilobits per second whether the audio material is sym(cid:173)
`phonic music or silence. Variable Bit Rate encoding uses a 30
`variable number of bits to represent sounds, with more bits
`required for complex (symphonic) sounds than for simple
`sounds or silence. The standard encoding scheme used for
`streaming media is CBR because the resulting data rate is
`more predictable than for VBR.
`The server stores a predetermined amount of media data in
`a First-In First-Out (FIFO) buffer in an arrangement that
`receives media data directly or indirectly from a real-time
`source, such as a radio station. For example, the server buffer
`might be set to store up 30 seconds of media data. Because the 40
`source produces media data in real time, the media data is
`delivered to the server approximately at the rate it is gener(cid:173)
`ated. Of course there can be variability's in this data delivery
`process due to networking, disk accesses, and so on, causing
`the delivery rate of the media data to be variable over short 45
`periods of time, typically measured in seconds. But over a
`longer period of time measured in minutes or tens of minutes
`or longer, the media data is delivered from source to server at
`the rate it is generated, and the server in turn provides that
`media data to the FIFO buffer at that same rate. Since CBR 50
`encoding is normally used for streaming media, the media
`data is generated, received by the server, and provided to the
`buffer approximately at a fixed rate. Once the buffer is full, for
`each new data element received into the buffer the oldest data
`element is deleted from the buffer. Once a connection is made 55
`to a user's computer, the server sends the media data to the
`user computer's buffer in the following manner. First, media
`data is sent to the user at the highest rate that the data con(cid:173)
`nection between the server and the user computer will support
`until the predetermined amount of data that had been stored in
`the server buffer has been transferred to the user's computer.
`Once the buffer has been transferred a steady state condition
`is reached wherein as each media data element arrives at the
`server, it is immediately sent out to the user computer. In this
`steady state condition, the media data is sent at a rate that
`matches the constant fill rate of the server buffer, and is
`received at the same rate by the user computer ifthere are no
`
`6
`interruptions in the transmission of media data between the
`server and the user's computer. If interruptions have inter(cid:173)
`fered with the arrival of sent media data to the user's com(cid:173)
`puter, that data may have been "dropped" by routers in the
`Internet and needs to be resent. This causes data to "back up"
`into the server FIFO for that user.
`In one method of operation, the resending of missing data
`is the responsibility of a reliable transport mechanism, such as
`TCP. The server buffer "sends" data by delivering it to the
`transport mechanism. The transport mechanism actually
`"sends" the data across the communications medium, and has
`processes which determine if all the data that has been sent
`has been received by the destination. If not, missing pieces of
`data are automatically resent to the destination, and are
`15 arranged to be delivered to the target software on the desti(cid:173)
`nation system in an ordered fashion. In the circumstance of
`this invention, the destination is the user computer, and the
`target software on the destination system is the media player.
`If the transport mechanism determines that data is missing, it
`20 retransmits that data to the destination as fast as the connec(cid:173)
`tion between the server and destination will allow. The net
`effect of this invention is that all media data to be delivered to
`a user computer is always sent as fast as the communications
`medium will support, either by the server buffer passing
`media data to the transport mechanism, or by the transport
`mechanism delivering or redelivering the media data to the
`user computer. This is enabled by buffering data at the server,
`and is distinctly different from prior art in which media data
`is only sent from the server to the user computer at the rate at
`which it is to be played out.
`In another method of operation, the server can use an
`unreliable transport mechanism, such as UDP, and rely on a
`streaming software process to manage data delivery and the
`resending of data elements not received by the media player.
`As an example of the preceding description, if the server
`had been set to store 30 seconds of audio in its buffer, when a
`user connects that 30 seconds worth of media, data is trans(cid:173)
`ferred to the user's media player buffer as fast as the data
`connection between the two will allow. The media player can
`begin playing as soon as it has received a very minimum
`amount of data, perhaps comprising only a single packet of
`media data. For ease of understanding, consider the server
`buffer and the media player buffer to be an elastic system that
`between the two stores up 30 seconds of audio data. The
`server starts with 30 seconds of buffered audio data which it
`transfers to the media player until the server has no buffered
`media data and the media player has 30 seconds of buffered
`media data. Regardless of how much of the buffered media
`data has been transmitted to the media player, there always is
`30 seconds of media data being buffered between the two
`locations. Consequently, the audio being played out by the
`media player will always be 30 seconds behind the audio at
`the source. If there were a media player in the radio station
`studio, an announcer would hear themselves through the
`media player with a 30 second delay.
`Routinely, once a steady state has been achieved, the next
`data element to be sent is the next sequential data element
`from that which has already been received by the user's
`computer buffer. However, ifthere is more data to be sent than
`60 at the routine constant fill rate, such as in the condition where
`some media data has been resent by the reliable transport
`layer, the server transport mechanism will again send the
`buffered media data as fast as the connection between the
`server and the user's computer will support. Similarly, if the
`65 media player buffer begins to deplete or becomes depleted
`due to networking interruptions, the server will attempt to
`send as much data as is necessary to rebuild the user comput-
`
`Amazon / WAG Acquisition
`Exhibit 1015
`Page 8
`
`
`
`US 8,122,141 B2
`
`7
`er's buffer to the proper level. This allows for rebuilding the
`user's computer buffer under circumstances wherein Internet
`interruptions have blocked the normal flow of data. When
`compared to conventional systems, which provide no capa(cid:173)
`bility to rebuild the user's computer buffer when data is lost, 5
`the streaming media buffering system of the present invention
`provides for recovery oflost data elements and the restoration
`of the user's buffer, even while the user media player contin(cid:173)
`ues to play.
`Under conditions in which interruptions have interfered 10
`with the arrival of sent media data to the user's computer, data
`loss exceeding certain levels will cause the transport mecha(cid:173)
`nism software to stop accepting data for transmission from
`the application software, namely the streaming media server
`software. Although other arrangements are possible within 15
`the scope of this invention, in the preferred embodiment, the
`streaming media server software keeps track of the last data
`element in the FIFO buffer that has been "sent" to each user
`using a software pointer. An interruption in the ability to send
`media data to a user results in this pointer "backing up" in the 20
`FIFO in such a way that the server knows from what point in
`the buffer to restart sending data when the transport mecha(cid:173)
`nism again requests data to send. When the server software
`receives that notification, it will begin sending data to the user
`starting from the next data element to send as indicated by the 25
`pointer, and sending as much data as the transport mechanism
`will accept. The transport mechanism will again send this
`data as fast as it can to the user. This process continues until
`the steady state condition is again reached wherein each data
`element is sent to the user as soon as it arrives from the media 30
`source.
`In another embodiment, the server is connected to the
`Internet, and to a broadcast media source, such as a radio
`station. A radio station computer is provided with a means for
`receiving media data elements as they are generated by the 35
`audio and/or video source, and for transmitting those media
`data elements to the server as they are generated. As before,
`the server provides a buffer manager and a FIFO buffer, and
`provides a means for receiving the sequentially arranged
`media data elements from the broadcast media source and 40
`storing those data elements in the FIFO buffer. The buffer
`manager comprises means for: supplying the FIFO buffer
`with a predetermined number of data elements; maintaining a
`pointer into the buffer for each user computer indicating the
`last media data element that has been sent to that user, thus 45
`indicating the next element or elements to be sent; and, once
`the FIFO buffer is full, deleting the oldest data element in the
`buffer as each new data element is received. Importantly, the
`buffer manager is arranged to maintain the pre-determined
`number of data elements in the FIFO buffer. At least one user 50
`computer is connected to the server via the Internet or other
`data communications medium.
`The user computer is associated with a media player soft(cid:173)
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined number of media data
`elements, playing the data out sequentially as audio and/or
`video, and deleting media data elements from the buffer as
`they are played out. As data is played out, the next sequential
`data elements a