`
`(12) United States Patent
`Price
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,122,141 B2
`*Feb. 21, 2012
`
`(54) STREAMING MEDIA BUFFERING SYSTEM
`
`(75) Inventor: Harold Edward Price, Bethel Park, PA
`(
`)
`
`(73) Assignee: WAG Acquisition, LLC, Flanders, NJ
`(US)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`This patent is Subject to a terminal dis-
`claimer.
`(21) Appl. No.: 12/800,152
`
`(22) Filed:
`(65)
`
`May 10, 2010
`Prior Publication Data
`US 2010/0235536A1
`Sep. 16, 2010
`
`Related U.S. Application Data
`(63) Continuation of application No. 10/893,814, filed on
`Jul. 19, 2004, now Pat. No. 7,716,358, which is a
`continuation-in-part of application No. 09/819,337,
`filed on Mar. 28, 2001, now Pat. No. 6,766,376.
`(60) Provisional application No. 60/231,997, filed on Sep.
`12, 2000.
`
`(51) Int. Cl.
`(2006.01)
`G06F 15/16
`(52) U.S. Cl. ....................................................... 709/231
`(58) Field of Classification Search .................. 709/230,
`709/231
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,526,353 A
`6/1996 Henley et al. ................ 370/60.1
`5,793,980 A
`8, 1998 Glaser et al. .....
`... 395.200.61
`5,822,537 A 10, 1998 Katseffet al. ............ 395/200.61
`
`
`
`7, 1999 Emura .......................... TO9,219
`5,922,048 A
`7/1999 Veschi et al. ...
`370,394
`5,923,655 A
`5,928,330 A * 7/1999 Goetz et al. ................... TO9,231
`5.999,525 A 12/1999 Krishnaswamy et al. ... 370/352
`6,002,720 A 12/1999 Yurt et al. ..................... 375,240
`6,014,693 A
`1/2000 Ito et al. .........
`TO9,219
`6,014,694. A
`1/2000 Aharoni et al. ....
`TO9,219
`6,014,706 A
`1/2000 Cannon et al. .
`TO9,231
`6,029, 194 A
`2, 2000 Tilt
`. . . . . . . . . .
`.
`. . . .
`TO9,219
`88: A
`$398 Pisa et al.
`(232
`6061731 A
`5, 2000 Blakeslee ...
`TO9,231
`6,061,732 A
`5, 2000 Korst et al. .
`TO9,231
`6,065,050 A
`5/2000 DeMoney ...
`TO9,219
`6,085,221 A
`7/2000 Graf .............................. TO9,202
`6,292,834 B1
`9, 2001 Ravi et al.
`(Continued)
`
`OTHER PUBLICATIONS
`Salehi et al., “Supporting Stored Video: Reducing Rate Variability
`and End-to-End Resource Requirements through Optimal Smooth
`ing.” IEEE/ACM Transactions on Networking, vol. 6, Issue 4, p.
`397-410, Aug. 1998.
`
`(Continued)
`
`Primary Examiner — Joseph Avellino
`Assistant Examiner — Marshall McLeod
`(74) Attorney, Agent, or Firm — Ernest D. Buff; Ernest D.
`Buff & Assoc. LLC; Gordon E. Fish
`
`(57)
`
`ABSTRACT
`
`Streaming media, such as audio or video files, is sent via the
`Internet. The media are immediately played on a user's com
`puter. Audio/video data is transmitted from the server more
`rapidly than it is played out by the user system. The audio/
`Video data in the user buffer accumulates; and interruptions in
`playback as well as temporary modem delays are avoided.
`
`28 Claims, 3 Drawing Sheets
`
`Petitioners' Exhibit 1061
`Page 0001
`
`
`
`US 8,122,141 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`6,588,015 B1
`7/2003 Eyer et al.
`6,594,699 B1
`7/2003 Sahai et al.
`6,598,228 B2
`7/2003 Hejna
`6,637,031 B1 * 10/2003 Chou .............................. 725/87
`3.
`R lS. S. et i al
`
`tal.
`.
`.
`.
`
`w sy
`6,728,753 B1 * 4/2004 E. et
`6.829,368 B2 12/2004 Meyer et al.
`6,889,257 B1
`5, 2005 Patel
`6,907,481 B2
`6, 2005 Kovacevic
`7,054,500 B1
`5, 2006 Lillevold
`7,111,058 B1
`9/2006 Nguyen et al.
`7,170,856 B1
`1/2007 Ho et al.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. TO9,203
`
`3/2008 Hannaway
`7,346,698 B2
`5/2008 Nguyen et al.
`7,373,413 B1
`ck
`2001/0047377 A1* 1 1/2001 Sincaglia et al. ................. 709/1
`OTHER PUBLICATIONS
`Rexford et al., “Smoothing variable-bit-rate video in an
`
`s
`internetwork.” IEEE/ACM Transactions on Networking, vol.7. Issue
`2, p. 202-215, Apr. 1999.
`Bianchi. “Buffer sizing for high speed video information retrieval on
`ATM networks.” Globecom New York , 1997, vol. 2, pp. 1057
`1061.
`
`* cited by examiner
`
`Petitioners' Exhibit 1061
`Page 0002
`
`
`
`U.S. Patent
`U.S. Patent
`
`Feb. 21, 2012
`Feb. 21, 2012
`
`Sheet 1 of 3
`Sheet 1 of 3
`
`US 8,122,141 B2
`US 8,122,141 B2
`
`
`
`Fig. 1
`
`Fig. 1
`
`Petitioners’ Exhibit 1061
`Page 0003
`
`Petitioners' Exhibit 1061
`Page 0003
`
`
`
`U.S. Patent
`U.S. Patent
`
`Feb. 21, 2012
`Feb. 21, 2012
`
`Sheet 2 of 3
`Sheet 2 of 3
`
`US 8,122,141 B2
`US 8,122,141 B2
`
`Fig. 2
`
`Fig. 2 IAA
`
`
`
`
`Petitioners’ Exhibit 1061
`Page 0004
`
`Petitioners' Exhibit 1061
`Page 0004
`
`
`
`U.S. Patent
`U.S. Patent
`
`Feb. 21, 2012
`Feb. 21, 2012
`
`Sheet 3 of 3
`Sheet 3 of 3
`
`US 8,122,141 B2
`US 8,122,141 B2
`
`Fig. 3
`Fig. 3
`
`
`
`32
`
`34
`
`36
`
`38
`
`40Pir
`
`Petitioners’ Exhibit 1061
`Page 0005
`
`Petitioners' Exhibit 1061
`Page 0005
`
`
`
`1.
`STREAMING MEDIA BUFFERING SYSTEM
`
`US 8,122,141 B2
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. patent applica
`tion Ser. No. 10/893,814, filed Jul. 19, 2004 (published on
`Dec. 9, 2004 as U.S. patent publication number 2004/
`0249969 A1, and now U.S. Pat. No. 7,716.358), which was a
`continuation-in-part of U.S. patent application Ser. No.
`09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No. 6,766,
`376), which claimed the benefit under 35 U.S.C. S 119(e) of
`U.S. provisional patent application Ser. No. 60/231.997, filed
`Sep. 12, 2000; claims the benefit, under 35 U.S.C. S 120, of
`the respective filing dates of said applications, as well as
`benefit of the filing date of copending U.S. patent application
`Ser. No. 10/825,869, filed Apr. 16, 2004 (published on Dec.
`23, 2004 as U.S. patent publication number 2004/260828
`A1), which was a continuation of said U.S. patent application
`Ser. No. 09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No.
`6,766,376), which claimed the benefit under 35 U.S.C. S 119
`(e) of said U.S. provisional patent application Ser. No.
`60/231.997, filed Sep. 12, 2000; and hereby incorporates by
`reference the entire disclosure of each of said prior applica
`tions.
`
`10
`
`15
`
`25
`
`BACKGROUND OF THE INVENTION
`
`35
`
`45
`
`1. Field of the Invention
`The present invention relates to multimedia computer com
`30
`munication systems; and more particularly, to a buffering
`system for streaming media, such as audio/video, on the
`Internet.
`2. Description of the Related Art
`Prior to the development of Internet streaming media tech
`nologies, audio and video were formatted into files, which
`users needed to download to their computer before the files
`could be heard or viewed. Real time, continuous media, as
`from a radio station, was not suitable for this arrangement in
`that a file of finite size must be created so it could be down
`40
`loaded. The advent of streaming media technologies allowed
`users to listen or view the files as they were being down
`loaded, and allowed users to “tune-in' to a continuous media
`broadcast, or “stream', such as from a radio station. There are
`two fundamental types of streaming media: (i) material that
`originates from a source having a real-time nature. Such as a
`radio or TV broadcast, and (ii) material that originates from a
`non-real-time source Such as from a disk file. An example of
`non-real-time material might be a piece of music stored as a
`disk file, or a portion of a broadcast that originally was real
`time, perhaps yesterday's TV evening news, and was
`recorded into a disk file. For purposes of clarity within this
`document, streaming media of type (i) will be referred to as
`“broadcast media, and streaming media of type (ii) will be
`referred to as “file based media. Broadcast streaming media
`has as its source a system or arrangement that by definition
`can only be transmitted to users as fast as the material is
`generated; for example, a disk jockey speaking into a micro
`phone. Broadcast streaming media is the focus of this patent
`application.
`Since audio and video media must play out over a period of
`time it is more appropriate to think of bandwidth require
`ments than file size. The bandwidth requirement of an audio
`or video media refers to the data rate in bits per second that
`must be transmitted and received in order to listen or view the
`material uninterrupted. Transmitting the audio or video mate
`rial over a connection slower than the bandwidth requirement
`
`50
`
`55
`
`60
`
`65
`
`2
`results in unsatisfactory viewing or listening, if viewing or
`listening is possible at all. The connection available to most
`Internet users is by dial-up modem, which has a maximum
`receive data rate of 56,000 bits per second. Most audio and
`video available on the Internet has been compressed to be
`listenable or viewable within the 56,000 bits per second
`modem bandwidth. Requirements for achieving adequate
`audio and video over the Internet generally consume a con
`siderable portion of the listener's available bandwidth.
`Internet connection quality can vary rapidly overtime, with
`two primary factors responsible for degradation of the instan
`taneous bandwidth actually available to the user. These fac
`tors are the quality of the user's modem connection over
`telephone lines, which can have periods of interference caus
`ing reduced available bandwidth, and momentary Internet
`congestion at various points along the route over which the
`user's data flows. Each of these factors can cause delays and
`interruptions in the transmission of data to the user. Internet
`data communications devices such as routers are designed to
`drop data “packets' if they get overloaded. For material that is
`not time sensitive, these dropped packets will usually be
`resent, and the user will eventually be presented with the
`material. However, since streaming media is time sensitive,
`dropped packets can have a significant impact on the receipt
`and playback of an audio or video stream. These degradations
`in the receipt of Internet data are very common, and prevent
`most users from being able to listen to or view streaming
`media without interruption unless some special provisions
`have been incorporated into the user's computer software to
`accommodate data transmission interruptions.
`These interruptions are commonly referred to as “drop
`outs', meaning that the data flow to the user has been inter
`rupted (i.e., the audio “drops out'). Dropouts can be
`extremely annoying for example, while listening to music.
`The current state-of-the-art solution to the problem uses a
`pre-buffering technique to store up enough audio or video
`data in the user's computer so that it can play the audio or
`Video with a minimum of dropouts. This process requires the
`user to wait until enough of the media file is buffered in
`memory before listening or viewing can begin. The media
`data is delivered by a server computer which has available to
`it the source of the media data, Such as by a connection to a
`radio station. When the user connects to the server via the
`Internet, audio/video output at the user's system is delayed
`while the user's buffer is filled to a predetermined level.
`Typical pre-buffering wait times range from 10 to 20 seconds
`or more, determined by the vendor providing the audio or
`video media. Even with this pre-buffering process, interrup
`tions in playback still occur.
`In this process, the user has a Software application on the
`computer commonly called a “media player”. Using the fea
`tures built into the media player, the user starts the audio or
`Video stream, typically by clicking on a "start button, and
`waits 10-20 seconds or so before the material starts playing.
`During this time data is being received from the Source and
`filling the media player's buffer. The audio or video data is
`delivered from the source at the rate it is to be played out. If,
`for example, the user is listening to an audio stream encoded
`to be played-out at 24,000 bits per second, the source sends
`the audio data at the rate of 24,000 bits per second. Provided
`that the user waits 10 seconds, and the receipt of the buffering
`data has not been interrupted, there is enough media data
`stored in the buffer to play for 10 seconds.
`Gaps in the receipt of audio/video data, due to Internet
`slowdowns, cause the buffer to deplete. Because transmission
`of audio/video media data to the user takes place at the rate it
`is played out, the user's buffer level can never be increased or
`
`Petitioners' Exhibit 1061
`Page 0006
`
`
`
`3
`replenished while it is playing. Thus, gaps in the receipt of
`audio/video media data inexorably cause the buffer level to
`decrease from its initial level. In time, extended or repeated
`occurrences of these gaps empty the user's buffer. The audio/
`video material stops playing, and the buffer must be refilled to
`its original predetermined level before playing of the media
`CSU.S.
`By way of illustration in a 10 second pre-buffering sce
`nario, if the data reception stopped the instant that the media
`started playing, it would play for exactly 10 seconds. Once it
`starts playing, the media data plays out of the buffer as new
`media data replenishes the buffer. The incoming data rate
`equals the rate at which the data is played out of the user's
`buffer, assuming the receipt of data across the Internet is
`unimpeded. If there are no interruptions in the receipt of the
`media data for the duration of the time the user listens to or
`watches the material, the buffer level remains constant and
`there will still be 10 seconds of data stored in the media
`player's buffer when the user stops the player. On the other
`hand, if the media player encounters interruptions totaling 6
`seconds while playing the material, there would only be 4
`seconds of media data remaining in the buffer when the user
`stopped it. If data reception interruptions at any time during
`the playing exceed 10 seconds, the user's media player buffer
`becomes exhausted. There is no media data to play, and the
`audio or video stops—a dropout has occurred. At this point a
`Software mechanism in the media player stops attempting to
`play any more of the material, and starts the buffering process
`again. The media player remains silent until the buffer refills,
`at which time the media player will once again start playing
`the material.
`There are two fundamental types of streaming media: (i)
`material that originates from a source having a real-time
`nature, such as a radio or TV broadcast, and (ii) material that
`originates from a non-real-time source Such as from a disk
`file. An example of non-real-time material might be a piece of
`music stored as a disk file, or a portion of a broadcast that
`originally was real-time, perhaps yesterday's TV evening
`news, and was recorded into a disk file. For purposes of clarity
`within this document, streaming media of type (i) will be
`referred to as “broadcast’ media, and streaming media of type
`(ii) will be referred to as “file based media.
`Both streaming media types are handled similarly in con
`ventional systems, and both are handled similarly by the
`45
`streaming media buffering system of the present invention.
`The two streaming media types are readily distinguished.
`Broadcast streaming media has as its source a system or
`arrangement that by definition can only be transmitted to
`users as fast as the material is generated; for example, a disk
`jockey speaking into a microphone. File based media, on the
`other hand, can be transmitted to users at any data rate, since
`there is no inherent time element to a file residing on a com
`puter disk. With conventional Internet streaming media sys
`tems for streaming media of either type, media data is trans
`mitted from the server to the user at the rate at which it will be
`played out, regardless of the data rate capabilities of the
`connection between the server and the user.
`Conventional streaming media systems may incorporate
`buffering systems for programmatic purposes. For example,
`the system may buffer media data at the server for the purpose
`of packet assembly/disassembly. Media data may also be
`buffered at the server to permit programming conveniences
`Such as dealing with chunks of data of a specific size. Such
`server buffering of media data is not used by conventional
`streaming media systems to mitigate long term Internet per
`formance degradation as described hereinafter.
`
`25
`
`30
`
`35
`
`40
`
`50
`
`55
`
`60
`
`65
`
`US 8,122,141 B2
`
`10
`
`15
`
`4
`The sending of audio or video files via a network is known
`in the art. U.S. Pat. No. 6,029,194 to Tilt describes a media
`server for the distribution of audio/video over networks, in
`which retrieved media frames are transferred to a FIFO
`buffer. A clock rate for a local clock is adjusted according to
`the fullness of the buffer. The media frames from the buffer
`are sent in the form of data packets over the networks in
`response to interrupts generated by the local clock. In this
`manner, the timing for the media frames is controlled by the
`user to assure a continuous stream of video during editing.
`U.S. Pat. No. 6,014,706 to Cannon, et al. discloses an appa
`ratus and method for displaying streamed digital video data
`on a client computer. The client computer is configured to
`receive the streamed digital video data from a server com
`puter via a computer network. The streamed digital video data
`is transmitted from the server computer to the client computer
`as a stream of video frames. U.S. Pat. No. 6,002,720, to Yurt,
`et al. discloses a system of distributing video and/or audio
`information wherein digital signal processing is employed to
`achieve high rates of data compression. U.S. Pat. No. 5,923,
`655, to Veschi et al. discloses a system and method for com
`municating audio/video data in a packet-based computer net
`work wherein transmission of data packets through the
`computer network requires variable periods of transmission
`time. U.S. Pat. No. 5,922,048 to Emura discloses a video
`server apparatus having a stream control section which deter
`mines a keyframe readout interval and a keyframe playback
`interval that satisfy a playback speed designated by a terminal
`apparatus. Finally, U.S. Pat. No. 6,014,694 to Aharoni, et al.
`discloses a system and method for adaptively transporting
`video over networks, including the Internet, wherein the
`available bandwidth varies with time.
`There remains a need in the art for a method and system
`that afford immediate and uninterrupted listening/viewing of
`streaming media by the user.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides a system and method for
`sending streaming media, Such as audio or video files, via the
`Internet. Immediate playing of the media on a user's com
`puter is afforded while reducing interruptions in playback due
`to Internet congestion and temporary modem delays due to
`noisy lines. Nearly instantaneous playback is achieved, while
`maintaining protection against playback interruption.
`Delayed starts, heretofore required to provide protection
`against interruption, are avoided. Data loss due to interrup
`tions in the receipt of media data by the media player can be
`recovered while the player continues to play out the audio or
`video material. If the interruptions are so severe as to deplete
`the user's buffer and stop the play out, the media player will
`begin to play out again as soon as the media player begins to
`receive media data without waiting to first buildup the buffer.
`Generally stated, the invention provides a system for dis
`tributing via the Internet streaming media composed of a
`plurality of time-sequenced data elements. The system has a
`server connected to the Internet for transmitting the data
`elements. Associated with the server area buffer manager and
`a FIFO buffer for storing at least one of the data elements for
`transmission. The buffer manager comprises means for:
`receiving the media data; Supplying media data in order to the
`FIFO buffer; supplying the FIFO buffer with a predetermined
`number of data elements; maintaining a pointerinto the buffer
`for each user computer indicating the last media data element
`that has been sent to that user, thus indicating the next element
`or elements to be sent; and, once the FIFO buffer is full,
`deleting the oldest data elements in the buffer as new data
`
`Petitioners' Exhibit 1061
`Page 0007
`
`
`
`5
`elements are received, said means arranged to maintain the
`pre-determined number of data elements in the FIFO buffer.
`At least one user computer is connected to the server via the
`Internet or other data communications medium.
`This invention presumes the existence of a data communi
`cations transport mechanism, such as the TCP protocol, for
`the reliable delivery of data in an ordered sequence from the
`source of the media data to the server, or from the server to the
`media player software of the user computer. Thus, the deliv
`ery of data in the proper sequence is outside the scope of this
`invention.
`The user computer is associated with a media player Soft
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined number of media data
`elements which are received sequentially by the media player,
`playing the data out sequentially as audio and/or video, and
`deleting media data elements from the buffer as they are
`played out. As data is played out, the next sequential data
`elements are received from the server in such a fashion as to
`approximately maintain the predetermined number of data
`elements in the user's buffer.
`There are two types of encoding schemes used for audio
`and video material “Variable Bit Rate' VBR, and “Con
`stant Bit Rate CBR. CBR encoding represents the encoded
`media with a constant bit rate per second, regardless of the
`complexity of the material being encoded, for example, if an
`audio Source is encoded at 20 kilobits per secondata Constant
`Bit Rate, the media data being produced from the encoding is
`at 20 kilobits per second whether the audio material is sym
`phonic music or silence. Variable Bit Rate encoding uses a
`30
`variable number of bits to represent sounds, with more bits
`required for complex (symphonic) sounds than for simple
`Sounds or silence. The standard encoding scheme used for
`streaming media is CBR because the resulting data rate is
`more predictable than for VBR.
`The server stores a predetermined amount of media data in
`a First-In First-Out (FIFO) buffer in an arrangement that
`receives media data directly or indirectly from a real-time
`Source. Such as a radio station. For example, the server buffer
`might be set to store up 30 seconds of media data. Because the
`Source produces media data in real time, the media data is
`delivered to the server approximately at the rate it is gener
`ated. Of course there can be variability’s in this data delivery
`process due to networking, disk accesses, and so on, causing
`the delivery rate of the media data to be variable over short
`periods of time, typically measured in seconds. But over a
`longer period of time measured in minutes or tens of minutes
`or longer, the media data is delivered from Source to server at
`the rate it is generated, and the server in turn provides that
`media data to the FIFO buffer at that same rate. Since CBR
`50
`encoding is normally used for streaming media, the media
`data is generated, received by the server, and provided to the
`buffer approximately at a fixed rate. Once the buffer is full, for
`each new data element received into the buffer the oldest data
`element is deleted from the buffer. Once a connection is made
`to a user's computer, the server sends the media data to the
`user computer's buffer in the following manner. First, media
`data is sent to the user at the highest rate that the data con
`nection between the server and the user computer will support
`until the predetermined amount of data that had been stored in
`the server buffer has been transferred to the user's computer.
`Once the buffer has been transferred a steady state condition
`is reached wherein as each media data element arrives at the
`server, it is immediately sent out to the user computer. In this
`steady state condition, the media data is sent at a rate that
`matches the constant fill rate of the server buffer, and is
`received at the same rate by the user computer if there are no
`
`45
`
`35
`
`40
`
`55
`
`60
`
`65
`
`US 8,122,141 B2
`
`10
`
`15
`
`25
`
`6
`interruptions in the transmission of media data between the
`server and the user's computer. If interruptions have inter
`fered with the arrival of sent media data to the user's com
`puter, that data may have been “dropped by routers in the
`Internet and needs to be resent. This causes data to “back up'
`into the server FIFO for that user.
`In one method of operation, the resending of missing data
`is the responsibility of a reliable transport mechanism, Such as
`TCP. The server buffer “sends' data by delivering it to the
`transport mechanism. The transport mechanism actually
`'sends' the data across the communications medium, and has
`processes which determine if all the data that has been sent
`has been received by the destination. If not, missing pieces of
`data are automatically resent to the destination, and are
`arranged to be delivered to the target software on the desti
`nation system in an ordered fashion. In the circumstance of
`this invention, the destination is the user computer, and the
`target Software on the destination system is the media player.
`If the transport mechanism determines that data is missing, it
`retransmits that data to the destination as fast as the connec
`tion between the server and destination will allow. The net
`effect of this invention is that all media data to be delivered to
`a user computer is always sent as fast as the communications
`medium will support, either by the server buffer passing
`media data to the transport mechanism, or by the transport
`mechanism delivering or redelivering the media data to the
`user computer. This is enabled by buffering data at the server,
`and is distinctly different from prior art in which media data
`is only sent from the server to the user computer at the rate at
`which it is to be played out.
`In another method of operation, the server can use an
`unreliable transport mechanism, such as UDP and rely on a
`streaming Software process to manage data delivery and the
`resending of data elements not received by the media player.
`As an example of the preceding description, if the server
`had been set to store 30 seconds of audio in its buffer, when a
`user connects that 30 seconds worth of media, data is trans
`ferred to the user's media player buffer as fast as the data
`connection between the two will allow. The media player can
`begin playing as soon as it has received a very minimum
`amount of data, perhaps comprising only a single packet of
`media data. For ease of understanding, consider the server
`buffer and the media player buffer to be an elastic system that
`between the two stores up 30 seconds of audio data. The
`server starts with 30 seconds of buffered audio data which it
`transfers to the media player until the server has no buffered
`media data and the media player has 30 seconds of buffered
`media data. Regardless of how much of the buffered media
`data has been transmitted to the media player, there always is
`30 seconds of media data being buffered between the two
`locations. Consequently, the audio being played out by the
`media player will always be 30 seconds behind the audio at
`the source. If there were a media player in the radio station
`studio, an announcer would hear themselves through the
`media player with a 30 second delay.
`Routinely, once a steady state has been achieved, the next
`data element to be sent is the next sequential data element
`from that which has already been received by the user's
`computer buffer. However, if there is more data to be sent than
`at the routine constant fill rate, such as in the condition where
`some media data has been resent by the reliable transport
`layer, the server transport mechanism will again send the
`buffered media data as fast as the connection between the
`server and the user's computer will support. Similarly, if the
`media player buffer begins to deplete or becomes depleted
`due to networking interruptions, the server will attempt to
`send as much data as is necessary to rebuild the user comput
`
`Petitioners' Exhibit 1061
`Page 0008
`
`
`
`7
`er's buffer to the proper level. This allows for rebuilding the
`user's computer buffer under circumstances wherein Internet
`interruptions have blocked the normal flow of data. When
`compared to conventional systems, which provide no capa
`bility to rebuild the user's computer buffer when data is lost,
`the streaming media buffering system of the present invention
`provides for recovery of lost data elements and the restoration
`of the user's buffer, even while the user media player contin
`ues to play.
`Under conditions in which interruptions have interfered
`with the arrival of sent media data to the user's computer, data
`loss exceeding certain levels will cause the transport mecha
`nism Software to stop accepting data for transmission from
`the application Software, namely the streaming media server
`Software. Although other arrangements are possible within
`the scope of this invention, in the preferred embodiment, the
`streaming media server Software keeps track of the last data
`element in the FIFO buffer that has been “sent to each user
`using a Software pointer. An interruption in the ability to send
`media data to a user results in this pointer"backing up' in the
`FIFO in such a way that the server knows from what point in
`the buffer to restart sending data when the transport mecha
`nism again requests data to send. When the server Software
`receives that notification, it will begin sending data to the user
`starting from the next data element to send as indicated by the
`pointer, and sending as much data as the transport mechanism
`will accept. The transport mechanism will again send this
`data as fast as it can to the user. This process continues until
`the steady state condition is again reached wherein each data
`element is sent to the user as soon as it arrives from the media
`SOUC.
`In another embodiment, the server is connected to the
`Internet, and to a broadcast media source. Such as a radio
`station. A radio station computer is provided with a means for
`receiving media data elements as they are generated by the
`audio and/or video source, and for transmitting those media
`data elements to the server as they are generated. As before,
`the server provides a buffer manager and a FIFO buffer, and
`provides a means for receiving the sequentially arranged
`media data elements from the broadcast media source and
`storing those data elements in the FIFO buffer. The buffer
`manager comprises means for: Supplying the FIFO buffer
`with a predetermined number of data elements; maintaining a
`pointer into the buffer for each user computer indicating the
`last media data element that has been sent to that user, thus
`indicating the next element or elements to be sent; and, once
`the FIFO buffer is full, deleting the oldest data element in the
`buffer as each new data element is received. Importantly, the
`buffer manager is arranged to maintain the pre-determined
`number of data elements in the FIFO buffer. At least one user
`computer is connected to the server via the Internet or other
`data communications medium.
`The user computer is associated with a media player Soft
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined number of media data
`elements, playing the data out sequentially as audio and/or
`video, and deleting media data elements from the buffer as
`they are played out. As data is played out, the next sequential
`data elements are received from the server in Such a fashion as
`to approximately maintain the predetermined number of data
`elements in the user's buffer. It should be understood that data
`might arrive at the media player out-of-sequence and that
`processes in the media player or the media player buffer
`manager are responsible for properly arranging this data.
`In another embodiment, the server is connected to the
`Internet and provisioned as initially described, and has avail
`able to it file based media data as the source material. The file
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 8,122,141 B2
`
`10
`
`15
`
`8
`based media data can be read by the server which can deliver
`media data elements to the server FIFO buffer at a constant
`time-sequenced rate, as if the data were arriving from abroad
`cast media source. As