`Price
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,122,141 B2
`*Feb. 21, 2012
`
`US008l22141B2
`
`5,922,048 A
`7/1999 Emura ........................ .. 709/219
`5,923,655 A
`7/1999 Veschiet al.
`370/394
`
`8:358:38 2 * 13/1333 88Z8nZZ33;11;;;';1:":::: Z88/88%
`6,002,720 A
`12/1999 Yuit et al.
`................... .. 375/240
`6,014,693 A
`1/2000 It
`tal.
`........ ..
`709/219
`
`6,014,694 A
`1/2000 A(1)1:roniet 31.
`709/219
`..
`6,014,706 A
`1/2000 Cannon et al.
`709/231
`6,029,194 A
`2/2000 Tilt
`................... ..
`709/219
`
`
`
`709/217
`4/2000 Bisdikian et 31'
`:1: 883881
`28888 8821221.; ‘:::"“
`709/231
`5/2000 Korstetal.
`.
`709/219
`5/2000 D M
`7/2000 Gfaffifififl................ H 709/202
`9/2001 Ravi et al.
`
`6947317 A
`8:88:88? 2
`6,061,732 A
`6,065,050 A
`6,085,221 A
`6,292,834 B1
`
`(C°mi““ed)
`
`OTHER PUBLICATIONS
`
`Salehi et a1., “Supporting Stored Video: Reducing Rate Variability
`and End-to-End Resource Requirements through Optimal Smooth-
`ing,” IEEE/ACM Transactions on Networking, vol. 6, Issue 4, p.
`397410’ Aug’ 1998'
`
`(Continued)
`
`.
`.
`.
`PWWWV EW'”"e’ ‘ J°SePh AVGHIDO
`Assistant Exammer T Marshall McLeod
`(74) Attorney, Agent, or Firm — Ernest D. Buff; Ernest D.
`Buff & Assoc. LLC; Gordon E. Fish
`
`(57)
`ABSTRACT
`Streaming media, Such as audio or Video files, is Sent Via the
`Internet. The media are immediately played on a user’s com-
`puter. Audio/video data is transmitted from the server more
`
`rapidly than it is played out by the user system. The audio/
`video data in the user buffer accumulates; and interruptions in
`playback as well as temporary modem delays are avoided.
`
`28 Claims, 3 Drawing Sheets
`
`(54) STREAMING MEDIA BUFFERING SYSTEM
`
`<75)
`
`1nVem°r=
`
`1§;g°'°‘ Edward P“°°= Be‘he1Park= PA
`(
`)
`(73) Assignee: WAG Acquisition, LLC, Flanders, NJ
`(US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U~S~C~ 154(b)bY 0daYS~
`.
`.
`.
`.
`.
`This patent is subject to a terminal dis-
`C1a1mer'
`
`(21) Appl. No.: 12/800,152
`
`(22)
`
`Filed:
`
`May 10, 2010
`
`(65)
`
`Prior Publication Data
`
`Sep' 16’ 2010
`US 2010/0235536A1
`Related U.S. Application Data
`
`(63) Continuation of application No. 10/893,814, filed on
`Jul. 19, 2004, now Pat. No. 7,716,358, which is a
`continuation-in-part of application No. 09/819,337,
`filed on Mar. 28, 2001, now Pat. No. 6,766,376.
`(60) Provisional application No. 60/231,997, filed on Sep.
`12, 2000.
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 15/16
`(52) U.S. Cl.
`..................................................... .. 709/231
`(58) Field of Classification Search ..................
`77%99//223301,
`See application file for Complete Search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,526,353 A
`5,793,980 A
`5,822,537 A
`
`6/1996 Henley et al.
`8/1998 Glaser et al.
`10/1998 Katseffet al.
`
`.............. .. 370/60.1
`. . . . . .
`. . .. 395/200.61
`.......... .. 395/200.61
`
`
`
`PAGE 1 OF 13
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 1 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`US 8,122,141 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`6,588,015 B1
`7/2003 Eyer et al.
`6,594,699 131
`7/2003 Sahaietal.
`6,598,228 B2
`7/2003 Hejna
`6,637,031 B1* 10/2003 Chou ............................ .. 725/87
`5,555,751 B1
`12/2003 Chen et 31~
`6,708,213 B1
`3/2004 Bommaiah et al.
`6,728,753 131*
`4/2004 Parasnisetal.
`............. .. 709/203
`6,829,368 132
`12/2004 Meyer et al.
`6,889; 57 B1
`5/2005 Patel
`6,907,481 B2
`6/2005 Kovacevic
`7,054,500 B1
`5/2006 Lillevold
`7,111,058 B1
`9/2006 Nguyen et al.
`7,170,856 B1
`1/2007 Ho et al.
`
`3/2008 Harmaway
`7,346,698 B2
`5/2008 Nguyen et al.
`7,373,413 B1
`-
`-
`“/2001 S”‘°"g1“’ et 31‘ """""""" " 709/1
`2°01/0047377 A1
`OTHER PUBLICATIONS
`
`,1,
`
`“Smoothing Variable-bit-rate Video
`Rexford et
`-
`-
`7,
`-
`‘“‘e”‘e‘W°rk’ IEEE/ACMT“‘“Sa°“°“S °“Ne‘W°rk‘“g’ V°1'7’I”“‘°'
`2a_P~ 20?-215s APT; 1999
`_
`_
`_
`_
`_
`Blanchl, “Buffer s1z1ng for hlgh speed VlC1e0 lnformatlon retrleval on
`ATM networks,” G1obecom—New York—, 1997, Vol. 2, pp. 1057-
`1061,
`
`in
`
`an
`
`a1.,
`
`* cited by examiner
`
`PAGE 2 OF 13
`
`|.M.L. SLU'S EXHIBIT ‘I001
`
`PAGE 2 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`U.S. Patent
`
`Feb. 21, 2012
`
`Sheet 1 013
`
`US 8,122,141 B2
`
`Fig. 1
`
`L 18
`
`
`
`20
`
`V‘;
`
`x—
`E8
`
`22
`
`l 2220
`
`PAGE 3 OF 13
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 3 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`U.S. Patent
`
`Feb. 21, 2012
`
`Sheet 2 013
`
`US 8,122,141 B2
`
`Fig. 2
`
`PAGE 4 OF 13
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 4 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`U.S. Patent
`
`Feb. 21, 2012
`
`Sheet 3 013
`
`US 8,122,141 B2
`
`Fig. 3
`
`32
`
`34
`
`J.
`
`a 6
`
`PAGE 5 OF 13
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 5 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`US 8,122,141 B2
`
`1
`STREAMING MEDIA BUFFERING SYSTEM
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. patent applica-
`tion Ser. No. 10/893,814, filed Jul. 19, 2004 (published on
`Dec. 9, 2004 as U.S. patent publication number 2004/
`0249969 A1, and now U.S. Pat. No. 7,716,358), which was a
`continuation-in-part of U.S. patent application Ser. No.
`09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No. 6,766,
`376), which claimed the benefit under 35 U.S.C. §119(e) of
`U.S. provisional patent application Ser. No. 60/231,997, filed
`Sep. 12, 2000; claims the benefit, under 35 U.S.C. §120, of
`the respective filing dates of said applications, as well as
`benefit of the filing date of copending U.S. patent application
`Ser. No. 10/825,869, filed Apr. 16, 2004 (published on Dec.
`23, 2004 as U.S. patent publication number 2004/260828
`A1), which was a continuation of said U.S. patent application
`Ser. No. 09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No.
`6,766,376), which claimed the benefit under 35 U.S.C. §119
`(e) of said U.S. provisional patent application Ser. No.
`60/231,997, filed Sep. 12, 2000; and hereby incorporates by
`reference the entire disclosure of each of said prior applica-
`tions.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to multimedia computer com-
`munication systems; and more particularly, to a buffering
`system for streaming media, such as audio/video, on the
`Internet.
`
`2. Description of the Related Art
`Prior to the development of Internet streaming media tech-
`nologies, audio and video were formatted into files, which
`users needed to download to their computer before the files
`could be heard or viewed. Real time, continuous media, as
`from a radio station, was not suitable for this arrangement in
`that a file of finite size must be created so it could be down-
`
`loaded. The advent of streaming media technologies allowed
`users to listen or view the files as they were being down-
`loaded, and allowed users to “tune-in” to a continuous media
`broadcast, or “stream”, such as from a radio station. There are
`two fundamental types of streaming media: (i) material that
`originates from a source having a real-time nature, such as a
`radio or TV broadcast, and (ii) material that originates from a
`non-real-time source such as from a disk file. An example of
`non-real-time material might be a piece of music stored as a
`disk file, or a portion of a broadcast that originally was real-
`time, perhaps yesterday’s TV evening news, and was
`recorded into a disk file. For purposes of clarity within this
`document, streaming media of type (i) will be referred to as
`“broadcast” media, and streaming media of type (ii) will be
`referred to as “file based” media. Broadcast streaming media
`has as its source a system or arrangement that by definition
`can only be transmitted to users as fast as the material is
`generated; for example, a disk jockey speaking into a micro-
`phone. Broadcast streaming media is the focus of this patent
`application.
`Since audio and video media must play out over a period of
`time it is more appropriate to think of bandwidth require-
`ments than file size. The bandwidth requirement of an audio
`or video media refers to the data rate in bits per second that
`must be transmitted and received in order to listen or view the
`
`material uninterrupted. Transmitting the audio or video mate-
`rial over a connection slower than the bandwidth requirement
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`results in unsatisfactory viewing or listening, if viewing or
`listening is possible at all. The connection available to most
`Internet users is by dial-up modem, which has a maximum
`receive data rate of 56,000 bits per second. Most audio and
`video available on the Internet has been compressed to be
`listenable or viewable within the 56,000 bits per second
`modem bandwidth. Requirements for achieving adequate
`audio and video over the Internet generally consume a con-
`siderable portion of the listener’s available bandwidth.
`Internet connection quality can vary rapidly over time, with
`two primary factors responsible for degradation ofthe instan-
`taneous bandwidth actually available to the user. These fac-
`tors are the quality of the user’s modem connection over
`telephone lines, which can have periods of interference caus-
`ing reduced available bandwidth, and momentary Internet
`congestion at various points along the route over which the
`user’s data flows. Each of these factors can cause delays and
`interruptions in the transmission of data to the user. Internet
`data communications devices such as routers are designed to
`drop data “packets” ifthey get overloaded. For material that is
`not time sensitive, these dropped packets will usually be
`resent, and the user will eventually be presented with the
`material. However, since streaming media is time sensitive,
`dropped packets can have a significant impact on the receipt
`and playback of an audio or video stream. These degradations
`in the receipt of Internet data are very common, and prevent
`most users from being able to listen to or view streaming
`media without interruption unless some special provisions
`have been incorporated into the user’s computer software to
`accommodate data transmission interruptions.
`These interruptions are commonly referred to as “drop-
`outs”, meaning that the data flow to the user has been inter-
`rupted (i.e.,
`the audio “drops out”). Dropouts can be
`extremely annoying—for example, while listening to music.
`The current state-of-the-art solution to the problem uses a
`pre-buffering technique to store up enough audio or video
`data in the user’s computer so that it can play the audio or
`video with a minimum of dropouts. This process requires the
`user to wait until enough of the media file is buffered in
`memory before listening or viewing can begin. The media
`data is delivered by a server computer which has available to
`it the source of the media data, such as by a connection to a
`radio station. When the user connects to the server via the
`
`Internet, audio/video output at the user’s system is delayed
`while the user’s buffer is filled to a predetermined level.
`Typical pre-buffering wait times range from 10 to 20 seconds
`or more, determined by the vendor providing the audio or
`video media. Even with this pre-buffering process, interrup-
`tions in playback still occur.
`In this process, the user has a software application on the
`computer commonly called a “media player”. Using the fea-
`tures built into the media player, the user starts the audio or
`video stream, typically by clicking on a “start” button, and
`waits 10-20 seconds or so before the material starts playing.
`During this time data is being received from the source and
`filling the media player’s buffer. The audio or video data is
`delivered from the source at the rate it is to be played out. If,
`for example, the user is listening to an audio stream encoded
`to be played-out at 24,000 bits per second, the source sends
`the audio data at the rate of 24,000 bits per second. Provided
`that the user waits 10 seconds, and the receipt ofthe buffering
`data has not been interrupted, there is enough media data
`stored in the buffer to play for 10 seconds.
`Gaps in the receipt of audio/video data, due to Internet
`slowdowns, cause the buffer to deplete. Because transmission
`of audio/video media data to the user takes place at the rate it
`is played out, the user’s buffer level can never be increased or
`
`PAGE 6 OF 13
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 6 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`US 8,122,141 B2
`
`3
`replenished while it is playing. Thus, gaps in the receipt of
`audio/video media data inexorably cause the buffer level to
`decrease from its initial level. In time, extended or repeated
`occurrences of these gaps empty the user’s buffer. The audio/
`video material stops playing, and the buffer must be refilled to
`its original predetermined level before playing of the media
`resumes.
`
`By way of illustration in a 10 second pre-buffering sce-
`nario, if the data reception stopped the instant that the media
`started playing, it would play for exactly 10 seconds. Once it
`starts playing, the media data plays out of the buffer as new
`media data replenishes the buffer. The incoming data rate
`equals the rate at which the data is played out of the user’s
`buffer, assuming the receipt of data across the Internet is
`unimpeded. If there are no interruptions in the receipt of the
`media data for the duration of the time the user listens to or
`watches the material, the buffer level remains constant and
`there will still be 10 seconds of data stored in the media
`
`player’s buffer when the user stops the player. On the other
`hand, if the media player encounters interruptions totaling 6
`seconds while playing the material, there would only be 4
`seconds of media data remaining in the buffer when the user
`stopped it. If data reception interruptions at any time during
`the playing exceed 10 seconds, the user’s media player buffer
`becomes exhausted. There is no media data to play, and the
`audio or video stops—a dropout has occurred. At this point a
`software mechanism in the media player stops attempting to
`play any more ofthe material, and starts the buffering process
`again. The media player remains silent until the buffer refills,
`at which time the media player will once again start playing
`the material.
`
`There are two fundamental types of streaming media: (i)
`material that originates from a source having a real-time
`nature, such as a radio or TV broadcast, and (ii) material that
`originates from a non-real-time source such as from a disk
`file. An example ofnon-real-time material might be a piece of
`music stored as a disk file, or a portion of a broadcast that
`originally was real-time, perhaps yesterday’s TV evening
`news, and was recorded into a disk file. For purposes of clarity
`within this document, streaming media of type (i) will be
`referred to as “broadcast” media, and streaming media oftype
`(ii) will be referred to as “file based” media.
`Both streaming media types are handled similarly in con-
`ventional systems, and both are handled similarly by the
`streaming media buffering system of the present invention.
`The two streaming media types are readily distinguished.
`Broadcast streaming media has as its source a system or
`arrangement that by definition can only be transmitted to
`users as fast as the material is generated; for example, a disk
`jockey speaking into a microphone. File based media, on the
`other hand, can be transmitted to users at any data rate, since
`there is no inherent time element to a file residing on a com-
`puter disk. With conventional Internet streaming media sys-
`tems for streaming media of either type, media data is trans-
`mitted from the server to the user at the rate at which it will be
`
`played out, regardless of the data rate capabilities of the
`connection between the server and the user.
`
`Conventional streaming media systems may incorporate
`buffering systems for programmatic purposes. For example,
`the system may buffer media data at the server for the purpose
`of packet assembly/disassembly. Media data may also be
`buffered at the server to permit programming conveniences
`such as dealing with chunks of data of a specific size. Such
`server buffering of media data is not used by conventional
`streaming media systems to mitigate long term Internet per-
`formance degradation as described hereinafter.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`The sending of audio or video files via a network is known
`in the art. U.S. Pat. No. 6,029,194 to Tilt describes a media
`server for the distribution of audio/video over networks, in
`which retrieved media frames are transferred to a FIFO
`
`buffer. A clock rate for a local clock is adjusted according to
`the fullness of the buffer. The media frames from the buffer
`
`are sent in the form of data packets over the networks in
`response to interrupts generated by the local clock. In this
`manner, the timing for the media frames is controlled by the
`user to assure a continuous stream of video during editing.
`U.S. Pat. No. 6,014,706 to Cannon, et al. discloses an appa-
`ratus and method for displaying streamed digital video data
`on a client computer. The client computer is configured to
`receive the streamed digital video data from a server com-
`puter via a computer network. The streamed digital video data
`is transmitted from the server computer to the client computer
`as a stream ofvideo frames. U.S. Pat. No. 6,002,720, to Yurt,
`et al. discloses a system of distributing video and/or audio
`information wherein digital signal processing is employed to
`achieve high rates of data compression. U.S. Pat. No. 5,923,
`655, to Veschi et al. discloses a system and method for com-
`municating audio/video data in a packet-based computer net-
`work wherein transmission of data packets through the
`computer network requires variable periods of transmission
`time. U.S. Pat. No. 5,922,048 to Emura discloses a video
`server apparatus having a stream control section which deter-
`mines a keyframe readout interval and a keyframe playback
`interval that satisfy a playback speed designated by a terminal
`apparatus. Finally, U.S. Pat. No. 6,014,694 to Aharoni, et al.
`discloses a system and method for adaptively transporting
`video over networks,
`including the Internet, wherein the
`available bandwidth varies with time.
`
`There remains a need in the art for a method and system
`that afford immediate and uninterrupted listening/viewing of
`streaming media by the user.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides a system and method for
`sending streaming media, such as audio or video files, via the
`Internet. Immediate playing of the media on a user’s com-
`puter is afforded while reducing interruptions in playback due
`to Internet congestion and temporary modem delays due to
`noisy lines. Nearly instantaneous playback is achieved, while
`maintaining protection against playback interruption.
`Delayed starts, heretofore required to provide protection
`against interruption, are avoided. Data loss due to interrup-
`tions in the receipt of media data by the media player can be
`recovered while the player continues to play out the audio or
`video material. If the interruptions are so severe as to deplete
`the user’s buffer and stop the play out, the media player will
`begin to play out again as soon as the media player begins to
`receive media data without waiting to first build up the buffer.
`Generally stated, the invention provides a system for dis-
`tributing via the Internet streaming media composed of a
`plurality of time-sequenced data elements. The system has a
`server connected to the Internet for transmitting the data
`elements. Associated with the server are a buffer manager and
`a FIFO buffer for storing at least one of the data elements for
`transmission. The buffer manager comprises means for:
`receiving the media data; supplying media data in order to the
`FIFO buffer; supplying the FIFO buffer with a predetermined
`number ofdata elements; maintaining a pointer into the buffer
`for each user computer indicating the last media data element
`that has been sent to that user, thus indicating the next element
`or elements to be sent; and, once the FIFO buffer is full,
`deleting the oldest data elements in the buffer as new data
`
`PAGE 7 OF 13
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 7 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`US 8,122,141 B2
`
`5
`elements are received, said means arranged to maintain the
`pre-determined number of data elements in the FIFO buffer.
`At least one user computer is connected to the server via the
`Internet or other data communications medium.
`
`This invention presumes the existence of a data communi-
`cations transport mechanism, such as the TCP protocol, for
`the reliable delivery of data in an ordered sequence from the
`source ofthe media data to the server, or from the server to the
`media player software of the user computer. Thus, the deliv-
`ery of data in the proper sequence is outside the scope of this
`invention.
`
`The user computer is associated with a media player soft-
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined number of media data
`elements which are received sequentially by the media player,
`playing the data out sequentially as audio and/or video, and
`deleting media data elements from the buffer as they are
`played out. As data is played out, the next sequential data
`elements are received from the server in such a fashion as to
`
`approximately maintain the predetermined number of data
`elements in the user’s buffer.
`
`There are two types of encoding schemes used for audio
`and video material—“Variable Bit Rate”—VBR, and “Con-
`stant Bit Rate”—CBR. CBR encoding represents the encoded
`media with a constant bit rate per second, regardless of the
`complexity of the material being encoded, for example, if an
`audio source is encoded at 20 kilobits per second at a Constant
`Bit Rate, the media data being produced from the encoding is
`at 20 kilobits per second whether the audio material is sym-
`phonic music or silence. Variable Bit Rate encoding uses a
`variable number of bits to represent sounds, with more bits
`required for complex (symphonic) sounds than for simple
`sounds or silence. The standard encoding scheme used for
`streaming media is CBR because the resulting data rate is
`more predictable than for VBR.
`The server stores a predetermined amount of media data in
`a First-In First-Out (FIFO) buffer in an arrangement that
`receives media data directly or indirectly from a real-time
`source, such as a radio station. For example, the server buffer
`might be set to store up 30 seconds ofmedia data. Because the
`source produces media data in real time, the media data is
`delivered to the server approximately at the rate it is gener-
`ated. Of course there can be variability’s in this data delivery
`process due to networking, disk accesses, and so on, causing
`the delivery rate of the media data to be variable over short
`periods of time, typically measured in seconds. But over a
`longer period of time measured in minutes or tens of minutes
`or longer, the media data is delivered from source to server at
`the rate it is generated, and the server in turn provides that
`media data to the FIFO buffer at that same rate. Since CBR
`
`encoding is normally used for streaming media, the media
`data is generated, received by the server, and provided to the
`buffer approximately at a fixed rate. Once the buffer is full, for
`each new data element received into the buffer the oldest data
`element is deleted from the buffer. Once a connection is made
`
`to a user’s computer, the server sends the media data to the
`user computer’ s buffer in the following manner. First, media
`data is sent to the user at the highest rate that the data con-
`nection between the server and the user computer will support
`until the predetermined amount ofdata that had been stored in
`the server buffer has been transferred to the user’ s computer.
`Once the buffer has been transferred a steady state condition
`is reached wherein as each media data element arrives at the
`
`server, it is immediately sent out to the user computer. In this
`steady state condition, the media data is sent at a rate that
`matches the constant fill rate of the server buffer, and is
`received at the same rate by the user computer if there are no
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`interruptions in the transmission of media data between the
`server and the user’s computer. If interruptions have inter-
`fered with the arrival of sent media data to the user’s com-
`
`puter, that data may have been “dropped” by routers in the
`Internet and needs to be resent. This causes data to “back up”
`into the server FIFO for that user.
`
`In one method of operation, the resending of missing data
`is the responsibility ofa reliable transport mechanism, such as
`TCP. The server buffer “sends” data by delivering it to the
`transport mechanism. The transport mechanism actually
`“sends” the data across the communications medium, and has
`processes which determine if all the data that has been sent
`has been received by the destination. If not, missing pieces of
`data are automatically resent to the destination, and are
`arranged to be delivered to the target software on the desti-
`nation system in an ordered fashion. In the circumstance of
`this invention, the destination is the user computer, and the
`target software on the destination system is the media player.
`If the transport mechanism determines that data is missing, it
`retransmits that data to the destination as fast as the connec-
`tion between the server and destination will allow. The net
`effect of this invention is that all media data to be delivered to
`
`a user computer is always sent as fast as the communications
`medium will support, either by the server buffer passing
`media data to the transport mechanism, or by the transport
`mechanism delivering or redelivering the media data to the
`user computer. This is enabled by buffering data at the server,
`and is distinctly different from prior art in which media data
`is only sent from the server to the user computer at the rate at
`which it is to be played out.
`In another method of operation, the server can use an
`unreliable transport mechanism, such as UDP, and rely on a
`streaming software process to manage data delivery and the
`resending of data elements not received by the media player.
`As an example of the preceding description, if the server
`had been set to store 30 seconds of audio in its buffer, when a
`user connects that 30 seconds worth of media, data is trans-
`ferred to the user’s media player buffer as fast as the data
`connection between the two will allow. The media player can
`begin playing as soon as it has received a very minimum
`amount of data, perhaps comprising only a single packet of
`media data. For ease of understanding, consider the server
`buffer and the media player buffer to be an elastic system that
`between the two stores up 30 seconds of audio data. The
`server starts with 30 seconds of buffered audio data which it
`
`transfers to the media player until the server has no buffered
`media data and the media player has 30 seconds of buffered
`media data. Regardless of how much of the buffered media
`data has been transmitted to the media player, there always is
`30 seconds of media data being buffered between the two
`locations. Consequently, the audio being played out by the
`media player will always be 30 seconds behind the audio at
`the source. If there were a media player in the radio station
`studio, an armouncer would hear themselves through the
`media player with a 30 second delay.
`Routinely, once a steady state has been achieved, the next
`data element to be sent is the next sequential data element
`from that which has already been received by the user’s
`computer buffer. However, ifthere is more data to be sent than
`at the routine constant fill rate, such as in the condition where
`some media data has been resent by the reliable transport
`layer, the server transport mechanism will again send the
`buffered media data as fast as the connection between the
`
`server and the user’s computer will support. Similarly, if the
`media player buffer begins to deplete or becomes depleted
`due to networking interruptions, the server will attempt to
`send as much data as is necessary to rebuild the user comput-
`
`PAGE 8 OF 13
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 8 OF 13
`
`I.M.L. SLU'S EXHIBIT 1001
`
`
`
`US 8,122,141 B2
`
`7
`er’s buffer to the proper level. This allows for rebuilding the
`user’ s computer buffer under circumstances wherein Internet
`interruptions have blocked the normal flow of data. When
`compared to conventional systems, which provide no capa-
`bility to rebuild the user’ s computer buffer when data is lost,
`the streaming media buffering system ofthe present invention
`provides for recovery of lost data elements and the restoration
`of the user’s buffer, even while the user media player contin-
`ues to play.
`Under conditions in which interruptions have interfered
`with the arrival of sent media data to the user’ s computer, data
`loss exceeding certain levels will cause the transport mecha-
`nism software to stop accepting data for transmission from
`the application software, namely the streaming media server
`software. Although other arrangements are possible within
`the scope of this invention, in the preferred embodiment, the
`streaming media server software keeps track of the last data
`element in the FIFO buffer that has been “sent” to each user
`
`using a software pointer. An interruption in the ability to send
`media data to a user results in this pointer “backing up” in the
`FIFO in such a way that the server knows from what point in
`the buffer to restart sending data when the transport mecha-
`nism again requests data to send. When the server software
`receives that notification, it will begin sending data to the user
`starting from the next data element to send as indicated by the
`pointer, and sending as much data as the transport mechanism
`will accept. The transport mechanism will again send this
`data as fast as it can to the user. This process continues until
`the steady state condition is again reached wherein each data
`element is sent to the user as soon as it arrives from the media
`source.
`
`In another embodiment, the server is connected to the
`Internet, and to a broadcast media source, such as a radio
`station. A radio station computer is provided with a means for
`receiving media data elements as they are generated by the
`audio and/or video source, and for transmitting those media
`data elements to the server as they are generated. As before,
`the server provides a buffer manager and a FIFO buffer, and
`provides a means for receiving the sequentially arranged
`media data elements from the broadcast media source and
`
`storing those data elements in the FIFO buffer. The buffer
`manager comprises means for: supplying the FIFO buffer
`with a predetermined number of data elements; maintaining a
`pointer into the buffer for each user computer indicating the
`last media data element that has been sent to that user, thus
`indicating the next element or elements to be sent; and, once
`the FIFO buffer is full, deleting the oldest data element in the
`buffer as each new data element is received. Importantly, the
`buffer manager is arranged to maintain the pre-determined
`number of data elements in the FIFO buffer. At least one user
`
`computer is connected to the server via the Internet or other
`data communications medium.
`
`The user computer is associated with a media player soft-
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined number of media data
`elements, playing the data out sequentially as audio and/or
`video, and deleting media data elements from the buffer as
`they are played out. As data is played out, the next sequential
`data elements are received from the server in such a fashion as
`
`to approximately maintain the predetermined number of data
`elements in the user’ s buffer. It should be understood that data
`
`might arrive at the media player out-of-sequence and that
`processes in the media player or the media player buffer
`manager are responsible for properly arranging this data.
`In another embodiment, the server is connected to the
`Internet and provisioned as initially described, and has avail-
`able to it file based media data as the source material. The file
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`based media data can be read by the server which can deliver
`media data elements to the server FIFO buffer at a constant
`
`time- sequenced rate, as ifthe data were arriving from a broad-
`cast media source. As before, the server provides a buffer
`manager and a FIFO buffer, and provides a means for receiv-
`ing the sequentially arranged media data elements from the
`file based media source and storing those data elements in the
`FIFO buffer. The buffer manager comprises means for:
`receiving the media data; supplying media data in order to the
`FIFO buffer; supplying the FIFO buffer with a predetermined
`number of data elements at a constant time-sequenced fill
`rate; maintaining a pointer into the buffer for each user com-
`puter indicating