`(10) Patent No.:
`a2) United States Patent
`Price
`(45) Date of Patent:
`*Feb. 21, 2012
`
`
`US008122141B2
`
`(54) STREAMING MEDIA BUFFERING SYSTEM
`
`(75)
`
`.
`
`Inventor: Harold Edward Price, Bethel Park, PA
`(US)
`
`‘
`
`(*) Notice:
`
`(73) Assignee: WAG Acquisition, LLC, Flanders, NJ
`(US)
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`US.C. 154(b) by 0 days.
`:
`:
`:
`:
`:
`This patent is subject to a terminal dis-
`claimer.
`(21) Appl. No.: 12/800,152
`
`5,922,048 A
`5,923,655 A
`5,928,330 A *
`5999525 A
`6,002,720 A
`6,014,693 A
`6,014,694 A
`6,014,706 A
`6,029,194 A
`eerey ‘
`6,061,731 A
`6,061,732 A
`6,065,050 A
`6,085,221 A
`6,292,834 Bl
`
`FN999 EMUL occ 709/219
`.
`.. 370/394
`7/1999 Veschi et al.
`
`7/1999 Goetzetal. oe 709/231
`12/1990 Krishnaswamy et al... 370/352
`12/1999 Yurt etal. oo 375/240
`
`1/2000 Ito et al.
`......
`.. 709/219
`1/2000 Aharonietal. .
`.. 709/219
`
`1/2000 Cannonetal. ..
`.. 709/231
`2/2000 Tilt Rereeenseeateey
`.. 709/219
`45000 rsdtian etal.
`” ea
`
`"300/231
`5/2000 Blakeslee ...
`
`..
`.. 709/231
`5/2000 Korst etal.
`5/2000 DeMoney ........ ee 709/219
`FI2000 Graf cccccccccccccccccececcscecesee 709/202
`9/2001 Ravi et al.
`(Continued)
`
`(22)
`
`Filed:
`
`May 10, 2010
`
`OTHER PUBLICATIONS
`
`(65)
`
`Prior Publication Data
`5
`US 2010/0235536 Al
`Sep. 16, 2010
`Related U.S. Application Data
`(63) Continuation of application No. 10/893,814, filed on
`Jul. 19, 2004, now Pat. No. 7,716,358, which is a
`continuation-in-part of application No. 09/819,337,
`filed on Mar. 28, 2001, now Pat. No. 6,766,376.
`(60) Provisional application No. 60/231,997,filed on Sep.
`12, 2000.
`
`Salehi et al., “Supporting Stored Video: Reducing Rate Variability
`and End-to-End Resource Requirements through Optimal Smooth-
`ing,” IEEE/ACM Transactions on Networking, vol. 6, Issue 4, p.
`397-410, Aug. 1998.
`
`(Continued)
`
`.
`.
`.
`Primary Examiner — Joseph Avellino
`Assistant Examiner — Marshall McLeod
`(74) Attorney, Agent, or Firm — Ernest D. Buff; Ernest D.
`Buff & Assoc. LLC; Gordon E. Fish
`
`(51)
`
`Int. Cl.
`(2006.01)
`GO6F 15/16
`
`
`(52) US. CD. cecceescscesscseecssesssseesssessssssesnnesseneesenees 709/231=(57) ABSTRACT
`:
`:
`:
`/
`(58) Field of Classification Search .................. oy Streaming media, such as audio or videofiles, is sent via the
`See applicationfile for complete search history
`Internet. The media are immediately played on a user’s com-
`,
`puter. Audio/video data is transmitted from the server more
`rapidly than it is played out by the user system. The audio/
`video data in the user buffer accumulates; and interruptions in
`playback as well as temporary modem delays are avoided.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,526,353 A
`5,793,980 A
`5,822,537 A
`
`6/1996 Henley etal. ou... 370/60.1
`ws 395/200.61
`8/1998 Glaseret al.
`..
`
`10/1998 Katseff et al. 0.0.0... 395/200.61
`
`28 Claims, 3 Drawing Sheets
`
`
`
`
`
`
`
`
`
`
` porseee poset ee
`
`
`
`
`
`
`
`
`
`
` pocscsipeeee ee
`
`
`
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0001
`EXHIBIT 1005 - PAGE 0001
`
`
`
`US 8,122,141 B2
` Page 2
`
`U.S. PATENT DOCUMENTS
`
`7/2003 Eyeretal.
`6,588,015 Bl
`7/2003. Sahaiet al.
`6.594.699 Bl
`7/2003 Hejna
`6,598,228 B2
`6,637,031 B1* 10/2003 Chow wii 725/87
`een3 BI
`cysoos Ben eta al
`£08,
`t
`al.
`6.728.753 Bl*
`4/2004 Parasnis et al C. 709/203
`6,829,368 B2
`12/2004 Meyeretal.
`6,889,257 Bl
`5/2005 Patel
`6,907,481 B2
`6/2005 Kovacevic
`7,054,500 Bl
`5/2006 Lillevold
`7,111,058 Bl
`9/2006 Neuyenet al.
`7,170,856 Bl
`1/2007 Hoetal.
`
`x
`
`3/2008 Hannaway
`7,346,698 B2
`5/2008 Nguyen etal.
`7,373,413 Bl
`5
`:
`:
`1
`11/2001 Sincagliaetal. oo. 709/1
`2001/0047377 A1l*
`OTHER PUBLICATIONS
`an
`in
`“Smoothing variable-bit-rate video
`Rexford et al.
`:
`:
`:
`”
`internetwork,” IEEE/ACMTransactions on Networking,vol. 7, Issue
`2, p. 202-215, Apr. 1999.
`Bianchi, “Buffer sizing for high speed video informationretrieval on
`AIMnetworks,” Globecom—New York—, 1997, vol. 2, pp. 1057-
`1061.
`
`* cited by examiner
`
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0002
`EXHIBIT 1005 - PAGE 0002
`
`
`
`U.S. Patent
`
`Feb. 21, 2012
`
`Sheet 1 of 3
`
`US 8,122,141 B2
`
`Fig. 1
`
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0003
`EXHIBIT 1005 - PAGE 0003
`
`
`
`U.S. Patent
`
`Feb. 21, 2012
`
`Sheet 2 of 3
`
`US 8,122,141 B2
`
`Fig. 2 llTT
`
`
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0004
`EXHIBIT 1005 - PAGE 0004
`
`
`
`U.S. Patent
`
`Feb. 21, 2012
`
`Sheet 3 of 3
`
`US 8,122,141 B2
`
`Fig. 3
`
`32
`
`34
`
`36
`
`38
`
`40PELL
`
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0005
`EXHIBIT 1005 - PAGE 0005
`
`
`
`US 8,122,141 B2
`
`1
`STREAMING MEDIA BUFFERING SYSTEM
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`25
`
`BACKGROUNDOF THE INVENTION
`
`1. Field of the Invention
`The presentinventionrelates to multimedia computer com-
`munication systems; and more particularly, to a buffering
`system for streaming media, such as audio/video, on the
`Internet.
`
`2. Description of the Related Art
`Prior to the developmentof Internet streaming media tech-
`nologies, audio and video were formatted into files, which
`users needed to download to their computer before the files
`could be heard or viewed. Real time, continuous media, as
`from a radio station, was not suitable for this arrangementin
`thata file of finite size must be created so it could be down-
`
`40
`
`This application is a continuation of U.S. patent applica-
`tion Ser. No. 10/893,814, filed Jul. 19, 2004 (published on
`Dec. 9, 2004 as U.S. patent publication number 2004/
`0249969 Al, and now U‘S. Pat. No. 7,716,358), which was a
`continuation-in-part of U.S. patent application Ser. No.
`09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No. 6,766,
`376), which claimedthe benefit under 35 U.S.C. §119(e) of
`USS. provisional patent application Ser. No. 60/231,997,filed
`Sep. 12, 2000; claims the benefit, under 35 U.S.C. §120, of
`the respective filing dates of said applications, as well as
`benefit ofthe filing date of copending U.S. patent application
`Ser. No. 10/825,869, filed Apr. 16, 2004 (published on Dec.
`23, 2004 as U.S. patent publication number 2004/260828
`A1), which was a continuation of said U.S. patent application
`Ser. No. 09/819,337, filed Mar. 28, 2001 (now U.S. Pat. No.
`6,766,376), which claimed the benefit under 35 U.S.C. §119
`(e) of said U.S. provisional patent application Ser. No.
`60/231,997, filed Sep. 12, 2000; and hereby incorporates by
`reference the entire disclosure of each of said prior applica-
`tions.
`
`2
`results in unsatisfactory viewing or listening, if viewing or
`listening is possibleat all.he connection available to most
`Internet users is by dial-up modem, which has a maximum
`receive data rate of 56,000 bits per second. Most audio and
`video available on the Internet has been compressed to be
`listenable or viewable within the 56,000 bits per second
`modem bandwidth. Requirements for achieving adequate
`audio and video over the Internet generally consumea con-
`siderable portion ofthe listener’s available bandwidth.
`Internet connection quality can vary rapidly over time, with
`twoprimary factors responsible for degradation ofthe instan-
`taneous bandwidth actually available to the user. These fac-
`tors are the quality of the user’s modem connection over
`telephonelines, which can have periods of interference caus-
`ing reduced available bandwidth, and momentary Internet
`congestion at various points along the route over which the
`user’s data flows. Each of these factors can cause delays and
`interruptions in the transmission of data to the user. Internet
`data communications devices such as routers are designed to
`drop data “packets” ifthey get overloaded. For material thatis
`not time sensitive, these dropped packets will usually be
`resent, and the user will eventually be presented with the
`material. However, since streaming media is time sensitive,
`dropped packets can have a significant impact on the receipt
`and playback of an audio or video stream. These degradations
`in the receipt of Internet data are very common,and prevent
`most users from being able to listen to or view streaming
`media without interruption unless some special provisions
`have been incorporated into the user’s computer software to
`accommodate data transmission interruptions.
`These interruptions are commonly referred to as “drop-
`outs”, meaning that the data flow to the user has been inter-
`rupted (i.e.,
`the audio “drops out”). Dropouts can be
`extremely annoying—for example, while listening to music.
`The current state-of-the-art solution to the problem uses a
`pre-buffering technique to store up enough audio or video
`data in the user’s computer so that it can play the audio or
`video with a minimum ofdropouts. This process requires the
`user to wait until enough of the media file is buffered in
`memory before listening or viewing can begin. The media
`data is delivered by a server computer which has available to
`loaded. The advent of streaming media technologies allowed
`it the source of the media data, such as by a connection to a
`users to listen or view the files as they were being down-
`radio station. When the user connects to the server via the
`loaded, and allowedusersto “tune-in”to a continuous media
`broadcast, or “stream”, such as fromaradio station. There are
`Internet, audio/video output at the user’s system is delayed
`45
`two fundamental types of streaming media: (1) materialthat
`while the user’s buffer is filled to a predetermined level.
`originates from a source having a real-time nature, such as a
`Typical pre-buffering wait times range from 10 to 20 seconds
`radio or TV broadcast, and (ii) material that originates from a
`or more, determined by the vendor providing the audio or
`non-real-time source such as from a diskfile. An example of
`video media. Even with this pre-buffering process, interrup-
`non-real-time material might be a piece of music stored as a
`tions in playbackstill occur.
`disk file, or a portion of a broadcastthat originally wasreal-
`In this process, the user has a software application on the
`time, perhaps yesterday’s TV evening news, and was
`computer commonlycalled a “media player’. Using the fea-
`recorded into a disk file. For purposes of clarity within this
`tures built into the media player, the user starts the audio or
`document, streaming media of type (i) will be referred to as
`video stream, typically by clicking on a “start” button, and
`“broadcast” media, and streaming media of type (ii) will be
`waits 10-20 secondsor so before the material starts playing.
`referred to as “file based” media. Broadcast streaming media
`During this time data is being received from the source and
`has as its source a system or arrangementthat by definition
`filling the media player’s buffer. The audio or video data is
`can only be transmitted to users as fast as the material is
`delivered from the source at the rate it is to be played out. If,
`generated; for example, a disk jockey speaking into a micro-
`for example, the user is listening to an audio stream encoded
`phone. Broadcast streaming media is the focus of this patent
`to be played-out at 24,000 bits per second, the source sends
`application.
`the audio data at the rate of 24,000 bits per second. Provided
`Since audio and video media mustplay out over a period of
`thatthe user waits 10 seconds, and the receipt ofthe buffering
`time it is more appropriate to think of bandwidth require-
`data has not been interrupted, there is enough media data
`ments than file size. The bandwidth requirementof an audio
`stored in the buffer to play for 10 seconds.
`or video media refers to the data rate in bits per second that
`Gaps in the receipt of audio/video data, due to Internet
`must be transmitted and received in orderto listen or view the
`slowdowns, cause the buffer to deplete. Because transmission
`of audio/video media data to the user takes placeat therate it
`is played out, the user’s buffer level can never be increased or
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0006
`EXHIBIT 1005 - PAGE 0006
`
`60
`
`material uninterrupted. Transmitting the audio or video mate-
`rial over a connection slowerthan the bandwidth requirement
`
`
`
`US 8,122,141 B2
`
`3
`replenished while it is playing. Thus, gaps in the receipt of
`audio/video media data inexorably cause the buffer level to
`decrease from its initial level. In time, extended or repeated
`occurrences of these gaps empty the user’s buffer. The audio/
`video material stops playing, and the buffer mustberefilled to
`its original predetermined level before playing of the media
`resumes.
`
`By wayofillustration in a 10 second pre-buffering sce-
`nario, if the data reception stopped the instant that the media
`started playing, it would play for exactly 10 seconds. Onceit
`starts playing, the media data plays out of the buffer as new
`media data replenishes the buffer. The incoming data rate
`equals the rate at which the data is played out of the user’s
`buffer, assuming the receipt of data across the Internet is
`unimpeded. If there are no interruptions in the receipt of the
`media data for the duration of the time the user listens to or
`watches the material, the buffer level remains constant and
`there will still be 10 seconds of data stored in the media
`
`player’s buffer when the user stops the player. On the other
`hand, if the media player encounters interruptionstotaling 6
`seconds while playing the material, there would only be 4
`seconds of media data remaining in the buffer when the user
`stoppedit. If data reception interruptions at any time during
`the playing exceed 10 seconds, the user’s media player buffer
`becomes exhausted. There is no media data to play, and the
`audio or video stops
`a dropouthas occurred.Atthis point a
`software mechanism in the media player stops attempting to
`play any more ofthe material, andstarts the buffering process
`again. The media player remains silent until the bufferrefills,
`at which time the media player will once againstart playing
`the material.
`There are two fundamental types of streaming media: (i)
`material that originates from a source having a real-time
`nature, such as a radio or TV broadcast, and (11) material that
`originates from a non-real-time source such as from a disk
`file. An example ofnon-real-time material might be a piece of
`music stored as a disk file, or a portion of a broadcast that
`originally was real-time, perhaps yesterday’s TV evening
`news, and was recordedinto a diskfile. For purposes of clarity
`within this document, streaming media of type (4) will be
`referred to as “broadcast” media, and streaming media oftype
`(i1) will be referred to as “file based” media.
`Both streaming media types are handled similarly in con-
`ventional systems, and both are handled similarly by the
`streaming media buffering system of the present invention.
`The two streaming media types are readily distinguished.
`Broadcast streaming media has as its source a system or
`arrangement that by definition can only be transmitted to
`users as fast as the material is generated; for example, a disk
`jockey speaking into a microphone. File based media, on the
`other hand, can be transmitted to users at any data rate, since
`there is no inherent time elementto a file residing on a com-
`puter disk. With conventional Internet streaming media sys-
`tems for streaming mediaof either type, media datais trans-
`mitted from the serverto the userat the rate at which it will be
`
`played out, regardless of the data rate capabilities of the
`connection between the server andthe user.
`Conventional streaming media systems may incorporate
`buffering systems for programmatic purposes. For example,
`the system may buffer media dataat the server for the purpose
`of packet assembly/disassembly. Media data may also be
`buffered at the server to permit programming conveniences
`such as dealing with chunksof data of a specific size. Such
`server buffering of media data is not used by conventional
`streaming media systems to mitigate long term Internet per-
`formance degradation as described hereinafter.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`The sending of audio or video files via a network is known
`in the art. U.S. Pat. No. 6,029,194 to Tilt describes a media
`server for the distribution of audio/video over networks, in
`which retrieved media frames are transferred to a FIFO
`buffer. A clock rate for a local clock is adjusted according to
`the fullness of the buffer. The media frames from the buffer
`are sent in the form of data packets over the networks in
`response to interrupts generated by the local clock. Inthis
`manner, the timing for the media framesis controlled by the
`user to assure a continuous stream of video during editing.
`USS. Pat. No. 6,014,706 to Cannon,et al. discloses an appa-
`ratus and method for displaying streamed digital video data
`on a client computer. The client computer is configured to
`receive the streamed digital video data from a server com-
`puter via a computer network. The streameddigital video data
`is transmitted from the server computerto the client computer
`as a stream of video frames. U.S. Pat. No. 6,002,720, to Yurt,
`et al. discloses a system of distributing video and/or audio
`information wherein digital signal processing is employed to
`achieve high rates of data compression. U.S. Pat. No. 5,923,
`655, to Veschi et al. discloses a system and method for com-
`municating audio/video data in a packet-based computernet-
`work wherein transmission of data packets through the
`computer network requires variable periods of transmission
`time. U.S. Pat. No. 5,922,048 to Emura discloses a video
`server apparatus having a stream control section which deter-
`mines a keyframe readout interval and a keyframe playback
`interval that satisfy a playback speed designated by a terminal
`apparatus. Finally, U.S. Pat. No. 6,014,694 to Aharoni, et al.
`discloses a system and method for adaptively transporting
`video over networks,
`including the Internet, wherein the
`available bandwidth varies with time.
`There remains a need in the art for a method and system
`that afford immediate and uninterrupted listening/viewing of
`streaming media by the user.
`
`SUMMARYOF THE INVENTION
`
`The present invention provides a system and method for
`sending streaming media, such as audioorvideofiles, via the
`Internet. Immediate playing of the media on a user’s com-
`puteris afforded while reducinginterruptions in playback due
`to Internet congestion and temporary modem delays due to
`noisy lines. Nearly instantaneous playbackis achieved, while
`maintaining protection against playback interruption.
`Delayed starts, heretofore required to provide protection
`against interruption, are avoided. Data loss due to interrup-
`tions in the receipt of media data by the media player can be
`recovered while the player continues to play out the audio or
`video material. If the interruptions are so severe as to deplete
`the user’s buffer and stop the play out, the media player will
`begin to play out again as soon as the media player begins to
`receive media data without waitingto first build up the buffer.
`Generally stated, the invention provides a system for dis-
`tributing via the Internet streaming media composed of a
`plurality of time-sequenced data elements. The system has a
`server connected to the Internet for transmitting the data
`elements. Associated with the server are a buffer manager and
`a FIFO buffer for storing at least one of the data elements for
`transmission. The buffer manager comprises means for:
`receiving the media data; supplying media data in order to the
`FIFO buffer; supplying the FIFO buffer with a predetermined
`numberofdata elements; maintaining a pointerinto the buffer
`for each user computerindicating the last media data element
`that has been sentto that user, thus indicating the next element
`or elements to be sent; and, once the FIFO buffer is full,
`deleting the oldest data elements in the buffer as new data
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0007
`EXHIBIT 1005 - PAGE 0007
`
`
`
`5
`elements are received, said means arranged to maintain the
`pre-determined numberof data elements in the FIFO butter.
`Atleast one user computer is connectedto the server via the
`Internet or other data communications medium.
`This invention presumes the existence of a data communi-
`cations transport mechanism, such as the TCP protocol, for
`the reliable delivery of data in an ordered sequence from the
`source ofthe media data to the server, or from the server to the
`media player software of the user computer. Thus, the deliv-
`ery of data in the proper sequenceis outside the scopeofthis
`invention.
`The user computer is associated with a media player soft-
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined number of media data
`elements which are received sequentially by the media player,
`playing the data out sequentially as audio and/or video, and
`deleting media data elements from the buffer as they are
`played out. As data is played out, the next sequential data
`elements are received from the server in such a fashion as to
`
`approximately maintain the predetermined numberof data
`elements in the user’s buffer.
`
`puter, that data may have been “dropped”by routers in the
`Internet and needs to be resent. This causes data to “back up”
`into the server FIFO for that user.
`In one method of operation, the resending of missing data
`is the responsibility ofa reliable transport mechanism, suchas
`TCP. The server buffer “sends” data by delivering it to the
`transport mechanism. The transport mechanism actually
`“sends”the data across the communications medium, and has
`processes which determineif all the data that has been sent
`has been receivedby the destination. Ifnot, missing pieces of
`data are automatically resent to the destination, and are
`arranged to be delivered to the target software on the desti-
`nation system in an ordered fashion. In the circumstance of
`this invention, the destination is the user computer, and the
`target software on the destination system is the media player.
`If the transport mechanism determines that data is missing,it
`retransmits that data to the destination as fast as the connec-
`tion between the server and destination will allow. The net
`effect of this inventionis that all media datato be delivered to
`There are two types of encoding schemes used for audio
`and video material—‘Variable Bit Rate’—VBR,and “Con-
`a user computer is alwayssentas fast as the communications
`medium will support, either by the server buffer passing
`stant Bit Rate’—CBR. CBRencodingrepresents the encoded
`media data to the transport mechanism, or by the transport
`media with a constant bit rate per second, regardless of the
`mechanism delivering or redelivering the media data to the
`complexity of the material being encoded, for example, if an
`user computer. This is enabled by buffering data at the server,
`audio source is encodedat 20 kilobits per second at a Constant
`and is distinctly different from prior art in which media data
`Bit Rate, the media data being produced from the encodingis
`is only sent from the server to the user computerat the rate at
`at 20 kilobits per second whether the audio material is sym-
`whichit is to be played out.
`phonic music or silence. Variable Bit Rate encoding uses a
`In another method of operation, the server can use an
`variable numberof bits to represent sounds, with more bits
`unreliable transport mechanism, such as UDP, and rely ona
`required for complex (symphonic) sounds than for simple
`streaming software process to manage data delivery and the
`sounds or silence. The standard encoding scheme used for
`resending of data elements not received by the media player.
`streaming media is CBR because the resulting data rate is
`As an example of the preceding description,if the server
`morepredictable than for VBR.
`had beenset to store 30 seconds of audio in its buffer, when a
`Theserver stores a predetermined amountof media data in
`user connects that 30 seconds worth of media, data is trans-
`a First-In First-Out (FIFO) buffer in an arrangement that
`ferred to the user’s media player buffer as fast as the data
`receives media data directly or indirectly from a real-time
`connection betweenthe two will allow. The media player can
`source, such as a radio station. For example, the server buffer
`begin playing as soon as it has received a very minimum
`mightbeset to store up 30 seconds ofmedia data. Because the
`amount of data, perhaps comprising only a single packet of
`source produces media data in real time, the media data is
`delivered to the server approximately at therate it is gener-
`media data. For ease of understanding, consider the server
`
`ated. Of course there can be variability’sin this data delivery buffer and the media player bufferto be an elastic system that
`process due to networking, disk accesses, and so on, causing
`between the two stores up 30 seconds of audio data. The
`server starts with 30 seconds of buffered audio data whichit
`the delivery rate of the media data to be variable over short
`periods of time, typically measured in seconds. But over a
`longer period of time measured in minutesor tens of minutes
`or longer, the media data is delivered from source to server at
`the rate it is generated, and the server in turn provides that
`media data to the FIFO buffer at that same rate. Since CBR
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`US 8,122,141 B2
`
`6
`interruptions in the transmission of media data between the
`server and the user’s computer. If interruptions have inter-
`fered with the arrival of sent media data to the user’s com-
`
`encoding is normally used for streaming media, the media
`data is generated, received by the server, and provided to the
`buffer approximately ata fixed rate. Once the bufferis full, for
`each new data element received into the buffer the oldest data
`elementis deleted from the buffer. Once a connection is made
`to a user’s computer, the server sends the media data to the
`user computer’s buffer in the following manner. First, media
`data is sent to the user at the highest rate that the data con-
`nection betweenthe server and the user computer will support
`until the predetermined amountofdata that had been stored in
`the server buffer has been transferred to the user’s computer.
`Oncethe buffer has been transferred a steady state condition
`is reached wherein as each media data element arrives at the
`
`55
`
`server, it is immediately sent out to the user computer. In this
`steady state condition, the media data is sent at a rate that
`matches the constant fill rate of the server buffer, and is
`received at the samerate by the user computer if there are no
`
`65
`
`transfers to the media player until the server has no buffered
`media data and the media player has 30 secondsof buffered
`media data. Regardless of how much ofthe buffered media
`data has been transmitted to the mediaplayer, there alwaysis
`30 seconds of media data being buffered between the two
`locations. Consequently, the audio being played out by the
`media player will always be 30 seconds behind the audio at
`the source. If there were a media player in the radio station
`studio, an announcer would hear themselves through the
`media player with a 30 seconddelay.
`Routinely, once a steady state has been achieved, the next
`data element to be sent is the next sequential data element
`from that which has already been received by the user’s
`computer buffer. However,ifthere is more data to be sent than
`at the routine constantfill rate, such as in the condition where
`some media data has been resent by the reliable transport
`layer, the server transport mechanism will again send the
`buffered media data as fast as the connection between the
`
`server and the user’s computer will support. Similarly, if the
`media player buffer begins to deplete or becomes depleted
`due to networking interruptions, the server will attempt to
`send as much data as is necessary to rebuild the user comput-
`IPR2022-01228
`IPR2022-01228
`EXHIBIT 1005 - PAGE 0008
`EXHIBIT 1005 - PAGE 0008
`
`
`
`US 8,122,141 B2
`
`7
`er’s buffer to the proper level. This allows for rebuilding the
`user’s computer buffer under circumstances wherein Internet
`interruptions have blocked the normal flow of data. When
`compared to conventional systems, which provide no capa-
`bility to rebuild the user’s computer buffer when datais lost,
`the streaming media buffering system ofthe present invention
`provides for recovery of lost data elements andtherestoration
`of the user’s buffer, even while the user media player contin-
`ues to play.
`Under conditions in which interruptions have interfered
`with thearrival of sent media data to the user’s computer, data
`loss exceeding certain levels will cause the transport mecha-
`nism software to stop accepting data for transmission from
`the application software, namely the streaming media server
`software. Although other arrangements are possible within
`the scopeofthis invention, in the preferred embodiment, the
`streaming media server software keeps track of the last data
`elementin the FIFO buffer that has been “sent” to each user
`using a software pointer. An interruption in the ability to send
`media data to a userresults in this pointer “backing up”in the
`FIFOin such a waythat the server knows from whatpoint in
`the buffer to restart sending data when the transport mecha-
`nism again requests data to send. When the server software
`receives that notification, it will begin sending data to the user
`starting from the next data elementto sendas indicated by the
`pointer, and sending as muchdataas the transport mechanism
`will accept. The transport mechanism will again send this
`data as fast as it can to the user. This process continues until
`the steady state condition is again reached wherein each data
`elementis sent to the useras soon asit arrives from the media
`source.
`
`In another embodiment, the server is connected to the
`Internet, and to a broadcast media source, such as a radio
`station. A radio station computer is provided with a meansfor
`receiving media data elements as they are generated by the
`audio and/or video source, and for transmitting those media
`data elements to the server as they are generated. As before,
`the server provides a buffer manager and a FIFO buffer, and
`provides a means for receiving the sequentially arranged
`media data elements from the broadcast media source and
`
`storing those data elements in the FIFO buffer. The buffer
`manager comprises means for: supplying the FIFO buffer
`with a predetermined numberofdata elements; maintaining a
`pointer into the buffer for each user computer indicating the
`last media data element that has been sent to that user, thus
`indicating the next element or elements to be sent; and, once
`the FIFO bufferis full, deleting the oldest data elementin the
`buffer as each new data element is received. Importantly, the
`buffer manager is arranged to maintain the pre-determined
`numberof data elements in the FIFO buffer. At least one user
`
`computer is connected to the servervia the Internet or other
`data communications medium.
`The user computer is associated with a media player soft-
`ware incorporating a user buffer and comprises means for
`receiving and storing a predetermined numberof media data
`elements, playing the data out sequentially as audio and/or
`video, and deleting media data elements from the buffer as
`theyare played out. As datais played out, the next sequential
`data elements are received from the server in such a fashion as
`
`to approximately maintain the predetermined numberofdata
`elements in the user’s buffer. It should be understoodthat data
`
`might arrive at the media player out-of-sequence and that
`processes in the media player or the media player buffer
`managerare responsible for properly arranging this data.
`In another embodiment, the server is connected to the
`Internet and provisionedas initially described, and hasavail-
`ableto it file based media data as the source material. Thefile
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`based media data can be read by the server which can deliver
`media data elements to the server FIFO buffer at a constant
`
`time-sequencedrate, as ifthe data werearriving from a broad-
`cast media source. As before, the server provides a buffer
`manager and a FIFO buffer, and provides a meansfor receiv-
`ing the sequentially arranged media data elements from the
`file based media source andstoring those data elementsin the
`FIFO buffer. The buffer manager comprises means for:
`receiving the media data; supplying media data in order to the
`FIFO buffer; supplying the FIFO buffer with a predetermined
`numberof data elements at a constant time-sequencedfill
`rate; maintaining a pointer into the buffer for each user com-
`puter indicating the last media data elementthat has been sent
`to that user, thus indicating the next element or elements to be
`sent; and, once the FIFO bufferis full, deleting the oldest data
`element in the buffer as each new data elementis received,
`said means arranged to maintain the p