(12) Ulllted States Patent
`Price
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,364,839 B2
`Jan. 29, 2013
`
`US008364839B2
`
`(54) STREAMING MEDIA DELIVERY SYSTEM
`(75)
`Inventor:
`gJaSr)old Edward Price, Bethel Park, PA
`
`(73) Assignee: WAG Acquisition, LLC, Flanders, NJ
`<US>
`fthi
`th t
`S b. H
`d.
`1
`.
`s
`e erm o
`u JCC o any 1sc a1mer,
`patent is extended or adjusted under 35
`1s4<b>byodays1
`
`(,,) N t.
`o 1ce:
`
`(21) APP1. N04 13/385,375
`
`(22)
`
`Filed:
`
`Feb. 16, 2012
`
`(65)
`
`Prior Publication Data
`US 2012/0166669 A1
`Jun. 28, 2012
`
`Related U.S. Application Data
`(63) Continuation of application No. 12/800,177, filed on
`May 10, 2010, now Pat. No. 8,185,611, which is a
`continuation of application No. 10/893,814, filed on
`Jul. 19, 2004, now Pat. No. 7,716,358, which is a
`continuation-in-part of application No. 09/819,337,
`filed on Mar. 28, 2001, now Pat. No. 6,766,376.
`(60) Provisional application No. 60/231,997, filed on Sep.
`12, 2000
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 15/16
`(52) U.S. Cl.
`...................................... .. 709/231; 709/219
`(58) Field of Classification Search ................ .. 709/231,
`709/230, 217, 218, 219
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`US. PATENT DOCUMENTS
`4,953,995 A
`10/1990 Lang
`5,057,932 A
`10/1991 Lang
`5,065,396 A
`11/1991 Castellano
`5,164,839 A
`11/1992 Lang
`
`,
`
`,
`
`2
`2
`
`5,481,543 A
`5,493,514 A
`5,525,353 A
`2333335 2
`,
`,
`5,583,859 A
`5,510,341 A
`5,613,032 A
`,
`,
`272235321:
`5,668,948 A
`5,710,970 A
`5,719,786 A
`5,734,119 A
`5,737,536 A
`
`gincer et 81.
`unt
`§V(:Et1:,: et al,
`
`1/1996 Veltman
`2/1996 Keith
`5/1995 Henley Gt 31-
`11/1332 %°“g
`reeman
`12/1996 Feldmeier
`3/1997 Tanaka
`3/1997 Cruz
`annee S
`3123:
`9/1997 B lkn
`1/1998 Wfalteilspet al.
`2/1998 Nelson
`3/1998 France
`4/1998 Herrmarm
`
`5’751’951 A
`
`5/1998 Osborne
`C t'
`d
`( on mue )
`FOREIGN PATENT DOCUMENTS
`
`CA
`EP
`
`1/2004
`2247588 C
`614317 A2 , 9/1994
`(COI1UI111ed)
`
`OTHER PUBLICATIONS
`Bing Zheng and Mohan1med Atiquzzaman, Multimedia Over High
`Speed Networks: Reducing Network Requirements With Fast Buffer
`Flllup’ 1998'
`
`(Continued)
`
`Primary Examiner — Joseph Avellino
`Assistant Examiner — Marshall McLeod
`(74) Attorney, Agent, or Firm — Ernest D. Buff; Ernest D.
`B11ff& ASSOC. LLC
`
`ABSTRACT
`(57)
`Streaming media, such as audio or video files, is sent via the
`Internet. The media are immediately played on a user’s com-
`puter. Audio/video data is transmitted from the server more
`rapidly than it is played out by the user system. The audio/
`video data in the user buffer accumulates; and interruptions in
`playback as well as temporary modem delays are avoided.
`
`21 Claims, 3 Drawing Sheets
`
`@37.5
`14 243
`
`2:41:
`
`PAGE 1 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 1 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`US 8,364,839 B2
`Page2
`
`7,085,842 B2
`3113333 31
`7,170,856 B1
`7,212,574 B2
`7’260’564 B1
`7,272,298 B1
`7,287,083 B1
`7,318,017 B2
`7,334,016 B2
`7,334,044 B1
`7,346,698 B2
`7’349’663 B1
`7,373,413 B1
`7,376,710 B1
`7,471,834 B2
`7’478’164 B1
`7,496,676 B2
`7,590,656 B2
`7,647,297 B2
`7,689,510 B2
`7,818,444 B2
`7,890,631 B2
`7’917’557 B2
`2001/0047,377 A1
`2002/0007418 A1
`2002/0023165 A1
`2002/0029166 A1
`2002/0069218 A1
`2002/0078174 A1
`2002/0083182 A1
`2002/0120675 A1
`2002/0131443 A1
`2002/0147634 A1
`2002/0177914 A1
`2003/0014488 A1
`2003/0018978 A1
`2003/0061305 A1
`2003/0068046 A1
`2003/0093790 A1
`2003/0186645 A1
`2004/0078812 A1
`2004/0086120 A1
`2004/0123725 A1
`2004/0131340 A1
`2004/0162910 A1
`2004/0186733 A1
`2004/0231004 A1
`2004/0260835 A1
`2005/0005025 A1
`2005/0080876 A1
`2005/0108320 A1
`2005/0190915 A1
`2005/0203917 A1
`2005/0251832 A1
`2005/0262251 A1
`2006/0095472 A1
`2006/0143667 A1
`2006/0153537 A1
`2007/0005428 A1
`2007/0016865 A1
`2007/0038728 A1
`2007/0079327 A1
`2007/0088804 A1
`2007/0226365 A1
`2007/0233784 A1
`2007/0274672 A1
`
`U.S. PATENT DOCUMENTS
`51751968 A
`5/1998 Cohen
`5,758,527 A
`571998 Z11“
`5,778,374 A
`771998 Dang
`5,793,980 A
`8/1998 Glaseretal.
`5,805,823 A
`971998 85112
`5,809,239 A
`971998 D11“
`5,815,552 A
`971998 011%
`5,819,150 A
`1071998 1351115515
`5,821,985 A
`1071998 Y115“
`5,822,537 A
`10/1998 Katseffetal.
`5,835,495 A
`11/1998 Ferriere
`5,857,230 A
`271999 Wang
`5,881,245 A
`371999 111551155511
`5,892,915 A
`471999 D1155
`5,910,876 A
`6/1999 Sharmaetal.
`5,922,048 A
`771999 E51515
`5,923,655 A
`7/1999 Veschietal.
`5,928,330 A
`771999 G55”
`5,933,503 A
`871999 8511111111
`5,938,734 A
`871999 Y”
`5,955,715 A
`971999 K5111151
`5,963,202 A
`10/1999 Polish
`5,974,503 A
`1071999 V5“1“115S11
`5,978,557 A
`1171999 R5151“ 51111,
`5,987,510 A
`1171999 151111
`5,995,705 A
`1171999 Lang
`5,995,015 A
`1171999 Day
`5,999,525 A
`12/1999 Krishnaswamyet al.
`5,002,720 A
`1271999 8551151111,
`5,014,593 A
`172000 11° 5151,
`5,014,594 A
`172000 A11‘“°“1 51 111,
`5,014,705 A
`172000 51111115“ 51111,
`5,029,194 A
`272000 1111
`5,032,193 A
`272000 81111511“
`6,047,317 A
`4/2000 Bisdikian etal.
`5,047,355 A
`472000 A11°151S°“
`5,057,832 A
`572000 15V 5151,
`5,051,731 A
`572000 1311555155
`5,051,732 A
`572000 K5151 51 51,
`5,055,050 A
`572000 D5M°“5Y
`5,085,221 A
`772000 G151
`5,151,532 A
`1172000 51111551111
`5,173,328 131
`172001
`8515
`5,173,340 131
`172001 G15*15Y 5151,
`6,233,226 B1
`5/2001 Gringerietal.
`5,279,040 131
`872001 M5 5151,
`5,292,834 131
`972001 R11“ 5151,
`5,321,259 B1
`1172001 W*111‘51
`5,377,995 132
`472002 Ag”111‘1“1m
`5,385,595 131
`572002 W15“
`5,389,473 B1
`572002 51115151
`5,405,255 B1
`572002 L1“
`5,430,520 131
`872002 051515
`5,449,719 131
`972002 135851
`5,452,943 B1
`972002 Fumya
`5,535,920 131
`372003 Pmy
`5,588,015 131
`772003 EY515”11~
`5,594,599 B1
`772003 81111111 5151,
`5,598,228 132
`772003 1151511
`5,525,750 131
`972003 D1155
`5,537,031 B1
`1072003 C115“
`5,555,751 B1
`1272003 C115“ 51111,
`5,708,213 131
`372004 135515151511 5151,
`6,728,753 B1
`4/2004 Parasnis
`6,757,796 B1
`6/2004 Hofmann
`6,829,368 B2
`12/2004 Meyer et al.
`6,831,892 B2
`12/2004 Robinett etal.
`
`6,850,965 B2
`6,889,257 B1
`6,907,481 B2
`6,925,495 B2
`6,938,047 B2
`6,988,144 B1
`6,990,497 B2
`7,020,710 B2
`7,039,784 B1
`7,054,500 B1
`
`2/2005 Allen
`5/2005 Patel
`6/2005 Kovacevic
`8/2005 Hegde
`8/2005 Kryeziu
`1/2006 Luken
`1/2006 O’R0urke
`3/2006 Weber
`5/2006 Chen
`5/2006 Lillevoid
`
`,
`
`'
`
`,
`
`8/2006 Reid
`373332 83283“ 9 41'
`1/2007 Ho etal
`5/2007 Abrams, Jr
`8/2007 Lynn
`’
`9/2007 Lang etal
`10/2007 Nay
`1/2008 Swoboda
`2/2008 Fishhaut
`2/2008 Allen
`3/2008 Hmmaway
`3/2008 Joseph
`5/2008 Nguyen etal
`5/2008 Cromwell
`12/2008 Sull
`1/2009 Lango
`2/2009 Kryeziu
`9/2009 Plasma
`1/2010 LaChapelle
`3/2010 Lamkin
`10/2010 Brueck
`2/2011 Allen
`3/2011 Shteyn
`11/2001 Sincaglia
`1/2002 Hegde
`2/2002 Lahr
`3/2002 Jacobs
`6/2002 Sull
`6/2002 Sim
`6/2002 Alvarado
`8/2002 Everett
`9/2002 Robinett
`10/2002 Jacoby
`11/2002 Chase
`1/2003 Dalal
`1/2003 Singal
`3/2003 Copley
`4/2003 Lindqvist
`5/2003 Logan
`10/2003 Mori
`4/2004 Calvert
`5/2004 Akins
`7/2004 Kim
`7/2004 Antoun
`8/2004 Kryeziu
`9/2004 Looml-S
`11/2004 seo
`12/2004 Welk
`1/2005 Harville
`4/2005 Peiffer
`5/2005 Lord
`9/2005 Pare
`9/2005 Freeberg
`11/2005 Chiueh
`11/2005 Klernets
`5/2006 Krikorian
`6/2006 Kurosawa
`7/2006 Kaneko
`1/2007 Jacobs
`1/2007 Johnson
`2/2007 Jacobs
`4/2007 Khoo
`4/2007 Qureshey
`9/2007 Hildreth
`10/2007 O’R0uIke
`11/2007 Itoi
`
`200870059532 A1
`
`372008 K1125“,
`
`2008/0195743 Al
`
`8/2008 Brueck
`
`EP
`EP
`EP
`EP
`EP
`EP
`EP
`
`FOREIGN PATENT DOCUMENTS
`680185 A2
`11/1995
`720374 A1
`7/1996
`762300 A2
`3/1997
`827336 A2
`3/1998
`859535 A2
`8/1998
`984584 A1
`3/2000
`1113642 A2
`7/2001
`
`PAGE 2 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 2 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`US 8,364,839 B2
`Page 3
`
`EP
`JP
`JP
`JP
`JP
`JP
`W0
`W0
`W0
`W0
`W0
`W0
`W0
`W0
`
`6/2004
`1427218 A2
`12/1998
`10336626 A
`7/1999
`11184780 A
`5/2000
`2000151595 A
`6/2000
`2000172599 A
`8/2000
`2000228669 A
`4/1997
`9712447 A2
`5/1997
`9717775 A1
`8/1997
`9730551 A1
`10/1998
`9844733 A1
`5/2001
`0138993 A1
`7/2002
`02057943 A1
`3/2003
`03023781 A1
`1/2005
`2005004485 A1
`OTHER PUBLICATIONS
`
`an
`in
`“Smoothing variable-bit-rate video
`al.,
`Rexford et
`internetwork,” IEEE/ACM Transactions on Networking, vol. 7, Issue
`2, pp. 202-215, Apr. 1999.
`Bianchi, “Buffer sizing for high speed video information retrieval on
`ATM networks,” GLOBECOM—New York—, 1997, vol. 2, pp.
`1057-1061.
`Salehi et al., “Supporting Stored Video: Reducing Rate Variability
`and End-to-End Resource Requirements through 0ptimal Smooth-
`ing,” IEEE/ACM Transactions on Networking, vol. 6, Issue 4, pp.
`397-410, Aug. 1998.
`
`Mielke et al., A Multi-level Buffering and Feedback Scheme for
`Distributed Multimedia Presentation Systems, IEEE, 1998.
`Nam et al, Adaptive Multimedia Stream Presentation in Mobile Com-
`puting Environment, IEEE TENCON, Sep. 1999.
`Deleon et al., An Adaptive Predictor for Media Playout Buffering,
`1999.
`
`Zhao et al., Bandwidth-Efficient Continuous Media Streaming
`through 0ptimal Multiplexing, 1999.
`Gollapudi et al., Buffer Management in Multimedia Database Sys-
`tems, 1996.
`Hui et al., Client-Server Synchronization and Buffering for Variable
`Rate Multimedia Retrievals, IEEE Journal on SelectedAreas in Com-
`munications, vol. 14, No. 1, Jan. 1996.
`Zheng et al., Multimedia 0ver High Speed Networks: Reducing
`Network Requirements With Fast Buffer Fillup, 1998.
`Zheng et al., Traffic Management of Multimedia over ATM Net-
`works, Jan. 1999.
`Chen et al., Video and Audio: Organization and Retrieval in the
`WW, 1996.
`Yee et al., Server and Buffer Allocation in Client Server Multimedia
`System, IEEE, 1993.
`Rejaie et al., RAP: An end-to-end rate-based congestion control
`mechanism for realtime streams in the Internet, IEEE, Mar. 1999.
`
`PAGE 3 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 3 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 1 of3
`
`US 8,364,839 B2
`
`Fig. 1
`
`PAGE 4 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 4 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 2 of3
`
`US 8,364,839 B2
`
`Fig. 2
`
`
`
`PAGE 5 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 5 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`U.S. Patent
`
`Jan. 29, 2013
`
`Sheet 3 of3
`
`US 8,364,839 B2
`
`32
`
`34
`
`t »
`
`PAGE 6 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 6 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`US 8,364,839 B2
`
`1
`STREAMING MEDIA DELIVERY SYSTEM
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of U.S. patent applica-
`tion Ser. No. 12/800,177, filed May 10, 2010 (published on
`Sep. 2, 2010 as U.S. patent publication number 2010/
`0223362 A1 and now U.S. Pat. No. 8,185,611), which was a
`continuation of U.S. patent application Ser. No. 10/893,814,
`filed Jul. 19, 2004 (published on Dec. 9, 2004 as U.S. patent
`publication number 2004/0249969A1, and now U.S. Pat. No.
`7,716,358), which was a continuation-in-part of U.S. patent
`application Ser. No. 09/819,337, filed Mar. 28, 2001 (now
`U.S. Pat. No. 6,766,376), which claimed the benefit under 35
`U.S.C. §1 1 9(e) ofU.S. provisional patent application Ser. No.
`60/231,997, filed Sep. 12, 2000; and it claims the benefit,
`under 35 U.S.C. §120, of the respective filing dates of said
`applications, as well as benefit of the filing date of copending
`U.S. patent application Ser. No. 10/825,869, filed Apr. 16,
`2004 (published on Dec. 23, 2004 as U.S. patent publication
`number 2004/260828 A1), which was a continuation of said
`U.S. patent application Ser. No. 09/819,337, and of copend-
`ing U.S. patent application Ser. No. 12/800,152, filed May 10,
`2010 (published on Sep. 16, 2010 as U.S. patent publication
`number2010/0235536A1, andnow U.S. Pat. No. 8,122,141),
`which was also a continuation of said U.S. patent application
`Ser. No. 10/893,814; and hereby incorporates by reference
`the entire disclosure of each of said prior applications.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to multimedia computer com-
`munication systems; and more particularly, to systems and
`methods for delivering streaming media, such as audio and
`video, on the Internet.
`2. Description of the Related Art
`Prior to the development of Internet streaming media tech-
`nologies, audio and video were formatted into files, which
`users needed to download in their entirety to their computers
`before the files could be heard or viewed. Real time, continu-
`ous media, as from a radio station, was not suitable for this
`arrangement, in that a file of finite size must be created so it
`could be downloaded. The advent of streaming media tech-
`nologies allowed users to listen to or view the files as they
`were being downloaded, and allowed users to “tune-in” to a
`continuous media broadcast, or “stream”, such as from a
`radio station.
`
`Sending audio or video files via a network is known in the
`art. U.S. Pat. No. 6,029,194 to Tilt describes a media server
`for the distribution of audio/video over networks, in which
`retrieved media frames are transferred to a FIFO buffer. A
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`clock rate for a local clock is adjusted according to the full-
`ness of the buffer. The media frames from the buffer are sent
`
`55
`
`in the form of data packets over the networks in response to
`interrupts generated by the local clock. In this manner, the
`timing for the media frames is controlled by the user to assure
`a continuous stream of video during editing. U.S. Pat. No.
`6,014,706 to Cannon, et al. discloses an apparatus and
`method for displaying streamed digital video data on a client
`computer. The client computer is configured to receive the
`streamed digital video data from a server computer via a
`computer network.
`The streamed digital video data is transmitted from the
`server computer to the client computer as a stream of video
`frames. U.S. Pat. No. 6,002,720, to Yurt, et al. discloses a
`
`60
`
`65
`
`2
`
`system for distributing video and/or audio information,
`wherein digital signal processing is employed to achieve high
`rates of data compression. U.S. Pat. No. 5,923,655, to Veschi
`et al. discloses a system and method for communicating
`audio/ video data in a packet-based computer network,
`wherein transmission of data packets through the computer
`network requires variable periods of transmission time. U.S.
`Pat. No. 5,922,048 to Emura discloses a video server appa-
`ratus having a stream control section that determines a key-
`frame readout interval and a keyframe playback interval,
`which satisfy a playback speed designated by a terminal
`apparatus. Finally, U.S. Pat. No. 6,014,694 to Aharoni, et al.
`discloses a system and method for adaptively transporting
`video over networks,
`including the Internet, wherein the
`available bandwidth varies with time.
`
`Despite these developments, users viewing or listening to
`streaming content over Internet connections often encounter
`interruptions, due to the frequency of unanticipated transmis-
`sion delays and losses that are inherent in many Internet
`protocols. These interruptions are commonly referred to as
`“dropouts”, meaning that the data flow to the user has been
`interrupted (i.e., the audio “drops out”).
`Dropouts can be extremely annoying—for example, while
`listening to music. The current state-of-the-art solution to the
`problem uses a pre-buffering technique to store up enough
`audio or video data in the user’s computer so that it can play
`the audio or video with a minimum of dropouts. This process
`requires the user to wait until enough of the media file is
`buffered in memory before listening or viewing can begin.
`The media data is delivered by a server computer, which has
`available to it the source of the media data, such as by a
`connection to a radio station. When the user connects to the
`
`server via the Internet, audio/video output at the user’ s system
`is delayed while the user’s buffer is filled to a predetermined
`level. Typical pre-buffering wait times range from ten to
`twenty seconds or more, determined by the vendor providing
`the audio or video media. Even with this pre-buffering pro-
`cess, interruptions in playback still occur.
`In this process, the user has a software application on the
`computer commonly called a “media player”. Using the fea-
`tures built into the media player, the user starts the audio or
`video stream, typically by clicking on a “start” button, and
`waits ten to twenty seconds or so before the material starts
`playing. During this time data is being received from the
`source and filling the media player’s buffer. The audio or
`video data is delivered from the source at the rate it is to be
`
`played out. If, for example, the user is listening to an audio
`stream encoded to be played-out at 24,000 bits per second, the
`source sends the audio data at the rate of 24,000 bits per
`second. Provided that the user waits ten seconds, and the
`receipt of the buffering data has not been interrupted, there is
`enough media data stored in the buffer to play for ten seconds.
`Gaps in the receipt of audio/video data, due to Internet
`slowdowns, cause the buffer to deplete. Because transmission
`of audio/video media data to the user takes place at the rate it
`is played out, the user’s buffer level can never be increased or
`replenished while it is playing. Thus, gaps in the receipt of
`audio/video media data inexorably cause the buffer level to
`decrease from its initial level. In time, extended or repeated
`occurrences ofthese gaps empty the user’s buffer. The audio/
`video material stops playing, and the buffer must be refilled to
`its original predetermined level before playing of the media
`resumes.
`
`By way of illustration, if, in a ten second pre-buffering
`scenario, data reception stopped the instant that the media
`started playing, it would play for exactly ten seconds. Once
`the media data starts playing, it plays out of the buffer as new
`
`PAGE 7 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 7 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`US 8,364,839 B2
`
`3
`media data replenishes the buffer. The incoming data rate
`equals the rate at which the data is played out of the user’s
`buffer, assuming the receipt of data across the Internet is
`unimpeded. If there are no interruptions in the receipt of the
`media data for the duration of the time the user listens to or
`watches the material, the buffer level remains constant and
`there will still be ten seconds of data stored in the media
`player’s buffer when the user stops the player.
`On the other hand, ifthe media player encounters interrup-
`tions totaling six seconds while playing the material, there
`would only be four seconds of media data remaining in the
`buffer when the user stopped it. Ifdata reception interruptions
`at any time during the playing exceed ten seconds, the user’s
`media player buffer becomes exhausted. There is no media
`data to play, and the audio or video stops—a dropout has
`occurred. At this point a software mechanism in the media
`player stops attempting to play any more of the material, and
`starts the buffering process again. The media player remains
`silent until the buffer refills, at which time the media player
`will once again start playing the material. This pattern has
`brought about considerable consumer
`frustration with
`streaming media over the Internet.
`
`SUMMARY OF THE INVENTION
`
`There is a need for improved systems and methods for
`delivering streaming content over the Internet or other com-
`munications medium, which facilitate continuous transmis-
`sion of streaming content, respond on demand without obj ec-
`tionable buffering delay, and perform without disruption or
`dropouts.
`To address these objectives, various embodiments for
`delivering streaming content are provided, which envision
`that both the server and user systems involved in the content
`delivery may have buffering capacity. The embodiments
`make varying uses of this capacity to facilitate continuous
`content transmission on demand. Nearly instantaneous play-
`back is achieved, while maintaining protection against play-
`back interruption.
`In one aspect, the server and user-sides of the transmission
`are coordinated, by (a) sending initial streaming media ele-
`ments to the user system at a sending rate more rapid than the
`playback rate, to fill the user buffer; and (b) after the user
`buffer has been filled, sending further streaming media data
`elements to the user system at about the playback rate.
`In another embodiment, the user system may be used to
`regulate transmission of streaming media to it, by a streaming
`media server. In such embodiment, the server may operate by
`(a) assigning identifiers to the sequential media data elements
`comprising the program; (b) receiving requests from the user
`system for media data elements corresponding to specified
`identifiers; and (c) sending media data elements to the user
`system responsive to said requests. A user system used in
`connection with such an embodiment may operate by (i)
`maintaining a record of the identifier of the last sequential
`media data element that has been received by said player; (ii)
`requesting transmission of the next sequential media data
`elements following said last sequential media data element,
`as said media player requires for continuous and uninter-
`rupted playback.
`Other aspects and advantages of the invention will be
`apparent from the accompanying drawings and the detailed
`description that follows.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The invention will be more fully understood and further
`advantages will become apparent when reference is had to the
`following detailed description and the accompanying draw-
`ings, in which:
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`FIG. 1 is a schematic/block diagram illustrating the ele-
`ments of a streaming media buffering system in accordance
`with one embodiment of the present invention;
`FIG. 2 is a schematic/block diagram of an alternative
`embodiment of the system shown by FIG. 1; and
`FIG. 3 is a flowchart illustrating a method employed in one
`embodiment of the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`The following is a detailed description of certain embodi-
`ments ofthe invention chosen to provide illustrative examples
`of how it may preferably be implemented.
`Audio and video media must play out over a period oftime.
`Thus, in considering the delivery of such media, it is more
`appropriate in certain respects to think of bandwidth require-
`ments than file size. The bandwidth requirement of audio or
`video media refers to the data rate in bits per second that must
`be transmitted and received in order to listen to or view the
`
`material uninterrupted.
`Transmitting the audio or video material over a connection
`slower than the bandwidth requirement results in unsatisfac-
`tory viewing or listening, if viewing or listening is possible at
`all. The connection available may, for example, be by dialup
`modem, which has a maximum receive data rate of 56,000
`bits per second. Audio and video encoded for distribution
`over the Internet may be compressed to be listenable or view-
`able within such a 56,000 bits per second bandwidth.
`Requirements for achieving adequate audio and video over
`the Internet may consume a considerable portion of the lis-
`tener’s available bandwidth.
`
`There are two types of encoding schemes used for audio
`and video material—“Variable Bit Rate” (VBR), and “Con-
`stant Bit Rate” (CBR). CBR encoding represents the encoded
`media with a constant bit rate per second, regardless of the
`complexity of the material being encoded. For example, if an
`audio source is encoded at 20 kilobits per second at a Constant
`Bit Rate, the media data being produced from the encoding is
`at 20 kilobits per second, whether the audio material is com-
`plex (e.g., symphonic) or silence. Variable Bit Rate encoding
`uses a variable number of bits to represent sounds or video,
`with more bits required for complex material (e.g., sym-
`phonic sounds or action scenes) than for simple sounds,
`silence, or still scenes. The most usual encoding scheme used
`for streaming media is CBR, because the resulting data rate is
`more predictable than for VBR. Statements in this specifica-
`tion concerning “constant” data rates and the like should be
`understood as subject to appropriate variation where VBR-
`encoded data may be involved.
`Even if a user’s Internet connection has the requisite aver-
`age bandwidth capacity to allow reception ofthe program, the
`actual rate of delivery of data to the user can fluctuate widely
`above, and more particularly, below, this average, as a func-
`tion ofthe quality ofthe user’s connectivity at any given time.
`Internet connection quality can vary rapidly over time, with
`two primary factors responsible for degradation ofthe instan-
`taneous bandwidth actually available to the user. These fac-
`tors are the quality of the user’s Internet connection, which
`can have periods of interference causing reduced available
`bandwidth, and momentary Internet congestion at various
`points along the route over which the user’s data flows. Each
`of these factors can cause delays and interruptions in the
`transmission of data to the user. Internet data communications
`
`devices such as routers are designed to drop data packets if
`they get overloaded. For material that is not time sensitive,
`these dropped packets will usually be resent, and the user will
`
`PAGE 8 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 8 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`US 8,364,839 B2
`
`5
`eventually be presented with the material. However, since
`streaming media is time sensitive, dropped packets can have
`a significant impact on the receipt and playback of an audio or
`video stream. Such degradation in the receipt of Internet data
`is very common, and prevents most users from being able to
`listen to or view streaming media without interruption unless
`some special provisions have been incorporated into the
`user’s computer software to accommodate data transmission
`interruptions.
`There are two fundamental types of streaming media,
`which affect, in some respects, the requirements for smooth
`and continuous delivery: (i) material that originates from a
`source having a realtime nature, such as a radio or TV broad-
`cast, and (ii) material that originates from a non-real-time
`source such as from a disk file. An example of non-real-time
`material might be a piece of music stored as a disk file, or a
`portion of a broadcast that originally was realtime, perhaps
`yesterday’s TV evening news, and was recorded into a disk
`file. For purposes of clarity within this document, streaming
`media of type (i) will be referred to as “real time” or “broad-
`cast” media, and streaming media of type (ii) will be referred
`to as “file based” media.
`
`In many respects, both streaming media types are handled
`similarly in conventional systems, and both are handled simi-
`larly (in a number of respects) by the streaming media deliv-
`ery system of the present invention. Nevertheless, the two
`streaming media types are readily distinguished. Broadcast
`streaming media has as its source a system or arrangement
`that by definition can only be transmitted to users as fast as the
`material is generated; for example, a disk jockey speaking
`into a microphone. File based media, on the other hand, can
`be transmitted to users at any available data rate, since in the
`context ofdata communications, the time required for reading
`a small portion of data from a file residing entirely on a locally
`accessible, random access storage device may be considered
`negligible.
`In conventional systems for streaming media over the
`Internet, media data (whether real-time or file based) is sim-
`ply transmitted from the server to the user at the rate at which
`it will be played out (the “playback rate”), regardless of the
`data rate capabilities ofthe connection between the server and
`the user.
`
`Conventional streaming media systems may incorporate
`server-side buffering systems for programmatic purposes.
`For example, the system may buffer media data at the server
`for the purpose of packet assembly/disassembly. Media data
`may also be buffered at the server to permit programming
`conveniences such as dealing with blocks of data of a specific
`size. However, conventional streaming media systems have
`not utilized server-side buffering for the purpose ofmitigating
`long term Internet performance degradation. Rather, prior art
`systems, in which data is continuously transmitted at the
`playback rate, have performed buffering for continuity pur-
`poses solely on the user side, with the consequences dis-
`cussed above of startup delays and dropouts. The present
`invention addresses such shortcomings.
`The present invention provides a system and method for
`delivering streaming media, such as audio or video media, via
`the Internet or other communications medium. Immediate
`
`playing of the media on a user’s computer is afforded, while
`reducing interruptions in playback due to Internet congestion,
`and temporary modem delays due to noisy lines. Nearly
`instantaneous playback is achieved, while maintaining pro-
`tection against playback interruption. Delayed starts, hereto-
`fore required to provide protection against interruption, are
`avoided. Data lost due to interruptions in the receipt of media
`data by the media player can be recovered while the player
`
`6
`continues to play out the audio or video material. If the
`interruptions are so severe as to deplete the user’s buffer and
`stop the play out, the media player can quickly recover as
`well, by beginning to play out again without waiting to first
`build up the buffer, as soon as the media player begins to
`receive media data elements.
`
`In one embodiment, the invention provides a system for
`distributing via the Internet streaming media composed of a
`plurality of time-sequenced data elements. As shown in FIG.
`1, the system is provided with a server 12 connected to the
`Internet 10 for transmitting the streaming media data ele-
`ments. Associated with the server 12 is a server buffer 14 for
`
`storing at least one of the data elements for transmission, and
`a buffer manager 16. Buffer 14 is a conventional computer
`storage mechanism such as a hard disk, as shown for conve-
`nience of illustration, or, preferably, an electronic storage
`arrangement such as Random Access Memory (RAM).
`The media may come from a live source, shown as 26 in
`FIG. 1, or from a stored file on the server 12, or another
`storage device, such as a hard drive.
`A number of different implementations of such a server,
`involving different ways of handling server buffer 14, will be
`discussed.
`
`10
`
`15
`
`20
`
`25
`
`In the various implementations, there is in each case at least
`one user computer 18 (or similar device) connected to the
`server 12 via the Internet 10 or other data communications
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`medium. User computer 18 is associated with media player
`software incorporating user buffer 20. The user buffer 20 is
`provided with means for storing a predetermined number of
`the data elements. User buffer 20 is a conventional computer
`storage mechanism such as a hard disk, or, preferably, an
`electronic storage arrangement such as Random Access
`Memory (RAM) as suggested by the illustration. A buffer
`manager 22 is also associated with the user computer 18. The
`buffer manager 22, having the form of software or firmware,
`is provided with means for receiving and storing a predeter-
`mined number of media data elements which are received
`
`sequentially by the media player, playing the data out sequen-
`tially as audio and/or video, and deleting media data elements
`from the buffer as they are played out (or displacing them by
`newly arrived elements). As data is played out, the next
`sequential data elements are received from the server in such
`a fashion as to approximately maintain the predetermined
`number of data elements in the user’s buffer. It should be
`
`understood that data might arrive at the media player out-of-
`sequence and that processes in the media player or the media
`player buffer manager are responsible for properly arranging
`this data.
`
`Alternatively, user computer 18 may be replaced by an
`Internet radio or Internet Appliance, which is comprised of a
`dedicated processor for receiving Internet radio or audio/
`video material. Examples of such devices might range from
`familiar computing devices such as palmtops, PDAs (Per-
`sonal Digital Assistants), and wireless phones, to devices that
`appear and operate similarly to conventional consumer elec-
`tronic devices such as radios and televisions, but with the
`additional capability of Internet access.
`FIFO Server Buffer Implementation
`There are a large number of ways of managing server
`buffer 14 in order to implement the systems and methods
`described in this specification. In one implementation, buffer
`manager 16 is adapted to effectively render server buffer 14 a
`FIFO device. In this implementation, buffer manager 16 is
`provided in the form of software or firmware that provides
`means for: receiving the media data; supplying media data in
`order to the FIFO buffer; supplying the buffer 14 with a
`predetermined number of data elements; maintaining point-
`
`PAGE 9 OF 16
`
`|.M.L. SLU'S EXHIBIT 1001
`
`PAGE 9 OF 16
`
`I.M.L. SLU'S EXHIBIT 1001
`
`

`

`US 8,364,839 B2
`
`7
`ers 2411 through 2411 into the buffer, one for each user com-
`puter indicating the last media data element that has been sent
`to that user, thus indicating the next elem