(12) United States Patent
`Goldhor et al.
`
`(10) Patent N0.2
`(45) Date of Patent:
`
`US 6,625,656 B2
`Sep. 23, 2003
`
`US006625656B2
`
`(54) METHOD AND APPARATUS FOR
`CONTINUOUS PLAYBACK ()R
`DISTRIBUTION OF INFORMATION
`
`5,652,627 A * 7/1997 Allen ....................... .. 348/497
`5,692,213 A * 11/1997 Goldberg et al. ......... .. 345/302
`5,694,521 A * 12/1997 Shlomot et al.
`704/262
`
`_
`VISUAL STREAMED
`
`5,749,064 A * 5/1998 Pawate et al. . . . . . .
`5,758,076 A * 5/1998 Wu et al.
`
`. . . .. 704/213
`709/231
`
`5,767,863 A * 6/1998 Kimura .................... .. 345/531
`5,793,980 A * 8/1998 Glaser et al. ............. .. 709/231
`5,806,023 A * 9/1998 Satyamurti . . . . . .
`. . . .. 704/211
`5,808,662 A * 9/1998 Kinney et al.
`..... .. 348/15
`5,818,436 A * 10/1998 Imaietal. . . . . . . .
`. . . .. 345/302
`5,822,537 A * 10/1998 Katseff et al.
`709/231
`5,842,172 A * 11/1998 Wilson . . . . . . . .
`. . . .. 704/503
`
`,
`(75) Inventors: Rlchard 5' Goldhorg Belmont’ MA
`(Us); Donald J- HeJlla, Jr» L05 A1t°$>
`CA (US)
`
`(73) Assignee: Enounce, Incorporated, Palo Alto, CA
`
`(US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U'S'C' 154(k)) byo days‘
`
`(21) APPI- NO-I 09/519,487
`
`.
`(22) Flled:
`
`(65)
`
`Ma“ 6’ 2000
`-
`-
`-
`Prlor Pubhcatlon Data
`
`US 2002/0052967 A1 May 2, 2002
`
`Related US. Application Data
`
`(63) Continuation-in-part of application No. 09/304,761, ?led on
`May 4, 1999.
`
`7
`
`""""""""""""
`
`(58) Field Of Search ............................... .. 709/217, 219,
`709/218, 231, 232, 233, 234, 235; 348/6,
`7 513 516 518 725/101 92 93 94 95
`’
`’
`’
`’ 96 88 ég 60 £34 ’142’
`’
`’
`’
`’
`’
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`
`5,864,678 A * 1/1999 Riddle . . . . . .
`. . . .. 709/235
`5,864,682 A * 1/1999 Porter et al. .............. .. 709/247
`5,893,062 A * 4/1999 Bhadkamkaf et a1~ ~~~~ ~~ 704/270
`2 i
`$119} et a1- -
`6,018,359 A * 1/2000 Kermode et al.
`
`an]1 ...................... ..
`
`725/101
`
`,
`
`,
`
`6,086,537 A * 7/2000 Urbano et al. . . . . . .
`
`. . . .. 600/443
`
`6,123,548 A * 9/2000 Tallal et al. . . . . .
`
`. . . .. 434/185
`
`6,124,878 A * 9/2000 Adams et al. . . . . . .
`. . . .. 725/118
`6,169,843 B1 * 1/2001 Lenihan et al. .......... .. 386/46
`6,357,047 B1 * 3/2002 KurtZe et al.
`725/151
`6,397,251 B1 * 5/2002 Graf . . . . . . . . . . . .
`. . . .. 709/225
`
`6,415,326 B1 * 7/2002 Gupta et al. .............. .. 709/231
`6,449,653 B2 * 9/2002 Klemets et al. ........... .. 709/231
`_
`_
`* cited by examiner
`
`Primary Examiner—Mark R. Powell
`Assistant Examiner—William C. Vaughn, Jr.
`(74) Attorney, Agent, or Firm—Michael B. Einschlag
`
`(57)
`
`ABSTRACT
`.
`.
`.
`.
`An embodiment of the present invention is an apparatus for
`preparing media for playback Which comprises: (a) a buffer
`Which stores data corresponding to the media; (b) a time
`scale modi?cation system that time-scale modi?es data
`output from the buffer at a playback rate; and (c) a rate
`determiner that determines the playback rate over an interval
`
`5,544,324 A * 8/1996 Edem et al. .............. .. 709/231
`
`to Control the amount of data m the buffer‘
`
`5,630,013 A * 5/1997 Suzuki et al. ............. .. 704/216
`5,649,050 A * 7/1997 Hardwick et al. ........ .. 704/203
`
`2 Claims, 16 Drawing Sheets
`
`10o
`
`STREAMING
`DATA
`SOURCE
`20(]\
`
`400
`
`CAPTURE
`BUFFER
`
`CAPTURE
`500\ BUFFER
`MONITOR
`
`m
`
`3pm
`USER SYSTEM
`
`H00
`/
`TSH
`SUBSYSTEH
`
`PLAYBACK /500
`SYSTEM
`E
`E
`i
`TSH RATE _ _____ ___,
`DETEHMINEH
`
`700
`
`USER
`INTERFACE
`
`500
`
`PAGE 1 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 1 0f 16
`
`US 6,625,656 B2
`
`FIG. 1
`
`- 2100
`STORAGE
`
`fzooo
`
`MEDIA
`BROADCAST
`SERVER
`
`2200
`l
`
`MEDIA
`RECIPIENT
`
`MEDIA
`RECIPIENT
`
`MEDIA
`RECIPIENT
`
`MEDIA
`RECIPIENT
`
`\
`23001
`
`\
`23002
`
`\
`23003
`
`X
`23004
`
`_
`
`MEDIA
`RECIPIENT
`
`\
`2300
`
`PAGE 2 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 2 0f 16
`
`US 6,625,656 B2
`
`100\ STREAMING
`DATA
`SOURCE
`
`200\_
`
`FIG . 2
`
`USER SYSTEM
`
`CAPTURE
`BUFFER
`
`—_
`
`TSH
`SUBSYSTEM
`__
`
`PLAYBACK / 500
`SYSTEM
`
`USER
`INTERFACE
`
`_/900
`
`PAGE 3 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 3 0f 16
`
`US 6,625,656 B2
`
`FIG. 3
`
`DATA om-—‘
`
`_——DATA IN
`
`0
`
`TL
`
`TH
`
`MAX
`
`’ FIG. 4
`
`PLAYBACK
`HATE
`
`0
`
`TL
`
`TH
`
`DATA LEVEL
`MAX
`
`0 f x5 TL
`TL< XL< TH
`TH f x f MAX
`
`PLAYBABK RATE = (S?ALEHanh'i?x-TL)HTLH
`PLAYBACK RATE = 1.0
`PLAYBACK HATE = tS?ALEltanh'illX-THl I (MAX-THU
`
`PAGE 4 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 4 0f 16
`
`US 6,625,656 B2
`
`INPUT
`T0 TSH
`SUBSYSTEM
`
`OUTPUT
`0F TSM
`SUBSYSTEM
`
`\
`
`FIG. 5
`
`J\
`
`FIG. 5
`
`J\
`
`\
`
`»
`1
`
`a,
`t
`
`’
`I
`INPUT
`T0 TSM A .
`SUBSYSTEM l V m t
`
`~
`
`.
`
`OUTPUT Lnm
`
`SUBSYSTEM
`
`PAGE 5 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 5 0f 16
`
`US 6,625,656 B2
`
`FIG. 7
`
`STREAMING
`100
`\ DATA
`SOURCE
`
`3000
`
`200\
`
`300
`
`.._..
`
`F
`
`_ ............ ..!_ ........................ ._
`
`T
`i
`
`i
`
`DATA ARRIVAL /5400
`—- TIME-STAMP
`SYSTEM
`
`1
`
`L
`
`800 Z
`
`400\
`
`TSM
`‘
`CAPTURE
`BUFFER _' SUBSYSTEM
`
`PLAYBACK /500
`’ SYSTEM
`
`700 f
`1
`*- uSER
`TSM RATE
`DETEHMINEH -—Y INTERFACE
`
`/900
`
`I
`
`’
`55°°\ DATA DEPARTURE
`TIME-STAMP j'_’ TIHE'STAMP /5500
`
`5300
`SYSTEM /
`CLOCK
`
`PAGE 6 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 6 6f 16
`
`US 6,625,656 B2
`
`STREAMING
`100
`\ DATA
`SOURCE
`20o\
`
`FIG. 8
`
`4000
`
`3210
`
`‘_ ...... .._ _ ......................................... .__|
`
`l
`l
`000
`1
`/
`PLAYBACK /500'
`,
`TSM
`,
`400\ CAPTURE
`BUFFER """ SUBSYSTEH —' SYSVTEM
`
`I
`
`1
`_
`INTERFACE —- DETERSINER
`
`DATA
`DETECTOR
`
`A5100:
`-
`
`700
`
`PAGE 7 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 7 0f 16
`
`US 6,625,656 B2
`
`FIG. 9
`
`330
`
`STREAMING
`100
`\ DATA
`sounce
`2nn\
`
`5m
`
`PM
`
`_
`
`_
`
`........... .1
`
`i
`s2oo\
`i
`‘ \——- INPUT RATE
`: -— momma
`
`900
`/
`PLAYBACK /500
`,
`ISM
`“" SUBSYSTEH "" SYSTEM
`
`l
`
`CAPTURE
`BUFFER
`
`400
`\
`
`i
`
`l
`
`1
`
`700
`
`‘ f
`TSM RATE -—‘ USER
`DETEHHINEH '—- INTERFACE
`l
`
`fsoo
`
`1
`
`L
`OUTPU: RATE
`RATE
`B400\
`" COMPARATOR ‘— MONITOR
`
`5300i
`
`.
`
`./
`
`-
`
`L.
`
`5300
`SYSTEM
`CLOCK /
`
`J
`
`PAGE 8 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 8 0f 16
`
`US 6,625,656 B2
`
`FIG. 9A
`
`m
`
`STREAMING
`100
`\ DATA
`SOURCE
`
`200\
`
`w
`
`s2oo\
`
`l
`
`400\ CAPTURE
`BUFFER
`
`300
`
`8?”
`
`_
`
`7
`
`53°
`
`PLAYBACK
`SYSTEM
`
`1
`
`——- INPUT HATE
`-—=\ MONITOR
`
`TSM
`SUBSYSTEH
`
`I fmo
`
`900
`
`I
`
`TSM HATE -'\-' USER
`DETEHMINEH ‘-'\-- INTERFACE
`
`I
`
`6300
`
`I
`7
`—- OUTPUT RATE
`HATE
`“' COMPARATOR -——' MONITOR ""
`\
`6400
`
`I
`' “
`
`5300
`SYSTEM
`CLOCK /
`
`PAGE 9 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 9 0f 16
`
`US 6,625,656 B2
`
`100\ STFEJaATIkING
`SOURCE
`
`STREAMING
`150
`\ DATA
`INFORMATION
`SOURCE
`
`FIG. 10
`
`m
`
`MEDIA AND
`200\. INFORMATION
`DATA PACKETS
`
`300
`L
`
`
`
`T ............ .__ __ ............................................ 5700
`
`Z
`
`INFORMATION
`
`,
`
`BANOWIOTH
`
`,
`
`TE
`
`CLOCK
`
`=
`
`fsaoo .
`
`,
`7°°\ TSM HATE F-—~ USER
`DETERMINE“ ‘-——-J INTERFACE fem
`
`400\ CAPTURE
`,
`TSM
`BUFFER "'_’ SUBSYSTEM
`
`,
`
`,
`
`PLAYBACK
`SYSTEM
`
`'
`
`fs?o
`
`PAGE 10 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 10 0f 16
`
`US 6,625,656 B2
`
`FIG. 11
`
`STREAMING
`100
`\ DATA
`SOUHCE
`
`1m
`
`25o
`
`2oo\
`
`l_ """"" “L """""""" "_ "- """" "_|
`
`i INTERMEDIATE SEHVEHNODE
`
`i
`
`'
`
`'
`
`s0o\ CQYFTFUERRE
`
`l__.
`
`MONITOR
`
`A
`
`i
`CAPTURE /4°0
`
`BUFFER.
`
`~
`
`!
`
`TSHlRAT
`
`E l
`
`i7°°\ DETEHMINER _“' SUBSYSTEM
`
`TJSTT
`
`800'
`
`'
`
`/ '
`
`i_ ........ _, .............. .-
`200
`311
`ir-n-USEFy-ui
`:
`SYSTEM T/
`
`.._.._.._i
`
`PAGE 11 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 11 0f 16
`
`US 6,625,656 B2
`
`
`
`m G. on . M _ n N 0 E . E ERR _
`
`n 0 .
`
`0 . .
`
`
`
`_ MUM l- I MQTU _ _ I MEN “ CALHUnmuL B“ _ _ MW WWN .
`
`_ m k m W Fm ............. MIL _| .... M ........... .m. ..... L
`
`w 6\ o\
`
`_ _ 2 m m
`
`fmwll. m 0\, 661656 2
`
`
`. _ m 6 m
`
`n _ _
`
`m _ Hm _
`
`.............. _ a m
`
`6 9 n a n
`
`H w 6 m
`. m _ _
`
`2 “ HS m u
`1 _ I _ 0 . u
`
`a _ Em _
`w.(_ 7f . _
`
`. mm "an . _ AV. Urr. _
`
`
`
` _\ .... ........... |||| m A ? _ m m
`
`n K w n
`
`. TN "
`
`PAGE 12 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 12 0f 16
`
`US 6,625,656 B2
`
`T .................. -._
`
`'
`
`i
`
`-
`
`170\ STREAMING
`DATA
`GENERATOR
`
`180
`r/
`
`FIG- 13
`
`TSM
`175\ 'ENCODEH' _
`SUBSYSTEM
`
`'
`
`200
`/
`
`10000
`
`L ...... .._
`
`200\
`
`330
`
`|_ ...... .._ _ ............... .533 ..... -._ ................... ..j
`
`|
`0
`z
`!400\ CAPTURE
`,
`.DEQQER. __,,__ PLAYBACK /500!
`
`
`SYSTEM
`II |
`BUFFER —’ SUBSYSTEM | 1 I __
`
`
`I
`
`r
`l
`i5°°\ CAPTURE
`1
`|
`
`I
`900i
`USER
`03M RATE ~—-—-‘
`‘—’ DETEHMINEH
`- INTERFACE / i
`
`PAGE 13 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 13 0f 16
`
`US 6,625,656 B2
`
`FIG. 14
`
`“ED
`1
`
`YELLOW
`1
`
`910
`J
`
`920
`)
`
`YELLOW
`)
`
`|
`0.3
`
`|
`
`|
`0.5
`
`|
`
`|
`
`c
`
`|
`
`|
`
`l
`
`|
`2 .0
`
`|
`3 . 0
`
`/ HED
`
`FIG. 15
`
`sliders
`
`0.3
`
`0.5
`
`1.0
`
`2.0
`
`3.0
`
`906
`[-5 speed Liming
`
`931
`Speed Information tn 0%
`Current Speed
`Max. Sustain Speed
`Requested Speed I? Min. Sustain Speed
`
`941/
`
`PAGE 14 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 14 0f 16
`
`US 6,625,656 B2
`
`FIG. 15
`
`sliders
`
`X
`
`I
`l
`0.3
`
`I
`
`I
`
`l
`
`I
`0.5
`
`l
`1.0
`
`I]
`
`I
`
`l
`
`I
`
`I
`
`I
`2.0
`
`|
`l
`3.0
`
`905
`Bi] Speed uniting
`911
`921
`
`931
`Speed Information to 0%
`Current Speed
`Max. Sustain Speed
`Requested Speed I? Hin. Sustain Speed
`
`I
`341
`
`PAGE 15 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 15 0f 16
`
`US 6,625,656 B2
`
`FIG. 17
`
`sliders
`
`Playback Speed
`
`910
`
`03
`
`'
`
`0T5
`
`I
`
`I
`
`1T0
`
`I
`
`I
`
`210
`
`\Q'ZOEHU
`
`906
`[gm Speed Information to Display
`Current Speed
`Max. Sustain Speed
`Requested Speed
`Min. Sustain Speed
`
`91-1
`
`321
`
`931
`
`$1
`941/
`
`PAGE 16 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`U.S. Patent
`
`Sep. 23, 2003
`
`Sheet 16 0f 16
`
`US 6,625,656 B2
`
`FIG. 1B
`
`sliders
`
`910
`/
`BLUE
`'////7/////////////////////
`A\sno
`
`I
`0.3
`
`I
`
`l
`
`l
`
`l
`0.5
`
`l
`1.0
`
`920
`
`|
`.
`
`I
`
`l
`
`I
`.
`
`3°
`
`2130f 2°
`
`I
`
`90B
`speed Uniting
`911
`921
`
`931
`
`Speed Information to Display
`Current Speed
`Hax. Sustain Speed
`Requested Speed ? Min. Sustain Speed
`
`1
`941
`
`PAGE 17 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`US 6,625,656 B2
`
`1
`METHOD AND APPARATUS FOR
`CONTINUOUS PLAYBACK OR
`DISTRIBUTION OF INFORMATION
`INCLUDING AUDIO-VISUAL STREAMED
`MULTIMEDIA
`
`This is a continuation-in-part of a patent application
`entitled “Method and Apparatus for Continuous Playback of
`Streaming Media” Which Was ?led on May 4, 1999, Ser. No.
`09/304,761.
`
`10
`
`2
`annoying since they occur randomly and have a random
`duration. In addition, intermittent interruptions often cause
`the context of the playback stream to be lost as a user Waits
`for playback to be resumed When neW data is received.
`As one can readily appreciate from the above, a need
`exists in the art for a method and apparatus for providing
`substantially continuous playback of media such as audio
`and audio-visual Works received from sources having non
`deterministic delays such as a server, for example, a ?le
`server or a streaming media server, broadcasting data via the
`Internet.
`
`SUMMARY OF THE INVENTION
`Embodiments of the present invention advantageously
`satisfy the above-identi?ed need in the art and provide
`method and apparatus for providing substantially continuous
`playback of media such as audio and audio-visual Works
`received from sources having non-deterministic delays such
`as a server, for example, a ?le server or a streaming media
`server, broadcasting data via the Internet.
`One embodiment of the present invention is an apparatus
`for preparing media for playback Which comprises: (a) a
`buffer Which stores data corresponding to the media; (b) a
`time-scale modi?cation system that time-scale modi?es data
`output from the buffer at a playback rate; and (c) a rate
`determiner that determines the playback rate over an interval
`to control the amount of data in the buffer.
`
`BRIEF DESCRIPTION OF THE FIGURE
`
`FIG. 1 shoWs, in schematic form, hoW audio or audio
`visual Works are broadcast from a server, for example, a ?le
`server or a streaming media server, to recipients over a
`communication medium, such as, for example, a network
`such as the Internet;
`FIG. 2 shoWs a block diagram of an embodiment of the
`present invention Which provides substantially continuous
`playback of an audio or audio-visual Work received from a
`source having non-deterministic delays such as a server, for
`example, a ?le server or a streaming media server, broad
`casting data via a communication medium, such as, for
`example, a netWork such as the Internet;
`FIG. 3 shoWs, in pictorial form, loW and high thresholds
`used in one embodiment of Capture Buffer 400 in the
`embodiment of the present invention shoWn in FIG. 2;
`FIG. 4. shoWs a graph of playback rate versus the amount
`of data in Capture Buffer 400 in the embodiment of the
`present invention shoWn in FIG. 2;
`FIG. 5. shoWs, in graphical form, relative amounts of data
`at an input and an output of TSM Subsystem 800 in the
`embodiment of the present invention shoWn in FIG. 2 during
`time-scale expansion, i.e., sloW doWn of the playback-rate of
`the streaming media;
`FIG. 6. shoWs, in graphical form, relative amounts of data
`at an input and an output of TSM Subsystem 800 in the
`embodiment of the present invention shoWn in FIG. 2 during
`time-scale compression, i.e., speed up of the playback rate of
`the streaming media;
`FIGS. 7—13 shoW block diagrams of alternative embodi
`ments of the present invention; and
`FIGS. 14—18 shoW displays produced by various embodi
`ments of a graphical interface used to fabricate at least some
`embodiments of the present invention.
`
`15
`
`25
`
`35
`
`45
`
`55
`
`TECHNICAL FIELD OF THE INVENTION
`
`The present invention pertains to the ?eld of playback of
`media such as audio and audio-visual Works Which are
`retrieved from sources having non-deterministic delays such
`as, for example, a server such as a ?le server or a streaming
`media server, broadcasting data via the Internet. In
`particular, the present invention pertains to method and
`apparatus for providing playback of an audio or audio-visual
`Work received from sources having non-deterministic
`delays. In further particular, the present invention pertains to
`method and apparatus for providing continuous playback of
`media from sources having non-deterministic delays such
`as, for example, a server such as a ?le server or a streaming
`media server, broadcasting data via the Internet, an Intranet,
`or the like.
`
`BACKGROUND OF THE INVENTION
`
`Many digitally encoded audio and audio-visual Works are
`stored as data on servers such as ?le servers or streaming
`media servers that are accessible via the Internet for users to
`doWnload. FIG. 1 shoWs, in schematic form, hoW such audio
`or audio-visual Works are distributed over the Internet. As
`shoWn in FIG. 1, media broadcast server 2000 accesses data
`representing the audio or audio-visual Work from storage
`medium 2100 and broadcasts the data to multiple recipients
`23001 to 2300” across non-deterministic delay netWork
`2200. In the system shoWn in FIG. 1, there are tWo main
`sources of random delay: (a) delay due to the broadcast
`server’s accessing storage medium 2100 and (b) delay due
`to the congestion, interference, and other delay mechanisms
`Within netWork 2200. In more complex systems, delays can
`also arise from decoders, multicasting CPU time-slices, and
`other concurrently operating softWare components.
`One Well knoWn technique for providing playback of the
`audio or audio-visual Work is referred to as batch playback.
`Batch playback entails doWnloading an entire Work and
`initiating playback after the entire Work has been received.
`Another Well knoWn technique for providing playback of the
`audio or audio-visual Work is referred to as “streaming.”
`Streaming entails doWnloading data Which represents the
`audio or audio-visual Work and initiating playback before
`the entire Work has been received.
`There are several disadvantages inherent in both of these
`techniques. A prime disadvantage of batch playback is that
`the vieWer/listener must Wait for the entire Work to be
`doWnloaded before any portion of the Work may be played.
`This can be tedious since the vieWer/listener may Wait a long
`time for the transmission to occur, only to discover that the
`Work is of little or no interest soon after playback is initiated.
`The streaming technique alleviates this disadvantage of
`batch playback by initiating playback before the entire Work
`has been received. HoWever, a disadvantage of streaming is
`that playback is often interrupted When the How of data is
`interrupted due to netWork traf?c, congestion, transmission
`errors, and the like. These interruptions are tedious and
`
`65
`
`DETAILED DESCRIPTION
`FIG. 2 shoWs a block diagram of embodiment 1000 of the
`present invention Which provides substantially continuous
`
`PAGE 18 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`US 6,625,656 B2
`
`3
`playback of an audio or audio-visual Work received from a
`source having non-deterministic delays such as a server, for
`example, a ?le server or a streaming media server, broad
`casting via the Internet. As shoWn in FIG. 2, Streaming Data
`Source 100 provides media data representing an audio or
`audio-visual Work through netWork 200 to User System 300
`(US 300), Which media data is received at a non
`deterministic rate by US 300. Capture Buffer 400 in US 300
`receives the media data as input. In a preferred embodiment
`of the present invention, Capture Buffer 400 is a FIFO (First
`In First Out) buffer existing, for example, in a general
`purpose memory store. It should be understood that Capture
`Buffer 400 may also be implemented using pointers and
`conventional system memory, circular buffers, and any one
`of a number of apparatus and methods that are Well knoWn
`to those of ordinary skill in the art.
`In the absence of delays in data arrival at US 300 from
`netWork 200, the amount of data in Capture Buffer 400
`ought to remain substantially constant as the data transfer
`rate is typically chosen to be substantially equal to the
`playback rate. HoWever, as is Well knoWn to those of
`ordinary skill in the art, pauses and delays in transmission of
`the media data through netWork 200 to Capture Buffer 400
`cause data depletion therein since data is simultaneously
`being output (for example, at a constant rate) from Capture
`Buffer 400 to satisfy data requirements of Playback System
`500. As is Well knoWn, if the media data transmitted to US
`300 is delayed long enough, a sufficient amount of data in
`Capture Buffer 400 Will be consumed so that Playback
`System 500 must pause until enough media data has arrived
`to enable resumption of playback. Thus, a typical playback
`system must constantly check for arrival of neW data While
`the playback system is paused, and it must initiate playback
`once neW data is received.
`In accordance With the present invention, data input to
`Capture Buffer 400 of US 300 is buffered for a predeter
`mined amount of time Which typically varies, for example,
`from one (1) second to several seconds. Then, Time-Scale
`Modi?cation (TSM) methods are used to sloW the playback
`rate of the audio or audio-visual Work to substantially match
`a data drain rate required by Playback System 500 With a
`streaming data rate of the arriving data representing the
`audio or audio-visual Work. As is Well knoWn to those of
`ordinary skill in the art, presently knoWn methods for
`Time-Scale Modi?cation (“TSM”) enable digitally recorded
`audio to be modi?ed so that a perceived articulation rate of
`spoken passages, i.e., a speaking rate, can be modi?ed
`dynamically during playback. During Time-Scale
`expansion, TSM Subsystem 800 requires less input data to
`generate a ?xed interval of output data. Thus, in accordance
`With the present invention, if a delay occurs during trans
`mission of the audio or audio-visual Work from netWork 200
`to US 300 (of course, it should be clear that such delays may
`result from any number of causes such as delays in accessing
`data from a storage device, delays in transmission of the data
`from a media server, delays in transmission through netWork
`200, delays Waiting for CPU resources on softWare
`implementations, and so forth), the playback rate is auto
`matically sloWed to reduce the amount of data drained from
`Capture Buffer 400 per unit time. As a result, and in
`accordance With the present invention, more time is pro
`vided for data to arrive at US 300 before the data in Capture
`Buffer 400 is exhausted. Advantageously, this delays the
`onset of data depletion in Capture Buffer 400 Which Would
`cause Playback System 500 to pause.
`As shoWn in FIG. 2, Capture Buffer 400 receives the
`folloWing as input: (a) media data input from netWork 200,
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`4
`(b) requests for information about the amount of data stored
`therein from Capture Buffer Monitor 600; and (c) media
`stream data requests from TSM Subsystem 800. In response,
`Capture Buffer 400 produces the folloWing as output: (a) a
`stream of data representing portions of an audio or audio
`visual Work (this is output to TSM Subsystem 800); (b) a
`stream of location information used to identify the position
`in the stream of data (this is output to TSM Subsystem 800);
`and (c) the amount of data stored therein (this is output to
`Capture Buffer Monitor 600). It should be Well knoWn to
`those of ordinary skill in the art that Capture Buffer 400 may
`include a digital storage device. There are many methods
`Well knoWn to those of ordinary skill in the art for utiliZing
`digital storage devices, for example a “hard disk drive,” to
`store and retrieve general purpose data. There exist many
`commercially available apparatus Which are Well knoWn to
`those of ordinary skill in the art for use as a digital storage
`device such as, for example, a CD-ROM, a digital tape, a
`magnetic disc.
`It should be understood that the data in Capture Buffer
`400 may be comprised of samples of a signal Which are
`usable by TSM SubSystem 800, or alternatively, the data in
`Capture Buffer 400 may be comprised of an encoded rep
`resentation Which, When decoded, provides samples of a
`signal that are usable by TSM SubSystem 800. As one can
`readily appreciate, a decoder utiliZed to decode encoded
`representations of signals can be disposed Within Capture
`Buffer 400, or it can be disposed in any logical location
`betWeen Capture Buffer 400 and TSM SubSystem 800.
`There are numerous apparatus and methods that are Well
`knoWn to those of ordinary skill in the art for implementing
`such a decoder, Which decoder is not shoWn in the ?gures for
`ease of understanding the present invention.
`Capture Buffer Monitor 600 receives, as input, informa
`tion representing (or that can be used to determine) the
`amount of data in Capture Buffer 400. Capture Buffer
`Monitor 600 produces, as output: (a) data requests to Cap
`ture Buffer 400; and (b) information representing (or that can
`be used to determine) the amount of data in Capture Buffer
`400, Which information is output to TSM Rate Determiner
`700. Capture Buffer Monitor 600 utiliZes any one of a
`number of methods that are Well knoWn to those of ordinary
`skill in the art to obtain the information representing (or that
`can be used to determine) the amount of data in Capture
`Buffer 400. For example, and Without limitation, Capture
`Buffer Monitor 600 may periodically poll Capture Buffer
`400 to determine the amount of data in the buffer; Capture
`Buffer Monitor 600 may monitor the arrival and departure of
`data from Capture Buffer 400; and Capture Buffer Monitor
`600 may compute data arrival and departure rates from
`Capture Buffer 400.
`As further shoWn in FIG. 2, and in accordance With the
`present invention, TSM Rate Determiner 700 receives the
`folloWing as input: (a) a signal (from Capture Buffer Moni
`tor 600) that represents the amount of data present in
`Capture Buffer 400 and possibly a data arrival rate for
`Capture Buffer 400; (b) a signal (output, for example, from
`Playback System 500 or from another module of US 300
`such as TSM SubSystem 800) that represents a current data
`consumption rate of Playback System 500; and (c) a number
`of parameters (to be described beloW), Which parameters
`may optionally be supplied by User Interface 900. In some
`embodiments of the present invention, the signal that rep
`resents the current data consumption rate may not be nec
`essary since TSM Rate Determiner 700 can calculate the
`data consumption rate because it is generating the playback
`rate used by TSM SubSystem 800. One or more of the
`
`PAGE 19 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`US 6,625,656 B2
`
`5
`following parameters are input to TSM Rate Determiner
`700: (a) a loW threshold value parameter (TL Which is
`described in detail below) for the amount of data in Capture
`Buffer 400; (b) a high threshold value parameter (TH Which
`is described in detail beloW) for the amount of data in
`Capture Buffer 400; (c) a scale parameter (Scale Which is
`described in detail beloW) Which is used to adjust the
`playback rate; (d) a parameter designated Interval SiZe; and
`(d) a parameter designated SpeediChangeiResolution.
`These parameters may be input in any one of a number of
`methods that are Well knoWn to those of ordinary skill in the
`art. For example, they may be set as predetermined param
`eters for embodiment 1000 in accordance With methods
`Which are Well knoWn to those of ordinary skill in the art,
`i.e., system constants Which are loaded When the system is
`initialiZed, they can be entered and/or vieWed and varied by
`receiving user input through a user interface, for example
`User Interface 900, in accordance With methods Which are
`Well knoWn to those of ordinary skill in the art, and so forth.
`HoWever, the manner in Which these parameters are set
`and/or varied are not shoWn for ease of understanding the
`present invention.
`In response to the input, TSM Rate Determiner 700
`produces, as output, a rate signal representing a TSM rate, or
`playback rate, Which can help better balance the data con
`sumption rate of Playback System 500 With an arrival rate of
`data at Capture Buffer 400.
`It should be understood that some embodiments of the
`present invention can operate in numerous modes. For
`example, one embodiment of the present invention may
`operate in a mode that attempts to balance a data consump
`tion rate With a data arrival rate. In this mode, the embodi
`ment utiliZes changes in playback rate to alter the data
`consumption rate, and as a result, the playback rate of
`material presented by the embodiment is determined by the
`data delivery rate of information from the source, for
`example, a media server. For convenience, this mode is
`referred to as “Rate Determined by Data Arrival” mode. In
`another mode, an embodiment of the present invention: (a)
`monitors various system conditions and user input playback
`presentation rate requests; (b) computes or infers data arrival
`and departure rates; and (c) intervenes Whenever a user
`request Would cause data under?oW or over?oW in Capture
`Buffer 400 or a disruption in playback. For convenience, this
`mode is referred to as “Rate Restricted by Data Arrival”
`mode.
`In some embodiments, the playback rate Will be altered in
`a continuous, for example, sloWly varying, fashion. For
`example, in some embodiments, buffers of time-scale modi
`?ed output may be queued for playback in Playback System
`500. In this case, the queued data may not be modi?ed When
`a user requests a change in playback rate. As a result,
`Whenever a user requests a change in playback rate, there
`may be a delay betWeen the time the request is made and the
`time the change in playback rate is effected. This is because,
`in these embodiments, although time-scale modi?cation
`may begin immediately for data sent to Playback System
`500 after the request for a change in playback rate Was
`received, there may be a delay until data processed With the
`previous rate (and buffered for output) has been played back.
`For this reason, such embodiments may appear to be a bit
`sluggish. To mitigate this perception, in accordance With one
`aspect of such embodiments, feedback is provided to the
`user to indicate a Current Playback Rate (CPR) and a
`Requested Playback Rate (RPR). CPR and RPR shoW the
`user that a neWly requested playback rate has been received
`and that the embodiment is initiating a response.
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`Advantageously, such feedback mitigates a tendency the
`user might have to “overcorrect” in an effort to elicit a
`salient response from any embodiment in Which it is utiliZed.
`In a preferred embodiment of this aspect of the present
`invention, the playback rate of each buffer of data available
`to Playback System 500 is associated With the buffer (this
`can be done by TSM SubSystem 800 or by other compo
`nents of User System 300). Thus, When such buffers are
`presented (or are expected to be presented), User Interface
`900 is provided an indication of the event of presentation of
`the buffer to the user (or expected event of presentation of
`the buffer to the user). In addition, User Interface 900 is
`presented the playback rate for the associated buffer or
`information that can be used to obtain the playback rate. For
`example, Playback System 500 may report the playback rate
`associated With each buffer as the buffer is played back.
`Alternatively, Playback System 500 may report the event of
`presentation of each buffer, and User Interface 900 or TSM
`Rate Determiner 700 may access a table Which contains
`playback rates for each buffer that Was dispatched to Play
`back System 500. This playback rate information is used to
`determine CPR and report it to the user, if desired. For
`example, in one embodiment, CPR is determined to be the
`playback rate of the most recent buffer played back. In other
`embodiments, CPR may be computed using any of several
`mathematical functions, for example, a mathematical
`average, of multiple values of playback rates of a predeter
`mined number of the buffers played. Furthermore, in accor
`dance With some embodiments that display RPR and CPR,
`the user receives con?rmation of his/her request and is able
`to observe the embodiment’s response. In operation, CPR
`Will folloW or chase RPR as the embodiment responds to
`user playback rate requests. As discussed above, such feed
`back may be useful to avoid overcorrections by users When
`the embodiment’s response to user requests appears slug
`gish.
`In a preferred embodiment of the present invention, TSM
`Rate Determiner 700 uses the parameter IntervaliSiZe to
`segment the input digital data stream in Capture Buffer 400
`and to determine a single TSM rate for each segment of the
`input digital stream. Note the length of each segment is
`given by the value of the IntervaliSiZe parameter. Further,
`TSM Rate Determiner 700 uses the parameter Speedi
`ChangeiResolution to determine appropriate TSM rates to
`pass to TSM SubSystem 800. A desired TSM rate is con
`verted to one of the quantiZed levels in a manner Which is
`Well knoWn to those of ordinary skill in the art. This means
`that the TSM rate, or playback rate, can change only if the
`desired TSM rate changes by an amount that exceeds the
`difference betWeen quantiZed levels, i.e., SpeediChangei
`Resolution. As a practical matter then, parameter Speedi
`ChangeiResolution ?lters small changes in TSM rate, or
`playback rate.
`
`In another embodiment of TSM Rate Determiner 700, it
`determines a maximum playback rate that can be used (over
`a given reporting time interval (rti)) Without draining Cap
`ture Buffer 400 so much that playback Would have to pause
`to Wait for more data to arrive. This maximum playback rate
`is referred to as the current maximum sustainable playback
`rate (CmaxSR), and its value represents a scale factor
`applied to a normal playback rate, i.e., a playback rate With
`no time-scale modi?cation. In accordance With the present
`invention, CmaxSR is given as folloWs:
`
`PAGE 20 of 31
`
`PETITIONERS' EXHIBIT 1007
`
`

`

`US 6,625,656 B2
`
`8
`rli=time interval over which a playback rate must be
`sustained
`Thus if Capture Buffer 400 is capable of holding a 64,000
`samp