(12) United States Patent
`Walker et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,974,200 B2
`Jul. 5, 2011
`
`US007974200B2
`
`(54) TRANSMITTING AND RECEIVING
`REAL-TIME DATA
`
`(75) Inventors: Matthew DWalker, Felixstowe (GB);
`Richard J. Jacobs, Woodbridge (GB);
`Michael E Nilsson, Ipswich (GB)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`4,813,044 A
`3/1989 Kumar et al. ................. T14,809
`(Continued)
`
`(73) Assignee: British Telecommunications public
`limited company, London (GB)
`
`EP
`
`FOREIGN PATENT DOCUMENTS
`O 418 396 A1
`3, 1991
`(Continued)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1843 days.
`
`(21) Appl. No.:
`
`10/415,819
`
`(22) PCT Filed:
`
`Nov. 28, 2001
`
`PCT/GBO1A5246
`
`(86). PCT No.:
`S371 (c)(1),
`(2), (4) Date: May 2, 2003
`(87) PCT Pub. No.: WO02/45372
`PCT Pub. Date: Jun. 6, 2002
`O
`O
`Prior Publication Data
`US 2004/O153951 A1
`Aug. 5, 2004
`
`(65)
`
`(30)
`
`Foreign Application Priority Data
`
`Nov. 29, 2000 (EP) ..................................... OO310594
`
`(51) Int. Cl.
`(2006.01)
`H04 IAI6
`(52) U.S. Cl. .......................................... 370/235; 725/87
`(58) Field of Classification Search .................. 370/230,
`370/329, 337,342, 412,441, 487, 229, 230.1,
`370/231, 232,233,234, 235, 395.2, 395.21;
`375/147, 240.01, 240.12; 709/229; 34.5/531;
`348/390.1; 725/87, 93, 94, 96, 118, 119
`See application file for complete search history.
`
`24
`
`OTHER PUBLICATIONS
`“Receiver-driven layered multicast'. S. McCanne, V. Jacobson, and
`M. Vetterli. In Proc. ACM Sigcomm 96, pp. 117-130, Palo Alto, CA,
`Aug. 1996.
`
`(Continued)
`Primary Examiner — Ricky Ngo
`Assistant Examiner — Kan Yuen
`(74) Attorney, Agent, or Firm — Nixon & Vanderhye P.C.
`(57)
`ABSTRACT
`Real-time data (e.g. Video) is streamed over packet networks
`(e.g. the Internet). Streamed video is provided without the
`start-up delay by transmitting data from a video streamer to
`the video viewer more rapidly than the video viewer con
`Sumes the data and using the excess data to build a buffer at
`the video viewer. When a suitable sized buffer is built the
`transmission rate of data to the buffer may be reduced. In
`order to deliver the best quality material for the available
`bandwidth, the supply of video data may be switched to a
`higher bit-rate source when the reservoir is filled. Fluctua
`tions in network throughput may be accommodated during
`the transmission of data on a fine Scale by adjusting the
`transmission rate of the data and on a coarse scale by Switch
`ing between data streams encoded at different bit-rates. Fluc
`tuations in network throughput are determined by counting
`the number of missing packets at the video viewer which
`information may then be fed back to the video streamer to
`adjust the flow of data accordingly.
`
`12 Claims, 2 Drawing Sheets
`
`
`
`Buffer
`manager
`
`Switch
`
`
`
`
`
`23
`
`VIMEO/IAC EXHIBIT 1001
`VIMEO ET AL., v. BT, IPR2019-00833
`
`

`

`US 7,974,200 B2
`Page 2
`
`5.535.209 A
`
`7, 1996 Glaser et al.
`
`J. WW
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`6,226.329 B1* 5, 2001 Ishibashi
`4- 4 W
`7/2001 Lakshman et al.
`6.269,078 B1
`299; St.
`3.22. R
`370,260
`656 El
`25s.I.
`6,310,857 B1 * 10/2001 Duffield et al. ............... 370,232
`6,324,165 B1 * 1 1/2001 Fan et al. ...................... 370,232
`6,373,855 B1
`4/2002 Downing et al.
`- -
`8:36 R
`$383 Shelby et al.
`6,470,378 B1
`10/2002 Tracton et al.
`6,480,448 B1 * 1 1/2002 Kawano et al. .............. 369/47.5
`6,480,541 B1 * 1 1/2002 Girod et al. ...
`375,240.12
`6,487.528 B1 * 1 1/2002 Vossing et al. ................ TO4,229
`6,493,388 B1
`12, 2002 Wang
`6,501,797 B1
`12, 2002 Wall der Schaar et al.
`3:56 R* 3: Er 370/487
`ck
`6,593.930 B1* 7/2003. Sheaffer et al. .
`345,531
`6,600,737 B1 * 7/2003 Lai et al. ....................... 370/352
`6,614,843 B1
`9/2003 Gordon et al.
`6,618,363 B1* 9/2003 Bahl ............................. 370,329
`6,618,381 B1* 9/2003 Miyamoto et al. ....... 370,395.43
`6,625,119 B1* 9/2003 Schuster et al. .............. 370/230
`6,637,031 B1 10/2003 Chou
`6,640,086 B2 10/2003 Wall
`6,661,777 B1 * 12/2003 Blanc et al. ................... 370,252
`6,697.369 B1* 2/2004 Dziong et al. .
`370,395.2
`6,700,893 B1* 3/2004 Radha et al. .................. 370/412
`6,701,372 B2
`3/2004 Yano et al.
`6,731,097 B1
`5/2004 Richards et al.
`6.738,386 B1* 5/2004 Holmqvist .................... 370/412
`6,744,815 B1
`6/2004 Sackstein et al.
`6,754,189 B1* 6/2004 Cloutier et al. ............... 370,329
`6,778.499 B1* 8/2004 Senarath et al. .............. 370,232
`6,810,425 B2 10/2004 Yamamoto
`6,813,275 B1
`1 1/2004 Sharma et al.
`6.850,564 B1
`2/2005 Pejhan et al.
`6,909,693 B1* 6/2005 Firoiu et al. .................. 370,235
`
`7/2005 Curet et al.
`6,920, 178 B1
`U.S. PATENT DOCUMENTS
`29. (all
`239: R:
`5,140,417 A * 8, 1992 Tanaka et al. ............ 375,240.01
`T10,56
`6993,604 B2* /2006 Dixon.
`5,159,447 A * 10/1992 Haskell et al. ........... 375/240.05
`7027516 B2
`4.2006 Anderson et al."
`5,363,138 A * 1 1/1994 Hayashi et al. ............ 348/390.1
`7.058.723 B2
`62006 Wilson
`RE34,824 E *
`1/1995 Morrison et al.
`348/419.1
`7106758 B2 + 92006 Belketal
`370,503
`5,511,054 A * 4, 1996 Oishi et al. ................. 369759.27
`7,116,714 B2 10/2006 Hannuksela
`5.561466 A
`10/1996 Kiriyama
`7,142,509 B1
`1 1/2006 Rovner et al.
`5.566,308 A 10, 1996 Rhnan
`256', 38 Rial
`5,675,696 A * 10/1997 Ishimoto et al. .............. 386, 112
`- ww.
`Ima et al.
`5,706.504. A
`1/1998 Atkinson et al.
`7,444,419 B2 10/2008 Green
`WW
`7,542.435 B2
`6/2009 Leon et al.
`5,748.955 A
`5/1998 Smith
`7,761.901 B2
`7/2010 Walker et al.
`3. A
`3.
`th s
`38885 A.
`858 Rial.
`5,818,818 A * 10/1998 Soumiya et al. .............. 370,252
`2001/0028789 A1 * 10, 2001 Uchide ......................... 386,111
`3.
`A 1988 R al.
`2001/0040700 A1 11/2001 HannukSela et al.
`5.874,997 A
`2/1999 E.
`39882. A.
`29: Rou
`5,892,881. A * 4, 1999 Takishima et al. .............. 386,52
`5,898,671. A
`4, 1999 Hunt
`2002/0009096 A1
`1/2002 Odenwalder ................. 370,441
`5,909.434 A * 6/1999 Odenwalder et al
`370,342
`2002/0010938 A1
`1/2002 Zhang et al.
`5915.30 A
`6/1999 Kim
`T10/68
`.
`.
`.
`.
`.
`.
`.
`.
`.
`39882. A
`398: Massal.
`5,918,020 A * 6/1999 Blackard et al. .............. TO9,228
`upta et al.
`5,928.330 A
`7, 1999 Goetz et al.
`2002/0041585 A1
`4/2002 Bahl ............................. 370,337
`5.956.32 A
`9/1999 Yao'etal
`370,230
`5.960.45. A
`6/1999 Chi " 2002/0057889 A1
`5/2002 Ando et al.
`6,01779 A *
`1/2000 Wills. 370,236
`2002, 0083184 A1
`6, 2002 Elliott
`W .
`.
`.
`2002/0114292 A1* 8, 2002 Kawabata et al. ............ 370,329
`6,014,694. A
`3. Aharoni et al
`2002/0131408 A1
`9, 2002 HSu et al.
`709,232
`8:52 A. 39 Stal
`2002/0131496 A1
`9, 2002 Vasudevan et al.
`6061732 A
`5/2000 Korsteal."
`2002fO167942 A1 11, 2002 Fulton
`Ww J.
`2003/0037158 A1
`2/2003 Yano et al.
`6,065,104 A
`5/2000 Tng
`2003, OO72370 A1* 4, 2003 Girod et al. .............. 375,240.13
`6,081,843 A
`6/2000 Kikki et al.
`2003/0076858 A1
`4/2003 Deshpande
`6,092,115 A
`7/2000 Choudhury et al.
`2003/0103515 A1
`6/2003 Brown et al. ................. 370,412
`6,097.697 A
`8/2000 Yao et al.
`2003/0153311 A1* 8, 2003 Black ............................ 455,436
`; A
`$39. S.
`2003/0169932 A1
`9, 2003 Li et al.
`6,124,878 A * 9/2000 Adams et al. ................. 725/118
`58.85% A. 3.
`. . . . . . . . . . . . . . . . . . . . . . .
`Out . . . . .
`6.181.821 B1
`1/2001 Lim .............................. 382,232
`6216,173 B1
`4/2001 Jonesetal
`2004/0078460 Al
`42004 Valaviet al.
`
`.
`
`.
`
`.
`
`. 375,240.26
`
`725/87
`
`
`
`20040114684 A1
`6/2004 Karczewicz et al.
`2004/O181817 A1
`9, 2004 Larner
`2004/0190600 A1* 9, 2004 Odenwalder ................. 375,147
`2005/0010697 A1
`1/2005 Kinawi et al.
`2005/0021830. A 1
`1/2005 Urzaiz et al.
`2839; A.
`338 SE, al
`2005/0172028 A1
`8, 2005 Nilsson etal
`... 386,112
`2005/019 1038 A1* 9/2005 Ogawa et al.
`2006/0064.501 A1
`3/2006 Harris et al. .................. 709,232
`388-3. A.
`858. Walker
`2006/0182016 A1
`8/2006 Walker et all
`2008/0250454 Al
`10/2008 Nishina et al.
`2009/0116551 A1
`5/2009 Nilsson et al.
`2009/O133075 A1
`5/2009 Nishina et al.
`FOREIGN PATENT DOCUMENTS
`O 763 944 A2
`3, 1997
`O939 545 A2
`9, 1999
`O 948 211 A2 10, 1999
`1045,555 A2 10/2000
`1 120966 A2
`8, 2001
`1 128 610 A2
`8, 2001
`1 130921 A1
`9, 2001
`1241891. A
`9, 2002
`2363. 277 A 10, 2000
`2367219. A
`3, 2002
`O7 O123172
`5, 1995
`7-236.136
`9, 1995
`T-264580
`10, 1995
`T-303117
`11, 1995
`9-93.553 A
`4, 1997
`9-261613
`10, 1997
`9-298.734
`11, 1997
`10-126771
`5, 1998
`10-164533
`6, 1998
`10-262.245 A
`9, 1998
`11-164270
`6, 1999
`
`EP
`EP
`EP
`EP
`EP
`EP
`EP
`EP
`GB
`GB
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`
`

`

`US 7,974,200 B2
`Page 3
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`KR
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`
`11-184780
`11-187367
`11-2391.63
`11-3O8271
`11-313301
`2000-151705
`2000-183958
`2000-228669
`2001-1448O2
`2000-28335
`WO98,26604
`OO/OO1151
`WOOO/O1151
`OO,352O1
`WOOO,352O1
`WOOOf 41365
`WOOOf 49810
`WOOOf 62552
`WOO1,395.08 A1
`WO O2/O54776 A1
`
`7, 1999
`7, 1999
`8, 1999
`11, 1999
`11, 1999
`5, 2000
`6, 2000
`8, 2000
`5, 2001
`5, 2000
`6, 1998
`1, 2000
`1, 2000
`6, 2000
`6, 2000
`T 2000
`8, 2000
`10, 2000
`5, 2001
`T 2002
`
`OTHER PUBLICATIONS
`“TCP-like congestion control for layered multicast data transfer”. L.
`Vicisano, L. Rizzo, J. Crowcroft, In Proc. IEEE Infocom '98, vol. 3,
`pp. 996-1003, San Francisco, CA, Mar. 1998.
`“Layered audiovisual coding for multicast distribution on IP net
`works”. J. O. M. Nilsson, D. Dalby, in 6' IEEE, European Workshop
`on distributed Imaging, Nov. 1999.
`“Viewer response to time-varying video quality”. D. Pearson. In
`Human Vision and Electronic Imaging III (B.E. Rogowitz and T. N.
`Pappas, eds.), vol. Proc. SPIE, vol. 3299, (San Jose, CA), pp. 16-25.
`Jan. 1998.
`“Two-layer coding of video signals for VBR networks'. M.
`Ghanbari, IEEE Journal on Selected Areas of Communications, vol.
`7, No. 5, pp. 771-781, Jun. 1989.
`"Retransmission Scheme for MPEG Streams in Mission Critical
`Multimedia Applications'. Sugh-Hoon Lee, Sungyoung Lee,
`EUROMICRO 1998, pp. 574-580.
`"Error-resilient video transmission using long-term memory motion
`compensated prediction'. T. Wiegand, N. Farber, K. Stuhlmuller, B.
`Girod, IEEE Journal on Selected Areas in Communications, vol. 18,
`No. 6, Jun. 2000, pp. 1050-1062.
`“Connections with multiple congested gateways in packet-switched
`networks, part 1: one-way traffic'. S. Floyd, ACM Computer Com
`munications Review, vol. 21. No. 5, pp. 30-47, Oct. 1991.
`“TCP-Friendly Unicast Rate-Based Flow Control'. J. Mandavi and
`S. Floyd, Available from http://www.psc.edu/networking?papers/
`tep friendly.html, Jan. 1997.
`“TCP Congestion Control”. M. Allman, V. Paxson, W. Stevens,
`Internet Engineering Taskforce Request for Comment 2581, posted
`on the Internet Engineering Taskforce Internet site at http://www.jetf.
`org/rfc/rfc2581.txt.
`Rejaie et al. "Quality Adaptation for Congestion Controlled Video
`Playback over the Internet”. Proc. ACMSIGCOMM 99, Aug.-Sep.
`1999.
`Rejaie et al., “RAP: An End-to-end Rate-based Congestion Control
`Mechanism for Realtime', Proc. IEEE Infocom, Mar. 1999.
`Walker et al., “A Study of the Efficiency of Layered Video Coding
`Using H.263.” Proc. of Packet Video 99, The 9th International Packet
`Video Workshop, New York, Apr. 1999, pp. 1-10.
`Morrison et al., “Reduction of the Bit-Rate of Compressed Video
`While In Its Coded Form.” In Sixth International Workshop on Packet
`Video; Portland, Oregon; Sep. 1994, 4 pages.
`European Search Report Oct. 11, 2002 in EP 022.52216.
`International Search Report Jul. 5, 2004 in PCT/GB2004/000974.
`International Search Report Jun. 21, 2002 in PCT/GB 01/05246.
`International Search Report May 21, 2003 in PCT/GB 03/01358.
`International Search Report Dec. 10, 2002 in PCT/GB 02/04203.
`Rejaie et al., “Layered Quality Adaptation for Internet Video Stream
`ing”, IEEE Journal on Selected Areas in Communications, Dec.
`2000, IEEE, USA, vol. 18, No. 12, pp. 2530-2543.
`Sisalem et al., “MLDA: A TCP-Friendly Congestion Control Frame
`work for Heterogeneous Multicast Environments'. International
`
`Workshop on Quality of Service (IWQOS)—Conference Proceed
`ings Article, Jun. 5, 2000, pp. 65-74.
`Kurceren et al., Synchronization-Predictive Coding for Video Com
`pression: The SP Frames Design for JVT/H.26L, IEEE ICIP 2002.
`Postel, Transmission Control Protocol, Prepared for Defense
`Advanced Research Projects Agency, Information Processing Tech
`niques Office, 1400 Wilson Boulevard, Arlington, Virginia 22209,
`Jan. 1980.
`International Search Report Dec. 16, 2002 in PCT/GB 02/04182.
`Bolotet al.: “Experience with Control Mechanisms for Packet Video
`in the Internet”, XP000751635, Computer Communication Review,
`ACM SIGCOMM, 2004, pp. 4-15.
`Office Action dated Oct. 3, 2008 in U.S. Appl. No. 10/507,111.
`Office Action dated Dec. 17, 2007 in U.S. Appl. No. 10/507, 111.
`Office Action dated Aug. 16, 2007 in U.S. Appl. No. 10/507,826.
`Office Action dated Mar. 25, 2008 in U.S. Appl. No. 10/507,826.
`Office Action dated Sep. 4, 2008 in U.S. Appl. No. 10/507,826.
`Office Action dated Aug. 22, 2006 in U.S. Appl. No. 10/508,122.
`Office Action dated Feb. 20, 2007 in U.S. Appl. No. 10/508,122.
`Office Action dated Aug. 14, 2007 in U.S. Appl. No. 10/508,122.
`Office Action dated Nov. 1, 2007 in U.S. Appl. No. 10/508,122.
`Office Action dated Jun. 20, 2008 in U.S. Appl. No. 10/508,122.
`Office Action dated Dec. 30, 2008 in U.S. Appl. No. 10/508,122.
`Office Action dated Aug. 27, 2007 in U.S. Appl. No. 10/488,345.
`Office Action dated Feb. 1, 2008 in U.S. Appl. No. 10/488,345.
`Office Action dated Sep. 11, 2008 in U.S. Appl. No. 10/488,345.
`Office Action dated Mar. 31, 2008 in U.S. Appl. No. 10/549,582.
`Office Action dated Sep. 30, 2008 in U.S. Appl. No. 10/549,582.
`Sullivan et al., “Using the Draft H.26L Video Coding Standard for
`Mobile Applications.” IEEE Image Processing 2001 International
`Conference.
`Karcewicz et al., “VCEG-L27' ITU-Telecommunications Standard
`ization Sector, Jan. 4, 2001.
`Postel, “Transmission Control Protocol.” Prepared for Defense
`Advanced Research Projects Agency, Information Processing Tech
`niques Office, 1400 Wilson Boulevard, Arlington, Virginia 22209,
`Jan. 1980.
`Naoki Wakayama et al., “On TCP-friendly Video Transfer", IEICE
`Technical Report, vol. 99 No. 428, Nov. 15, 1999, pp. 79-84.
`Kurceren R. et al: “Improved SP-frame Encoding. VCEG-M73.” ITU
`Telecommunications Standardization Sector Video Coding Experts
`Group, Apr. 2, 2001, pp. 1-5.
`Office Action dated Apr. 6, 2009 in U.S. Appl. No. 10/549,582.
`Office Action dated Jun. 23, 2009 in U.S. Appl. No. 10/488,345.
`Office Action dated Apr. 13, 2009 in U.S. Appl. No. 10/508,122.
`Office Action dated Jun. 25, 2009 in U.S. Appl. No. 10/507,826.
`Office Action (21 pgs.) dated May 28, 2010 in U.S. Appl. No.
`10/488,345.
`Office Action (13 pgs.) dated Apr. 13, 2010 in U.S. Appl. No.
`10/507,826.
`Applicant Response (33 pgs.) to Office Action dated Jun. 23, 2009 in
`U.S. Appl. No. 10/488,345.
`Applicant Response (4pgs.) to Office Action dated Jun. 25, 2009 in
`U.S. Appl. No. 10/507,826.
`Office Action (11 pgs.) dated Jan. 13, 2010 in U.S. Appl. No.
`10/508,122.
`Office Action (8 pgs.) dated Aug. 3, 2009 in U.S. Appl. No.
`10/549,582.
`Applicant Response (10 pgs.) to Office Action dated Aug. 3, 2009 in
`U.S. Appl. No. 10/549,582.
`Kim, et al., “Network-Aware Error Control Using Smooth and Fast
`Rate Adaptation Mechanism for TCP-Friendly Internet Video.”
`IEEE, pp. 320-325 (2000).
`Wu et al., “The core experiment on the SP frame coding.” Joint Video
`Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, 2" meeting
`Geneva, CH, Jan. 29-Feb. 1, 2002, document: JVT-B112, Feb. 3,
`2002, Microsoft Nokia.
`Office Action mailed Apr. 2, 2010 in U.S. Appl. No. 12/230,550.
`Notice of Allowance mailed Mar. 12, 2010 in U.S. Appl. No.
`10/549,582.
`Office Action mailed Feb. 3, 2009 in U.S. Appl. No. 10/507,826.
`* cited by examiner
`
`

`

`U.S. Patent
`
`Jul. 5, 2011
`
`Sheet 1 of 2
`
`US 7,974,200 B2
`
`
`
`1
`
`Dual rate
`encoder
`
`Streamer
`
`Transmission
`Rate TE
`
`Transmission
`Rate Ts
`Figure 1
`
`24
`
`Buffer
`
`2.
`
`manager
`
`
`
`R
`
`Buffer
`
`
`
`
`
`It
`
`
`
`27
`
`28
`
`23
`
`Figure 2
`
`
`
`
`
`
`
`
`
`31
`
`Packet
`OSS
`detector
`
`Client
`Status
`monitor
`
`Display
`
`Figure 3
`
`

`

`U.S. Patent
`
`Jul. 5, 2011
`
`Sheet 2 of 2
`
`US 7,974,200 B2
`
`Video Streamer
`
`Initialize
`buffers
`
`25
`
`Start read-Out
`Of R data at
`Ts=2T
`
`25
`
`
`
`Switch to RH
`data at
`TS = 2TE
`
`Client
`
`Select
`program
`
`Start client
`viewer
`
`"Send data"
`Signal
`
`Buffer R. data
`at TE
`
`Read-Out cata to
`display at TE
`
`Detect buffer
`full
`
`"Send SWitch"
`Signal
`
`40
`
`42
`
`44
`
`46
`
`48
`
`50
`
`52
`
`54
`
`Buffer R data. Read
`out Rdata
`
`56
`
`Read-Out RH
`data
`
`Figure 4
`
`

`

`US 7,974,200 B2
`
`1.
`TRANSMITTING AND RECEIVING
`REAL-TIME DATA
`
`This application is the U.S. national phase of international
`application PCT/GB01/05246 filed 28 Nov. 2001 which des- 5
`ignated the U.S.
`
`BACKGROUND
`
`2
`operating said display device to:
`receive second-encoding-rate data packets representing a
`Subsequent part of real-time presentation into said store;
`remove second-encoding-rate data packets from said store at
`said second encoding rate for decoding to present said
`real-time presentation to said user at a second level of
`quality higher than said first level of quality.
`According to another aspect of the present invention there
`is provided a method of presenting time-sensitive data at a
`client while constructing a buffer of time-sensitive data, said
`method comprising receiving time-sensitive data which has
`been transmitted to said client, passing said time-sensitive
`data to a data buffer, and, monitoring the quantity of time
`sensitive data in the data buffer, reading said time dependent
`data out of the data buffer to be processed for viewing:
`wherein the method is characterised in that the rate at which
`the time-sensitive data is read out of the data buffer is lower
`than the rate at which the time-sensitive data is passed to th
`data buffer; and the time-sensitive data is read out of the data
`buffer when it arrives in the data buffer, such that there is
`substantially no delay between the client receiving the time
`sensitive data and making the time-sensitive data available;
`and, presenting the time-sensitive data.
`There will come a point when the data buffer becomes
`sufficiently full. The rate of transmission can then be reduced
`to equal the rate of consumption by the viewing means which
`will bring the quantity of data in the buffer to an equilibrium.
`However, in this situation the bandwidth of the connection
`may not be employed to full capacity.
`Inafurther aspect of the present invention there is provided
`a method of presenting time-sensitive data at a client,
`wherein, time-sensitive data encoded at a first bit-rate is
`received until a pre-determined quantity of data fills the data
`buffer, whereupon time-sensitive data encoded at a second
`bit-rate is received, wherein said second bit-rate is higher than
`the first bit-rate.
`A still further aspect of the present invention provides a
`method of providing time-sensitive data to a client is taught,
`wherein time-sensitive data encoded at a first bit-rate is read
`from a first data buffer at a first transmission rate to be trans
`mitted to the client; and, upon request, time-sensitive data
`encoded at a second bit-rate is read from a second data buffer
`at a second rate.
`It is desirable to use as much of the available bandwidth of
`a link as possible to transmit data because with a higher
`bit-rate of video data comes better quality reproduction.
`However, loss of data in the network causes severe degrada
`tion of service—far outstripping the benefits of increased
`bit-rate. For example, with predictive coding schemes such as
`H.263 and MPEG, receiving half of a 500 kbits' video
`stream is likely to give a much worse quality than all of a 250
`kbits' stream. It is therefore important to reduce transmis
`sion rate in a controlled way, rather than letting databe lost to
`the network. The Internet protocol TCP has a built-in control
`mechanism whereby the data transmission rate is steadily
`increased until packet loss is detected, whereupon the data
`rate is reduced. The data rate is then increased again until
`packet loss reoccurs. A variable transmission rate is said to be
`elastic and applications which are able to control the trans
`mission rate of data in response to network conditions are said
`to be TCP-friendly. It is desirable to provide video data in a
`TCP-friendly way so that the as much of the bandwidth avail
`able at any particular time is utilised. A further benefit of
`TCP-friendly data delivery is that congestion in the network
`is managed as individual applications themselves reduce data
`rates until each has a fair share of the bandwidth.
`
`10
`
`15
`
`1. Technical Field
`The invention is in the field of handling of time-sensitive
`data over packet Switched networks, and more particularly
`transmitting and receiving video data over the Internet.
`The invention relates to a method of providing a streaming
`Video service to a client across a packet network whilst reduc
`ing the start-up delay usually associated with preparing a
`buffer of data while maintaining the use of a buffer. The
`invention also relates to a method of controlling the transmis
`sion rate of the streaming video to adapt to congestion in the 20
`network.
`2. Related Art
`Traditionally the Internet has supported traffic such as FTP,
`e-mail and web-surfing, where the overall delay does not
`intrinsically detract from the final presentation of the media. 25
`The advent of faster processing multimedia PCs has driven
`the delivery of multimedia, including video, over the Internet.
`Time-sensitive applications however require continuous,
`quality of service guaranteed, high bandwidth data channels,
`which is seemingly at odds with the packet-based nature of
`the Internet and has the potential to disrupt transmissions with
`unacceptable packet jitter, i.e. the variation in the inter-arrival
`times of packets caused by variable routing and changeability
`of delivery rates owing to congestion. Currently, commercial
`streaming technologies overcome jitter by constructing a
`large buffer (5-30 seconds) before starting to playback video
`material. This start-up delay is non-optimal for a user, who
`may have to wait for this period, before realizing that the
`content requested is incorrect; and generally detracts from the ao
`users experience of the multimedia presentation.
`
`30
`
`BRIEF SUMMARY
`
`50
`
`According to a first aspect of the present invention there is 45
`provided a method of operating a real-time communication
`apparatus comprising a real-time data sender, a real-time data
`display device having a store and a network connecting said
`sender and said display device, said method comprising the
`steps of:
`operating said sender to transmit a plurality first-encoding
`rate data packets representing a first part of a real-time
`presentation to said display device, said transmission rate
`being higher than said encoding rate;
`operating said display device to:
`receive said first-encoding-rate data packets into said store;
`remove first-encoding-rate data packets from said store at
`said first encoding rate for decoding to present said real
`time presentation to said user at a first level of quality;
`on said store being filled with said first-encoding-rate data to 60
`a predetermined level, sending an indication that said level
`has been reached to said sender,
`operating said sender, on receipt of said indication, to send
`second-encoding-rate data packets representing Subse
`quent parts of said real-time presentation to said display 65
`device, said second encoding rate being higher than said
`first encoding rate;
`
`55
`
`

`

`US 7,974,200 B2
`
`3
`Standard compression technologies, such as MPEG4 or
`H.263 can be managed to exhibit TCP-friendly behaviour, see
`for example the applicant’s co-pending patent application
`number GB 9928.023.2. This solution, however, requires a
`high-speed, dedicated PC per video stream. Transcoding an
`encoded data stream from a high bit-rate to a low bit-rate
`when network congestion is detected also suffers from the
`problem of being computationally demanding. Another
`approach is to use layering of video streams, whereby quality
`adaption is achieved by adding or dropping layers of the video
`stream. The disadvantage of this method is that it is ineffi
`cient, as a certain proportion of the available bandwidth must
`be allocated to instructions for integrating the layers.
`The present invention further provides a method wherein
`the rate at which time-sensitive data is read out from first and
`second buffers may be dynamically varied in dependence
`upon the condition of a link to the client, and further, time
`sensitive data encoded at a first bit-rate is read from a first data
`buffer at a first transmission rate to be transmitted to the
`client; or, time-sensitive data encoded at a second bit-rate is
`read from a second data buffer at a second rate, independence
`upon the condition of a link to the client, wherein said first
`bit-rate is lower than the second bit-rate.
`
`10
`
`15
`
`25
`
`4
`FIG.2 shows the arrangement of the video streamer 2. Low
`quality encoded video data encoded at a low bit-rate R, and
`high quality encoded video data encoded at a high bit-rate R.
`from the encoder 2 is received at the input connections 21 and
`22 respectively and fed to buffers 23 and 24 respectively. It
`should be noted that there is provided one buffer per channel
`ofencoded video data that is received by the video streamer 2.
`Encoded video data is read out from each buffer 23, 24 via a
`switch26 which selects which encoded video data stream is to
`be sent to the output connection 27. There is provided a buffer
`manager 25 which is capable of controlling the rate at which
`data is read out from each of the buffers 23, 24 and thus
`defines the transmission rate Ts of the video streamer 2. The
`buffer manager is also in connection with the switch26 and is
`further capable of receiving signals from connection 28. Tsis
`selected by varying the time delay between the transmission
`of each packet, such that Ts may be less than, equal to or
`greater than the encoder transmission rate T.Those skilled in
`the art will realise that the limiting factor on the sustainability
`of transmission where T-T is the size of the buffer 23, 24
`such that a buffer of size Skbits will be able to sustain a
`transmission rate of Ts-2T for twice as long as a buffer of
`size S/2 kbits. Through the control of both switch 26 and the
`transmission rate Ts the buffer manager is able to control the
`bit-rate which is output from the video streamer 2 on two
`scales; by adjusting the transmission rate T's fine control of the
`bit-rate is achieved, and by switching between the two
`encoded data streams encoded at bit-rates R, and R control
`of the bit-rate on a coarse scale may be achieved. The buffer
`manager 25 makes adjustments to Ts or Switches the output
`between buffers in response to signals received from connec
`tion 28.
`FIG.3 shows the arrangement of the client running on a PC
`3a, b, c etc. The encoded video data that is sent from the video
`streamer 2 is received at the client via a connection 27 and
`checked for completeness by a packet loss detector 31. The
`data is then sent into a client buffer 32 which is of a size
`suitable to absorb fluctuations in network throughput. The
`client buffer32 is connected directly to a decoder 33 and from
`there decoded data is sent to be displayed at the client screen
`(not shown). A client status monitor 34 is connected to the
`packet loss detector 31 and client buffer 32. The client status
`monitor 34 is able to send signals via connection 28.
`The packet loss detector 31 monitors incoming packets. If
`packet loss is detected then a signal is sent to the client status
`monitor 34, which is informs the buffer manager at the video
`streamer 2 via connection 28. Missing packets can be retrans
`mitted. The buffer manager 25 steadily increases the trans
`mission rate Tsuntil a consistent pattern of packet loss occurs,
`indicating that the maximum bandwidth is being utilised. In
`the interest of maintaining a congestion free network, the
`transmission rate Ts may then be exponentially reduced. The
`client status monitor 34 monitors the volume of data in the
`client buffer 32 such that a signal is sent via connection 28 to
`the buffer manager 25 at the video streamer 2 when the client
`buffer 32 becomes sufficiently full of data.
`The system of video streamer 2 and client 3 as described
`above allows user-friendly video streaming, i.e. the client
`buffer 32 enables the quality of the video to be despite varia
`tions in network conditions, which might otherwise have a
`detrimental effect on the overall perceived quality of the
`media.
`The operation of the present embodiment of the invention
`will now be described with reference to FIG. 4.
`The video streamer 2 is initialised, which involves filling
`the buffers 23, 24 with a quantity of data from the encoder 1.
`For a live broadcast, data is constantly fed into the buffers 23,
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Embodiments of the invention will now be described, by
`way of example only, with reference to the figures, where:
`FIG. 1 is a schematic overview of the relationship between
`encoder, video streamer and clients;
`FIG. 2 shows the arrangement of the video streamer;
`FIG.3 shows the arrangement of a client; and
`FIG. 4 shows the stepwise operation of one embodiment of
`the present invention.
`35
`
`30
`
`DETAILED DESCRIPTION OF EXEMPLARY
`EMBODIMENTS
`
`As shown in FIG. 1, a first embodiment of the present
`invention consists of a source of compressed video data,
`encoder 1, which encodes data both at allow bit-rate R, which
`may have a value of for example 500 kbits', and a high
`bit-rate R of for example 1500 kbits'. The compression
`codec used is H.263 but equally may be any other codec, such
`as MPEG4. Encoder 1 takes live' video data as its input, for
`instance a broadcast of a sporting event.
`The two encoded data streams are transmitted via separate
`logical connections to the video streamer 2 at a transmission
`rate TE. The video streamer 2 may be on the same premises as
`the encoder 1 and linked via an intranet. The video streamer 2
`runs on a server computer, for instance one comprising a
`Pentium III 700 MHz, 256MB RAM which has access to the
`Internet.
`A video viewer, hitherto referred to as the client, running
`on a PC (a, b, c etc in FIG. 1) suitably configured to have
`access to the Internet, may connect to the video streamer 2 via
`the Internet and thus the client is able to access content. A
`suitable PC terminal is a 266 MHz. Pentium II laptop PC. The
`Video streamer 2 can Support a large number of clients (typi
`cally up to 1000) viewing the same stream.
`For a live broadcast, the encoder 1 will transmit at a trans
`mission rate TE which is real-time. The two streams of data
`R, and R coded at different bit-rates offer different quality
`Video reproduction, but each data stream has the same trans
`mission rate, T. The data must be decoded at this rate for the
`program to play back in real-time.
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`5
`24 and is Subsequently discarded after an amount of time
`defined by the size of the buffer and the quality of data being
`received.
`APC running browser software to browse web pages on the
`Internet may be used to select a link to, for example, a live
`broadcast on a site hosted by the entity providing streamed
`Video. Being interested in viewing the particular clip or
`broadcast, the user clicks (selects) the link at 40. The brows
`ing software detects that streamed video data has been
`requested and launches the video viewing client software at
`42 which embodies the client 3. The client 3 issues a “send
`data command at 44 via connection 28 to the buffer manager
`25, which sets switch 26 to read encoded video data from the
`low bit-rate data buffer 23 and requests a transmission rate of
`T-2T. The data is transmitted to the data connection 27 and
`thence to the client 3. Using the example encoding bit-rate
`cited above of 500kbits' for R, data flows into the networ