ELSEVIER
`
`SECOND EDITION
`
`Praise for the first edition:
`For broadcasters and web developers
`involved in media delivery across the web,
`the book could be a very useful first step
`in understanding the basics of streaming
`technologies".
`—European Broadcasting Union
`vwvw.ebu.ch
`
`David Austerberry
`
`VOF
`
`VIDEO & AUDIO
`STREAMING
`
`A quick-Start guide to digital media—includes streaming
`to wireless devices and up-to-date technology information
`for streaming companies and products
`
`Focal
`Press
`
`Petitioners' Exhibit 1020
`Page 0001
`
`

`

`The Technology of
`Video and Audio
`Streaming
`Second Edition
`David Austerberry
`
`AMSTERDAM • BOSTON • HEIDELBERG • LONDON
`NEW YORK • OXFORD • PARIS • SAN DIEGO
`SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO
`Focal Press is an Imprint of Elsevier
`
`ELSEVIER
`
`0 Focal
`
`Press
`
`Petitioners' Exhibit 1020
`Page 0002
`
`

`

`Focal Press
`Is An imprint of Elsevier,
`200 Wheeler Road, Burlington, MA 01803, USA
`Linacre House, Jordan Hill, Oxford 0X2 8DP, UK
`Copyright © 2005, David Austerberry. All rights reserved.
`The right of David Austerberry to be identified as the author of this work has been
`asserted in accordance with the Copyright, Designs and Patents Act 1988
`No part of this publication may be reproduced in any material form (including
`photocopying or storing in any medium by electronic means and whether or not
`transiently or incidentally to some other use of this publication) without the written
`permission of the copyright holder except in accordance with the provisions of the
`Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by
`the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London, England
`W1T4LP. Applications for the copyright holder's written permission to reproduce
`any part of this publication should be addressed to the publisher.
`^ Recognizing the importance of preserving what has been written, Elsevier prints its
`^ books on acid-free paper whenever possible.
`Library of Congress Cataloglng-ln-Publlcation Data
`Austerberry, David.
`The technology of video and audio streaming / David Austerberry. - 2nd ed.
`p. cm.
`Includes bibliographical references and index.
`ISBN 0-240-80580-1
`1. Streaming technology (Telecommunications) 2. Digital video. 3. Sound -
`Recording and reproducing - Digital techniques.
`I. Title.
`TK5105.386 .A97 2004
`006.7'876 - dc22
`
`2004017485
`
`British Library Cataloguing-in-Publication Data
`A catalogue record for this book is available from the British Library.
`ISBN: 0240805801
`
`For information on all Focal Press publications visit our website at
`www.books.elsevier.com
`
`04 05 06 07 08 09 10 9 8 7 6 5 4 3 2 1
`
`Printed in the United States of America
`
`Petitioners' Exhibit 1020
`Page 0003
`
`

`

`Contents
`
`Preface
`Acknowledgments
`
`Section 1. Basics
`
`1 Introduction
`500 years of print development
`100 years of the moving image
`The Web meets television
`Convergence
`What is streaming?
`Applications
`How this book is organized
`Summary
`
`2 IP networks and telecommunications
`Introduction
`Network layers
`Telecommunications
`The local loop
`Summary
`
`3 The World Wide Web
`Introduction
`WWW
`Web graphics
`Proprietary tools
`Web servers
`Summary
`
`'
`
`ix
`xi
`
`1
`
`3
`3
`4
`5
`7
`7
`9
`10
`10
`
`13
`13
`14
`25
`30
`38
`
`40
`40
`42
`44
`48
`48
`51
`
`Petitioners' Exhibit 1020
`Page 0004
`
`

`

`VI
`
`Contents
`
`4 Video formats
`Introduction
`Scanning
`Color space conversion
`Digital component coding
`Videotape formats
`Time code
`Interconnection standards
`High definition
`Summary
`
`5 Video compression
`Introduction
`Compression basics
`Compression algorithms
`Discrete cosine transform
`Compression codecs
`MPEG compression
`Proprietary architectures
`Summary
`
`6 Audio compression
`Introduction
`Analog compression
`Digital audio
`The ear and psychoacoustics
`The human voice
`Lossy compression
`Codecs
`Codec standards
`Proprietary codecs
`Open-source codecs
`Summary
`
`Section 2. Streaming
`
`7 Introduction to streaming media
`Introduction
`What are the applications of streaming?
`The streaming architecture
`Bandwidth, bits, and bytes
`
`52
`52
`53
`56
`61
`65
`72
`74
`76
`77
`
`78
`78
`79
`80
`84
`87
`89
`98
`101
`
`102
`102
`103
`104
`110
`112
`114
`117
`118
`127
`128
`129
`
`131
`
`133
`133
`134
`138
`147
`
`Petitioners' Exhibit 1020
`Page 0005
`
`

`

`Contents
`
`Proprietary codec architectures
`Summary
`
`8 Video encoding
`Introduction
`Video capture
`Compression
`Encoding enhancements
`Encoding products
`Limits on file sizes
`Summary
`
`9 Audio encoding
`Introduction
`Audio formats
`Capture
`Encoding
`File formats
`Summary
`
`10 Preprocessing
`Introduction
`Video processing
`Audio
`Summary
`
`11 Stream serving
`Introduction
`Streaming
`Webcasting
`On-demand serving
`Inserting advertisements
`Playlists
`Logging and statistics
`Proprietary server architectures
`Server deployment
`Summary
`
`12 Live webcasting
`Introduction
`Planning a webcast
`Video capture
`
`VII
`
`149
`152
`
`154
`154
`159
`167
`170
`173
`175
`177
`
`179
`179
`181
`184
`186
`189
`192
`
`193
`193
`193
`200
`207
`
`209
`209
`211
`218
`222
`222
`224
`225
`227
`229
`232
`
`233
`233
`233
`237
`
`,
`
`Petitioners' Exhibit 1020
`Page 0006
`
`

`

`viii
`
`Graphics
`Audio capture
`Encoding
`Summary
`
`13 Media players
`Introduction
`Portals, players, and plug-ins
`Digital Rights Management
`Summary
`
`Section 3. Associated Tectinoiogies and Applications
`
`14 Rights management
`Introduction
`The value chain
`Digital Rights Management
`The rights management parties
`System integration
`Encryption
`Watermarking
`Security
`XrML
`Examples of DRM products
`MPEG-4
`Summary
`
`15 Content distribution
`Introduction
`Content delivery networks
`Corporate intranets
`Improving the QoS
`Satellite delivery
`Summary
`
`16 Applications for streaming media
`Introduction
`Summary
`
`Glossary
`Abbreviations
`Index
`
`Contents
`
`238
`238
`241
`243
`
`244
`244
`245
`256
`257
`
`259
`
`261
`261
`264
`265
`270
`274
`276
`277
`279
`280
`282
`286
`287
`
`289
`289
`291
`300
`304
`306
`307
`
`309
`309
`322
`
`327
`331
`335
`
`Petitioners' Exhibit 1020
`Page 0007
`
`

`

`rECHNOLOGY/Web, Multimedia
`
`VIDEO & AUDIO
`STREAMING
`
`SECOND EDITION
`
`David Austerberry
`Learn the end-to-end process, starting with
`capture from a video or audio source through to
`the consumer's media player
`A quick-start guide to streaming media technologies
`How to monetize content and protect revenue
`with digital rights management
`
`For broadcasters, web developers, and project managers
`implementing streaming media systems, David Austerberry
`shows how to deploy the technology on your site, from video
`and audio capture through to the consumer's media player.
`
`The book first deals with Internet basics and gives a
`thorough coverage of telecommunications networks and the
`last mile to the home. Video and audio formats are covered,
`as well as compression standards including Windows Media
`and MPEG-4. The book then guides you through the streaming
`process, showing in-depth how to encode audio and video.
`The deployment of media servers, live webcasting and how
`the stream is displayed by the consumer's media player are
`also covered.
`
`A final section on associated technologies illustrates how
`you can protect your revenue sources with digital rights
`management, looks at content delivery networks, and pro­
`vides examples of successful streaming applications.
`The supporting website www.davidausterberry.com/streamina.html
`offers updated links to sources of information, manufacturers
`and suppliers.
`
`0 Focal
`
`Press
`
`Focal Press
`An Imprint of Elsevier
`www.focalpress.com
`
`ELSEVIER
`
`If you X.
`enjoyed this
`book please post
`a review to your
`favorite online
`bookstore
`today. y'
`
`S.
`
`David Austerberry is
`co-owner of the new
`media communications
`consultancy. Informed
`Sauce. He has worked
`with streaming media
`since the late nineties.
`Before the move to
`streaming, he was a
`product manager for
`several broadcast equip­
`ment manufacturers, and
`can also count many years
`with a leading radio and
`television broadcaster.
`
`CONTENTS
`
`• Introduction
`• IP Networks and
`Telecommunications
`•The World Wide Web
`•Video Formats
`•Video Compression
`•Audio Compression
`
`• Introduction to
`Streaming Media
`•Video Encoding
`•Audio Encoding
`• Pre-Processing
`• Stream Serving
`• Live Webcasting
`• Media Players
`:liiiologies
`
`• Rights Management
`• Content Distribution!
`•Applications
`'
`
`Cover Imagery: © Getty Images
`
`Petitioners' Exhibit 1020
`Page 0008
`
`

`

`11 stream serving
`
`Introduction
`
`What happens once you successfully have encoded your multimedia content?
`Much like publishing a web page, the file is uploaded to the delivery server.
`That is where things diverge. A conventional web server simply downloads the
`media file. A streaming server has to manage the delivery rate of the stream to
`give real-time playback. In addition, the streaming server supports VCR-like
`control of the media clip.
`When a browser requests a web page, the files are delivered as fast as the
`network connection allows. TCP manages an error-free transmission by retrans­
`mitting lost packets, but the download time depends upon the intervening band­
`width available. TCP starts at a low rate then ramps up to the maximum that
`can be achieved. An accurate delivery is ensured, but timely delivery cannot be
`guaranteed. Streaming media has opposite requirements: the delivery must be
`in real-time, but reasonable levels of transmission errors can be accepted.
`Streaming servers can be proprietary to an architecture or designed to handle
`standard formats like MPEG-4. The system architecture can vary from a single
`machine serving a small corporate training site, to large distributed server farms,
`capable of serving hundreds of thousands of streams for live events like break­
`ing news footage, fashion shows, and rock concerts.
`Streaming can be delivered as a push or pull process. Push is used to stream
`live or prerecorded content as a webcast - this is the television model. Push
`streaming can be used for web channels or live events. Alternatively, the user
`can pull prerecorded content on-demand. This interactive experience is akin to
`using a CD-ROM or a web browser.
`A webcast can be a mix of live and prerecorded content. With live events the
`server is acting as a distribution point, just echoing the stream onto the viewers.
`For the prerecorded content the server has two functions. The first is to recall
`the content from the local disk storage arrays and the second is to control the
`stream delivery rate.
`
`Petitioners' Exhibit 1020
`Page 0009
`
`

`

`210
`
`The Technology of Video and Audio Streaming
`
`In the case of interactive content the client or player is requesting the files
`from the server. With the simuiated-live webcast, the server runs a piayiist,
`which streams fiies at the scheduied time to the player.
`
`Table 11.1 Web Server versus Streaming Server
`
`Web server
`
`Streaming server
`
`Advantages
`
`Part of existing infrastructure
`No additional expertise or training
`for IT staff
`
`Disadvantages
`
`None of the streaming server
`advantages
`Only supports progressive
`download
`
`Optimized media delivery
`Dynamic stream control
`Interactive media control
`Multicast support
`Improved server hardware utilization
`Supports live webcasting
`Additional equipment required
`
`source
`
`distribution
`
`serve
`
`deiivery
`
`stream
`
`encode
`
`Live event
`
`encode
`
`FTP
`
`Pre-recorded
`content
`
`Live
`
`Simulated
`live
`
`Interactive
`r
`
`WEBCAST
`
`server
`ON-DEMAND
`
`Figure 11.1 Webcasting and on-demand.
`
`Petitioners' Exhibit 1020
`Page 0010
`
`

`

`stream serving
`
`Streaming
`
`211
`
`What is a streaming server? The most-used server for the delivery of multime­
`dia content is the web server, typified by Apache. Web servers use HTTP over
`TCP/IP to deliver HTML pages and their associated image files.
`TCP/IP is used as the transport layer over the Internet. The files are down­
`loaded to the web browser cache as fast as the system allows. TCP incorpo­
`rates flow control to manage the download rate. There is no predetermined rate
`for delivery. TCP will increase the data rate until network packet loss indicates
`that the network is congested. At this point, the rate backs off. Another con­
`straint is the receive buffer. TCP uses a sliding window of data in transit. The
`receiver processes packets as they arrive. If data arrives too fast, the receive
`buffer will overflow. The receiver sends messages to the transmitter to slow
`down, to stop the buffer from filling.
`Suppose that you want to stream a stream encoded at 40 kbit/s. The TCP
`transmissions could start at 10 kbit/s. The transmitter then ramps up to 100
`kbit/s, where network congestion sets the upper limit. Suppose other users
`come on to the network, and the transmission throttles back to 30 kbit/s.
`At no time has the data rate matched the data rate at which the stream was
`encoded.
`Now consider if this clip lasts for 30 seconds, the complete file size is 150
`kbytes. This is downloaded to the browser cache - not a great problem.
`Now suppose we move up to a 20-minute presentation encoded at 300 kbit/s.
`Now the file size is 45 Mbytes - very large for the cache. This has been the
`way that the Flash player handled video files, but Flash was limited to short
`clips.
`When you stream content in real-time, the media packets are processed by
`the player as they arrive. There is no local caching, so the local storage issues
`are solved. This may not seem an issue to PC users, but many media players
`have very limited memory, for example set-top boxes and mobile devices. The
`problem with Flash also has gone, Macromedia now has developed the Flash
`player to support streaming of longform video, and the content is rendered then
`discarded.
`There is still the rate control problem. If the stream is encoded at 40 kbit/s it
`must be delivered at that rate for satisfactory viewing. One of the functions of
`the transport layer protocol is to regulate the stream rate. But what happens
`in the example where the network is congested and the best rate is 30 kbit/s?
`The player runs out of data and stops, one of the main complaints about
`streaming.
`There are ways around this, but the first is to encode at a rate below that
`which will suit the worst-case network conditions. That may be hard to predict,
`so there are more sophisticated ways; the usual is to encode at several rates,
`
`Petitioners' Exhibit 1020
`Page 0011
`
`

`

`212
`
`The Technology of Video and Audio Streaming
`
`then automatically select the optimum rate for the propagation conditions. This
`switching between different rate files is another task for the server.
`One of the great attractions of streaming is the interactivity. The user can nav­
`igate the clip with VCR controls. The server has to locate and serve the correct
`portions of the clip using an index.
`From these examples, it can be seen that the streaming server has several
`additional functions over a standard web server:
`
`• Real-time flow control
`• Intelligent stream switching
`• Interactive clip navigation
`
`HTTP does not support any of this functionality, so new protocols were devel­
`oped for streaming media. Under the auspices of the IETF several new proto­
`cols were developed for multimedia real-time file exchange: RTSP, RTP, and
`RTCP. There are also a number of proprietary protocols using similar principles.
`Windows Media originally used the Microsoft Media Server (MMS) for the deliv­
`ery framework (but now supports RTSP); the stream is in Advanced System
`Format (ASF).
`Real-Time Streaming Protocol (RTSP) is the framework that can be used for
`the interactive VCR-like control of the playback (Play, Pause, etc.). It is also
`used to retrieve the relevant media file from the disk storage array. RTSP also
`can be used to announce the availability of additional media streams in, for
`example, a live webcast. Real-Time Protocol (RTP) is used for the media data
`packets. The Real-Time Control Protocol (RTCP) provides feedback from the
`player to indicate the quality of the stream. It can report packet loss and out-
`of-order packets. The server can then react to congested network conditions
`by lowering the video frame rate or gear-shifting to a file encoded at a lower bit
`rate. The real-time media stream can be delivered by UDP or TCP over IP; the
`choice depends upon propagation conditions. The control protocols use TCP/IP
`for the bidirectional client-server connection.
`
`Streaming file formats
`To stream media files in real-time, they must be wrapped by one of the stream­
`ing formats. These formats have timing control information that can be used by
`the server to manage the flow rate. If the client is using interactive control, the
`file index aids the navigation.
`The main formats are MPEG-4 (mp4), the Microsoft advanced system
`format (.wmv and .wma extensions if created by Windows Media codecs,
`.asf if not), RealNetworks (.rm and .ra), and QuickTime hinted movies (.mov
`extension).
`
`Petitioners' Exhibit 1020
`Page 0012
`
`

`

`• •
`•
`
`server
`
`stream serving
`
`control
`& info
`
`V
`RTSP
`
`TCP
`
`IP
`1
`m
`1
`
`Figure 11.2 The streaming protocol stack.
`
`•
`•
`1 •
`
`213
`
`media player
`
`I-• I • • T
`
`control
`& info
`
`RTSP
`
`TCP
`
`IP
`
`control
`ctiannel
`
`network
`
`video
`& audio
`
`RIP
`
`UDP
`
`IP
`1
`
`video
`& audio
`
`RIP
`
`UDP
`
`IP
`
`media
`data
`ctiannel
`
`ASF file
`
`Header
`
`Object iData
`
`Object
`JL
`JL
`[fixed size packets]
`miiiwiiiuJ '
`
`Index
`Object(s)
`
`Figure 11.3 Typical streaming fiie format.
`
`Web server and streaming server
`If you already have a web site with web server capacity, and you stream only
`occasionally, it is possible to use that web server to deliver streaming files. The
`web server will use HTTP over TCP/IP. There will be no control of the stream
`delivery rate beyond buffer overflow in the TCP/IP stack.
`
`Petitioners' Exhibit 1020
`Page 0013
`
`

`

`214
`
`The Technology of Video and Audio Streaming
`
`Hint tracks
`The MPEG-4 team has adopted the QuickTime concept of hint tracks for
`control of the stream delivery. A streaming file is called a movie. The movie
`file container contains tracks, which could be video, audio, or other clip data.
`The track consists of control information that references the media data (or
`objects) that constitute the track. This means that several different movie
`files could reference the same video media object. This can be very useful for
`rich media presentations. One video file can be repurposed into several
`different presentations - maybe multiple languages or a number of levels of
`detail (introduction, overview, and in-depth). A movie is not the video and
`audio media files; it is the metadata or instructions for a specific presentation
`of the media data. The files are flattened into a single file when the stream is
`encoded.
`A streamable movie has a hint track in addition to the video and audio (MPEG-
`4 files are not limited streaming media applications). The hint track gives the
`server software pointers to the RTP information in order to serve the relevant
`media chunks. This information allows the server to deliver the correct video
`material in the sequence stipulated in the track file, and at the correct rate for
`the player display.
`
`movie
`(video) track
`
`Hinted
`movie
`
`(RTP hint) track
`
`RTP packet hint
`
`pointer
`sample
`
`header
`chunk
`
`RTP packet hint
`
`pointer
`sample
`
`header
`chunk
`
`RTP packet hint
`<D k-
`•a B
`c
`2 /
`/]}
`•C
`0)0.1
`« £
`
`frame
`
`frame
`
`frame
`
`media data
`
`RTP
`metadata
`
`media data
`
`Video
`file
`
`Figure 11.4 Typical streaming file format.
`
`Petitioners' Exhibit 1020
`Page 0014
`
`

`

`stream serving
`
`215
`
`Adapting to network congestion
`Both RealNetworks and Windows Media offer a way of changing the bit rate of
`a stream as network congestion varies. To get the best viewing experience we
`want to stream at the highest rate possible. But if the network slows down,
`rather than attempting to continue with a high bit rate, it makes sense to
`throttle back the bit rate. If the congestion eases then the bit rate can revert to
`a higher level. That way the viewer is not subject to stalling streams, just a
`graceful degradation in quality. These technologies work only with unicasting.
`The Real-Time Protocol maintains the correct delivery rate over UDP/IP (or
`TCP/IP if bandwidth permits). The RTSP framework supports the client
`interaction with the stream, the VCR-like controls Play, Pause, and so on. The
`streaming server application can use RTCP reports from the player to measure
`network congestion and switch stream rates for multiple bit rate media files.
`The player can report lost and delayed packets, and the reception of out-of-
`sequence packets.
`
`Media files
`low-
`bandwidth
`
`40k
`
`100k
`
`Ok
`
`500k
`
`high-
`bandwidth
`
`Figure 11.5 Sireaming control.
`
`©
`
`time
`reference
`
`
`media media
`
`fiie fiie
`
`parse parse
`
`RTF RTF
`
`
`packetizer packetizer
`
`media data
`
`I SETUP
`PUW
`1 PAUSE
`1 TEARDOWN
`1
`
`RTSP RTSP
`
`
`
`piayer commands piayer commands
`
`player reports
`
`select
`encoded
`bandwidth
`
`1
`1
`1
`,
`1
`
`'
`'
`'
`1
`I
`
`
`
`RTCP RTCP
`
`Petitioners' Exhibit 1020
`Page 0015
`
`

`

`216
`
`The Technology of Video and Audio Streaming
`
`RealNetworks SureStrecm
`SureStream allows different encoding rates to be combined into a single media
`file. Ttie streaming server will choose the appropriate data rate for the prevail­
`ing conditions by negotiating with the player. The lowest rate, called a duress
`stream, is a backstop, and will be streamed if congestion is very bad. When
`you are calculating the server disk space, a SureStream file takes the space of
`the sum of all the components. Because only the Helix Server can extract the
`correct component, a web server will serve the file in its entirety, including all
`the different rates.
`
`Windows Medio Intelligent Streaming
`This is a similar feature, allowing multiple constant bit rate streams to
`be encoded and wrapped in a single file. The streaming server will then
`stream the best rate for the current network conditions. Windows Media
`Encoder comes with predefined multiple bit rate profiles, but the profiles also
`can be customized to suit your special requirements. If you want to multicast
`a file that has been coded at multiple bit rates, only the highest rate will be
`transmitted.
`
`Drawbacks
`If you want to serve at different rates, then better results can be obtained by
`encoding a small picture at low frame rates for low bit rate streams, and a larger
`picture for streams at higher rates. So you may want a 480 x 360 pixel frame
`at 30 fps for a 1 Mbits/s stream and 160 x 120 at 6 fps for an ISDN link. Multi­
`ple bit rate encoding has to use the same picture size, so that the player can
`switch seamlessly between the different rates. The automatic rate-shifters have
`their uses, but are not a complete answer to serving the same content over
`very different networks. The big advantage is that the process of changing bit
`rate is invisible to the user.
`
`QuickTime and alternate movies
`This is a less sophisticated method of offering different bit rates. You can encode
`a movie as several separate movies encoded at different bit rates, and maybe
`with different language audio tracks. These are the alternates: The player
`follows a pointer to the master movie, which then references the alternate
`movies. The player negotiates with the server to request the correct alternate
`file for the player settings.
`
`Petitioners' Exhibit 1020
`Page 0016
`
`

`

`stream serving
`
`217
`
`MPEG-4 and scalable streams
`The MPEG team has proposed a different way to cope with variable network
`bandwidth. The server transmits a basic low-resolution stream. Additional helper
`streams can carry more detail. If the bandwidth is available then these extra
`streams allow a better quality picture to be assembled by the player.
`MPEG-4 also supports scalable encoding. This means that a basic player may
`decode only part of the stream to create the video, albeit at a lower quality than
`a more complex player, which can decode and display all the stream information.
`
`Loading content
`Whether you are using a managed service or doing your own serving, the first
`step is to deliver your content to the streaming servers. The encoding probably
`takes place near the video editing facility or, for a live webcast, at the venue.
`The servers have to be located close to an Internet backbone, unless you are
`streaming only over a local area. So in all probability the encoder and server
`are separated geographically. The simplest way to deliver the content is via a
`file transfer, using FTP. Some encoding systems have the ability to transfer a
`file automatically, immediately after the encoding has finished.
`
`Live streaming
`The file can, of course, be sent on a CD-ROM. If the content is a live broad­
`cast, then neither of these methods is suitable; it has to be streamed. This is
`covered in the chapter on live webcasts.
`The media encoder typically connects to the server using TCP for a bidirec­
`tional control link and a unidirectional media stream, using UDP. It is very impor­
`tant that the circuit used for this connection has more than sufficient bandwidth
`and a high QoS; that means low packet loss and timing jitter. Any data loss or
`corruption will be visible by all receivers of the webcast. This generally means
`using an uncontended circuit like T-1/E-1, rather than a domestic circuit like
`ADSL or a cable modem.
`
`media data (UDP)
`>
`
`players
`
`control (TCP)
`
`media
`encoder
`Figure 11.6 Encoder connections.
`
`origin
`server
`
`Petitioners' Exhibit 1020
`Page 0017
`
`

`

`218
`
`The Technology of Video and Audio Streaming
`
`Announcing the content
`The player usually locates streaming media by a hyperlink embedded in a web
`page. This link contains not only the URL for the content, but also the instruc­
`tions to start the media player.
`
`Web links
`The usual way to announce streaming media files is by a hyperlink in a web
`page. The link points to a small file on the web server. Windows Media calls
`this the stream redirector, or ASX. Real uses the RealAudio Metafile or Ram
`file. Once the browser receives this file, using the MIME type, the metafile is
`passed to a streaming plug-in. The metafile has the full address and filename
`for the streaming content. The media player than transmits a request to the
`specified streaming server for the media file. This may use MMS or RTSP for
`communication rather than the HTTP used with the web server. If all goes well
`the correct clip is streamed to the player - success.
`The metafiles can list several files to be played in sequence.
`
`SMIL
`If you are streaming rich media, then a number of different clips and images
`have to be synchronized for correct playback at the client. One way to do this
`is to use the Synchronized Multimedia Integration Language (SMIL) to write a
`control file.
`SMIL
`is supported by
`architectures.
`
`the QuickTime, Real, and Windows Media
`
`Webcasting
`
`Webcasting can be live, prerecorded, or a mix of both. A webcast at its
`simplest can be just a single clip. If you want to play out a sequence of clips
`you can set up a playlist. Even if you are streaming a presentation, you may
`want an introductory promotional film, and possibly some follow-up material
`after the main presentation. The playlist controls the server to play the relevant
`clips at the specified time.
`
`Splitting and relays
`A streaming server can handle several hundred simultaneous clients. The only
`real way to establish how many clients would be by conducting load tests. To
`
`Petitioners' Exhibit 1020
`Page 0018
`
`

`

`stream serving
`
`219
`
`HYPERLINK
`
`web page
`
`pointer http: //address/filename.ram
`link to metafile
`
`4 http
`
`http
`
`web server
`
`@
`
`starts player
`or plugin
`
`rtsp : / /address/clipname . rm
`pointer
`metafile or redirector
`
`player requests
`media clip
`
`streaming server
`
`SUCCESS!
`
`Figure 11.7 Unking to streaming media.
`
`serve to a large number of clients, extra servers have to be used. For on-
`demand serving, they can be added in parallel, but for live streams a different
`architecture is used to save network resources. A relay server splits the live
`streams downstream, so what starts as a singie live stream from the encoder
`fans out like the branches of a tree. Take the example of a CEO's webcast from
`the headquarters on the west coast to two other remote sites on the south and
`east coasts.
`
`Petitioners' Exhibit 1020
`Page 0019
`
`

`

`220
`
`The Technology of Video and Audio Streaming
`
`Server-side playlist
`
`Live encoder
`
`router
`
`Streaming
`server
`
`live content
`
`1
`
`Disk
`storage
`array
`
`pre-recorded m content
`
`Internet
`
`control
`LAN
`
`Stats & logging
`
`Figure 11.8 Live and Simula fed-live webcasting.
`
`If only a few clients were watching at the remote sites, the streams could be
`delivered directly from the server at headquarters. But each client will be using
`bandwidth on the trunk networks, all carrying the same data. If a large number
`of clients at each site are watching the webcast, the server will be unable to
`handle the load. The alternative is to use local servers at the remote sites to
`receive a single stream, and then locally distribute it. This saves bandwidth on
`the long-distance circuits and reduces the load on the originating server.
`You may be thinking that this is a really important broadcast, what happens
`if the link goes down? Companies like RealNetworks have come up with a
`number of solutions.
`RealNetwork's Helix Universal Server includes the facility to set up backup
`links. You set up redundant transmitters, usually two servers in parallel. If the
`link from the designated transmitter breaks, the receiving server automatically
`falls back to the next available transmitting server. The clients will see the
`webcast has stopped; if they refresh the link, the stream will continue using the
`backup path.
`
`Petitioners' Exhibit 1020
`Page 0020
`
`

`

`stream serving
`
`221
`
`Multicasting
`If you are webcasting live to a large audience, a regular unicast establishes
`a separate network connection to each client. This uses considerable network
`resources, even though each client is watching exactly the same material.
`Multicasting offers a solution by serving a single stream. This is then routed to
`all the clients. So the network routers are doing the distribution rather than the
`streaming servers.
`The drawback with multicasting is that you need to have control over the
`routers. First they have to be multicast enabled, which is not necessarily the
`case. There are different algorithms for setting up multicast routes: dense and
`sparse. If you are streaming over a corporate network within a single domain,
`multicasting has much to offer. It is when you want to multicast to the public
`Internet that problems can arise. The network of peer-to-peer connections that
`link a server to a client may not be multicast enabled, or the routers may not
`be set up to multicast across domains.
`If you are multicasting you cannot use automatic rate changing. The server
`transmits a single stream and is not aware of the clients, so it cannot nego­
`tiate to serve at a certain rate. To get around this you may have to transmit
`three or four different rate streams from different server ports. The player con­
`nects to an appropriate port for the bandwidth available at their local end.
`
`Multicast network
`If you are setting up a multicast to several sites, and want to use Virtual Private
`Networks (VPN) for the intermediate connections across the corporate WAN,
`note that IPSec does not support multicasting. The way around this is to unicast
`to a splitter server at the remote site, then multicast locally. You will need to
`make sure that clients can see only one multicast server. Since the same IP
`address is used for the multicast, the client potentially could receive several
`duplicate packets. The player will not decode the stream correctly in these
`circumstances.
`Announcing a multicast is different from retrieving on-demand content. With
`on-demand, the browser requests the media file. When the file is retrieved, the
`header carries information necessary for the player configuration. Once you join
`a multicast, there is no header. Instead, a small message file gives the browser/
`player the necessary information (like the port number to use). This message
`can use Session Description Protocol (SOP - RFC.2327); Microsoft uses the
`media station NSC file. The media station is analogous to a television station,
`so the station represents the channel and not the media streamed (programs
`transmitted) over the chann