I 1111111111111111 11111 1111111111 111111111111111 IIIII IIIII 111111111111111111
`US008677428B2
`
`c12) United States Patent
`Lewis et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,677,428 B2
`Mar.18,2014
`
`(54) SYSTEM AND METHOD FOR RULE BASED
`DYNAMIC SERVER SIDE STREAMING
`MANIFEST FILES
`
`(75)
`
`Inventors: Jason Lewis, Issaquah, WA (US);
`William Watts, Seattle, WA (US); Peter
`Friedman, Kenmore, WA (US)
`
`(73) Assignee: Disney Enterprises, Inc., Burbank, CA
`(US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 369 days.
`
`(21) Appl. No.: 12/806,750
`
`(22) Filed:
`
`Aug. 20, 2010
`
`(65)
`
`Prior Publication Data
`
`US 2012/0047542Al
`
`Feb. 23, 2012
`
`(51)
`
`(2011.01)
`
`Int. Cl.
`H04N 71173
`(52) U.S. Cl.
`USPC .................................. 725/91; 725/36; 725/97
`(58) Field of Classification Search
`USPC ................................................ 725/91, 97, 36
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,892,915 A * 4/1999 Duso et al ..................... 709/219
`6,978,306 B2 * 12/2005 Miller et al. . ................. 709/226
`2005/0262245 Al* 11/2005 Menon et al .................. 709/226
`
`2007/0157231 Al*
`2008/0244669 Al *
`2009/0193473 Al*
`2009/0204901 Al*
`2009/0254952 Al *
`2010/0050083 Al*
`
`7/2007
`10/2008
`7/2009
`8/2009
`10/2009
`2/2010
`
`... ... ... .. . . . . . 725/35
`Eldering et al.
`Miner ........................... 725/109
`Moon et al ...................... 725/81
`Dharmaji et al. . ............ 715/745
`Sridhar et al. . ................. 725/92
`Axen et al. .. ... .... ... ... . . . . . 715/726
`
`* cited by examiner
`
`Primary Examiner Pankaj Kumar
`Assistant Examiner - Sahar Baig
`(74) Attorney, Agent, or Firm -Farjami & Farjami LLP
`
`(57)
`
`ABSTRACT
`
`There is provided a system and method for rule-based
`dynamic server-side streaming manifest files. There is pro(cid:173)
`vided a method comprising receiving a request to provide a
`first video content for playback, evaluating a plurality of rules
`for the first video content, generating a dynamic manifest file
`referencing the first video content, and providing the dynamic
`manifest file in response to the request, thereby enabling an
`application to playback the first video content for output on a
`display by interpreting the dynamic manifest file. The rules
`may implement various features such as dynamic advertise(cid:173)
`ment insertion, load balancing, client customization, user and
`device targeting, enhanced security mechanisms, global
`announcements, and others. As streaming protocols are
`widely supported by default on many client platforms, the
`dynamic manifest files can be utilized in a user friendly and
`transparent manner compared to client-side solutions requir(cid:173)
`ing cumbersome client software installations.
`
`18 Claims, 4 Drawing Sheets
`
`Producer 180
`
`100
`
`/
`
`Live Video
`Encoder .1ZQ
`
`Rule Resolution
`SeNer.1.Zll
`
`I Processor 121 I
`
`Dynamic Manifest
`File Server 11ll
`I Processor 111 I
`
`Display 1J!l!
`
`Client Device 150.
`Processor ID
`
`Media Player Application
`1li!l
`
`Usar185
`
`Google Exhibit 1011
`Google v. Ericsson
`
`

`

`U.S. Patent
`
`Mar.18,2014
`
`Sheet 1 of 4
`
`US 8,677,428 B2
`
`Fig. 1
`
`Producer 180
`
`Camera Rig 185
`
`100
`
`/
`
`Live Video
`Encoder170
`
`Live Video
`Segments 175
`
`Rule Resolution
`Server 120
`
`I Processor ill I
`
`Dynamic Manifest
`File Server 110
`
`I Processor ill I
`
`Display 160
`
`Client Device 150
`
`Processor 151
`
`Memory 155
`
`Media Player Application
`156
`
`Manifest
`~ File 157
`
`Advertisement
`Video Storage 140
`
`Ad Video
`Segments 145
`
`User185
`
`

`

`U.S. Patent
`
`Mar.18,2014
`
`Sheet 2 of 4
`
`US 8,677,428 B2
`
`Fig. 2
`
`200
`
`/
`
`Live Video Segments 275
`
`Segment
`276a
`
`Segment
`276b
`
`Segment
`276c
`
`Segment
`276d
`
`Segment
`276e
`
`8_01.ts
`10 sec.
`
`•
`
`S_02.ts
`10 sec.
`
`S_03.ts
`10 sec.
`
`S 04.ts
`10 sec .
`
`S_05.ts
`10 sec.
`
`•
`
`Dynamic Manifest
`._
`File Server 210
`I Processor 211 I
`◄ ------------------- -------------
`
`-
`
`Processed Video Segments 215
`
`Segment 216b
`
`Segment 216e
`
`S 02-cut.ts
`3 sec.
`
`•
`
`S OS-cut.ts
`7 sec.
`
`Ad Video Segments 245
`
`Segment
`246a
`
`Segment
`246b
`
`Segment
`246c
`
`A_01 .ts
`10 sec.
`
`A_02.ts
`10 sec.
`
`;
`;
`l Manifest File 257 l
`
`A_03.ts
`10 sec.
`
`•
`
`Entry
`258a
`
`8_01.ts
`
`Entry
`258b
`
`8_02
`-cut.ts
`
`Entry
`258c
`
`Entry
`258d
`
`Entry
`258e
`
`A_01 .ts
`
`A_02.ts
`
`A_03.ts
`
`'
`
`En ry
`258f
`
`S_05
`-cut.ts
`
`10 sec.
`
`3 sec.
`
`10 sec.
`
`10 sec.
`
`10 sec.
`
`7 sec.
`
`

`

`U.S. Patent
`
`Mar.18,2014
`
`Sheet 3 of 4
`
`US 8,677,428 B2
`
`Fig. 3
`
`300
`
`/
`
`Rule Resolution Server 320
`
`I
`
`+
`I Platform Rule Set 322a
`
`j~
`
`Processor 321
`
`+
`I Resolution Rule Set 322b
`
`I Stored Video Segments 375
`i
`
`Dynamic Manifest File Server 310
`
`I
`
`i
`I Manifest File 357a I
`
`Processor 311
`
`i
`I Manifest File 357b I
`
`~
`
`i
`
`Manifest File 357c
`
`' 1
`
`Processed Video
`Segments 315a
`I
`
`'
`
`Processed Video
`Segments 315b
`I
`
`t
`
`Network 330
`
`--::, ~
`
`~
`
`'
`Processed Video ..-
`Segments 315c
`i
`~ .r Content Delivery
`•
`Network 335
`-
`~ + ,~
`
`Client Device 350a
`
`Client Device 350b
`
`Client Device 350c
`
`Flash Player
`Plugin 356a
`
`+
`
`Display 360a
`(800 X 480)
`
`HLS Client 356b
`
`+
`
`Display 360b
`(960 X 640)
`
`Native Binary
`Application 356c
`
`+
`
`Display 360c
`(1280 X 720)
`
`

`

`U.S. Patent
`
`Mar.18,2014
`
`Sheet 4 of 4
`
`US 8,677,428 B2
`
`Fig. 4
`
`400
`
`/
`
`Receive, from an application executing on a
`first client device, a request to provide a first _,--- 410
`video content for playback
`
`'.
`
`Evaluate a plurality of rules for the first video
`content
`
`_,--- 420
`
`~r
`
`Generate a dynamic manifest file referencing
`a plurality of content video segments
`corresponding to the first video content
`
`V---430
`
`w
`
`Providing the dynamic manifest file to the
`application in response to the request,
`thereby enabling the application to playback
`the first video content for output on a display
`by interpreting the dynamic manifest file
`
`....,,,,.---- 440
`
`

`

`US 8,677,428 B2
`
`1
`SYSTEM AND METHOD FOR RULE BASED
`DYNAMIC SERVER SIDE STREAMING
`MANIFEST FILES
`
`BACKGROUND OF THE INVENTION
`
`2
`shown in and/or described in connection with at least one of
`the figures, as set forth more completely in the claims.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`10
`
`The features and advantages of the present invention will
`become more readily apparent to those ordinarily skilled in
`the art after reviewing the following detailed description and
`accompanying drawings, wherein:
`FIG. 1 presents a diagram of a system for providing rule(cid:173)
`based dynamic server-side streaming manifest files, accord(cid:173)
`ing to one embodiment of the present invention;
`FIG. 2 presents a diagram of a system for using rule-based
`dynamic server-side streaming manifest files to implement
`15 dynamic in-stream advertisements, according to one embodi(cid:173)
`ment of the present invention;
`FIG. 3 presents a diagram of a system for using rule-based
`dynamic server-side streaming manifest files to implement
`stream targeting for client devices, according to one embodi-
`20 ment of the present invention; and
`FIG. 4 shows a flowchart describing the steps, according to
`one embodiment of the present invention, by which rule(cid:173)
`based dynamic server-side streaming manifest files may be
`provided.
`
`1. Field of the Invention
`The present invention relates generally to media playback.
`More particularly, the present invention relates to media play(cid:173)
`back using dynamic manifest files.
`2. Background Art
`Streaming platforms based on widely supported protocols
`such as Hypertext Transfer Protocol (HTTP), Internet Proto-
`col (IP) multicast and peer to peer (P2P) streaming allow
`content producers to continue harnessing standard web deliv(cid:173)
`ery technologies for streamlined implementation using exist(cid:173)
`ing infrastructure, avoiding the need to develop and imple(cid:173)
`ment new data streaming protocols. As a result, such
`streaming platforms are seeing widespread adoption, with
`supporting player applications built for a wide range of oper(cid:173)
`ating systems and devices. By utilizing applications based on
`widely supported streaming protocols, users can enjoy live or
`recorded video content streamed conveniently to their favor-
`ite media consumption devices, whether it be a laptop or
`desktop computer, a mobile phone, a video game console, a 25
`digital video recorder, a set top box, or another network
`enabled media device.
`Conventionally, many streaming platforms rely on a mani(cid:173)
`fest file retrieved as a static play list file from a web host. Thus,
`it is difficult to provide dynamic content where the contents of
`the manifest file may change. Moreover, the manifest file
`must be strictly sequential and provides no mechanisms for
`conditional logic or implementing business rules. This lack of
`flexibility makes it difficult to provide desirable features such
`as dynamic advertisement insertion, load balancing, client
`customization, user and device targeting, enhanced security
`mechanisms, and other features to more flexibly meet the
`needs of advertisers and content producers. For example,
`advertisers may want to use demographic information and
`user profile information to provide targeted advertising cus(cid:173)
`tomized for each viewer, which is not possible if streaming
`media playback is only by a static and sequential manifest
`file.
`Client side dynamic media streaming player applications
`may be provided to implement conditional logic and business 45
`rules. However, one of the most compelling features of
`streaming media is the capability for users to stream from
`many different media devices due to the widespread support
`of streaming protocols. As a result, a dynamic client side
`solution may necessitate significant development and main(cid:173)
`tenance costs to provide customized streaming applications
`for each major client platform, preventing cost effective
`implementation and deployment. Moreover, users may be
`required to search, download, and update dynamic media
`streaming applications for each individual device. This pro(cid:173)
`cess may be inconvenient for many users, and the installation
`or updating of applications may be restricted in public or
`corporate environments.
`Accordingly, there is a need to overcome the drawbacks
`and deficiencies in the art by providing flexible dynamic 60
`streaming media in a cost effective manner that is convenient
`and transparent for end users.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`The present application is directed to a system and method
`for rule-based dynamic server-side streaming manifest files.
`30 The following description contains specific information per(cid:173)
`taining to the implementation of the present invention. One
`skilled in the art will recognize that the present invention may
`be implemented in a manner different from that specifically
`discussed in the present application. Moreover, some of the
`35 specific details of the invention are not discussed in order not
`to obscure the invention. The specific details not described in
`the present application are within the knowledge of a person
`of ordinary skill in the art. The drawings in the present appli(cid:173)
`cation and their accompanying detailed description are
`40 directed to merely exemplary embodiments of the invention.
`To maintain brevity, other embodiments of the invention,
`which use the principles of the present invention, are not
`specifically described in the present application and are not
`specifically illustrated by the present drawings.
`FIG. 1 presents a diagram of a system for providing rule-
`based dynamic server-side streaming manifest files, accord(cid:173)
`ing to one embodiment of the present invention. Diagram 100
`of FIG. 1 includes producer 180, camera rig 185, live video
`encoder 170, rule resolution server 120, dynamic manifest
`50 file server 110, content delivery network 135, display 160,
`network 130, advertisement video storage 140, client device
`150, and user 185. Live video encoder 170 includes live video
`segments 175. Rule resolution server 120 includes processor
`121. Dynamic manifest file server 110 includes processor
`55 111. Advertisement video storage 140 includes ad video seg(cid:173)
`ments 145. Client device 150 includes processor 151 and
`memory 155. Memory 155 includes media player application
`156 and manifest file 157.
`As shown in diagram 100 of FIG. 1, producer 180 may
`direct camera rig 185 to record live footage, such as a live
`event, to be streamed to users such as user 185. Camera rig
`185 may then record and send live video footage to live video
`encoder 170, which may then encode the live video footage in
`real-time into live video segments 175. For example, the live
`65 video footage may be encoded into MPEG transport stream
`(TS) fragment files of a fixed length, such as 10 seconds, for
`ease of distribution using streaming protocols such as HTTP
`
`SUMMARY OF THE INVENTION
`
`There are provided systems and methods for rule-based
`dynamic server-side streaming manifest files, substantially as
`
`

`

`US 8,677,428 B2
`
`3
`streaming, IP multicast, P2P streaming. As new live footage is
`captured and approved, new video segments may be encoded
`and stored within live video segments 175, which may then be
`distributed over content delivery network 135. Content deliv-
`ery network 135 may, for example, comprise a network of
`servers for storage and distribution of media content. Live
`video encoder 170 may also provide rule resolution server
`120 with an updated list of references for accessing live video
`segments 175 through content delivery network 135, which
`may then be passed to dynamic manifest file server 110 for
`generating manifest file 157. Producer 180 may provide spe(cid:173)
`cific rules to rule resolution server 120 to customize the
`generation of manifest file 157, such a rule to insert advertis-
`ing segments to overwrite a particular time block in the live
`stream.
`As shown in diagram 100, a client device 150 may send a
`request to dynamic manifest file server 110 for live video
`content. Client device 150 may comprise a laptop or desktop
`computer, a mobile phone, a video game console, a digital
`video recorder, a set top box, or another network enabled 20
`media device. User 185 may direct client device 150 to
`execute media player application 156, for example by click-
`ing on an application icon shown on display 160. Media
`player application 156 may comprise, for example, a web
`browser. User 185 may navigate to a video streaming portal
`site accessible over network 130 to click on a link directed to
`dynamic manifest file server 110 for access to live video
`stream content. Media player application 156 may then send
`a request, such as a HTTP GET request over network 130 to
`dynamic manifest file server 110. Dynamic manifest file
`server 110 may then forward various parameters received
`from the request originating from client device 150 to rule
`resolution server 120. Client device 150 may also explicitly
`send parameter data to dynamic manifest file server 110 vol(cid:173)
`untarily or in response to a request for client parameters.
`Dynamic manifest file server 110 may then utilize rule reso(cid:173)
`lution server 120 to evaluate various business rules and create
`a dynamically tailored manifest file accordingly, which may
`then be passed back to client device 150 over network 130 and
`placed into memory 155 as manifest file 157, as shown in 40
`FIG. 1.
`Media player application 156 may then interpret manifest
`file 157 to playback video content on display 160. For
`example, manifest file 157 may reference live video segments
`175 and ad video segments 145 on servers hosted in content
`delivery network 135, accessible over network 130. Thus,
`manifest file 157 may comprise a play list file such as a M3U8
`play list file. Media player application 156 may then request,
`stream, decode, and output the referenced video segments
`seamlessly to display 160 to playback the requested live video 50
`stream for user 185. As camera rig 185 captures new live
`footage and live video encoder 170 adds new segments to live
`video segments 175, media player application 156 can peri(cid:173)
`odically request an updated manifest file 157 from dynamic
`manifest file server 110. Producer 180 may dynamically add
`new rules to rule resolution server 120, for example to sched(cid:173)
`ule advertising blocks. Rule resolution server 120 may then
`direct dynamic manifest file server 110 to switch to advertis-
`ing content at specific time blocks specified by the new rules
`added by producer 180.
`While the example system shown in diagram 100 of FIG. 1
`is directed towards a live content streaming embodiment with
`advertisement insertion, the rule based dynamic server-side
`streaming manifest files provided by dynamic manifest file
`server 110 may also be utilized for alternative embodiments
`and use cases. For example, pre-recorded content may be
`provided rather than live content. If the pre-recorded content
`
`4
`has existing advertising blocks, then rules may be provided at
`rule resolution server 120 to replace the existing advertising
`blocks, facilitating the reuse of existing video segments with(cid:173)
`out re-encoding. Rules evaluated by rule resolution server
`120 may also be directed towards other functions besides
`advertisement insertion, such as enhanced security, video
`content customization, client or user targeting, geographic or
`region targeting, content delivery network load balancing,
`priority broadcasts for concurrent playback, and other func-
`10 tions. Moreover, diagram 100 only depicts one simplified
`exemplary implementation for clarity. Alternative embodi(cid:173)
`ments may, for example, combine, separate, or duplicate cer(cid:173)
`tain components and functions shown in diagram 100. For
`example, dynamic manifest file server 110 and rule resolution
`15 server 120 may be combined into a single server, multiple
`servers may be utilized for load balancing to multiple client
`devices, multiple content delivery networks may be utilized
`for higher quality of service, and several different video
`streams may be offered for streaming.
`Moving to FIG. 2, FIG. 2 presents a diagram ofa system for
`using rule-based dynamic server-side streaming manifest
`files
`to
`implement dynamic
`in-stream advertisements,
`according to one embodiment of the present invention. Dia(cid:173)
`gram 200 of FIG. 2 includes live video segments 275, pro-
`25 cessed video segments 215, dynamic manifest file server 210,
`ad video segments 245, and manifest file 257. Live video
`segments 275 include segments 275a through 275e. Pro(cid:173)
`cessed video segments 215 include segments 216b and 216e.
`Dynamic manifest file server 210 includes processor 211. Ad
`30 video segments 245 include segments 246a through 246c.
`Manifest file 257 includes entries 258a through 258/ With
`regards to FIG. 2, it should be noted that live video segments
`275 may correspond to live video segments 175 from FIG. 1,
`that dynamic manifest file server 210 may correspond to
`35 dynamic manifest file server 110 from FIG. 1, that ad video
`segments 245 may correspond to ad video segments 145 from
`FIG. 1, and that manifest file 257 may correspond to manifest
`file 157 from FIG. 1.
`As previously discussed, media files may be encoded and
`segmented into fragment files of a fixed length to facilitate
`integration with streaming platforms. Thus, as shown in FIG.
`2, live video segments 275 may be prepared as successive ten
`second segments, shown as segments 276a through 276e.
`Similarly, ad video segments 245 are also prepared as three
`45 ten second segments, or segments 246a through 246c, which
`may comprise one complete thirty second commercial. As
`additional live footage is recorded and encoded into new
`segments, the new segments may be appended within live
`video segments 275.
`Dynamic manifest file server 210 may then dynamically
`create manifest file 257 referencing live video segments 275
`and ad video segments 245. For example, as previously
`described, producer 180 may provide a rule to rule resolution
`server 120 specifying a time block to switch to a commercial,
`55 such as thirteen (13) seconds into the live broadcast. How(cid:173)
`ever, since live video segments 275 are fragmented into ten
`second segments, an insertion of advertisement content may
`only be supported at ten second intervals since many standard
`manifest file formats do not support offset playback of seg-
`60 ments. Thus, if commercial playback is desired at an offset of
`thirteen seconds, insertion must be delayed until the twenty
`second offset, resulting in dead time while waiting for the
`segment to finish.
`To avoid such dead time, dynamic manifest file server 210
`65 may be configured to generate processed video segments 215,
`which may be preferably directly stream copied from the
`original sources and trimmed accordingly, or alternatively
`
`

`

`US 8,677,428 B2
`
`5
`trimmed and re-encoded if direct stream copies are not fea(cid:173)
`sible. Assuming a rule specifying commercial insertion at
`thirteen seconds for ad video segments 245 running at thirty
`seconds, segments 216b and 216e may thus be created from
`segments 276b and 276e, respectively. More specifically, seg(cid:173)
`ment 216b may comprise the first three seconds of segment
`276b, and segment 216e may comprise the last seven seconds
`of segment 276e. Processed video segments 215 may then be
`referenced accordingly instead of the original live video seg(cid:173)
`ments 275, allowing advertisement or other content insertion
`at any desired playback offset or position without dead time.
`For pre-recorded content where it is desirable not to discard
`any of the original video content, dynamic manifest file server
`210 may be configured to continue referencing the original
`video segments even after insertion of advertising.
`Thus, dynamic manifest file server 210 may generate mani(cid:173)
`fest file 257, with entry 258a referencing original segment
`276a, entry 258b referencing processed segment 216b,
`entries 258c through 258e referencing segments 246a
`through 246c respectively, and entry 258/ referencing pro(cid:173)
`cessed segment 216e. When entries 258a through 258/are
`played back successively without gaps by a media player
`application, the result is an advertisement smoothly inte(cid:173)
`grated into a live stream with no interruptions or dead time.
`Since additional rules may be specified, for example to pro(cid:173)
`vide targeted advertising based on user tracking profiles, cli(cid:173)
`ent device profiles, geographic regions, and other parameters,
`highly targeted advertising is possible even in live streaming
`embodiments.
`Moving to FIG. 3, FIG. 3 presents a diagram ofa system for
`using rule-based dynamic server-side streaming manifest
`files to implement stream targeting for client devices, accord-
`ing to one embodiment of the present invention. Diagram 300
`of FIG. 3 includes rule resolution server 320, stored video
`segments 375, dynamic manifest file server 310, processed
`video segments 315a through 315c, content delivery network
`335, network 330, client devices 350a through 350c, and
`displays 360a through 360c. Rule resolution server 320
`includes processor 321, platform rule set 322a, and resolution
`rule set 322b. Dynamic manifest file server 310 includes
`processor 311 and manifest files 357a through 357c. Client
`device 350a includes Flash player plugin 356a. Client device
`350b includes HTTP Live Streaming or HLS client 356b.
`Client device 350c includes native binary application 356c.
`With respect to FIG. 3, it should be noted that rule resolution
`server 320 may correspond to rule resolution server 120 of
`FIG. 1, that dynamic manifest file server 310 may correspond
`to dynamic manifest file server 110 from FIG. 1, that pro(cid:173)
`cessed video segments 315a through 315c may each corre(cid:173)
`spond to processed video segments 215 of FIG. 2, that content
`delivery network 335 may correspond to content delivery
`network 135 of FIG. 1, that network 330 may correspond to
`network 130 of FIG. 1, that client devices 350a through 350c
`may each correspond to client device 150 of FIG. 1, and that
`displays 360a through 360c may each correspond to display 55
`160 of FIG. 1.
`While FIGS. 1 and 2 illustrate an advertisement insertion
`embodiment, FIG. 3 illustrates client or device targeting,
`wherein video content is processed and customized according
`to particular client device parameters. As shown in FIG. 3,
`dynamic manifest file server 310 provides manifest files for a
`diverse range of client device platforms, including Flash
`Player plugin 356a at client device 350a, HTTP Live Stream(cid:173)
`ing client 356b at client device 350b, and native binary appli(cid:173)
`cation 356c at client device 356c. Platform rule set 322a may
`include various rules as how to customize video content based
`on the target device platform to be supported. Additionally,
`
`6
`displays 360a, 360b, and 360c each utilize different screen
`resolutions to display video content, and resolution rule set
`322b may include various rules as how to resize video content
`based on the target display resolution.
`For example, for client device 350a, platform rule set 322a
`may dictate that if a request originates from a client device
`indicating Flash Player plug-in support, then dynamic mani(cid:173)
`fest file server 310 should preferably generate a F4M Flash
`Zeri manifest file, or manifest file 357a, referencing F4F
`10 Flash video files, or processed video segments 315a. Pro(cid:173)
`cessed video segments 315a may be encoded from stored
`video segments 375, and may be resized and cropped accord(cid:173)
`ing to resolution rule set 322b to optimally fit the 800 by 480
`resolution provided by display 360a. Thus, when client
`15 device 350a requests video content represented by stored
`video segments 375, dynamic manifest file server 310 may
`provide manifest file 357a in the Flash Zeri HTTP streaming
`format for facilitated playback by Flash Player plugin 356a.
`Flash Player plugin 356a may then access the referenced
`20 video content in processed video segments 315a over net(cid:173)
`work 330 via content delivery network 335.
`Moving to client device 350b, platform rule set 322a may
`dictate that if a request originates from a client device indi(cid:173)
`cating HLS or HTTP Live Streaming support, then dynamic
`25 manifest file server 310 should preferably generate a M3U8
`manifest file, or manifest file 357 b, referencing MPEG trans(cid:173)
`port stream video files, or processed video segments 315b.
`Processed video segments 315b may be encoded from stored
`video segments 375, and may be resized and cropped accord-
`30 ing to resolution rule set 322b to optimally fit the 960 by 640
`resolution provided by display 360b. Thus, when client
`device 350b requests video content represented by stored
`video segments 375, dynamic manifest file server 310 may
`provide manifest file 357b in the HTTP Live Streaming for-
`35 mat for facilitated playback by HLS client 356b. HLS client
`356b may then access the referenced video content in pro(cid:173)
`cessed video segments 315b over network 330 via content
`delivery network 335.
`Moving to client device 350c, platform rule set 322a may
`40 dictate that if a request originates from a set top box running
`a native binary application, then dynamic manifest file server
`310 should preferably generate a M3U8 manifest file, or
`manifest file 357 c, referencing MPEG transport stream video
`files, or processed video segments 315c. Processed video
`45 segments 315c may be encoded from stored video segments
`375, and may be resized and cropped to optimally fit the 1280
`by 720 resolution provided by display 360c. Thus, when
`client device 350c requests video content represented by
`stored video segments 375, dynamic manifest file server 310
`50 may provide manifest file 357c as a standard M3U8 playlist
`for facilitated playback by native binary application 356c.
`Native binary application 356c may then access the refer(cid:173)
`enced video content in processed video segments 315c over
`network 330 via content delivery network 335.
`Thus, dynamic manifest file 310 and rule resolution server
`320 may target specific client platforms to provide the opti(cid:173)
`mal format for the manifest files and the processed video
`segments, and may also resize and process video for the best
`appearance on the specific display for each client device.
`60 While display resolution is shown as one example display
`parameter, other display parameters may also be considered
`such as color space or gamut, color bit-depth, refresh rate, and
`other parameters. Thus, as one example, rule resolution server
`320 may include a color space conversion rule to adjust video
`65 colors to the color space of a targeted display. Another
`example rule may comprise a framerate conversion rule con(cid:173)
`verting video framerates to adjust to specific refresh rates of a
`
`

`

`US 8,677,428 B2
`
`7
`targeted display, or downsampling three-dimensional stereo(cid:173)
`scopic video intended for three-dimensional displays into
`two-dimensional video for standard two-dimensional dis(cid:173)
`plays. To provide faster response time for client devices,
`video for the most common client configurations may be
`pre-processed and cached in advance. Additionally, multiple
`bit-rate streams may be encoded and referenced in the gen(cid:173)
`erated manifest files to allow graceful degradation to lower
`bit-rate video in response to adverse network conditions.
`In addition to preparing video content for optimal presen(cid:173)
`tation quality, substantive content such as advertising content
`may also be targeted and prepared based on various param(cid:173)
`eterized rules evaluated by rule resolution server 320. Thus,
`for example, a user tracking profile associated with client
`device 350a may be analyzed to formulate a targeted adver(cid:173)
`tising campaign for manifest file 357a, adding appropriate
`pre-roll, post-roll, and mid-roll advertising content estimated
`to be most relevant and interesting for the user of client device
`350a. The geographic location or region of client device 350a
`may also be detected, for example through GPS tracking or
`geo-IP address look-up, and advertising may be adjusted
`accordingly for the detected region, for example by selecting
`from a regional advertising campaign. The detected region
`may also affect the requested substantive content, for
`example by configuring processed video segments 315a to 25
`include subtitles or a dubbed language track based on the
`detected region.
`Besides advertisement insertion and user or client device
`targeting, rule resolution server 320 may also implement a
`wide variety of other rules to enhance, target, and customize 30
`the video streaming experience for the end user. For example,
`one rule may rewrite the URLs within a manifest file to point
`to the content delivery network in closest proximity to the
`client device, providing improved network performance and
`responsiveness. The proximity rule may be overridden or 35
`supplemented by another rule implementing client load bal(cid:173)
`ancing parameters, for example by steering users to content
`delivery networks with greater free user capacity. Another
`rule may implement enhanced security features, such as lim(cid:173)
`iting or granting access or providing keys based on time 40
`windows or rental periods, HTTP cookie or login status,
`client device identification, or other criteria. For example, if
`the enhanced security checks fail, then the client may be
`redirected to a video showing how the user can remedy any
`failed security checks.
`Other rules may provide features desirable for content
`creators and network administrators. For example, one rule
`may insert a high priority video segment into multiple mani(cid:173)
`fest files for a specific group of clients or for all clients
`globally. Thus, for example, administrators can provide 50
`important news bulletins, alerts, or other high priority video
`to the displays of multiple clients concurrently. Content pro(cid:173)
`viders may also use such global notifications to advertise or
`promote specific content to all streaming clients regardless of
`the particular stream each client is watching. Since such glo- 55
`bal notifications may be configured to display concurrently
`on many different clients, vira