as) United States
`a2) Patent Application Publication co) Pub. No.: US 2012/0254456 Al
`Visharamet al.
`(43) Pub. Date:
`Oct. 4, 2012
`
`
`US 20120254456A1
`
`(54) MEDIA FILE STORAGE FORMAT AND
`ADAPTIVE DELIVERY SYSTEM
`
`(22)
`
`Filed:
`
`Mar.31, 2011
`
`(75)
`
`Inventors:
`
`Zubair Visharam, Santa Clara, CA
`(US); Sunil Mukundan, Chennai
`(IN); Karthik Narayanan, Chennai
`US Kepnar Narayanan,San
`IN); Jaspel Kohli, Sunnyvale, CA
`
`Jose, CA (US); Prabakar
`Sundarrajan, Saratoga, CA (US)
`
`(73) Assignee:
`
`Juniper Networks,Inc.,
`Sunnyvale, CA (US)
`
`(21) Appl. No.:
`
`13/077,831
`
`Publication Classification
`
`(51)
`Int. Cl.
`GO6F 13/16
`(2006.01)
`(52) USe CM. ccccccssssssssssessssssessssevsessssesesssusenenses 709/231
`
`(57)
`
`ABSTRACT
`
`A method and apparatus for creating universal adaptive bit
`rate streams using a generic container format to store audio,
`video, and supplementaldata that allows seamless trans-con-
`tainerization from one adaptive streaming format to another.
`
`
`
`
`client
`client
`
`
`
`101b
`
`10la
`
`client
`
`101n
`
`
`102 video
`
`central
`
`
`management
`
`station 106
`
`network
`104
`
`
`origin
`server
`
`switch
`appliance
`105
`
`video
`
`switch
`appliance
`103
`
`1
`
`APPLE 1008
`
`APPLE 1008
`
`1
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 1 of 34
`
`US 2012/0254456 Al
`
`1‘DIA
`
`OapIA
`
`YouIMs
`
`sourrdde
`
`£01
`
`OdpIA
`
`YOuIMs
`
`sourrlidde
`
`Sol
`
`quay
`
`UTOL
`
`
`
`quatyTo
`
`q101
`
`quota
`
`eT0l
`
`
`
`UISTLIO
`
`IOAIOS
`
`COT
`
`yIomyou
`
`POT
`
`ye-uas
`
`
`
`JUDUIASeULUI
`
`
`
`QQ,Wonerys
`
`2
`
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4,2012 Sheet 2 of 34
`
`US 2012/0254456 Al
`
`S0C
`
`jabeuey
`
`gss
`
`soz]BENE)seBeuelmura
`
`BuynpayssOfSiupeasues,
`
`
`node7ySIgMemesBugqniogJe(hpayos
`
`
`
`
`
`wate)100¢fsay
`
`
`|JoBeueyyJeung
`
`FIPSIABPISJBAIES
`
`UBIO
`
`jeBeuriy
`
`|
`
`WSR
`
`sogdjeuyT0Z
`
`
`
`
`
`MO|-|e}eq)pueJuUsUOdWIO4Y
`
`
`
`
`
`|aquibusjoonding
`
`
`
`
`|60C—efit‘SHINLLL
`
`
`auiBuzjopejal,gindu}
`aINJOSIUDIY
`
`SuynpayosUVESDY
`
`OlticSENLL
`
`
`
`
`
`BYoeD|BUIaKGalexuogIauUIEg>SUBD
`
`
`CleTzcapmpuegCcI¢
`J8AI8SUIBLOYORBISYOMIONpeziwindo
`
`COI
`
`3
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 3 of 34
`
`US 2012/0254456 Al
`
`©
`
`OngmoO
`
`cy
`
`ro
`
`oS
`ny
`
`Mm
`om)
`on
`
`\O
`n
`
`oo
`
`—
`35
`=
`3m
`>
`
`“
`ge
`=
`Om
`>
`
`on
`32
`=
`3m
`>
`
`=
`Qo st
`ge
`5
`
`WwW
`
`©i
`
`an)
`
`4
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 4 of 34
`
`
`
`buyspxqcopA
`
`
`SAINAISOOPIA
`SFAo
`
`MMS
`
`WE|
`
`
`
`SAINAIS[DJ1Od
`
`UOIINGIASIGBIINIIS
`
`90?LOP
`
`SAYIUIMS
`
`
`
`ssaynoyabp3
`
`ysOMjau
`
`800
`
`LON
`
`\,Nss
`
`US 2012/0254456 Al
`
`
`
`
`
`zop10Paouelj|ddyyoumMsoapiAjad+SdqdOTe
`
`
`
`VolaSIBAIVSQZJOpeaysulWSAZYUMdnajedse
`
`5
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4,2012 Sheet 5 of 34
`
`US 2012/0254456 Al
`
`
`
`
`
`$10e3ai33y1U9}U07
`
`
`
`SIIIAIISSULYSI|qnd
`
`§‘DIA
`
`
`
`SIQUMQCUa}U07)
`
`c0S
`
`
`
`JO9UMO1Ua4U0D
`
`TOS
`
`6
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 6 of 34
`
`US 2012/0254456 Al
`
`829
`
`929
`
`0e9
`
`YaAYsS
`
`
`
`
`
`MYOMLAN
`WOOT
`MYOMLAN
`
`AOVAYSLNI
`
`y09
`
`NOILVOINANWOO
`
`YOSsSs00ud
`
`
`
`019
`
`AOVYOLS
`
`AOA
`
`909
`
`NIV
`
`AYOWSAN
`
`9SI
`
`oO
`
`AV1dSI0
`
`v9
`
`
`
`AOIAACLNANI
`
`919
`
`YOsaNnd
`
`TOYULNOOD
`
`7
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 7 of 34
`
`US 2012/0254456 Al
`
`
`
`
`
`TRANSLATION
`LOGIC
`
`104
`
`BANDWIDTH
`MONITOR
`
`203
`
`
`
`
`fT
`
`
`
`
`y
`
`SERVER SIDE
`PLAYER(SSP)
`201
`
`(2)
`
`(1)
`
`Step 7:
`Step 8:
`
`Step 3:
`
`Step 4:
`Step 5:
`
`
`
`
`
`
`
`
`
`HTTP
`PROCESSING
`ENGINE(HPE)
`701
`
`Step 6:
`
`TASK
`SCHEDULER
`202
`
`Step 2:
`
`Step I:
`
`
`
`OPTIMIZED
`NETWORK
`STACK
`211
`
`
`
`
`
`y:
`
`
`
`Step 6a:
`
`\
`
`\
`}
`
`/
`
`/
`
`(1) @) Two calls for every new
`connection/request.
`
`Onecall for every
`(@)
`subsequent connection
`
`8
`
`

`

`US 2012/0254456 Al
`
`.
`Fi
`
`- 7b
`
`Patent Application Publication
`
`Oct. 4, 2012 Sheet 8 of 34
`
`Step 1: User request comes to HTTP
`Processing Engine (HPE) through Netstack
`after a connection has been established
`
`Step 2: HPE does minimalparsing to get the
`query string and name/value pairs
`
`Step 3: HPEinitiates a call to Server Side
`Player (SSP) with the Query String
`
`send the data retrieved on the wire.
`
`Step 4:
`1. For every first call per newconnection, SSP
`remaps the Query String to internal URL
`pointing to the Metafile.
`2. This Metafile stores number of profiles
`present, their rates, time period and other
`information of the video. This Metafile can be
`stored at the root of the Video file(foo)
`
`Step 5: SSP returns the remapped URL to HPE
`and also sets a flag(indicates to HPE to not send
`the returned metafile on the wire, but back to
`
`the SSP)
`
`Step 6: HPEcreates a tasks to fetch the Meta
`File, Gets it and passes it back to SSP
`
`Step 6a: The SSP stores the information in the
`meta-file in the socket structure of that
`
`connection. (The metafile information is stored
`in the cache as a result of this fetch forfast
`
`Step 7: Once the socketstructure is populated,
`SSP remapsthe original URL with the
`Translation Logic Plugin. The Translation
`Logic (does both seek and rate adaptation) uses
`the information present in the socket to do the
`necessary translation to different
`profiles/chunks
`
`Step 8: SSP returns the remapped URL and
`resets the flag in HPE to conveythatit’s ok to
`
`9
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 9 of 34
`
`US 2012/0254456 Al
`
`Policy Module
`801
`
`Media Manager
`204
`
`Analytics Module
`802
`
`Disk Manager SAS Disks
`
`Fig. 8a
`
`10
`
`10
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 10 of 34
`
`US 2012/0254456 Al
`
`Disk Manager
`
`206
`
`
`
`
`dictionary
`dictionary
`804a
`804b
`
`
`
`
`dictionary
`804c
`
`
`
`SAS Disk
`803c
`
`SAS Disk
`803a
`
`SAS Disk
`803b
`
`/alb/c/one.gif
`
`/alb/c/two. gif
`
`/a/b/c/three. gif
`
`Fig. 8b
`
`11
`
`11
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 11 of 34
`
`US 2012/0254456 Al
`
`Buffer Manager
`203
`
`RAM buffers
`
`813
` GETobject B
`
`Disk Manager
`206
`
`Single Disk IO
`
`
`
`Large disk block with multiple related objects
`806
`
`Fig. 8c
`
`12
`
`12
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 12 of 34
`
`US 2012/0254456 Al
`
`Buffer Pool 904
`
`
`
`Buffer Ops
`
`Buffer Manager
`Buffer Cache Hash Map,
`Free List, Buffer Operations,
`Eviction 203
`
`STAT/
`
`GET
`
`
`
`GETTask
`
`Cache Request Handler
`Nea Request
`Task
`<———} Handle GET Task 902
`ander
`<>
`Create Media STAT/GET
`
`Task 903
`
`
`Cache Request
`Media Request Table
`Private Data 905
`
`
`906
`
`
`13
`
`13
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 13 of 34
`
`US 2012/0254456 Al
`
`Buffer Manager
`203
`
`Media Provider:
`Solid State Device
`(SSD) Manager
`205
`
`Media Provider: Disk
`Manager
`
` Media Manager
`
`Media Provider:
`Origin Manager
`207
`
`Fig. 10a
`
`14
`
`14
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 14 of 34
`
`US 2012/0254456 Al
`
`report
`
`Analytics Manager
`
`1002
`
`promote
`
`Fig. 10b
`
`CLIInterface
`1001
`
`Buffer Manager
`
`203
`204
`
`config
`
`Media Manager
`
`Media Provider:
`Flash Manager
`1006
`
`Media Provider:
`Solid State Device
`(SSD) Manager
`
`205
`207
`
`Media Provider: Disk
`Manager
`206
`
`Media Provider:
`Origin Manager
`
`15
`
`15
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 15 of 34
`
`US 2012/0254456 Al
`
`Calculate deadline time. Post a task
`with deadline time. Deadline =
`required size / AFR
`
`Based on AFR and timerinterval,
`calculate the max allowed size (how
`muchdata to send over the connection in
`one timer interval).
`
`Send the data over the socket
`
`
` Send incomplete?
`
`Wait for socket space and
`
`
`
`send data.
` More data to send?
`
` Get next data chunk to send.
`
`16
`
`16
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 16 of 34
`
`US 2012/0254456 Al
`
`ISO file
`
`hint)
`
`metadata
`moov
`1202
`...other boxesAytrack (video)
`Interleaved, time-ordered, video
`and audio frames, and hint
`instructions for progressive
`download
`
`track (audio)
`
`track (pdh-
`
`17
`
`17
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 17 of 34
`
`US 2012/0254456 Al
`
`System Hostname
`
`
`
`Add Namespace
`
` $35
`
`aad
`
` E? Config
`
`AG tST
`
`AEE __I 304
`AAA
`
`1 3 05 oa ok st
`
`
`Enable Interfaces
`
`18
`
`18
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 18 of 34
`
`US 2012/0254456 Al
`
`
`exca
`
`
`CeLetewsTh “rn
`
`nnn}
`
`= nooo}
`
`1400
`
`totsizefromcache:
`Bag0064,00
`
`Slotsizefromorigi: Wtot_sizefromnfs:
`{RODGN00,00
`O00
`
`Wrtsp_outbytes:
`Bet.
`
`00007
`
`mo}
`
`mo=mE
`
`
`
`163910
`
`1639.20
`
`16380
`
`1 S40
`
`163950
`
`16:40:
`
`nokeena statise’ counter desplay toa
`
`1401
`
`1402
`
`1403
`
`19
`
`19
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 19 of 34
`
`US 2012/0254456 Al
`
`1500
`
`Network Usage (Last Hour}
`
`NOOO
`
`
`athé statistics
`
`
`
`BS
`BX bytes
`Bee
`RX packets
`RX micast packels
`EX discards
`a
`RX errore
`a
`
`[SS8
`
`TR bytes
`Tk packets
`Tx discards
`TX etrars
`Tx avers&
`
`RX Overs
`RX frame
`
`a
`Q
`
`Tx camer
`TX collisions
`
`6
`&
`
`1501
`
`Fig. 15
`
`20
`
`20
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 20 of 34
`
`US 2012/0254456 Al
`
`ATITAXYOu
`
`
`
`
`
`[=JS2APOXV=PISLATOO00j/FsAuooJodrunl-erpour//:dyy
`
`
`
`OPISJAIN
`
`JaARIq
`
`LO9I
`
`
`
`
`
`T=ISRATOXV=PIS([Uxooj/fsoo-Jodrunf-erpour//:dyyOLDLLSONDVHLVdVIVa
`
`
`
`
`
`
`
`
`
`A]['QdoOoj/fsfuooJodrunl-erpour//:dyy
`
`Ye[OU0Y)WVadaLS.LAN
`
`JoyerauayqIn
`
`SO9T
`
`foo;SuIpurg
`
`
`
`UOISSasIOFJ
`
`
`
`€=J49C=]SBTAOXV=PIS,004/Js/W09-Jodrunl“erpouy/-dyy
`
`
`
`
`
`
`
`AONVHDATAOUdYOA
`
`WVALS
`
`21
`
`
`
`
`
`ATdLNVITIdNODVSHAINONAAINO
`
`MOTHHLOOWS
`
`21
`
`
`
`
`
`

`

`Patent Appl
`
`icat
`
`ion
`
`Publ
`
`icat
`
`ion
`
`Oct
`
`4,
`
`2012 Sheet 21 of 34
`
`US 2012/0254456 Al
`
`
`
`
`
`JOSSOIOIdMOTJUJOOULS[e.NUISBSBSdAIOSUISLIQGAIW27oleuads
`
`
`
`
`
`
`
`
`
`
`
`sayoede8p_yCAWadnynuau}0}ulstiouesvsuoljounyua)pue
`
`qHDOra
`4
`
`4
`
`mmmrrrrr,
`
`~o_o—eee
`
`
`
`ISBIOISUISLIO
`
`
`
`SOLTSHAN
`
`cOLTGHW
`
`7
`
`ea =
`
`AHOVOD
`
`a_==wwSSSSBSSSSES|
`
`MOTJYJOOUS
`
`SUISSIOOIg
`
`q90L1
`
`
`
`pYADORILPome
`
`SSIIanovolSSI
`
`\WHOORLLI—___
`
`SOLT
`
`SOaPLA
`
`ONTHSITE#Nd
`
`W4LSAS
`
`vOLt
`
`MOLYIOOUS
`
`SSTINTHOVO
`
`SUISSIIOIg
`qoOrb
`
`UISIIO
`
`I9AIIS
`
`TOLT
`
`cOLTGHIA
`
`ee
`
`\
`
`
`
`JOSSIN0IgMOTJYIOOUISpuvaspzULseJOGSUIAIasGIA?[O1eu90S
`
`yo)
`
`IdAvid
`
`POLL
`
`JoARIjust
`
`POLLL
`
`quot)
`
`JoAP]g
`
`SOTLT
`
`qual
`
`JoART
`
`YOULL
`
`yuatT
`
`Jahvid
`
`BOTLT
`
`22
`
`22
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 22 of 34
`
`US 2012/0254456 Al
`
`
`
`
`
`-UdJOJSJUaWA|aSNanbajdjnwyansjsuoD-
`
`
`dooj|es0|8ulsnsjesseaed1gWjNwWYyd}e4-
`ayoedCj
`
`
`
`
`
`d|NpowBulsssvodald|JeD-
`
`
`
`8081SUIBUZBUISSIIO1dAld
`
`
`
`
`
`ayes|}]/NWJUaWseJ)‘AzZWaU/e}UOI-ay
`
`
`
`
`
`eJawAseuIq|eUJaU!GIA8389‘sayLL
`
`eyep
`
`QT“SI
`
`
`
`HOIAIO1U!ananb
`
`
`
`sajnpowdnueayd|eD-
`
`AydJesalH
`
`
`
`
`
`908TajnpoyjeusayxXy
`
`
`
`
`
`all,evepelawvaysijqndyoje4-
`
`
`
`sulysijqngananb-uq
`
`jsanbay
`
`
`
`THISRETLOSBISSAYTOMO04/js/luoovadiun[eipeu//:dq3yc08I
`
`
`
`ysasu|pasequassuy
`
`
`
`
`
`posijepelsurooj/is/iuoouediuntepau/day
`
`MO|JYOOUWS
`
`ananb-uy
`
`sjessy
`
`LOOT“SSIAPYIED
`
`dSSMojjyIOOWS
`
`23
`
`
`
`
`
`SSI]8Yyde>sNndul[JIMJeyJSanbawseynday
`
`
`
`
`
`vO8I
`
`23
`
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 23 of 34
`
`US 2012/0254456 Al
`
`oS Se www rst wk KK“ ---
`
`~
`
`“O]
`
`oNsousYy
`
`TI6T
`
`
`OTSO6T
`606I!SUdTIANVH|!JHUO0O|
`CNHAOVaPeseqNIOMTd
`
`wdCL)ursnidO/1
`8061O/l
`winpozH=OVCH
`Aegyoip—s«sELL
`
`SUITpueUUO;)
`
`9dA
`
`O16!
`
`6[“Sly
`
`8081
`
`sUuTTIO
`
`HdA
`
`Jaseurly
`
`COST
`
`oulfuy
`
`WW
`
`rool
`
`24
`
`24
`
`
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 24 of 34
`
`US 2012/0254456 Al
`
`aZISyoyoedeyeqWd|paureqos
`
`slag
`
`800¢
`
`900¢
`
`100c
`
`Nd
`
`€10C
`
`yoyovied
`
`O10c
`
`CcO0T
`
`25
`
`OcSIH
`
`
`yoyoegeyeq|yyorgvieq
`poureywosJas
`cLOCTT0C
`
`yoyoegvyeq
`pourr}uosJas
`joyoegeyeq|Jepray
`
`TopeoHtd
`pO0T£00C
`
`25
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 25 of 34
`
`US 2012/0254456 Al
`
`IOI?
`
`
`
`qHdVaHOPAANAD
`
`cOTT
`
`TOVAaLVad
`
`COI?
`
`
`
`COVEdanLVds
`
`SOIC
`
`VIVd—~¢OVL
`
`90TC
`
`VIVd—[OVL
`
`pOTT
`
`26
`
`26
`
`
`

`

`Patent Application Publication
`
`Oct. 4,2012 Sheet 26 of 34
`
`US 2012/0254456 Al
`
`£0ccAYMOUs
`
`YJOOUISJIAT[IP
`
`
`cOccSYseo
`
`ngspdrypnu
`
`+Salayer
`
`TTSIA
`
`yIqa]sUIsoUO
`
`10cceIoer
`
`27
`
`27
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 27 of 34
`
`US 2012/0254456 Al
`
`]JEULIOT
`
`ZJRULIO}
`
`<+—____—_
`
`
`
`POETPueursoj
`
`CT‘SIq
`
`YJOOUISIOAI[APwe
`
`
`SOC?AIMOTFUWVULIOJ
`COETATLoesWgo[SUIS
`0po
`
`OCT,ap
`
`paeorpep/NdD
`
`
`
`COLTMWeMpley
`
`yIqa]SUISU0
`
`TOEAYyet
`
`28
`
`28
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 28 of 34
`
`US 2012/0254456 Al
`
`
`
`YJOOUWSIOAT[ap
`
`SOVSIFMOLY
`
`nqoydiyypnuu
`
`
`
`+SoplJayer
`
`
`
`COVESYovo
`
`JJeurIOy
`
`voSIA
`
`PeBITPSP/11dD
`
`
`
`COPEoPMpIEY
`
`POTeueuoy
`
`
`
`ygo[SUIssUO
`
`lOvePIFyet
`
`29
`
`29
`
`

`

`US 2012/0254456 Al
`
`CZ“SIA
`
`Patent Application Publication
`
`Oct. 4, 2012 Sheet 29 of 34
`
`JaAJasaspe
`
`90S8¢
`
`IDAISSOSPO
`
`LOST
`
`IDAIOSASpPd
`
`80SC
`
`PelBoTpep/{dt)
`
`YpPIMpuegysty
`
`POSTAY
`
`ippIMpuegAo]
`
`COSCMIU
`
`
`
`COSTceMp.ey
`
`
`
`Iqa[suIsaUO
`
`COSCPIevel
`
`30
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 30 of 34
`
`US 2012/0254456 Al
`
`YIOMION
`
`c09C
`
`97‘SIA
`
`SOOPIA
`
`
`
`MOL]BIPIA
`
`
`
`(dAIA)A94sTIqQNg
`
`109¢
`
`
`
`UISLIGCA4yeg¢pa
`
`IIATIS
`
`c09T
`
`31
`
`31
`
`
`

`

`Oct. 4, 2012 Sheet 31 of 34
`
`Patent Application Publication
`
`
`
`JYSYAIATISOWS!£=9f@--------------------------------------------------------------------------------
`
`
`
`bdnay)auruyVINSI/AIST
`
`
`
`ades0j¢UoneUlseg
`
`
`
`JVULIOJUDUISBA
`
`60LT
`
`
`
`
`
`TOLc(paseg214)eOLe
`
`COA
`
`
`
`
`
`JEULIOYIL]9da1n0G
`
`
`
`LOLTdW1.44)SUTIO‘T
`
`“DAIPITH
`
`
`
`‘quoyd!snew80LE90L¢NAWqasieg
`
`
`—
`
`:SLIgUdUIseL]sy):Sr:[eyuswepun.f
`
`asedT
`
`srt
`
`—SOLT
`
`32
`
`LTSIASVANIIAAT
`
`
`
`TOLT(@uTTUT)
`
`vIpIAWAVV/A9CH
`
`—Pd
`
`
`
`
`
`juaysaansSWSJBULIOJWeg30.1n0S
`
`US 2012/0254456 Al
`
`‘anoyd!sen
`
`,INUUIseL]SL
`
`
`s}luQ[TeyWeuTepun,y
`
`80L¢
`
`90L¢NAW
`
`IOATIOOYY
`
`OTL7
`
`PIP
`
`YseLAATdba
`
`
`
`W/OVV/P9TH
`
`dI/ddadn/S.LdS
`
`ddiVdLW/S.Lds
`
`
`
`
`
`CIL¢dWHaq}ouryuyFOLZ
`
`32
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 32 of 34
`
`US 2012/0254456 Al
`
`18d*TOMION
`
`
`
`GLI8zIDAI08
`
`ospa/CIW
`
`oTIQTISAIS$
`
`93p9/C-TW
`
`BTIQoISAI3$
`
`33po/CHW 608TAY
`
`ipprpueqXATL
`
`LOST719
`
`wppmpueqXANA
`
`SO8TIY
`
`ippmpueg7AAA
`
`OZJoIaro
`
`TYOAW
`
`QZ“SI
`
`CO8T
`
`
`
`MP
`
`addy
`
`t08c
`
`aqopy
`
`VO8T
`
`LASW
`
`SO8C
`
`33
`
`33
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 33 of 34
`
`US 2012/0254456 Al
`
`8906C
`
`addy
`
`LAISW
`
`4906¢
`
`Sqopy
`
`9906C
`
`9906C
`
`ddD¢
`
`tpprapueq&(HIN
`
`€067AY
`
`pO6C9ipprapueqAAW
`
`(ppiapueq7NAN
`
`S06caI
`
`67‘SI
`
`UTSLIO
`
`
`
`TN6ZTreed
`
`19NAW
`
`JaAIas
`
`106¢
`
`34
`
`34
`
`

`

`Patent Application Publication
`
`Oct. 4, 2012 Sheet 34 of 34
`
`US 2012/0254456 Al
`
`099199-NJSUIPOsUYVIPS]
`
`
`
`
`Z00¢ATIATTIIG'LOOESynduyoo.n0s
`
`
`SISITIUPAA+PIA-3C1
`
`YAaAyS*TOTGa
`
`
`
`SOO,WUBINSIUTOY
`
`SYVIPSSEL8S8POL)
`
`
`
`SUOUI:(S}eUHGy
`
`
`
`dexsjoog/ysayueyy900¢-SVBULAOToULO‘TOASSO
`
`
`
`
`
`
`+Waz-seayJUISUY]wenssndepyDUNT)[BOYABOUTT/AATT]
`
`CapaSO0¢eursug
`
`
`
`
`
`BIDITAJEULIO,YEPYPoposus-aiqswaRAngqaystyqngsoreangpeson1papoousue©MOLT
`
`
`
`
`
` \SISTART+SEFRLT3qopy
`
`
`
`JEULIG]WEPyPopoous-alyssurwasy
`SOSACAINST“pdSRwacg
`
`
`
`ardeyPORESOOANOSSHGOA/ATTLAST"3SE‘
`
`IVVIPGOH3SSOpol)
`
`35
`
`
`
`/2I01SJUAISISI9g
`
`LOOE“481
`
`O¢‘SI
`
`
`
`
`
`pueutedUOIALsvauyyyrysoutLyp/GOA~
`
`SOYP.HIGPAIYEPSpaokSWPAoO
`
`35
`
`
`
`
`
`
`
`
`
`
`

`

`US 2012/0254456 Al
`
`Oct. 4, 2012
`
`MEDIA FILE STORAGE FORMAT AND
`ADAPTIVE DELIVERY SYSTEM
`
`TECHNICAL FIELD
`
`[0001] The present disclosure generally relates to media
`contentdelivery.
`
`BACKGROUND
`
`MED, and a central managementstation across a network,
`according to a possible embodimentof the invention;
`[0009]
`FIG.2 illustrates an example of an MFD component
`and data flow architecture, according to a possible embodi-
`mentof the invention;
`[0010]
`FIG. 3 illustrates an example of dynamically deter-
`mining the popularity of portions of content, according to a
`possible embodimentofthe invention;
`[0011]
`FIG. 4 illustrates a MFD deploymentthat replaces a
`conventional video server site, according to a possible
`embodimentof the invention;
`[0012]
`FIG. 5 illustrates an MFD network delivering con-
`tent to client systems, according to a possible embodimentof
`the invention;
`[0013]
`FIG. 6 illustrates a computer system upon which a
`possible embodiment may be implemented;
`[0014]
`FIG. 7a illustrates flow diagram of the workflow
`between the HPE andthe server side player, according to a
`possible embodimentofthe invention;
`[0015]
`FIG. 76 illustrates flow diagram of the workflow
`between the HPE andthe server side player, according to a
`possible embodimentofthe invention;
`[0016]
`FIG. 8a illustrates the interaction between a disk
`manager and a policy module and an analytics module,
`according to a possible embodimentof the invention;
`[0017]
`FIG. 8& illustrates the interaction between a disk
`manager and dictionaries for storage devices, according to a
`possible embodimentofthe invention;
`[0018]
`FIG. 8c illustrates the interaction between a disk
`manager and a reverse block map, according to a possible
`embodimentof the invention;
`[0019]
`FIG. 9 illustrates the interaction between a buffer
`pool and buffer manager, according to a possible embodiment
`of the invention;
`[0020]
`FIG. 10a illustrates the interaction between a media
`manager, buffer manager, and media providers, according to
`a possible embodimentof the invention;
`[0021]
`FIG. 10d illustrates the interaction between a media
`manager and analytics module for determining hot video
`segments, according to a possible embodimentof the inven-
`tion;
`FIG. 11 illustrates a flow chart of anetwork manager
`[0022]
`in a MFD,according to a possible embodimentofthe inven-
`tion;
`FIG. 12 illustrates a progressive downloadhinttrack
`[0023]
`location in a containerfile, according to a possible embodi-
`mentof the invention;
`[0024]
`FIG. 13 illustrates a graphical user interface screen-
`shot of a parameter specification screen, according to a pos-
`sible embodimentof the invention;
`[0025]
`FIG. 14 illustrates a graphicaluser interface screen-
`shot of a real time byte delivery monitoring graph, according
`to a possible embodimentofthe invention;
`[0026]
`FIG. 15 illustrates a graphicaluser interface screen-
`shot of a network connection performance monitoring graph,
`according to a possible embodimentof the invention;
`[0027]
`FIG. 16 illustrates an media flow director (MFD)in
`communication with client systems across a network, accord-
`ing to a possible embodimentofthe invention;
`[0028]
`FIG. 17 illustrates MFD applications in a network,
`according to a possible embodimentof the invention;
`[0029]
`FIG. 18 illustrates a publishing system that delivers
`
`
`[0008] FIG.1illustrates a media flow director (MFD)in videofiles to an MFD,according to a possible embodiment of
`the invention;
`communication with client systems, an origin server, a peer
`
`[0002] The approaches described in this section could be
`pursued, but are not necessarily approaches that have been
`previously conceivedor pursued. Therefore, unless otherwise
`indicated herein, the approaches describedin this section are
`not prior art to the claims in this application and are not
`admitted to be prior art by inclusionin this section.
`[0003] Delivery of video content over the Internet has
`evolved overthe years. Thefirst applications ofvideo delivery
`from content servers to client computers were restricted by
`technology and bandwidth capacity. Video files had to be
`dramatically reduced in size to accommodate the low band-
`width oftelephonelines. Low resolution video content had to
`be downloadedto a client computer in whole before the video
`file could be played to a user. This was due to file system
`limitations that required an entire file to be downloaded
`before the file was available in the file system and video
`player software only having the ability to play back complete
`files. Users were forced to endure long delays waiting for the
`video file download to complete.
`[0004]
`Proprietary file formats and video player software
`werecreatedto allow the user to view the video content as the
`
`content was being downloaded. Thefile was either saved or
`discarded after the download was complete and the user
`viewedthe video content. This approach was highly suscep-
`tible to download delays due to bandwidth limitations, server
`loads, or network delays. The playback of the video content
`hadto be periodically stopped because the video playersoft-
`ware starved from the lack of video content to playback.
`[0005] A more sophisticated method was developed that
`streamed the video content to the video player software. The
`delivery systems typically had a single bit rate videofile for
`each video contenttitle. The single bit rate video files were
`served to all users regardless of their bandwidth availability.
`Users with higher than normal bandwidth were penalized by
`being forced to view video content that was a lower quality
`than the bandwidth justified.
`[0006] A certain amountof the video content was buffered
`before the user was able to playback the video content. The
`buffer was large enough to hold an amountofvideo contentto
`mask over minor delays in video content delivery caused by
`bandwidth limitations, server loads, or network delays. Long
`delivery delays, typically a few seconds or longer, were
`caused by erratic last mile bandwidths. The delivery delays
`would starve the video player software and cause the video
`player softwareto stop the playback ofthe video contentuntil
`delivery resumed and the buffer was sufficiently filled.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`invention is illustrated by way of
`[0007] The present
`example, and not by wayof limitation, in the figures of the
`accompanying drawings and in whichlike reference numer-
`als refer to similar elements and in which:
`
`36
`
`36
`
`

`

`US 2012/0254456 Al
`
`Oct. 4, 2012
`
`FIG. 19 illustrates a preprocessing engine that pre-
`[0030]
`pares video file for delivery by the MFD, according to a
`possible embodimentof the invention;
`[0031]
`FIG. 20 illustrates examplesoffile formats that can
`be used by an MFD,according to a possible embodiment of
`the invention;
`[0032]
`FIG. 21 illustrates an internal file structure, accord-
`ing to an embodimentofthe invention;
`[0033]
`FIG. 22 illustrates an MFD caching multiplebit rate
`files, according to an embodimentofthe invention;
`[0034]
`FIG.23 illustrates an MFD dynamically creating bit
`rate specific files from a single bit rate file in response to
`segment requests, according to an embodimentof the inven-
`tion;
`FIG. 24 illustrates an MFD caching multiplebit rate
`[0035]
`files and dynamically creating bit rate specific files from one
`of the cached multiple bit rate files, according to an embodi-
`mentof the invention;
`[0036]
`FIG. 25illustrates an origin server sending twobit
`rate files to edge servers, according to an embodimentof the
`invention;
`[0037]
`FIG.26 illustrates an Media Flow Publisher (MFP)
`creating smoothflow formatted files from source input files
`from an origin server, according to an embodimentof the
`invention;
`[0038]
`FIG. 27 illustrates an MFPcreating Media Funda-
`mental Unit (MFU)formattedfiles from differing formatted
`source input streams/files, according to an embodimentofthe
`invention;
`[0039]
`FIG. 28 illustrates an MFPcreating MFU formatted
`files from differing source input streams/files from an origin
`server, according to an embodimentof the invention;
`[0040]
`FIG. 29 illustrates an MFD creating MFU formatted
`files from differing source input streams/files from an origin
`server, according to an embodimentof the invention; and
`[0041]
`FIG. 30 illustrates an MFD dynamically creating
`MFU formatted files from live source input streams and
`source input files, according to an embodimentofthe inven-
`tion.
`
`DETAILED DESCRIPTION
`
`[0042] A dynamic variable rate media delivery system is
`described. In the following description, for the purposes of
`explanation, numerousspecific details are set forth in order to
`provide a thorough understanding of the present invention.It
`will be apparent, however, to one skilled in the art that the
`present invention may be practiced without these specific
`details. In other instances, well-knownstructures and devices
`are shownin block diagram form in order to avoid unneces-
`sarily obscuring the present invention.
`[0043] Embodiments are described herein accordingto the
`following outline:
`
`1.0 General Overview
`
`2.0 Structural and Functional Overview
`
`[0044]
`[0045]
`[0046]
`[0047]
`[0048]
`[0049]
`[0050]
`[0051]
`
`2.1 Adaptive Content Delivery over a Network
`2.1.1 Media Flow Director Architecture
`2.1.2 Media Flow Director Placement
`2.2 Variable Rate Media Delivery over a Network
`2.2.1 Smoothflow Architecture
`2.2.2 Transitioning Between Different Bit Rates
`2.2.3 Media Flow Director in the Network
`2.2.4 Preparing Smoothflow Files
`
`[0052]
`[0053]
`
`2.2.5 Media Flow Publisher
`
`2.2.6 Universal Media File Formatting
`
`3.0 Implementation Mechanisms—Hardware Overview
`
`4.0 Examples
`5.0 Extensions and Alternatives
`
`1.0 General Overview
`
`[0054] This overview presents a basic description of some
`aspects of possible embodiments of the present invention.It
`should be noted that this overview is not an extensive or
`
`exhaustive summary ofaspects of the possible embodiment.
`Moreover,
`it should be noted that
`this overview is not
`intended to be understood as identifying any particularly
`significant aspects or elements of the possible embodiment,
`nor as delineating any scope of the possible embodiment in
`particular, nor the invention in general. This overview merely
`presents some concepts that relate to the example possible
`embodiments in a condensed and simplified format, and
`should be understood as merely a conceptual prelude to a
`more detailed description of example possible embodiments
`that follows below.
`
`[0055] A variable rate media delivery system is described.
`In other embodiments, the invention encompasses a computer
`apparatus and a computer-readable medium configured to
`carry out the described steps. In the text below, video content
`and video data are used as examples of media content and
`media data, but the examplesare not limited to video content
`alone and may include other types of media content and
`media data, e.g., audio, multimedia presentations, slide
`shows, etc. The system adapts to changes in the bit rate
`between a server and client by delivering media data thatis
`appropriate for the changing bit rate. In an embodiment,
`video data is sent to match the bandwidth between the server
`and client. Bandwidth between a client and server is con-
`
`stantly varying. At a certain point in time,thereis a bit rate at
`which the video data has to be delivered for the video to be
`
`continuously played back by the client’s video player without
`pausesor interruptions.
`[0056] The system ensures that the client video player is
`neverstarved of data. The bandwidth betweenthe server and
`
`client can be automatically detected and the bandwidth is
`mapped by the server to an appropriate video bit rate. The
`client video player is supplied with a steady stream of video
`data. The time scale is not compromised but the amount of
`data delivered to the client can vary to match the available
`bandwidth. As such, the quality of the video being played
`back can vary. The client player always plays the same
`amountof data; just the quality of the video changes.
`[0057] The server seamlessly switches between video bit
`rates without the user and client video player seeing any
`interruptions.
`In a possible embodiment,
`the server can
`switch bit rates upon significant drops or increases in orderto
`stop from reacting to spikes or valleys too quickly. The only
`noticeable change is the quality of the video being played.
`Standard video players can be used to interface with the
`server. A possible embodiment provides a video streaming
`experience to the user via a progressive download approach.
`Tn another possible embodiment, a video player can have the
`intelligence to communicate with the server using control
`signals or commandsto enable other features of the server
`such as trick play features/bit rate adaptation or providing
`information about client side resources such as CPU usage/
`
`37
`
`37
`
`

`

`US 2012/0254456 Al
`
`Oct. 4, 2012
`
`memory. Trick play is a feature that allows users to control
`their media playback experience using flow commands such
`as fast forward, rewind, framestep, pause, etc., ona videothat
`is currently being streamed.
`[0058]
`In another possible embodiment, the video player
`can be modified slightly to be able to send theserver a control
`signal or commandthattells the server that the video player
`has been paused. The pausenotification allows the server to
`halt the progressive downloadto the video player. This frees
`up someof the bandwidth ofthe server and allows the server
`to handle other downloads.
`
`[0059] An adaptive network content delivery system is
`described. Referring to FIG. 1, a possible embodimentpro-
`vides a media flow director (MFD) 103. A MFDis a network
`device that can efficiently deliver video content (and other
`types of file content of various sizes, e.g., media (audio,
`pictures, etc.), games, software, HTML,scripts, etc.) over a
`network 104 to a plurality of clients 101¢, 1015, 1017, via
`HTTP, FTP, streaming, or other protocols. Video content
`includes such contentas feature length movies, sitcoms, vari-
`ety shows, talk shows, news shows, advertisements, etc. Cli-
`ent devices 101a, 1015, 101”, may be a personal computing
`device such as a desktop or laptop computer, set top box,
`handheld computing device, cellular phone, portable media
`player, or any other portable device capable of displaying
`multimedia content, and are also coupled to network 104
`through any properinterface.
`[0060] The MFD 103 stores video content internally in a
`variety of storage devices, including HDD, SSD, RAM,non-
`volatile memory, etc. The MFD 103 can deliver video to a
`large number of clients while maintaining a high level of
`viewing experience for each client. The MFD 103 automati-
`cally adapts the bit rate of a video being delivered to a client
`101 by measuring the client’s last mile bit rate variation. The
`MEDprovidesthe clients with smooth viewing ofvideo with-
`out buffering stops. It assures delivery of in-progress videos
`to clients using the adaptive bit rate delivery. The MFD
`dynamically adapts the bit rate of the video to match the
`effective last mile bandwidth and to adapt to changing net-
`work conditions for the client. The MFD can dynamically
`measure the last mile bandwidth to the client in order to
`
`dynamically adjust the bit rate. The client does not need a
`custom video content player to communicate with the MFD.
`The MFDsupports industry standard video players such as
`Flash, Quicktime, SilverLight, Windows Media Player, etc.
`[0061] The MFD 103 providespolicy controls on the stor-
`age and delivery of videos that can be defined by a customer
`as well as the MFD provider. The MFD can be deployed in a
`variety of ways across the network 104, including at an edge
`cache location or an origin server location. The network 104
`may be implemented by any medium or mechanism that
`provides for the exchange of data between devices in the
`communication system. Examples of network 104 include,
`without limitation, a network such as a Local Area Network
`(LAN), Wide Area Network (WAN), Ethernet, intranet, the
`Internet, or one or moreterrestrial, satellite or wireless links,
`etc.
`
`[0062] The MFD 103 communicates with network 104
`through any proper communication interface, such as an Eth-
`ernet or wireless communications port. Alternatively or in
`addition, any numberof devices connected to network 104
`mayalso be directly connected to each other through a com-
`munications link. For example, the MFD 103 can request
`content from an origin server 102 via the network 104 or via
`
`a local network in order to obtain URL content whena client
`101 requests a URL content that the MFD 103 does not have
`stored in its memory. Additionally, MFD 103 can communi-
`cate with another MFD 105via the network 104 orvia a local
`network in order to obtain URL content, MFD content infor-
`mation, MFDstatus,etc.
`[0063] Acentral managementstation 106 maybe available
`to allow administrators to set policies for MFD management
`such as cache management, bandwidth limitations, content
`popularity determination, etc. The central managementsta-
`tion 106 also allows administrators to manage where and how
`content is stored in the multiple storage devices within a
`MED. The central managementstation 106 may communi-
`cate via a local network or connection to MFDs 103, 105 or
`via network 104. An administrator logs into the central man-
`agement system 106 and selects a MFD orset of MFDsthat
`the administrator wants to interact with. The administrator
`can define policies and have the policies sent to a MFD or
`pushed out to multiple MFDs by the central management
`station 106. Alternatively, the MFDs may contact the central
`managementstation 106 periodically or whennotified that by
`the central managementstation that updates are available for
`that MFD.
`
`2.0 Structural and Functional Overview
`
`2.1 Adaptive Content Delivery Over a Network
`[0064]
`2.1.1 Media Flow Director Architecture
`[0065]
`FIG. 2 illustrates a possible embodiment showing
`the architecture of a content delivery switchthat is configured
`as a MFD to deliver video content. The components in the
`architecture are described below.
`
`Output Protocol Engine 210
`
`[0066] The output protocol engine 210 handles requests
`from clients. A client sends a request to the MFD to make a
`connection with client, e.g., an HTTP get request for a par-
`ticular URL,
`the output protocol engine 210 passes for
`requests URL cont