SPIEProceedingsVol.CR,StandardsandCommonInterfacesforVideo
`InformationSystems,October-, ,Philadelphia,USA
`ComparisonoftheH. andH. Video
`CompressionStandards
`BerndGirod,EckehardSteinbach,NikoF(cid:127)arber
`TelecommunicationsInstitute
`UniversityofErlangen-Nuremberg
`Cauerstrasse, Erlangen,Germany
`girod@nt.e-technik.uni-erlangen.de
`InvitedPaper
`DedicatedtoProf.Dr.-Ing.H.G.Musmannontheoccasionofhis
`thbirthday
`ABSTRACT
`ThedraftinternationalstandardITU-TH. iscloselyrelatedtothewell
`knownandwidelyusedITU-TRecommendationH. .However,H. does
`providethesamesubjectiveimagequalityatlessthanhalfthebit-rate.Inthis
`paperweinvestigatetowhatextendsingleenhancementsofH. contributeto
`thisperformancegain,andconsiderthetrade-o(cid:11)qualityvs.complexity.Based
`onthetestsequence\Foreman",H. initsdefault-andoptionalcoding-
`modesiscomparedtoH. onthebasisofratedistortioncurvesatbit-rates
`upto kbps.Atkbps,theperformancegainofH. initsdefaultmode
`comparedtoH. isapproximatelydBPSNR.Thisimprovementisachieved
`withonlylittleincreaseofcomplexity,andismainlyduetomoreaccuratemo-
`tioncompensationwithhalf-pelaccuracy.Consideringthetrade-o(cid:11)qualityvs.
`complexity,thecombinationoftheoptionalcoding-modes\Advancedpredic-
`tionmode"and\PB-framesmode"seemstobeagoodcompromise,resulting
`inanadditionalperformancegainof dBPSNRatkbps.The\Syntax-
`basedarithmeticcodingmode"ontheotherhand,o(cid:11)ersonlyaverysmall
`performancegain(.dBPSNRatkbps)foritsincreasedcomputational
`complexity.Resultsfrompro(cid:12)lingaH. softwarecodecarepresentedin
`ordertosupportcomplexityconsiderationsoftheoptionalcoding-modes.
`
`IPR2018-01413
`Sony EX1011 Page 1
`
`

`

` INTRODUCTION
`Thedemandforvideophoneapplicationsovercommunicationchannelswith
`lowbandwidthlikePSTNormobilelinksrequiresnewstandardsforthecom-
`pressionofimagesequencesatverylowbit-rates.Thoughvideotransmission
`atkbpswasalreadyreportedin  [ ],reasonablepicturequalityatbit-
`ratesbelowkbpsisstillchallenging.However,theITU-TStudyGroupXV
`hasnowdraftedsuchastandardsuitableforvideotransmissionbelowkbps.
`TheITU-TdraftinternationalstandardH. [ ]iscloselyrelatedtothe
`wellknownandwidelyusedITU-TrecommendationH. [],whichhasbeen
`designedbythesameStudyGroup.Thiscloserelationshiphelpedtoarrive
`atthenewstandardinashortperiodoftime,includingnotonlythevideo
`codingalgorithmbutalsothecorrespondingaudio(G. ),multiplex(H. ),
`control(H.)andsystem(H. )aspects.ThoughH. andH. sharethe
`samebasiccodecstructure,thereisasigni(cid:12)cantimprovementinperformance.
`Side-by-sidecomparisonsshowthatthesamesubjectiveimagequalitycanbe
`achievedwithlessthanhalfthebit-rate.Thisperformancegainisdueto
`improvedandoptimizedcodingtechniques,aswellasoptionalcoding-modes
`(\options")whichmaybeswitchedonbythecoder.Foragivenapplication,
`however,thetrade-o(cid:11)qualityvs.complexityhastobeconsideredaswell.
`Therefore,acomparisonoftheH. andH. videocompressionstandards
`regardingqualityandcomplexitywillbeinvestigatedinthispaper.Weassume
`thatthereaderisfamiliarwithH. .
`Thepaperisorganizedasfollows.Insectionwebrie(cid:13)ydescribethehybrid
`codingstructurecommontoH. andH. .
`Insection thedi(cid:11)erences
`betweenthetwostandardsarepointedout,followedbyadescriptionofthe
`additionaloptionsofH. insection.Thenwecomparetheperformanceof
`H. andH. withseveraloptionsinsection.Theincreaseincomplexity
`duetotheH. optionsisdiscussedinsection.
`CODERSTRUCTURE
`BothH. []andH. [ ]usethesamebasicstructureoftheencoder
`(Fig. ).Itisahybridofinterframepredictionexploitingtemporalredun-
`dancyandtransformcodingoftheresidualpredictionerrorexploitingspatial
`redundancyandadaptivelyreducingspatialresolution.Temporalpredictionis
`basedonablock-basedmotionestimation(ME)andcompensation(MC),while
`adiscretecosinetransform(DCT)isusedforspatialredundancyreduction.
`
`IPR2018-01413
`Sony EX1011 Page 2
`
`

`

`Themotioncompensatedpredictionerrorissubdividedinto(cid:2)blocks
`andeachblockistransformedbytheDCT.Thecoe(cid:14)cientsarequantized
`(Q)andpairsofzero-runsandquantizerlevelsarecombined,resultinginan
`variablelengthcodeword(VLC)foreachblock.Oneof uniformthreshold
`quantizersmaybeselectedadaptively.Finally,run-levelpairs,motionvectors
`(MV),andquantizationparameters(Qp)areentropycodedalongwithother
`sideinformationandmultiplexedtothebitstream.
`Figure :ThehybridcodingstructureofH. andH.
` DIFFERENCES
`ThoughH. andH. sharethesamebasiccodingstructure,thereare
`bothminorandmajordi(cid:11)erences.Thesearecoveredinthissection.Aspecial
`sectionisdedicatedtotheH. -options,whicharefurtherenhancementsnot
`includedinH. .
`Targetbit-rate
`Thetargetbit-rateofH. isp(cid:2)kbps(p= ,,:::, )whereasH.
`aimsatbit-ratesbelowkbps.Thosebit-ratesre(cid:13)ectthetypicalapplications
`envisionedwhenthestandardsweredesigned.InthecaseofH. ,thiswas
`visualtelephonyoverISDN,whereasH. shallenablethesameserviceover
`PSTN(e.g.,usingV. ).
`
`DCT Q
`
`frame
`buffer
`
`MV
`
`MC
`
`ME
`
`intra/inter
`
`Coding
`Control
`
`Video in
`
`mode
`
`Qp
`
`VLC
`
`-1
`Q
`
`-1
`DCT
`
`IPR2018-01413
`Sony EX1011 Page 3
`
`

`

`Pictureformats
`H. operatesontwopictureformats,CIF(CommonIntermediateFor-
`mat)andquarter-CIF(QCIF).Asthenameimplies,thespatialresolutionof
`QCIFisonequarteroftheCIF-resolution.
`InadditiontoCIFandQCIF,
`H. supportsathirdformatcalledsub-QCIF.AscanbeseeninTable ,the
`spatialresolutionisfurtherreducedbyafactorofapproximately.Sub-QCIF
`resolutiondoesfurtherlimittheimagequality,butitdoesprovideabettersub-
`jectiveimpressionatextremelylowbit-rates,e.g.,below kbps.Inaddition,
`thereducednumberofsamplesisattractiveforinexpensivelow-endterminals.
`CIF (cid:2)
`pels
`QCIF (cid:2) 
`pels
`sub-QCIF (cid:2) 
`pels
`Table :Spatialresolutionofluminancecomponents
`Motioncompensationaccuracy
`WhileH. islimitedtomotioncompensation(MC)withinteger-pelac-
`curacy,H. provideshalf-pelaccuracy.Wewillseeinsection,thatthis
`extensionimprovesperformancesigni(cid:12)cantly.Theimprovementduetohalf-
`pelMCisawellestablishedfact[]andhasalreadybeenutilizedsuccessfully
`inITU-TH.(MPEG-)[].
`Filterintheloop
`H. utilizesaspatiallowpass(cid:12)lterinthepredictor,the\(cid:12)lterinthe
`loop",whichcanbeswitchedonamacroblockbasis.H. doesnotinclude
`suchaloop(cid:12)lter.ThisisinfacttheonlyfeaturefromH. notadopted
`byH. .Sincethereisasigni(cid:12)cantperformancegainduetoloop(cid:12)ltering
`inH. ,thismayseemsurprising.Thereason,however,isthatthebilinear
`interpolationusedinH. forhalf-pelMCintroducesspatiallowpass(cid:12)ltering
`asasidea(cid:11)ect.Inaddition,oneoftheH. optionsincludesoverlappedblock
`MC(seesection),whichhasaninherent(cid:12)lteringe(cid:11)ectaswell.
`GOBlayer
`Bothstandardsuseahierarchicalsyntaxdecomposingasequenceintopic-
`tures,groupofblocks(GOB),andmacroblocks(MB).InH. ,thesizeof
`aGOBhasbeenreducedtoasingleMB-row( MBsforQCIF,MBsfor
`sub-QCIF)comparedtothreeMB-rows( (cid:2) MBs)inH. .Thesecond
`di(cid:11)erenceintheGOBlayeristhatH. allowstoinsertheaderinformation
`optionally.Thisenablesthecoder,e.g.,toinsertextrasynchronizationwords,
`ortoreducetheoverheadforapplicationsrequiringalowerrobustness.
`
`IPR2018-01413
`Sony EX1011 Page 4
`
`

`

`Errorcorrection
`TheITU-TRecommendationH. includesforwarderrorcorrectionusing
`aBCH( , )code,whilenoparticularerrorprotectionschemeisrecom-
`mendedforH. .However,ITU-TStudyGroupXVcontinuestoworkon
`errorprotectionforapplicationsofH. ,e.g.,inmobileenvironments.
`OtherBesidesthedi(cid:11)erencesmentionedaboveH. containsseveralminorim-
`provementscomparedtoH. ,whichshallbeexplainedbrie(cid:13)y.Thougheach
`singleimprovementonlymakesasmalldi(cid:11)erence,theyalladdupandcontrib-
`utetotheoverallperformance.
`(cid:15)VLCtables
`Accordingtothenewsyntax,thereareseveralnewtablesforvariable
`lengthcodes(VLC).Othertablesareoptimizedforbetterperformance.
`E.g.,theeventscodedintheVLCtableforDCTcoe(cid:14)cientsnowconsist
`oftriplets(run,level,eob)ratherthanpairs(run,level).
`(cid:15)Motionvectorprediction
`Motionvectorsarecodeddi(cid:11)erentiallyasinH. ,butwithamore
`sophisticatedpredictor.NotonlytheprecedingMBisusedforprediction,
`butalsoMBsinthepreviousMB-row.
`(cid:15)Adaptationofquantizationparameter
`WhileH. allowsanyquantizerforanyMB,H. restrictsthetran-
`sitionsfromMBtoMBwithinaGOBtothetwonextcoarseror(cid:12)ner
`quantizers.Thisyieldsasmallerandmoree(cid:11)ectiveVLCtable.
`(cid:15)Macroblockaddressing
`ForskippedMBs,H. usesaMBaddress(MBA)indicatinghowmany
`MBsareskipped.InH. ,foreveryskippedMBasinglebit(COD)is
`transmitted.
`H. OPTIONS
`AnH. -codermayuseoptionalcodingtechniques(\options")tofurther
`improveitsperformance.Optionshavetobenegotiatedwiththedecodervia
`externalmeans(forexampleaccordingtoITU-TH.).Therearefouroptions
`availableinH. ,whichwillbebrie(cid:13)yexplainedinthefollowing.
`
`IPR2018-01413
`Sony EX1011 Page 5
`
`

`

`(cid:15)UnrestrictedMotionVectormode(UMV-mode)
`(cid:15)AdvancedPredictionmode(AP-mode)
`(cid:15)PB-framesmode(PB-mode)
`(cid:15)Syntax-basedArithmeticCodingmode(SAC-mode)
`. Unrestrictedmotionvectormode
`InthedefaultpredictionmodeofH. ,motionvectorsmustnotreference
`pixelsoutsidethecodedpicturearea.Thisrestrictionneednotholdifthe
`UMV-modeisswitchedon.Whenapixelreferencedbyamotionvectorlies
`outsidethepicture,theclosestedgepixelisusedinstead.
`.Advancedpredictionmode
`Thisoptionincludesoverlappedblockmotioncompensationandthepossi-
`bilityoffourmotionvectorspermacroblock(insteadofone).IftheAP-mode
`isswitchedon,itisautomaticallyusedincombinationwiththeUMV-mode.
`.. Fourmotionvectorspermacroblock
`Themotionvectors(MV)foreachxluminanceblockareobtainedby
`addingpredictorstothetransmittedvectordi(cid:11)erences.Threeneighboring
`motionvectorsMV ,MVandMV areusedforprediction,asindicatedin
`Fig..Ifonlyonemotionvectorpermacroblockispresent,MV ,MVand
`MV arede(cid:12)nedasfortheupperleftblockinFig..Foreachdisplacement
`component(horizontalandvertical)thepredictionisthemedianvalueofthe
`threevectorsMV ,MVandMV .
`Themotionvectorsforbothchrominanceblocksarederivedbycalculating
`thesumofthefourluminancemotionvectorsfollowedbydividingthissumby
`.Theresultisthenroundedtowardsthenearesthalfpixelpositionusinga
`prede(cid:12)nedroundingtable.
`
`IPR2018-01413
`Sony EX1011 Page 6
`
`

`

`MV2
`
`MV3
`
`MV2 MV3
`
`MV1 MV
`
`MV2 MV3
`
`MV1
`
`MV
`
`MV2 MV3
`
`MV1
`
`MV
`
`MV1 MV
`
`Figure:MV-predictorsusedwhenMVperMBarepresent.\MV"indicates
`therelativepositionofthecurrent(cid:2)blockintheMB.
`..Overlappedblockmotioncompensation
`Inthismode,motioncompensationfortheluminancesamplesisalways
`performedona(cid:2)blockbasis.Eachpixelisaweightedsumofthreepredic-
`tionvalues.Inordertoobtainthethreepredictionvalues,threemotionvectors
`areused:themotionvectorofthecurrentluminanceblockandtwooutoffour
`remotevectors.Remotevectorsarethemotionvectorsofhorizontallyandver-
`ticallydirectlyadjacent(cid:2)blocks.Thesemotionvectorsdonotingeneral
`belongtothesameGOB.Foreachpixel,theremotemotionvectorsofthe
`blocksatthetwonearestblockbordersareused.Fig. showsthepositionof
`theremotemotionvectorsforthecurrentluminanceblock.
`Figure :Selectionoftheremotemotionvectors
`
`remote luminance
`block
`
`Current luminance
`block
`
`remote luminance
`block
`
`remote luminance
`block
`
`remote luminance
`block
`
`Macroblock
`
`IPR2018-01413
`Sony EX1011 Page 7
`
`

`

`4
`5
`5 5 5 5 5
`55
`5 5 5 5 5 5
`55666655
`5 5 6 6 6 6 5 5
`55666655
`5 5 6 6 6 6 5 5
`55555555
`4 5 5 5 5 5 5 4
`horizontal position in block
`
`Fig.showsthematrixofweightsusedforoverlappedblockmotioncom-
`pensation.Dependingonthepositionintheblock,theluminanceamplitudes
`aremultipliedbytheappropriateweightandthesumofthethreeweighted
`valuesisdividedby.Foradetaileddescriptionofthetheoryunderlying
`overlappedblockmotioncompensationreferto[].
`Figure:Upperleft:weightingvaluesforpredictionwithmotionvectorof
`currentluminanceblock.Upperright:weightingvaluesforpredictionwith
`motionvectorsoftheluminanceblocksontopandbottomofcurrentluminance
`block.Bottom:weightingvaluesforpredictionwithmotionvectorsofthe
`luminanceblockstotheleftandrightofcurrentluminanceblock
`H. alsode(cid:12)neshowtohandlespecialcasesthatoccurif
`(cid:15)oneofthesurroundingblockswasnotcodedorwascodedinINTRA
`mode;
`(cid:15)thecurrentblockisattheborderofthepicture;
`(cid:15)thecurrentblockisatthebottomoftheMBandhencethecorresponding
`MVshavenotbeendecodedyet.
`ThereaderisreferredtoAnnexFoftheH. standard[ ]formoredetailed
`informationonthesespecialcon(cid:12)gurations.
`
`4 5
`
`vertical position in block
`
`2 2 2 2 2 2 2 2
`112
`22211
`1
`1 1 1 1
`1 1 1
`11111111
`1 1 1 1 1 1 1 1
`11111111
`1 1 2 2 2 2 1 1
`22222222
`
`2 1 1 1 1 1 1 2
`22111122
`2 2 1 1 1 1 2 2
`22111122
`2 2 1 1 1 1 2 2
`22111122
`2 2 1 1 1 1 2 2
`21111112
`
`IPR2018-01413
`Sony EX1011 Page 8
`
`

`

`forward prediction
`
`. PB-framesmode
`APB-frameconsistsoftwopicturescodedasoneunit.TheP-pictureis
`predictedfromthelastdecodedP-pictureandtheB-pictureispredictedboth
`fromthelastandcurrentP-picture.Fig.illustratesthepositionsofP-and
`B-picturesandthepredictiondirections.
`Figure:PredictioninPB-framesmode
`InaPB-frame,thenumberofblocksperMBis ratherthan.Within
`eachMB,theblocksbelongingtotheP-pictureare(cid:12)rsttransmitted,followed
`bytheblocksoftheB-picture.OnlypartsofaB-blockarebidirectionally
`predictedasindicatedinFig..Forpixelswherethebackwardmotionvector
`pointstotheinsideofthecurrentP-macroblock,bidirectionalpredictionis
`used.AllotherpixelsoftheB-blockarepredictedusingonlytheforward
`motionvector.
`Figure:ForwardandbidirectionalpredictionofB-blocks
`
`forward
`prediction
`
`B-block
`
`bidirectional
`prediction
`
`P-macroblock
`
`P
`
`B
`
`P
`
`bidirectional
`prediction
`
`bidirectional
`prediction
`
`PB-Frame
`
`IPR2018-01413
`Sony EX1011 Page 9
`
`

`

`.Syntax-basedarithmeticcodingmode
`InthismodeofH. allvariablelengthcoding/decodingoperationswith
`Hu(cid:11)mantablesarereplacedbyarithmeticcoding/decodingoperations.By
`usingsyntax-basedarithmeticcoding,therestrictionofa(cid:12)xedintegralnumber
`ofbitsforeachsymbolisremoved.
`PERFORMANCECOMPARISON
`InthissectionwecomparetheratedistortionperformanceofH. and
`H. .Weusetheaveragedpeaksignal-to-noiseratio(PSNR)asadistortion
`measureforawholesequence,i.e.,(cid:12)rstwecalculatethePSNRforeachframe
`naccordingto
`PSNRn= log MMXi=
`
`(oi(cid:0)ci);
`( )
`whereMisthenumberofsamplesinaframe,andoiandciaretheamplitudes
`oftheoriginalandcodedframerespectively.ThenthePSNRvaluesforeach
`frameareaveragedforNframesinthesequenceaccordingto
`PSNR= NNXn= PSNRn:
`()
`Alldatapresentedinthefollowingwereobtainedforthe\Foreman"test
`sequenceinQCIFresolution(frames-).Allsequenceswerecodedata(cid:12)xed
`frame-rateusingthe(cid:12)xedquantizers ,,, , , ,andrespectively.
`Simulationswerecarriedoutusingavailablesoftwarecodecs[][].
`Theresultsarepresentedintwoparts.The(cid:12)rstpartcoverstheactual
`comparisonofH. andH. .Weinvestigatetowhatextendvariousim-
`provementscontributetotheoverallperformancegain.Inthispartwewill
`useeitheralloptionsofH. ornone.Inthesecondpart,weevaluatethe
`e(cid:11)ectivenessoftheH. optionscomparedtothedefaultmodeofH. ,thus
`providingamoredi(cid:11)erentiatedviewoftheperformancegainduetosingle
`options.
`
`IPR2018-01413
`Sony EX1011 Page 10
`
`

`

`TheresultsfromthecomparisonofH. andH. aresummarizedin
`Fig..Atabit-rateofkbps,thefollowingobservationscanbemade.
`Figure:PerformanceofH. andH. ataframe-rateof .fps.
`Curvesand showthatH. w/ooptionsoutperformsH. byapproxi-
`matelydB.AnotherdBisgainedifweusealloftheH. options(curve ).
`Twothirdsofthemaximumperformancegainareapparentlyduetofeatures
`notincludedintheH. options.
`Forfurtheranalysiscurvedemonstratestheimportanceofhalf-pelMCin
`H. .ItshowstheperformanceofH. whenwerestrictthemotionvectors
`tointeger-pelaccuracy.Adramaticlossof dBPSNRatkbpscanbe
`observed.However,itshouldbenotedthatthislossisnotonlyduetoless
`accurateMC.Whenusingintegermotionvectorsonly,theinherent(cid:12)ltering
`e(cid:11)ectofbilinearinterpolationislostaswell.ThisisalsothereasonwhyH.
`nowperformsevenworsethanH. .
`InordertocompareH. andH. underthesameconditions,withre-
`specttoMCand(cid:12)ltering,considercurveinFig..Itshowstheperformance
`ofH. withtheloop(cid:12)lterswitchedo(cid:11).Incurveandneitheraloop(cid:12)lter
`orhalf-pelMCisused.AtkbpsH. stilloutperformsH. by.dB.
`Thissmalladvantageisduetothe'other'minordi(cid:11)erencesmentionedatthe
`endofsection .
`
`1) H.263
`2) H.263 w/o options
`3) H.261
`4) H.263 w/o options, integer−pel ME
`5) H.261 w/o loop filter
`
`32
`
`64
`
`rate [kbps]
`
`128
`
`192
`
`1)
`
`2)
`
`3)
`
`4)
`
`5)
`
`35
`
`34
`
`33
`
`32
`
`31
`
`30
`
`29
`
`28
`
`27
`
`26
`
`25
`
`24
`
`23
`
`PSNR [dB]
`
`IPR2018-01413
`Sony EX1011 Page 11
`
`

`

`36
`
`36
`
`(b)
`(a)
`Figure:PerformanceoftheH. options'AdvancedPredictionmode'(a)
`and'Syntax-basedArithmeticCoding'(b).Theframe-rateis .fps.
`Advancedpredictionmode
`InFig.(a)theperformancegainduetotheH. options'Advanced
`Predictionmode'(AP)isillustrated.Becausetheunrestrictedmotionvector
`modeisautomaticallyincludedintheadvancedpredictionmode,thisoptionis
`notinvestigatedseparately.Atkbps,theAP-moderesultsinaperformance
`gainofapproximately .dB.Itshouldbementionedthatfourmotionvectors
`permacroblockwerenotusedveryfrequentlyduringthesimulations(lessthan
` %).Syntax-basedarithmeticcodingmode
`TheimprovementduetotheSAC-modeisverysmall,approximately.dB
`atkbps(Fig.(b)).BecauseSACissimplyadi(cid:11)erent(lossless)entropy
`codingscheme,thePSNRforagivenquantizerisune(cid:11)ected,butfewerbitsare
`produced.Intermsofreducedbit-rate,theaveragegainforinter-codedma-
`croblocksis -%.Forintra-codedmacroblocks,thegainishigher,onaverage
`about %.
`
`1)
`
`2)
`
`1) SAC−mode
`
`2) w/o options
`
`32
`
`64
`rate [kbps]
`
`128
`
`35
`
`34
`
`33
`
`32
`
`31
`
`30
`
`29
`
`28
`
`27
`
`26
`
`PSNR [dB]
`
`1)
`
`2)
`
`1) AP−mode
`
`2) w/o options
`
`32
`
`64
`rate [kbps]
`
`128
`
`35
`
`34
`
`33
`
`32
`
`31
`
`30
`
`29
`
`28
`
`27
`
`26
`
`PSNR [dB]
`
`IPR2018-01413
`Sony EX1011 Page 12
`
`

`

`PB-framesmode
`TheuseofPB-framesinH. istoincreasetheframe-ratewithoutin-
`creasingthebit-ratetoomuch.Typically,theframe-rateisdoubledwhenthe
`PB-modeisused.Considerasequencecodedat.fpsusingH. w/oop-
`tions(Fig. (a),curve ).Thoughthequalityofsingleframesisgood( .
`dBatkbps),thelowtemporalresolutionresultsinjerkymotion.However,
`increasingtheframe-rateto .fps(curve)whilemaintainingthebit-rate
`atkbps,resultsinansigni(cid:12)cantlossofimagequality( .dB).
`(a)
`(b)
`Figure :PerformanceoftheH. option\PB-framesmode"(a)andacom-
`binationofalloptions(b).
`Abettercompromisebetweentemporalandspatialresolutionispossible
`whenthePB-modeofH. isused.Curve inFig. (a)showstheperfor-
`manceofthePB-modeat .fps.BecausethequalityofP-andB-frames
`di(cid:11)erssigni(cid:12)cantly,theaveragedPSNRaccordingtoequationiscalculated
`separately(curve aand b).Ascanbeseen,thePSNRforP-framesdrops
`onlyby.dBcomparedtocurve .NotethatthesamenumberofP-frames
`persecond(.)arenowtransmittedwithonlylittlelossofquality.With
`theuseoftheB-frames,however,theframe-rateisdoubled.Thoughthequal-
`ityoftheB-framesislow,theyprovidethesubjectiveimpressionofsmooth
`motion.Accordingtothisconcept,theamountofbandwidthallocatedtothe
`B-partofaPB-frameiskeptlow,onaverageabout -%.Infact,H.
`
`64
`rate [kbps]
`
`1)
`
`2)
`
`3a)
`
`3b)
`
`1) w/o options (6.25 fps)
`
`2) w/o options (12.5 fps)
`
`3) AP−PB−SAC (12.5 fps)
`
`a) P−frames
`
`b) B−frames
`
`32
`
`128
`
`36
`
`35
`
`34
`
`33
`
`32
`
`31
`
`30
`
`29
`
`28
`
`27
`
`26
`
`PSNR [dB]
`
`1)
`
`3a)
`
`2)
`
`3b)
`
`1) w/o options (6.25 fps)
`
`2) w/o options (12.5 fps)
`
`3) PB−mode (12.5 fps)
`
`a) P−frames
`
`b) B−frames
`
`32
`
`64
`rate [kbps]
`
`128
`
`36
`
`35
`
`34
`
`33
`
`32
`
`31
`
`30
`
`29
`
`28
`
`27
`
`26
`
`PSNR [dB]
`
`IPR2018-01413
`Sony EX1011 Page 13
`
`

`

`speci(cid:12)esthataB-macroblockalwayshastobequantizedmorecoarselythan
`itscorrespondingP-macroblock.
`BecausethequalityoftheB-framesisintentionallykeptlow,theneedfor
`e(cid:11)ectivebidirectionalpredictionisnotasbig.Instead,thedesignofPB-frames
`focusesonlessoverheadfortheB-part,whichisveryimportantforlowbit-
`rates.Hence,PB-framesinH. arenotmeanttoworkinthesamewayas
`e.g.B-framesinMPEG[]andwillactuallybelesse(cid:11)ective.Thisisbecause
`noindependentforwardandbackwardmotionvectorscanbeusedforthebidi-
`rectionalprediction.Instead,asmalldelta-vectorisusedtoobtaintheforward
`andbackwardmotionvectorsfromascaledversionoftheP-vector.Thisre-
`duced(cid:13)exibilitydoeslimittheperformanceandwillalsoreducetheoptimum
`sizeofthesearchrangeforthedelta-vector.Increasingthesearchrange(which
`ispossiblewithinthestandard)doesnotguaranteeabetterprediction.Infact,
`duetomoresideinformationtheperformancemightdecrease.Wewillreferto
`thisobservationwhenweconsidercomplexityissuesinsection.
`Fig. (b)showstheperformanceofH. whenalltheoptionsareused.
`Again,curve aand bshouldbecomparedtoH. w/ooptionsathalfthe
`frame-rate(.fps).Ascanbeseen,thequalityoftheP-framesatkbps
`increasesby.dBwhiletheframe-rateisdoubledfrom.fpsto .fps.
`Becausethein(cid:13)uenceoftheSAC-modeissmall,mostoftheperformancegain
`isduetothecombinationoftheAP-andPB-mode.
`COMPUTATIONALCOMPLEXITY
`Sofarweinvestigatedtheperformancegainregardlessoftheinvolvedcom-
`plexity.Foragivenapplicationhowever,thetrade-o(cid:11)qualityvs.complexity
`hastobeconsideredaswell.Inthissectionwetrytogivesomeindicationof
`theinvolvedcomplexity.Similartotheprevioussection,wewill(cid:12)rstcompare
`H. initsdefaultmodewithH. ,andthenfocusontheH. options.
`. ComparisonH. vs.H.
`ThoughtheratedistortionperformanceofH. w/ooptionsissigni(cid:12)-
`cantlybetterthanH. ,thecomplexitydoesnotdi(cid:11)ertoomuch.Onone
`hand,H. requiresadditionalcomplexityforhalf-pelMEandMC.Compa-
`redtointeger-pelME,however,theadditionalcomplexityduetothehalf-pel
`re(cid:12)nementsearchandbilinearinterpolationistypicallysmall.Ontheother
`hand,H. doesnotrequireanyoperationsforloop(cid:12)ltering,likeH. .
`
`IPR2018-01413
`Sony EX1011 Page 14
`
`

`

`WeconcludethatH. w/ooptionsisane(cid:11)ectivealternativetoH. ,
`sinceasigni(cid:12)cantperformancegainisprovidedforaminorincreaseincom-
`plexity.
`.ComplexityofH. options
`Inthefollowingwetrytoassesstheadditionalcomplexitywhensingleop-
`tionsofH. areused.Togetherwiththeknowledgeofthecorresponding
`performancegain,thiswillbeusefultodecidewhichoptionsshouldbesuppor-
`tedbyacoderforagivenapplication.Inadditiontosomebasicconsiderations,
`weprovideresultsfrompro(cid:12)lingaH. softwarecodeconaSunWorkstation
`(SuperSPARC,MHz, MIPS,MBmemory,Solaris. OperatingSys-
`tem).Ofcoursethesedatahavetobeusedwithgreatcare,sincetheydepend
`heavilyontheimplementationandmaynotgeneralize,especiallyforhardware
`implementations.Thecoderwewereusinginoursimulations[]wasbased
`onTestModelNumber(TMN-).Besidesotherissueslikeratecontroland
`modedecision,TMN-doesalsospecify,thatmotionvectorsareobtainedby
`fullsearchmotionestimation.Becausefullsearchmotionestimationiscompu-
`tationallyintense,thecomplexityofthecoderisdominatedbythisoperation.
`TheH. softwarecodecwaspro(cid:12)ledunderthefollowingconditions.We
`usedthesameimagesequenceasinsection(frames- )andaconstantbit-
`rateof kbpswhilemaintaininga(cid:12)xedframe-rateof fps.Singleoptions
`ofH. orcombinationsofthemwereusedinsixsimulations,whichwillbe
`abbreviatedasfollows:
`(cid:15)NO:Nooptions
`(cid:15)AP:AdvancedPredictionmode
`(cid:15)PB:PB-framesmode
`(cid:15)SAC:Syntax-basedArithmeticCodingmode
`(cid:15)AP-PB:BothAPandPBareused
`(cid:15)AP-PB-SAC:AP,PBandSACareused
`TheresultsaresummarizedinFig. andarediscussedinthefollowing
`paragraphs.Thecase\nooptions"(NO)servesasthebaselineinourdiscus-
`sions.
`
`IPR2018-01413
`Sony EX1011 Page 15
`
`

`

`Figure :ComparisonoftheconsumedCPU-timeforsingleoptionsofH.
`orcombinationsofthem.Thedashedlinerepresentsreal-time.Graybars
`indicatetheamountoftimeconsumedbymotionestimation.
`Advancedpredictionmode
`The(cid:12)rstsimulation(AP)showsthattheperformancegainofapproximately
` .dB(atkbps)hastobepaiedbyaconsiderableincreaseincomplexity.
`BecausethecodermayeitheruseonemotionvectorperMB(MV)orfour
`(MV -MV),all(cid:12)vemotionvectorshavetobecalculatedtoenableane(cid:11)ective
`decision.TheadditionalcomplexityforMEhowever,stronglydependson
`theimplementation.Thereferedsoftwarecoderforexample,usedasmall
`re(cid:12)nementsearchtoobtainMV -MVfromMV,whereasTMN-speci(cid:12)esa
`fullsearchMEforeachmotionvector.
`Inaddition,extracomplexityisintroducedbyoverlappedmotioncompen-
`sation.Aweightedsumofthreepredictionvalueshastobecomputedtoyield
`the(cid:12)nalpredictionvalue.Becausethishastobedoneforeverypixel,the
`computationalcostishigh.IncontrasttoME,whichisonlydoneattheco-
`der,overlappedmotioncompensationdoesalsoincreasethecomplexityofthe
`decoder.
`PB-framesmode
`WhencomparingthePB-framesmode(PB)tothedefaultmode(NO),the
`computationtimeforthecoderdecreases.Thereasonisthelessexpensive
`motionestimationfortheblocksofaB-frame.AfullMEisdoneonlybetween
`twoP-frames-everyotherframe.IntheB-framesasmallersearchrangeof
`typically(cid:6)pelsisused.Aswehavediscussedinsection,thelimitedsearch
`rangeismainlyselectedduetoperformanceconsiderations,notcomplexity.A
`biggersearchrangewillnotnecessarilyincreasetheperformance.Notethatin
`oursimulationstheframe-rateofthePB-modeisthesameasinthedefault
`mode.IfthePB-modewouldbeusedtodoubletheframe-rate,asdiscussedin
`
`CPU-time [s]
`
`Coder
`
`AP-PB-SAC
`
`AP-PB
`
`SAC
`
`PB
`
`Options
`
`AP
`
`NO
`
`70
`
`60
`
`50
`
`40
`
`30
`
`20
`
`10
`
`0
`
`CPU-time [s]
`
`Decoder
`
`AP-PB-SAC
`
`AP-PB
`
`SAC
`
`PB
`
`Options
`
`AP
`
`NO
`
`01234567
`
`IPR2018-01413
`Sony EX1011 Page 16
`
`

`

`section,thecomplexitywouldincreasesigni(cid:12)cantly.Forthesameframe-rate
`however,thereducedcomplexityisapositivesidee(cid:11)ect.
`Again,themorecomplexprediction(averagingvaluefromtwoframes)does
`increasethecomplexityatthecoderanddecoder.Becausethisisdoneonly
`everyotherframe,anditislesscomplexthanoverlappedmotioncompensation,
`theincreaseincomputationtimeatthedecoderislessthanintheAP-mode.
`AnotherparticularlyimportantpointtomentionforPB-framesisthein-
`creaseofthedelaycausedbythisoption.Onthecodersidetwoframeshave
`tobestoredinmemoryandarecodedasoneunitwhichcausesanextradelay
`ofoneframe.Assumingframeskippingtoreducebit-rate,thisdelaycanbeas
`muchasormoreframesoftheoriginalsequence.Again,ifthePB-modeis
`usedtodoubletheframe-rate,noadditionaldelayisintroducedcomparedto
`thedefaultmode.
`Syntax-basedarithmeticcodingmode
`Theimprovedperformanceduetosyntax-basedarithmeticcoding(SAC)
`ofapproximately.dBresultsinanincreasedcomplexityforthecoderand
`decoder.Becausetheconsumedtimebymotionestimationdominatesthe
`overallcomputationalcomplexityatthecoder,therelativeincreasemaybe
`neglected.Forthedecoder,however,theexecutiontimeincreasesnoticeably.
`Combinationofoptions
`ThelasttwobarsinFig. showthecombinationofseveraloptions.The
`combinationoftheAP-andPB-moderesultsinacomplexitysimilartothe
`defaultmode.Consideringthecorrespondingperformancegain,thiscombina-
`tionseemstobeagoodchoice.AddingSACtothiscombinationdoesincrease
`complexity,butthecorrespondingperformancegainissmallasdiscussedin
`section.
`CONCLUSIONS
`Thesimilaritiesanddi(cid:11)erencesbetweenthedraftinternationalstandard
`ITU-TH. andthewellknownandwidelyusedITU-TRecommendation
`H. havebeenpresented.Mandatoryenhancementsandoptionalcoding-
`modeshavebeenevaluatedwithrespecttoimagequalityandcomplexity.Ithas
`beenshownthathalf-pelmotioncompensationisoneofthemainadvantages
`ofthedraftstandard.ThereforeH. providesabetterimagequalitythan
`H. ,evenwhenitisusedinitsdefaultmode.Furthermorethecomplexity
`
`IPR2018-01413
`Sony EX1011 Page 17
`
`

`

`ofH. inthiscon(cid:12)gurationisonlymarginallyhigherthanthecomplexityof
`H. .UsingtheAP-modeofH. doesincreasetheexecutiontimeofthecoder
`anddecoder.ForthePB-modehoweverareducedcomplexityatthecoder
`sidecanbeobservedduetolessexpensivemotionestimationintheB-frames.
`Atthedecoderthecomplexityisincreased,butlessthanintheAP-mode.
`ThecorrespondingperformancegainandthespecialuseofB-framesinH.
`havebeendiscussed.Consideringthetrade-o(cid:11)qualityvs.complexity,the
`combinationofAP-andPB-modeseemtobeagoodcompromise.Syntax-
`basedarithmeticcoding,ontheotherhand,o(cid:11)ersonlylittleperformancegain
`foritsincreasedcomputationalcomplexity.
`ACKNOWLEDGMENT
`ThispaperisdedicatedtoProf.Dr.-Ing.H.G.Musmannontheoccasion
`ofhisthbirthday.Hehasbeenaninspirationtomanyimagecompression
`researchers,includingtheauthorsofthispaper.Hisunerringvisionovertwo
`decadesthatmovingvideocanbetransmittedovertelephonelineshasbeenan
`importantcontributiontothestandardsreviewedinthispaperthatisgratefully
`acknowledged.
`REFERENCES
`[ ]H.G.MusmannandJ.Klie,\TVTransmissionUsingakbit/sTrans-
`missionRate",InternationalConferenceonCommunications,pp. . . -
` . .,  .
`[]\VideoCodecforaudiovisualservicesatpxkbits/s",ITU-TRecom-
`mendationH. ,Geneva, .
`[ ]\VideoCodingfornarrowtelecommunicationchannelsat<kbit/s",
`DRAFTITU-TRecommendationH. ,April .
`[]TMN-Testmodel
`forH. ,TelenorResearch,
` (Internet:
`http://www.nta.no/brukere/DVC/).
`[]H. softwarecodec,PortableVideoResearchGroup,Stanford(Internet:
`ftp://havefun.stanford.edu/pub/p/).
`[]B.Girod,\Motion-CompensatingPredictionwithFractional-PelAccu-
`racy",IEEETransactionsonCommunications,vol. ,no.,pp.- ,
`April .
`
`IPR2018-01413
`Sony EX1011 Page 18
`
`

`

`[]ISO/IEC  (ITU-TH.),\Genericcodingofmovingpicturesand
`associatedaudioinformation,Part :Systems,Part:Video,Part :
`Audio",March .
`[]M.T.Orchard,G.J.Sullivan,\OverlappedBlockMotionCompensation:
`AnEstimation-TheoreticApproach",IEEETransactionsonImagePro-
`cessing,vol. ,no.,pp. - ,September .
`
`IPR2018-01413
`Sony EX1011 Page 19
`
`