•
`
`•
`
`•
`
`IPR2021-00827
`Unified EX1010 Page 260
`
`

`

`j
`
`•
`
`•
`ress1on
`
`CHAPTER
`
`•
`
`•
`
`•
`
`•
`
`•
`
`IPR2021-00827
`Unified EX1010 Page 261
`
`

`

`•
`
`242
`
`. . ,,
`.
`~-
`
`...
`
`. . ,. .......
`. .
`-~-~
`. •. .
`
`Introduction
`
`Chapter 14
`
`The DV compression scheme is the product of cooperatio11 among a
`large number of 1nanufacturers , but with four co1npanies perfor111-
`ing most of the development
`and owning most of the intellectual
`property . It began , I am told , as an exercise to produce a compression
`scheme for use in high-definitio11 consumer ca1ncorders , using a data
`rate of 50 Mbits /s. It soon became apparent
`th at the work presented
`an opportunity
`for a new generation
`of standard-definition
`cam (cid:173)
`corders working at 25 Mbits /s and off erin g th e "digital'" cachet The 25
`Mbits/s rate was very suitable , becau se reco rd in g at this rate can be
`implemented quite easily with robust , in exp en sive technology .
`Even then , DV exceeded its expect ation s; it is sometimes described
`as the technology that went from en gi11eers to u ser s before
`the mar(cid:173)
`is it suc(cid:173)
`keting departments realized what wa s happ enin g! N ot only
`cessful in the consumer market , but the techn o logy and its derivative s
`have been adopted in the profession
`televisio n broadcast market
`in
`fact, video recorders based 011 DV it is claim ed that have been adopted
`more rapidly than any other format
`. Two vers ions of DV are in use
`professionally. 25 Mbits / s systems , very similar
`to the consumer D' '
`products, are used for acquisition , particularly
`in news environments.
`A 50 Mbits/s version is used in studios and f or po stproduction
`.
`Altho·ugh DV is based on the DCT transform , ma11y aspects are
`quite different
`from
`the MP EG approach
`. Before
`looking at che
`details, it is importan .t to review the design criteria , remembering
`that
`. We have seen chac
`the original intention was a consumer product
`motion estimation is a very expensive process most suitable
`for an
`(few encoders , many decoders ). For a con(cid:173)
`asymmetric environment
`sumer product , it was determjned
`that there was no sufficiently
`eco(cid:173)
`~omical system of temporal
`compression
`, so DV had
`to be an
`1?traframe system. This approach allows simple editing , also a substan(cid:173)
`t1al advantage.
`When one is recording on a videotape , it is most helpful
`if the b~t
`stream to be recorded has a constant number of bits per frame. Th 15
`was also a design criterion for DV.
`MPEG can of course work with I-frames only. However,
`the ~{PEG
`model controls bit rate by a feedback system. It is possible to approxj(cid:173)
`mate constant-bit rate compression with MPEG (as specified for the D-
`ff·
`·
`10 format m
`·
`eral
`ent1oned in the previous chapter), but stu mg 1s gen
`-
`1
`h.
`.
`.
`Y necessary t
`o ac 1eve an exact f 1xed b1 t rate.
`
`IPR2021-00827
`Unified EX1010 Page 262
`
`

`

`DV Compression
`
`I
`243
`
`s would argue that fixed bit rate is, in fact, unde(cid:173)
`MPEG enthusiast
`sirable . As we discussed earlier , tl1e human psycl1ovisual system is
`1nore sen sitive to changes in perceived quality than it is to absolute
`quality . Constant bit rate with images of varying complexity necessar(cid:173)
`ily implies va1·iable quality of the compressed image. DV proponents ,
`011 the other hand , show research results that supposedly demonstrate
`that DV techniques give a lower mi11imum quality in a sequence of
`frames with varying complexity . The arguments will continue!
`Fi11ally, the fact
`that DV was intended
`for con sumer equipment
`1neant cost effectivene ss was a prime design goal. It was required that
`comp1·ession be performed on a single chip , and the hope was that the
`same chip could be u sed for both compression a11d decompression .
`The se goals were achieved.
`
`'"-----U1: Basic Concepts of DV Compression
`
`DV use s tl1e D CT transform ; it also performs quantizatio11 of the
`, as in JPEG and MPEG. However , DV is specifically
`coefficients
`designed
`to generate a fixed number of bits from each frame . It uses
`some very sophisticated
`techniques
`to ensure that bit allocation is
`optimal
`for every frame .
`in detail below , but we start with a brief
`The process
`is described
`summary. The f i1·st step , if necessary , is to reduce the 11un1ber of
`samples. For 25 Mbits /s consu1ner prodt1cts there are
`color difference
`different
`approaches
`for 525/ 60 and 625/50 video. For 525/60 systems
`the video (normally 4:2:2) is first reduced to 4:1:1. This is an appropriate
`the video will eventually be coded to NTSC. For 625/50
`choice when
`that DV video should be compatible . with the
`systems
`it was decided
`using the Digital Video Broadcast1:1g (DVB)
`MPEG-2 video transmitted
`system
`adopted
`i11 Europe for satellite , cable, and terrestrial. trans-
`.
`.
`.
`d F
`h
`of essional version of
`so 4:2:0 coding was selecte
`. or t e pr
`d
`m1ss1on
`.
`.
`4.1.1 ·
`sed for both 525/60 an
`1
`is u
`Mbits/s
`DV compression
`described be ow ,
`· ·
`50
`625/50 at 25 Mbits / s; 4:2:2 is used for both standards for
`
`. to smaller pieces, each of
`.
`compression.
`The next step is to break the image llp in DV uses macroblocks ,
`which is allocated a fixed nu~ber of by~~:ks make a segment, and
`much as JPEG and MPEG do . F1ve macrob
`ber of bytes. The
`d to the same num
`f
`. different parts o
`f
`each segment must be compresse
`roin
`i11 one segment are taken
`five macroblocks
`
`IPR2021-00827
`Unified EX1010 Page 263
`
`

`

`244
`
`Chapter 14
`
`tl1at no seg1ne11t gets 1nore than its
`
`tl1e f ratne , in a11 atten1pt to ensure
`sl1are of complexity.
`Each 8x8 block in the seg1nent is DCT tra11sformed and analyzed
`for complexity a11d, depending 011 the complexity , is allocated to one
`of four banks of quantizatio11 tables . Each bank is optimized
`for com(cid:173)
`pressio11 of blocks in a particular
`e11ergy range . There are effectively
`16 tables in each bank , and eacl1 of these corresponds
`to a level of
`quantization
`scaling. Tl1e syste1n the11 quantizes
`the coefficients of all
`blocks in the segn1ent , t1si11g a table f ron1 tl1e correct bank for each
`block. This is perfor1ned for each of the 16 possible quantization-sca l(cid:173)
`ing factors , and the system selects the scaling factor closest to , but n ot
`exceeding, 385 bytes of quantized
`is
`coef ficie11ts. Thus , each segment
`con1pressed to an equal nun1ber of b y te s, but within
`that segment
`each block is quantized using a table appr opriate
`to the energy of that
`block. Comp lex blocks receive a great er share , and simple blocks a less(cid:173)
`er share , of the bits available . Ther e is a third mechanism
`that help s
`distribute
`the bits optimally ; this is discussed
`in the more derailed
`description that follows .
`
`I•
`
`•
`
`Detailed Description
`
`25 Mbits /s Compression
`
`is specified by IEC document
`DV compression for consumer products
`61834. The DV-based compression
`used
`for professional
`produces
`described in this section is specified by SMPTE Standard 31411, and
`this standard , by
`figures used in this section are reproduced
`from
`kind permission of the Vice President
`, Engineering , of SMPTE These
`two documents are the definitive descriptions of DV compression and
`recommended reading for anyone working with DV video.
`Prior to compression
`to the 25 Mbits/s professional
`standard ., _the
`signal must be reduced from the (normal) 4:2:2 coding
`to 4:1:1 ~odJng.
`The DV standard does not specify any filtering, but merel! discarct;
`pixels from the 4:2:2 input , as shown
`in Figure 14-1 and F1g~re 14-(cid:173)
`n~a}T
`Depending on the source of the images, appropriate
`prefilter~g
`be necessary to prevent artifacts being generated by this decimaaon
`process.
`
`IPR2021-00827
`Unified EX1010 Page 264
`
`

`

`DV Compressio11
`
`Figure 14-1
`Reduction to 4: I : I
`sampling for 525/60
`systems.
`
`l / 13.5Ml-Iz
`
`Luminance
`
`(Y)
`
`First active lin e ->
`in a field
`
`>
`
`I
`
`245
`
`• • •
`
`'
`•
`
`•
`•
`•
`•
`•
`•
`•
`
`•
`
`•
`•
`•
`•
`•
`•
`•
`
`•
`
`•
`•
`•
`•
`•
`•
`•
`
`•
`
`•
`•
`•
`•
`•
`•
`•
`
`•
`
`•
`•
`•
`•
`•
`•
`•
`
`•
`
`•
`
`•
`
`•
`•
`•
`•
`•
`•
`•
`
`•
`
`9'.'
`
`-
`
`-
`
`....._ __ LI..Dl / 6.75MHz
`• ---
`~ --
`• --
`•
`•
`• --
`•
`•
`• •
`r --
`•
`•
`-----
`. --
`•
`•
`. . . . . . . .
`•
`•
`. . .
`._ ~-
`• • • • •
`•
`•
`. . .
`... --
`.
`•
`.
`•
`.
`. .
`.
`.
`.
`.
`.
`
`····6 0(~--~~-: -~ -------)66666666 ······
`
`•
`
`····00(
`
`::~: : )00000
`
`Color difference
`
`First act ive Ii11e ~>
`in a f ield
`>
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`r
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`• •
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`• '
`' '
`• •
`• '
`• '
`'
`'
`'
`'
`'
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`~
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`(CR, C8 )
`-~:-=:~-~:-· --..)··1··6 ··;··6 ·-L,() .. l .. 6······
`.... (). 4--c,....._:
`• -
`•
`•
`•
`•
`,.. •
`~ •
`•
`• • •
`'--
`•
`-
`-
`'
`•
`)··~··O ··~··O··~··O ··~··O······
`•
`~ ==~:: ~= •
`····O··~··C
`)··~··O··~··O··~··O··~··O······
`,,-=-.--~
`• •
`• • •
`• • • •
`•
`•
`•
`
`Line 285
`Line 23
`Line 286
`Line 24
`0 00 ······ Line 287
`Line 25
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`Line 285
`Line 23
`Line 286
`Line 24
`Line 287
`Line 25
`
`-
`
`"
`
`~
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`1111 1-: _ First pixel in active period
`
`Wh-axxi c• . •LLere O: Tran smit ting sampl es
`0 : Discarded samples
`
`· :: :: OllO blocks and macro blocks. In the con(cid:173)
`The image is then divided int c
`sumer 625/ 50 standard with 4:2= 11 ,: ==::O coding , this process is straightfor-
`=: ::1:1 coding , however , presents a slight
`ward , and the same as MPBG. 4:
`- - 180 samples of each color difference
`prob lem , because there are (Jnly
`sig11al on a line , and 180 is l)_ot c1.__L.( l lli3ivisible by 8; we are left witl1 a "h~lf
`block " at the end of the li~ s
`his is resolved by creating a special
`· L. oure 14 3
`"end" macroblock , as shown 'i~ ·p·
`" ,( 1 "'l sa&.a .,_ o
`'
`- '
`into superblocks, as shown i~ Figures
`The macroblocks are gro~ ed ·
`"'
`the se~ P er.....n;,.n:::its are formed by takin_g five mac-
`14-4 and 14-5. Then
`roblocks for each segment . {!2
`t are assembled 1n a pseudo-
`.
`.
`'\'he . ·~... ..
`segmen s
`k
`f. e macroblocks are ta en
`random manner w1th1n a
`h
`•
`
`from d ifferent positions in ~egi .. -:_-c_·c-•rwtiient t e
`rblocks no two in the
`iv
`t•
`a•fferent supe
`1
`same row or colun1n of su~ 1ve
`h.
`e the segment con-
`
`1 1 1
`
`I
`
`.
`
`.
`'5, Sl=#xt::
`
`bl k· (four lum1nan
`: . X OCS
`
`IPR2021-00827
`Unified EX1010 Page 265
`
`

`

`•
`
`I
`
`, 2~6
`-
`
`-
`
`Figure 14-2
`Reduction to 4: 1 : 1
`sampling for 625/50
`systems.
`
`116.75MHz
`
`Luminance
`
`(Y)
`
`>
`
`Chapter 14
`
`Line 33.5
`Line 23
`
`Line 336
`Line 24
`
`······ Line 337
`Line 25
`
`00000
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`,
`
`.
`
`'
`
`,
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`,
`
`'
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`•
`
`,
`
`'
`.
`
`-
`
`,
`•
`•
`• • "
`•
`
`I
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`,
`
`'
`,
`•
`•
`•
`•
`
`'
`,
`
`•
`•
`•
`•
`
`····000000
`• • • • • • • • • • • •
`• • • • • • • • • • • •
`• • • •
`• • • • •
`• • • • • • • •
`• • • • • • •
`····O··~··O··~··O··~··O··~··O··~··O······
`Line 335
`-
`l ine 23
`• -
`-
`-
`-
`-
`-
`"
`····O ··~··O··~··O ··f ··0 ··1··0 ··f ··O······
`'Line 336
`line 2-f
`····O··}··O··~··O··} ··O·· ~-·O··}· ·O·· ····
`Line 337
`Line 25
`-
`-
`-
`-
`-
`"
`
`Color difference
`
`(CR, C 8)
`
`First active line -->
`in a field
`>
`
`,
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`
`•
`•
`•
`•
`•
`•
`•
`•
`
`I
`
`'
`,
`
`-
`
`-
`•
`•
`•
`•
`
`,
`
`•
`•
`•
`•
`
`•
`
`First pixel in active period
`
`Wl1ere O: Transmitting
`samp Jes
`0 : Discard ed sampl es
`
`one of each color difference block ) in each macroblock . Each pixel
`value is expressed as an 8-bit word , so the segment comprises 30X8X8
`= 1920 bytes of data.
`Each block is OCT transformed. DV provides for both frame and
`field OCT coding , but unlike MPEG-2, the decision is made for each
`block, and affects only that block. The two modes are known as 8-8-
`DCT, used for blocks where there is little co11tent variation bet\veen
`the two fields, and 2-4-8-DCT, used where the content variation is sig(cid:173)
`nificant (usually as a result of motion in that area of the image ). In D\ 7
`a 2-4-8-0CT is coded as two 8X4 blocks ; the top block is the sum ?f
`adjacent rows of pixels, the bottom block the difference between adp(cid:173)
`cent rows. This is illustrated in Figure 14-6.
`
`IPR2021-00827
`Unified EX1010 Page 266
`
`

`

`DV Compression
`
`•
`
`i. ·: "1
`
`r.:::J c ·. J
`figure 14-3
`Arrangement of OCT
`blockS for 4: I: I
`encoding.
`
`...
`8 lu1es
`
`8 lin es
`
`•
`
`•
`
`I
`
`I
`
`I
`
`1
`I
`I • ' I
`I •
`• I
`I •
`
`247
`
`•
`
`Lumina11ce DCT block
`
`Left
`
`Top
`
`Color d ifference OCT block
`
`Left
`
`Top
`
`90 DCT blocks
`
`Right
`
`•••••• ••••••••• •••••••
`
`..
`
`22.5 DCT blocks
`
`••••••• •••• •••••••••••
`
`••••••• ••••••••••••• •
`
`\
`
`'
`' '
`8x8 pixels
`Right
`
`/
`
`/
`
`16x4 pixels
`
`IPR2021-00827
`Unified EX1010 Page 267
`
`

`

`248
`
`L~- ~ .. - _, ___ __,
`
`C'" :··3 C: ::J t:' _J
`Figure 14-4
`Superblocks and
`macroblocks in one
`television frame for
`525/60 system with
`4: I : I compression .
`
`Left
`
`0
`
`50 ,0
`S1 ,0
`S2 ,0
`S3 ,0
`S4 ,0
`S5 ,0
`
`l
`so ,1
`SI , I
`
`S2 ,1
`53 ,1
`
`S-i I
`
`I
`
`S5 ,1
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`l
`
`1'op
`
`•
`1
`
`0
`1
`
`2
`
`3
`4
`
`Chapter 14
`
`'
`.\
`
`720 pixels
`3
`so ,3
`S1 ,3
`
`S2 ,3
`
`S3 ,3
`
`S4 ,3
`
`S5 ,3
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`Rjght
`
`4
`so ,4
`SJ ,4
`
`52 ,4
`
`53 ,4
`
`S4 ,4
`
`•
`
`J >
`2
`so ,2
`Sl ,2
`
`I
`
`I
`
`S2 ,2
`
`S3 ,2
`S4 ,2
`
`55 ,2
`
`I
`
`I
`
`I
`
`I
`
`-180 I ines
`
`B0tton1
`
`5
`6
`
`7
`8
`9
`
`I
`
`I
`I
`I
`
`S6 10
`S7 ,0
`
`S8 ,0
`S9 ,0
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`l
`
`I
`
`S6 ,J
`S7 ,I
`
`S8 ,1
`$9 ,1
`
`S6 ,2
`
`S7 ,2
`
`S8 ,2
`
`S9 ,2
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`56 ,3
`
`Sl ,3
`
`$8 ,3
`
`S9 ,3
`
`' "'
`
`1 su perblock = 27 111acro blocks
`/
`~
`r---... ~
`
`S5 ,4
`S6 4
`
`I
`
`S7 ,4
`58 ,4
`
`S9 4
`'
`
`I
`I
`
`I
`I
`
`St1per blocks j = 9 :
`j = 3 : I
`
`V
`
`/
`
`/
`
`"
`
`...... r-
`
`0
`
`1
`
`3
`
`4
`
`2
`
`•
`
`[' • I
`
`- . .
`Figure 14-5
`Superblocks and
`macrobfo cks in one
`television frame for
`625/50 system with
`4: 1: l compression .
`
`Left
`
`0
`so ,0
`Sl ,0
`S2 ,0
`S3 ,0
`
`S4 ,0
`
`S5 .,0
`S6 ,0
`
`Tep
`
`• l
`
`0
`
`J
`
`2
`
`3
`
`4
`5
`6
`
`1
`so ,1
`S 1 , l
`S2 ,1
`S3 ·1
`
`I
`
`S4 ,1
`
`$5 , 1
`
`I
`
`I
`
`I
`
`I
`
`I
`r
`
`I
`
`I
`
`l
`
`I
`
`I
`
`l
`
`I
`r
`l
`
`---'-=> ~ 720 pixe.ls
`2
`3
`so ,2
`so ,3
`S'I 2
`SL ,3
`
`I
`
`S2 ,2
`
`S3 ,2
`
`$4 ,2
`
`55 ,2
`S6 ,2
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`Right
`
`4
`so 14
`SI ,4
`S2 14
`53 ,4
`
`S4 ,4
`S5 ,4
`
`S6 ,4
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`S2 ,3
`
`$3 ,3
`
`S4 ,3
`
`S5 ,3
`S6 ,3
`
`576 lines
`
`7
`
`8
`9
`10
`
`Bottom 11
`
`S7 ,0
`
`S8 ,0
`S9 ,0
`
`S 10, 0
`S 11, 0
`
`S6 ,1
`S7 ·1
`,
`
`58 , 1
`
`S9 , 1
`
`I
`
`l
`
`I
`S 10, 1
`r S J I, 1
`
`I
`
`I
`
`I
`
`I
`
`I
`
`S7 ,2
`
`S8 ,2
`
`59 ,2
`S 10, 2
`S I 1, 2
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`S7 ,3
`
`S8 ,3
`
`59 , 3
`S 10, 3
`5 11, 3
`
`S7 ,4
`
`S8 ,4
`
`S9 .,4
`S I 0, 4
`
`S J 1, 4
`
`•
`
`~
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`I
`I
`I
`
`"'
`Sup er block s i = 11 l
`• 3 I
`1 st1peT bl ock= 27 macro blocks
`-
`J -
`/ ~
`
`I/
`
`I/
`
`/
`
`I'-.
`
`~
`
`t'-......
`' I'-
`
`0
`
`1
`
`2
`
`3
`
`I '
`
`4
`
`IPR2021-00827
`Unified EX1010 Page 268
`
`

`

`DV Compressio11
`
`Figure 14-6
`The two OCT mo des
`of DV, and the
`scanning orde r of the
`resulting OCT
`coefficients .
`
`•
`
`8-8-DCT
`hor izontal
`:>
`2 3 4 5 6 7
`6 7 15 16 28 29
`8 14 17 27 30 43
`13 18 26 31 42 44
`19 25 32 41 45 54
`24 33 40 46 53 55
`34 39 47 52 56 61
`38 48 51 57 60 62
`49 50 58 59 63 64
`
`0 1
`0
`1 2
`I
`3 5
`2
`4 9
`3 10 12
`4 11 20
`5 21 23
`6 22 35
`7 36 37
`
`Vertica J
`
`(sum )
`
`0
`l
`2
`3
`
`l
`0
`1 3
`5 9
`11 15
`13 27
`
`249
`
`2-4-8-Der
`
`horizontal
`
`-- ~
`2 3 4 5 6
`7 19 21 35 37
`17 23 33 39 49
`25 31 41 47 55
`29 43 45 57 59
`
`7
`51
`53
`61
`63
`
`Vertical
`'¥ 4
`2 4 8 20 22 36 38 52
`5
`6 10 18 24 34 40 50 54
`(differen ce) 6 12 16 26 32 42 48 56 62
`
`7 14 28 30 44 46 58 60 64
`
`/
`
`/
`
`to nine bits (- 255 to +255), and ini(cid:173)
`is quantized
`The DC coefficient
`tially the weighted AC coefficients are quantized to ten bits (-511 to
`+ 511). Each block is then allocated to one of four classes labeled 0-3
`depending on the value of the largest AC coefficient in the block. Class
`0 is used for the lowest energy blocks/ class 3 for the highest. For class-3
`blocks / the least significant bit of each AC coefficient value is discarded.
`Otherwise / the most significant digit of every AC coefficient is discard(cid:173)
`than 255 im1nediately qualifies the
`ed . (Any AC coefficient greater
`block for class 3/ so that for a11y other class the MSB of every coeffi(cid:173)
`cie11t must be zero .) At this stage both DC and AC coefficients are 9 bits.
`Each 9-bit AC value is then divided by a quantization step/ deter(cid:173)
`mined by the class number an d an area n umber 1 dependent on the
`pos ition of the coefficient
`in the transformed block. The area num(cid:173)
`bers are illu strated in Figure 14-7/ and the derivation of the quantiza (cid:173)
`tion step is shown in Figu re 14-8.
`
`Figure 14-7
`Area nu m bers.
`
`8-8-DCT
`horizontal
`>
`4 5 67
`0123
`.,.
`0 0 1 1 1 2 2
`I H
`,-
`0 0 1 1 1 2 2 2
`0 1 1 1 2 2 2 3
`1 1 1 2 2 2 3 3
`1 1 2 2 2 3 3 3
`.
`1 2 2 2 3 3 3 3
`2 2 2 3 3 3 3 3
`2 2 3 3 3 3 3 3
`
`0
`1
`2
`3
`4
`5
`6
`7
`
`2-4-8-0CT
`
`horizontal
`:>
`0123
`4567
`- -
`llX 0 1
`l 1 2 2 3
`0 1 1 2 2 2 3 3
`1 1 2 2 2 3 3 3
`1 2 2 2 3 3 3 3
`
`0 0 1 1 2 2 2 3
`0 1 1 2 2 2 3 3
`1 1 2 2 2 3 3 3
`1 2 2 3 3 3 3 3
`
`0
`1
`2
`3
`
`4
`5
`6
`7
`
`(sum )
`
`Vertical
`
`(difference )
`
`Vertica l
`
`IPR2021-00827
`Unified EX1010 Page 269
`
`

`

`r
`
`250
`
`-
`
`-
`
`Figure 14-8
`Derivation of
`quantization steps.
`
`•
`
`I ;
`
`Quantiz.acion
`numb er
`(QNO )
`
`0
`15
`14
`13
`12
`11
`10
`9
`8
`7
`6
`5
`4
`3
`2
`1
`0
`
`Class number
`2
`1
`
`3
`
`15
`14
`13
`12
`11
`10
`9
`8
`7
`6
`5
`4
`3
`2
`1
`0
`
`15
`14
`13
`12
`11
`10
`9
`8
`7
`6
`5
`4
`3
`2
`1
`0
`
`15
`14
`13
`12
`11
`10
`9
`8
`7
`6
`5
`4
`3
`2
`1
`0
`
`0
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`2
`2
`2
`2
`4
`4
`4
`4
`8
`8
`
`Chapter 14
`
`Area number
`1
`2
`1
`I
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`1
`2
`1
`2
`2
`2
`2
`2
`2
`4
`2
`4
`4
`4
`4
`4
`4
`8
`8
`4
`8
`8
`8
`8
`8
`16
`16
`8
`
`3
`1
`1
`1
`1
`1
`1
`1
`2
`2
`2
`4
`4
`4
`4
`8
`8
`8
`8
`16
`16
`16
`16
`
`This table needs a little expla11ation. The quantization nu1nber is the
`compression control , and varies between O and 15. Zero represents the
`highest compression (coarser quantization of the coefficients , so fewer
`bits in the encoded output ) and 15 the lowest compression. The system
`seeks the highest quanti zation number
`that can be used , consistent
`with the final output of the compression system being not more than
`385 bytes for the complete segment. Each block in the segment has
`already been assigned to one of the four classes. Now the system can
`test each quantization level. Each AC coefficient
`in each block is divid(cid:173)
`ed by the appropriate quantizatio11 step . For example , for quantizat ion
`number 7, in a class-2 block, coefficients in area O would be divided by
`2, coefficients in areas 1 and 2 would be divided by 4, and those in area
`3 by 8. For the same quantization number , coefficients
`in a class-I
`block would be divided by 1, 2, 2, and 4 respectively .
`It may appear that the table entries for class 3 are "out of place:
`This is because the AC coefficients of class-3 blocks have already been
`divided by 2 as a result of discarding the least significant bit.
`The quantized coefficients are read out
`in the scanning order
`shown in Figure 14-6. The combination of run-length zeros and the
`following coefficient value is then variable-length encoded in a man(cid:173)
`ner similar to JPEG and MPEG, but with code tables optimized
`for
`
`IPR2021-00827
`Unified EX1010 Page 270
`
`

`

`DV Compression
`
`Figure 14-9
`Arrangement of a
`compressed
`macroblock w ith
`4: J: I compression .
`
`I 251 I I
`DV. As in JPEG and MPEG a 4-bit end-of-block (EOB) code is inserted
`whe11 there are no more non -zero coefficients. Note that the variable(cid:173)
`length-encoding
`step must be inside the loop that measures the num(cid:173)
`be1· of bytes required for each quantization number . In other words,
`for each quantization number , the coef ficie11ts must be quantized ,
`scanned 1 and variable-length-encoded
`before
`the length of the
`encoded segment is known .
`When the correct quantizatio11 number has been fo und (the high(cid:173)
`est number
`that results in 11ot more than 385 bytes for the segment),
`the resulting data is arranged as shown in Figure 14-91 which shows
`the data for one macroblock . After a 4-byte header that identifies the
`•
`n1acrob lock 1 77 bytes are provided for n1acroblock data (5X77 = 385).
`The first byte carries 4 bits for the quant ization number (0 to 15) a11d
`four b its to report error status and error co11cealmen t action in a
`videotape machine . Then comes the first luminance block DC coeffi(cid:173)
`cient (9 bits ) plus one bit to indicate fie ld or frame DCT1 and two bits
`to indicate the class of tl1e bl ock (0 to 3). Then there is a space of 121~
`bytes for AC coeffic ients. The ot her five blocks of the macro block
`fo llow in simi lar manne r1 except that on ly 81/i bytes are provided for
`the AC coefficients of the color -difference blocks.
`
`Byte pos irio11 number
`. . . . . 2 0 2 1 . . . . . 3 4 3 5 . . . . . 4 8 4 9 . . . . . 6 2 6 3 . . . . . 72 73 . . 81
`..... _.,.
`.
`•
`~mo
`D i cl
`C .
`i cO
`3
`
`5 6 7
`s
`............
`• .
`•
`T
`: 010
`D 1 cl
`A C \ cO
`.
`0
`~
`0
`
`v1SB
`
`..SB
`
`: ..............
`.
`: mo
`D I cl
`C ; cO
`AC 1
`
`•
`
`AC
`
`..........
`.
`,.
`I =mo
`D l c l
`• C 1 cO
`• .
`2
`
`AC
`
`..... ·---
`•
`:
`: mo
`D 1 ct
`•
`C Jc0
`AC 4
`
`-·- ..
`.
`•
`: mo
`D 1 cl
`C fc0
`AC 5
`
`AC
`
`Yo
`
`--
`14 bytes
`
`~
`
`Y1
`
`- ..,.
`14 bytes
`
`~
`
`Y2
`
`-
`-
`..,.
`-
`14 bytes
`
`..,.
`
`-
`
`Y3
`
`14 bytes
`
`-
`-
`
`-
`-
`
`CR
`
`10 b;tes
`
`- -
`-
`
`Cn
`
`-
`10 bytes
`
`~
`
`NOTES
`STA:
`QNO :
`DC:
`AC:
`EOB:
`mo :
`cO, cl :
`
`Error stat us
`Quantization number
`DC compo 11enc
`AC component
`End of block (0110)
`DCTmode
`Classnumber
`
`O f cou rse not all blocks will convenie 11tly generate exactly 101/i or
`81/i bytes of :ariable- leng th -co ded AC coeffic ien ts. In some blocks th e
`
`IPR2021-00827
`Unified EX1010 Page 271
`
`

`

`t 252
`
`Chapter 14
`
`EOB will occur before the e11d of tl1e all ocat ed spa ce; for o ther s tl1ere
`will be too 1nuch data for the spac e pr ov ided . DV e1nploy s a comple x
`three-pass algorithm
`to take maximun1 adv anta ge of the space avail(cid:173)
`able. The first pass places the data as describ ed ; th e second a11d third
`passes attempt
`to place all th e 1·e1naini11g d ata in th e "empt y" spaces
`after EOB in the blocks with littl e data.
`So, for 4:1:1 compre ssion of 525/ 60 th ere are 1,350 1na c1·obl ocks per
`frame , or 270 seg1ne11ts per fr am e; app rox i111ate ly 30 fr ame s/second .
`Each segment has a 4-byte h eader pl L1s 385 by tes of data, so th at the
`data rate is:
`
`270 x 389 x 30 x 8 = 25,207,200 bits/ secon d
`
`For 625/ 50 system s with 576 acti ve lines th ere ar e, 1,620 macr o blocks
`or 324 segments per frame , a11d 25 f ran1es per secon d , so th e data rate is:
`324 x 389 x 25 x 8 = 25,207,200 bi ts/ second
`
`Note that the 4:1:1 input d ata (at 8-bit precisio n ) re pr esent s just
`under 125 Mbits /s, so that thi s m ode o f DV provi des a bo ut 5:1 com-
`press1on.
`
`•
`
`50 Mbits / s Compression
`
`This section is much shorter because 50 M b its/ s compre ssion uses all
`the techniques of the 25 Mbit s/ s co mpre ssion ju st d escribed . In fact , in
`practical implementation s, the pro cess u ses t wo standard D V ch ips
`instead of one ! 50 Mbit s/s DV compre ssion u ses 4:2:2 vid eo with o u t dis(cid:173)
`carding an y pixel value s and gene ra tes two stre am s of 25 Mbit s/s data
`that may be multiplexed
`int o one 50 M bit s/ s stre am . The differences
`lie in how the video data is distributed
`to the tw o channels and hoi, ,
`the data is arranged . The 4:22 video data at 8-bit precisi on repre senrs
`about 166 Mbits /s, so this mod e of DV pr o vid es about 3.3:1 compr es(cid:173)
`sion and is virtually transparent ..
`The macro block for 4:2:2 DV compres si.on ha s o nl y four . D~ T
`blocks , two luminance and one each color difference , as shown m F1g(cid:173)
`ure 14-10. Again five macroblocks are grouped
`into one segment , and
`half the segments are sent to each channel
`.
`fact or
`With only four blocks per macro block (and a compression
`of 3.3:1), quantization
`is less aggressive , and more space is needed for
`AC coefficients . This results in a revised layout
`for the compressed
`macroblock , as shown in Figure 14-11. As with the 25 Mbits /s system , a
`
`•
`
`•
`
`•
`
`IPR2021-00827
`Unified EX1010 Page 272
`
`

`

`DV Compressio11
`
`Figure 1 4 -1 0
`Macroblock an d OCT
`bf ock.s for 4 : 2: 2
`.
`com pre ssion .
`
`C
`13
`C R
`y
`
`DCTO
`
`DCTl
`
`\
`
`....
`
`DCT3
`
`Left
`
`•
`
`Right DCT2
`
`NOT ES - DCTI: DCT block order
`1= 0, 1, 2, 3
`
`I
`
`•
`
`253
`
`three -pass algorithm distrib u tes excess coefficient data into otherwise
`unused spaces in th e data struc tu re.
`
`Figure 1 4- 11
`Arrangement of a
`compressed
`macroblock w ith
`4:2 :2 compression .
`
`NISB
`
`LSB
`
`•
`•
`•
`
`Byte positio n numbe r
`5 6 7 . . . . . 20 21 . . ... 34 35 . . . . . 48 49 ..... 62 63 ... . . 7 2 73 .. 81
`s
`
`...........
`.. .... ...-....
`: mo
`.
`:•--c-o•
`T
`•
`•
`:mo
`•
`D i cl
`•
`: lllO
`D i ct
`A
`D l ct
`C i cO
`Clc0
`C l co
`0
`AC 3 AC
`AC 2
`~
`0
`
`~ ...........
`•
`•
`•
`: ITIO
`D ! ct
`C l cO
`AC 1
`
`AC
`
`XO
`
`AC
`
`Xl
`
`•
`
`•
`
`~ Yo
`--
`14 byte s
`
`-
`.,
`
`~ -
`~ -
`~ ...
`2 bytes 12 byte s 14 bytes
`
`Y1
`
`-
`-
`-
`.,
`... ~ ...
`--
`2 byte s 12 bytes 10 bytes
`
`CR
`
`-
`
`..,
`
`Cs
`
`-
`--
`10 bytes
`
`8 bits
`
`MSB R
`e
`s
`e
`r
`V
`e
`LSB __._ d
`
`0
`1
`1
`0
`
`<
`
`4 bits
`
`The
`arrange1nent
`of data area
`(XO, Xl)
`
`There are more differences in the detailed syntax of the bitstream,
`bu t th at is the essence of 4:2:2 50 Mb its/s compression in DV. All the
`ot h er processes descri bed u11der the previo us section apply.
`
`IPR2021-00827
`Unified EX1010 Page 273
`
`

`

`•
`
`|PR2021-00827
`
`Unified EX101O Page 274
`
`IPR2021-00827
`Unified EX1010 Page 274
`
`

`

`CHAPTER
`
`ave ets
`
`IPR2021-00827
`Unified EX1010 Page 275
`
`

`

`256
`
`r e
`
`....
`
`
`
`.__ ______ .. Introduction
`
`Chapter 15
`
`to DCT for
`Wavelet technology off e1·s the most viable alternative
`in1age co111p1·essio11. TI1e tech11iques were i11 vestigated by workers
`in
`n1any di£ f ere11t fields / and existed u11de1- various names until
`the con(cid:173)
`nectio11 was spotted by a 111athe1natician in th e 1nid 1980s. Although
`the favo1·ite child of theorjsts f 01· many years / L1ntil recently practical
`image compressio11 systems l1ave failed to 1neet expectations
`.
`Tl1e theory of wavelets
`is q Ltite complex / but
`the principles
`a11d DCT / wavelets
`involved are 11ot. Like Fourier
`transforms
`traru(cid:173)
`forn1 inforn1ation
`into a 1·ep1·esentatio11 using frequency-dependent
`coef ficie11ts, but wavelets diff er in th at useful po sitio11al information
`is 1·etained. Before we look at wavelet s, it will be useful
`to review some
`of tl1e qualitative aspects of Fourier transf orn1s .
`
`r
`
`. J
`
`More about Fourier Transfor111s
`
`for a
`information
`Let's look at the isst1e of f reqt1ency and location
`n1ome11t because it's really in1porta11t. If we have a series of samples of
`a signal , say in the ho1-izo11tal direction / we can look at any sample
`a11d know i1nmediately the amplitude
`informatio11
`for that particular
`locatio11 in the horizo11tal di1·ection . What do we ·k11ow about
`the hor(cid:173)
`izontal spatial freqL1ency? Nothing! We need the i11formation
`from
`many samples to gai11 mt1ch infor1nation
`about
`the frequency
`con(cid:173)
`te11t of the signal/ and from all of the san1ples to get all of the inf or(cid:173)
`mation about freque11cy content.
`Now let's perform a Fourier
`transform. This gives us a number of
`of the signal . W~ :3-n
`values representing
`the frequency components
`get the information pertaini11g to any given frequency by exarrun1ng
`the single coefficient
`tl1at represents
`that frequency. However , \Ve
`have no information about where any a1nplit1.1de event (such as a step)
`may occur . To find this we need to assess the contributions
`of all the
`frequency components
`in fact/ perform a inverse Fourier
`transfo:m .
`In general terms/ in the spatial domain we have excellent
`locaaon
`information
`but little frequency
`information;
`in the f requei~C) '
`domain we have excellent frequency
`information
`but little locat 1on
`infor111ation. Put like this it sounds very obvious/ but it is a f u1:da(cid:173)
`mental issue that is important
`to understand. One of the great thmgs
`
`IPR2021-00827
`Unified EX1010 Page 276
`
`

`

`Wavelets
`
`l •
`
`257
`
`is that
`
`they ca11 give both frequency
`
`and location
`
`about wavelets
`i11f 01·n1a ti 011.
`It will l1elp i11 understa11ding the operation of wavelets to examine
`i11 a Jittle 1nore detail l1ow tl:ie Fourier
`transf or1n works . The general
`f or111 of a Fourier se1·ies for a periodic function
`is:
`Ao+ A1si11( 2n. f + cp1) + A2sin ( 2n.2 f + cp2
`+ A3si11( 2n.3 f + cp3) + - - -
`
`)
`
`wl1e1·e A 0 is the DC co1npone11t, f is the frequency of periodicity ; 2f 3~
`etc., are the 111ultiples of the first frequency, and cp
`, etc. are the phase
` cp
`2
`a11gles of eacl1 of tl1e compo11ent frequencies. Remember , as discussed in
`Ch apter 5, Fo ur ier transforms
`require
`two coefficie11ts for each fre(cid:173)
`qt1ency ; j11stead of using phase angles directly , we could have used one
`si11e and on e cosi11e ter111 for each f1·equency , each with an appropriate
`coeff icient . Let's exam in e how we arrive at these coefficients . Let's build
`a si111ple periodic waveform by adding a few Fou1ie1· terms.
`
`1,
`
`f (x ) = 3sin(2n.x )+ 5sin(2n.3x)+ 4sin(2n.5x)
`
`Th e wavef o1·m is sl1ow n in Figure 15-1. 111 this case I have made all
`the pha se a11gles ze1·0, so we need only worry about one coefficient
`f 01· each f reque11cy. The DC co1nponent is also zero .
`
`f (x)
`
`0
`
`0.5
`
`X
`
`1
`
`A Fourie1· coefficient may be calculated by multiplying
`the wave(cid:173)
`form by th e correspo11ding Fourier component
`frequency and inte(cid:173)
`grati11g the 1·esuJt ove1· the period of the input waveform. (Th~ DC
`coefficient
`is obtained by integrating
`tl1e original waveform
`itself
`over the san1e period .) Normally we would have to use both sine and
`cosine components
`to get both coefficients , b11t in this case sine waves
`
`•
`
`- - .
`Figure 15-1
`A periodi c wafveform
`made from three sine
`components.
`
`IPR2021-00827
`Unified EX1010 Page 277
`
`

`

`I
`258
`
`•
`
`•
`
`Chapter 15
`
`alone will be sufficie11t. So, if we m11ltiply our sa1nple waveform by
`the funda111e11tal frequency
`(the frequency of periodicity ), we have :
`g (x) = 2.sin (2n. x). [3si11(2n.x) + 5sin (2n .3x) + 4sin (2n.5x )]
`
`Figure 15-2
`The waveform
`multiplied by a sine
`wave at the
`frequency of
`periodicity. The result
`is asymmetric, so
`yields a non-zero
`coefficient.
`
`g(x)
`
`10
`
`5 ....
`0 -
`
`-5
`
`0
`
`I
`
`,.
`
`\
`
`I
`0.5
`
`X
`
`"
`
`-
`
`1
`
`i11 Figure 15-2. (There
`a good
`is probably
`is shown
`This product
`to make
`mathematjcal .reason for the multiplier
`"2" but I put it there
`the numbers con1e out right! ) If we integrat e this waveform over the
`full period:
`
`1 f g (x )dx= 3
`
`0
`we have our first coefficient
`, if \Ve
`the simplification
`. Just to validate
`in Figure
`change the sine to a cosine , the waveform product
`is shown
`15-3, and the integral over the same period
`is zero . In fact , all cosine
`products will integrate to zero , so we can continue
`to look just at sine
`products .
`
`Figure 15-3
`In this special case,
`multiplying by a
`cosine wave at the
`same frequency gives
`a symmetric product
`that integrates to
`zero.
`
`•
`
`g(x) O
`
`0.5
`
`X
`
`1
`
`IPR2021-00827
`Unified EX1010 Page 278
`
`

`

`Wavelets
`
`259
`
`The beha vjor I a1n tryin