US006249549B1
`
`
`
`
`
`
`
`
`
`(12) United States Patent
`US 6,249,549 B1
`(10) Patent N0.:
`Kim
`
`
`
`
`
`
`
`(45) Date of Patent:
`Jun. 19, 2001
`
`
`
`(54) DOWN CONVERSION SYSTEM USINGA
`
`
`
`
`PRE-DECIMATION FILTER
`
`
`
`
`
`8/2000 Boyce et a1.
`.................... 375/240.16
`
`
`
`8/2000 Nakagawa et al.
`............. 375/240.16
`
`
`
`
`
`
`
`
`(75)
`
`( * ) Notice:
`
`
`
`
`
`
`
`
`
`Inventor: Hee-Yong Kim, Plainsboro, NJ (US)
`
`
`
`
`
`(73) Assignee: Matsushita Electric Industrial Co.,
`Ltd., Osaka (JP)
`
`
`
`Subject to any disclaimer, the term of this
`
`
`
`
`
`
`patent is extended or adjusted under 35
`
`
`
`
`
`
`
`U.S.C. 154(b) by 0 days.
`
`
`
`
`(57)
`
`
`
`6,100,932 *
`
`6,104,434 *
`
`* cited by examiner
`
`
`
`Primary Examiner—Chris Kelley
`
`
`
`Assistant Examiner—Gims Philippe
`
`
`
`
`
`
`
`(74) Attorney, Agent, or Firm—Ratner & Prestia
`
`
`ABSTRACT
`An HDTV down conversion system including an apparatus
`
`
`
`
`
`
`
`for forming a low resolution video signal from an encoded
`
`
`
`
`
`
`
`
`Video signal representing a Video image. The encoded Video
`
`
`
`
`
`
`
`
`signal is a frequency-domain transformed high resolution
`
`
`
`
`
`Video signal with motion vectors. The apparatus includes a
`
`
`
`
`
`
`
`
`receiver for receiving the encoded video signal as a plurality
`
`
`
`
`
`
`
`
`of blocks of high resolution frequency-domain video coef-
`
`
`
`
`
`
`
`icient values. Aplurality of blocks comprises a macroblock.
`
`
`
`
`
`
`A down-conversion filter weights selected ones of the high
`
`
`
`
`
`
`
`
`esolution frequency-domain video coefficient values Within
`
`
`
`
`
`
`each block to generate corresponding blocks of filtered
`
`
`
`
`
`
`
`‘requency-domain Video coefficients. An inverse-transform
`
`
`
`
`
`3rocessor
`transforms each block of filtered frequency—
`
`
`
`
`
`
`
`domain video coefficients into a block of first-filtered pixel
`
`
`
`
`
`
`
`values. A pre-decimation filter performs inter-macroblock
`
`
`
`
`
`inter-block filtering of the plurality of blocks of first-filtered
`
`
`
`
`
`
`3ixel values and provides corresponding blocks of second-
`
`
`
`
`
`
`
`iltered pixel values. Adecimating processor deletes selected
`
`
`
`
`
`
`
`ones of the second-filtered pixel values Within each block to
`
`
`
`
`
`
`
`arovide blocks of low resolution video signal pixel values.
`
`
`
`
`
`
`
`
`28 Claims, 8 Drawing Sheets
`
`
`
`
`(22)
`
`
`
`
`
`(21) Appl. N0.: 09/169,790
`Filed:
`Oct. 9, 1998
`
`
`
`
`
`
`
`
`
`
`(51)
`Int. Cl.7 ....................................................... H04B 1/66
`
`(52) US. Cl.
`................
`375/240.21; 382/248
`
`
`
`
`
`
`(58) Field of Search .....
`375/240.16, 240.2,
`
`
`
`
`
`
`
`, - 0.12, 240.13, 240.21;
`375/2402 , 2
`
`
`
`
`
`348/4261, 441, 443, 445, 449, 458, 459,
`
`
`
`
`
`
`
`556, 565, 384.1, 408.1, 427.1; 382/269,
`
`
`
`
`
`264, 248, 232, 239
`
`
`
`
`(56)
`
`
`
`References Cited
`
`
`U.S. PATENT DOCUMENTS
`
`
`
`......................... 348/556
`6/1997 Boyce et al.
`
`
`
`
`
`. 382/232
`1/1998 Merhav et al.
`
`
`
`
`
`
`..... 382/239
`7/1999 Kim et al.
`
`
`
`
`
`
`5/2000 Pearlstein et al.
`375/240.12
`
`
`
`
`
`5/2000 Boyce et al.
`375/240
`
`
`
`
`
`
`5,635,985 *
`
`5,708,732 *
`
`5,926,573 *
`
`6,061,400 *
`
`6,061,402 *
`
`
`
`
`
`
`
`
`
`
`
`
`
`DOMAIN
`
`FILTER
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MOTION BLOCK
`
`
`
`
`
` DISPLAY CONVERSION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 1 of 20
`
`GOOGLE EXHIBIT 1008
`
`Page 1 of 20
`
`GOOGLE EXHIBIT 1008
`
`

`

`
`US. Patent
`
`
`
`
`
`Jun. 19, 2001
`
`
`
`Sheet 1 0f 8
`
`
`
`
`
`US 6,249,549 B1
`
`_
`
`
`
`
`E252zQEszEoo
`20:22-mommmooE
`
`
`
`man?2252
`
`ofo:
`
`
`
`ESEEx003EdoEzoo
`
`
`
`fa:
`
`E<moi;
`
`mg;82>
`
`oi
`
`emf
`
`owe
`
`a:
`
`mmmmSZ
`
`ENfiz/Bo
`
`
`
`szflm._m<_m<>
`
`1525-22oz<
`
`wzaooma
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 2 of 20
`
`Page 2 of 20
`
`
`
`
`
`
`
`
`

`

`
`
`
`
`m£2;sz@1ng$9sz;mmSEE:>2"01,,nn.lllllluhIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII_
`
`
`
`
`
`US 6,249,549 B1
`
`
`
`><EmE
`
`Eamomm
`
`NNN
`
`
`
`
`
`mzo_w$>28>§maV352$
`
`
`
`
`70.52%?
`a503295:5m..............................................................................u
`
`
`
`
`
`$25$5:m238233%u2&4?uwnua__lPmm30mm3%EEem
`$5:23%E
`
`
`
`
`
`S”U
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 3 of 20
`
`Page 3 of 20
`
`
`
`
`

`

`US. Patent
`
`Jun. 19, 2001
`
`Sheet 3 0f 8
`
`US 6,249,549 B1
`
`a
`
`H
`
`N
`
`
`
`zoEmEEimzm
`
`
`
`a
`
`H
`
`
`
`29:8;.mximzm
`
`
`
` DQQQQ<<<<<<<<<QQQQO.
`
`D.QQQQQQQQQQQQQQQQQ.D
`
`.OQQQQQQQQQQQQQQQQQD
`
`<m.0:
`
`fiuflofififlfiflfiflfiflfiflfiflflfl
`
`fluflfiflfifififlfiflfiflfiflfl.flfl
`
`Page 4 of 20
`
`Page 4 of 20
`
`

`

`
`US. Patent
`
`
`
`
`
`Jun. 19, 2001
`
`
`
`
`Sheet 4 0f 8
`
`
`
`US 6,249,549 B1
`
`
`RECEIVE MOTION
`
`VECTOR
`
`
`
`310
`
`
`
`MVTRANSLAHON
`
`
`
`
`312
`
`
`
`
`
`
`GET COORDINATE OF PREDICTION
`
`
`BLOCK IN DOWNSAMPLED PICTURE
`
`314
`
`
`
`
`
`GET SUBPIXELED POINT
`
`
`
`316
`
`
`
`
`SET INITIAL POLYPHASE
`
`
`
`318
`
`
`
`
`
`
`PULL THE PIXELS FROM
`
`
`REFERENCE FRAME
`
`
`
`320
`
`
`
`
`INITIALIZE REGISTERS
`
`
`
`322
`
`
`
`
`UPDATE REGISTERS
`
`
`
`
`
`324
`
`
`
`
`
`326
`
`
`
`
` FILTER
`
`
`UPDATEPHASE
`
`
`
`
`
`
`
`
`
`
`
`ALL
`
`
`17 PIXELS
`
`
`DONE?
`
`
`YES
`
`
`FIG. 33
`
`332
`
`
`
`328
`
`
`
`Page 5 of 20
`
`Page 5 of 20
`
`

`

`nu
`
`
`m“waPS”
`
`
`
`
`
`mm.
`
`
`
`
`
`
`
`US 6,249,549 B1
`
`mx
`
`ox
`
`
`
`
`
`v.6:
`
`CA
`
`___.._.._____L___--____._._...._J.__.__...___..___
`
`
`.w.....................
`oxfixmxmxvxmxmxRx
`o:Em:m:w:m:a:ENxexmxvxmxmxFxox
`oxfixmxmxvxmxoxRX
`m:a:S::5NE2::5
`
`mE%oxfixmxmxvxmx@xRx
`Em:m:Em:a:Ew:
`xxexmxvxmxmxFxox
`a:E:3:NEmE3.;
`exFxmxmxvxmxoxRx
`
`
`
`.meSagoazoomw
`
`de:EEQ51E
`
`Page 6 of 20
`
`Page 6 of 20
`
`

`

`
`US. Patent
`
`
`
`Jun. 19, 2001
`
`
`
`
`
`Sheet 6 0f 8
`
`
`
`
`US 6,249,549 B1
`
`
`
`
`
`M82
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MB1
`
`
`FIG.5A
`
`
`
`
`
`
`
`MBO
`
`
`
`
`
`
`
`
`
`
`
`Page 7 of 20
`
`Page 7 of 20
`
`

`

`
`US. Patent
`
`
`
`
`
`Jun. 19, 2001
`
`
`
`
`Sheet 7 0f 8
`
`
`
`US 6,249,549 B1
`
`
`
`
`
`
`
`Page 8 of 20
`
`Page 8 of 20
`
`

`

`
`US. Patent
`
`
`
`
`
`Jun. 19, 2001
`
`
`
`
`Sheet 8 0f 8
`
`
`
`US 6,249,549 B1
`
`dB
`
`10
`
`0
`
`—10
`
`—20
`
`-50
`
`
`
`
`
`AMPLITUDES
`
`(dB)
`
`:'
`§--;-'UPSAMPL|NG FILTER
`n_“,: ..................................... En\v& 3
`
`
`
`DECIMAHON FILTER
`
`—TOTAL CASCADED FTLTERE
`
`
`
`-
`
`
`
`
`
`0
`
`
`
`
`0.1
`
`0.2
`
`
`
`0,3
`
`
`
`0.4
`
`
`
`0.5
`
`
`
`0.6
`
`
`
`0.7
`
`
`
`0.8
`
`
`
`0.9
`
`
`
`1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`——-T0TAL0A80A0E0 FILTER
`............... . ........ .....; ......... ........
`
`;
`
`g
`g
`........g ........ g........
`
`
`
`
`
`
`——-5DECIMAT10MFILTER
`
`
`
`0
`
`
`0.1
`
`0.2
`
`
`
`0.3
`
`
`
`0.4
`
`0.5
`
`0.6
`
`
`
`0.7
`
`
`
`0.8
`
`
`
`0.9
`
`
`
`1
`
`
`
`FIG.7
`
`
`
`AMPLITUDES(dB)
`
`-100
`
`—120
`
`Page 9 of 20
`
`Page 9 of 20
`
`

`

`
`
`US 6,249,549 B1
`
`1
`
`DOWN CONVERSION SYSTEM USING A
`
`
`
`PRE-DECIMATION FILTER
`
`FIELD OF THE INVENTION
`
`
`
`The present invention relates to a decoder which converts
`
`
`
`
`
`
`
`and formats an encoded high resolution video signal, e.g.
`
`
`
`
`
`
`
`
`MPEG-2 encoded video signals, to a decoded lower reso-
`
`
`
`
`
`
`
`lution output video signal, and more specifically to a down
`
`
`
`
`
`
`
`
`conversion system for the decoder.
`
`
`
`
`
`BACKGROUND OF THE INVENTION
`
`
`
`In the United States a standard has been proposed for
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`digitally encoded high definition television signals (HDTV).
`A portion of this standard is essentially the same as the
`
`
`
`
`
`
`
`
`
`MPEG-2 standard, proposed by the Moving Picture Experts
`
`
`
`
`
`
`
`
`
`
`
`
`
`Group (MPEG) of the International Organization for Stan—
`dardization (ISO). The standard is described in an Interna-
`
`
`
`
`
`
`
`
`
`
`
`
`tional Standard (IS) publication entitled, “Information
`Technology—Generic Coding of Moving Pictures and Asso-
`
`
`
`
`
`
`ciated Audio, Recommendation H.626”, ISO/IEC 13818-2,
`
`
`
`
`
`
`IS, November 1994 which is available from the ISO and
`
`
`
`
`
`
`
`
`
`which is hereby incorporated by reference for its teaching on
`
`
`
`
`
`
`the MPEG-2 digital video coding standard.
`
`
`
`
`
`
`The MPEG-2 standard is actually several different stan-
`
`
`
`
`
`
`
`dards. In MPEG—2, several different profiles are defined,
`
`
`
`
`
`
`
`
`each corresponding to a different level of complexity of the
`
`
`
`
`
`
`encoded image. For each profile, different levels are defined,
`
`
`
`
`
`
`
`
`
`each level corresponding to a different image resolution.
`
`
`
`
`
`
`One of the MPEG-2 standards, known as Main Profile, Main
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Level is intended for coding video signals conforming to
`existing television standards (i.e., NTSC and PAL). Another
`
`
`
`
`
`
`
`standard, known as Main Profile, High Level, is intended for
`
`
`
`
`
`
`
`
`coding high-definition television images.
`
`
`
`
`Images encoded according to the Main Profile, High
`
`
`
`
`
`
`Level standard may have as many as 1,152 active lines per
`
`
`
`
`
`
`
`
`image frame and 1,920 pixels per line.
`
`
`
`
`
`
`
`The Main Profile, Main Level standard, on the other hand,
`
`
`
`
`
`
`
`
`defines a maximum picture size of 720 pixels per line and
`
`
`
`
`
`
`
`
`
`
`576 lines per frame. At a frame rate of 30 frames per second,
`
`
`
`
`
`
`
`
`
`signals encoded according to this standard have a data rate
`
`
`
`
`
`
`
`
`of 720*576*30 or 12,441,600 pixels per second. By
`
`
`
`
`
`
`
`contrast, images encoded according to the Main Profile,
`
`
`
`
`
`
`
`High Level standard have a maximum data rate of 1,152*1,
`
`
`
`
`
`
`
`
`920*30 or 66,355,200 pixels per second. This data rate is
`
`
`
`
`
`
`
`
`more than five times the data rate of image data encoded
`
`
`
`
`
`
`
`
`
`
`
`according to the Main Profile, Main Level standard. The
`
`
`
`
`
`
`
`
`standard proposed for HDTV encoding in the United States
`
`
`
`
`
`
`
`
`is a subset of this standard, having as many as 1,080 lines per
`
`
`
`
`
`
`
`
`
`frame, 1,920 pixels per line and a maximum frame rate, for
`
`
`
`
`
`
`
`
`
`
`this frame si7e, of 30 frames per second. The maximum data
`
`
`
`
`
`
`
`
`
`rate for this proposed standard is still far greater than the
`
`
`
`
`
`
`
`
`
`
`maximum data rate for the Main Profile, Main Level stan—
`
`
`
`
`
`
`
`
`
`
`dard.
`
`The MPEG-2 standard defines a complex syntax which
`
`
`
`
`
`
`
`contains a mixture of data and control information. Some of
`
`
`
`
`
`
`
`this control information is used to enable signals having
`
`
`
`
`
`
`
`several different formats to be covered by the standard.
`
`
`
`
`
`
`
`
`These formats define images having differing numbers of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`picture elements (pixels) per line, differing numbers of lines
`per frame or field, and differing numbers of frames or fields
`
`
`
`
`
`
`
`
`per second. In addition, the basic syntax of the MPEG—2
`
`
`
`
`
`
`
`
`
`Main Profile defines the compressed MPEG-2 bit stream
`
`
`
`
`
`
`
`representing a sequence of images in five layers,
`the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`sequence layer, the group of pictures layer, the picture layer,
`the slice layer and the macroblock layer. Each of these layers
`
`
`
`
`
`
`
`
`
`is introduced with control information. Finally, other control
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`2
`
`
`
`
`
`
`
`information, also known as side information, (e.g. frame
`type, macroblock pattern, image motion vectors, coefficient
`
`
`
`
`
`
`
`
`
`
`
`
`zig-Lag patterns and dequantization information) are inter-
`spersed throughout the coded bit stream.
`
`
`
`
`
`A down conversion system converts a high definition
`
`
`
`
`
`
`
`input picture into lower resolution picture for display on a
`
`
`
`
`
`
`
`
`lower resolution monitor. Down conversion of high resolu-
`
`
`
`
`
`
`
`tion Main Profile, High Level pictures to Main Profile, Main
`
`
`
`
`
`
`
`
`
`Level pictures, or other lower resolution picture formats, has
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`gained increased importance for reducing implementation
`costs of HDTV. Down conversion allows replacement of
`
`
`
`
`
`
`
`
`expensive high definition monitors used with Main Profile,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`High Level encoded pictures with inexpensive existing
`monitors which have a lower picture resolution to support,
`
`
`
`
`
`
`
`for example, Main Profile, Main Level encoded pictures,
`
`
`
`
`
`
`
`
`such as NTSC or 525 progressive monitors.
`
`
`
`
`
`
`
`
`
`
`
`
`To effectively receive the digital images, a decoder should
`process the video signal information rapidly. To be optimally
`
`
`
`
`
`
`effective, the coding systems should be relatively inexpen-
`
`
`
`
`
`
`sive and yet have sufficient power to decode these digital
`
`
`
`
`
`
`
`
`signals in real time.
`
`
`
`One method of down conversion of the prior art simply
`
`
`
`
`
`
`
`
`low pass filters and decimates the decoded high resolution,
`
`
`
`
`
`
`
`
`
`Main Profile, High Level picture to form an image suitable
`
`
`
`
`
`
`
`
`for display on a conventional
`television receiver.
`
`
`
`
`
`
`
`
`
`
`
`
`Consequently, using existing techniques, a decoder employ-
`ing down conversion may be implemented using a single
`
`
`
`
`
`
`
`
`
`
`
`
`
`processor having a complex design, considerable memory,
`and operating on the spatial domain image at a high data rate
`
`
`
`
`
`
`
`
`to perform this function. The high resolution, and high data
`
`
`
`
`
`
`
`
`
`
`rate, however,
`requires very expensive circuitry, which
`
`
`
`
`
`
`
`would be contrary to the implementation of a decoder in a
`
`
`
`
`
`consumer television receiver in which cost is a major factor.
`
`
`
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`The present invention is embodied in an apparatus for
`
`
`
`
`
`
`
`forming a low resolution video signal from an encoded
`
`
`
`
`
`
`
`
`video signal representing a video image. The encoded video
`
`
`
`
`
`
`
`
`signal is a frequency-domain transformed high resolution
`
`
`
`
`
`video signal. The apparatus includes a means for receiving
`
`
`
`
`
`
`
`the encoded video signal as a plurality of blocks of high
`
`
`
`
`
`
`
`
`
`resolution frequency—domain video coefficient values. A
`
`
`
`
`
`down-conversion filter weights selected ones of the high
`
`
`
`
`
`
`
`resolution frequency-domain video coefficient values within
`
`
`
`
`
`
`each block to generate corresponding blocks of filtered
`
`
`
`
`
`
`
`requency-domain video coefficients. An inverse-transform
`
`
`
`
`
`3rocessor
`transforms each block of filtered frequency-
`
`
`
`
`
`
`
`domain video coefficients into a block of first—filtered pixel
`
`
`
`
`
`
`
`values. A pre-decimation filter performs inter-block filtering
`
`
`
`
`
`
`of the plurality of blocks of first-filtered pixel values and
`
`
`
`
`
`
`
`
`
`3rovides corresponding blocks of second-filtered pixel val-
`
`
`
`
`
`
`ues. Adecimating means deletes selected ones of the second-
`
`
`
`
`
`
`
`iltered pixel values within each block to provide blocks of
`
`
`
`
`
`
`
`
`ow resolution down sampled video signal pixel values.
`
`
`
`
`
`
`
`
`According to one aspect of the invention, the decimating
`
`
`
`
`
`
`
`me ans is coupled to an up—sampling filter which converts the
`
`
`
`
`
`
`
`alocks of low resolution down sampled video signal pixel
`
`
`
`
`
`
`
`
`values into up-sampled blocks of low resolution video signal
`
`
`
`
`
`
`
`
`aixel values. An adder then adds the upsampled blocks of
`
`
`
`
`
`
`
`
`0w resolution video signal pixel values to the block of
`
`
`
`
`
`
`
`
`
`first—filtered pixel values to provide a sum which is filtered
`
`
`
`
`
`
`
`3y the pro-decimation filter and then decimated by the
`
`
`
`
`
`
`
`
`
`decimating means. The combined frequency response char-
`
`
`
`
`
`
`
`acteristic of the up-sampling filter and the pre-decimation
`
`
`
`
`
`
`
`
`ilter conforms to a response characteristic of a Lagrange
`
`
`
`
`
`
`
`interpolator.
`
`
`
`
`
`10
`
`I\)m
`
`LA)LII
`
`
`
`
`40
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`60
`
`
`
`
`
`Page 10 of 20
`
`Page 10 of 20
`
`

`

`
`
`US 6,249,549 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`I\)m
`
`LA)LII
`
`40
`
`60
`
`
`3
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`
`
`
`These and other features and advantages of the present
`
`
`
`
`
`
`
`invention will become apparent from the following detailed
`
`
`
`
`
`
`
`
`
`
`
`
`description, taken in conjunction with the accompanying
`drawings, wherein:
`
`
`FIG. 1 is a high level block diagram of a video decoding
`
`
`
`
`
`
`system of the prior art;
`
`
`
`
`
`
`
`
`
`
`FIG. 2 is a high level block diagram of an exemplary
`10
`embodiment of a down conversion system having a DCT
`
`
`
`
`
`
`domain filter and a pie-decimation filter;
`
`
`
`
`
`FIG. 3A illustrates subpixel positions and corresponding
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`predicted pixels for exemplary embodiments of 3:1 and 2:1
`down conversion systems;
`
`
`
`FIG. 3B shows the up-sampling process which is per-
`
`
`
`
`
`
`
`formed for each row of an input macroblock for an exem—
`
`
`
`
`
`
`
`plary down conversion system;
`
`
`
`
`FIG. 4 illustrates the multiplication pairs for the first and
`
`
`
`
`
`
`
`
`
`second output pixel values of an exemplary embodiment of
`
`
`
`
`
`
`a block mirror filter;
`
`
`
`
`FIG. 5A shows input and decimated output pixels for
`
`
`
`
`
`
`
`
`4:2:0 video signal using 3:1 decimation;
`
`
`
`
`
`
`FIG. 5B shows input and decimated output pixels for
`
`
`
`
`
`
`
`
`4:2:0 video signal using 2:1 decimation;
`
`
`
`
`
`
`FIG. 6 shows the frequency response characteristics of an
`
`
`
`
`
`up-sampling filter, a pre-decimation filter and their cascaded
`
`
`
`
`
`
`
`response for a horizontal 3:1 down conversion system; and
`
`
`
`
`
`
`
`
`FIG. 7 shows the frequency response characteristics of an
`
`
`
`
`
`up-sampling filter, a pre-decimation filter and their cascaded
`
`
`
`
`
`
`
`response for a horizontal 2:1 down conversion system.
`
`
`
`
`
`
`
`DETAILED DESCRIPTION
`
`I. Decoder Overview
`
`
`
`The exemplary embodiment of the invention filters
`
`
`
`
`
`
`decoded HDTV signals which have been encoded according
`
`
`
`
`
`
`
`
`to the MPEG-2 standard and in particular, the Main Profile,
`
`
`
`
`
`
`
`
`
`High Level MPEG-2 standard.
`
`
`
`
`The invention described herein, however, is not limited to
`
`
`
`
`
`
`
`down conversion filtering of decoded HDTV signals. The
`
`
`
`
`
`
`
`
`filtering method described below may also be used to filter
`
`
`
`
`
`
`
`
`other types of frequency-domain encoded digital signals
`
`
`
`
`
`
`
`which may be divided into sections, filtered, and then
`
`
`
`
`
`
`
`
`recombined.
`
`The MPEG-2 Main Profile standard defines a sequence of
`
`
`
`
`
`
`
`images in five levels:
`the sequence level,
`the group of
`
`
`
`
`
`
`
`
`
`pictures level,
`the picture level,
`the slice level and the
`
`
`
`
`
`
`
`
`
`
`macroblock level. Each of these levels may be considered to
`
`
`
`
`
`
`
`be a record in a data stream, with the later-listed levels
`
`
`
`
`
`
`
`
`occurring as nested sub-levels in the earlier listed levels. The
`
`
`
`
`
`
`
`
`records for each level include a header section which con-
`
`
`
`
`
`
`
`
`
`tains data that is used in decoding its sub-records.
`
`
`
`
`
`
`Macroblocks are composed of six blocks, 4 luminance
`
`
`
`
`
`
`blocks Y and 2 chrominance blocks, Cr and Cb. Each block
`
`
`
`
`
`
`
`of the encoded HDTV signal contains data representing 64
`
`
`
`
`
`
`
`
`respective coefficient values of a two dimensional discrete
`
`
`
`
`
`
`
`
`
`
`
`
`
`cosine transform (DCT) representation of 64 picture ele-
`
`
`
`
`
`ments (pixels) in the HDTV image.
`In the encoding process, the pixel data is subject to motion
`
`
`
`
`
`
`
`
`
`compensated differential coding prior to the discrete cosine
`
`
`
`
`
`
`
`transformation and the blocks of transformed coefficients are
`
`
`
`
`
`
`further encoded by applying run-length and variable length
`
`
`
`
`
`
`
`encoding techniques. A decoder which recovers the image
`
`
`
`
`
`
`
`sequence from the data stream reverses the encoding pro-
`
`
`
`
`
`
`
`
`
`cess. This decoder employs an entropy decoder (e.g. a
`
`
`
`
`
`
`
`
`variable length decoder), an inverse discrete cosine trans-
`
`
`
`
`
`
`
`
`form processor, a motion compensation processor, and an
`
`
`
`
`
`
`
`
`interpolation filter.
`
`4
`
`FIG. 1 is a high level block diagram of a typical video
`
`
`
`
`
`
`
`decoding system of the prior art. The video decoder of the
`
`
`
`
`
`
`
`
`
`prior art includes an entropy decoder 110, which is usually
`
`
`
`
`
`
`
`
`a variable length decoder and a run length decoder, an
`
`
`
`
`
`
`
`
`
`inverse quantizer 120, and an inverse discrete cosine trans-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`form (IDCT) processor 130. The exemplary system also
`includes a controller 170 which controls the various com-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ponents of the decoding system responsive to the control
`information retrieved from the input bit stream by the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`entropy decoder 110. For processing of prediction images,
`the prior art system further includes a memory 160, adder
`
`
`
`
`
`
`
`
`
`140, a motion compensation processor 150, and a block to
`
`
`
`
`
`
`
`raster converter 180.
`
`
`
`The variable length decoder 110 receives the encoded
`
`
`
`
`
`
`
`
`video image signal, and reverses the encoding process to
`
`
`
`
`
`
`
`
`produce control
`information including motion vectors
`
`
`
`
`
`
`describing the relative displacement of a matching macrob-
`
`
`
`
`
`
`lock in a previously decoded image. This matching mac-
`
`
`
`
`
`
`
`roblock corresponds to a macroblock of the predicted picture
`
`
`
`
`
`that is currently being decoded. The variable length decoder
`
`
`
`
`
`
`
`
`110 also receives the quantized DCT transform coefficients
`
`
`
`
`
`
`
`
`of the blocks of either the current video image which is
`
`
`
`
`
`
`
`
`
`referred to as the residual video image, if intraframe encod-
`
`
`
`
`
`
`
`ing is used, or the difference between the current and the
`
`
`
`
`
`
`
`
`
`
`predicted video image, if interframe encoding is used. The
`
`
`
`
`
`
`
`inverse quantizer 120 receives the quantized DCT transform
`
`
`
`
`
`
`
`
`coeflicients and reconstructs the quantized DCT coefficients
`
`
`
`
`
`
`
`for a particular macroblock. The quatization matrix to be
`
`
`
`
`
`
`used for a particular block is received from the variable
`
`
`
`
`
`
`
`
`length decoder 110.
`
`
`
`The IDCT processor 130 transforms the reconstructed
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DCT coefficients to pixel values in the spatial domain (for
`each block of 8x8 matrix values representing luminance or
`
`
`
`
`
`
`
`chrominance components of the macroblock, and for each
`
`
`
`
`
`
`
`alock of 8x8 matrix values representing the differential
`
`
`
`
`
`
`
`
`uminance or differential chrominance components of the
`
`
`
`
`
`
`
`aredicted macroblock).
`If the current macroblock is not predictively encoded,
`
`
`
`
`
`
`
`hen the output matrix values are the pixel values of the
`
`
`
`
`
`
`
`
`
`
`corresponding macroblock of the current video image. If the
`
`
`
`
`
`
`macroblock is interframe encoded, the corresponding mac-
`
`
`
`
`
`
`roblock of the previous video picture frame (a reference
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`rame) is stored in memory 160 for use by the motion
`compensation processor 150. The motion compensation
`
`
`
`
`
`
`3rocessor 150 receives the previous macroblock from
`
`
`
`
`
`
`
`. memory 160 responsive to the motion vector which is
`
`
`
`
`
`
`
`
`
`received from the entropy decoder 110. The motion com-
`
`
`
`
`
`
`
`
`
`3ensation processor 150 then adds the previous macroblock
`
`
`
`
`
`
`
`
`o the current IDCT transformed macroblock (corresponding
`
`
`
`
`
`
`
`0 a residual component of the present predictively encoded
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`rame) in adder 140 to produce the corresponding macrob-
`ock of pixels for the current video image, which is then
`
`
`
`
`
`
`
`
`
`stored into the memory 160.
`
`
`
`
`
`II. Down Conversion System
`
`
`
`
`A. Overview
`
`
`FIG. 2 is a high level block diagram of an exemplary
`
`
`
`
`
`
`embodiment of a down conversion system. As shown in
`
`
`
`
`
`
`the down conversion system includes a variable
`FIG. 2,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`length decoder (VLD) 210, a run-length (R/L) decoder 212,
`an inverse quantizer 214, and an inverse discrete cosine
`
`
`
`
`
`
`
`
`transform (IDCT) processor 218.
`In addition,
`the down
`
`
`
`
`
`
`
`
`conversion system includes a down conversion filter (DCT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`filter) 216, a pre—decimation filter 240, and a down sampling
`processor 232 for filtering of encoded pictures. While the
`
`
`
`
`
`
`
`
`following describes the exemplary embodiment for a Main
`
`
`
`
`
`
`Profile, High level encoded input,
`the down conversion
`
`
`
`
`
`
`
`
`system may be implemented with any similarly encoded
`
`
`
`
`
`
`
`high resolution image bit stream.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 11 of 20
`
`Page 11 of 20
`
`

`

`
`
`US 6,249,549 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`6
`the DCT coefficients of the current
`the MPEG standard,
`
`
`
`
`
`
`
`
`received image represent
`the DCT coefficients of the
`
`
`
`
`
`
`
`residual components of the predicted image macroblocks.
`
`
`
`
`
`The horizontal components of the motion vectors are scaled
`
`
`
`
`
`
`
`since the low resolution reference pictures of previous
`
`
`
`
`
`
`
`frames stored in the reference frame memory 222 do not
`
`
`
`
`
`
`
`have the same number of pixels as the high resolution
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`predicted frame (Main Profile, High Level).
`Referring to FIG. 2,
`the motion vectors of the Main
`
`
`
`
`
`
`
`
`
`Profile, High Level bit stream provided by the VLD 210 are
`
`
`
`
`
`
`provided to the MV translator 220. Each motion vector is
`
`
`
`
`
`
`
`
`scaled by the MV translator 220 to reference the appropriate
`
`
`
`
`
`
`prediction block of the reference frame of a previous image
`
`
`
`
`
`
`
`stored in reference frame memory 222. The size (number of
`
`
`
`
`
`
`
`
`pixel values) in the retrieved block is smaller than a block of
`
`
`
`
`
`
`
`
`the corresponding high resolution block used to encode the
`
`
`
`
`
`
`
`
`current
`image; consequently,
`the retrieved block is
`
`
`
`
`
`
`up-sampled to form a prediction block having the same
`
`
`
`
`
`
`
`number of pixels as the residual block provided by the IDCT
`
`
`
`
`
`
`
`rocessor 218.
`
`
`
`
`
`
`
`The prediction block is up-sampled by the up-sampling
`arocessor 226 responsive to a control signal from the MV
`
`
`
`
`
`
`
`ranslator 220 to generate a block corresponding to the
`
`
`
`
`
`original high resolution block of pixels. Then, half pixel
`
`
`
`
`
`
`
`values are generated, if indicated by the motion vector for
`
`
`
`
`
`
`
`
`
`
`
`
`
`he tip-sampled prediction block in the half-pixel generator
`
`
`
`
`
`
`228,
`to ensure proper spatial alignment of the prediction
`alock. The up-sampled and aligned prediction block is added
`
`
`
`
`
`
`
`'n adder 230 to the current filtered block, which is, for this
`
`
`
`
`
`
`
`
`
`example, the reduced resolution residual component from
`
`
`
`
`
`
`he predicted block. All the processing is done on a mac-
`
`
`
`
`
`
`
`roblock by macroblock basis. After the motion compensa-
`
`
`
`
`
`
`ion process is complete for the current macroblock in the
`
`
`
`
`
`
`
`upsampling domain, the reconstructed macroblock is filtered
`
`
`
`
`3y the pre-decimation filter 240 and then decimated accord-
`
`
`
`
`
`
`
`ingly in the down sampling processor 232. The pre-
`
`
`
`
`
`
`
`
`decimation filter 240 operates with a fixed kernel size to
`
`
`
`
`
`
`
`
`ilter spatial pixel values across block and macroblock
`
`
`
`
`
`
`
`Doundaries. Thus, the pre-decimation filter 240 is an inter-
`
`
`
`
`
`
`alock filter. The pre-decimation filter 240 acts to reduce
`
`
`
`
`
`
`
`
`alocking artifacts in the decoded image which may result,
`
`
`
`
`
`
`
`or example, from the intra-block frequency domain filter
`
`
`
`
`
`
`
`216. The decimation process does not reduce the resolution
`
`
`
`
`
`
`
`
`of the image but simply removes redundant pixels from the
`
`
`
`
`
`
`
`
`ow resolution filtered image.
`
`
`
`
`Once the downsampled macroblocks for an image are
`
`
`
`
`
`
`
`
`available,
`the display conversion block 280 adjusts the
`
`
`
`
`
`
`
`
`image for display on a low resolution television display by
`
`
`
`
`
`
`
`filtering the vertical and horizontal components of the down-
`
`
`
`
`
`
`
`sampled image in the VPF 282 and the HZPF 284 respec-
`
`
`
`
`
`
`
`
`
`
`tively.
`B. Macroblock Prediction
`
`
`
`Since the reference frames of previous images are down
`
`
`
`
`
`
`
`
`sized, the received motion vectors pointing to these frames
`
`
`
`
`
`
`
`
`may also be translated according to the conversion ratio. The
`
`
`
`
`
`
`
`
`following describes the motion translation for the luminance
`
`
`
`
`
`
`
`
`block, for example, in the horizontal direction. One skilled
`
`
`
`
`
`
`
`
`in the art would easily extend the following discussion to
`
`
`
`
`
`
`
`
`
`motion translation in the vertical direction if used. Denoting
`
`
`
`
`
`
`
`x and y as the current macroblock address in the original
`
`
`
`
`
`
`
`
`image frame, Dx as the horizontal decimation factor and mvx
`
`
`
`
`
`
`
`
`as the half pixel horizontal motion vector of the original
`
`
`
`
`
`
`
`
`
`
`
`5
`
`The down conversion system also includes a motion
`
`
`
`
`
`
`
`vector (MV) translator 220, a high resolution motion block
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`generator 224 including up-sampling processor 226 and
`half-pixel generator 228 and a reference frame memory 222.
`
`
`
`
`
`
`
`
`In addition,
`the system includes a display conversion
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`block 280 including a vertical programmable filter (VPF)
`282 and horizontal programmable filtcr (HZPF) 284. The
`
`
`
`
`
`
`
`
`display conversion block 280 converts downsampled images
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`into images for display on a particular display having a
`lower resolution.
`
`
`The down conversion filter 216 performs an intra—block
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`lowpass filtering of the high resolution (e.g. Main Profile,
`
`
`
`
`
`
`
`
`High Level DCT) coefficients in the frequency domain. The
`pre-decimation filter 240 performs an inter-block low pass
`
`
`
`
`
`
`
`filtering of the spatial pixel values. The down sampling
`
`
`
`
`
`
`
`
`
`processor 232 eliminates selected spatial pixel values by
`
`
`
`
`
`
`
`decimation of the lowpass filtered Main Profile, High Level
`
`
`
`
`
`
`
`
`
`picture to produce a set of pixel values which can be
`
`
`
`
`
`
`
`
`
`
`displayed on a monitor having lower resolution than that
`
`
`
`
`
`
`
`required to display a Main Profile, High Level picture. The
`
`
`
`
`
`
`
`
`exemplary reference frame memory 222 stores the spatial
`
`
`
`
`
`
`
`
`pixel values corresponding to at
`least one previously
`
`
`
`
`
`
`
`
`decoded reference frame having a resolution corresponding
`
`
`
`
`
`
`to the down-sampled picture. For non-intra macroblock
`
`
`
`
`
`
`
`encoding, the MV translator 220 scales the motion vectors
`
`
`
`
`
`
`
`
`for each block of the received picture consistent with the
`
`
`
`
`
`
`
`
`
`reduction in resolution, and the low resolution motion block
`
`
`
`
`
`
`
`
`generator 224 receives the decimated low resolution motion
`
`
`
`
`
`
`
`
`blocks provided by the reference frame memory 222,
`
`
`
`
`
`
`
`
`up-samples these motion blocks and generates half pixel
`
`
`
`
`
`
`
`
`values to provide motion blocks at the half pixel accuracy
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`which exhibit good spatial correspondence to the decoded
`and filtered differential pixel blocks.
`
`
`
`
`
`The operation of this exemplary embodiment of a down
`
`
`
`
`
`
`conversion system for intra-macroblock encoding is now
`
`
`
`
`
`
`described. The Main Profile, High Level bit-stream is
`
`
`
`
`
`
`
`
`received and decoded by VLD 210. In addition to header
`
`
`
`
`
`
`
`information used by the HDTV system,
`the VLD 210
`
`
`
`
`
`
`
`
`
`provides DCT coefficients for each block and macroblock,
`
`
`
`
`
`
`
`
`and motion vector information. The DCT coefficients are run
`
`
`
`
`
`
`
`
`
`length decoded in the R/L decoder 212 and inverse quan-
`
`
`
`
`
`
`
`
`
`tized by the inverse quantizer 214. The VLD 210 and R/L
`
`
`
`
`
`
`
`
`
`decoder 212 correspond to the entropy decoder 110 of FIG.
`
`
`
`
`
`
`
`
`1.
`
`Since the received video image represented by the DCT
`
`
`
`
`
`
`
`
`coefficients is a high resolution picture, the DCT coefficients
`
`
`
`
`
`
`
`of each block are lowpass filtered before decimation of the
`
`
`
`
`
`
`
`
`
`high resolution video image.