USOO6456663BI
`
`
`
`
`
`
`
`
`US 6,456,663 B1
`(10) Patent No.:
`(12) United States Patent
`
`
`
`
`
`
`
`
`(45) Date of Patent:
`Sep. 24, 2002
`Kim
`
`
`
`
`
`(54) DCT DOMAIN DOWN CONVERSION
`
`
`
`
`
`SYSTEM THAT COMPENSATES FOR IDCT
`
`
`
`
`MISMATCH
`
`
`
`
`(75)
`
`
`
`
`
`
`
`Inventor: Hee-Yong Kim, Plainsboro, NJ (US)
`
`
`
`
`
`
`(73) Assignee: Matsushita Electric Industrial Co.,
`
`
`
`
`
`Ltd., Osaka (JP)
`
`
`( * ) Notice:
`
`
`
`
`
`
`
`
`
`Subject to any disclaimer, the term of this
`
`
`
`
`patent is extended 01' adjusted under 35
`
`
`
`use 1546’) by 0 days.
`
`
`
`(58) Field of Search
`
`
`
`
`
`
`
`(21) Appl. No.: 09/537,346
`.
`.
`
`
`
`
`
`(22) Flled'
`Mar. 29’ 2000
`Int. Cl.
`.................................................. H04N 7 30
`51
`
`
`
`
`
`
`7
`()
`/
`
`
`
`
`
`
`(52) US. Cl. ''''''''''''''''''''' 375/24025; 375/240'24;
`
`375/24021
`
`
`375,240 25 240 24
`.................. 595/240] 21‘.H04N 7.30,
`
`
`
`
`
`.
`’
`
`
`(56)
`
`
`
`
`
`References Cited
`US. PATENT DOCUMENTS
`
`
`
`.. 348/3901
`6/1992 Raychaudhuri et a1.
`
`
`
`
`12/1996 Suzuki et a1.
`............... 714/800
`
`
`
`
`
`
`/1997 Haskell et al.
`341/200
`
`
`
`
`
`10/1998 Malladi et al.
`........ 375/24003
`
`
`
`
`
`
`5,122,875 A
`
`5,590,139 A
`
`5,604,502 A
`
`5,818,532 A
`
`
`............. 375/24016
`1/2001 Kim et al.
`6,175,592 B1 *
`
`
`
`
`
`
`6,310,919 B1 * 10/2001 Florencio ............... 375/24016
`
`
`
`
`
`
`6,373,905 B1 *
`4/2002 Yasuda et al.
`....... 375/340
`
`
`
`
`
`
`6,384,864 B1 *
`5/2002 Kim ........................... 348/441
`
`
`
`
`
`
`
`
`>I< cited by examiner
`
`
`
`Primary Examiner—Howard Brillon
`
`
`
`
`
`
`(74) Attorney, Agent, or Firm—RatnerPrestia
`
`
`
`
`
`
`
`
`
`
`ABSTRACT
`(57)
`
`
`
`
`
`
`
`
`
`To reduce aliasing during the down conversion and decoding
`
`
`
`
`
`
`
`
`
`of video signals that have been encoded according to the
`
`
`
`
`
`
`
`moving picture experts group (MPEG) standard, a discrete
`
`
`
`
`
`
`
`
`
`cosine transform (DCT) domain filter is applied to the
`unquantized DCT coefficient values. Also, partly because
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the inverse discrete transform (IDCT) operation of the
`stan ar ma
`e 1H1 emente ,
`mismatc
`
`
`
`
`
`
`IDCT
`MPEG
`d d
`yb ' pl
`'
`d
`h
`
`
`
`
`
`control processing is implemented. Concurrent implemen-
`
`
`
`
`
`
`
`
`
`
`tation of the IDCT mismatch control process and the DCT
`
`
`
`
`
`
`
`
`domain filter does not consistently produce the highest
`quality picture. Thus, the current invention is related to a
`
`
`
`
`
`
`
`robust DCT domain filter designed to maintain the higher
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`quality in downconverted images. The DCT domain filter
`sets the filter coefficient corresponding to the highest fre-
`
`
`
`
`
`
`
`
`quency band to unity to prevent modification of any coef-
`
`
`
`
`
`
`
`ficient value that has been modified by the IDCT mismatch
`
`
`
`
`
`
`
`
`operation.
`
`
`13 Claims, 4 Drawing Sheets
`
`
`
`
`
`
`
`
`30
`
`32
`
`
`
`
`
`33
`I
`
`36
`
`
`
`38
`
`
`
`28
`
`
`
`
`
`
`200
`
`—
`
`
`
`HD
`
`BIT STREAM
`
`
`
`
`6
`
`2 0
`
`
`
`
`DCT COEFFICIENT 0 62
`[PROCESSOR 41‘
`
`
`
`a
`
`
`
`3mm
`
`
`
`
`
`R/L
`DECODER
`
`
`
`
`
`FILTER
`
`
`
`
`
`
`
`
`
`
`IDCT
`
`
`
`MISMATCH
`
`
`
`
`
`
`nc'HDOMAIN IDCT
`CONTROL
`
`
`
`MV
`
`
`
`
`
`POLY
`PHASE
`
`UPSAMPLE
`
`CONTROLLER
`
`
`
`40
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`REFERENCE
`FRAME
`
`1
`
`REFERENCE
`FRAME
`
`2
`
`
`
`
`45
`
`MOTION
`
`BLOCK
`
`
`HALE PIXEL
`GENERATOR
`
`62
`
`DISPLAY CONVERSION
`
`
`
`
`.
`
`O
`
`|
`
`
`I
`DISPLAY
`
`L _ _
`RE—OROER
`
`
`Page 1 of 12
`
`GOOGLE EXHIBIT 1013
`
`Page 1 of 12
`
`GOOGLE EXHIBIT 1013
`
`

`

`
`US. Patent
`
`
`
`
`
`Sep.24,2002
`
`
`
`Sheet1,0f4
`
`
`
`
`
`US 6,456,663 B1
`
`om
`
`
`
`zQEEEmw2952
`
`oz<2255mm;
`
`zo:<m2mn_§oo
`
`SE
`
`
`
`923%
`
`255m
`
`
`
`:m-
`
`S
`
`Sago
`
`Etzm
`
`59
`
`283292
`
`SE28
`
`mmmm>z_
`
`ENfiz/So
`
`ENE/So
`
`89>
`
`:52
`
`._<zw_m
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 2 of 12
`
`Page 2 of 12
`
`
`
`
`
`

`

`US. Patent
`
`Sep.24,2002
`
`Sheet2 0f4
`
`US 6,456,663 B1
`
`m;£38095
`
`£55588
`
` mmam
`
`mm
`
`SEE102222E$805EEEEM%
`
`22,8593GI
`
`E28
`
`fix:5%
`
`mob/$25
`
`.mmsi
`
`£522::2mesmzé6%>2
`
`
` _E5:ESEVI_nEEn_
`"maxim
`
`><._n_m_o
`
`N
`
`g9
`
`_...............LWEE_
`2055248:55V5295g20:9235%;
`
`
`
`
`
`mofimfimm$3”:on
`
`Page 3 of 12
`
`Page 3 of 12
`
`
`
`
`
`

`

`
`US. Patent
`
`
`
`
`
`Sep. 24, 2002
`
`
`
`
`Sheet 3 0f4
`
`
`
`US 6,456,663 B1
`
`
`DCT COEFFICIENTS
`
`
`
`
`PERFORM TWO DIMENSIONAL SUMMATION
`
`
`
`OF DCT COEFFICIENTS:
`
`
`N
`M
`
`
`2 F(m,n)
`SUM=Z
`
`m=0 n=0
`
`
`
`
`
`
`
`42
`
`
`
`
`
`
`
`
`SUM
`
`ODD
`
`EVEN OR
`
`
`ODD?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F(M,N)
`
`
`REPLACE F(M,N)
`
`
`
`EVEN OR
`
`
`, VWTHFXNLN)—1
`
`ODD?
`
`WITH F(M,N) +1
`
`REPLACE F(M,N)
`
`
`
`
`
`
`PROVIDE RESULTANT
`
`
`DCT COEFFICIENTS
`
`
`
`TO DCT COEFFICIENT
`
`PROCESSOR
`
`
`
`52
`
`
`
`FIG.3
`
`Page 4 of 12
`
`Page 4 of 12
`
`

`

`
`US. Patent
`
`
`
`
`
`Sep. 24, 2002
`
`
`
`
`Sheet 4 0f4
`
`
`
`US 6,456,663 B1
`
`I
`I
`
`|
`I
`
`
`
`
`I
`FIRST OUTPUT PIXEL
`I
`x7 x5 x5 x4 x3 x2 x XD'x0 x1 x2 x3 x4 x5 X5 x7I x7 x5 x5 x4 x3 x2 x1 x0
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`“14“13“12“11“10“9 “8'
`“1 “6 “5 “4 “3 “2 “1 “O :T
`|
`
`1
`SECOND OUTPUT PIXEL
`4
`
`
`
`
`
`
`X7 X5 X5 X4 X3 X2 X1 X0' X0 X1 X2 X3 X4 X5 X5 X7'1-X7l X5 X5 X4 X3 X2 X1 X0
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`“14“13“12“11“11O “9.'
`“O “1 “O “5 “4 “3 “2 “1
`' “O
`l
`
`
`
`
`
`111
`
`1
`
`
`
`X0
`
`x2
`
`
`
`
`FIG.4
`
`
`(PRIOR ART)
` PERFORM
`
`
`
`
`DOWN
`
`60
`
`
`
`
`
`
`
`
`
`._______
`
`
`
`
`
`
`
`CONVERSION?
`
`
`
`HMODIFIEDBY\,_____
`
`
`61m
`
`
`
`\
`11113111111011 /
`
`\ CONTROL?,
`\.__ __
`
`
`
`
`_———______1
`
`
`
`
`
`
`
`
`
`
`
`PERFORM OCT DOMAIN
`
`
`LOW PASS FILTERING
`
`ON THE DCT COEFFICIENTS
`
`
`
`
`
`68
`
`
`
`Page 5 of 12
`
`Page 5 of 12
`
`

`

`1
`
`DCT DOMAIN DOWN CONVERSION
`
`
`
`
`SYSTEM THAT COMPENSATES FOR IDCT
`
`
`
`
`MISMATCH
`
`
`
`
`
`US 6,456,663 B1
`
`
`FIELD OF THE INVENTION
`
`
`
`This invention relates to a decoder which converts and
`
`
`
`
`
`
`
`
`formats an encoded high resolution video signal, e.g.,
`
`
`
`
`
`
`
`
`MPEG-2 encoded video signal, and more specifically to a
`
`
`
`
`
`
`
`
`method and apparatus for adaptively compensating for
`
`
`
`
`
`
`
`encoder/decoder mismatch
`
`
`BACKGROUND OF THE INVENTION
`
`
`
`In the United States a standard has been proposed for
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`digitally encoded high definition television signals (HDTV).
`A portion of this standard is essentially the same as the
`
`
`
`
`
`
`
`
`
`MPEG-2 standard, proposed by the Moving Picture Experts
`
`
`
`
`
`
`
`
`
`
`
`
`
`Group (MPEG) of the International Organization for Stati-
`dardization (ISO). The standard is described in an Interna-
`
`
`
`
`
`
`
`
`
`
`
`
`tional Standard (IS) publication entitled, “Information
`
`
`
`
`
`
`Technology—Generic Coding of Moving Pictures and Asso—
`ciated Audio, Recommendation H.626”, ISO/IEC 13818-2,
`
`
`
`
`
`
`IS, 11/94 which is available from the ISO and which is
`
`
`
`
`
`
`
`
`
`hereby incorporated by reference for its teaching on the
`
`
`
`
`
`
`
`
`
`MPEG-2 digital video coding standard.
`
`
`
`
`
`The MPEG-2 standard is actually several different stan-
`
`
`
`
`
`
`
`dards. In MPEG-2, several dilferent profiles are defined,
`
`
`
`
`
`
`
`
`each corresponding to a different level of complexity of the
`
`
`
`
`
`
`encoded image. For each profile, different levels are defined,
`
`
`
`
`
`
`
`
`
`each level corresponding to a different image resolution.
`
`
`
`
`
`
`One of the MPEG-2 standards, known as Main Profile, Main
`
`
`
`
`
`
`
`Level is intended for coding video signals conforming to
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`existing television standards (i.e., NTSC and PAL). Another
`standard, known as Main Profile, High Level, is intended for
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`coding high—definition television images. Images encoded
`according to the Main Profile, High Level standard may
`
`
`
`
`
`
`
`
`have as many as 1,152 lines per image frame and 1,920
`
`
`
`
`
`
`
`
`
`pixels per line.
`
`
`
`The Main Profile, Main Level standard, on the other hand,
`
`
`
`
`
`
`
`
`defines a maximum picture size of 720 pixels per line and
`
`
`
`
`
`
`
`
`
`
`576 lines per frame. At a frame rate of 30 frames per second,
`
`
`
`
`
`
`
`
`
`signals encoded according to this standard have a data rate
`
`
`
`
`
`
`
`
`of 720x576><30 or 12,441,600 pixels per second. By
`
`
`
`
`
`
`
`contrast, images encoded according to the Main Profile,
`
`
`
`
`
`
`
`High Level standard have a maximum data rate of 1,152><
`
`
`
`
`
`
`
`
`
`1,920><30 or 66,355,200 pixels per second. This data rate is
`
`
`
`
`
`
`
`
`more than five times the data rate of image data encoded
`
`
`
`
`
`
`
`
`
`
`
`according to the Main Profile, Main Level standard. The
`
`
`
`
`
`
`
`
`standard proposed for HDTV encoding in the United States
`
`
`
`
`
`
`
`
`is a subset of this standard, having as many as 1,080 lilies per
`
`
`
`
`
`
`
`
`
`frame, 1,920 pixels per line and a maximum frame rate, for
`
`
`
`
`
`
`
`
`
`
`this frame size, of 30 frames per second. The maximum data
`
`
`
`
`
`
`
`
`
`rate for this proposed standard is still far greater than the
`
`
`
`
`
`
`
`
`
`
`maximum data rate for the Main Profile, Main Level stan—
`
`
`
`
`
`
`
`
`
`
`dard.
`
`The MPEG-2 standard defines a complex syntax which
`
`
`
`
`
`
`
`contains a mixture of data and control information. Some of
`
`
`
`
`
`
`
`this control information is used to enable signals having
`
`
`
`
`
`
`
`several different formats to be covered by the standard.
`
`
`
`
`
`
`
`
`These formats define images having differing numbers of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`picture elements (pixels) per line, differing numbers of lines
`per frame or field, and differing numbers of frames or fields
`
`
`
`
`
`
`
`
`per second. In addition, the basic syntax of the MPEG-2
`
`
`
`
`
`
`
`
`
`Main Profile defines the compressed MPEG-2 bit stream
`
`
`
`
`
`
`
`representing a sequence of images in five layers,
`the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`sequence layer, the group of pictures layer, the picture layer,
`the slice layer and the macroblock layer. Each of these layers
`
`
`
`
`
`
`
`
`
`
`10
`
`
`
`
`
`
`
`
`
`
`I\)m
`
`40
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`2
`
`is introduced with control information. Finally, other control
`
`
`
`
`
`
`
`information, also known as side information, (e.g. frame
`
`
`
`
`
`
`type, macroblock pattern, image motion vectors, coefficient
`
`
`
`
`
`
`
`
`
`
`
`zig-zag patterns and dequantization information) is inter-
`spersed throughout the encoded bit stream.
`
`
`
`
`
`Implementation of this standard in television studios and
`
`
`
`
`
`in viewer’s homes is expected to be incremental. At least
`
`
`
`
`
`
`until the television studios provide a large amount of pro-
`
`
`
`
`
`
`
`gramming in HDTV format, viewers are likely to retain their
`
`
`
`
`
`
`
`
`
`
`
`
`
`standard definition television (SDTV) receivers but may
`want to view HDTV programming in SDTV format. Thus,
`
`
`
`
`
`
`the operation of decoding the encoded bitstream may
`
`
`
`
`
`
`
`include the process of down conversion. Down conversion
`
`
`
`
`
`
`converts a high definition input picture into a lower resolu-
`
`
`
`
`
`
`
`tion picture for display on a lower resolution monitor. Down
`
`
`
`
`
`
`
`conversion of high resolution Main Profile, High Level
`
`
`
`
`
`
`
`pictures to Main Profile, Main Level pictures, or other lower
`
`
`
`
`
`
`
`resolution picture formats, has gained increased importance
`
`
`
`
`
`
`for reducing implementation costs of HDTV Down conver-
`
`
`
`
`
`
`sion allows replacement of expensive high definition moni-
`
`
`
`
`
`
`tors used with Main Profile, High Level encoded pictures
`
`
`
`
`
`
`
`
`with inexpensive existing monitors that have a lower picture
`
`
`
`
`
`
`
`resolution to support, for example, Main Profile, Main Level
`
`
`
`
`
`
`
`encoded pictures, such as NTSC or PAL.
`
`
`
`
`
`Processing of video signals in the MPEG—2 standard
`
`
`
`
`
`
`
`
`includes converting the video signals between the spatial
`
`
`
`
`
`
`
`
`domain and the frequency domain using discrete cosine
`
`
`
`
`
`
`
`
`transforms (DCTs) and inverse discrete cosine transforms
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(IDC’l‘s) during the respective encoding and decoding stages
`of the process. When the DCT used by an encoder and the
`
`
`
`
`
`
`
`
`
`IDCT used by a decoder have different implementations, a
`
`
`
`
`
`
`dilference may occur in the reconstructed pixels between the
`
`
`
`
`
`
`
`encoder and the decoder. This difference may accumulate
`
`
`
`
`
`
`
`
`and become visible in the decoded picture. This distortion is
`
`
`
`
`
`
`
`
`LA) .11
`~ called IDCT mismatch distortion because the visible distor—
`
`
`
`
`
`
`
`
`
`tion in the decoded picture is caused by different DCT/IDCT
`
`
`
`
`
`
`
`implementations in the encoder and decoder. IDCT mis-
`
`
`
`
`
`
`
`match is a serious problem for high quality coding schemes
`
`
`
`
`
`
`
`
`such as those conforming to the MPEG-1 and MPEG-2
`
`
`
`
`
`
`
`standards. Thus, in order to achieve high coding quality,
`
`
`
`
`
`
`
`IDCT mismatch must be controlled.
`
`
`
`
`IDCT mismatch occurs when the result of an IDCT is very
`
`
`
`
`
`
`
`close to a half integer. A slight difference between the
`
`
`
`
`
`
`
`
`encoder and decoder can result in two different rounded
`
`
`
`
`
`
`
`
`
`integer values. This difference is most
`likely to cause
`
`
`
`
`
`
`
`
`
`problems when the values of the IDCT results are close to
`
`
`
`
`
`
`
`
`a half integer (e.g., 1.5). When the IDCT results are rounded
`
`
`
`
`
`
`
`
`
`
`to the nearest integer, one implementation may round up,
`
`
`
`
`
`
`
`
`
`because its resultant value is only slightly greater than the
`
`
`
`
`
`
`
`
`value of the half integer, while the other implementation
`
`
`
`
`
`
`
`
`
`rounds down, because its resultant value is only slightly less
`
`
`
`
`
`
`
`
`than the value of the half integer. Accordingly, if a decoder
`
`
`
`
`
`
`
`
`that rounds up processes a signal from an encoder that
`
`
`
`
`
`
`
`
`
`rounds down or vice versa, IDCT mismatch errors may
`
`
`
`
`
`
`
`
`occur. When a decoded frame containing errors is used to
`
`
`
`
`
`
`
`decode a sequence of predicted frames,
`the error may
`
`
`
`
`
`
`
`
`become more visible with each predicted frame that is based
`
`
`
`
`
`
`
`
`
`on the erroneous frame. One approach to control IDCT
`
`
`
`
`
`
`
`
`mismatch includes oddification methods. The processing of
`
`
`
`
`
`
`oddification typically involves setting specific coefficients to
`
`
`
`
`
`
`an odd value. In this approach, the reconstructed or dequan-
`
`
`
`
`
`
`
`
`tized DCT data is oddified at the decoder before the IDCT
`
`
`
`
`
`
`
`
`step.
`
`
`
`
`~
`
`
`
`
`
`
`
`60
`
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`The present invention provides an apparatus for use in a
`
`
`
`
`
`
`
`video decoder which decodes digital video signals that have
`
`
`
`
`
`
`
`
`
`
`Page 6 of 12
`
`Page 6 of 12
`
`

`

`
`3
`been encoded into frequency domain coefficient values. The
`
`
`
`
`
`
`
`
`apparatus comprises a mismatch control processor, a fre-
`
`
`
`
`
`
`quency domain filter having filter coellicients corresponding
`
`
`
`
`
`
`
`to frequency bands, and an inverse frequency domain trans-
`
`
`
`
`
`
`
`
`form processor. The mismatch control processor is coupled
`
`
`
`
`
`
`
`to receive the frequency domain coefficient values and to
`
`
`
`
`
`
`
`
`process the frequency domain coefficient values according
`
`
`
`
`
`
`
`to a mismatch control algorithm to produce processed fre-
`
`
`
`
`
`
`
`quency domain coefficient values. The frequency domain
`
`
`
`
`
`
`
`filter is coupled to receive the processed frequency domain
`
`
`
`
`
`
`
`
`coefficient values and to provide lowpass filtered frequency
`
`
`
`
`
`
`
`domain coefficient values. If down conversion is performed,
`
`
`
`
`
`
`the frequency domain filter coefficient corresponding to the
`
`
`
`
`
`
`
`highest frequency band is set to 1 at least for image blocks
`
`
`
`
`
`
`
`
`tha have been modified by the mismatch control processor.
`
`
`
`
`
`
`
`
`The inverse frequency domain transform processor
`is
`
`
`
`
`
`
`
`coupled to the frequency domain filter for transforming the
`
`
`
`
`
`
`
`
`output coefficient values provided by the frequency domain
`
`
`
`
`
`
`
`filter into spatial domain picture elements.
`
`
`
`
`
`
`It is to be understood that both the foregoing general
`
`
`
`
`
`
`
`
`
`
`description and the following detailed description are
`
`
`
`
`
`
`
`exemplary, but are not restrictive, of the invention.
`
`
`
`
`
`
`
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`
`
`
`The invention is best understood from the following
`
`
`
`
`
`
`
`
`detailed description when read in connection with the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`accompanying drawing. It is emphasized that, according to
`common practice, the various features of the drawing are not
`
`
`
`
`
`
`
`
`to scale. On the contrary,
`the dimensions of the various
`
`
`
`
`
`
`
`
`
`features are arbitrarily expanded or reduced for clarity.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Included in the drawing are the following figures:
`
`
`
`
`
`
`
`
`FIG. 1 (prior art)
`is a block diagram illustrating an
`
`
`
`
`
`
`exemplary configuration of a Moving Picture Experts Group
`
`
`
`
`
`(MPEG) encoding and compression system;
`FIG. 2 is a block diagram illustrating an exemplary
`
`
`
`
`
`
`
`
`
`
`configuration of a MPEG decoding and decompression
`system incorporating down conversion;
`
`
`
`
`FIG. 3 is a flow diagram illustrating an exemplary IDCT
`
`
`
`
`
`mismatch control process in MPEG; and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 4 (prior art) illustrates the multiplication pairs for the
`first and second output pixel values of a block mirror filter.
`
`
`
`
`
`
`
`
`
`FIG. 5 is a flow diagram of an exemplary embodiment of
`
`
`
`
`
`a DCT lowpass filter in accordance with the invention;
`
`
`
`
`
`
`
`
`
`DETAILED DESCRIPTION OF THE
`
`
`INVENTION
`
`
`
`
`
`
`
`
`
`Although illustrated and described above with reference
`
`
`
`
`
`
`
`to certain specific embodiments,
`the present invention is
`
`
`
`
`
`
`
`nevertheless not intended to be limited to the details shown.
`
`
`
`
`
`
`
`Rather, various modifications may be made in the details
`
`
`
`
`
`
`
`within the scope and range of equivalents of the claims and
`
`
`
`
`
`
`
`
`
`without departing from the invention.
`
`
`
`
`
`FIG. 1 is a block diagram illustrating an exemplary
`
`
`
`
`
`
`
`
`
`
`
`
`
`configuration of a Moving Picture Experts Group (MPEG)
`
`
`
`
`encoding and compression system.
`The system shown in FIG. 1 compresses each picture of
`
`
`
`
`
`
`
`the video input signal, block-by-block, until all the blocks
`
`
`
`
`
`
`
`
`
`constituting the picture have been processed. A block may
`
`
`
`
`
`
`
`
`comprise a group of 8x8 pixels and a macroblock may
`
`
`
`
`
`
`
`comprise a group of 16x16 luminescence pixels and two to
`
`
`
`
`
`
`
`four 8x8 blocks of chrominance pixels. A current macrob-
`
`
`
`
`
`
`
`lock is fed into motion estimation block 22 to generate a
`
`
`
`
`
`
`
`motion estimation based on a previous reference picture.
`
`
`
`
`
`
`The summation circuit 2 is coupled to receive both the video
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`input signal and the motion compensated prediction signal 4.
`
`
`
`US 6,456,663 B1
`
`4
`
`The summation circuit 2, determines the pixel—by—pixel
`
`
`
`
`
`
`
`difference between the current video input signal picture
`
`
`
`
`
`
`
`
`block and its corresponding motion compensated prediction
`
`
`
`
`
`
`block 4. The resulting blocks of differences 6, are coupled to
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the discrete cosine transform (DCT) processor 8. The DCT
`
`
`
`
`
`
`processor 8, applies orthogonal transform processing to the
`difference blocks 6. The resulting blocks of frequency
`
`
`
`
`
`
`
`domain transform coefficients are provided to the quantizer
`
`
`
`
`
`
`
`10. The quantizer 10 quantizes the blocks of transform
`
`
`
`
`
`
`
`
`
`coefficients to reduce the number of bits used to represent
`
`
`
`
`
`
`
`the transform coefficients. The variable—length coder 12
`
`
`
`
`
`
`
`subjects the blocks of quantized transform coefficients from
`
`
`
`
`
`
`
`the quantizer 10 to variable-length coding, such as Huff-
`
`
`
`
`
`
`
`mann coding and run-length coding. The resulting blocks of
`
`
`
`
`
`
`
`
`coded transform coefficients, along with motion vectors, are
`
`
`
`
`
`
`
`
`then fed as a bit stream, via the output buffer 14, to a digital
`
`
`
`
`
`
`
`
`
`
`transmission medium.
`
`
`Acontrol signal indicating the number of bits stored in the
`
`
`
`
`
`
`
`
`
`output buffer 14 is fed back to the quantizer 10. The
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`quantizer 10 adjusts the quantizing step size in response to
`the control signal to prevent
`the output buffer 14 from
`
`
`
`
`
`
`
`
`
`
`overflowing or underflowing and also to maintain a required
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`bit rate. Increasing or decreasing the quantizing step size
`respectively decreases or increases the number of bits fed
`
`
`
`
`
`
`
`into the output buffer 14.
`
`
`
`
`
`The block of quantized transform coefficients provided by
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`quantizer 10, is also coupled to the inverse quantizer 16. The
`
`
`
`
`
`inverse quantizer 16 performs processing complementary to
`
`
`
`
`
`
`
`he quantizing processing performed by the quantizer 10.
`The inverse quantized data is subjected to mismatch control
`
`
`
`
`
`
`
`17, and the resulting block of transform coefficients is fed to
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`he inverse discrete cosine transform (IDCT) processor 18,
`where it is inversely orthogonally transformed by processing
`
`
`
`
`
`complementary to the orthogonal transform processing per-
`
`
`
`
`
`
`ormed by the discrete cosine transform processor 8.
`
`
`
`
`
`
`The resulting restored spatial domain difference block is
`
`
`
`
`
`
`
`coupled to the summation circuit 20. The summation circuit
`
`
`
`
`
`
`
`
`20 is also coupled to receive the motion compensated
`
`
`
`
`
`
`
`3rediction block 4 for the current video input signal picture
`
`
`
`
`
`
`
`
`alock from the motion estimation, prediction, and compen—
`
`
`
`
`
`
`
`
`sation circuit 22. The summation circuit 20 performs pixel-
`
`
`
`
`
`
`
`
`Jy-pixel addition between the restored difference block from
`
`
`
`
`
`
`
`he inverse discrete cosine transform circuit 18 and the
`
`
`
`
`
`
`
`
`
`matching motion compensated prediction block 4 from the
`
`
`
`
`
`
`
`motion estimation, prediction, and compensation circuit 22
`
`
`
`
`
`
`0 provide a reconstructed picture block to the motion
`
`
`
`
`
`
`
`estimation, prediction, and compensation circuit 22.
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 2 is a block diagram illustrating an exemplary
`configuration of a MPEG decoding and decompression
`
`
`
`
`
`
`system incorporating down conversion. This embodiment of
`
`
`
`
`
`
`a decoding and decompression system 200 includes a vari-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`able length decoder (VLD) 28, a run-length (R/L) decoder
`30, an inverse quantizer 32, IDCT mismatch control 33, a
`
`
`
`
`
`
`
`
`controller 40, and a DCT coefficient processor 34. As shown
`
`
`
`
`
`
`
`
`in FIG. 2, the DCT coefficient processor 34 comprises a
`
`
`
`
`
`
`
`DCT domain filter 36, and an inverse discrete cosine trans-
`
`
`
`
`
`
`
`
`
`form (IDCT) processor 38. In an alternate embodiment of a
`
`
`
`
`
`
`
`
`
`
`
`
`decoding and decompression system without down
`conversion, the DCT coefficient processor comprises only
`
`
`
`
`
`
`
`the IDCT processor. Note that, for completeness, FIG. 2
`
`
`
`
`
`
`
`
`depicts the primary components of a MPEG decoding sys-
`
`
`
`
`
`
`
`tem incorporating down conversion. A more detailed
`
`
`
`
`
`
`
`
`
`
`
`
`
`description of this decoding processor may be found in
`pending US. patent application No. 09/169,790.
`
`
`
`
`
`
`The digital television system may receive either high-
`
`
`
`
`
`
`
`definition television (HDTV) signals, that need to be filtered
`
`
`
`
`
`
`
`10
`
`
`
`I\)m
`
`
`
`LA)LII
`
`
`
`40
`
`
`
`
`
`
`
`
`
`60
`
`
`
`
`
`
`
`
`
`
`Page 7 of 12
`
`Page 7 of 12
`
`

`

`
`
`US 6,456,663 B1
`
`6
`IDCT mismatch control. Thus it is desirable to reduce the
`
`
`
`
`
`
`
`possible occurrence of half integer resultant values.
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 3 is a flow diagram illustrating an exemplary IDCT
`mismatch control process in MPEG. DCT coefficients as
`
`
`
`
`
`
`
`provided by the inverse quantizer 32 are subjected to a
`
`
`
`
`
`
`
`
`summation process in step 42. This summation process is
`
`
`
`
`
`
`
`typically performed on a block of 8x8 DCT coefficients. The
`
`
`
`
`
`
`
`summation process in step 42 is in accordance with the
`
`
`
`
`
`
`
`
`following formula.
`
`
`M N
`
`
`
`m:0 n:0
`Sum=ZZF(m,n)
`
`
`
`10
`
`
`
`
`
`
`
`
`
`5
`
`
`
`
`
`
`
`
`
`
`and downsampled before they can be displayed on the
`viewer’s standard definition television (SDTV) monitor, or
`
`
`
`
`
`
`SDTV signals that may be displayed on the SDTV monitor.
`
`
`
`
`
`
`
`
`Controller 40 determines Whether the DCT coefficients are
`
`
`
`
`
`
`
`to be downsampled and generates a control signal 62.
`
`
`
`
`
`
`
`
`Control signal 62 is provided to switches 41 and 45, and to
`
`
`
`
`
`
`
`the DCT coefficient processor 34. For example, when an
`
`
`
`
`
`
`
`
`HDTV signal
`is received, controller 40 provides control
`
`
`
`
`
`
`
`signal 62 such that switch 41 is open and switch 45 provides
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`upsampled data to the half pixel generator (i.e., switch 45 is
`
`
`
`
`
`
`
`
`
`
`
`in the up position in FIG. 2). Control signal 62 is also
`provided to the DCT coefficient processor 34 such that the
`
`
`
`
`
`
`
`
`DCT coefficients of each block are lowpass filtered in the
`
`
`
`
`
`
`
`
`
`
`DCT domain during HDTV reception, before conversion to
`
`
`
`
`
`
`
`the spatial domain.
`
`
`
`W'hen SDTV signals are received, no down conversion or
`
`
`
`
`
`
`
`filtering is needed as these signals may be decoded and
`
`
`
`
`
`
`
`
`displayed on the SDTV monitor. In this instance, the con-
`
`
`
`
`
`
`
`
`troller 40 provides control signal 62 such that switch 41 is
`
`
`
`
`
`
`
`closed and switch 45 provides motion block data to the half
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`pixel generator (i.e., switch 45 is in the lower position in
`
`
`
`
`
`
`
`
`FIG. 2), thus bypassing the downsampling and upsampling
`operations. The controller 40 also controls the DCT coeffi-
`
`
`
`
`
`
`
`
`cient processor 34 to bypass the DCT domain filter when
`
`
`
`
`
`
`
`
`
`
`
`
`decoding the SDTV signals.
`Processor 34 may also monitor the IDCT mismatch
`
`
`
`
`
`
`
`
`control processor 33 to determine which blocks of DCT
`
`
`
`
`
`
`
`coefficients are modified by the processor 33 and which
`
`
`
`
`
`
`
`
`blocks are not modified. The processor 34 then uses this
`
`
`
`
`
`
`
`
`
`information to control the value of the highest frequency
`
`
`
`
`
`
`
`
`filter coefficient of the DCT domain filter 36, as described
`
`
`
`
`
`
`
`
`below. According to this alternate embodiment of the
`
`
`
`
`
`
`
`invention, the highest frequency filter coefficient of the DCT
`
`
`
`
`
`
`
`
`domain filter 36 is set to unity only when the filter 36 is
`
`
`
`
`
`
`
`
`
`
`
`LA)LII
`processing a block that was modified by the mismatch
`
`
`
`
`
`
`
`
`control processor 33. As a further refinement of this
`
`
`
`
`
`
`
`
`embodiment, the highest frequency filter coefficient of the
`
`
`
`
`
`
`
`DCT domain filter may be set to unity only when processing
`
`
`
`
`
`
`
`
`the row of coefficients in the modified block that includes the
`
`
`
`
`
`
`
`
`
`modified coefficient value F(M,N).
`
`
`
`
`In operation,
`the encoded bit-stream is received and
`
`
`
`
`
`
`
`
`decoded by VLD 28. In addition to header information used
`
`
`
`
`
`
`by digital television system, the VLD 28 provides run length
`
`
`
`
`
`
`
`
`
`encoded DCT coefficients for each block and macroblock,
`
`
`
`
`
`
`
`
`and motion vector information. The DCT coellicients are run
`
`
`
`
`
`
`
`
`
`length decoded in the R/L decoder 30 and inverse quantized
`
`
`
`
`
`
`
`
`by the inverse quantizer 32.
`
`
`
`
`The inverse quantizer 32 provides the DCT coefficients to
`
`
`
`
`
`
`
`the IDCT mismatch controller 33. The IDCT mismatch
`
`
`
`
`
`
`
`
`controller 33 provides DCT coefficients to the DCT filter 36
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`which may perform a lowpass filtering in the frequency
`domain by weighting the DCT coefficients with predeter—
`
`
`
`
`
`
`
`mined filter coefficient values before providing them to the
`
`
`
`
`
`
`
`
`IDCT processor 38. The IDCT processor 38 converts the
`
`
`
`
`
`
`
`
`filtered DCT coefficients into spatial pixel values by per-
`
`
`
`
`
`
`
`
`forming an inverse discrete cosine transform operation.
`
`
`
`
`
`
`
`MPEG does not specify the detail of the IDCT implemen—
`
`
`
`
`
`
`
`
`
`tation. Therefore, forms of implementation can differ. This
`
`
`
`
`
`
`
`difference is most likely to become manifest when the values
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`of the IDCT results are close to a half integer (e.g., 1.5).
`When the IDCT results are rounded to the nearest integer, it
`
`
`
`
`
`
`
`
`is possible that one implementation will round up, because
`
`
`
`
`
`
`
`
`
`its resultant value is only slightly greater than the value of
`
`
`
`
`
`
`
`
`
`the half integer, While the other will round down, because its
`
`
`
`
`
`
`
`
`
`
`resultant value is only slightly less than the value of the half
`
`
`
`
`
`
`
`
`
`integer. This mismatch becomes bigger when there are more
`
`
`
`
`
`
`
`
`
`prediction frames. To reduce the mismatch, MPEG employs
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`F(m,n) represents a two dimensional matrix of DCT
`coefficients located by indices m and n. M is the highest
`
`
`
`
`
`value of the index In, and N is the highest value of the index
`
`
`
`
`
`
`
`
`n. At step 44, it is determined if the value produced by the
`
`
`
`
`
`
`
`
`
`summation process of step 42 is even or odd. If the sum—
`
`
`
`
`
`
`
`mation value is odd, the DCT coefficients are provided to the
`
`
`
`
`
`
`
`
`
`DCT coeflicient processor 34 as provided by the inverse
`
`
`
`
`
`
`quantizer 32. If, however, the value of the summation is
`
`
`
`
`
`
`
`
`
`even, then at step 46 it
`is determined if the value of a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`particular coefficient, F(M,N) is even or odd. If F(M,N) is
`
`
`
`
`
`
`
`
`
`the value of F(M,N) is replaced with the value
`even,
`
`
`
`
`
`
`
`
`
`{F(M,N)+1} at step 48. Then the DCT coefficients with the
`replacement value are provided, at step 52,
`to the DCT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`coefficient processor 34, If the value of F(M,N) is odd, the
`
`
`
`
`
`
`
`value of F(M,N) is replaced with the value {F(M,N)—1} at
`step 50. This is equivalent to toggling the least significant bit
`
`
`
`
`
`
`
`
`
`(I .813) of the coefficient F(M,N). Then the DCT coefficients
`
`
`
`
`
`
`
`with the replacement value are provided to the DCT coef-
`
`
`
`
`
`
`
`
`
`ficient processor 34.
`
`
`
`the exemplary
`When down conversion is performed,
`
`
`
`
`
`
`embodiment of the DCT coefficient processor 34 as depicted
`
`
`
`
`
`in FIG. 2 comprises a DCT domain filter 36 and an IDCT
`
`
`
`
`
`
`
`
`processor 38. The derivation and advantages of using the
`
`
`
`
`
`
`
`
`DCT domain filter are described in an application for patent,
`
`
`
`
`
`
`
`
`DOWN CONVERSION SYSTEM USING A PRE-
`
`
`
`
`
`DECIMATION FILTER, Ser. No. 09/169,790. Briefly, the
`
`
`
`
`
`
`
`
`‘ DCT domain filter 36, which processes the DCT coefficients
`
`
`
`
`
`
`
`
`
`in the frequency domain, is an alternative to implementing
`
`
`
`
`
`
`
`a lowpass filter in the spatial domain. For example, lowpass
`
`
`
`
`
`
`
`
`
`filtering in the spatial domain is accomplished in the fre-
`
`
`
`
`
`
`
`quency domain by multiplying the DCT coefficients by
`
`
`
`
`
`
`
`
`weighting coefficients prior to performing the IDCT process.
`
`
`
`
`
`
`
`In a mathematical illustration, spatial values, x(n), can be
`
`
`
`
`
`
`
`obtained by the IDCT process described by the following
`
`
`
`
`
`
`
`
`equation:
`
`40
`
`50
`
`60
`
`
`
`
`
`
`
`_
`1””1
`71k(n+1/2)
`
`
`
`
`xm) = N;a(k)-C(kl-cos—,
`
`
`where ot(k)=1/z for k=0 and 1 otherwise.
`
`
`
`
`
`Here a one dimensional DCT is illustrated for simplicity.
`
`
`
`
`
`
`The weighting coefficients, used to accomplish lowpass
`
`
`
`
`
`
`
`
`
`
`
`
`filtering, are obtained by transforming the lowpass filter
`impulse response in the spatial domain to weighting coef-
`
`
`
`
`
`
`ficients in the frequency domain. These weighting coeffi-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`cients are represented by H‘(k) in the followi