`
`
`
`
`
`
`(12) United States Patent
`
`
`
`US 6,792,045 B2
`(10) Patent N0.:
`
`
`
`
`
`
`Matsumura et al.
`
`
`(45) Date of Patent:
`Sep. 14, 2004
`
`USOO6792045B2
`
`......... 382/239
`4/2003 Miyagoshi et a1.
`6,549,671 B1 *
`
`
`
`
`
`
`3/2003 Zhang et al.
`.......... 375/24027
`2003/0043923 A1 *
`
`
`
`
`
`
`4/2003 Yoshinari .......
`375/24025
`2003/0067989 A1 *
`
`
`
`
`
`5/2003 Higuchi et al.
`............... 386/68
`2003/0091332 A1 *
`
`
`
`
`
`
`FOREIGN PATENT DOCUMENTS
`
`
`0 779 744
`6/1997
`
`
`9—214979
`8/1997
`11—252566
`9/1999
`
`EP
`JP
`JP
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(54)
`
`
`
`(75)
`
`
`
`
`
`IMAGE SIGNAL TRANSCODER CAPABLE
`
`
`
`OF BIT STREAM TRANSFORMATION
`
`
`
`SUPPRESSING DETERIORATION 0F
`
`
`PICTURE QUALITY
`
`
`
`
`
`
`Inventors: Tetsuya Matsumura, Hyogo (JP);
`
`
`
`
`Toyohiko Yoshida, IIyogo (JP)
`
`
`
`
`
`
`(73) Assignee: Renesas Technology Corp., Tokyo (JP)
`
`
`
`
`
`
`
`( * ) Notice:
`Subject to any disclaimer, the term of this
`
`patent is extended or adjusted under 35
`
`
`
`
`
`
`
`
`U.S.C. 154(b) by 388 days.
`
`(65)
`
`(30)
`
`(56)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(21) Appl. N0.: 09/769,415
`
`
`(22) Filed:
`Jan. 26, 2001
`
`
`
`
`
`Prior Publication Data
`
`
`
`US 2001/0010707 A1 Aug. 2, 2001
`
`
`
`
`
`
`
`
`Foreign Application Priority Data
`
`(JP)
`....................................... 2000—022541
`Jan. 31, 2000
`
`
`
`
`
`
`
`
`
`
`(51)
`Int. Cl.7 .................................................. H04N 7/12
`(52) U.S. Cl. ................................ 375/2401
`
`
`
`
`
`(58) Field of Search
`375/24001, 240.02,
`
`
`
`
`
`375/24003, 240.1, 240.11, 240.12, 24013—24016,
`
`
`
`
`240.25, 240.26, 240.28, 240.29; 348/397.1—399.1,
`
`
`
`
`390.1, 424.1, 424.2, 425.1, 425.3, 425.4,
`
`
`
`
`
`426.1, 441, 452, 382/233—236, 238—239;
`
`
`
`
`386/27, 33, 109, 1117112; H04N 7/12
`
`
`
`
`
`References Cited
`
`
`U.S. PATENT DOCUMENTS
`
`
`
`Perlman ................ 375/24003
`6,151,361 A * 11/2000
`
`
`
`
`
`
`
`6,160,844 A * 12/2000 Wilkinson .
`....... 375/240
`
`
`
`
`
`
`6,285,716 B1 *
`9/2001 Knee et a1.
`............ 375/24028
`
`
`
`
`
`
`
`
`6,421,385 B1 *
`7/2002 Uenoyama et a1.
`...... 375/2402
`
`
`
`
`
`
`
`
`6,516,135 B1 *
`2/2003 Higuchi et a1.
`............. 386/112
`
`
`
`
`
`
`
`
`
`(57)
`
`ABSTRACT
`
`
`
`
`
`
`
`OTHER PUBLICATIONS
`
`
`Wee et a1, “Efficient processing of compressed Video”,
`
`
`
`
`
`
`Conference Record of the Thirty—Second Asilomar Confer-
`
`
`
`
`
`ence on Signal, Systems & Computers, vol. I, pp. 853—857,
`
`
`
`
`
`
`NOV. 1998*
`
`
`“MPEG Digital Video—Coding Standards”, IEEE Signal
`
`
`
`
`
`Processing Magazine, pp. 82—100, Sep. 1997.*
`
`
`
`
`
`
`“VTR Will Be Obsolete, Heated Competition for the Suc-
`
`
`
`
`
`
`
`cessor,” by Harada et a1., Nikkei Electronics, Sep. 6, 1999,
`
`
`
`
`
`
`pp. 107—116.
`
`
`* cited by examiner
`
`
`
`Primary Examiner—Young Lee
`
`
`
`
`
`
`
`
`
`
`(74) Attorney, Agent, or Firm—McDermott Will & Emery
`LLP
`
`
`
`An MPEG2 dccodcr portion dccodcs an input bit stream and
`
`
`
`
`
`
`
`
`
`outputs a digital decoded image while extracting coding
`
`
`
`
`
`
`
`information and supplying the same to a control portion. An
`
`
`
`
`
`
`
`MPEG2 encoder portion re—encodes the digital decoded
`
`
`
`
`
`
`
`image output from the MPEG2 decoder portion. Coding
`
`
`
`
`
`
`
`
`information supplied from the control portion is reflected on
`
`
`
`
`
`
`
`a coding parameter in re-encoding. Transcoding between the
`
`
`
`
`
`
`
`MPEG2 standard and the DV standard can also be executed
`
`
`
`
`
`
`
`
`by arranging a decoder or an encoder corresponding to the
`
`
`
`
`
`
`DV standard in place of either the MPEG2 decoder portion
`
`
`
`
`
`
`
`
`or the MPEG2 encoder portion.
`
`
`
`
`
`
`13 Claims, 12 Drawing Sheets
`
`
`
`
`
`
`TO 50
`
`
`
`
`TO 50
`
`
`
`MB LAYER
`UPPER LAYER
`
`
`
`INFORMATION
`INFORMATION
`
`
`
`STORAGE
`STORAGE
`
`
`
`PORTION
`PORTION
`
`
`
`~ MACROBLOCK TYPE
` - Gd(n)
`
`
`' MOTION VECTOR PREDICTIVE RANGE
`' CBP
`
`
`
`
`
`
`
`,NFORMATION
`- MOTION PREDICTION MODE
`- PICTURE TYPE
`
`
`
`' MOTION VECTOR
`' BIT RATE
`
`
`
`
`
`' PICTURE SIZE
`
`
`STREAM
`STREAM
`INPUT BIT
`FRAME
`MPEGZ
`
`
`
`
`
`
`
`
`
`
`
`STREAM
`ANALYSIS
`DECODING
`BUFFER
`BUFFER
`
`
`
`
`
`
`
`
`
`
`PORTION
`PORTION
`FUNCTION
`PORTION
`(MPEG2)
`
`
`
`
`
`
`
`
`
`
`PORTION
`
`
`16
`
`
`
`
`
`
`
`
`
`INFORMATION
`
`
`
`
`
`DIGITAL
`
`DECODED
`
`IMAGE (TO 31)
`
`
`
`
`
`
`
`
`
`
`
`Page 1 of21
`
`GOOGLE EXHIBIT 1009
`
`Page 1 of 21
`
`GOOGLE EXHIBIT 1009
`
`

`

`
`
`
`100
`
`
`
`CONTROL PORTION
`
`50
`
`
`
`
`
`
`
`
`CODING
`.
`CODING
`
`
`INFORMATION
`PARAMETER
`
`
`
`
`
`
`
`
`
`
`
`FIG. 1
`
`INPUT BIT
`
`STREAM
`
`(MPEG2)
`
`
`
`
`
`DECODER
`
`
`PORTION
`PORTION
`DECODED
`
`
`
`
`
`
`DIGITAL
`
`ENCODEFI
`
`OUTPUT BIT
`
`STREAM
`
`(MPEGZ)
`
`IMAGE
`
`
`
`10
`
`30
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mama'S'fl
`
`
`
`vooz‘171ms
`
`ZIJOImus
`
`zasvo‘Z6L‘9sn
`
`Page 2 of 21
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 2
`
`20
`
`T050
`
`T050
`
`
`
`
`
`22
`
`11
`
`UPPERLAYER
`INFORMARON
`
`
`
`STORAGE
`PORRON
`
`MBLAYER
`WFORMARON
`
`
`
`STORAGE
`PORRON
`
`-de)
`- MOTION VECTOR PREDICTIVE RANGE,
`‘ GOP STRUCTURE
`- PICTURE TYPE
`- BIT RATE
`-HCTURESEE
`
`UPPER LEVEL
`INFORMATION
`
`~MACROBLOCKTYPE
`« CBP
`‘ Q SCALE
`- MOTION PREDICTION MODE
`- MOTION VECTOR
`
`MB LEVEL
`INFORMATION
`
`INPUTBH‘
`STREAM
`(MPEG3
`
`STREAM
`ANALYSB
`PORNON
`
`
`
`STREAM
`BUFFER
`PORRON
`
`
`MPEG2
`DECOONN3
`
`FUNCHON
`
`
`PORRON
`
`
`
`12
`
`I4
`
`16
`
`DKHTAL
`DECODED
`IMAGE(TOBM
`
`FRAME
`BUFFER
`PORUON
`
`18
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1119chST]
`
`
`
`vooz‘171ms
`
`ZIJOZ199118
`
`zasvo‘Z6L‘9sn
`
`Page 3 of 21
`
`

`

`
`
`.%C%NTG%WFE
`GEW)Pw[WWCDGEENENETGR.GmMT.RJCInFluNBNDPIUEEES.DDD-EE0wWGR3C0A0
`
`P
`
`BH313B
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`GDP
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`8
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`GOP
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`gm.5:2:m8.E.3imag,maScm385%5
`
`Page 4 of 21
`
`

`

`
`
`mama'S'fl
`
`m 8 E
`
`g
`h
`
`g3
`8,
`4:
`3N
`
`:
`m
`E;\D
`.NG
`1‘}.
`e:N
`
`FIG. 4
`
`GOP
`
`DECODED1MAGE(n)H
`
`.__.,
`
`dGGdddG de GdGde Gd Gd Gd Gd Gd Gd Gd Gd
`CODE QUANTITY
`(."UMBEROFGENERATEDBITS)11) (2))13)
`(4d)5)(
`15)
`17)
`18) 19)10)111)(12)113)(1411(5)(15)1)1117)118)
`
`GdG
`
`OF EACH PICTURE Gd(n)
`
`
`
`TARGET CODING QUANTIW Ge(n
`FOR EACH P1CTURE1N REENCODING Ge Ge Ge GeGeeG Ge Ge Ge Ge Ge Ge Ge Ge Ge Ge Ge Ge
`(1)12)(3)
`14)
`15))16))17)
`13)
`91) 110) (11) 12) (13) 14) 115) 115)(17) 113)
`
`IMAGE SUBJECTED TO RE-ENCODING
`
`—>
`
`|_____.______.__J
`GOP
`
`Page 5 of 21
`
`

`

`FIG. 5
`
`DECODED GOP STRUCTURE
`
`I
`
`GOPI
`
`GOP2
`
`II
`
`II
`
`GOPS
`
`I
`
`II,
`
`[134—11)
`
`m‘-—UJ
`
`“ad—"0'
`
`CD‘———U3
`
`m*——UJ
`
`'U‘———-"O
`
`(134—03
`
`UJ‘—'—“‘CU
`
`1
`
`w+~—-w
`
`CD<—‘Ul
`
`03‘—"’—ED
`
`W‘fl—w
`
`v4——'U
`
`DECODED PICTURE (n)
`
`DECODED PICTURE TYPE
`
`PICTURE'TYPE IN RE-ENCODING
`
`IMAGE SUBJECTED
`
`TO FIE—ENCODING
`
`REvENCODED GOP STRUCTURE L_________JL_____|L____.___I
`GOP1
`GOP2
`GOP3
`
`
`
`mama'S'fl
`
`
`
`vooz‘171ms
`
`ZIJ0E199118
`
`zasvo‘Z6L‘9sn
`
`Page 6 of 21
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`m a 5Nc
`
`
`
`
`
`
`
`E
`
`g3
`a
`:‘
`E
`
`C!CI)
`
`'3}\D
`.Nc
`4}
`U1
`WN
`
`FIG. 6
`
`
`
`51
`
`
`
`
`mamaST]
`
`
`
`
`
`
`
`
`
`
`
`CODING
`INFORMATION
`
`INPUT BIT
`
`STREAM
`
`(MPEG2)
`
`DECODER
`PORTION
`
`DIGITAL
`DECODED
`JMAGE
`
`.
`
`ENCODER
`PORTION
`
`OUTPUT BIT
`SJSEQM
`(
`2)
`
`110
`
`130
`
`Page 7 of 21
`
`

`

`FIG. 7
`
`210
`
`CONTROL PORTION
`
`52
`
`CODING
`
`INFORMATION
`
`
`
`CODING
`PARAMETER
`
`
`
`
`I PREPROCESSING I MPEGZ
`I
`MPEG2 I POSTPROCESSING
`
`F w,
`DECODcR m I m
`PORTION
`ENCODER
`PORTION
`PORTION
`
`
`
`
`
`mmB”
`STREAM —«~~
`.(MPEGQ)
`
`STANDARD DIGITAL
`DECODED IMAGE
`
`(exJTU-R-656/601)
`
`OUTPUTBIT
`STREAM
`(MPEGZ)
`
`210
`
`230
`
`1119chST]
`
`70%‘I71'das
`
`ZIJ0L199118
`
`zasvo‘Z6L‘9sn
`
`Page 8 of 21
`
`

`

`FIG. 8
`
`INPUT BIT
`
`STREAM
`
`(MPEG2)
`
`
`
`am
`
`
`
`
`
`
`mamaST]
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`70%‘171ms
`
`
`
`
`
`Z1.108mus
`
`ZHsvo‘Z6L‘9sn
`
`.
`
`‘
`
`CONTROL PORTION
`
`
`
`53
`
`
`
`
`
`
`
`
`
`CODING INFORMATION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`I MPEG2
`PREPROCESSING' MPEG2
` STANDARD DIGITAL
`DECODER
`‘ PORTION
`
` PORTION
`
`DECODED IMAGE
`(ex.|TU-R-656/601)
`
`
`I
`
`OUTPUT BIT
`
`STREAM
`
`(MPEG2)
`
`ENCODER
`
`PORTION
`
`10
`
`230
`
`Page 9 of 21
`
`

`

`FIG. 9
`
`202
`
`CODING
`INFORMATION
`
`CONTROL PORTION
`
`'
`
`
`
`CODING
`PARAMETER
`
`
`
`
`
`I MPEGZ
`
`
`
`
`
`PORTION
` ENCODER
`
`PORTION
`PORTION
`
`
`
`
`
`INPUT BIT
`
`STREAM
`
`(MPEG2)
`
`POSTPROCESSING
`
`DECQDER
`
`STANWIRD DIGITAL
`DECODED IMAGE
`
`(ex. lTU-R-656/601 )
`
`OUTPUT BIT
`
`STREAM
`
`(MPEG2)
`
`210
`
`230
`
`
`
`mama'S'fl
`
`
`
`vooz‘171ms
`
`ZIJO6mus
`
`zasvo‘Z6L‘9sn
`
`Page 10 of 21
`
`

`

`FIG. 10
`
`.
`
`‘
`
`CONTROL PORTION
`
`
`
`3&3
`
`55
`
`CODING
`
`
`
`PARAMETER
`
`
`
`OUTPUT BIT
`
`STREAM
`M0262)
`
`270
`
`230
`
`INPUT BIT
`
`STREAM
`
`(DY)
`
`PORTION
`
`POSTPROCESSIIG
`
`PORTION
`
`
`
`
`'PREPROCESSING
`MPEGB
`I
`avDECODER
`
`
`
`
`
`
`ENCODEFI
`STANDARD DIGn'AL
`'
`PORTION
`DECODED IMAGE
`
`
`
`
`(ex.ITU-R£56/601)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mama'S'fl
`
`
`
`vooz‘171ms
`
`ZIJOOI199118
`
`zasvo‘Z6L‘9sn
`
`Page 11 of 21
`
`

`

`FIG. 11
`
`TO 55
`
`
` MB LAYER
`
`
`INFORMATION
`
`
`
`STORAGE
`
`PORTION
`
`NO
`
`STREAM
`
`STREAM
`
`(DV)
`
`
`
`
`
`
`FRAME
`STREAM
`DV DECODING
`INPUT BIT
`BUFFER
`
`BUFFER
`ANALYSIS
`
`
`
`
`PORTION
`PORTION
`PORTION
`PORTION
`
`
`
`
`
`
`78
`72
`74
`76
`
`FUNCTION
`
`mamaST]
`
`70%‘171ms
`
`ZIJOII1991IS
`
`zasvo‘Z6L‘9sn
`
`DIGITAL
`DECODED
`
`IMAGE (TO 17)
`
`Page 12 of 21
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 12
`
`
`
`mamaST]
`
`
`
`g);
`
`
`
`
`
`CONTROL PORTION
`
`
`
`56
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`70%‘I71'das
`
`ZIJOZI199118
`
`zasvo‘Z6L‘9sn
`
`INPUT BIT
`
`STREAM
`(WEE?)
`
`CODING
`
`INFORMATION
`
`
`PREPROCESSING I DV ENCODER I
`
`
`POSTPROCESSWG
`PORIION
`roman
`PORTION
`
`
`STANDARD DBLTAL
`
`DECODEDW
`
`(exJTU-H 656/601 )
`
`
`
`
`
`
`
`
`CODING
`PARAMETER
`
`OUTPUTB”
`mm
`
`210
`
`370
`
`Page 13 of 21
`
`

`

`
`
`US 6,792,045 B2
`
`1
`
`IMAGE SIGNAL TRANSCODER CAPABLE
`
`
`
`OF BIT STREAM TRANSFORMATION
`
`
`
`SUPPRESSING DETERIORATION OF
`
`
`PICTURE QUALITY
`
`
`
`
`
`2
`
`bit stream once coded is deteriorated in picture quality as
`
`
`
`
`
`
`
`compared with the source image. When re-coding the dete-
`
`
`
`
`
`
`
`
`riorated image, the picture quality may be further remark-
`
`
`
`
`
`
`
`ably deteriorated depending on the coding system.
`
`
`
`
`
`
`SUMMARY OF THE INVENTION
`
`
`
`An object of the present invention is to provide an image
`
`
`
`
`
`
`
`
`
`
`signal transcoder capable of performing transcoding while
`suppressing deterioration of picture quality between coding
`
`
`
`
`
`standards identical to or different from each other.
`
`
`
`
`
`
`
`Briefly stated, the present invention provides an image
`
`
`
`
`
`
`
`signal transcoder temporarily decoding an input bits stream
`
`
`
`
`
`
`
`of coded image signal and re-coding the image signal under
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`a prescribed system, comprising a decoding processing
`
`
`
`
`
`
`
`portion, a coding processing portion and a control portion.
`
`
`
`
`
`
`
`
`The decoding processing portion decodes the input bit
`stream according to a prescribed coding standard and out—
`
`
`
`
`
`
`
`puts decoded image data while extracting coding informa-
`
`
`
`
`
`
`
`
`tion of the input bit stream. The coding processing portion
`
`
`
`
`
`
`
`
`
`re-codes the decoded image data under the prescribed cod-
`
`
`
`
`
`
`
`
`
`ing standard and forms an output bit stream. The control
`
`
`
`
`
`
`
`
`
`portion sets a coding parameter in the re-coding on the basis
`
`
`
`
`
`
`
`of the coding information.
`
`
`
`
`According to another aspect of the present invention, an
`
`
`
`
`
`
`
`
`
`
`
`
`
`image signal transcoder temporarily decoding an input bits
`stream of coded image signal and re-coding the image signal
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`under a prescribed system comprises a decoding processing
`
`
`
`
`
`
`
`portion, a coding processing portion and a control portion.
`
`
`
`
`
`
`
`
`The decoding processing portion decodes the input bit
`stream according to a first coding standard and outputs
`
`
`
`
`
`
`
`decoded image data while extracting coding information of
`
`
`
`
`
`
`
`the input bit stream. The coding processing portion re-codes
`
`
`
`
`
`
`
`
`
`the decoded image data under a second coding standard and
`
`
`
`
`
`
`
`
`
`forms an output bit stream. The control portion sets a coding
`
`
`
`
`
`
`
`
`
`parameter in the re-coding on the basis of the coding
`
`
`
`
`
`
`
`
`
`information.
`
`Accordingly, a principal advantage of the present inven-
`
`
`
`
`
`
`tion resides in that the coding parameter in the re—coding is
`
`
`
`
`
`
`
`
`set on the basis of the coding information obtained when
`
`
`
`
`
`
`
`
`
`
`decoding the input bit stream and hence deterioration of the
`
`
`
`
`
`
`
`
`
`picture quality following transcoding between identical cod-
`
`
`
`
`
`
`
`ing standards can be suppressed.
`
`
`
`
`Further, the coding parameter in the re-coding under the
`
`
`
`
`
`
`
`second coding standard is set on the basis of the coding
`
`
`
`
`
`
`
`
`
`information obtained when decoding the input bit stream
`
`
`
`
`
`
`
`according to the first coding standard and hence deteriora-
`
`
`
`
`
`
`
`tion of the picture quality following transcoding between
`
`
`
`
`
`
`
`different coding standards can be suppressed.
`
`
`
`
`
`The foregoing and other objects, features, aspects and
`
`
`
`
`
`
`
`advantages of the present
`invention will become more
`
`
`
`
`
`
`
`apparent from the following detailed description of the
`
`
`
`
`
`
`
`invention when taken in conjunction with the
`present
`
`
`
`
`
`
`
`
`
`accompanying drawings.
`BRIEF DESCRIPTION OF TIIE DRAWINGS
`
`
`
`
`FIG. 1 is a schematic block diagram showing the structure
`
`
`
`
`
`of an image signal
`transcoder 100 according to a first
`
`
`
`
`
`
`embodiment of the present invention;
`
`
`
`
`FIG. 2 is a block diagram illustrating the structure of an
`
`
`
`
`
`
`
`MPEG2 decoder portion 11;
`
`
`
`
`FIG. 3 illustrates re—encoding by an MPEG2 encoder
`
`
`
`
`
`
`portion 31 with reference to picture types;
`
`
`
`
`
`FIG. 4 illustrates bit rate control in the re—encoding by the
`
`
`
`
`
`
`
`MPEG2 encoder portion 31,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`BACKGROUND OF THE INVENTION
`
`
`1. Field of the Invention
`
`
`
`
`The present invention relates to an apparatus for perform-
`
`
`
`
`
`
`
`ing bit stream transformation for re-coding a temporarily
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`decoded input image signal under a prescribed system (such
`a series of decoding and re—coding are hereinafter collec—
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tively referred to as transcoding), and more particularly, it
`relates to an image signal transcoder capable of executing
`
`
`
`
`
`
`
`transcoding on an input bit stream as to an image signal
`
`
`
`
`
`
`
`
`coded under the MPEG2 standard or the DV standard.
`
`
`
`
`
`
`
`
`
`
`
`
`
`2. Description of the Background Art
`In recent years, data processing of image signals such as
`
`
`
`
`
`
`
`
`Video data remarkably tends to digitization. While an impor-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tant point of digital data processing of image signals is
`
`
`
`
`
`
`compression coding according to a signal compression
`
`
`
`
`
`
`
`technique, the standards of compression coding systems for
`
`
`
`
`
`
`
`
`image signals (hereinafter also referred to as image com-
`
`
`
`
`
`
`pression systems) are not necessarily unified.
`
`
`
`
`
`One of representative standards is the MPEG2 (Moving
`
`
`
`
`
`
`Picture Experts Group 2) standard employed for a public
`
`
`
`
`
`
`
`set-top box (STB) capable of receiving and reproducing
`
`
`
`
`
`digital broadcasting, a DVD-Video which is a DVD (Digital
`Versatile Disc) for movie/music Video works, a DVD—RTR
`
`
`
`
`
`
`which is a recording/reproducing DVD and the like. The
`
`
`
`
`
`MPEG2 standard is described in literature “ISO-IEC/JTCl
`
`
`
`
`SC29 DISI3818, Part 2, 1994” in detail.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A DVC (Digital Video Cassette) which is a public digital
`VTR employs a dedicated DV standard referred to as Mini
`
`
`
`
`
`
`
`DV as the image compression system. The DV standard is
`
`
`
`
`
`
`
`described in literature “SD Specifications of Consumer-Use
`
`
`
`
`
`Digital VCR", HD Digital VCR Conference, December
`
`
`
`
`
`
`
`I994, in detail.
`
`
`Such digitization of image data processing is conceivably
`
`
`
`
`
`further expanded in the future in consideration of transition
`
`
`
`
`
`of TV broadcasting to digitization, accessibility to other
`
`
`
`
`
`devices indispensable to formation of a home network and
`
`
`
`
`
`the like.
`
`
`
`
`
`
`
`
`Such progress in digitization of signal processing increas—
`ingly leads to the necessity for an application product
`
`
`
`
`
`
`
`
`
`capable of treating the aforementioned image compression
`
`
`
`
`
`
`systems in fusion, i.e., an application product receiving a bit
`
`
`
`
`
`
`
`stream formed under each coding standard, decoding the bit
`
`
`
`
`
`
`
`
`
`stream and re-coding the decoded bit stream according to a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`prescribed image compression system thereby enabling stor-
`age of image data between media having different coding
`
`
`
`
`
`
`
`
`standards, i.e., different image compression systems.
`
`
`
`
`
`
`For example, application of a VTR receiving an input bit
`
`
`
`
`
`
`
`stream according to the MPEG2 standard corresponding to
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`a set-top box (STB), transcoding the bit stream to a different
`bit rate and writing the bit stream in a recording/reproducin g
`
`
`
`
`
`
`
`
`
`
`
`
`
`DVD (DVD-RTR)
`is assumable. Therefore,
`transcoding
`between bit streams of image data under different coding
`
`
`
`
`
`
`
`
`standards or between bit streams of image data having
`
`
`
`
`
`
`
`
`
`different bit rates under the same coding standard.
`
`
`
`
`
`
`
`
`Both of the MPEG2 standard and the DV standard
`
`
`
`
`
`
`
`
`
`referred to as the representative coding standards, which are
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`not reversible coding systems (lossless coding systems), are
`subjected to some distortion in the process of compression
`
`
`
`
`
`
`and expansion. Consequently, a decoded image based on a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`60
`
`65
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 14 of21
`
`Page 14 of 21
`
`

`

`
`
`US 6,792,045 B2
`
`3
`
`FIG. 5 illustrates re-encoding reflecting GOP structures of
`
`
`
`
`
`
`a decoded image;
`
`
`
`FIG. 6 is a schematic block diagram showing the structure
`
`
`
`
`
`
`of an image signal transcoder 101 according to a second
`
`
`
`
`
`
`
`embodiment of the present invention;
`
`
`
`
`FIG. 7 is a schematic block diagram showing the overall
`
`
`
`
`
`
`structure of an image signal transcoder 200 according to a
`
`
`
`
`
`
`
`third embodiment of the present invention;
`
`
`
`
`
`FIG. 8 is a schematic block diagram showing the overall
`
`
`
`
`
`structure of another image signal transcoder 201 according
`
`
`
`
`
`
`
`to the third embodiment of the present invention;
`
`
`
`
`
`
`
`FIG. 9 is a schematic block diagram showing the overall
`
`
`
`
`
`
`structure of still another image signal transcoder 202 accord-
`
`
`
`
`
`
`
`
`ing to the third embodiment of the present invention;
`
`
`
`
`
`
`
`FIG. 10 is a schematic block diagram showing the struc-
`
`
`
`
`
`
`ture of an image signal transcoder 300 according to a fourth
`
`
`
`
`
`
`
`embodiment of the present invention;
`
`
`
`
`FIG. 11 is a block diagram showing the structure of a DV
`
`
`
`
`
`20
`decoder portion 70; and
`
`
`
`
`FIG. 12 is a schematic block diagram showing the struc-
`
`
`
`
`
`
`ture of another image signal transcoder 301 according to the
`
`
`
`
`
`
`
`fourth embodiment of the present invention.
`
`
`
`
`
`DESCRIPTION OF THE PREFERRED
`
`
`EMBODIMENTS
`
`10
`
`
`
`
`15
`
`
`
`
`
`
`
`Embodiments of the present invention are now described
`
`
`
`
`
`
`in detail with reference to the drawings.
`
`
`
`
`
`
`As for transcoding of image signals described with ref-
`
`
`
`
`
`
`erence to the embodiments of the present invention; the
`
`
`
`
`
`
`
`following three cases are assumed by way of illustration:
`
`
`
`
`
`
`
`
`
`
`
`
`
`(1) transcoding between identical coding standards for
`decoding a bit stream coded under the MPEG2 standard
`
`
`
`
`
`and re-encoding the bit stream under the MPEG2
`
`
`
`
`
`
`
`standard;
`
`
`
`
`
`
`
`(2) transcoding between different coding standards for
`decoding a bit stream coded under the DV standard and
`
`
`
`
`
`
`
`re-encoding the bit stream under the MPEG2 standard;
`
`
`
`
`
`
`
`and
`
`
`
`
`
`
`
`
`(3) transcoding between different coding standards for
`decoding a bit stream coded under the MPEG2 standard
`
`
`
`
`
`
`and re-encoding the bit stream under the DV standard.
`
`
`
`
`
`
`
`
`First Embodiment
`
`
`With reference to a first embodiment of the present
`
`
`
`
`
`
`
`invention; an image signal transcoder 100 corresponding to
`
`
`
`
`
`
`the case (1) among the aforementioned three cases is
`
`
`
`
`
`
`
`
`described. The case (2) and (3) will be described in a fourth
`
`
`
`
`
`
`
`
`
`embodiment later.
`
`
`Referring to FIG. 1,
`the image signal transcoder 100
`
`
`
`
`
`
`
`
`according to the first embodiment of the present invention
`
`
`
`
`
`
`
`comprises an MPEG2 decoding block 10 decoding an input
`
`
`
`
`
`
`bit stream encoded under the MPEG2 standard and trans-
`
`
`
`
`
`
`
`
`
`forming the same to a digital decoded image and an MPEG2
`
`
`
`
`
`
`
`coding block 30 re-encoding the digital decoded image
`
`
`
`
`
`
`
`
`under the MPEG2 standard and forming an output bit
`
`
`
`
`
`
`
`
`
`stream.
`
`The MPEG2 decoding block 10 and the MPEG2 coding
`
`
`
`
`
`
`
`
`block 30 include an MPEG2 decoder portion 11 and an
`
`
`
`
`
`
`
`
`MPEG2 encoder portion 31 executing actual decoding and
`
`
`
`
`
`
`
`
`
`
`actual re-encoding respectively.
`The image signal
`transcoder 100 further comprises a
`
`
`
`
`
`
`
`control portion 50 having a function of holding coding
`
`
`
`
`
`
`
`
`information obtained in decoding in the MPEG2 decoder
`
`
`
`
`
`
`portion 11 and fitting the coding information to calculation
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`of a coding parameter
`in re—encoding by the MPEG2
`encoder portion 31.
`
`
`
`
`
`
`
`
`
`30
`
`
`
`35
`
`
`
`
`
`
`40
`
`45
`
`60
`
`65
`
`Page 15 of21
`
`4
`
`the MPEG2 decoder portion 11
`Referring to FIG. 2;
`
`
`
`
`
`
`
`includes a stream analysis portion 12 receiving the input bit
`
`
`
`
`
`
`
`
`stream; a stream buffer portion 14 for temporarily holding an
`
`
`
`
`
`
`output of the stream analysis portion 12, an MPEG2 decod-
`
`
`
`
`
`
`
`
`ing function portion 16 decoding the bit stream output from
`
`
`
`
`
`
`
`
`
`the stream buffer portion 14 by a constant quantity at a time
`
`
`
`
`
`
`
`and a frame buffer portion 18 for temporarily storing the
`
`
`
`
`
`
`
`
`
`decoded image obtained by the MPEG2 decoding function
`
`
`
`
`
`
`
`portion 16. The frame buffer portion 18 outputs the digital
`
`
`
`
`
`
`
`
`
`
`decoded image to the MPEG2 encoder portion 31.
`
`
`
`
`
`
`
`The MPEG2 decoder portion 11 further includes an upper
`
`
`
`
`
`
`
`layer information storage portion 20 for storing information
`
`
`
`
`
`
`of upper layers extracted in the stream analysis portion 12
`
`
`
`
`
`
`
`
`and an MB layer information storage portion 22 for storing
`
`
`
`
`
`
`information of a macroblock layer level obtained by the
`
`
`
`
`
`
`
`MPEG2 decoding function portion 16 in MPEG2 decoding
`
`
`
`
`
`
`
`processing.
`The bit stream of an image signal according to the
`
`
`
`
`
`
`
`
`MPEG2 standard has a hierarchical structure formed by the
`
`
`
`
`
`
`
`
`following layers:
`
`sequence layer
`information)
`
`
`
`
`
`
`
`
`GOP (group of picture) layer (including GOP level header
`information)
`
`
`
`
`
`
`
`picture layer (including picture level header information)
`
`
`
`
`
`
`
`slice layer (including slice level header information)
`
`
`
`
`
`macroblock (MB)
`layer (including macroblock level
`information)
`
`block layer
`
`
`sequence encode code
`
`
`
`
`
`
`
`
`
`
`Abit stream of arbitrary dynamic image data is formed by
`the aforementioned layers; each of which includes a portion
`
`
`
`
`
`
`
`referred to as a header portion. Each header portion has
`
`
`
`
`
`
`
`
`
`information related to the layer inserted as a code under a
`
`
`
`
`
`
`
`
`
`predetermined syntax.
`The stream analysis portion 12 extracts information nec-
`
`
`
`
`
`
`
`essary for transcoding including the bit rate and the picture
`
`
`
`
`
`
`
`
`
`
`size of the input bit stream; the picture type which is picture
`
`
`
`
`
`
`
`
`
`
`level information indicating the type of the coding system of
`
`
`
`
`
`
`
`each picture and the GOP structure of each picture frotn the
`
`
`
`
`
`
`
`
`
`header portions of the sequence; GOP and picture layers
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(these layers are hereinafter also referred to as upper layers)
`of the bit stream having such a hierarchical structure.
`
`
`
`
`
`
`
`
`Simultaneously with such extraction of the information
`
`
`
`
`
`
`
`related to the upper layers, the stream analysis portion 12
`
`
`
`
`
`
`
`
`
`counts and obtains the code quantity of each picture for each
`
`
`
`
`
`
`
`
`
`
`picture layer level. In the following description, symbol
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Gd(n) represents the code quantity of an n-th (n: natural
`
`
`
`
`
`
`
`number) picture in the input bit stream.
`The stream analysis portion 12 stores each obtained
`
`
`
`
`
`
`
`
`picture code quantity Gd(n) and the information related to
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the upper layers necessary for transcoding in the upper layer
`information storage portion 20. The information held in the
`
`
`
`
`
`
`
`
`upper layer information storage portion 20 is supplied to the
`
`
`
`
`
`
`
`control portion 50 and the MPEG2 decoding function por-
`
`
`
`
`
`
`
`
`tion 16.
`
`
`The stream buffer portion 14 temporarily buffers the bit
`
`
`
`
`
`
`
`
`stream output from the stream analysis portion 12 and
`
`
`
`
`
`
`
`
`outputs the same by a prescribed quantity, set in correspon-
`
`
`
`
`
`
`dence to the throughput of the MPEG2 decoding function
`
`
`
`
`
`
`portion 16; at a time.
`
`
`
`The MPEG decoding function portion 16 decodes the bit
`
`
`
`
`
`
`
`
`stream output from the stream buffer portion 14 by the
`
`
`
`
`
`
`
`
`
`prescribed quantity at a time with the information related to
`
`
`
`
`
`
`
`the upper layers extracted in the stream analysis portion 12
`
`
`
`
`
`
`
`
`and outputs the decoded image.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(including sequence level header
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 15 of 21
`
`

`

`5
`
`The frame buffer portion 18 temporarily holds the
`
`
`
`
`
`
`
`
`decoded image output from the MPEG2 decoding function
`
`
`
`
`
`
`
`
`portion 16 and outputs the same to the MPEG2 encoder
`
`
`
`
`
`
`
`
`
`portion 31 as the digital decoded image.
`
`
`
`
`
`The MPEG2 decoding function portion 16 extracts mac-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`roblock (MB) level information in the process of decoding.
`The macroblock level information is represented by quan-
`
`
`
`
`
`
`tization characteristic values (MQUANT), a macroblock
`
`
`
`
`
`type and information of motion vectors. The extracted
`
`
`
`
`
`
`
`
`macroblock level
`information is held in the MB layer
`
`
`
`
`
`
`
`
`
`information storage portion 22 and supplied to the control
`
`
`
`
`
`
`
`
`
`portion 50.
`The control portion 50 holds the coding information
`
`
`
`
`
`
`
`
`related to the upper layers and the macroblock level infor-
`
`
`
`
`
`
`
`
`
`mation extracted through decoding of the input bit stream
`
`
`
`
`
`
`
`
`and uses the information for re-encoding.
`
`
`
`
`
`
`Re—encoding by the MPEG encoder portion 31 based on
`
`
`
`
`
`
`the coding information supplied from the control portion 50
`
`
`
`
`
`
`
`
`is now described.
`
`
`
`The first embodiment is described with reference to a
`
`
`
`
`
`
`
`
`
`
`
`
`
`technique of minimizing image deterioration resulting from
`re-encoding When the bit rate of the input bit stream is
`
`
`
`
`
`
`
`
`
`
`equivalent to or higher than a target bit rate in re-encoding.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`This technique is applicable for re-encoding an input bit
`stream of 6 Mbps at a bit rate of 3 Mbps, for example.
`
`
`
`
`
`
`
`FIG. 3 illustrates re-encoding by the MPEG2 encoder
`
`
`
`
`
`
`
`
`
`
`
`
`portion 31 With reference to the picture types (picture coding
`
`types).
`In the MPEG2 standard,
`three coding types for an I
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`picture (intra-coded picture), a P picture (predictive-coded
`
`
`
`
`
`picture) and a B picture (bidirectionally predictive-coded
`
`
`
`
`
`
`
`
`
`picture) are present as the types of pictures. The picture type
`is information indicating with which one of the I, P and B
`
`
`
`
`
`
`
`
`pictures each picture is coded.
`
`
`
`
`The I picture, which is coded With closed information in
`
`
`
`
`
`
`
`only the single picture, can be decoded with only the
`
`
`
`
`
`
`
`
`
`
`information of the I picture itself.
`
`
`
`
`As to the P picture for which a temporally preceding
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`decoded I or P picture is used as a predictive picture (picture
`
`
`
`
`
`
`employed as the reference for computing difference),
`Whether to code the difference between the same and a
`
`
`
`
`
`
`
`
`motion-compensated predictive picture or to code the I
`
`
`
`
`
`
`
`picture Without computing the difference can be selected in
`
`
`
`
`
`
`macroblock units in response to the efficiency.
`
`
`
`
`
`For the B picture, three types of pictures, i.e., a temporally
`
`
`
`
`
`
`
`preceding decoded I or P picture, a temporally subsequent
`
`
`
`
`
`
`decoded I or P picture and an interpolation picture formed by
`
`
`
`
`
`
`these pictures are used. The most efficient method is selected
`
`
`
`
`
`
`
`
`
`from coding of difference of these three types of pictures
`
`
`
`
`
`
`
`
`after motion compensation and intra coding in macroblock
`
`
`
`
`
`
`
`units.
`
`Referring to FIG. 3, image signal data under the MPEG2
`
`
`
`
`
`
`
`
`standard has a GOP formed by one or a plurality ofI pictures
`
`
`
`
`
`
`
`and zero or a plurality of non-I pictures. In the example
`
`
`
`
`
`
`
`
`
`
`shown in FIG. 3, the GOP is formed by 15 pictures (n=1 to
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`15) and the cycle of the I or P pictures is 3.
`The MPEG2 encoder portion 31 decides the coding
`
`
`
`
`
`
`
`
`parameter in re—encoding under the MPEG2 standard on the
`
`
`
`
`
`
`basis of the picture types extracted by the stream analysis
`
`
`
`
`
`
`
`
`portion 12 and supplied to the control portion 50 through the
`
`
`
`
`
`
`
`
`upper layer information storage portion 20.
`
`
`
`
`
`
`More specifically,
`the MPEG2 encoder portion 31
`
`
`
`
`
`
`matches the picture types in re-encoding With the corre-
`
`
`
`
`
`
`
`sponding pictures included in the input bit stream.
`
`
`
`
`
`
`
`In other words,
`the decoded image decoded by the
`
`
`
`
`
`
`
`
`
`MPEG2 decoding function portion 16 is decoded on the
`
`
`
`
`
`
`
`basis of the picture types previously described on the bit
`
`
`
`
`
`
`
`
`
`
`
`
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`60
`
`
`
`
`65
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`US 6,792,045 B2
`
`6
`
`stream as the picture level information. Further, re-encoding
`
`
`
`
`
`
`
`is executed While holding the same relation between the
`
`
`
`
`
`
`
`
`
`image subjected to re-coding and the picture types thereof as
`
`
`
`
`
`
`
`between the original decoded image and the decoded picture
`
`
`
`
`
`
`
`types. In other words, pictures decoded With I, P and B
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`pictures in decoding in the MPEG2 decoding function
`portion 16 are re-encoded With the I, P and B pictures
`
`
`
`
`
`
`
`
`respectively. Thus, deterioration of the picture quality result-
`
`
`
`
`
`
`
`
`
`
`
`
`
`ing from transcoding can be suppressed When transforming
`the input bit stream according to the MPEG2 standard to a
`
`
`
`
`
`
`
`
`lower bit rate.
`
`
`
`Bit rate control in re-encoding is now described.
`
`
`
`
`
`
`
`
`
`
`
`
`
`Referring to FIG. 4,
`the code quantity Gd(n) of each
`
`
`
`
`
`
`
`picture corresponding to an n-th (n: natural number)
`decoded image is counted by the stream analysis portion 12
`
`
`
`
`
`
`
`and held in the upper layer information portion 20.
`
`
`
`
`
`
`
`
`The MPEG2 encoder portion 31 sets a coding quantity in
`
`
`
`
`
`
`
`re-coding in correspondence to the counted code quantity
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Gd(n) of each picture on the input bit stream thereby
`executing assignment of the code quantity. When the bit rate
`
`
`
`
`
`
`
`of the input bit stream is 6 Mbps and a target bit rate in
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`re-encoding is 3 Mbps,
`for example,
`the target coding
`
`
`
`
`
`
`
`
`
`quantity for each picture is 1/2. Assuming that Ge(n) repre-
`
`
`
`
`
`
`
`
`
`sents the target coding quantity for each picture in re-coding,
`the relation between the same and the counted code quantity
`
`
`
`
`
`
`
`
`
`