`
`
`
`
`
`
`a2) United States Patent
`
`
`
`US 6,792,045 B2
`(10) Patent No.:
`
`
`
`
`
`
`
`
`Sep. 14, 2004
`(45) Date of Patent:
`Matsumuraet al.
`
`US006792045B2
`
`(54)
`
`
`
`
`
`(75)
`
`
`(73)
`
`
`(*)
`
`(21)
`
`(22)
`
`(65)
`
`
`
`
`
`
`
`
`
`
`
`
`IMAGE SIGNAL TRANSCODER CAPABLE
`OF BIT STREAM TRANSFORMATION
`
`
`
`SUPPRESSING DETERIORATION OF
`
`
`
`
`PICTURE QUALITY
`
`
`
`
`‘Tetsuya Matsumura, Hyogo (JP);
`
`
`
`
`Toyohiko Yoshida, ITyogo (JP)
`
`
`
`
`
`
`Assignee: Renesas Technology Corp., Tokyo (JP)
`
`
`
`
`
`
`
`Subject to any disclaimer, the term of this
`
`
`
`
`patent is extended or adjusted under 35
`
`
`
`
`US.C. 154(b) by 388 days.
`
`Inventors:
`
`Notice:
`
`
`
`09/769,415
`
`
`
`Jan. 26, 2001
`Prior Publication Data
`
`
`
`
`
`US 2001/0010707 Al Aug. 2, 2001
`
`
`
`
`Foreign Application Priority Data
`(30)
`
`
`
`
`
`Jan. 31, 2000
`cesscsseessecsessesseessessecsecseeress 2000-022541
`
`
`
`
`
`(SL) Unt. C07 onceeee HO4N 7/12
`
`
`
`
`(52) US. Ch. ccc
`cecesceeieensnecesesees 375/240.1
`
`
`
`
`
`
`(58) Field of Search.
`... 375/240.01, 240.02,
`...
`
`
`
`
`375/240.03, 240.1, 240.11, 240.12, 240.13-240.16,
`
`
`
`
`240.25, 240.26, 240.28, 240.29; 348/397.1-399.1,
`
`
`
`
`
`390.1, 424.1, 424.2, 425.1, 425.3, 425.4,
`
`
`
`
`426.1, 441, 452; 382/233-236, 238-239;
`
`
`
`
`
`386/27, 33, 109, 111-112; HO4N 7/12
`
`
`Appl. No.:
`Filed:
`
`
`(TP)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`......... 382/239
`4/2003 Miyagoshi et al.
`6,549,671 Bl *
`
`
`
`
`
`
`3/2003 Zhang etal.
`.......... 375/240.27
`2003/0043923 Al *
`4/2003 Yoshinari .......
`we 375/240,25
`2003/0067989 Al *
`
`
`
`
`
`
`
`
`
`
`
`5/2003 Higuchi et al.
`.............. 386/68
`2003/0091332 Al *
`FOREIGN PATENT DOCUMENTS
`
`
`
`0 779 744
`6/1997
`
`
`9-214979
`8/1997
`11-252566
`9/1999
`
`
`
`
`
`
`
`
`
`EP
`
`JP
`
`JP
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`Wee et al, “Efficient processing of compressed video”,
`
`
`
`
`
`Conference Record of the Thirty-Second Asilomar Confer-
`
`
`
`
`
`
`ence on Signal, Systems & Computers, vol. 1, pp. 853-857,
`Nov. 1998.*
`
`
`
`
`
`
`
`“MPEG Digital Video—Coding Standards”, IEEE Signal
`
`
`
`
`
`
`Processing, Magazine, pp. 82-100, Sep. 1997.*
`
`
`
`
`
`
`
`“VIR Will Be Obsolete, Heated Competition for the Suc-
`
`
`
`
`
`
`cessor,” by Haradaetal., Nikkei Electronics, Sep. 6, 1999,
`
`
`pp. 107-116.
`
`
`
`* cited by examiner
`
`
`
`
`Primary Examiner—Young Lee
`
`
`
`
`
`
`(74) Attorney, Agent, or Firm—McDermott Will & Emery
`LLP
`
`
`
`ABSTRACT
`(57)
`
`
`
`
`
`
`
`
`
`An MPEG?decoderportion decodes an inputbit stream and
`
`
`
`
`
`
`
`outputs a digital decoded image while extracting coding
`
`
`
`
`
`
`
`information and supplying the sameto a control portion. An
`
`
`
`
`
`
`
`MPEG2 encoder portion re-encodes the digital decoded
`
`
`
`
`
`
`
`
`image output from the MPEG2 decoder portion. Coding
`
`
`
`
`
`
`
`information supplied from the control portion is reflected on
`
`
`
`
`
`
`
`a coding parameter in re-encoding. Transcoding between the
`MPEG?standard and the DV standard can also be executed
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`by arranging a decoder or an encoder corresponding to the
`
`
`
`
`
`
`
`
`DV standard in place of either the MPEG2 decoder portion
`
`
`
`
`
`or the MPEG2encoderportion.
`
`
`
`
`13 Claims, 12 Drawing Sheets
`
`
`
`TO 50
`
`
`
`
`
`
`DIGITAL
`
`DECODED
`
`
`
`IMAGE(TO 31)
`
`
`
`(56)
`
`
`
`References Cited
`
`
`U.S. PATENT DOCUMENTS
`
`
`
`
`
`Perlman ............000 375/240.03
`
`6,151,361 A * 11/2000
`
`
`
`
`
`
`
`
`
`
`6,160,844 A * 12/2000 Wilkinson ................. 375/240
`
`
`
`
`6,285,716 Bl *
`9/2001 Knee et al. 0.0... 375/240.28
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`6,421,385 B1 *
`7/2002 Uenoyamaetal. ...... 375/240.2
`
`
`
`
`
`
`
`
`6,516,135 Bl *
`2/2003 Higuchi et al. 386/112
`
`TO 50
`
`
`
`
`
`
`
`
`
`
` MB LAYER
`UPPER LAYER
`
`INFORMATION
`INFORMATION
`
`
`STORAGE
`STORAGE
`
`
`PORTION
`PORTION
`
`
`
`
`+ MACROBLOCK TYPE
`
` + Gd{n)
`
`+ CBP
`* MOTION VECTOR PREDICTIVE RANGE
`
`
`
`
`
`
`
`
`
`
`
`MB LEVEL
`*Q SCALE
`* GOP STRUCTURE
`UPPER LEVEL
`
`
`
`
`
`
`
`
`* PICTURE TYPE * MOTION PREDICTION MODEjINFORMATIONINFORMATION
`
`
`> BIT RATE
`* MOTION VECTOR
`
`
`
`
`
`» PICTURE SIZE
`
`
`
`
`STREAM
`INPUT BIT
`STREAM
`MPEG2
`FRAME
`
`
`
`
`
`
`
`ANALYSIS
`DECODING
`BUFFER.
`STREAM
`BUFFER
`
`
`
`
`
`PORTION
`
`
`
`
`FUNCTION
`
`
`
`
`
` (MPEG2)
`PORTION
`PORTION
`
`
`
`
`PORTION
`
`
`
` 16
`
`
`Page 1 of 21
`
`GOOGLEEXHIBIT 1009
`
`Page 1 of 21
`
`GOOGLE EXHIBIT 1009
`
`

`

`FIG. 1
`
`
`
`
`100
`
`
`
`CONTROL PORTION
`
`50
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`OUTPUTBIT
`INPUTBIT ea
`STREAM
`STREAM
`
`
`DIGITAL
`ENCODER
`DECODER
`(MPEG2)
`(MPEG2)
`PORTION
`PORTION
`
`
`DECODED
`
`IMAGE
`
`
`CODING
`INFORMATION
`
`
`
`
`
`CODING
`
`PARAMETER
`
`
`MPEG2
`
`
`
`
`
`10
`
`30
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`yuajed‘Sn
`
`$007‘FI“das
`ZIJO139948
`7HSP0'76L'9SA
`
`Page 2 of 21
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 2
`
`TO 50
`
`TO 50
`
`20
`
`
`
`UPPER LAYER
`INFORMATION
`STORAGE
`PORTION
`
`
`
`
`
`ti
`
`22
`
`
`
`MB LAYER
`INFORMATION
`STORAGE
`PORTION
`
`INPUT BIT
`STREAM
`(MPEG2)
`
`
`
`STREAM
`ANALYSIS
`PORTION
`
`STREAM
`BUFFER
`PORTION
`
`12
`
`14
`
`
`MPEG2
`DECODING
`
`
`
`
`FUNCTION
`
`PORTION
`
`16
`
`FRAME
`BUFFER
`PORTION
`
`18
`
`DIGITAL
`DECODED
`IMAGE (TO 31)
`
`* MACROBLOCKTYPE
`+ Gd{n}
`“CBP
`- MOTION VECTOR PREDICTIVE RANGE.
`MB LEVEL
`*Q SCALE
`UPPER LEVEL
`* GOP STRUCTURE
`
`* PICTURE TYPE INFORMATION||‘ MOTION PREDICTION MODE|INFORMATION
`
`BIT RATE
`* MOTION VECTOR
`- PICTURE SIZE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`yuayed‘SA
`
`$007‘FI“das
`crJOZWONS
`7HSP0'76L'9SA
`
`Page 3 of 21
`
`

`

`
`
`FIG. 3
`
`DECODED IMAGE(n)
`
`DECODED PICTURE TYPE
`
`PICTURE TYPE IN RE-ENCODING
`
`IMAGE SUBJECTED
`TO RE-ENCODING
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`GOP
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`B
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`GOP
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`U.S.PatentSep.14,2004Sheet3of12US6,792,045B2
`
`
`
`
`
`Page 4 of 21
`
`

`

`FIG. 4
`
`GOP
`
`CODE QUANTITY
`
`(NUMBER OF GENERATED BITS)
`
`OF EACH PICTURE Gd(n)
`
`Gd Gd Gd
`
`F (7) a)8 (10) (11) (42) (13) (14) (15) (16) (17) (18)
`
`Gd Gd Gd Gd Gd Gd Gd Gd Gd Gd
`
`DECODEDIMAGE(ni
`“eee TILT
`
`FOR EACH PICTUREIN RE-ENCODING Ge es Ge Ge Ge Ge Ge Ge
`Ge Ge Ge Ge Ge Ge Ge G
`(1)
`(2) 3)
`(4 6))
`(6)
`(7)
`(8) 0) (10) (411) (12) (13) (14) (15) (16)(17) “a
`
`Gd Gd Gd Gd Gd
`
`——{1):_:(2).:(3)_
`
`|
`
`IMAGE SUBJECTED TO RE-ENCODING
`
`GOP
`
`—e
`
`—
`
`yuajed‘Sn
`
`$007‘FI“das
`crJ9FONS
`7HSP0'76L'9SA
`
`Page 5 of 21
`
`

`

`FIG. 5
`
`DECODED GOP STRUCTURE
`
`GOP
`
`GOP2
`
`i
`
`|
`
`GOP3
`
`DECODED PICTURE (n}
`
`{
`
`DECODED PICTURE TYPE
`
`BBtstBBPBBPBBPBB B BP
`
`PICTURE TYPE IN RE-ENCODING
`
`B Bt BBP BBP BB
`
`B
`
`B
`
`B B
`
`P
`
`IMAGE SUBJECTED
`TO RE-ENCODING
`
`RE-ENCODED GOP STRUCTURE|
`
`GOP?
`
`IL
`GOP2
`
`GOP3
`
`yuayed“Sn
`$007‘FI“das
`ZTJO§WIS
`7HSP0'76L'9SA
`
`Page 6 of 21
`
`

`

`FIG. 6
`
`
`
`—
`
`—_
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`yuayed‘S'A
`
`$007‘FI“das
`ZTJO9WIS
`7HSP0'76L'9SA
`
`51
`
`CONTROL PORTION
`
`
`
`
`
`
`
`
`
`
`
`
`CODING
`CODING
`
`
`PARAMETER
`INFORMATION
`
`
`INPUT BIT
`MPEG2
`STREAM
`DECODER
`(MPEG2)
`PORTION
`
`
`
`110
`
`130
`
`OUTPUTBIT
`STREAM
`(MPEG2)
`
`Page 7 of 21
`
`

`

`yuajed‘S'0
`
`$007‘FI“das
`
`
`
`ZIJO£948
`
`
`STAEAN —-MPEG2
`i PREPROCESSING = STEAMBIT
`POSTPROCESSING
`INPUTBIT—lcm
`
`
`
`
`wweeGa)|[DECODER||FORTION STANDARD DIGITAL||PORTION ENCODER||"fupegay
`PORTION
` DECODED IMAGE
`
`
`
`(ex.ITU-R-656/601)
`7HSP0'76L'9SA
`
`CODING
`INFORMATION
`
`CONTROL PORTION
`
`200
`
`CODING
`PARAMETER
`
`FIG. 7
`
`
`
`210
`
`230
`
`Page 8 of 21
`
`

`

`FIG. 8
`
`INPUT BIT
`STREAM
`(MPEG2)
`
`
`
`201
`
`
`
`a CONTROLPORTION
`
`
`
`
`
`
`
`
`
`
`
`
`po
`CODINGINFORMATION
`
`
`|
`ok MPEG?
` MPEG2
`
`ENCODER
`| PORTION
`STANDARDDIGITAL
`DECODER
`
`PORTION
`PORTION
`DECODED IMAGE
`
`
`
`
`(ex.ITU-R-656/601)
`
`
`
`
`
`
`
`
`
`OUTPUTBIT
`STREAM
`(MPEG2)
`
`10
`
`230
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`yuayed‘SA
`
`$007‘FI“das
`ZIJog39048
`7HSP0'76L'9SA
`
`Page 9 of 21
`
`

`

`FIG. 9
`
`202
`
`CONTROL PORTION
`
`.
`
`54
`
`CODING
`INFORMATION
`
`(ex.ITU-R-656/601)
`
`INPUT BIT
`STREAM
`(MPEG?)
`
`POSTPROCESSING
`
`STANDARD DIGTAL
`DECODED IMAGE
`
`210
`
`230
`
`PARAMETER
`
`MPEG2
`PORTION
`CODER
`
`OUTPUTBIT
`STREAM
`(MPEG?)
`
`yuayed‘SA
`
`$007‘FI“das
`ZTJ96WIS
`7HSP0'76L'9SA
`
`Page 10 of 21
`
`

`

`FIG. 10
`
`|
`
`|
`
`CONTROL PORTION
`
`
`
`
`
`
`300
`
`
`
`CODING
`PARAMETER
`
`
`
`
`
`
`
`
`
`
`
`yuayed“SN
`
`$007‘FI“das
`
`
`
`crJOOFHONS
`
`DVDECODER|_|POSTPROCESSING E PREPROCESSINGi MPEG? | SIREBIT
`
`
`(ex.ITU-R-656/601}
`
` DECODED IMAGE
`
`7HSP0'76L'9SA
`
`INPUT BIT
`
`STREAM
`PORTION
`PORTION
`STREAM
`STANDARD DIGITAL||PORTION ENCODER
`
`(MPEG2)
`)
`PORTION
`(DY)
`
`
`
`
`270
`
`230
`
`Page 11 of 21
`
`

`

`FIG. 11
`
`TO 55
`
`
`
`
`MB LAYER
`INFORMATION
`STORAGE
`PORTION
`
`yuayed‘SA
`
`$007‘FI“das
`cLF°TE39948
`7HSP0'76L'9SA
`
`70
`
`
`
`
`
`
`
`
`
`
`
`
`INPUT BIT|STREAM STREAM DV DECODING FRAME DIGITAL
`
`
`
`ANALYSIS
`BUFFER
`FUNCTION
`BUFFER
`STREAM
`DECODED
`
`
`
`
`PORTION
`PORTION
`PORTION
`PORTION
`(DV)
`IMAGE(TO 17)
`
`
`
`
`
`
`72
`74
`76
`78
`
`Page 12 of 21
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 12
`
`
`
`CONTROL PORTION
`
`56
`
`
`
`
`
`PARAMETER
`
`CODING
`INFORMATION
`
`INPUTBIT
`
`STREAM
`
`-4
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`yuayed“SN
`
`$007‘FI“das
`cLJOCLPONS
`7HSP0'76L'9SA
`
`STREAM
`
`(DV)
`
`(ex.TU-A-656/601)
`
`
`
`PREPROCESSINGi DVENCODER
`
`ATION
`
`PORTION
`
`DECODED MAGE
`
`210
`
`370
`
`Page 13 of 21
`
`

`

`
`
`US 6,792,045 B2
`
`
`1
`IMAGE SIGNAL TRANSCODER CAPABLE
`
`
`
`OF BIT STREAM TRANSFORMATION
`
`
`
`SUPPRESSING DETERIORATION OF
`
`
`
`
`PICTURE QUALITY
`
`
`2
`
`
`
`
`
`
`
`bit stream once coded is deteriorated in picture quality as
`
`
`
`
`
`
`
`
`compared with the source image. When re-coding the dete-
`
`
`
`
`
`
`
`riorated image, the picture quality may be further remark-
`
`
`
`
`
`
`ably deteriorated depending on the coding system.
`SUMMARYOF THE INVENTION
`
`
`BACKGROUND OF THE INVENTION
`
`
`
`
`
`
`
`Aoobject of the present invention is to provide an image
`
`
`
`
`
`1. Field of the Invention
`
`
`
`
`signal transcoder capable of performing transcoding while
`
`
`
`
`
`
`
`
`
`
`
`
`suppressing deterioration of picture quality between coding
`The present invention relates to an apparatus for perform-
`standards identical to or different from each other.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ing bit stream transformation for re-coding a temporarily
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`decoded input image signal under a prescribed system (such
`Briefly stated, the present invention provides an image
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`signal transcoder temporarily decoding an input bits stream
`a series of decoding and re-coding are hereinafter collec-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tively referred to as transcoding), and more particularly, it
`of coded image signal and re-coding the image signal under
`
`
`
`
`
`
`
`
`
`
`
`
`
`relates to an image signal transcoder capable of executing
`a prescribed system, comprising a decoding processing
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`transcoding on an input bit stream as to an image signal
`portion, a coding processing portion and a control portion.
`coded under the MPEG2 standard or the DV standard.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The decoding processing portion decodes the input bit
`
`
`
`
`
`
`
`
`
`
`
`
`2. Description of the Background Art
`stream according to a prescribed coding standard and out-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`puts decoded image data while extracting coding informa-
`In recent years, data processing of image signals such as
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tion of the input bit stream. The coding processing, portion
`video data remarkably tendsto digitization. While an impor-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`re-codes the decoded image data under the prescribed cod-
`tant point of digital data processing of image signals is
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ing standard and forms an output bit stream. The control
`compression coding according to a signal compression
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`portion sets a coding parameter in the re-coding onthe basis
`technique, the standards of compression coding systemsfor
`
`
`
`
`
`
`
`
`
`
`
`
`of the coding information.
`image signals (hereinafter also referred to as image com-
`
`
`
`
`
`
`
`
`
`
`
`
`pression systems) are not necessarily unified.
`According to another aspect of the present invention, an
`
`
`
`
`
`
`
`
`
`
`
`
`image signal transcoder temporarily decoding an input bits
`One ofrepresentative standards is the MPEG2 (Moving
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`stream of coded imagesignal and re-coding the image signal
`Picture Experts Group 2) standard employed for a public
`
`
`
`
`
`
`
`
`
`
`
`
`
`undera prescribed system comprises a decoding processing
`sel-lop box (STB) capable of receiving and reproducing
`
`
`
`
`
`
`
`
`
`
`
`
`portion, a coding processing portion and a control portion.
`digital broadcasting, a DVD-Video which is a DVD (Digital
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Versatile Disc) for movie/music video works, a DVD-RTR
`The decoding processing portion decodes the input bit
`
`
`
`
`
`
`
`
`
`
`
`
`which is a recording/reproducing DVD andthe like. The
`stream according to a first coding standard and outputs
`
`
`
`
`
`
`
`
`
`
`
`MPEG2standard is described in literature “ISO-IECJTC1
`decoded image data while extracting coding information of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`$C29 DIS13818, Part 2, 1994” in detail.
`the input bit stream. The coding processing portion re-codes
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the decoded image data under a second coding standard and
`ADVC (Digital Video Cassette) which is a public digital
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`forms an output bit stream. The controlportion sets a coding
`VTR employs a dedicated DV standard referred to as Mini
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`parameter in the re-coding on the basis of the coding
`DV as the image compression system. The DV standard is
`information.
`
`
`
`
`
`
`described in literature “SD Specifications of Consumcr-Use
`
`
`
`
`
`
`
`
`
`
`
`
`
`Digital VCR”, HD Digital VCR Conference, December
`Accordingly, a principal advantage of the present inven-
`
`
`
`
`
`
`
`
`
`
`1994, in detail.
`tion resides in that the coding parameter in the re-coding is
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`set on the basis of the coding information obtained when
`Suchdigitization of image data processing is conceivably
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`decoding the input bit stream and hence deterioration of the
`further expanded in the future in consideration oftransition
`
`
`
`
`
`
`
`
`
`
`
`
`picture quality following transcoding betweenidentical cod-
`of ‘I'V broadcasting to digitization, accessibility to other
`
`
`
`
`
`
`
`
`
`ing standards can be suppressed.
`devices indispensable to formation of a home nctwork and
`the like.
`
`
`
`
`
`
`
`
`
`
`Further, the coding parameter in the re-coding under the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`second coding standard is set on the basis of the coding
`Such progressin digitization of signal processing increas-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`information obtained when decoding the input bit stream
`ingly leads to the necessity for an application product
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`according to the first coding standard and hence deteriora-
`capable of treating the aforementioned image compression
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`systems in fusion, 1.e., an application product receivingabit tion of the picture quality following transcoding between
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`stream formed under each coding standard, decoding the bit
`different coding standards can be suppressed.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`stream and re-coding the decoded bit stream according to a
`The foregoing and other objects, features, aspects and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`prescribed image compression system thereby enablingstor-
`advantages of the present
`invention will become more
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`age of image data between media having different coding
`apparent from the following detailed description of the
`
`
`
`
`
`
`
`
`
`
`
`
`
`standards, i.c., different image compression systems.
`invention when taken in conjunction with the
`present
`
`
`
`
`
`
`
`
`
`For example, application of a VIR receiving an input bit
`accompanying drawings.
`
`
`
`
`
`
`stream according to the MPEG2 standard corresponding to
`
`
`
`
`
`
`
`
`
`a set-top box (STB), transcoding the bit streamto a different
`
`
`
`
`
`
`
`
`bit rate and writing the bit stream in a recording/reproducing
`
`
`
`
`
`transcoding
`DVD (DVD-RTR)
`is assumable. Therefore,
`
`
`
`
`
`
`
`
`between bit streams of image data under different coding
`
`
`
`
`
`
`
`
`
`standards or between bit streams of image data having
`
`
`
`
`
`
`
`
`different bit rates under the same coding standard.
`Both of the MPLEG2 standard and the DV standard
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`referred to as the representative coding standards, which are
`
`
`
`
`
`
`
`
`not reversible coding systems (lossless coding systems), are
`
`
`
`
`
`
`subjected to some distortion in the process of compression
`
`
`
`
`
`
`and expansion. Consequently, a decoded image based on a
`
`BRIEF DESCRIPTION OP THE DRAWINGS
`
`
`
`
`
`
`
`
`FIG.1 is aschematic block diagram showingthe structure
`
`
`
`
`
`
`of an image signal
`transcoder 100 according to a first
`
`
`
`
`embodimentof the present invention;
`
`
`
`
`
`
`
`FIG. 2 is a block diagram illustrating the structure of an
`
`
`
`
`MPEG2decoder portion 11;
`
`
`
`
`
`
`FIG. 3 illustrates re-encoding by an MPEG2 encoder
`
`
`
`
`
`portion 31 with reference to picture types;
`
`
`
`
`
`
`
`FIG.4 illustrates bit rate control in the re-encoding by the
`
`
`
`
`MPEG?encoder portion 31;
`
`
`
`
`
`
`
`10
`
`1S
`
`
`
`20
`
`
`
`
`
`
`
`
`
`
`
`30
`
`35
`
`
`
`40
`
`
`
`45
`
`
`
`
`
`
`
`
`
`
`60
`
`
`
`65
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 14 of 21
`
`Page 14 of 21
`
`

`

`
`3
`
`
`
`
`
`
`FIG.5 illustrates re-encoding reflecting GOPstructures of
`
`
`
`a decoded image;
`
`
`
`
`
`
`FIG.6 is a schematic block diagram showingthe structure
`
`
`
`
`
`
`
`of an image signal transcoder 101 according to a second
`
`
`
`
`embodiment of the present invention;
`
`
`
`
`
`
`FIG. 7 is a schematic block diagram showing the overall
`
`
`
`
`
`
`
`structure of an image signal transcoder 200 according to a
`
`
`
`
`
`third embodiment of the present invention;
`
`
`
`
`
`FIG. 8 is a schematic block diagram showing the overall
`
`
`
`
`
`
`
`structure of another image signal transcoder 201 according
`
`
`
`
`
`
`
`to the third embodimentof the present invention;
`
`
`
`
`
`
`FIG. 9 is a schematic block diagram showing the overall
`
`
`
`
`
`
`
`
`structure of still another image signal transcoder 202 accord-
`
`
`
`
`
`
`
`ing to the third embodimentof the present invention;
`
`
`
`
`
`
`FIG. 10 is a schematic block diagram showing the struc-
`
`
`
`
`
`
`
`ture of an image signal transcoder 300 accordingto a fourth
`
`
`
`
`embodiment of the present invention;
`
`
`
`
`
`FIG. 11 is a block diagram showing the structure of a DV
`20
`
`
`
`
`decoder portion 70; and
`
`
`
`
`
`
`FIG. 12 is a schematic block diagram showing the struc-
`
`
`
`
`
`
`
`ture of another imagesignal transcoder 301 accordingto the
`
`
`
`
`
`fourth embodiment of the present invention.
`DESCRIPTION OF THE PREFERRED
`
`
`EMBODIMENTS
`
`
`
`
`
`
`
`Embodiments of the present invention are now described
`
`
`
`
`
`
`in detail with reference to the drawings.
`
`
`
`
`
`
`As for transcoding of image signals described with ref-
`
`
`
`
`
`
`
`erence to the embodiments of the present invention,
`the
`
`
`
`
`
`
`
`following three cases are assumed by wayofillustration:
`
`
`
`
`
`
`(1) transcoding between identical coding standards for
`
`
`
`
`
`decodinga bit stream coded under the MPEG2 standard
`
`
`
`
`
`
`
`and re-cncoding the bit stream undcr the MPEG2
`
`standard;
`
`
`
`
`
`
`
`(2) transcoding between different coding standards for
`
`
`
`
`
`
`
`
`decoding a bit stream coded under the DV standard and
`
`
`
`
`
`
`
`
`re-encoding the bit stream under the MPEG2 standard;
`and
`
`
`
`
`
`
`
`
`(3) transcoding between different coding standards for
`
`
`
`
`
`
`decodinga bit stream coded under the MPEG2 standard
`
`
`
`
`
`
`
`
`and re-encoding the bit stream under the DV standard.
`First Embodiment
`
`
`
`
`
`
`
`
`
`With reference to a first embodiment of the present
`
`
`
`
`
`
`invention, an image signal transcoder 100 corresponding to
`
`
`
`
`
`
`
`
`the case (1) among the aforementioned three cases is
`
`
`
`
`
`
`
`
`
`described. The case (2) and (3) will be described in a fourth
`embodimentlater.
`
`
`
`
`
`
`
`
`
`
`Referring to FIG. 1,
`the image signal transcoder 100 5
`
`
`
`
`
`
`
`according to the first embodiment of the present invention
`
`
`
`
`
`
`comprises an MPEG2decoding block 10 decoding an input
`bit stream encoded under the MPEG2standard and trans-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`forming the sameto a digital decoded image and an MPEG2
`
`
`
`
`
`
`
`
`coding block 30 re-encoding the digital decoded image
`
`
`
`
`
`
`
`
`
`under the MPEG2 standard and forming an output bit
`stream.
`
`
`
`
`
`
`
`
`
`The MPEG2 decoding block 10 and the MPEG2 coding
`
`
`
`
`
`
`
`
`block 30 include an MPEG2 decoder portion 11 and an
`
`
`
`
`
`
`
`MPEG2encoder portion 31 executing actual decoding, and
`
`
`
`actual re-encoding respectively.
`
`
`
`
`
`
`
`The image signal
`transcoder 100 further comprises a
`
`
`
`
`
`
`
`
`control portion 50 having a function of holding coding
`
`
`
`
`
`
`information obtained in decoding in the MPEG2 decoder
`
`
`
`
`
`
`
`portion 11 and fitting the coding information to calculation
`
`
`
`
`
`
`
`of a coding parameter
`in re-encoding by the MPEG2
`
`
`
`encoder portion 31.
`
`
`
`10
`
`
`
`
`1S
`
`
`
`
`
`
`
`
`
`30
`
`
`
`
`35
`
`40
`
`45
`
`60
`
`65
`
`
`
`US 6,792,045 B2
`
`
`4
`
`
`
`
`
`
`
`the MPEG2 decoder portion 11
`Referring to FIG. 2,
`
`
`
`
`
`
`
`
`includes a stream analysis portion 12 receiving the inputbit
`
`
`
`
`
`
`stream, a strcam buffer portion 14 for temporarily holding an
`
`
`
`
`
`
`
`
`output of the stream analysis portion 12, an MPEG2 decod-
`
`
`
`
`
`
`
`
`
`ing function portion 16 decoding the bit stream output from
`
`
`
`
`
`
`
`the stream buffer portion 14 by a constant quantity at a time
`
`
`
`
`
`
`
`
`
`and a frame buffer portion 18 for temporarily storing the
`
`
`
`
`
`
`
`decoded image obtained by the MPEG2 decoding function
`
`
`
`
`
`
`
`
`
`
`portion 16. The frame buffer portion 18 outputs the digital
`
`
`
`
`
`
`
`decoded image to the MPEG2encoderportion 31.
`
`
`
`
`
`
`
`The MPEG2decoderportion 11 further includes an upper
`
`
`
`
`
`
`layer information storage portion 20 for storing information
`
`
`
`
`
`
`
`
`of upper layers extracted in the stream analysis portion 12
`
`
`
`
`
`
`and an MBlayerinformation storage portion 22 for storing
`
`
`
`
`
`
`
`information of a macroblock layer level obtained by the
`
`
`
`
`
`
`MPEG?decoding function portion 16 in MPEG? decoding
`
`processing.
`
`
`
`
`
`
`
`
`The bit stream of an image signal according to the
`
`
`
`
`
`
`MPEG?standard has a hierarchical structure formed by the
`
`
`following layers:
`
`
`
`
`(including sequence level header
`sequence layer
`
`information)
`
`
`
`
`
`
`
`GOP(groupof picture) layer (including GOPlevel header
`
`information)
`
`
`
`
`
`
`picture layer (including picture level header information)
`
`
`
`
`
`
`
`slice layer (including slice level header information)
`
`
`
`
`
`macroblock (MB)
`layer (including macroblock level
`
`information)
`
`
`block layer
`
`
`
`sequence encode code
`
`
`
`
`
`
`
`A bit stream of arbitrary dynamic imagedata is formed by
`
`
`
`
`
`
`
`the aforementioned layers, each of which includes a portion
`
`
`
`
`
`
`
`
`
`referred to as a header portion. Each header portion has
`
`
`
`
`
`
`
`information related to the layer inserted as a code under a
`
`
`predetermined syntax.
`
`
`
`
`
`
`
`The stream analysis portion 12 extracts information nec-
`
`
`
`
`
`
`
`
`
`
`essary for transcoding including the bit rate and the picture
`
`
`
`
`
`
`
`
`
`
`size of the input bit stream, the picture type whichis picture
`
`
`
`
`
`
`
`level information indicating the type of the coding system of
`
`
`
`
`
`
`
`
`
`eachpicture and the GOP structure of each picture from the
`
`
`
`
`
`
`
`
`header portions of the sequence, GOP and picture layers
`
`
`
`
`
`
`
`
`(these layers are hereinafter also referred to as upper layers)
`
`
`
`
`
`
`
`
`of the bit stream having such a hicrarchical structure.
`
`
`
`
`
`
`
`Simultaneously with such extraction of the information
`
`
`
`
`
`
`
`
`
`related to the upper layers, the stream analysis portion 12
`
`
`
`
`
`
`
`
`
`
`counts and obtains the code quantity of each picture for each
`
`
`
`
`
`
`
`
`picture layer level. In the following description, symbol
`
`
`
`
`
`
`
`
`
`Gd(n) represents the code quantity of an n-th (n: natural
`
`
`
`
`
`
`
`number) picture in the input bit stream.
`
`
`
`
`
`
`
`
`The stream analysis portion 12 stores each obtained
`
`
`
`
`
`
`
`
`picture code quantity Gd(n) and the information related to
`
`
`
`
`
`
`
`
`
`the upper layers necessary for transcoding in the upper layer
`
`
`
`
`
`
`
`
`information storage portion 20. The information held in the
`
`
`
`
`
`
`
`upperlayer information storage portion 20 is suppliedto the
`
`
`
`
`
`
`
`
`control portion 50 and the MPEG2 decoding function por-
`tion 16.
`
`
`
`
`
`
`
`
`
`
`The stream buffer portion 14 temporarily buffers the bit
`
`
`
`
`
`
`
`
`stream output from the stream analysis portion 12 and
`
`
`
`
`
`
`outputs the same by a prescribed quantity, set in correspon-
`
`
`
`
`
`
`dence to the throughput of the MPEG2 decoding function
`
`
`
`portion 16, at a time.
`
`
`
`
`
`
`
`
`The MPEG decoding function portion 16 decodes the bit
`
`
`
`
`
`
`
`
`
`stream output from the stream buffer portion 14 by the
`
`
`
`
`
`
`
`prescribed quantity at a time with the information related to
`
`
`
`
`
`
`
`
`the upper layers extracted in the stream analysis portion 12
`
`
`
`
`
`and outputs the decoded image.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 15 of 21
`
`Page 15 of 21
`
`

`

`
`
`US 6,792,045 B2
`
`(4)
`
`
`
`Ge(n=f(k)'Gdn)
`
`
`
`
`
`In this expression, k is expressed as BR(d)/BR(e), where
`
`
`
`
`
`
`
`
`BR(d)represents the bit rate of the input bit stream, BR(e)
`
`
`
`
`
`
`
`
`
`represents the target bit rate in re-encoding, and f(k) repre-
`
`
`
`
`
`
`sents a prescribed function whose valueis decided bythe bit
`
`
`
`
`
`
`
`
`
`rate ratio k.
`‘The expression (1) indicates that the target
`
`
`
`
`
`
`
`
`coding quantity for each picture in re-encoding is set in
`
`
`
`
`
`
`response to the bit rate ratio k.
`
`
`
`
`
`
`
`
`
`In place of the above expression (1), the following com-
`
`
`
`
`
`
`
`
`putational expression (2) employing not the simple function
`
`
`
`
`
`
`
`
`f(k) of the bit rate ratio k but a function f(BR(e), BR(d)) by
`
`
`
`
`
`the bil rates is also applicable:
`
`
`
`
`
`
`
`
`
`Ge(n)=f(BR(e), BR(d))Gd{n)
`
`
`(2)
`
`
`
`5
`6
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`stream as the picture level information. Further, re-encoding
`The frame buffer portion 18 temporarily holds the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`is executed while holding the same relation between the
`decoded image output from the MPEG2 decoding function
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`image subjccted to re-coding and the picture types thercof as
`portion 16 and outputs the same to the MPEG? encoder
`
`
`
`
`
`
`
`
`
`
`
`
`betweenthe original decoded image and the decodedpicture
`portion 31 as the digital decoded image.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`types. In other words, pictures decoded with I, P and B
`The MPEG? decoding function portion 16 extracts mac-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`pictures in decoding in the MPEG2 decoding function
`roblock (MB)level information in the process of decoding.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`portion 16 are re-encoded with the I, P and B pictures
`The macroblock level information is represented by quan-
`
`
`
`
`
`
`
`
`
`
`
`respectively. Thus, deterioration of the picture quality result-
`tization characteristic values (MQUANT), a macroblock
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ing from transcoding can be suppressed when transforming
`type and information of motion vectors. The extracted
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the input bit stream according to the MPEG2standard to a
`macroblock level
`information is held in the MB layer
`
`
`
`
`
`
`
`lowerbit rate,
`
`
`
`information storage portion 22 and supplied to the control
`
`
`
`
`
`
`
`
`Bit rate control in re-encoding is now described.
`portion 50.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`‘The control portion 50 holds the coding information
`Referring to FIG. 4,
`the code quantity Gd(n) of each
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`related to the upper layers and the macroblock level infor-
`picture corresponding to an n-th (n: natural number)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`decoded image is counted by the stream analysis portion 12
`mation extracted through decoding of the input bit stream
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and held in the upper layer information portion 20.
`and uses the information for re-encoding.
`
`
`
`
`
`
`
`
`
`
`
`
`
`The MPEG2encoderportion 31 sets a coding quantity in
`Re-encoding by the MPEG encoderportion 31 based on
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`re-coding in correspondence to the counted code quantity
`the coding information supplied from the control portion 50
`
`
`
`
`
`
`
`
`
`
`is now described.
`
`
`
`Gdn) of each picture on the inpul bil stream thereby
`
`
`
`
`
`
`
`The first embodiment is described with reference to a
`
`
`
`
`
`
`
`executing assignmentof the code quantity. Whenthebit rate
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`of the input bit stream is 6 Mbps anda target bit rate in
`technique of minimizing image deterioration resulting from
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`re-encoding is 3 Mbps,
`for example,
`the target coding
`re-encoding when the bit rate of the input bit stream is
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`equivalent to or higher thanatarget bit rate in re-encoding.
`quantity for each picture is %. Assuming that Ge(n) repre-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`This technique is applicable for re-encoding an input bit
`sents the target coding quantity for each picture in re-coding,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`stream of 6 Mbpsat a bit rate of 3 Mbps, for example.
`the relation between the same and the counted code quantity
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 3 illustrates re-encoding by the MPEG2 encoder
`Gd(n) of the decoded image obtained from the input bit
`
`
`
`
`
`
`
`
`
`
`
`stream can be expressed as follows:
`portion 31 with reference to the picture types (picture coding
`
`types).
`
`
`
`
`
`
`
`
`
`In the MPEG? standard,
`three coding types for an I
`
`
`
`
`
`picture (intra-coded picture), a P picture (predictive-coded
`
`
`
`
`
`picture) and a B picture (bidirectionally predictive-coded
`
`
`
`
`
`
`
`
`
`picture) are present as the types of pictures. The picture type
`
`
`
`
`
`
`
`
`is information indicating with which one of the I, P and B
`
`
`
`
`pictures each picture is coded.
`
`
`
`
`
`
`
`The I picture, which is coded with closed information in
`
`
`
`
`
`
`
`
`
`
`only the single picture, can be decoded with only the
`
`
`
`
`information of the I picture itself.
`
`
`
`
`
`
`
`As to the P picture for which a temporally preceding
`
`
`
`
`
`
`
`decodedI or P picture is used as a predictive picture (picture
`
`
`
`
`
`
`employed as the reference for computing difference),
`whether to code the difference between the same and a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mation-compensated predictive picture or to code the I
`
`
`
`
`
`
`picture without computing the difference can be selected in
`
`
`
`
`
`macroblock units in response to the efficiency.
`
`
`
`
`
`
`
`Forthe B picture, three types of pictures, i.e., a temporally
`
`
`
`
`
`
`preceding decoded I or P picture, a temporally subsequent
`
`
`
`
`
`
`decodedI or P picture and an interpolation picture formed by
`
`
`
`
`
`
`
`
`
`these pictures are used. The mostefficient methodis selected
`
`
`
`
`
`
`
`
`from coding of difference of these three types of pictures
`
`
`
`
`
`
`
`after motion compensation and intra coding in macroblock 5
`units.
`
`
`
`
`
`
`
`
`
`Referring to FIG. 3, image signal data under the MPEG2
`
`
`
`
`
`
`
`standard has a GOP formed by oneora plurality of I pictures
`
`
`
`
`
`
`
`
`
`
`and zero or a plurality of non-I pictures. In the example
`
`
`
`
`
`
`
`shown in FIG.3, the GOPis formed by15 pictures (n=1 to
`
`
`
`
`
`
`
`15) and the cycle of the I or P pictures is 3.
`
`
`
`
`
`
`
`
`The MPEG? encoder portion 31 decides the coding
`
`
`
`
`
`
`parameter in re-encoding under the MPEG2standard on the
`
`
`
`
`
`
`
`
`basis of the picture types extracted by the stream ana