United States Patent [191
`Galbi
`
`[54] MPEG AUDIO/VIDEO DECODER
`
`[76] Inventor: David E. Galbi, 1063 Morse Ave.,
`#19-301, Sunnyvale, Calif. 94089
`
`[21] Appl. No.: 311,659
`[22] Filed:
`Sep. 23, 1994
`
`Related US. Application Data
`
`[63] Continuation-impart of Ser. No. 288,652, Aug. 10, 1994,
`abandoned, which is a continuation of Ser. No. 890,732,
`May 28, 1992, abandoned, which is a continuation-in-part of
`Ser. No. 669,818, Mar. 15, 1991, abandoned.
`
`[51] Int. Cl.6 ..................................................... .. G06K 9/36
`[52] US. Cl.
`382/233
`[58] Field of Search ................................... .. 382/232, 250;
`348/403, 404, 416; 358/432
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,302,775 11/1981 Widergren et al. ................... .. 382/232
`4,394,774
`7/1983 Widergnen et a1.
`382/250
`4,831,440
`5/1989 Borgers et al. . . . .
`. . . .. 348/403
`
`5,021,891
`
`6/1991 Lee . . . . . . . . . . . . . . . . . . . . .
`
`. . . .. 358/432
`
`5,148,292
`
`9/1992 Acampura et a1. . . . .
`
`. . . .. 358/133
`
`. . . .. 358/133
`5,196,930
`3/1993 Kadono et a]. . . . . . . . .
`348/416
`5,414,469 5/1995 Gonzales etal ..
`5,452,104
`9/1995 Lee ........................................ .. 348/404
`
`FOREIGN PATENT DOCUMENTS
`
`O 598 295 5/1994 European Pat. O?'. .
`
`USOO5649029A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,649,029
`Jul. 15, 1997
`
`Primary Examiner-Jose L. Couso
`Attorney, Agent, or Firm-$kjerven, Morrill, MacPherson,
`Franklin & Friel; David T. Millers
`
`[57]
`
`ABSTRACT
`
`An MPEG audio/video decoder has memories, a signal
`processing unit (SPU) including a multiplier and a butter?y
`unit, a main CPU, and a memory controller which are time
`division multiplexed between decoding video and audio
`data. For audio decoding, the butter?y unit determines
`combinations of components of a frequency-domain vector
`to reduce the number of multiplies required to transform to
`the time domain (matrixing). Matrixing is interwoven with
`MPEG ?ltering to increases throughput of the decoder by
`increasing parallel use of the multiplier, the butter?y unit,
`and a memory controller. The decoder includes a degrouping
`circuit which performs two divisions in three clock cycles to
`degroup a subband code. Three cycles matches the write
`time of three components so that subband codes are
`degrouped and written to memory with a minimum delay.
`Performing two divides in three clock cycles allows the
`divider to be smaller. In response to an error signal from a
`source of an MPEG audio data stream, the decoder replaces
`data with an error code and temporarily enables error
`handling. The error code is a valid bit combination rarely
`found in MPEG audio data frames. During audio decoding
`with error handling enabled, the decoder checks for the error
`code and replaces the error code with reconstructed data.
`Typically, some subband data are replaced with zeros so that
`an error only changes some of the frequency components.
`
`5 Claims, 13 Drawing Sheets
`
`10o \
`115 \
`k
`118 \
`125 1
`Code FIFO 4
`Decoder FIFO 4 ‘L6
`Audio Error
`(64x16)
`‘
`Code Injector ‘“ (31x18)
`‘
`
`no \
`
`‘4°51
`Interface
`
`104
`,1
`Sm‘ Bus
`Host Bus
`
`VLC/FLC : r
`
`f
`
`Decoder
`
`‘ _ Memmy ‘ _
`7 ' Controller
`7 ' DRAM
`
`171 \
`?_ Y, CbCrFIFOs
`(56x16, 64x16)
`
`150 \
`CPU
`
`_ :
`
`TMEM
`(64 X 22)
`136 j
`
`v K134
`ZMEM
`(192x16)
`1
`135 \
`v K140
`QMEM
`172 \
`(64x16) < 7 Signal _
`F’rqcessmg ‘ = g Overlay FIFO
`7 Un"
`(64x16)
`4]
`137 \ v
`PMEM
`(128 x 18)
`11
`170 \ ‘y
`
`\rso
`
`173 \
`i Vert, Horz
`' Interpolation
`
`174 \ ]
`Overlay
`Blending
`
`175 \ 4
`Conversion
`to RGB
`
`176
`Video
`Output
`
`191 \
`
`192
`
`190 \
`
`4 g ; Audio FIFO
`Motion _
`Compensatron
`(4 x 16)
`
`:
`
`Audio
`Sen'alizer
`
`Audio
`Output
`
`Apple Inc. v. Parthenon
`Ex. 1015 / Page 1 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 1
`
`

`
`U.S. Patent
`
`Jul. 15, 1997
`
`Sheet 1 of 13
`
`5,649,029
`
`mam_m_$m
`
`mammo:
`
`E:
`
`N9
`
`mo:
`
`8mt2:_
`
`o:
`
`m:
`
`Samo_u:<
`
`
`
`§8.E_250
`
`8;xE8:5889
`
`3
`
`8.
`
`o9
`
`at
`
`E;
`
`
`
`N5:.:m>
`
`:o_§oe9c_
`
`mou__.._58.>
`
`32%.263
`
`saso
`
`82>
`
`mt
`
`.350
`
`o__8<
`
`N2
`
`mom2:o_m.m>:oo
`
`2mm
`
`8.
`
`8_
`
`3.xs9.:%_2
`
`>m_§o
`
`a:_u:.a._m
`
`EmEgo
`§xV9
`
`o:§>
`
`auooma
`
`§Xmm5smzw
`
`9__$8o¢
`
`_§_m
`
`E:
`
`_>ms_n_
`
`§xmm:
`
`:o_§2
`
`cozmmcmaeoo
`
`E
`
`me
`
`3.Xismzo
`
`55
`
`amXwe
`
`3
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 2
`
`Ex. 1015/ Page 2 of 39
`
`Ex. 1015 / Page 2 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 2
`
`
`
`
`
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Jul. 15, 1997
`
`Sheet 2 of 13
`
`5,649,029
`
`/ 200
`
`I
`
`1
`
`.
`
`C'L
`
`CiH
`
`
`‘A ? CIM ' 1\ /a
`
`221
`
`220
`
`4mm
`
`Rin
`
`Rout
`
`illbits
`
`{210
`
`@230
`
`231
`\~
`
`4bits
`
`Din
`
`Q
`
`{debits
`
`R
`
`QL
`
`3,4
`
`5
`
`240
`
`TO ZMEM
`
`232
`\_
`
`'
`
`QH
`
`FIG. 2
`
`Ex. 1015 / Page 3 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 3
`
`

`
`US. Patent
`
`Jul. 15, 1997
`
`Sheet 3 of 13
`
`5,649,029
`
`s11
`321
`300 \
`\jvh r320
`Ti‘ r310
`dividend
`dividend
`312
`1Z¢->Rin
`Rout —2"—H Rin
`Rout
`
`2
`
`324
`
`Q
`
`i1 313
`FIG. 3A
`
`Q
`
`{7L 323
`
`
`
`.m D...
`
`A
`
`B
`
`ividend
`H C
`
`000011111 Wd
`00110011
`01010101
`
`hm ROOTOOTXX
`
`.m S_O._IOO4IOXX
`\iivvlj
`
`_A B
`
`C 520
`CC
`AA
`
`QOOO111XX
`
`FIG. 3B
`
`8 TRANSISTORS
`
`1O TRANSISTORS
`
`12 TRANSISTORS
`__
`s OH Q
`
`FIG. 3C
`
`Ex. 1015 / Page 4 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 4
`
`

`
`U.S. Patent
`
`Jul. 15, 1997
`
`Sheet 4 of 13
`
`5,649,029
`
`400 f
`411
`421
`431
`\i‘ /410
`\i‘ [420
`i1 r430
`dividend
`dividend
`dividend
`412
`1%» Rin
`Rout —@'——> Rin
`Rout —@~> Rin
`Rout
`
`434
`
`1L 413
`
`1L 423
`FIG. 4A
`
`Q 433
`
`u q
`
`m
`
`m @0000011111
`
`mnm 80000111100
`
`
`0001100111100
`
`A0000000011
`
`~Wm0101010101.
`
`w_l_o4lo1nuo1o4lo
`
`0011000110
`
`0000100001
`
`FIG. 4B
`
`Ex. 1015 / Page 5 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 5
`
`

`
`US. Patent
`
`Jul. 15, 1997
`
`Sheet 5 0f 13
`
`5,649,029
`
`D
`
`B ___.___...._
`
`DID
`‘’ 3D
`A
`‘
`
`12 TRANSISTORS
`
`DEKJ
`T
`Q
`
`D”
`10 TRANSISTORS
`
`g ...__..T
`_Q—-—-
`D ------~
`A ——‘
`
`12 TRANSlSTORS
`D‘ R
`
`A
`9
`D
`o
`D
`E
`9.
`0
`
`24 TRANSISTORS
`
`E
`S
`
`FIG. 4C
`
`Ex. 1015 / Page 6 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 6
`
`

`
`U.S. Patent
`
`Jul. 15, 1997
`
`Sheet 6 of 13
`
`5,649,029
`
`220
`
`221
`
`l
`
`->
`
`‘
`
`[21_°1_____‘ -_----- "zé'g'o‘
`_
`R{3:0]
`M3111
`l
`1 + Address Generator
`:
`i W
`A[7:0]
`l
`i
`{8
`[550 i
`g
`200 X 8 ROM
`!
`L
`Rout
`Quotient J
`y
`i
`
`/4 H _ — 1'4 — —
`
`230/
`———————¢
`
`231/ £___
`
`232 J
`
`240
`
`FIG. 5A
`
`Ex. 1015 / Page 7 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 7
`
`

`
`U.S. Patent
`
`Jul. 15, 1997
`
`Sheet 7 of 13
`
`5,649,029
`
`2
`
`2
`
`%
`
`2
`
`
`
`
`
`S
`
`:2
`
`CO
`.2.
`‘‘ E
`'
`
`0 =
`
`2
`
`F5
`
`8
`
`8
`
`an
`LO
`
`0'
`
`5
`
`553
`
`E I
`R2R1 V
`Y A6A5
`DIV3DIV3
`
`
`Q °
`
`‘
`
`550\
`
`-.">3“2’
`an
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 8
`
`Ex. 1015/ Page 8 of 39
`
`Ex. 1015 / Page 8 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 8
`
`

`
`U.S. Patent
`
`Jul. 15, 1997
`
`Sheet 8 of 13
`
`5,649,029
`
`/610
`V0 0
`o oo
`
`V0 V0 31
`000 I
`V
`—__
`We
`
`V4
`
`V-1 32
`0 0 o
`V4
`
`__
`V-2 0
`
`V_2 V-2 31
`0 0 0
`V-2
`
`
`V-3
`
`V'3 0
`
`V-3 32
`0 0 0
`V-3
`
`______
`
`
`
`/620
`V0 33
`o00
`
`V0 31
`. . .
`V0 O
`
`
`.2
`
`V-1 63
`
`V-1 32
`0 0 a
`V4 O
`
`
`V-2 63
`
`V-2 31
`0 0 0
`V'2 0
`
`
`V-3 63
`
`V-3 32
`0 0 0
`V''3 0
`
`
`/630
`o
`V..4;8
`
`V0 33
`V0
`0 0 0
`V0 17
`V-1 48
`
`V-1 33
`V-1
`o 0 0
`
`V-1 17
`V-2 48
`
`V-2 33
`V-2
`0 0 0
`
`V'2 17
`V-3 48
`
`V'3 33
`V-3
`0 0 0
`
`M-3
`
`631
`
`632
`
`V-4 to \/-14
`
`I
`
`I
`
`Z
`
`
`
`V-15 0
`' ' '
`
`0 0 0
`V4564
`
`
`
`
`
`
`V-15 33
`0 I O
`
`628 V-15 48
`‘J
`0 0 0
`
`638
`
`o o o
`V450
`
`
`V-15 32
`000
`
`M45
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 9
`
`Ex. 1015/ Page 9 of 39
`
`Ex. 1015 / Page 9 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 9
`
`

`
`U.S. Patent
`
`Jul. 15,1997
`
`Sheet 9 of 13
`
`5,649,029
`
`mm.&9
`
`20:".
`
`=:E:~_IIII.u_I_ENE%o3+o_$88;
`mt.=:EI>IanFIII.NQH32205NE_.u88oEtr.m.9.28%+,m,_x_§n_+a_m:2m2%
`<mmL.205___¢mI=_:EI>QVOANm_uu.o.nEn_nE
`.205opt.
`
`
`
`Em&<+.m.._x_§n_+a§_8I._mmA4_8.§eaNo_peaE,....&<+.max_§n:S£2m_omxmfi_4X.55xm.o__m
`
`axma_sé:o%asIE4mtn!58%;
`
`2..w_....EOE_¢3_=exV4Nsizmzs./oio—.E.2".
`
`
`
`
`
`afiwommSaromm
`
`
`
`_$.z.x_o§E<E
`
`oOE
`_3m__2m_E%I ofl
`
`
`
`siflmoa;m_sm_EgNEI8_§_2m2%VLyn:soo__2§m:mo
`
`
`
`om:>._n_mmE:m
`
`_oz_%_Sox
`
`
`
`«EV95§._-Em
`
`.3“?s_&:.5o
`
`RNsfizzfifi
`
`.Iul.IIl|Il.lal||‘.I..I.lIIII..l.IIIN|.lI-l.lll.lIl|I-II
`
`
`
`
`
`ms.lmu“I_oEIHEEEI_>5wC.8“&_m§._:.m
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 10
`
`Ex. 1015/Page1O of39
`
`Ex. 1015 / Page 10 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 10
`
`
`

`
`U.S. Patent
`
`Jul. 15, 1997
`
`Sheet 10 of 13
`
`5,649,029
`
`an
`
`_o”,m_moM
`
`fiifima
`
`agmfma
`
`I..In|:.I.nI.l‘IllIIl|u|||Il..II.IlIIIIIIlI|I|I'.lIlI|III.|..
`
`ea9,.._s_<._o
`
`
`
`“.250oa=<
`
`Enasememrw2%_<._ou=_fi
`
`2:250Eémmmq
`
`zqEm_ua2ms:
`
`8w__2w_2n_
`
`
`
`N<n.O_u_
`
`OH>5.
`
`
`
`<.\..0_u_
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 11
`
`Ex. 1015/Page 11 of39
`
`Ex. 1015 / Page 11 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 11
`
`
`
`
`
`
`

`
`US. Patent
`
`Jul. 15, 1997
`
`Sheet 11 of 13
`
`5,649,029
`
`520335
`
`E a... veg :t
`
`ON.
`0E
`
`EmEOU
`
`NE. x22 2.
`
`v vv a 2:623
`22 2 82 w
`
`
`
`r528": 6258250
`
`\l lhol
`
`m8. agwiomi
`@250
`
`66 3506mm
`
`358%565
`522E250
`
`:mEwmE
`
`7 m0 mwazsom
`/ 5
`
`ooh x22 2.
`
`mi. .GE
`
`@Qxg
`
`smzo
`mm;
`
`52520
`
`536 A 803250 r 5.
`
`5%
`3 $6
`
`Ex. 1015 / Page 12 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 12
`
`

`
`US. Patent
`
`Jul. 15, 1997
`
`Sheet 12 0f 13
`
`5,649,029
`
`[805
`HHANSFEH BIT ALLOCATION AND SCALEFACTORS TO QMEM]
`l
`[810
`ITRANSFER 33 COMPONENTS TO PMEM]
`820
`
`[825
`IwAIT uNTIL VLC/FLC IS IDLE]
`835
`
`840
`
`_
`
`IWAIT uNTIL TRANSFER OF SOUND AMPLITUDE VALUES IS coMPLETEI
`I
`[855
`+[wAIT uNTIL TRANSFER OF 33 COMPONENTS IS ooMPLETEl
`860
`
`[865
`| IF NOTTHE 8TH rrERATToN FoR THE LAST vEcToR, ISSUE TRANSFER 0m COMPONENTS j
`870
`mm
`I875
`[ISSUE TRANSFER OF 4 MATRIX RESULTS]
`880
`
`[885
`[TRANSFER 32 DECODER SOUND AMPLITUDE VALUES 1
`890
`‘ mm
`
`895
`I mm
`
`FIG. 8A
`
`Ex. 1015 / Page 13 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 13
`
`

`
`US. Patent
`
`Jul. 15, 1997
`
`Sheet 13 of 13
`
`5,649,029
`
`SPU
`
`I I I l l I IJ
`
`FIG. 8B
`
`Ex. 1015 / Page 14 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 14
`
`

`
`1
`MPEG AUDIO/VIDEO DECODER
`
`CROSSREFFERENCE TO RELATED
`APPLICATION
`This is a continuation-in-part of US. patent application
`Ser. No. 08/288,652 entitled “A Variable Length Code
`Decoder for Video Decompression Operations,” ?led Aug.
`10, 1994 now abandoned, which is a continuation of US.
`patent application Ser. No. 07/890,732, ?led May 28. 1992
`(now abandoned) which was a continuation-in-part of US.
`patent application Ser. No. 07/669,8l8, entitled “Decom
`pression Processor for Video Applications,” ?led Mar. 15,
`1991 (now abandoned), all of which are incorporated by
`reference in their entirety.
`
`10
`
`COPYRIGHT NOTICE
`A portion of the disclosure of this patent document
`contains material which is subject to copyright protection.
`The copyright owner has no objection to the facsimile
`reproduction by anyone of the patent document or the patent
`disclosure. as it appears in the Patent and Trademark O?ice
`patent ?les or records, but otherwise reserves all copyright
`rights whatsoever.
`
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`This invention relates to decoders for generating audio
`signals from digital data, and in particular to combined audio
`and video decoding according to the MPEG standard.
`
`25
`
`DESCRIPTION OF RELATED ARI‘
`The Motion Picture Experts Group (MPEG) developed an
`international standard (sometimes referred to herein as the
`“MPEG standard”) for representation, compression, and
`decompression of motion pictures and associated audio on
`digital media. The International Standards Organization
`(ISO) publication, No. ISO/IEC 1117211993 (E), entitled
`“Coding for Moving Pictures and Associated Audio-for
`digital storage media at up to about 1.5 Mbit/s,” describes
`the MPEG standard and is incorporated by reference herein
`in its entirety. The MPEG standard speci?es coded digital
`representations of audio and video and is intended for
`continuous data transfer from equipment such as compact
`disks, digital audio tapes, or magnetic hard disks, at rates up
`to 1.5 Mbits per second.
`Under the MPEG standard, parallel data streams or time
`multiplexed data streams provide video data frames and
`audio data frames. Methods and systems for decompressing
`video data frames are described in U.S. patent applications
`Ser. Nos. 07/890,732 and 07/669,818 which were incorpo
`rated by reference above. Audio data frames contain a
`header, side information, and subband data. Subband data
`indicate ?equency-domain vectors that are converted to
`time-domain output sound amplitudes by a transformation
`(matrixing) and a smoothing ?lter (windowing).
`Typically, MPEG audio/video decoding systems for
`decoding digital data include, two decoders, one for audio
`decoding and one for video decoding. on two separate
`integrated circuit chips. The audio decoder and video
`decoder are separated because of the diiferences between
`MPEG audio coding techniques and MPEG video coding
`techniques, but separate audio and video decoders increase
`the amount of circuitry in and the cost of an audio/video
`decoding system. A decoding architecture is needed that
`reduces the amount of circuitry and the cost of decoding
`MPEG audio and video data.
`
`35
`
`45
`
`50
`
`55
`
`65
`
`5,649,029
`
`2
`SUMMARY OF THE INVENTION
`In accordance with this invention, an MPEG audio/video
`decoder integrated on a single chip uses components such as
`memories, a main CPU, a memory controller, and a signal
`processing unit (SPU) for both audio and video decoding.
`The SPU contains a multiplier (or multiply-and-accumulate
`unit) and a butter?y unit which together alternately decode
`video data and audio data. The combination of a multiplier
`and a butter?y unit is efficient for both audio and video
`decoding. In particular, for audio decoding, determining
`particular sums and differences of the components of a
`frequency-domain vector with a butter?y unit reduces the
`number of multiplies required for matrixing (i.e. determin
`ing a component of a time-domain vector from a frequency
`domain sample vector). Determining combinations of the
`components can be performed in series with dequantizing
`and descaling of the components combined Additionally,
`matrixing and windowing (i.e. combining a present time
`domain vectors with previous time-domain vectors) are
`combined in a single instruction to increase throughput of a
`decoder by increasing parallel use of the multiplier, the
`butter?y unit, and a memory controller which reads and
`writes to an external memory.
`Also in accordance with this invention, a degrouping
`circuit for decoding MPEG standard subband codes includes
`a divider which uses three clock cycles to perform two
`divisions which convert a MPEG subband code into three
`vector components. Performing two divides in three clock
`cycles instead of two allows the divider to be smaller and
`less costly, but does not slow decoding because three clock
`cycles is the time required to write three vector components
`into a single-port memory. Accordingly, the smaller divider
`does not signi?cantly increase the time required to degroup
`subband codes and write the resulting components into
`memory. Using the known limits on input dividends of the
`divider, the size and cost of the divider can be further
`reduced from that of a general purpose divider.
`Also in accordance with this invention, in response to an
`error signal from an external source of an MPEG audio data
`stream, an MPEG audio decoder replaces errors in the audio
`data stream with an error code which is a bit combination
`rarely found in MPEG audio data frames, and then tempo
`rarily enables error handling. The audio data stream con
`taining error codes can be saved or buttered in the decoder.
`During audio decoding with error handling enabled, the
`decoder checks the audio data for the bit combination
`equaling the error code and replaces the bit combination
`with reconstructed data. The replacement attempts to mini
`mize the audible e?ects of an error. Typically, some subband
`data is replaced with zeros so that an error causes some of
`the frequency components to be lost.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 shows a block diagram of an MPEG audio/video
`decoder in accordance with an embodiment of this inven
`tion.
`FIG. 2 shows a block diagram of a degrouping circuit in
`accordance with an embodiment of this invention.
`FIGS. 3A, 3B, and 3C show a block diagram. a logic
`table, and a gate level diagram of a divide-by-three circuit in
`accordance with this invention.
`FIGS. 4A, 4B, and 4C show a block diagram, a logic
`table, and a gate level diagram of a divide-by-?ve circuit in
`accordance with this invention.
`FIGS. 5A and 5B show a block diagram of another
`embodiment of the degrouping circuit and a gate level
`
`Ex. 1015 / Page 15 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 15
`
`

`
`5,649,029
`
`3
`diagram of an address generator for dividing by three, ?ve,
`or nine in accordance with this invention.
`FIG. 6 shows memory maps of previous vector compo
`nents used during a windowing process in
`FIGS. 7A, 7B, and 7C show a block diagram of an
`embodiment of a signal processing unit in accordance with
`an embodiment of this invention.
`FIG. 8A shows a ?ow diagram of an audio decoding
`process in accordance with this invention.
`FIG. 8B shows a timing diagram for the process of FIG.
`8A.
`Use of the same reference symbols in di?‘erent ?gures
`indicates similar or identical elements.
`
`10
`
`20
`
`4
`frame is a digital description of a ?xed number of ?equency
`domain sound samples in up to two sound channels. The
`MPEG standard for video data ?ames and decoding of video
`data ?ames to produce a video signal are described in US.
`patent applications Ser. Nos. 07/890,732 and 07/669,818
`which were incorporated by reference above. The MPEG
`standard currently de?nes three types of audio data frames
`referred to as layer 1, layer 2, and layer 3 data ?ames.
`Decoder 100 in FIG. 1 decodes layer 1 and layer 2 audio data
`frames. Layer 1 and layer 2 audio data frames contain a
`header, side information, and subband data. The header
`indicates: the bitrate of the data stream providing the audio
`data frames; the sample frequency of the decoded sound;
`whether the subband data contains one or two sound chan
`nels; and a mode extension describing whether the sound
`channels in the subband data are independent, stereo, or
`intensity stereo. The side information indicates the number
`of bits allocated per subband in the subband data and an
`index to scalefactors F for dequantizing and descaling
`subband data as described below.
`CPU 150 controls the percentage of time SPU 140 spends
`decoding audio data. For audio decoding, CPU 150 directs
`memory controller 180 to move audio data from DRAM 160
`to decoder FIFO 125 and directs SPU 140 perform the
`calculations necessary for decoding audio data. SPU 140
`operates in parallel with CPU 150 and executes commands
`according to software stored in an instruction memory in
`SPU 140.
`When decoding an audio data frame, SPU 140 ?rst
`executes a “get bits” command which loads the header and
`side information of the audio data frame, ?om decoder FlFO
`buffer 125, through a VLC/FLC decoder 120, into CPU 150.
`The CPU 150 writes bit allocations and scalefactors ?om the
`side information through SPU 140, into QMEM 135. Header
`and side information pass through VLC/FLC decoder 120
`unchanged. Subband data follows the side information in the
`data stream from decoder FIFO bu?er 125. VLC/FLC
`decoder 120 contains circuits for decoding variable length
`codes (VLC) in video data and ?xed length codes (FLC) in
`audio and video data. VLC/FLC decoder 120 also contains
`degrouping circuits for audio data as described below.
`A “get subbands” command executed by SPU 140 causes
`VLC/FLC decoder 120 to parse and convert subband codes
`Ci ?om decoder FIFO buffer 125 into 192 scaled and
`quantized components Si". VLC/FLC decoder 120 preforms
`degrouping as required and writes the scaled and quantized
`components Si" into ZMEM 134. Each frequency-domain
`vector S” has 32 components Si" in 32 ?equency ranges
`(subbands i). The “get subbands” command writes compo
`nents Si" for three frequency-domain vector S” in each
`channel (six vectors S" total for two channels) to ZMEM
`134. For intensity stereo, some of the ?equency components
`Si" are used by both channels. VLC/FLC decoder 120 writes
`two copies of components that are shared by the channels so
`that each vector S" in ZMEM 134 has 32 components Si".
`For monophonic sound, VLC/FLC decoder 120 can write
`two copies of all components Si" so that both channels of a
`stereo output signal are the same. The number of vectors S"
`in an audio data ?ame depends on the number of channels
`and whether the audio data frame follows layer 1 or layer 2
`of the MPEG standard. Under layer 1, there are 12 vectors
`S" (384 samples) per channel. Under layer 2, there are 36
`vectors S" (1152 samples) per channel.
`SPU 140 executes a “dequant/descale” command to gen
`erate components Si of frequency-domain vectors S by
`descaling and dequantizing values Si" ?om ZMEM 134.
`SPU 140 writes a representation of a vector S to a portion of
`
`25
`
`30
`
`35
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`In accordance with this invention, an audio/video decoder
`decodes MPEG standard data streams to provide an audio
`signal and a video signal. The audio/video decoder alternates
`between decoding video data frames and decoding audio
`data frames and employs the same memories and signal
`processing unit (SPU) for both audio and video decoding.
`FIG. 1 shows a block diagram of an audio/video decoder
`100 for decoding MPEG standard audio and video data
`frames. Decoder 100 receives MPEG standard coded audio
`and video data via a serial bus 104, decodes the audio and
`video data, and provides the decoded data over a video bus
`176 and an audio bus 192. Decoder 100 includes static
`random access memories (SRAMs) 134 to 137 (also referred
`to herein as ZMEM 134, QMEM 135, TMEM 136, and
`PMEM 137) which alternate between holding video data for
`video decoding and holding audio data for audio decoding,
`and a signal processing unit (SPU) 140 which includes an
`instruction memory, a register ?le, a multiplier or a multiply
`and-aocumulate unit (MAC), and a butter?y unit for decod
`ing and decompressing video data or audio data depending
`on whether decoder 100 is currently decoding video or
`audio.
`Audio/video decoder 100 interfaces with a source of
`audio and video signals such as a host computer and a
`compact disk digital signal processor (CD-DSP) over a host
`bus 102 and serial bus 104. Serial bus 104 carries a stream
`of compressed audio and video data following the MPEG
`standard, which decoder 100 receives through a ?rst-in
`45
`?rst-out (FlFO) buffer 115 (“code FIFO 115”). A memory
`controller 180 reads the compressed data ?om code FlFO
`115 via a main bus 155 and Writes the compressed data to an
`external memory 160 (also referred herein as DRAM 160).
`As disclosed below, an audio error code injector 118 can
`50
`inject error codes into audio data written to DRAM 160. A
`central processing unit (CPU) 150, which is a microcoded
`processor having its own instruction memory controls access
`to main bus 155 and in particular, sends commands to
`memory controller 180 which cause the data transfer ?om
`55
`code FIFO 115 to DRAM 160.
`In this embodiment, DRAM 160 contains dynamic ran
`dom access memory (DRAM) components. Other suitable
`memory technologies can also be used. DRAM 160 holds
`compressed data from serial bus 104 and decompressed data
`for output to an audio bus 192 or a video bus 176. Under the
`direction of CPU 150, memory controller 180 transfers
`compressed audio or video data to a decoder FIFO 125 for
`decoding of an audio data frame or a video data ?ame by
`SPU 140.
`According to the MPEG standard, a video data frame is a
`compressed digital description of a picture and an audio data
`
`65
`
`Ex. 1015 / Page 16 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 16
`
`

`
`5
`TMEM 136. Mauixing as described below transforms a
`frequency-domain vector S to a time-domain vector V. SPU
`140 stores components Vi of a time-domain vector V in
`PMEM 137, and memory controller 180 writes components
`V1 from PMEM 137 to DRAM 160. Components from 16
`consecutive time-domain vectors V0 to V16 from DRAM
`160 are combined in a windowing process described below,
`and the combination is accumulated in TMEM 136 to
`provide 32 time-domain output sound amplitudes Ai. Time
`domain output sound amplitudes Ai are typically written to
`an audio output FIFO buffer in DRAM 160, and written as
`required from DRAM 160 through main bus 155, an output
`audio FIFO 190, and an audio serializer 191 to audio output
`bus 192. Output audio FIFO buffer 190 holds enough output
`sound amplitude values so that at the fastest sampling rate
`expected delayed access to main bus 155 does not interrupt
`sound. Audio serializer 191 converts the output audio data to
`a serial data stream, and a digital-to-analog converter (DAC)
`and ampli?er (not shown) generate a sound from the audio
`data.
`The side information indicates the number of possible
`values for each quantized component Si" (and each subband
`code Ci) in a subband i. For example, if subband codes Ci
`in subband i have 0, 2, 4, . . . , or 2” possible values, then
`0, 1, 2, . . . , or N bits are used for each code Ci. If no bits
`are used for a subband i, VLC/FLC decoder 120 writes zero
`into ZMEM 134 for components Si", and vector S has less
`than 32 non-zero components. For a bit allocation repre
`senting 2" possible values for a subband i, VLC/FLC
`decoder 120 uses the bit allocations from the side informa
`tion in QMEM 135 to identify the start and end of a
`component Si" in the data stream and writes component Si"
`to a word aligned location in ZMEM 134.
`The MPEG standard allows components Si" to have 3, 5,
`or 9 possible values and encodes three components Sli",
`S2i", and S3i" from subbandi of three dilferent vectors S1,
`S2, and S3 into a single code Ci. For example, there are 27
`possible combinations of three quantized and scaled com
`ponents Sli", S2i", and S3i", if each has three possible
`values 0, 1, or 2. A 5-bit subband code Ci given by eq. 1
`represents the 27 possible combinations.
`
`25
`
`Similarly, a 7-bit code Ci given by eq. 2 represents three
`components Sli", S2i", and S3i" having ?ve possible values
`0 to 4 each.
`
`45
`
`Eq. 3 gives a 10-bits code Ci representing three components
`Sli“, S2i", and S3i" which each have 9 possible values, 0 to
`8.
`
`VLC/FLC decoder 120 degroups a code Ci into three
`components S3i", S2i", and Sli" given by eqs. 1 to 3 before
`writing the scaled and quantized components S3i", S2i" and
`Sli" to ZMEM 134. Two divisions are su?icient to degroup
`a code Ci given by eqs. 1 to 3. For example, if Ci=x2-S3i"+
`x-S2i"+S1i” and components S3i". S2i" and Sli" are less
`than x, dividing Ci by x provides a quotient Q1 and a
`remainder R1 given by eq. 4.
`
`Dividing by x again provides a quotient Q2 and a remainder
`R2 given by eq. 5.
`
`55
`
`65
`
`5,649,029
`
`6
`(eq. 5)
`(Q1/x)=Q2=S3i" with remainder m=s2i"
`IfZMEM 134 has a single port connected to VLC/FLC 120,
`three clock cycles are required to write components S31",
`S2i", and Sli". Accordingly, VLC/FLC decoder 120 can use
`three clock cycles for two divides which degroup a code Ci,
`and not cause a pipeline delay in writing components S3i",
`S2i", and Sli".
`FIG. 2 shows decoding circuit 200 which performs two
`divides for degrouping a code Ci in three clock cycles. The
`?rst divide is an extended divide that takes two clock cycles.
`The second divide takes one clock cycle. Using two clock
`cycles for the ?rst divide permits use of a smaller divider and
`reduces the cost of VLC/FLC decoder 120. In the embodi
`ment of FIG. 2, a divider 210 receives dividend values from
`multiplexers 220 and 221 and divides the dividend values by
`a divisor x equal to 3, 5, or 9 to produce a quotient Q and
`a remainder Rout. Side information gives the bit allocation
`for each subband and determines the value of divisor x for
`each subband which requires degrouping.
`Code Ci is partitioned into three parts CiH, CiM, and CiL
`for the ?rst divide of degrouping. CiL contains the 2, 3, or
`4 least signi?cant bits of code Ci for divisor x equal to 3, 5,
`or 9 respectively. CiM contains the next 2, 3, or 4 more
`signi?cant bits of code Ci, and CiH contains the most
`signi?cant 1, 1, or 2 bits of Ci for divisor x equal to 3, 5, or
`9 respectively. CiH is padded on the left with zeros to 2, 3,
`or 4 bits;
`Degrouping proceeds as follows. During a ?rst clock
`cycle, multiplexers 220 and 221 assert signals CiH and CM
`to divider 210, and divider 210 produces a quotient Q1H and
`a remainder R1‘ which are written to registers 231 and 230
`at the end of the ?rst clock cycle. Registers 230 and 231 in
`the embodiment of FIG. 2 are edge triggered devices, but in
`alternative embodiments, registers 230 and 231 may be
`latches, memory locations, or any devices capable of hold
`ing and asserting digital data signals. During a second clock
`cycle, multiplexers 220 and 221 assert respectively remain
`der R1‘ from register 230 and signal CiL to divider 210, and
`divider 210 produces a quotient Q1L and remainder R1. At
`the end of the second clock cycle, quotient Q1L and remain
`der R1 are written to registers 231 and 230 respectively, and
`quotient Q1H is written from register 231 to a register 232.
`Quotients Q1H and Q1L are respectively the most signi?
`cant and least signi?cant bits of the quotient Q1 given in eq.
`4. Remainder R1 is value Sli" as in eq. 1, 2, or 3.
`During a third clock cycle, multiplexers 220 and 221
`assert respectively signals Q1H and Q1L from registers 230
`and 231 to divider 210, divider 210 produces quotient Q2
`and remainder R2 that are given in eq. 5, and a multiplexer
`240 selects value R1 from register 230 for writing to a
`memory such as ZMEM 134 of FIG. 1. At the end of the
`third clock cycle, quotient Q2 remainder and R2 are written
`to registers 231 and 230, and the quotient QlL is written
`from register 231 to register 232.
`During a fourth clock cycle, remainder R2 which equals
`S2i" passes through multiplexer 240 and is written to the
`memory. Quotient Q2 is written to register 232 at the end of
`the fourth clock cycle. Quotient Q2 which equals S3i" is
`written to memory during the ?fth clock cycle. A ?rst divide
`for a second code Ci’ can be performed during the fourth and
`?fth clock cycles and can proceed as disclosed above.
`Accordingly, if a series of codes C are degrouped, degroup
`ing proceeds with a pipeline delay only for the ?rst code in
`the series.
`Any known or yet to be developed digital divider circuit
`may be employed for divider 210 providing the divider
`circuit handles the correct size dividend, quotient. and
`
`Ex. 1015 / Page 17 of 39
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1015, p. 17
`
`

`
`7
`remainder. FIG. 3A shows a block diagram of a divide-by
`three circuit 300 which uses the limits on the values of codes
`C to reduce the number of gates and transistors required.
`Divide-by-three circuit 300 contains two identical circuit
`blocks 310 and 320. Each block 310 and 320 has input
`terminals for a 1-bit dividend signal C and a 2-bit remainder
`signal Rin and output terminals for a 1-bit quotient signal Q
`and a 2-bit remainder signal Rout Output remainder signal
`Rout from block 310 is asserted as input remainder signal
`Rin of block 320. When used in divider 210 of FIG. 2,
`divide-by-three circuit 300 has a remainder bus 312 con
`nected to multiplexer 220 (FIG. 2), dividend buses 311 and
`321 connected to multiplexer 221, remainder output bus 324
`connected to register 230, and quotient output busses con
`nected to register 231. Multiplexers (not shown) may pro
`vide the connections which select divide-by-three circuit
`300 in response to the corresponding bit allocation of a
`subband.
`FIG. 3B shows a logic table relating the input signals Rin
`and C to output signals Rout and Q for each of circuit blocks
`310 and 320. During the ?rst clock cycle, the maximum
`input signal Rin applied to bus 312 equals the maximum
`signal CiH which is 01 because code Ci Contains only ?ve
`bits. The maximum remainder for any divide-by-three is 10
`binary, a