`
`r‘ON_}
`
`c
`
`PATENT
`NUMBER
`
`587008?
`
`iiiiiiiiiiiiiiiiiiiiiiiiiiin
`
`i
`
`
`
`
`
`Gnopif
`
`UNIT
`
`EXAMINER
`
`\/0
`
` PATENT DATE
`£513 (33 M9
`
`
`SUBOLASS
`
`_ssi=uAL Ni_JMeEi_=i
`
`FILING DATE cuss
`
`
`
`T3,
`
`
`
`
`
`
`
`ATTORNEYS
`DOCKET NO.
`
`
`
`7-
`
`v
`
`1
`
`STATEOR
`
`TOTAL
`
`INDEP.
`
`Veriflodandmkmwiedsed ‘server --i '
`'
`.4
`
`
`
`2
`
`
`.:i
`L C:o¢/;;/ G’
`’
`I
`,-55/P/We %/-’it3‘f‘Z7/ EL Vb.
`/V}-5':
`lb;/17’ ’ A
`,'
`’T?r’i,Tr~/W L973” hf]: ,._,» —
`
`u.
`
`e
`
`i
`
`55
`
`PARTS or APPLICATION
`F“-ED SEPARATE‘-Y
`
`'
`
`,7
`
`‘
`
`/
`it
`A Iications Examiner
`cums ALLOWED .
`’
`
`Total Claims fr
`
`H
`Pi=iUt<.rw3urEri
`PFEIEMRY EXAMINER
`‘3”‘5“*’? 34'”
`
`Sheets,nNg. Figs. 4 g.
`l L
`( A i
`'Bs§IPcEH -‘féfi
`¢9%~‘/5-’5~av
`Primary Examiner NUMBER
`PREPAFIED FOR ISSUE
`
`'. uthorized disclosure may be prohibited
`WARNING: The information disclosed herein may be restricted.
`81 and 368. Possession outside the US.
`by the United States Code Title 35, Sections 12
`Patent 8- Trademark Office is restricted to aut rized employees and contractors only.
`
`‘
`
`
`
`Ni 0/
`
`ASUS Exhibit 1002 - Page 1
`
`Form PTO-436A
`(Ftev. 3192)
`
`_
`
`mi
`
`ASUS Exhibit 1002 - Page 1
`
`
`
`A-|"PL'|(.‘3ATJ|C)i;l
`~1sumInviwutwgggqgulxlImullulll»
`CONTENTS
`
`% AAPPVROVED FoHu;:ENsE" ,
`.5 1
`5
`
`
`
`W/748269
`
`erE’t2‘r‘Za
`Cotflnrgted
`
`.‘%
`
`,
`
`_,
`
`_,
`
`r
`
`.
`
`V
`
`1
`
`. V"
`
`''x.-
`
`.
`
`-
`
`I
`
`I
`
`.
`
`'
`
`.
`
`.
`
`'
`
`'
`
`W
`
`I‘
`’: _. I“
`‘
`I
`Gmm !EBO‘
`
`'
`
`.
`
`_-
`.1
`
`‘E
`
`I’U
`it} "
`
`’
`
`I’ U-
`
`(FRbNT)
`
`ASUS Exhibit 1002 - Page 2
`
`ASUS Exhibit 1002 - Page 2
`
`
`
`Staple Issue
`
`POSITION
`
`,_
`
`.
`
`9'-AS9'F'ER
`'
`‘
`«
`EXAMINER : 5
`— —
`VERIFIER - Z 7%
`‘
`~ convsconn.T
`SPEC-HAND -_
`“LE MA'NT- -—
`DRAWNG _—
`
`INDEX OF CLAIMS
`
`—
`-—
`—m_
`I%E
`
`——
`E-
`E-—
`I--
`
`-_
`
`—
`——
`——
`—
`
`SYMBOLS
`
`
`
`Wm
`
`Dale
`
`W44% XI‘
`“:6 -
`-
`
`.1“
`
`I
`- -
`I:
`I W‘: W
`= 5! A -
`ib ———
`gl I-—--
`13 " I --
`14? II V
`-
`"
`--I
`
`L
`
`.p..:>r.-zmwco——«c>toa:-.I0'>
`
`hM
`
`
`
`4:.;~..2.4:.D-
`
`L¢=
`
`¢
`_
`
`..
`
`Rejected
`Alfiuweu
`ml] Canceled
`,. Hestrficted
`. Non-elected
`. Interference
`. Appeal
`Dhieclad
`
`
`
`
`(LEFT INSIDE}
`
`
`ASUS Exhibit 1002 - Page 3
`
`ASUS Exhibit 1002 - Page 3
`
`
`
`
`
`
`
`SEARCH NOTESN
`Exmr.
`
`
`
`fnlflt-|'l' fll ITRIDFI
`
`ASUS Exhibit 1002 - Page 4
`
`ASUS Exhibit 1002 - Page 4
`
`
`
`PATENT NUMBER
`
`APPLICATION SERIAL NUMBER
`
`,- my
`_E
`3’-‘«’U.S.GOVER!_*£_l_u!§_'h‘JT r‘mrmm:-_or 1:-19.7-a“ap:.‘§‘?‘.:a “
`.- E/=
`oaIe1NALcLAssu§IcAtIoN
`CLASS
`SUBCLASS
`
`'3 L} (
`
`7:7 0 9'
`
`.
`
`.
`
`-
`
`IF HEISSUE. ORIGINAL PATENT NUMBER
`
`
`
`
`
`
`
`fl -A-‘
`
`
`
`ASSISTANT EXAMINER (PLEASE STAMP OR PRINT FULL NAME)
`
`
`CL-\ F1‘
`V 0
`
`PRIMARY EXAMINER (PLEASE STAMP OR PRINT FULL NAME}
`
`"MU K .
`
`PTO 270
`'
`u.s. nemnrmem or commence
`PATENT AND TRADEMRHK OFFICE
`ISSUE CLASSIFICATION sup
`(REV. 5-91)
`
`
`
`’-_1'—7‘\P"J'I.'.'l“V"-r"‘4:——---‘-
`
`ASUS Exhibit 1002 - Page 5
`
`
`
`
`
`
`
`r“_r_,_‘,‘___,,_,_,,,,,..,.,,.,fi__,..,_.(A..,...r_.H__.,.._.—.._..—.-.-.‘.-«.~...‘q-;—V-""‘b"‘l’7"‘*'x‘-tam-_r
`
`
`
`
`
`ASUS Exhibit 1002 - Page 5
`
`
`
`United States Patent
`
`[19]
`
`[11] Patent Number:
`
`5,870,087
`
`Chau
`
`[45] Date of Patent:
`
`Feb. 9, 1999
`
`US005870087A
`
`[54] MPEG DECODER SYSTEM AND METHOD
`HAVING A UNIFIED MEMORY FOR
`TRANSPORT DECODE AND SYSTEM
`CONTROLLER FUNCTIONS
`
`Inventor; Kwok Kit Chan, L05 Altos, Calif.
`
`Assignee; LSI Logic Corporation, Milpitas,
`Calif,
`
`[57]
`
`ABSTRACT
`
`.
`.
`An MPEG decoder system and method for performing video
`decoding or decompression which includes a unified
`memory for multiple functions according to the present
`invention. The video decoding system includes transport
`logic, a system controller, and MPEG decoder logic. The
`video decoding system of the present invention includes a
`single unified memory which stores code and data for the
`transport, system controller and MPEG decoder functions.
`
`Appl. No‘: 748,269
`
`NOV- 13, 1995
`Filctli
`Int. (31.6 ................................................. G061‘ 13/00
`U Q Cl
`345 302
`_“ ‘
`' “““““““
`/
`Field of Search
`345/302> 4183
`I
`707/1O1> 1027 1035 104
`References Cited
`
`U5. PATENT DOCUMENTS
`
`5,675,511
`5,692,213
`5,767,846
`
`......................... .. 345/302
`10/1997 Prasad et al.
`11/1997 Goldberg et al.
`.
`345/302
`6/1998 Nakamura et al.
`345/302
`
`P]‘in1a]yExan1ine]-—Phu K. Nguycn
`Assistant Exammer_C11ff N, V0
`Attorfle)’, Agent, or Fz'rm4Conley, Rose & Tayon; Jeffrey C.
`Hood
`
`The single unified memory is preferably a 16 Mbit memory.
`The MPEG decoder logic includes a memory controller
`which couples to the single unified memory, and each of the
`‘“‘“SP°“ 1°31?’ S-Y5“? "°“”°““" and MPEG d‘°'°°d‘°'r 1°31“
`access the single unified memory through the memory
`controller. The Video decoding system implements various
`frame memory saving schemes, such as compression or
`dynamic allocation, to more efficiently use the memory. In
`one embodiment,
`the memory is not required to store
`reconstructed frame data during B-frame reconstruction,
`thus considerably reducing the required amount of memory
`for this function. Alternatively, the memory is only required
`to store a portion of the reconstructed frame data.
`In
`addition,
`these savings in memory allow portions of the
`memory to also be used for transport and system controller
`functions. The present
`invention thus provides a video
`decoding system with reduced memory requirements.
`
`20 Claims, 16 Drawing Sheets
`
`Channel
`Receiver
`202
`—
`
`T
`
`rt
`
`5,afif§§
`204
`
`ASUS Exhibit 1002 - Page 6
`
`
`
`Feb. 9, 1999
`
`Sheet 1 of 16
`
`5,870,087
`
`ASUS Exhibit 1002 - Page 7
`
`
`
`U.S. Patent
`
`99919:heF
`
`Sheet 2 of 16
`
`5,870,087
`
`
`
`.mm%_o:n_o
`
`commwoo._n:
`
`gm_On_
`
`
`
`owns.omn__>_
`
`
`
`._oUoocm_Lwuoomo
`
`ASUS Exhibit 1002 - Page 8
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`6lf03teehS
`
`5,870,087
`
`_anoocm82>
`
`_.IIIIIIIll_
`
`:o%=m_._.
`
`
`
`wn___>_+_m::2o
`
`EH9gm.o>_m8m
`
`ASUS Exhibit 1002 - Page 9
`
`
`
`U.S. Patent
`
`99911,heF
`
`Sheet4 0f16
`
`5,870,087
`
`
`mm_aEmmco:mm:waEoo%E883cones.poomw._m>c_cozmmscmsd
`EH
`
`
`
`
`
`>_oEm_>_m:__uoown_
`
`mrmw@WNow
`
`
`
`m:o...m-mEEn_cmowmw._m>:_£mco._o_n_m_.m>
`
`ASUS Exhibit 1002 - Page 10
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`6lf05teehS
`
`5,870,087
`
`
`
`
`
`
`
`
`
`fococmoE§.nBzocm.M:9_2amEm._EEcocm
`
`
`
`$93oEm._E%o.NEo.Nmwsfl
`
`>n>_\+
`
`mom.Em?>n>_.+
`
`...$em$-o§N.m
`
`
`
`
`
`>___o252._.$8m2_.___wees?8mE§-p
`
`
`
`_mE8M.850%m$aE8
`
`
`
`
`
`:o_m$aE8:o_fiuo__mo_EmE€
`
`
`
`Em:Sofia
`
`E82
`
`
`
`cofimmasooEmofifi53x83
`
`
`
`_m8_Bmwmaeoo._EmEmom-Nm.m\x.Em_..a$-9.
`
`
`
`
`
`ASUS Exhibit 1002 - Page 11
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 6 of 16
`
`5,870,087
`
`
`
`co=_:mn_boEo_>_umu8on_wmn=>__238
`
`
`
`
`
`
`
`
`
`.o__o:<um>m_qm_n_mmmaeoo
`
`as.E:5
`a_m_,~_~..
`
`2o_So_m
`
`
`
`€82.m:o¢:mE£m:_-:o=82>
`
`Eran>a>
`
`2%>m_%_uéou_>
`
`5:3o_u=m
`
`Amman:8.8535
`
`Soumew:
`
`
`
`
`
`mocemgu.8:2m_o%_>-o_u=m
`
`5:3_mE_m;o.9_m§-n:m
`
`
`
`3:3commas:82>
`
`
`
`$.55toqmcma226
`
`wagea_2__._as
`
`oim>m_%_u-o_c=m
`
`EmuwEmzaom.2_m§-n=m
`
`5:3_oEm_t+22¢mew:
`
`_£o._.Ego
`
`80
`
`
`
`
`
`u$Eo>oScam;.208
`
`ASUS Exhibit 1002 - Page 12
`
`
`
`U.S. Patent
`
`99919:heF
`
`Sheet 7 of 16
`
`5,870,087
`
`
`
`co=_:mn_aoEos_umuoomo@mn__>_F22¢
`
`
`
`
`
`
`
`
`
`oESu_-m_w_oE=<wwoaeoomE£u_-m$22.00m_Ememmww_ofi:<mwmaeoo
`
`
`
`
`
`
`
`
`
`838.:3o_8N.2:33.:.
`
`mm.®_u_
`
`
`
`m:o.o$.mm5_o5_m
`
`an>30CEasmom:am20%End
`38:.Eng.38.“..ass
`
`
`
`
`
`§_£3.Efiqw%w_$$
`
`
`
`
`
`._<n_85._<..._8525.5%3_._<n_mficoemom$~.m692
`
`ASUS Exhibit 1002 - Page 13
`
`
`
`Feb. 9, 1999
`
`Sheet 8 of 16
`
`5,870,087
`
`$
`{D
`O’)
`
`$
`N
`l\
`
`$
`to
`O)
`
`V
`A L
`
`(uq/oes/alfiq) Mowewv/u1p!Mpu2av
`
`§
`
`.5
`
`E
`
`T
`8
`
`§
`
`3
`
`8
`
`S?
`
`8
`
`a
`‘EL .
`
`13,44?,3s5
`
`
`
`MemorySize(bit)
`
`(O
`J
`0.©
`‘
`
`GV
`
`+U
`
`(099/selfiq) U1P!MPU98
`
`%?C
`s
`.C
`
`Ca
`
`O0
`
`5
`an
`
`EE E
`
`;
`as
`.5‘
`U)
`E‘-o
`E<1:
`
`2(
`
`D> EE3
`
`"C
`o
`co
`E‘o
`E0:
`E
`
`Cc
`
`ASUS Exhibit 1002 - Page 14
`
`
`
`Feb. 9, 1999
`
`Sheet 9 of 16
`
`5,870,087
`
`D I]
`
`Kb 40 ¢ <1>
`
`(liq/069/elm) Mowew v/u1p!Mpueav
`G
`
`8 5
`
`.3
`5J
`
`%‘C
`O
`.C
`
`CC
`
`0o
`
`z:
`(D
`
`7:
`.0
`V
`&’
`<0
`g‘
`E
`as
`E
`
`&\\\\\\'
`V/////A
`
`(993/391/W) HIP!/“PU98
`
`EE E
`
`-,
`cu
`.E
`U)
`a~_,
`§<
`mo‘
`2
`W
`>
`
`E7
`
`23
`
`‘C!
`U
`an
`E‘O
`ECD
`2
`
`CC
`
`ASUS Exhibit 1002 - Page 15
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 10 of 16
`
`5,870,087
`
`E
`
`
`
`
`
`€mmmmaEoo._o_5mmemi82>
`
`arsmSam>m_%_n_-o%_>
`
`atzmmuoomo82>
`
`swamo_2_<
`
`
`
`Etsm>oc2m._o%_>-o__..5<
`
`_m...smaim>m_%_n_-o_u=<
`
`
`
`._mt=m_:o%:E._.
`
`atzm>m_>
`
`
`
`ummEm>oammo:mum
`
`28Em$8mn___>_
`
`omo
`
`
`
`
`
`was_....V.§m.95.
`
`ASUS Exhibit 1002 - Page 16
`
`
`
`U.S. Patent
`
`b.eF
`
`99919:
`
`Sheet 11 of 16
`
`5,870,087
`
`._§;Emm
`
`
`
` =o__.mm:8Eoo:o=o_>_H23253.5
`
`El
`
`
`
`
`
`m_gm__m><_%_%%m.N_._s_2:E9.25:s_<m_om_>:x£55>.
`
`
`
`
`
`
`
`EmaEmmuoomn___>_5..om293s_omm:_..o8xm_.
`
`
`
`
`
`oaaanzoomm_
`
`ASUS Exhibit 1002 - Page 17
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 12 of 16
`
`5,870,087
`
`
`
`|:|1-cycle/coeffS2—cycIe/coeffH4-cycle/coeff—E‘-27MHz 73-
`
`
`
`54MHz 6-81MHz
`
`55.
`\
` \\ ,
`
`HI
`
` §§§§i§§§$ E
`
`
`
`MemorySize(bit)
`
`H 4 1."
`
`1.
`
`>1oo|qoJoew/apflg
`
`CD
`.5
`(D
`
`%‘C
`U
`.C
`
`C(
`
`o0
`
`6
`(D
`
`EE
`
`LL,
`3.0
`gffl
`0,2
`’)
`--
`Q)
`3O
`D.
`O‘)
`.5(D
`U}
`(DO
`OL.
`D.
`
`2U
`
`>I
`
`ASUS Exhibit 1002 - Page 18
`
`
`
`Nzgo
`
`s_<m_o
`
`.\N\\\N%\\w_.=>_
`
`ASUS Exhibit 1002 - Page 19
`
`
`
`tHE
`
`b.eF
`
`99919;
`
`S
`
`N
`
`s_<mom
`
`m..
`
`V
`
`ASUS Exhibit 1002 - Page 20
`
`
`
`U.S. Patent
`
`b.eF
`
`99919;
`
`Sheet 15 of 16
`
`5,870,087
`
`
`
` mosommom.»9
`
`
`
`Euoomo>>n_z
`
`>n_w_._.
`
`Ioo
`
`:o_a__ommn_
`
`
`
`Ema=o_mm_Em=m#
`
`
`
`
`
`E2m>=o£wzmcm5330Sac=o_mm_Ew=m#
`
`
`
`soi.2_.EE__was225§_.,.$:02
`
`
`
`omosae228gm;.m._v22..520Sam
`
`
`
`Exomq26:5:zumom=o_mm_E2m_._
`
`ASUS Exhibit 1002 - Page 21
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`61f061teehS
`
`5,870,087
`
`IEmmoFmmV_._.NNNNm
`
`E..U_n_
`
`
`
`ouE=8U._o>>mus;
`
`ASUS Exhibit 1002 - Page 22
`
`
`
`5,870,087
`
`1
`MPEG DECODER SYSTEM AND METHOD
`HAVING A UNIFIED MEMORY FOR
`TRANSPORT DECODE AND SYSTEM
`CONTROLLER FUNCTIONS
`INCORPORATION BY REFERENCE
`
`The following references are hereby incorporated by
`reference.
`
`The ISO/IEC MPEG specification referred to as ISOHEC
`13818 is hereby incorporated by reference in its entirety.
`U.S. patent application Ser. No. 08/654,321 titled
`“Method and Apparatus for Segmenting Memory to Reduce
`the Memory Required for Bidirectionally Predictive-Coded
`Frames” and filed May 28, 1996 is hereby incorporated by
`reference in its entirety as though fully and completely set
`forth herein.
`
`U.S. patent application Ser. No. 08/653,845 titled
`“Method and Apparatus for Reducing the Memory Required
`for Decoding Bidirectionally Predictive-Coded Frames Dur-
`ing Pull—Down” and filed May 28, 1996 is hereby incorpo-
`rated by reference in its entirety as though fully and com-
`pletely set forth herein.
`U.S. patent application Ser. No. 08/689,300 titled
`“Method and Apparatus for Decoding B Frames in Video
`Codecs with Minimal Memory” and filed Aug. 8, 1996 now
`U.S. Pat. No. 5,818,533, whose inventors are David R. Auld
`and Kwok Chau, is hereby incorporated by reference in its
`entirety as though fully and completely set forth herein.
`1. Field of the Invention
`
`'lhe present invention relates generally to digital video
`compression, and more particularly to an MPEG decoder
`system which includes a single unified memory for MPEG
`transport, decode and system controller functions.
`2. Description of the Related Art
`Full-motion digital video requires a large amount of
`storage and data transfer bandwidth. Thus, video systems
`use various types of video compression algorithms to reduce
`the amount of necessary storage and transfer bandwidth. In
`general, different video compression methods exist for still
`graphic images and for full—motion video. Intraframe com-
`pression methods are used to compress data within a still
`image or single frame using spatial redundancies within the
`frame. Interframe compression methods are used to com-
`press multiple frames, i.e., motion video, using the temporal
`redundancy between the frames. Interframe compression
`methods are used exclusively for motion video, either alone
`or in conjunction with intraframe compression methods.
`Intraframe or still image compression techniques gener-
`ally use frequency domain techniques, such as the discrete
`cosine transform (DCT). Intraframe compression typically
`uses the frequency characteristics of a picture frame to
`efficiently encode a frame and remove spatial redundancy.
`Examples of video data compression for still gaphic images
`are JPEG (Joint Photographic Experts Group) compression
`and RLE (run-length encoding). JPEG compression is a
`group of related standards that provide either lossless (no
`image quality degradation) or lossy (imperceptible to severe
`degradation) compression. Although JPEG compression was
`originally designed for the compression of still images rather
`than video, JPEG compression is used in some motion video
`applications. The RLE compression method operates by
`testing for duplicated pixels in a single line of the bit map
`and storing the number of consecutive duplicate pixels
`rather than the data for the pixels themselves.
`In contrast to compression algorithms for still images,
`most video compression algorithms are designed to com-
`
`2
`press full motion video. As mentioned above, video com-
`pression algorithms for motion video use a concept referred
`to as interframe compression to remove temporal redundan-
`cies between frames. Interframe compression involves stor-
`ing only the diiferences between successive frames in the
`data file. Interframe compression stores the entire image of
`a key frame or reference frame, generally in a moderately
`compressed format. Successive frames are compared with
`the key frame, and only the differences between the key
`frame and the successive frames are stored. Periodically,
`such as when new scenes are displayed, new key frames are
`stored, and subsequent comparisons begin from this new
`reference point. It is noted that the interframe compression
`ratio may be kept constant while varying the video quality.
`Alternatively,
`interframe compression ratios may be
`content-dependent, i.e., if the video clip being compressed
`includes many abrupt scene transitions from one image to
`another, the compression is less efficient. Examples of video
`compression which use an interframe compression tech-
`nique are MPEG, DVI and Indeo, among others.
`MPEG BACKGROUND
`
`A compression standard referred to as MPEG (Moving
`Pictures Experts Group) compression is a set of methods for
`compression and decompression of full motion video images
`which uses the interframe and intraframe compression tech-
`niques described above. MPEG compression uses both
`motion compensation and discrete cosine transform (DCT)
`processes, among others, and can yield compression ratios
`of more than 30:1.
`
`The two predominant MPEG standards are referred to as
`MPEG-1 and MPEG-2. The MPEG-1 standard generally
`concerns frame data reduction using block-based motion
`compensation prediction (MCP), which generally uses tem-
`poral diiferential pulse code modulation (DPCM). The
`MPEG-2 standard is similar to the MPEG-1 standard, but
`includes extensions to cover a wider range of applications,
`including interlaced digital video such as high definition
`television (HDTV).
`Interframe compression methods such as MPEG are based
`on the fact that, in most video sequences, the background
`remains relatively stable while action takes place in the
`foreground. The background may move, but large portions
`of successive frames in a video sequence are redundant.
`MPEG compression uses this inherent redundancy to encode
`or compress frames in the sequence.
`An MPEG stream includes three types of pictures,
`referred to as the Intra (I) frame, the Predicted (P) frame, and
`the Bi-directional Interpolated (B)
`frame. The I or
`Intraframes contain the video data for the entire frame of
`video and are typically placed every 10 to 15 frames.
`Intraframes provide entry points into the file for random
`access, and are generally only moderately compressed.
`Predicted frames are encoded with reference to a past frame,
`i.e., a prior Intraframe or Predicted frame. Thus P frames
`only include changes relative to prior I or P frames. In
`general, Predicted frames receive a fairly high amount of
`compression and are used as references for future Predicted
`frames. Thus, both I and P frames are used as references for
`subsequent frames. Bi-directional pictures include the great-
`est amount of compression and require both a past and a
`future reference in order
`to be encoded. Bi-directional
`frames are never used as references for other frames.
`
`In general, for the frame(s) following a reference frame,
`i.e., P and B frames that follow a reference I or P frame, only
`small portions of these frames are different from the corre-
`
`ASUS Exhibit 1002 - Page 23
`
`
`
`5,870,087
`
`3
`sponding portions of the respective reference frame. Thus,
`for these frames, only the differences are captured, com-
`pressed and stored. The differences between these frames are
`typically generated using motion vector estimation logic, as
`discussed below.
`When an MPEG encoder receives a video file, the MPEG
`encoder generally first creates the I frames. The MPEG
`encoder may compress the I frame using an intraframe
`compression technique. The MPEG encoder divides respec-
`tive frames into a grid of 16x16 pixel squares called mac-
`roblocks in order
`to perform motion estimation/
`compensation. Thus, for a respective target picture or frame,
`i.c., a frame being encoded, the encoder searches for an
`exact, or near exact, match between the target picture
`macroblock and a block in a neighboring picture referred to
`as a search frame. For a target P frame the encoder searches
`in a prior I or P frame. For a target B frame, the encoder
`searches in a prior or subsequentl or P frame. When a match
`is found, the encoder transmits a vector movement code or
`motion vector. The vector movement code or motion vector
`only includes information on the difference between the
`search frame and the respective target picture. The blocks in
`target pictures that have no change relative to the block in
`the reference picture or I frame are ignored. Thus the amount
`of data that is actually stored for these frames is significantly
`reduced.
`After motion vectors have been generated, the encoder
`then encodes the changes using spatial redundancy. Thus,
`after finding the changes in location of the macroblocks, the
`MPEG algorithm further calculates and encodes the differ-
`ence between corresponding macroblocks. Encoding the
`difference is accomplished through a math process referred
`to as the discrete cosine transform or DCT. This process
`divides the macroblock into four sub blocks, seeking out
`changes in color and brightness. Human perception is more
`sensitive to brightness changes than color changes. Thus the
`MPEG algorithm devotes more effort to reducing color data
`than brightness.
`Therefore, MPEG compression is based on two types of
`redundancies in video sequences, these being spatial, which
`is the redundancy in an individual frame, and temporal,
`which is the redundancy between consecutive frames. Spa-
`tial compression is achieved by considering the frequency
`characteristics of a picture frame. Each frame is divided into
`non—overlapping blocks, and each block is transformed via
`the discrete cosine transform (DCT). After the transformed
`blocks are converted to the “DCT domain”, each entry in the
`transformed block is quantized with respect to a set of
`quantization tables. The quantization step for each entry can
`vary, taking into account the sensitivity of the human visual
`system (HVS)} to the frequency. Since the HVS is more
`sensitive to low frequencies, most of the high frequency
`entries are quantized to zero. In this step where the entries
`are quantized, information is lost and errors are introduced
`to the reconstructed image. Run length encoding is used to
`transmit
`the quantized values. To further enhance
`compression, the blocks are scanned in a zig-zag ordering
`that scans the lower frequency entries first, and the non-zero
`quantized values, along with the zero run lengths, are
`entropy encoded.
`When an MPEG decoder receives an encoded stream, the
`MPEG decoder reverses the above operations. Thus the
`MPEG decoder performs inverse scanning to remove the zig
`zag ordering, inverse quantization to de—quantize the data,
`and the inverse DCT to convert the data from the frequency
`domain back to the pixel domain. The MPEG decoder also
`performs motion compensation using the transmitted motion
`vectors to recreate the temporally compressed frames.
`
`4
`When frames are received which are used as references
`for other frames, such as I or P frames, these frames are
`decoded and stored in memory. When a temporally com-
`pressed or encoded frame is received, such as a P or B frame,
`motion compensation is performed on the frame using the
`prior decoded I or P reference frames. The temporally
`compressed or encoded frame, referred to as a target frame,
`will include motion vectors which reference blocks in prior
`decoded I or P frames stored in the memory. The MPEG
`decoder examines the motion vector, determines the respec-
`tive reference block in the reference frame, and accesses the
`reference block pointed to by the motion vector from the
`memory.
`A typical MPEG decoder includes motion compensation
`logic which includes local or on—chip memory. The MPEG
`decoder also includes an external memory which stores prior
`decoded reference frames. The MPEG decoder accesses the
`reference frames or anchor frames stored in the external
`memory in order to reconstruct
`temporally compressed
`frames. The MPEG decoder also typically stores the frame
`being reconstructed in the external memory.
`An MPEG decoder system also typically includes trans-
`port logic which operates to demultiplex received data into
`a plurality of individual multimedia streams. An MPEG
`decoder system also generally includes a system controller
`which controls operations in the system and executes pro-
`grams or applets.
`Prior art MPEG video decoder systems have generally
`used a frame store memory for the MPEG decoder motion
`compensation logic which stores the reference frames or
`anchor frames as well as the frame being reconstructed.
`Prior art MPEG video decoder systems have also generally
`included a separate memory for the transport and system
`controller functions. It has generally not been possible to
`combine these memories, due to size limitations. For
`example, current memory devices are fabricated on an 4
`Mbit granularity. In prior art systems, the memory require-
`ments for the transport and system controller functions as
`well as the decoder motion compensation logic would
`exceed 16 Mbits of memory, thus requiring 20 or 24 Mbits
`of memory. This additional memory adds considerable cost
`to the system.
`item in the
`The amount of memory is a major cost
`production of video decoders. Thus, it is desired to reduce
`the memory requirements of the decoder system as n111cl1 as
`possible to reduce its size and cost. Since practical memory
`devices are implemented using particular convenient dis-
`crete sizes, it is important to stay within a particular size if
`possible for commercial reasons. For example, it is desired
`to keep the memory requirements below a particular size of
`memory, such as 16 Mb, since otherwise a memory device
`of 20 or 24 Mb would have to be used, resulting in greater
`cost and extraneous storage area. As mentioned above, it has
`heretofore not been possible to combine the memory
`required for the transport and system controller functions
`with the memory required for the MPEG decoder logic due
`to the memory size requirements.
`Therefore, a new video decoder system and method is
`desired which efficiently uses memory and combines the
`memory subsystem for reduced memory requirements and
`hence reduced cost.
`
`SUMMARY OF THE INVENTION
`
`The present invention comprises an MPEG decoder sys-
`tem and method for performing video decoding or decom-
`pression which includes a unified memory for multiple
`
`ASUS Exhibit 1002 - Page 24
`
`
`
`5,870,087
`
`5
`invention. The video
`functions according to the present
`decoding system includes transport
`logic, a system
`controller, and MPEG decoder logic. The video decoding
`system of the present invention includes a single unified
`memory which stores code and data for the transport logic,
`system controller and MPEG decoder functions. The single
`unified memory is preferably a 16 Mbit memory. The
`present invention thus requires only a single memory, and
`thus has reduced memory requirements compared to prior
`art designs.
`The video decoding system includes transport logic which
`operates to demultiplex received data into a plurality of
`individual multimedia streams. The video decoding system
`also includes a system controller which controls operations
`in the system and executes programs or applets. The video
`decoding system further includes decoding logic, preferably
`MPEG decoder logic, which performs motion compensation
`between temporally compressed frames of a video sequence
`during video decoding or video decompression. The
`memory includes a plurality of memory portions, including
`a video frame portion for storing video frames, a system
`controller portion for storing code and data executable by
`the system controller, and a transport bulIer for storing data
`used by the transport
`logic. The MPEG decoder logic
`preferably includes a memory controller which couples to
`the single unified memory. Each of the transport
`logic,
`system controller, and MPEG decoder logic accesses the
`single unified memory through the memory controller.
`The video decoding system implements various frame
`memory saving schemes, such as compression or dynamic
`allocation,
`to reduce the required amount of frame store
`memory. Also,
`in one embodiment,
`the memory is not
`required to store reconstructed frame data during motion
`compensation,
`thus considerably reducing the required
`amount of memory for this function. Alternatively,
`the
`memory is only required to store a portion of the recon-
`structed frame data. These savings in memory allow portions
`of the memory to also be used for transport and system
`controller functions.
`
`The present invention thus provides a video decoding
`system with reduced memory requirements.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`A better understanding of the present invention can be
`obtained when the following detailed description of the
`preferred embodiment is considered in conjunction with the
`following drawings, in which:
`FIG. 1 illustrates a computer system which performs
`video decoding and which includes a motion compensation
`logic having a frame memory which stores reference block
`data according to the present invention;
`FIG. 2 is a block diagram illustrating the computer system
`of FIG. 1‘7
`FIG. 3 is a block diagram illustrating an MPEG decoder
`system including a unified memory for MPEG transport,
`system controller, and decode functions according to the
`present invention;
`FIG. 4 is a block diagram illustrating the MPEG decoder
`logic in the system of FIG. 3;
`FIG. 5 illustrates various frame memory saving schemes
`used in various embodiments of the invention;
`FIGS. 6a and 6b illustrate a table listing the memory
`partitions under different display schemes;
`FIG. 7 illustrates the relationship of memory bandwidth
`vs. memory size in the NTSC decoding scheme;
`
`6
`FIG. 8 illustrates the relationship of memory bandwidth
`vs. memory size in the PAI, encoding scheme;
`FIG. 9 illustrates the memory partitions according to the
`preferred embodiment of the invention;
`FIG. 10 illustrates the estimated memory bandwidth dis-
`tribution in the preferred embodiment of the invention;
`FIG. 11 illustrates the “worst case” relationship of pro-
`cessing power VS. memory size in the NTSC decoding
`scheme;
`FIG. 12 illustrates the clock domains in the system;
`FIG. 13 illustrates clock operating frequencies according
`to the preferred embodiment of the invention;
`FIG. 14 illustrates an example of the packet data interface
`between the transport controller and the source decoder; and
`FIG. 15 illustrates packet header formats used in the
`preferred embodiment.
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`Video Compression System
`Referring now to FIG. 1, a system for performing video
`decoding or decompression and including a unified memory
`according to the present invention is shown. The video
`decoding system of the present invention includes a single
`unified memory which stores code and data for the transport,
`system controller and MPEG decoder functions. This sim-
`plifies the design and reduces the memory requirements in
`the system.
`in one embodiment the video decoding or
`As shown,
`decompression system is comprised in a general purpose
`computer system 60. The video decoding system may com-
`prise any of various types of systems, including a computer
`system, set-top box, television, or other device.
`The computer system 60 is preferably coupled to a media
`storage unit 62 which stores digital video files which are to
`be decompressed or decoded by the computer system 60.
`The media storage unit 62 may also store the resultant
`decoded or decompressed video file.
`In the preferred
`embodiment, the computer system 60 receives a compressed
`video file or bitstream and generates a normal uncompressed
`digital video file. In the present disclosure, the term “com-
`pressed video file” refers to a video file which has been
`compressed according to any of Various video compression
`algorithms which use motion estimation techniques, includ-
`ing the MPEG standard, among others, and the term
`“uncompressed digital video file” refers to a stream of
`decoded or uncompressed video.
`As shown, the computer system 60 preferably includes a
`video decoder 74 which performs video decoding or decom-
`pression operations. The video decoder 74 is preferably an
`MPEG decoder. The computer system 60 optionally may
`also include an MPEG encoder 76. The MPEG decoder 74
`and MPEG encoder 76 are preferably adapter cards coupled
`to a bus in the computer system, but are shown external to
`the computer system 60 for illustrative purposes. The corri-
`puter system 60 also includes software,
`represented by
`floppy disks 72, which may perform portions of the video
`decompression or decoding operation and/or may perform
`other operations, as desired.
`The computer system 60 preferably includes various
`standard components, including one or more processors, one
`or more buses, a hard drive and memory. Referring now to
`FIG. 2, a block diagram illustrating the components com-
`prised in the computer system of FIG. 1 is shown. It is noted
`that FIG. 2 is illustrative only, and other computer architec-
`tures may be used, as desired. As shown,
`the computer
`
`ASUS Exhibit 1002 - Page 25
`
`
`
`5,870,087
`
`7
`system includes at least one processor 80 coupled through
`chipset logic 82 to a system memory 84. The chipset 82
`preferably includes a PCI
`(Peripheral Component
`Interconnect) bridge for interfacing to PCI bus 86, or another
`type of bus bridge for interfacing to another type of expan-
`sion bus. In FIG. 2, MPEG decoder 74 and MPEG encoder
`76 are shown connected to PCI bus 86. Various other
`components may be comprised in the computer system, such
`as video 88 and hard drive 90.
`As mentioned above, in the preferred embodiment of FIG.
`1 the computer system 60 includes or is coupled to one or
`more digital storage or media storage devices. For example,
`in the embodiment of FIG. 1,
`the computer system 60
`couples to media storage unit 62 through cable 64. The
`media storage unit 62 preferably comprises a RAID
`(Redundant Array of Inexpensive Disks) disk array, or
`includes one or more CD—ROM drives and/or one or more
`Digital Video Disk (DVD) storage units, or other media, for
`storing digital video to be decompressed and/or for storing
`the resultant decoded video data. The computer system may
`also include one or more internal RAID arrays, CD—ROM
`drives and/or may couple to one or more separate Digital
`Video Disk (DVD) storage units. The computer system 60
`also may connect to other types of digital or analog storage
`devices or media, as desired.
`Alternatively, the compressed digital video file may be
`received from an external source, such as a remote storage
`device or remote computer system. In this embodiment, the
`computer system preferably includes an input device, such
`as an ATM (Asynchronous Transfer Mode) adapter card or
`an ISDN (I11tegrated Services Digital Network) terminal
`adapter, or other digital data receiver, for receiving the
`digital video file. The digital video file may also be stored or
`received in analog format and converted to digital data,
`either externally to the computer system 60 or within the
`computer system 60.
`As mentioned above, the MPEG decoder 74 in the com-
`puter system 60 performs video decoding or video decom-
`pression functions. As discussed further below, the video
`decoding system includes transport logic which operates to
`demultiplex received data into a plurality of individual
`multimedia streams. The video decoding system also
`includes a system controller which controls operations in the
`system and executes programs or applets comprised in the
`stream. The video decoding system further includes decod-
`ing logic, preferably MPEG decoder logic, which performs
`motion compensation between temporally compressed
`frames of a video sequence during video decoding or video
`decompression. The video decoding system of the present
`invention includes a singl