`Chau
`
`US005870087A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,870,087
`Feb. 9, 1999
`
`[54] MPEG DECODER SYSTEM AND METHOD
`HAVING A UNIFIED MEMORY FOR
`TRANSPORT DECODE AND SYSTEM
`CONTROLLER FUNCTIONS
`
`[75] Inventor: KWok Kit Chau, Los Altos, Calif.
`
`[73] Assignee: LSI Logic Corporation, Milpitas,
`Calif.
`
`[21] Appl. No.: 748,269
`[22]
`Filed:
`Nov. 13, 1996
`
`[51] Int. Cl.6 ................................................... .. G06T 13/00
`[52] US. Cl. ............................................................ .. 345/302
`[58] Field of Search ................................... .. 345/302, 418;
`707/101, 102, 103, 104
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,675,511 10/1997 Prasad et a1. ......................... .. 345/302
`5,692,213 11/1997 Goldberg et a1.
`345/302
`5,767,846
`6/1998 Nakamura et a1. ................... .. 345/302
`
`Primary Examiner—Phu K. Nguyen
`Assistant Examiner—Cliff N. V0
`Attorney, Agent, or F irm—Conley, Rose & Tayon; Jeffrey C.
`Hood
`
`[57]
`
`ABSTRACT
`
`An MPEG decoder system and method for performing video
`decoding or decompression Which includes a uni?ed
`memory for multiple functions according to the present
`invention. The video decoding system includes transport
`logic, a system controller, and MPEG decoder logic. The
`video decoding system of the present invention includes a
`single uni?ed memory Which stores code and data for the
`transport, system controller and MPEG decoder functions.
`The single uni?ed memory is preferably a 16 Mbit memory.
`The MPEG decoder logic includes a memory controller
`Which couples to the single uni?ed memory, and each of the
`transport logic, system controller and MPEG decoder logic
`access the single uni?ed memory through the memory
`controller. The video decoding system implements various
`frame memory saving schemes, such as compression or
`dynamic allocation, to more ef?ciently use the memory. In
`one embodiment, the memory is not required to store
`reconstructed frame data during B-frame reconstruction,
`thus considerably reducing the required amount of memory
`for this function. Alternatively, the memory is only required
`to store a portion of the reconstructed frame data. In
`addition, these savings in memory alloW portions of the
`memory to also be used for transport and system controller
`functions. The present invention thus provides a video
`decoding system With reduced memory requirements.
`
`20 Claims, 16 Drawing Sheets
`
`1
`
`A
`
`A
`
`22
`
`Coded
`Stream
`
`'
`
`|_ _ _ _ 1L _ _ _ w
`‘
`1
`.
`Trans ort
`i
`“APE/SI
`+M|r9s
`Channel
`4 :l
`Decoder I
`204
`‘
`Regg'zver
`— @ ‘El
`:
`g I
`
`| ——————— - _,|
`l
`.
`
`E vldeozlggcoder :'__> Video
`——
`'- ——————— —-'
`
`l _______ — — ‘I
`
`|
`
`.
`
`,
`Audlo DAC |
`|
`228
`r—-> AUdlO
`|
`i — l
`
`l
`
`|_ _ _ _ _ _ _ _ I
`
`‘T
`210
`
`212
`
`16Mbit
`SDRAM
`
`ASUS Exhibit 1001 - Page 1
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 1 0f 16
`
`5,870,087
`
`FIG. 1
`
`
`
`Media Storage Unlt 62
`
`ASUS Exhibit 1001 - Page 2
`
`
`
`U.S. Patent
`
`F
`
`9m
`
`%
`
`8,5
`
`700
`
`0,.WNOE
`
`,82>M8
`
`
`om_n_s_M8EE
`om_n_s_M®>_._DEvoocmEvoomoM.22..
`
`
`
`mam_On_
`
`8
`
`swim63_._oDn_O5mm
`
`
`
`boEo_>_commmoenc
`
`ASUS Exhibit 1001 - Page 3
`
`ASUS Exhibit 1001 - Page 3
`
`
`
`
`U.S. Patent
`
`_b
`
`9m
`
`m
`
`5
`
`7000,
`
`MNE
`
`0.M,mGE
`
`322
`
`_2$5m
`
`_aooocm82>
`
`_.IIIIIIIll_
`
`mEN
`
`mn___>_+
`
`9.,SN%
`
`:o%:m_.—
`
`_mEm;o
`
`%_m>_8mm
`
`880
`
`E85
`
`ASUS Exhibit 1001 - Page 4
`
`ASUS Exhibit 1001 - Page 4
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 4 0f 16
`
`5,870,087
`
`QBwéEmE
`
`% boEoE
`
`F=>EO
`
`woEEmw “ uwvoomo “A
`
`|
`
`
`
`Q|S |
`
`25 3?:
`
`
`
`
`5:22 .50 @295 v2.95
`
`cozhwcmaEoo mom cosmwwwwms?
`
`
`
`w .0E
`
`583 2§a>
`
`950080
`
`UmUoO
`
`§
`
`Ema
`
`ASUS Exhibit 1001 - Page 5
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 5 of 16
`
`5,870,087
`
`
`
`_8o_.nmmmmaeoo,.EoEmmm-mm.m\x25503-8
`
`
`
`
`
`
`
`cofimmasoo58.55§§x$3
`
`Scocmw§_o:moEm_..-aEcocmmN.%.NwBcocmmE§-p
`
`
`
`
`
`
`
`
`Sm.Eon.V>9,$magioEeE%o.NEo.N
`
`
`
`25252238m25..2029if_mom5:NmE§-n
`
`_m_:8wBficmmmoaeoo
`
`m.O_n_
`
`>9,_+\x\\x\
`
`...z¢em$-oEN.m
`
`ASUS Exhibit 1001 - Page 6
`
`
`
`
`
`go:£_;
`
`389
`
`
`
`Em:So£_>>
`
`E82
`
`o_EmE€ :o_m$aE8co_..8o__m
`
`ASUS Exhibit 1001 - Page 6
`
`
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 6 of 16
`
`5,870,087
`
`
`
`:o=_:mn_boEm_>_u%8wn_wmn_s_F£3
`
`
`
`
`
`
`
`
`
`§_u:<8>m_%_n_mmmaeoo
`
`oz.R:Z._<n_
`a_mN.m.V
`
`90.23
`
`
`
`muoomum:oo=mEsm:_-=o=82>
`
`Etna>n_>
`
`oiw>m_%_u-o%_>
`
`5:3o_u=m
`
`wmsmcB.8535
`
`92mmeg
`
`
`
`
`
`mocemte>o:2m_oo_u_>-o_u=m
`
`
`
`Stan$5.2.82>
`
`oim>m_%_u-o_u=m
`
`
`
`SE5_oE_m;o5.._£oE=m
`
`mgza5_2___me...
`
`
`
`
`
`u$Eo>o.33...9_o8
`
`<m..U_n_
`
`
`
`Emuwemzaom5.._m§-n=m
`
`_£o._.Ego
`
`
`
`$.95_m_Em;o+2%meat
`
`omo
`
`ASUS Exhibit 1001 - Page 7
`
`
`
`
`
`5:33toamcm:o_c=m
`
`ASUS Exhibit 1001 - Page 7
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 7 of 16
`
`5,870,087
`
`
`
`:o___:mn_30.52umuoooo0m...__>_v23
`
`
`
`
`
`
`
`oE9n_-m_w_oE:<mmmaeoo
`
`
`
`mEmi-mwmmaeoo
`
`
`
`m_Ememmww_o:o:<mwmaeoo
`
`
`
`
`
`2&5?ow”._<n_25:53Newsnomhz
`
`mo.0_n_
`
`ASUS Exhibit 1001 - Page 8
`
`ASUS Exhibit 1001 - Page 8
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 8 of 16
`
`5,870,087
`
`100% 96% 87% 72% 100% 96% 87% 72%
`D [I
`III 43 <l> <15
`
`(uq/oes/emu) Mowewv/u1p!Mpu2av
`
`180
`
`160
`
`140
`
`20
`
`O0
`
`8
`
`8
`
`S?
`
`R’.
`
`O
`
`§ EC
`
`.)
`
`2
`
`to
`
`N
`9-
`LL
`
`K!
`
`E,
`
`_‘,_i’
`V
`3 E. 3%
`33
`E5
`g»
`5
`E
`«E
`(U
`U3
`
`3*
`.
`\
`3%
`"'Z."$:‘{’////
`>¢o°O,4i
`I;
`§
`T‘
`
`gr,
`
`3§“
`
`§ §
`
`§
`
`.3
`
`_
`§
`
`§
`
`Q
`§
`
`i
`§
`
`Q
`§
`
`3
`§
`
`3
`§
`
`(oes/semq) UIPIMPUBE
`
`ASUS Exhibit 1001 - Page 9
`
`ASUS Exhibit 1001 - Page 9
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 9 of 16
`
`5,870,087
`
`(liq/099/em) Mowew V/tnplmpueav
`C)
`3
`©
`3
`3
`C)
`C)
`3
`V‘
`C’)
`(\l
`\—
`
`G
`3
`L0
`
`3
`
`600
`
`5,3
`5
`3
`§ *
`
`5
`. § 00
`
`-9
`
`E3 <_D'
`3
`“-
`‘Q
`..,§
`:22
`
`*
`
`Rg 3
`
`(993/3915C!) HIP!/V\PU98
`
`ASUS Exhibit 1001 - Page 10
`
`§ E
`
`g
`
`.3
`‘$4
`E:
`%
`.2
`
`§
`
`2E é
`
`’
`
`ASUS Exhibit 1001 - Page 10
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 10 0f 16
`
`5,870,087
`
`5
`
`45 022
`
`
`03.8 8% E5 >223 85-062
`
`Nag Nag E5 23
`83 08.9 $5 2% aawagé
`$3 8% E5 r323
`838 @868 E5 2% $95-8?
`
`80.0w QOQON @8590 $8: $1
`
`
`
`
`
`80.8 08.8 was 25 2:,
`
`E5 @303 $5 $88 82>
`
`
`
`
`
`widow 05.2w Qwo
`
`a .QE
`
`
`
`$8.33 awmmqw ES Em Q80 25
`
`80.83 $0.83 Ea >m>
`
`
`
`
`
`
`
`
`
`
`
`$32.2 3%.? @3238 .65 $53 82>
`
`ASUS Exhibit 1001 - Page 11
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 11 of 16
`
`5,870,087
`
`_%_;Emm
`
`§§s_>_v
`
`
`
` cozmmcmgeoo=o=o_>_HE
`
`E!
`
`
`
`
`
`o_%__§<_%_%%m.N_.__>_2:E35::_>_<Em_>:x$55>.
`
`
`
`
`
`
`
`
`
`Sam25muoomn___>_.28m\o:5s_oN§§_xm_.
`
`o$E3_>_8Nm_
`
`S.O_n_
`
`ASUS Exhibit 1001 - Page 12
`
`ASUS Exhibit 1001 - Page 12
`
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 12 of 16
`
`5,870,087
`
`%%%
`%§§~~~££££££
`%%$§§§§§§§§§
`
`1—C\l<l'(\lLDOO\-'1—\—\—1—C\l
`,.
`Uf13<f3<1><1>9l‘#4|¢+
`
`E8
`
`
`
`
`0 0
`
`<1
`
`5
`
`E I
`
`8
`IIE
`
`6,000
`
`5000
`
`4,000
`
`3000
`
`2000
`
`1000
`
`>1oo|qoJoew/e|oA3
`
`ASUS Exhibit 1001 - Page 13
`
`ASUS Exhibit 1001 - Page 13
`
`
`
`////%
`
`A
`
`fl
`
`ooooomo
`ooooo m0
`
`
`mmmmmm.._.
`tttttt c.
`
`NI_>_oo
`
`s_<mom
`
`Nw.O_n_
`
`ASUS Exhibit 1001 - Page 14
`
`
`
`
`N_._s_w2
`
`ooooomo
`
`mmmmmmhe
`
`%/
`
`ASUS Exhibit 1001 - Page 15
`
`
`
`U.S. Patent
`
`9,
`
`9
`
`780,0
`
`u_m__o_EoomE88252Emma
`
`
`
`
`mosomm_OE.:o%:E._.
`
`Pr
`
`M.5,3er.
`
`
`m_nm:m_S930Ema=o_wm_Em=m_._M93=o_mm_Em=m_.—.m._V22..520Emau303%:9__away225E9352a3,2@503ME0825:5:zummm:o_mm_Em=m_._M28.62298#_m3
`
`
`
`
`cozazommo
`
`ASUS Exhibit 1001 - Page 16
`
`ASUS Exhibit 1001 - Page 16
`
`
`
`
`U.S. Patent
`
`Feb. 9, 1999
`
`Sheet 16 0f 16
`
`5,870,087
`
`$928
`
`$226
`
`LS8 20;
`
`L58 20;
`
`
`
`ouE=8 E0; : was; $9
`
`as. o: Emmi
`
`8mm 88 528
`
`“25> ES @960
`
`
`
`
`
`“255 E85 82>
`
`
`
`
`
`“25> 585m 22%
`
`ASUS Exhibit 1001 - Page 17
`
`
`
`1
`MPEG DECODER SYSTEM AND METHOD
`HAVING A UNIFIED MEMORY FOR
`TRANSPORT DECODE AND SYSTEM
`CONTROLLER FUNCTIONS
`
`INCORPORATION BY REFERENCE
`The following references are hereby incorporated by
`reference.
`The ISO/IEC MPEG speci?cation referred to as ISO/IEC
`13818 is hereby incorporated by reference in its entirety.
`US. patent application Ser. No. 08/654,321 titled
`“Method and Apparatus for Segmenting Memory to Reduce
`the Memory Required for Bidirectionally Predictive-Coded
`Frames” and ?led May 28, 1996 is hereby incorporated by
`reference in its entirety as though fully and completely set
`forth herein.
`US. patent application Ser. No. 08/653,845 titled
`“Method and Apparatus for Reducing the Memory Required
`for Decoding Bidirectionally Predictive-Coded Frames Dur
`ing Pull-DoWn” and ?led May 28, 1996 is hereby incorpo
`rated by reference in its entirety as though fully and com
`pletely set forth herein.
`US. patent application Ser. No. 08/689,300 titled
`“Method and Apparatus for Decoding B Frames in Video
`Codecs With Minimal Memory” and ?led Aug. 8, 1996 now
`US. Pat. No. 5,818,533, Whose inventors are David R. Auld
`and KWok Chau, is hereby incorporated by reference in its
`entirety as though fully and completely set forth herein.
`1. Field of the Invention
`The present invention relates generally to digital video
`compression, and more particularly to an MPEG decoder
`system Which includes a single uni?ed memory for MPEG
`transport, decode and system controller functions.
`2. Description of the Related Art
`Full-motion digital video requires a large amount of
`storage and data transfer bandWidth. Thus, video systems
`use various types of video compression algorithms to reduce
`the amount of necessary storage and transfer bandWidth. In
`general, different video compression methods exist for still
`graphic images and for full-motion video. Intraframe com
`pression methods are used to compress data Within a still
`image or single frame using spatial redundancies Within the
`frame. Interframe compression methods are used to com
`press multiple frames, i.e., motion video, using the temporal
`redundancy betWeen the frames. Interframe compression
`methods are used exclusively for motion video, either alone
`or in conjunction With intraframe compression methods.
`Intraframe or still image compression techniques gener
`ally use frequency domain techniques, such as the discrete
`cosine transform (DCT). Intraframe compression typically
`uses the frequency characteristics of a picture frame to
`ef?ciently encode a frame and remove spatial redundancy.
`Examples of video data compression for still graphic images
`are JPEG (Joint Photographic Experts Group) compression
`and RLE (run-length encoding). JPEG compression is a
`group of related standards that provide either lossless (no
`image quality degradation) or lossy (imperceptible to severe
`degradation) compression. Although J PEG compression Was
`originally designed for the compression of still images rather
`than video, JPEG compression is used in some motion video
`applications. The RLE compression method operates by
`testing for duplicated pixels in a single line of the bit map
`and storing the number of consecutive duplicate pixels
`rather than the data for the pixels themselves.
`In contrast to compression algorithms for still images,
`most video compression algorithms are designed to com
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,870,087
`
`2
`press full motion video. As mentioned above, video com
`pression algorithms for motion video use a concept referred
`to as interframe compression to remove temporal redundan
`cies betWeen frames. Interframe compression involves stor
`ing only the differences betWeen successive frames in the
`data ?le. Interframe compression stores the entire image of
`a key frame or reference frame, generally in a moderately
`compressed format. Successive frames are compared With
`the key frame, and only the differences betWeen the key
`frame and the successive frames are stored. Periodically,
`such as When neW scenes are displayed, neW key frames are
`stored, and subsequent comparisons begin from this neW
`reference point. It is noted that the interframe compression
`ratio may be kept constant While varying the video quality.
`Alternatively, interframe compression ratios may be
`content-dependent, i.e., if the video clip being compressed
`includes many abrupt scene transitions from one image to
`another, the compression is less ef?cient. Examples of video
`compression Which use an interframe compression tech
`nique are MPEG, DVI and Indeo, among others.
`
`MPEG BACKGROUND
`
`A compression standard referred to as MPEG (Moving
`Pictures Experts Group) compression is a set of methods for
`compression and decompression of full motion video images
`Which uses the interframe and intraframe compression tech
`niques described above. MPEG compression uses both
`motion compensation and discrete cosine transform (DCT)
`processes, among others, and can yield compression ratios
`of more than 30:1.
`The tWo predominant MPEG standards are referred to as
`MPEG-1 and MPEG-2. The MPEG-1 standard generally
`concerns frame data reduction using block-based motion
`compensation prediction (MCP), Which generally uses tem
`poral differential pulse code modulation (DPCM). The
`MPEG-2 standard is similar to the MPEG-1 standard, but
`includes extensions to cover a Wider range of applications,
`including interlaced digital video such as high de?nition
`television (HDTV).
`Interframe compression methods such as MPEG are based
`on the fact that, in most video sequences, the background
`remains relatively stable While action takes place in the
`foreground. The background may move, but large portions
`of successive frames in a video sequence are redundant.
`MPEG compression uses this inherent redundancy to encode
`or compress frames in the sequence.
`An MPEG stream includes three types of pictures,
`referred to as the Intra (I) frame, the Predicted (P) frame, and
`the Bi-directional Interpolated (B) frame. The I or
`Intraframes contain the video data for the entire frame of
`video and are typically placed every 10 to 15 frames.
`Intraframes provide entry points into the ?le for random
`access, and are generally only moderately compressed.
`Predicted frames are encoded With reference to a past frame,
`i.e., a prior Intraframe or Predicted frame. Thus P frames
`only include changes relative to prior I or P frames. In
`general, Predicted frames receive a fairly high amount of
`compression and are used as references for future Predicted
`frames. Thus, both I and P frames are used as references for
`subsequent frames. Bi-directional pictures include the great
`est amount of compression and require both a past and a
`future reference in order to be encoded. Bi-directional
`frames are never used as references for other frames.
`In general, for the frame(s) folloWing a reference frame,
`i.e., P and B frames that folloW a reference I or P frame, only
`small portions of these frames are different from the corre
`
`ASUS Exhibit 1001 - Page 18
`
`
`
`5,870,087
`
`3
`sponding portions of the respective reference frame. Thus,
`for these frames, only the differences are captured, com
`pressed and stored. The differences betWeen these frames are
`typically generated using motion vector estimation logic, as
`discussed beloW.
`When an MPEG encoder receives a video ?le, the MPEG
`encoder generally ?rst creates the I frames. The MPEG
`encoder may compress the I frame using an intraframe
`compression technique. The MPEG encoder divides respec
`tive frames into a grid of 16x16 pixel squares called mac
`roblocks in order to perform motion estimation/
`compensation. Thus, for a respective target picture or frame,
`i.e., a frame being encoded, the encoder searches for an
`exact, or near exact, match betWeen the target picture
`macroblock and a block in a neighboring picture referred to
`as a search frame. For a target P frame the encoder searches
`in a prior I or P frame. For a target B frame, the encoder
`searches in a prior or subsequent I or P frame. When a match
`is found, the encoder transmits a vector movement code or
`motion vector. The vector movement code or motion vector
`only includes information on the difference betWeen the
`search frame and the respective target picture. The blocks in
`target pictures that have no change relative to the block in
`the reference picture or I frame are ignored. Thus the amount
`of data that is actually stored for these frames is signi?cantly
`reduced.
`After motion vectors have been generated, the encoder
`then encodes the changes using spatial redundancy. Thus,
`after ?nding the changes in location of the macroblocks, the
`MPEG algorithm further calculates and encodes the differ
`ence betWeen corresponding macroblocks. Encoding the
`difference is accomplished through a math process referred
`to as the discrete cosine transform or DCT. This process
`divides the macroblock into four sub blocks, seeking out
`changes in color and brightness. Human perception is more
`sensitive to brightness changes than color changes. Thus the
`MPEG algorithm devotes more effort to reducing color data
`than brightness.
`Therefore, MPEG compression is based on tWo types of
`redundancies in video sequences, these being spatial, Which
`is the redundancy in an individual frame, and temporal,
`Which is the redundancy betWeen consecutive frames. Spa
`tial compression is achieved by considering the frequency
`characteristics of a picture frame. Each frame is divided into
`non-overlapping blocks, and each block is transformed via
`the discrete cosine transform (DCT). After the transformed
`blocks are converted to the “DCT domain”, each entry in the
`transformed block is quantiZed With respect to a set of
`quantiZation tables. The quantiZation step for each entry can
`vary, taking into account the sensitivity of the human visual
`system (HVS)} to the frequency. Since the HVS is more
`sensitive to loW frequencies, most of the high frequency
`entries are quantiZed to Zero. In this step Where the entries
`are quantiZed, information is lost and errors are introduced
`to the reconstructed image. Run length encoding is used to
`transmit the quantiZed values. To further enhance
`compression, the blocks are scanned in a Zig-Zag ordering
`that scans the loWer frequency entries ?rst, and the non-Zero
`quantiZed values, along With the Zero run lengths, are
`entropy encoded.
`When an MPEG decoder receives an encoded stream, the
`MPEG decoder reverses the above operations. Thus the
`MPEG decoder performs inverse scanning to remove the Zig
`Zag ordering, inverse quantiZation to de-quantiZe the data,
`and the inverse DCT to convert the data from the frequency
`domain back to the pixel domain. The MPEG decoder also
`performs motion compensation using the transmitted motion
`vectors to recreate the temporally compressed frames.
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`4
`When frames are received Which are used as references
`for other frames, such as I or P frames, these frames are
`decoded and stored in memory. When a temporally com
`pressed or encoded frame is received, such as a P or B frame,
`motion compensation is performed on the frame using the
`prior decoded I or P reference frames. The temporally
`compressed or encoded frame, referred to as a target frame,
`Will include motion vectors Which reference blocks in prior
`decoded I or P frames stored in the memory. The MPEG
`decoder examines the motion vector, determines the respec
`tive reference block in the reference frame, and accesses the
`reference block pointed to by the motion vector from the
`memory.
`A typical MPEG decoder includes motion compensation
`logic Which includes local or on-chip memory. The MPEG
`decoder also includes an external memory Which stores prior
`decoded reference frames. The MPEG decoder accesses the
`reference frames or anchor frames stored in the external
`memory in order to reconstruct temporally compressed
`frames. The MPEG decoder also typically stores the frame
`being reconstructed in the external memory.
`An MPEG decoder system also typically includes trans
`port logic Which operates to demultiplex received data into
`a plurality of individual multimedia streams. An MPEG
`decoder system also generally includes a system controller
`Which controls operations in the system and executes pro
`grams or applets.
`Prior art MPEG video decoder systems have generally
`used a frame store memory for the MPEG decoder motion
`compensation logic Which stores the reference frames or
`anchor frames as Well as the frame being reconstructed.
`Prior art MPEG video decoder systems have also generally
`included a separate memory for the transport and system
`controller functions. It has generally not been possible to
`combine these memories, due to siZe limitations. For
`example, current memory devices are fabricated on an 4
`Mbit granularity. In prior art systems, the memory require
`ments for the transport and system controller functions as
`Well as the decoder motion compensation logic Would
`exceed 16 Mbits of memory, thus requiring 20 or 24 Mbits
`of memory. This additional memory adds considerable cost
`to the system.
`The amount of memory is a major cost item in the
`production of video decoders. Thus, it is desired to reduce
`the memory requirements of the decoder system as much as
`possible to reduce its siZe and cost. Since practical memory
`devices are implemented using particular convenient dis
`crete siZes, it is important to stay Within a particular siZe if
`possible for commercial reasons. For example, it is desired
`to keep the memory requirements beloW a particular siZe of
`memory, such as 16 Mb, since otherWise a memory device
`of 20 or 24 Mb Would have to be used, resulting in greater
`cost and extraneous storage area. As mentioned above, it has
`heretofore not been possible to combine the memory
`required for the transport and system controller functions
`With the memory required for the MPEG decoder logic due
`to the memory siZe requirements.
`Therefore, a neW video decoder system and method is
`desired Which ef?ciently uses memory and combines the
`memory subsystem for reduced memory requirements and
`hence reduced cost.
`
`SUMMARY OF THE INVENTION
`
`65
`
`The present invention comprises an MPEG decoder sys
`tem and method for performing video decoding or decom
`pression Which includes a uni?ed memory for multiple
`
`ASUS Exhibit 1001 - Page 19
`
`
`
`5,870,087
`
`5
`functions according to the present invention. The video
`decoding system includes transport logic, a system
`controller, and MPEG decoder logic. The video decoding
`system of the present invention includes a single uni?ed
`memory Which stores code and data for the transport logic,
`system controller and MPEG decoder functions. The single
`uni?ed memory is preferably a 16 Mbit memory. The
`present invention thus requires only a single memory, and
`thus has reduced memory requirements compared to prior
`art designs.
`The video decoding system includes transport logic Which
`operates to demultipleX received data into a plurality of
`individual multimedia streams. The video decoding system
`also includes a system controller Which controls operations
`in the system and eXecutes programs or applets. The video
`decoding system further includes decoding logic, preferably
`MPEG decoder logic, Which performs motion compensation
`betWeen temporally compressed frames of a video sequence
`during video decoding or video decompression. The
`memory includes a plurality of memory portions, including
`a video frame portion for storing video frames, a system
`controller portion for storing code and data executable by
`the system controller, and a transport buffer for storing data
`used by the transport logic. The MPEG decoder logic
`preferably includes a memory controller Which couples to
`the single uni?ed memory. Each of the transport logic,
`system controller, and MPEG decoder logic accesses the
`single uni?ed memory through the memory controller.
`The video decoding system implements various frame
`memory saving schemes, such as compression or dynamic
`allocation, to reduce the required amount of frame store
`memory. Also, in one embodiment, the memory is not
`required to store reconstructed frame data during motion
`compensation, thus considerably reducing the required
`amount of memory for this function. Alternatively, the
`memory is only required to store a portion of the recon
`structed frame data. These savings in memory alloW portions
`of the memory to also be used for transport and system
`controller functions.
`The present invention thus provides a video decoding
`system With reduced memory requirements.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`A better understanding of the present invention can be
`obtained When the folloWing detailed description of the
`preferred embodiment is considered in conjunction With the
`folloWing draWings, in Which:
`FIG. 1 illustrates a computer system Which performs
`video decoding and Which includes a motion compensation
`logic having a frame memory Which stores reference block
`data according to the present invention;
`FIG. 2 is a block diagram illustrating the computer system
`of FIG. 1;
`FIG. 3 is a block diagram illustrating an MPEG decoder
`system including a uni?ed memory for MPEG transport,
`system controller, and decode functions according to the
`present invention;
`FIG. 4 is a block diagram illustrating the MPEG decoder
`logic in the system of FIG. 3;
`FIG. 5 illustrates various frame memory saving schemes
`used in various embodiments of the invention;
`FIGS. 6a and 6b illustrate a table listing the memory
`partitions under different display schemes;
`FIG. 7 illustrates the relationship of memory bandWidth
`vs. memory siZe in the NTSC decoding scheme;
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`FIG. 8 illustrates the relationship of memory bandWidth
`vs. memory siZe in the PAL encoding scheme;
`FIG. 9 illustrates the memory partitions according to the
`preferred embodiment of the invention;
`FIG. 10 illustrates the estimated memory bandWidth dis
`tribution in the preferred embodiment of the invention;
`FIG. 11 illustrates the “Worst case” relationship of pro
`cessing poWer vs. memory siZe in the NTSC decoding
`scheme;
`FIG. 12 illustrates the clock domains in the system;
`FIG. 13 illustrates clock operating frequencies according
`to the preferred embodiment of the invention;
`FIG. 14 illustrates an eXample of the packet data interface
`betWeen the transport controller and the source decoder; and
`FIG. 15 illustrates packet header formats used in the
`preferred embodiment.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`Video Compression System
`Referring noW to FIG. 1, a system for performing video
`decoding or decompression and including a uni?ed memory
`according to the present invention is shoWn. The video
`decoding system of the present invention includes a single
`uni?ed memory Which stores code and data for the transport,
`system controller and MPEG decoder functions. This sim
`pli?es the design and reduces the memory requirements in
`the system.
`As shoWn, in one embodiment the video decoding or
`decompression system is comprised in a general purpose
`computer system 60. The video decoding system may com
`prise any of various types of systems, including a computer
`system, set-top boX, television, or other device.
`The computer system 60 is preferably coupled to a media
`storage unit 62 Which stores digital video ?les Which are to
`be decompressed or decoded by the computer system 60.
`The media storage unit 62 may also store the resultant
`decoded or decompressed video ?le. In the preferred
`embodiment, the computer system 60 receives a compressed
`video ?le or bitstream and generates a normal uncompressed
`digital video ?le. In the present disclosure, the term “com
`pressed video ?le” refers to a video ?le Which has been
`compressed according to any of various video compression
`algorithms Which use motion estimation techniques, includ
`ing the MPEG standard, among others, and the term
`“uncompressed digital video ?le” refers to a stream of
`decoded or uncompressed video.
`As shoWn, the computer system 60 preferably includes a
`video decoder 74 Which performs video decoding or decom
`pression operations. The video decoder 74 is preferably an
`MPEG decoder. The computer system 60 optionally may
`also include an MPEG encoder 76. The MPEG decoder 74
`and MPEG encoder 76 are preferably adapter cards coupled
`to a bus in the computer system, but are shoWn eXternal to
`the computer system 60 for illustrative purposes. The com
`puter system 60 also includes softWare, represented by
`?oppy disks 72, Which may perform portions of the video
`decompression or decoding operation and/or may perform
`other operations, as desired.
`The computer system 60 preferably includes various
`standard components, including one or more processors, one
`or more buses, a hard drive and memory. Referring noW to
`FIG. 2, a block diagram illustrating the components com
`prised in the computer system of FIG. 1 is shoWn. It is noted
`that FIG. 2 is illustrative only, and other computer architec
`tures may be used, as desired. As shoWn, the computer
`
`ASUS Exhibit 1001 - Page 20
`
`
`
`5,870,087
`
`7
`system includes at least one processor 80 coupled through
`chipset logic 82 to a system memory 84. The chipset 82
`preferably includes a PCI
`(Peripheral Component
`Interconnect) bridge for interfacing to PCI bus 86, or another
`type of bus bridge for interfacing to another type of expan-
`sion bus. In FIG. 2, MPEG decoder 74 and MPEG encoder
`76 are shown connected to PCI bus 86. Various other
`components may be comprised in the computer system, such
`as video 88 and hard drive 90.
`As mentioned above, in the preferred embodiment of FIG.
`1 the computer system 60 includes or is coupled to one or
`more digital storage or media storage devices. For example,
`in the embodiment of FIG. 1,
`the computer system 60
`couples to media storage unit 62 through cable 64. The
`media storage unit 62 preferably comprises a RAID
`(Redundant Array of Inexpensive Disks) disk array, or
`includes one or more CD-ROM drives and/or one or more
`Digital Video Disk (DVD) storage units, or other media, for
`storing digital video to be decompressed and/or for storing
`the resultant decoded video data. The computer system may
`also include one or more internal RAID arrays, CD-ROM
`drives and/or may couple to one or more separate Digital
`Video Disk (DVD) storage units. The computer system 60
`also may connect to other types of digital or analog storage
`devices or media, as desired.
`Alternatively, the compressed digital video file may be
`received from an external source, such as a remote storage
`device or remote computer system. In this embodiment, the
`computer system preferably includes an input device, such
`as an ATM (Asynchronous Transfer Mode) adapter card or
`an ISDN (Integrated Services Digital Network) terminal
`adapter, or other digital data receiver, for receiving the
`digital video file. The digital video file may also be stored or
`received in analog format and converted to digital data,
`either externally to the computer system 60 or within the
`computer system 60.
`As mentioned above, the MPEG decoder 74 in the com-
`puter system 60 performs video decoding or video decom-
`pression functions. As discussed further below, the video
`decoding system includes transport logic which operates to
`demultiplex received data into a plurality of individual
`multimedia streams. The video decoding system also
`includes a system controller which controls operations in the
`system and executes programs or applets comprised in the
`stream. The video decoding system further includes decod-
`ing logic, preferably MPEG decoder logic, which performs
`motion compensation between temporally compressed
`frames of a video sequence during video decoding or video
`decompression. The video decoding system of the present
`invention includes a single unified memory which stores
`code and data for the transport, system controller and MPEG
`decoder functions. This simplifies the design and reduces the
`memory requirements in the system. The MPEG decoder 74
`thus performs functions with improved efficiency and
`reduced memory requirements according to the present
`invention.
`
`It is noted that the system for decoding or decompressing
`video data may comprise two or more interconnected
`computers, as desired. The system for decoding or decom-
`pressing video data may also comprise other hardware, such
`as a set top box, either alone or used in conjunction with a
`general purpose programmable computer. It is noted that any
`of various types of systems may be used for decoding or
`decompressing video data according to the present
`invention, as desired.
`FIG. 3—MPEG Decoder Block Diagram
`Referring now to FIG. 3, a block diagram illustrating an
`MPEG decoder system architecture according to one
`
`8
`embodiment of the present invention is shown. As shown,
`the MPEG decoder system includes a channel receiver 202
`for receiving a coded stream. As mentioned above, in the
`preferred embodiment,
`the coded stream is an MPEG
`encoded stream. The MPEG encoded stream may include
`interactive program content comprised within this stream, as
`desired. The channel receiver 202 receives the coded stream