`Diaz et al.
`
`[19]
`
`[54] VIDEO AND/OR AUDIO DECOMPRESSION
`AND/OR COMPRESSION DEVICE THAT
`SHARES A MEMORY INTERFACE
`
`[75] Inventors: Raul Zegers Diaz, Palo Alto; Je?'erson
`Eugene Owen, Freemont, both of Calif.
`
`[73] Assignee: STMicroelectronics, Inc., Carrollton,
`TeX.
`
`[21] Appl. No.: 702,911
`[22]
`Filed:
`Aug. 26, 1996
`
`[51] Int. Cl.6 .................................................... .. G06F 17/00
`
`[52] US. Cl. ....................................................... .. 395/200.77
`
`[58] Field of Search ....................... .. 395/200.77, 200.82,
`395/507, 890, 729; 348/402, 407, 10
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`395/729
`9/1988 Conforti ..... ..
`4,774,660
`.. 395/729
`1/1990 Marquardt
`4,894,565
`348/10
`6/1991 Baji et al.
`5,027,400
`395/729
`5,371,893 12/1994 Price et a1.
`348/431
`5,459,519 10/1995 Scalise et al.
`395/729
`5,522,080
`5/1996 Harney et al.
`.. 348/402
`5,557,538
`9/1996 Retter etal. ..
`395/507
`5,598,525
`1/1997 Nally etal.
`395/200.82
`5,621,893
`4/1997 Joh ......... ..
`5,623,672
`4/1997 Poppat .................................. .. 395/729
`
`FOREIGN PATENT DOCUMENTS
`
`0 673 171 A2 9/1995 European Pat. Off. .
`
`OTHER PUBLICATIONS
`
`Bheda, H. and P. Srinivasan, “A High—Performance Cross—
`Platform MPEG Decoder,” Digital Video Compression on
`Personal Computers: Algorithms and Technologies. SPIE
`Proceedings, Feb. 7—8, 1994, V0. 2187, pp. 241—248.
`Bursky, D., “Highly Integrated Controller Eases MPEG—2
`Adoption,” Electronic Design, Aug. 21, 1995, vol. 43, No.
`17, pp. 141—142.
`
`US005812789A
`Patent Number:
`Date of Patent:
`
`[11]
`[45]
`
`5,812,789
`Sep. 22, 1998
`
`Galbi, D. et al., “An MPEG—1 Audio/Video Decoder With
`Run—Length Compressed Antialiased Video Overlays,”
`1995 IEEE International Solid—State Circuits Conference,
`pp. 286—287, 381.
`Maturi, G., “Single Chip MPEG Audio Decoder,” IEEE
`Transactions on Consumer Electronics, vol. 38, No. 3, Aug.
`1992, pp. 348—356.
`Butler, B. and T. Mace, “The Great Leap Forward,” PC
`Magazine, Oct. 11, 1994, pp. 241—244, 246, 248, 250,
`253—254, 256, 260—261, 264, 266—268, 273—275, 278.
`Doquilo, J., “Symmetric Multiprocessing Servers: Scaling
`the Performance Wall,” Infoworla', Mar. 27, 1995, pp.
`82—85, 88—92.
`Video Electronics Standards Association, “VESA Uni?ed
`Memory Architechture Hardware Speci?cations Proposal,”
`Version: 1.0p, Oct. 31, 1995, pp. 1—38.
`Video Electronics Standards Association, “VESA Uni?ed
`Memory Architecture VESA BIOS Extensions VUMA—SBE
`Proposal,” Version: 1.0p, Nov. 1, 1995, pp. 1—26.
`King, A., Inside Windows 95, Microsoft Press, Redmond,
`Washington, 1994, pp. 85—90.
`“MPEG Video Overview,” SGS—Thomson Microelectronics
`Technical Note, Apr. 1992, pp. 1—4.
`“On the Bus Arbitration for MPEG 2 Video Decoder” ; VLSI
`Tech, System and Application, 1995 Symposium.
`“A Low Cost Graphics and Multimedia Workstation Chip
`Set”; IEEE Micro, 1994.
`
`Primary Examiner—Ellis B. Ramirez
`Attorney, Agent, or Firm—David V. Carlson; Theodore E.
`Galanthay; Lisa K. Jorgenson
`
`[57]
`
`ABSTRACT
`
`An electronic system that contains a ?rst device that requires
`a memory interface and video and/or audio decompression
`and/or compression device that shares a memory interface
`and memory with the ?rst device while still permitting the
`video and/or audio decompression and/or compression
`device to operate in real time is disclosed.
`
`33 Claims, 5 Drawing Sheets
`
`REGISTER
`INTERFACE
`B
`
`44
`
`12
`
`VIDEO
`0EG00|NG
`C'RCW
`
`14
`
`AUDIO
`DECODING
`cIRcuII
`DECODER
`
`VIDEO
`ENco0ING
`um“
`
`‘W0
`ENG00ING
`CIRCUIT
`
`4
`
`REGISTER
`INTERFACE
`A)
`
`ENCODER
`
`DMA ENGINE
`
`52
`
`46
`L5
`
`FIRST DEVICE
`
`Q
`
`60
`
`BM ENGINE
`
`u so
`
`w m
`
`REFRESH LOGIC I ARBIIER
`
`MEMORY CONTROLLER
`
`MEMORV [NIERFACE
`J
`70
`
`50
`
`Apple Inc. v. Parthenon
`Ex. 1001 / Page 1 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 1
`
`
`
`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 1 of5
`
`5,812,789
`
`VIDEO DECODING
`H CIRCUIT H
`L2
`MEMORY
`, INTERFACE
`
`MICRO-
`CONTROLLER
`
`is
`
`AUDIO DECODING
`CIRCUIT
`
`I
`
`18
`—
`
`IO /'
`
`MEMORY
`22
`
`FIG. 1 a
`(PRIOR ART)
`
`REGISTER
`MICRO-
`CONTROLLER <:L> INTERFACE k
`
`vIDEO DECODINC
`<—>
`CIRCuIT
`l2
`
`25
`
`2_()
`
`AUDIO DECODINC
`CIRCuIT
`H
`
`<—
`
`10 /'
`
`MEMORY
`_2_2
`
`F I G. 7 b
`(PRIOR ART)
`
`<—>
`
`MEMORY
`INTERFACE
`
`3»
`
`Ex. 1001 / Page 2 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 2
`
`
`
`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 2 of 5
`
`5,812,789
`
`$3Maya
`
`xma9::
`
`
`
`3;©w_.
`
`of
`
`288ea
`
`\
`
`mm
`
`58:
`
`8.
`
`ma<2
`
`2;
`
`we
`
`.8Ga
`
`moozm52$
`
`2::
`
`E05:
`
`8.
`
`2:
`
`55No_.5
`
`Emi
`
`222Em2m?
`
`898
`
`33N?
`
`E.._.._:m ><#_mE
`
`
`§E>z8SEW$98
`92"Sam82>1:5
`
`8_In_§o
`
`mo5E8<
`
`9:
`
`8§Ez_
`
`E.
`
`E30528
`
`25S_
`
`mamEn.
`
`oi
`
`3%SE9;2Gfiw
`
`Eooao
`
`EOE:
`
`om
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 3
`
`Ex.1001IPage 3 of 13
`
`Ex. 1001 / Page 3 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 3
`
`
`
`
`VIDEO
`DECODING
`CIRCUIT
`
`AUDIO
`DECODING
`CIRCUIT
`
`VIDEO
`ENCODING
`CIRCUIT
`
`AUDIO
`ENCODING
`CIRCUIT
`
`FIRST DEVICE
`
`FIG. 2
`
`DMA ENGINE
`
`
`
`SJ0‘E1991188661‘ZZ'd9S1II9Jl3(['S'f1
`
`
`
`
`
`68L‘ZI8‘S
`
`Ex. 1001 / Page 4 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 4
`
`
`
`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 4 of 5
`
`5,812,789
`
`2:
`
`o_§<
`
`H.500
`
`5%wt
`
`oi
`
`.3:mm:
`
`Es:
`
`
`
`flagEmma
`
`m
`
`95§Ez_:
`
`_
`
`2,:%:<882>2:5
`
`@5928mesa~§8
`
`~§<$:8<
`8_In_§omm:
`
`ESEZS
`
`25N:
`
`m.65..»
`
`mamUm
`
`E_
`
`X:
`
`xma92:
`
`mama
`
`asQ
`
`
`
`$82:$805
`
`_on_
`
`
`
` fiaomlwvlv.Q53M28
`
`
`
`65%;.$05:
`
`mfl
`
`M3at
`
`/42::
`
`ofE03:mm:
`
`of
`
`EHE
`
`m__._o<oS
`
`we
`
`N?
`
`3%
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 5
`
`Ex.1001IPage 5 of 13
`
`Ex. 1001 / Page 5 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 5
`
`
`
`
`
`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 5 of 5
`
`5,812,789
`
`E
`
`9:
`
`oi
`
`n_2<
`
`E_
`
`we
`
`6§Ez_E05:
`
`9:
`
`Es:
`
`~m_.._.._3m_
`
`O2
`
`898mom3052T23
`$300.:$9098mo%mE8<E
`
`
`H
`
`.%
`
`~_o§:d8<R
`
`me
`
`20m8as
`
`mom<2
`
`we
`
`N9
`
`_on_
`
`52%
`
`2:
`
`2
`
`8;222R:
`
`58:5Eofizwe
`
`
`
`.1:H._>_m_o
`
`xma92:
`
`am
`
`$2
`
`“V.Me~R~
`
`§$Ez_ma.3}:
`
`mamGa
`
`¢oN
`
`mom_6§m:z_
`
`E
`
`comNm_
`
`25
`
`oi
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 6
`
`Ex.1001IPage 6 of 13
`
`Ex. 1001 / Page 6 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 6
`
`
`
`
`
`1
`VIDEO AND/OR AUDIO DECOMPRESSION
`AND/OR COMPRESSION DEVICE THAT
`SHARES A MEMORY INTERFACE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application contains some text and drawings in
`common With pending US. patent applications entitled:
`“Video and/or Audio Decompression and/or Compression
`Device that Shares a Memory” by Jefferson E. OWen, Raul
`Z. DiaZ, and Osvaldo Colavin Ser. No. 08/702,910 ?led on
`Aug. 26, 1996, and has the same effective ?ling date and
`oWnership as the present application, and to that extent is
`related to the present application, Which is incorporated
`herein by reference.
`
`BACKGROUND
`
`The present invention relates to the ?eld of electronic
`systems having a video and/or audio decompression and/or
`compression device, and is more speci?cally directed to
`sharing a memory interface betWeen a video and/or audio
`decompression and/or compression device and another
`device contained in the electronic system.
`The siZe of a digital representation of uncompressed video
`images is dependent on the resolution, and color depth of the
`image. Amovie composed of a sequence of such images, and
`the audio signals that go along With them, quickly becomes
`large enough so that uncompressed such a movie typically
`cannot ?t entirely onto conventional recording medium,
`such as a CD. It is also typically noW prohibitively expen
`sive to transmit such a movie uncompressed.
`It is therefore advantageous to compress video and audio
`sequences before they are transmitted or stored. A great deal
`of effort is being expanded to develop systems to compress
`these sequences. There are several coding standards cur
`rently used that are based on the discrete cosine transfer
`algorithm including MPEG-1, MPEG-2, H.261, and H.263.
`(MPEG stands for “Motion Picture Expert Group”, a com
`mittee of the International OrganiZation for Standardization,
`ISO.) The MPEG-1, MPEG-2, H.261, and H.263 standards
`are decompression protocols that describe hoW an encoded
`bitstream is to be decoded. The encoding can be done in any
`manner, as long as the resulting bitstream complies With the
`standard.
`Video and/or audio compression devices (hereinafter
`encoders) are used to encode the video and/or audio
`sequence before it is transmitted or stored. The resulting
`bitstream is decoded by a video and/or audio decompression
`device (hereinafter decoder) before the video and/or audio
`sequence is displayed. HoWever, a bitstream can only be
`decoded by a decoder if it complies to the standard used by
`the decoder. To be able to decode the bitstream on a large
`number of systems it is advantageous to encode the video
`and/or audio sequences to comply to a Well accepted decom
`pression standard. The MPEG standards are currently Well
`accepted standards for one Way communication. H.261, and
`H.263 are currently Well accepted standards for video tele
`phony.
`Once decoded the images can be displayed on an elec
`tronic system dedicated to displaying video and audio, such
`as television or digital video disk (DVD) player, or on
`electronic systems Where image display is just one feature of
`the system, such as a computer. A decoder needs to be added
`to these systems to alloW them to display compressed
`sequences, such as received images and associated audio, or
`ones taken from a storage device. An encoder needs to be
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,812,789
`
`2
`added to alloW the system to compress video and/or audio
`sequences, to be transmitted or stored. Both need to be added
`for tWo Way communication such as video telephony.
`A typical decoder, such as an MPEG decoder 10 shoWn in
`FIG. 1a, contains video decoding circuitry 12, audio decod
`ing circuitry 14, a microcontroller 16, and a memory inter
`face 18. The decoder can also contain other circuitry
`depending on the electronic system the decoder is designed
`to operate in. For example, When the decoder is designed to
`operate in a typical television the decoder Will also contain
`an on screen display (OSD) circuit.
`FIG. 1b shoWs a better decoder architecture, used in the
`STi3520 and STi3520A MPEG Audio/MPEG-2 Video Inte
`grated Decoder manufactured by SGS-THOMSON Micro
`electronics. The decoder has a register interface 20 instead
`of a microcontroller. The register interface 20 is coupled to
`an external microcontroller 24. The use of a register inter
`face 20 makes it possible to tailor the decoder 10 to the
`speci?c hardWare the decoder 10 interfaces With or change
`its operation Without having to replace the decoder by just
`reprogramming the register interface. It also alloWs the user
`to replace the microcontroller 24, to upgrade or tailor the
`microcontroller 24 to a speci?c use, by just replacing the
`microcontroller and reprogramming the register interface
`20, Without having to replace the decoder 10.
`The memory interface 18 is coupled to a memory 22. A
`typical MPEG decoder 10 requires 16 Mbits of memory to
`operate in the main pro?le at main level mode (MP at ML).
`This typically means that the decoder requires a 2 Mbyte
`memory. Memory 22 is dedicated to the MPEG decoder 10
`and increases the price of adding a decoder 10 to the
`electronic system. In current technology the cost of this
`additional dedicated memory 22 can be a signi?cant per
`centage of the cost of the decoder.
`An encoder also requires a memory interface 18 and
`dedicated memory. Adding the encoder to an electronic
`system again increases the price of the system by both the
`price of the encoder and its dedicated memory.
`A goal in the semiconductor industry is to reduce the die
`area of an integrated circuit device for a given functionality.
`Some advantages of reducing the die area is the increase in
`the number of the die that can be manufactured on same siZe
`silicon Wafer, and the reduction in price per die resulting
`therefrom. This results in both an increase in volume and
`reduction in price of the device.
`Many of the functional circuits described above for FIG.
`1a and FIG. 1b take up a lot of die space. HoWever, each of
`them is needed to make the respective decoder operate.
`FIG. 1c shoWs a computer 25 containing a decoder 10, a
`main memory 168 and other typical components such as a
`modem 199, and graphics accelerator 188. The decoder 10
`and the rest of the components are coupled to the core logic
`chipset 190 through a bus 170. The bus is typically a PCI
`(peripheral component interface) or ISA (industry standard
`architecture) bus, and each component contains an appro
`priate interface for interfacing With the bus.
`When any component needs access to the memory 168
`either to read from or Write to the main memory 168, it
`generates a request Which is placed on the bus 26. When the
`request is a Write the data to be Written is also placed on the
`bus 26. The request is processed in the core logic chipset 190
`and the data is then either Written to or read from the main
`memory 168. When data is read from the main memory 168
`the data is noW placed on the bus and goes to the component
`that requested the read.
`There are typically many components in the computer
`systems that may require access to the main memory 168,
`
`Ex. 1001 / Page 7 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 7
`
`
`
`3
`and they are typically all coupled to the same bus 174, or
`possibly several buses 170, 198 connected together by a PCI
`bridge 192, if there are not enough connectors on one bus to
`accommodate all of the peripherals. HoWever, the addition
`of each bus is very expensive. Each request is typically
`processed according to a priority scheme. The priority
`scheme is typically based on the priority given to the device
`and the order in Which the requests are received. Typically,
`the priority scheme is set up so no device monopoliZes the
`bus, starving all of the other devices. Good practice suggests
`that no device on the bus require more than approximately
`50% of the bus’s bandWidth.
`The minimum bandWidth required for the decoder 10 can
`be calculated based on the characteristics and desired opera
`tion of the decoder. These characteristics include the stan
`dard to Which the bitstream is encoded to comply With,
`Whether the decoder is to operate in real time, to What eXtent
`frames are dropped, and hoW the images are stored.
`Additionally, the latency of the bus that couples the decoder
`to the memory should be considered.
`If the decoder does not operate in real time the decoded
`movie Would stop periodically betWeen images until the
`decoder can get access to the memory to process the neXt
`image. The movie may stop quite often betWeen images and
`Wait.
`To reduce the minimum required bandWidth and still
`operate in real time, the decoder 10 may need to drop
`frames. If the decoder 10 regularly does not decode every
`frame then it may not need to stop betWeen images.
`HoWever, this produces very poor continuity in the images.
`This is problematic With an image encoded to the MPEG-1
`or MPEG-2 standard, or any standards that uses temporal
`compression. In temporal (interpicture) compression some
`of the images are decoded based on previous images and
`some based on previous and future images. Dropping an
`image on Which the decoding of other images is based is
`unacceptable and Will result in many poor or even com
`pletely unrecogniZable images.
`The computer can also contain both a decoder and
`encoder to alloW for video telephony, as described above. In
`this case not operating in real time Would mean that the
`length of time betWeen the occurrence of an event, such as
`speaking, at one end of the conversation until the event is
`displayed at the other end of the conversation is increased by
`the time both the encoder and then the decoder must Wait to
`get access to the bus and the main memory. Not being able
`to operate in real time means that there Would be gaps in the
`conversation until the equipment can catch up. This
`increases the time needed to have a video conference, and
`makes the conference uncomfortable for the participants.
`One Widely used solution to alloW a component in a
`computer system to operate in real time is to give the
`component its oWn dedicated memory. Thus, as shoWn in
`FIG. 1c, the decoder 10 can be given its oWn dedicated
`memory 22, With a dedicated bus 26 to connect the decoder
`10 to its memory 22. The dedicated memory 22, its con
`troller and the pins to control this memory signi?cantly
`increase the cost of adding a decoder 10 to the computer.
`
`15
`
`25
`
`35
`
`45
`
`55
`
`SUMMARY OF THE INVENTION
`The present application discloses an electronic system
`that contains a ?rst device and video and/or audio decom
`pression and/or compression device capable of operating in
`real time. Both the ?rst device and the video and/or audio
`decompression and/or compression device require a
`memory interface. The video and/or audio decompression
`
`65
`
`5,812,789
`
`4
`and/or compression device shares a memory interface and
`the memory With the ?rst device. In the preferred embodi
`ment of the invention the shared memory interface contains
`an arbiter. The arbiter and DMA engines of the video and/or
`audio decompression and/or compression device and of the
`?rst device are con?gured to arbitrate betWeen the tWo
`devices When one of them is requesting access to the
`memory. This alloWs the use of one memory interface to
`control the access of both the video and/or audio decom
`pression and/or compression device and the ?rst device to
`the memory.
`When the video and/or audio decompression and/or com
`pression device used in an electronic system, such as a
`computer, already containing a device that has a memory
`interface the video and/or audio decompression and/or com
`pression device can share that memory interface and the
`memory of the device and the memory interface and
`memory of the video and/or audio decompression and/or
`compression device can be eliminated. Eliminating this
`memory interface reduces the die area Without changing the
`critical dimensions of the device. Therefore increasing the
`volume and reducing the cost of the decoder or encoder.
`Eliminating the memory greatly reduces the cost of adding
`the video and/or audio decompression and/or compression
`device to the electronic system While not requiring the video
`and/or audio decompression and/or compression device to
`be connected to the system bus, alloWing the video and/or
`audio decompression and/or compression device to operate
`in real time.
`An advantage of the present invention is signi?cant cost
`reduction due to the fact that the video and/or audio decom
`pression and/or compression device does not need its oWn
`dedicated memory but can share a memory With another
`device and still operate in real time.
`Another signi?cant advantage of the present invention is
`that the die space needed for the video and/or audio decom
`pression and/or compression device is smaller because the
`memory interface on the video and/or audio decompression
`and/or compression device is eliminated.
`A further advantage of the present invention is that the
`video and/or audio decompression and/or compression
`device can share the memory of the device With Which it is
`sharing the memory interface more ef?ciently.
`Another advantage of the present invention is that the cost
`of producing a video and/or audio decompression and/or
`compression device is reduced because the memory inter
`face on the video and/or audio decompression and/or com
`pression device is eliminated.
`Another advantage of the present invention is that the
`video and/or audio decompression and/or compression
`device can be monolithically integrated into the ?rst device
`and no eXtra packaging or pins are needed for the video
`and/or audio decompression and/or compression device, and
`no pins are needed for the ?rst device to connect to the video
`and/or audio decompression and/or compression device,
`saving pins on both devices and producing a better connec
`tion betWeen the tWo devices.
`Other advantages and objects of the invention Will be
`apparent to those of ordinary skill in the art having reference
`to the folloWing speci?cation together With the draWings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIGS. 1a and 1b are electrical diagrams, in block form, of
`prior art decoders.
`FIG. 1c is an electrical diagram, in block form, of a
`computer system containing a decoder according to the prior
`art.
`
`Ex. 1001 / Page 8 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 8
`
`
`
`5,812,789
`
`5
`FIG. 2 is an electrical diagram, in block form, of an
`electronic system containing a device having a memory
`interface and an encoder and decoder.
`FIG. 3 is an electrical diagram, in block form, of a
`computer system containing a core logic chipset designed
`for the CPU to share a memory interface With an encoder
`and decoder.
`FIG. 4 is an electrical diagram, in block form, of a
`computer system containing a graphics accelerator designed
`to share a memory interface With an encoder and/or decoder.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`
`FIG. 2 shoWs an electronic system 40 containing a ?rst
`device 42 having access to a memory 50 through a memory
`interface 48, and a decoder 44 and encoder 46, having access
`to the same memory 50 through the same memory interface
`48. First device 42 can be a processor, a core logic chipset,
`a graphics accelerator, or any other device that requires
`access to the memory 50, and either contains or is coupled
`to a memory interface. Any parts common to FIGS. 1
`through 4 are indicated using the same numbering system. In
`the preferred embodiment of the invention, electronic sys
`tem 40 contains a ?rst device 42, a decoder 44, an encoder
`46, a memory interface 48, and a memory 50. Although,
`either the decoder 44 or encoder 46 can be used in the
`decoder/encoder 45 Without the other. For ease of reference,
`a video and/or audio decompression and/or compression
`device 45 Will hereinafter be referred to as decoder/encoder
`45. The decoder/encoder 45 may be a single device, or cell
`on an integrated circuit, or may be tWo separate devices, or
`cells in an integrated circuit. In the preferred embodiment of
`the invention, the ?rst device 42, decoder/encoder 45, and
`memory interface 48 are on one integrated circuit, hoWever,
`they can be on separate integrated circuits in any combina
`tion.
`The decoder 44 includes a video decoding circuit 12 and
`an audio decoding circuit 14, both coupled to a register
`interface 20. The decoder 44 can be either a video and audio
`decoder, just a video, or just an audio decoder. If the decoder
`44 is just a video decoder it does not contain the audio
`decoding circuitry 14. The audio decoding can be performed
`by a separate audio codec coupled to the ?rst device 42, or
`through softWare. In the preferred embodiment of the
`invention, When the decoder/encoder 45 is in a system
`containing a processor and is coupled to the processor, the
`audio decoding is performed in softWare. This frees up space
`on the die Without causing signi?cant delay in the decoding.
`If the audio decoding is performed in softWare, the processor
`should preferably operate at a speed to alloW the audio
`decoding to be performed in real time Without starving other
`components of the system that may need to utiliZe the
`processor. For eXample, currently softWare to perform AC-3
`audio decoding takes up approximately 40% of the band
`Width of a 133 MHZ Pentium. The encoder 46 includes a
`video encoding circuit 62 and an audio encoding circuit 64,
`both coupled to a register interface 20. The encoder 46 can
`be either a video and audio encoder, just a video, or just an
`audio encoder. If the encoder 46 is just a video encoder, it
`does not contain the audio encoding circuitry 64. The audio
`encoding can be performed by a separate audio codec
`coupled to the ?rst device 42, or through softWare. In the
`preferred embodiment of the invention, When the decoder/
`encoder 45 is in a system containing a processor and is
`coupled to the processor, the audio encoding is performed in
`softWare presenting the same advantages of freeing up space
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`on the die Without causing signi?cant delay in the encoding.
`The register interfaces 20 of the decoder 44 and encoder 46
`are coupled to a processor.
`The decoder 44 and encoder 46 are coupled to the direct
`memory access (DMA) engine 52. The decoder and encoder
`can be coupled to the same DMA engine as shoWn in FIG.
`2, or each can have its oWn DMA engine, or share a DMA
`engine With another device. When the decoder/encoder 45
`are tWo separate devices or cells, decoder 44 and encoder 46
`can still be coupled to one DMA engine 52. When the
`decoder/encoder is one device or is one cell on an integrated
`circuit, the DMA engine 52 can be part of the decoder/
`encoder 45, as shoWn in FIG. 2. The DMA engine 52 is
`coupled to the arbiter 54 of the memory interface 48.
`The ?rst device 42 also contains a DMA engine 60. The
`DMA engine 60 of the ?rst device 42 is coupled to the arbiter
`54 of the memory interface 48. The arbiter is also coupled
`to the refresh logic 58 and the memory controller 56. The
`memory interface 48 is coupled to a memory 50. The
`memory controller 56 is the control logic that generates the
`address the memory interface 48 accesses in the memory 50
`and the timing of the burst cycles.
`In current technology, memory 50 is typically a DRAM.
`HoWever, other types of memory can be used. The refresh
`logic 58 is needed to refresh the DRAM. HoWever, as is
`knoWn in the art, if a different memory is used, the refresh
`logic 58 may not be needed and can be eliminated.
`The decoder/encoder 45 is coupled to the memory 50
`through devices, typically a bus 70, that have a bandWidth
`greater than the bandWidth required for the decoder/encoder
`45 to operate in real time. The minimum bandWidth required
`for the decoder/encoder 45 can be calculated based on the
`characteristics and desired operation of the decoder, includ
`ing the standard to Which the bitstream is encoded to comply
`With, Whether the decoder/encoder 45 is to operate in real
`time, to What eXtent frames are dropped, and Which images
`are stored. Additionally, the latency of the bus 70 that
`couples the decoder/encoder 45 to the memory 50 should be
`considered.
`A goal is to have the decoder/encoder 45 operate in real
`time Without dropping so many frames that it becomes
`noticeable to the human vieWer of the movie. To operate in
`real time the decoder/encoder 45 should decoder and/or
`encode images fast enough so that any delay in decoding
`and/or encoding cannot be detected by a human vieWer. This
`means that the decoder/encoder 45 has a required bandWidth
`that alloWs the decoder/encoder 45 to operate fast enough to
`decode the entire image in the time betWeen screen
`refreshes, Which is typically 1/30 of a second, With the human
`vieWer not being able to detect any delay in the decoding
`and/or encoding. To operate in real time the required band
`Width should be loWer than the bandWidth of the bus. In
`order not to starve the other components on the bus, i.e. deny
`these components access to the memory for an amount of
`time that Would interfere With their operation, this required
`bandWidth should be less the entire bandWidth of the bus.
`Therefore a fast bus 70 should be used. A fast bus 70 is any
`bus Whose bandWidth is equal to or greater that the required
`bandWidth. There are busses in current technology, includ
`ing the ISA bus, Whose bandWidth is signi?cantly beloW the
`bandWidth required for this.
`In the preferred embodiment of the invention the decoder/
`encoder 45 is coupled to the memory 50 through a fast bus
`70 that has a bandWidth of at least the bandWidth required
`for the decoder/encoder 45 to operate in real time, a thresh
`old bandWidth. Preferably the fast bus 70 has a bandWidth of
`
`Ex. 1001 / Page 9 of 13
`
`Petitioners HTC Corp. & HTC America, Inc. - Ex. 1001, p. 9
`
`
`
`5,812,789
`
`7
`at least approximately twice the bandwidth required for the
`decoder/encoder 45 to operate in real time. In the preferred
`embodiment the fast bus 70 is a memory bus, hoWever any
`bus having the required bandWidth can be used.
`The decoder/encoder 45 only requires access to the
`memory during operation. Therefore, When there is no need
`to decode or encode, the ?rst device 42, and any other
`devices sharing the memory 50 have exclusive access to the
`memory and can use the entire bandWidth of the fast bus 70.
`In the preferred embodiment, even during decoding and
`encoding the decoder/encoder 45 does not alWays use the
`entire required bandWidth. Since the fast bus 70 has a
`bandWidth a little less than tWice the required bandWidth the
`decoder/encoder 45 uses at most 60% of the bandWidth of
`the fast bus 70.
`The required bandWidth is determined based on the siZe
`and resolution of the image, and the type of frame (I, P, or
`B). In the preferred embodiment the decoder/encoder typi
`cally Will be using less than 40% of the bandWidth of the fast
`bus 70. This frees up the remaining bandWidth to be used by
`the other devices the decoder/encoder 45 is sharing the
`memory 50 With.
`The decoder/encoder 45 can decode a bitstream formatted
`according to any one or a combination of standards. In the
`preferred embodiment of the invention the decoder/encoder
`45 is a multi-standard decoder/encoder capable of decoding
`and encoding sequences formatted to comply to several Well
`accepted standards. This alloWs the decoder/encoder 45 to
`be able to decode a large number of video and/or audio
`sequences. The choice of Which standards the decoder/
`encoder 45 is capable of decoding bitstreams formatted to
`and of encoding sequences to comply to is based on the
`desired cost, ef?ciency, and application of the decoder/
`encoder 45.
`In the preferred embodiment, these standards are capable
`of both intrapicture compression and interpicture compres
`sion. In intrapicture compression the redundancy Within the
`image is eliminated. In interpicture compression the redun
`dancy betWeen tWo images are eliminated and only the
`difference information is transferred. This requires the
`decoder/encoder 45 to have access to the previous or future
`image that contains information needed to decode or encode
`the current image. These precious and/or future images need
`to be stored then used to decode the current image. This is
`one of the reasons the decoder/encoder 45 requires access to
`the memory, and requires a large bandWidth. The MPEG-1
`and MPEG-2 standards alloW for decoding based on both
`previous images and/or future images. Therefore for a
`decoder/encoder 45 capable of operating in real time to be
`able to comply With the MPEG-1 and MPEG-2 standards it
`should be able to access tWo images, a previous and a future
`image, fast enough to decode the current image in the 1/30 of
`a second betWeen screen refreshes.
`An MPEG environment is asymmetrical; there are much
`feWer encoders than decoders. The encoders are very dif?
`cult and expensive to manufacture and the decoders are
`comparatively easy and cheap. This encourages many more
`decoders than encoders, With the encoders in centraliZed
`locations, and decoders available such that every end user
`can have a decoder. Therefore, there are many receivers but
`feW transmitters.
`For video telephony and teleconferencing each end user
`has to be able to both receive and transmit. H.261, and H.263
`are currently Well accepted standards for video telephony.
`An encoder that can encode sequences to comply to the
`H.261 and H.263 standards is less complicated, having a
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`8
`loWer resolution and loWer frame rate than an encoder that
`complies to the MPEG-1 or MPEG-2 standards, possibly
`making the quality of the decoded images someWhat loWer
`than those from an encoder that complies With the MPEG-1
`or MPEG-2 standards. Such an encoder, since it should be
`inexpensive and operate in real time, is also less ef?cient
`than an encoder to encode sequences to comply to the
`MPEG-1 or MPEG-2 standards. This means that the com
`pression factor, Which is the ratio betWeen the source data
`rate and the encoded bitstream data rate, of such an encoder
`is loWer for a given image quality than the compression
`factor of an MPEG encoder. HoWever, because such an
`encoder is less com