United States Patent
`Diaz et al.
`
`19
`
`54) VIDEO AND/OR AUDIO DECOMPRESSION
`AND/OR COMPRESSION DEVICE THAT
`SHARES A MEMORY INTERFACE
`
`75 Inventors: Raul Zegers Diaz, Palo Alto; Jefferson
`Eugene Owen, Freemont, both of Calif.
`
`73 Assignee: STMicroelectronics, Inc., Carrollton,
`TeX.
`
`21 Appl. No.: 702,911
`22 Filed:
`Aug. 26, 1996
`(51) Int. Cl. ................................................ G06F 17/00
`52 U.S. Cl. ......................................................... 395/200.77
`58 Field of Search ......................... 395/200.77, 200.82,
`395/507, 890, 729; 348/402,407, 10
`
`56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`... 395/729
`4,774,660 9/1988 Conforti ...
`... 395/729
`4,894,565
`1/1990 Marquardt.
`5,027,400 6/1991 Baji et al. ................................. 34.8/10
`5,371,893 12/1994 Price et al.....
`... 395/729
`5,459,519 10/1995 Scalise et al. ...
`... 348/431
`5,522,080 5/1996 Harney et al. ...
`... 395/729
`5,557,538 9/1996 Retter et al. ......
`... 348/402
`5,598.525
`1/1997 Nally et al. ...
`... 395/507
`5,621,893 4/1997 Joh ............
`395/200.82
`5,623,672 4/1997 Poppat .................................... 395/729
`
`FOREIGN PATENT DOCUMENTS
`O 673 171 A2 9/1995 European Pat. Off..
`
`OTHER PUBLICATIONS
`
`Bheda, H. and P. Srinivasan, “A High-Performance Cross
`Platform MPEG Decoder.” Digital Video Compression on
`Personal Computers. Algorithms and Technologies. SPIE
`Proceedings, Feb. 7-8, 1994, Vo. 2187, pp. 241-248.
`Bursky, D., “Highly Integrated Controller Eases MPEG-2
`Adoption,” Electronic Design, Aug. 21, 1995, vol. 43, No.
`17, pp. 141-142.
`
`USOO5812789A
`Patent Number:
`11
`(45) Date of Patent:
`
`5,812,789
`Sep. 22, 1998
`
`Galbi, D. et al., “An MPEG-1 Audio/Video Decoder With
`Run-Length Compressed Antialiased Video Overlays.”
`1995 IEEE International Solid-State Circuits Conference,
`pp. 286-287, 381.
`Maturi, G., “Single Chip MPEG Audio Decoder," IEEE
`Transactions on Consumer Electronics, Vol. 38, No. 3, Aug.
`1992, pp. 348-356.
`Butler, B. and T. Mace, “The Great Leap Forward," PC
`Magazine, Oct. 11, 1994, pp. 241-244, 246, 248, 250,
`253–254, 256, 260-261, 264, 266-268, 273-275, 278.
`Doquilo, J., “Symmetric Multiprocessing Servers: Scaling
`the Performance Wall.” Infoworld, Mar. 27, 1995, pp.
`82-85, 88-92.
`Video Electronics Standards Association, “VESA Unified
`Memory Architechture Hardware Specifications Proposal.”
`Version: 10p, Oct. 31, 1995, pp. 1-38.
`Video Electronics Standards Association, “VESA Unified
`Memory Architecture VESABIOS Extensions VUMA-SBE
`Proposal.” Version: 1.0p, Nov. 1, 1995, pp. 1-26.
`King, A., Inside Windows 95, Microsoft Press, Redmond,
`Washington, 1994, pp. 85-90.
`“MPEG Video Overview,” SGS-Thomson Microelectronics
`Technical Note, Apr. 1992, pp. 1-4.
`“On the Bus Arbitration for MPEG 2 Video Decoder”; VLSI
`Tech, System and Application, 1995 Symposium.
`“A Low Cost Graphics and Multimedia Workstation Chip
`Set'; IEEE Micro, 1994.
`
`Primary Examiner Ellis B. Ramirez
`Attorney, Agent, or Firm-David V. Carlson; Theodore E.
`Galanthay, Lisa K. Jorgenson
`57
`ABSTRACT
`An electronic System that contains a first device that requires
`a memory interface and Video and/or audio decompression
`and/or compression device that shares a memory interface
`and memory with the first device while still permitting the
`Video and/or audio decompression and/or compression
`device to operate in real time is disclosed.
`
`33 Claims, 5 Drawing Sheets
`
`
`
`58
`
`56
`
`FRST DEVCE
`42
`
`80
`
`DMA ENGINE
`
`
`
`
`
`REGISTER
`NTERFACE 14
`20
`
`WIOE)
`DECODNC
`CIRCUIT
`
`AUDO
`DECODING
`CRCUIT
`DECOER
`
`
`
`
`
`
`
`
`
`WDEO
`ENCODNG
`CRCUIT
`
`AUDIO
`ENCODNC
`CIRCUIT
`
`REGISEER
`64 INTERFACE
`20
`
`ENCODER
`
`REFRESH LOGIC
`
`ARBITER
`
`54
`
`MEMORY CONTROLLER
`
`EORY NERFACE
`
`IPR2018-01413
`Sony EX1023 Page 1
`
`

`

`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 1 of 5
`
`5,812,789
`
`
`
`
`
`
`
`
`
`
`
`
`
`MICRO
`CONTROLLER
`
`16
`
`VIDEO DECODING
`CIRCUIT
`12
`
`. AUDIO DECODNG
`CIRCUIT
`14
`
`MEMORY
`INTERFACE
`
`
`
`
`
`18
`
`10/
`
`
`
`
`
`FIC. 1 O.
`(PRIOR ART)
`
`VIDEO DECODING
`CIRCUIT
`12
`
`AUDIO DECODING
`CIRCUIT
`14
`
`
`
`
`
`MEMORY
`INTERFACE
`
`18
`
`FIC. 1 b
`(PRIOR ART)
`
`IPR2018-01413
`Sony EX1023 Page 2
`
`

`

`US. Patent
`
`2
`
`8
`
`9m
`
`5,mm::88ea
`
`2,mm1x
`
`%Emoozmm—
`
`9mamE1,o:n.om:m@558
`aim”2:8$4828592”Sam82>1:525N:
`
`
`55880195.56%mm:
`052859%
`
`tmam<202E083mw?
`5N282%Egg5532EE222
`
`mm:
`
`
`
`m:BEE?
`
`w:m:Ea?WfimflmEmma
`
`.vw—Nwe
`
`E
`
`x:H.535
`
`
`
`xma$2:mewmm:mm5255mm:Ma
`
`o—
`
`mm
`
`:i«can:uNGEN
`
`|PR2018—O1413
`
`Sony EX1023 Page 3
`
`IPR2018-01413
`Sony EX1023 Page 3
`
`
`

`

`U.S. Patent
`
`Sep. 22, 1998
`
`Sheet 3 of 5
`
`5,812,789
`
`
`
`}}3000NE
`
`}}300030
`
`EN 10NE W WO
`
`|-
`
`ENIONE WWC
`
`2 (f)I, H.
`
`IPR2018-01413
`Sony EX1023 Page 4
`
`

`

`U.S. Patent
`
`009912,2neS
`
`of
`
`053
`
`H.500
`
`m:
`
`o:
`
`II
`
`Siam
`
`wm_mm:
`
`
`
`mm:onmEE
`
`:—
`
`2,:~3982>:53
`
`35:588%«38
`
`8E5
`
`
`
`“05562mm:
`
`mmjoEzoo
`
`25mm
`
`F
`
`m65%
`
`mam_on_
`
`o:
`
`5f04tw
`
`%X:
`
`of
`
`meom<z
`
`mama
`
`5,812,789
`
`20m8o>o
`
`of
`
`EVR:
`
`of$03:mm:J2::
`
`
`
`BEE.:55:
`
`
`
`mewl|53555%2i203$8822Q$82:$983E
`
`aa
`
`NM3
`
`N._
`
`H.=._o<o
`
`mm—
`
`of
`
`m?
`
`3%
`
`|PR2018—O1413
`
`Sony EX1023 Page 5
`
`IPR2018-01413
`Sony EX1023 Page 5
`
`
`
`
`
`

`

`US. Patent
`
`Sep. 22, 1998
`
`Sheet 5 0f 5
`
`5,812,789
`
`m:
`
`E
`
`858
`
`052n|o<
`
`Q
`
`09
`
`m3.gI
`
`ww
`
`mm—
`
`3:8
`
`
`
`mofimflzEOE:
`
`.3:mm:
`
`mofimzzmam15>:
`
`mam5a
`
`_.
`E
`
`./
`
`com
`
`<
`
`0awow
`
`mom
`
`25N
`
`C
`
`o:
`
`C
`
`of
`
`20m8o>o
`
`mam<2
`
`m?
`
`N?
`
`_on_
`
`59mm
`
`_on_
`
`Emnzzo
`
`2?:
`
`E052mo—
`
`o?
`
`N._
`
`I:mama
`
`meom<_._
`
`mo:
`
`ww—
`
`NVONE
`
`|PR2018—O1413
`
`Sony EX1023 Page 6
`
`IPR2018-01413
`Sony EX1023 Page 6
`
`
`

`

`5,812,789
`
`1
`VIDEO AND/OR AUDIO DECOMPRESSION
`AND/OR COMPRESSION DEVICE THAT
`SHARES A MEMORY INTERFACE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`This application contains Some text and drawings in
`common with pending U.S. patent applications entitled:
`“Video and/or Audio Decompression and/or Compression
`Device that Shares a Memory” by Jefferson E. Owen, Raul
`Z. Diaz, and Osvaldo Colavin Ser. No. 08/702.910 filed on
`Aug. 26, 1996, and has the same effective filing date and
`ownership as the present application, and to that extent is
`related to the present application, which is incorporated
`herein by reference.
`
`15
`
`2
`added to allow the System to compress Video and/or audio
`Sequences, to be transmitted or Stored. Both need to be added
`for two way communication Such as Video telephony.
`A typical decoder, such as an MPEG decoder 10 shown in
`FIG. 1a, contains video decoding circuitry 12, audio decod
`ing circuitry 14, a microcontroller 16, and a memory inter
`face 18. The decoder can also contain other circuitry
`depending on the electronic System the decoder is designed
`to operate in. For example, when the decoder is designed to
`operate in a typical television the decoder will also contain
`an on screen display (OSD) circuit.
`FIG. 1b shows a better decoder architecture, used in the
`ST3520 and ST352OAMPEG Audio/MPEG-2 Video Inte
`grated Decoder manufactured by SGS-THOMSON Micro
`electronics. The decoder has a register interface 20 instead
`of a microcontroller. The register interface 20 is coupled to
`an external microcontroller 24. The use of a register inter
`face 20 makes it possible to tailor the decoder 10 to the
`Specific hardware the decoder 10 interfaces with or change
`its operation without having to replace the decoder by just
`reprogramming the register interface. It also allows the user
`to replace the microcontroller 24, to upgrade or tailor the
`microcontroller 24 to a specific use, by just replacing the
`microcontroller and reprogramming the register interface
`20, without having to replace the decoder 10.
`The memory interface 18 is coupled to a memory 22. A
`typical MPEG decoder 10 requires 16 Mbits of memory to
`operate in the main profile at main level mode (MP at ML).
`This typically means that the decoder requires a 2 Mbyte
`memory. Memory 22 is dedicated to the MPEG decoder 10
`and increases the price of adding a decoder 10 to the
`electronic System. In current technology the cost of this
`additional dedicated memory 22 can be a significant per
`centage of the cost of the decoder.
`An encoder also requires a memory interface 18 and
`dedicated memory. Adding the encoder to an electronic
`System again increases the price of the System by both the
`price of the encoder and its dedicated memory.
`A goal in the Semiconductor industry is to reduce the die
`area of an integrated circuit device for a given functionality.
`Some advantages of reducing the die area is the increase in
`the number of the die that can be manufactured on Same size
`Silicon wafer, and the reduction in price per die resulting
`therefrom. This results in both an increase in volume and
`reduction in price of the device.
`Many of the functional circuits described above for FIG.
`1a and FIG. 1b take up a lot of die space. However, each of
`them is needed to make the respective decoder operate.
`FIG. 1c shows a computer 25 containing a decoder 10, a
`main memory 168 and other typical components Such as a
`modem 199, and graphics accelerator 188. The decoder 10
`and the rest of the components are coupled to the core logic
`chipset 190 through a bus 170. The bus is typically a PCI
`(peripheral component interface) or ISA (industry standard
`architecture) bus, and each component contains an appro
`priate interface for interfacing with the bus.
`When any component needs access to the memory 168
`either to read from or write to the main memory 168, it
`generates a request which is placed on the bus 26. When the
`request is a write the data to be written is also placed on the
`bus 26. The request is processed in the core logic chipset 190
`and the data is then either written to or read from the main
`memory 168. When data is read from the main memory 168
`the data is now placed on the bus and goes to the component
`that requested the read.
`There are typically many components in the computer
`Systems that may require access to the main memory 168,
`
`25
`
`35
`
`BACKGROUND
`The present invention relates to the field of electronic
`Systems having a Video and/or audio decompression and/or
`compression device, and is more specifically directed to
`Sharing a memory interface between a Video and/or audio
`decompression and/or compression device and another
`device contained in the electronic System.
`The size of a digital representation of uncompressed Video
`images is dependent on the resolution, and color depth of the
`image. A movie composed of a Sequence of Such images, and
`the audio signals that go along with them, quickly becomes
`large enough So that uncompressed Such a movie typically
`cannot fit entirely onto conventional recording medium,
`Such as a CD. It is also typically now prohibitively expen
`Sive to transmit Such a movie uncompressed.
`It is therefore advantageous to compress video and audio
`Sequences before they are transmitted or Stored. A great deal
`of effort is being expanded to develop Systems to compress
`these Sequences. There are Several coding Standards cur
`rently used that are based on the discrete cosine transfer
`algorithm including MPEG-1, MPEG-2, H.261, and H.263.
`(MPEG stands for “Motion Picture Expert Group”, a com
`mittee of the International Organization for Standardization,
`40
`ISO.) The MPEG-1, MPEG-2, H.261, and H.263 standards
`are decompression protocols that describe how an encoded
`bitstream is to be decoded. The encoding can be done in any
`manner, as long as the resulting bitstream complies with the
`Standard.
`Video and/or audio compression devices (hereinafter
`encoders) are used to encode the video and/or audio
`Sequence before it is transmitted or Stored. The resulting
`bitstream is decoded by a Video and/or audio decompression
`device (hereinafter decoder) before the video and/or audio
`Sequence is displayed. However, a bitstream can only be
`decoded by a decoder if it complies to the Standard used by
`the decoder. To be able to decode the bitstream on a large
`number of Systems it is advantageous to encode the Video
`and/or audio Sequences to comply to a well accepted decom
`pression standard. The MPEG standards are currently well
`accepted Standards for one way communication. H.261, and
`H.263 are currently well accepted standards for video tele
`phony.
`Once decoded the images can be displayed on an elec
`tronic System dedicated to displaying Video and audio, Such
`as television or digital video disk (DVD) player, or on
`electronic Systems where image display is just one feature of
`the System, Such as a computer. A decoder needs to be added
`to these Systems to allow them to display compressed
`Sequences, Such as received images and associated audio, or
`ones taken from a Storage device. An encoder needs to be
`
`45
`
`50
`
`55
`
`60
`
`65
`
`IPR2018-01413
`Sony EX1023 Page 7
`
`

`

`5,812,789
`
`15
`
`25
`
`35
`
`40
`
`3
`and they are typically all coupled to the same bus 174, or
`possibly several buses 170, 198 connected together by a PCI
`bridge 192, if there are not enough connectors on one bus to
`accommodate all of the peripherals. However, the addition
`of each bus is very expensive. Each request is typically
`processed according to a priority Scheme. The priority
`Scheme is typically based on the priority given to the device
`and the order in which the requests are received. Typically,
`the priority Scheme is set up So no device monopolizes the
`bus, Starving all of the other devices. Good practice Suggests
`that no device on the bus require more than approximately
`50% of the bus’s bandwidth.
`The minimum bandwidth required for the decoder 10 can
`be calculated based on the characteristics and desired opera
`tion of the decoder. These characteristics include the Stan
`dard to which the bitstream is encoded to comply with,
`whether the decoder is to operate in real time, to what extent
`frames are dropped, and how the images are Stored.
`Additionally, the latency of the bus that couples the decoder
`to the memory should be considered.
`If the decoder does not operate in real time the decoded
`movie would stop periodically between images until the
`decoder can get access to the memory to process the next
`image. The movie may stop quite often between images and
`wait.
`To reduce the minimum required bandwidth and still
`operate in real time, the decoder 10 may need to drop
`frames. If the decoder 10 regularly does not decode every
`frame then it may not need to Stop between images.
`However, this produces very poor continuity in the images.
`This is problematic with an image encoded to the MPEG-1
`or MPEG-2 standard, or any standards that uses temporal
`compression. In temporal (interpicture) compression Some
`of the images are decoded based on previous images and
`Some based on previous and future images. Dropping an
`image on which the decoding of other images is based is
`unacceptable and will result in many poor or even com
`pletely unrecognizable images.
`The computer can also contain both a decoder and
`encoder to allow for video telephony, as described above. In
`this case not operating in real time would mean that the
`length of time between the occurrence of an event, Such as
`Speaking, at one end of the conversation until the event is
`displayed at the other end of the conversation is increased by
`the time both the encoder and then the decoder must wait to
`get access to the bus and the main memory. Not being able
`to operate in real time means that there would be gaps in the
`conversation until the equipment can catch up. This
`increases the time needed to have a Video conference, and
`makes the conference uncomfortable for the participants.
`One widely used Solution to allow a component in a
`computer System to operate in real time is to give the
`component its own dedicated memory. Thus, as shown in
`FIG. 1c, the decoder 10 can be given its own dedicated
`memory 22, with a dedicated bus 26 to connect the decoder
`10 to its memory 22. The dedicated memory 22, its con
`troller and the pins to control this memory significantly
`increase the cost of adding a decoder 10 to the computer.
`SUMMARY OF THE INVENTION
`The present application discloses an electronic System
`that contains a first device and Video and/or audio decom
`pression and/or compression device capable of operating in
`real time. Both the first device and the video and/or audio
`decompression and/or compression device require a
`memory interface. The Video and/or audio decompression
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`and/or compression device shares a memory interface and
`the memory with the first device. In the preferred embodi
`ment of the invention the shared memory interface contains
`an arbiter. The arbiter and DMA engines of the video and/or
`audio decompression and/or compression device and of the
`first device are configured to arbitrate between the two
`devices when one of them is requesting access to the
`memory. This allows the use of one memory interface to
`control the access of both the video and/or audio decom
`pression and/or compression device and the first device to
`the memory.
`When the video and/or audio decompression and/or com
`pression device used in an electronic System, Such as a
`computer, already containing a device that has a memory
`interface the Video and/or audio decompression and/or com
`pression device can share that memory interface and the
`memory of the device and the memory interface and
`memory of the video and/or audio decompression and/or
`compression device can be eliminated. Eliminating this
`memory interface reduces the die area without changing the
`critical dimensions of the device. Therefore increasing the
`Volume and reducing the cost of the decoder or encoder.
`Eliminating the memory greatly reduces the cost of adding
`the Video and/or audio decompression and/or compression
`device to the electronic System while not requiring the Video
`and/or audio decompression and/or compression device to
`be connected to the System bus, allowing the video and/or
`audio decompression and/or compression device to operate
`in real time.
`An advantage of the present invention is significant cost
`reduction due to the fact that the Video and/or audio decom
`pression and/or compression device does not need its own
`dedicated memory but can share a memory with another
`device and Still operate in real time.
`Another significant advantage of the present invention is
`that the die Space needed for the Video and/or audio decom
`pression and/or compression device is Smaller because the
`memory interface on the Video and/or audio decompression
`and/or compression device is eliminated.
`A further advantage of the present invention is that the
`Video and/or audio decompression and/or compression
`device can share the memory of the device with which it is
`Sharing the memory interface more efficiently.
`Another advantage of the present invention is that the cost
`of producing a Video and/or audio decompression and/or
`compression device is reduced because the memory inter
`face on the Video and/or audio decompression and/or com
`pression device is eliminated.
`Another advantage of the present invention is that the
`Video and/or audio decompression and/or compression
`device can be monolithically integrated into the first device
`and no extra packaging or pins are needed for the Video
`and/or audio decompression and/or compression device, and
`no pins are needed for the first device to connect to the Video
`and/or audio decompression and/or compression device,
`Saving pins on both devices and producing a better connec
`tion between the two devices.
`Other advantages and objects of the invention will be
`apparent to those of ordinary skill in the art having reference
`to the following Specification together with the drawings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIGS. 1a and 1b are electrical diagrams, in block form, of
`prior art decoderS.
`FIG. 1c is an electrical diagram, in block form, of a
`computer System containing a decoder according to the prior
`art.
`
`IPR2018-01413
`Sony EX1023 Page 8
`
`

`

`S
`FIG. 2 is an electrical diagram, in block form, of an
`electronic System containing a device having a memory
`interface and an encoder and decoder.
`FIG. 3 is an electrical diagram, in block form, of a
`computer System containing a core logic chipset designed
`for the CPU to share a memory interface with an encoder
`and decoder.
`FIG. 4 is an electrical diagram, in block form, of a
`computer System containing a graphics accelerator designed
`to share a memory interface with an encoder and/or decoder.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`FIG. 2 shows an electronic system 40 containing a first
`device 42 having access to a memory 50 through a memory
`interface 48, and a decoder 44 and encoder 46, having acceSS
`to the same memory 50 through the same memory interface
`48. First device 42 can be a processor, a core logic chipset,
`a graphics accelerator, or any other device that requires
`access to the memory 50, and either contains or is coupled
`to a memory interface. Any parts common to FIGS. 1
`through 4 are indicated using the Same numbering System. In
`the preferred embodiment of the invention, electronic Sys
`tem 40 contains a first device 42, a decoder 44, an encoder
`46, a memory interface 48, and a memory 50. Although,
`either the decoder 44 or encoder 46 can be used in the
`decoder/encoder 45 without the other. For ease of reference,
`a Video and/or audio decompression and/or compression
`device 45 will hereinafter be referred to as decoder/encoder
`45. The decoder/encoder 45 may be a single device, or cell
`on an integrated circuit, or may be two Separate devices, or
`cells in an integrated circuit. In the preferred embodiment of
`the invention, the first device 42, decoder/encoder 45, and
`memory interface 48 are on one integrated circuit, however,
`they can be on Separate integrated circuits in any combina
`tion.
`The decoder 44 includes a video decoding circuit 12 and
`an audio decoding circuit 14, both coupled to a register
`interface 20. The decoder 44 can be either a video and audio
`decoder, just a video, or just an audio decoder. If the decoder
`44 is just a Video decoder it does not contain the audio
`decoding circuitry 14. The audio decoding can be performed
`by a separate audio codec coupled to the first device 42, or
`through software. In the preferred embodiment of the
`invention, when the decoder/encoder 45 is in a System
`containing a processor and is coupled to the processor, the
`audio decoding is performed in Software. This frees up Space
`on the die without causing Significant delay in the decoding.
`If the audio decoding is performed in Software, the processor
`should preferably operate at a speed to allow the audio
`decoding to be performed in real time without Starving other
`components of the System that may need to utilize the
`processor. For example, currently Software to perform AC-3
`audio decoding takes up approximately 40% of the band
`width of a 133 MHZ Pentium. The encoder 46 includes a
`Video encoding circuit 62 and an audio encoding circuit 64,
`both coupled to a register interface 20. The encoder 46 can
`be either a Video and audio encoder, just a video, or just an
`audio encoder. If the encoder 46 is just a video encoder, it
`does not contain the audio encoding circuitry 64. The audio
`encoding can be performed by a separate audio codec
`coupled to the first device 42, or through software. In the
`preferred embodiment of the invention, when the decoder/
`encoder 45 is in a System containing a processor and is
`coupled to the processor, the audio encoding is performed in
`Software presenting the same advantages of freeing up Space
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,812,789
`
`6
`on the die without causing Significant delay in the encoding.
`The register interfaces 20 of the decoder 44 and encoder 46
`are coupled to a processor.
`The decoder 44 and encoder 46 are coupled to the direct
`memory access (DMA) engine 52. The decoder and encoder
`can be coupled to the same DMA engine as shown in FIG.
`2, or each can have its own DMA engine, or share a DMA
`engine with another device. When the decoder/encoder 45
`are two separate devices or cells, decoder 44 and encoder 46
`can still be coupled to one DMA engine 52. When the
`decoder/encoder is one device or is one cell on an integrated
`circuit, the DMA engine 52 can be part of the decoder/
`encoder 45, as shown in FIG. 2. The DMA engine 52 is
`coupled to the arbiter 54 of the memory interface 48.
`The first device 42 also contains a DMA engine 60. The
`DMA engine 60 of the first device 42 is coupled to the arbiter
`54 of the memory interface 48. The arbiter is also coupled
`to the refresh logic 58 and the memory controller 56. The
`memory interface 48 is coupled to a memory 50. The
`memory controller 56 is the control logic that generates the
`address the memory interface 48 accesses in the memory 50
`and the timing of the burst cycles.
`In current technology, memory 50 is typically a DRAM.
`However, other types of memory can be used. The refresh
`logic 58 is needed to refresh the DRAM. However, as is
`known in the art, if a different memory is used, the refresh
`logic 58 may not be needed and can be eliminated.
`The decoder/encoder 45 is coupled to the memory 50
`through devices, typically a bus 70, that have a bandwidth
`greater than the bandwidth required for the decoder/encoder
`45 to operate in real time. The minimum bandwidth required
`for the decoder/encoder 45 can be calculated based on the
`characteristics and desired operation of the decoder, includ
`ing the Standard to which the bitstream is encoded to comply
`with, whether the decoder/encoder 45 is to operate in real
`time, to what extent frames are dropped, and which images
`are stored. Additionally, the latency of the bus 70 that
`couples the decoder/encoder 45 to the memory 50 should be
`considered.
`A goal is to have the decoder/encoder 45 operate in real
`time without dropping So many frames that it becomes
`noticeable to the human viewer of the movie. To operate in
`real time the decoder/encoder 45 should decoder and/or
`encode images fast enough So that any delay in decoding
`and/or encoding cannot be detected by a human viewer. This
`means that the decoder/encoder 45 has a required bandwidth
`that allows the decoder/encoder 45 to operate fast enough to
`decode the entire image in the time between Screen
`refreshes, which is typically /30 of a Second, with the human
`Viewer not being able to detect any delay in the decoding
`and/or encoding. To operate in real time the required band
`width should be lower than the bandwidth of the bus. In
`order not to Starve the other components on the bus, i.e. deny
`these components access to the memory for an amount of
`time that would interfere with their operation, this required
`bandwidth should be less the entire bandwidth of the bus.
`Therefore a fast bus 70 should be used. A fast bus 70 is any
`bus whose bandwidth is equal to or greater that the required
`bandwidth. There are buSSes in current technology, includ
`ing the ISA bus, whose bandwidth is significantly below the
`bandwidth required for this.
`In the preferred embodiment of the invention the decoder/
`encoder 45 is coupled to the memory 50 through a fast bus
`70 that has a bandwidth of at least the bandwidth required
`for the decoder/encoder 45 to operate in real time, a thresh
`old bandwidth. Preferably the fast bus 70 has a bandwidth of
`
`IPR2018-01413
`Sony EX1023 Page 9
`
`

`

`7
`at least approximately twice the bandwidth required for the
`decoder/encoder 45 to operate in real time. In the preferred
`embodiment the fast bus 70 is a memory bus, however any
`bus having the required bandwidth can be used.
`The decoder/encoder 45 only requires access to the
`memory during operation. Therefore, when there is no need
`to decode or encode, the first device 42, and any other
`devices sharing the memory 50 have exclusive access to the
`memory and can use the entire bandwidth of the fast bus 70.
`In the preferred embodiment, even during decoding and
`encoding the decoder/encoder 45 does not always use the
`entire required bandwidth. Since the fast bus 70 has a
`bandwidth a little less than twice the required bandwidth the
`decoder/encoder 45 uses at most 60% of the bandwidth of
`the fast bus 70.
`The required bandwidth is determined based on the size
`and resolution of the image, and the type of frame (I, P, or
`B). In the preferred embodiment the decoder/encoder typi
`cally will be using less than 40% of the bandwidth of the fast
`bus 70. This frees up the remaining bandwidth to be used by
`the other devices the decoder/encoder 45 is sharing the
`memory 50 with.
`The decoder/encoder 45 can decode a bitstream formatted
`according to any one or a combination of Standards. In the
`preferred embodiment of the invention the decoder/encoder
`45 is a multi-standard decoder/encoder capable of decoding
`and encoding Sequences formatted to comply to Several well
`accepted standards. This allows the decoder/encoder 45 to
`be able to decode a large number of Video and/or audio
`Sequences. The choice of which Standards the decoder/
`encoder 45 is capable of decoding bitstreams formatted to
`and of encoding Sequences to comply to is based on the
`desired cost, efficiency, and application of the decoder/
`encoder 45.
`In the preferred embodiment, these Standards are capable
`of both intrapicture compression and interpicture compres
`Sion. In intrapicture compression the redundancy within the
`image is eliminated. In interpicture compression the redun
`dancy between two images are eliminated and only the
`difference information is transferred. This requires the
`decoder/encoder 45 to have access to the previous or future
`image that contains information needed to decode or encode
`the current image. These precious and/or future images need
`to be Stored then used to decode the current image. This is
`one of the reasons the decoder/encoder 45 requires access to
`the memory, and requires a large bandwidth. The MPEG-1
`and MPEG-2 standards allow for decoding based on both
`previous images and/or future imageS. Therefore for a
`decoder/encoder 45 capable of operating in real time to be
`able to comply with the MPEG-1 and MPEG-2 standards it
`should be able to access two images, a previous and a future
`image, fast enough to decode the current image in the /30 of
`a Second between Screen refreshes.
`An MPEG environment is asymmetrical; there are much
`fewer encoders than decoders. The encoders are very diffi
`cult and expensive to manufacture and the decoders are
`comparatively easy and cheap. This encourages many more
`decoders than encoders, with the encoders in centralized
`locations, and decoderS available Such that every end user
`can have a decoder. Therefore, there are many receivers but
`few transmitters.
`For Video telephony and teleconferencing each end user
`has to be able to both receive and transmit. H.261, and H.263
`are currently well accepted Standards for Video telephony.
`An encoder that can encode Sequences to comply to the
`H.261 and H.263 standards is less complicated, having a
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,812,789
`
`8
`lower resolution and lower frame rate than an encoder that
`complies to the MPEG-1 or MPEG-2 standards, possibly
`making the quality of the decoded images. Somewhat lower
`than those from an encoder that complies with the MPEG-1
`or MPEG-2 standards. Such an encoder, since it should be
`inexpensive and operate in real time, is also less efficient
`than an encoder to encode Sequences to comply to the
`MPEG-1 or MPEG-2 standards. This means that the com
`pression factor, which is the ratio between the Source data
`rate and the encoded bitstream data rate, of Such an encoder
`is lower for a given image quality than the compression
`factor of an MPEG encoder. However, because Such an
`encoder is leSS complicated, it is much cheaper and faster
`than an encoder capable of complying with the MPEG-1
`and/or MPEG-2 standards. This makes video telephony
`possible, Since both a long delay in encoding the Signal and
`a cost that is prohibitively expensive for many users is
`unacceptable in Video telephony.
`In the preferred embodiment, the decoder/encoder 45 is
`capable of decoding a bitstream formatted to comply to the
`MPEG-1, MPEG-2, H.261, and H.263 standards, and encod
`ing a sequence to produce a bitstream to comply to the
`H.261, and H.263 standards. This allows the decoder/
`encoder 45 to be used for video telephony. Having the
`encoding comply to the H.261 and H.263 standards but not
`the MPEG-1 and MPEG-2 balances the desire to reduce the
`cost of transm