`SEP 2. 2. 1998
`
`PATENT
`NUMBER
`
`5812789
`iiQ((IHI
`
`- • .FJ L' i_.'} -,
`-
`FILING DATE CLASS
`
`-
`
`SUBCLASS, ~
`
`EXAMINER
`
`~
`5 ~ E , \<.Jt H. 1 F:;iL ·c.:
`3
`
`.. ,
`·~
`
`Foreign priority claimed
`35 use 119 conditions met
`
`no
`no
`
`STATE OR SHEETS
`COUNTRY DRWGS.
`
`TOTAL
`CLAIMS
`
`AS
`
`FILED -+
`
`INDEP.
`CLAIMS
`
`FILING· FEE.
`RECEIVED
`
`ATTORNEY'S
`DOCKET NO.
`
`.. U.S. DEPT. of_(:OMPJl_l:RCE •Paten! and Tra~emark Offlce-PCT-436_L (r!v. ?:_94)
`
`PARTS OF APPLICATION
`FILED SEPARATELY
`
`NOTICE OF ALLOWANCE MAILED
`
`Label
`Area
`
`Print Claim
`
`DRAWING
`
`Print Fig.
`l:}
`
`ELLIS B. RAMIREZ
`PRIMARY EXAMINER
`
`Primary Examiner
`
`PREPARED FOR. ISSUE
`
`WARNING: The information disclosed herein may be restricted. Unauthorized disclosure may be prohibited
`tiy the United States Code ntle 35, Sections 122, 181 and 368. Possession outside the U.S.
`Patent & Trademark Office is restricted to authorized employees and contractors only.
`
`(FACE)
`
`Apple Inc. v. Parthenon
`Ex. 1002 / Page 1 of 281
`
`
`
`r---.....__
`___ __ 1. Af!Plftation __ ...) __ , _ papers.
`v_~
`-----~,,, ,,
`3. --==='7) ~57=:''" ~--r--;:.-----;;-z-:--
`- - - - - 7
`-----.r ,,.....,,
`A.~,,;..o=~~~~~~;...__:::~~-
`_ _ __ _ 5~ _· ~~~..L...1..--~~~~...:ai..i...-~.c,..,,.L.
`q. d-47 y~.
`----}_,jd;J:=.~ ~-=£(!)~SJ ~.--L+-~~""""'4-
`___ /.Q,Y<l=r:>.R~aad\ ~,
`____ yiQ'}hc~ J'-
`fl-Uuw~1G-X
`_ _ __,___io. s<(2ef21~' /Jvh:s;,:& d Ak1N
`_J_._ . ....._/t3_{9 ........ '8_ 11.
`Fonc·: .···.·.:.:;::::.::~:I 5· sht;'; ;:,~Jt
`PTO Grant SEP 2 2 -1~98
`f) .
`("I
`\ __ )Ur~
`.~ ....... ~-
`_,
`
`j
`t ;_. i •) I.:'>
`I .'
`
`., '
`
`i
`
`I_
`
`Date
`Entered
`or
`Co1,1nted
`
`-- - ---,--
`
`-- - ~-- ~ - -------~---
`
`CONTENTS·
`
`Date
`Received
`or
`Mailed
`
`.Jl-4 '1(u.
`
`/•
`·'.1
`
`;'' ,-
`
`,,, ~~-!-
`
`12.
`
`13.·
`
`14.
`1's.
`16.
`
`( ·
`
`....--.. -~·:;,...
`
`~
`
`--- ,..
`
`. ........
`. ,
`
`;··
`
`:,- "
`
`,.,. __ :,.
`
`17.
`
`18.
`
`19.
`
`20.
`
`21.
`
`22.
`
`23.
`
`24.
`
`25.
`
`26.
`
`27.
`
`28.
`
`29.
`
`30.
`
`31.
`
`32.
`
`(FRONT)
`
`Ex. 1002 / Page 2 of 281
`
`
`
`Staple Issue Slip Here
`
`IDNO.
`
`~.ATE/
`
`POSITION
`CLASSIFIER
`EXAMINER
`TYPIST
`VERIFIER
`CORPS CORR.
`SPEC.HAND
`FILE MAINT.
`DRAFTING
`
`INDEX OF CLAIMS
`
`Claim
`
`Date
`
`<il ~
`c:
`iii
`:2>,
`c:
`u::
`i...Q.
`( r--- !) v
`.
`
`:/
`
`~
`.. ,,..
`"'
`
`,/ l/
`I./
`
`:
`I
`
`B
`
`/
`
`"
`Iv
`// 1,
`"'.I/·
`t:r• A
`It-,~
`i' ·v1,.,
`" ,
`
`Claim
`<il
`c:
`"6>
`
`iii
`c:
`
`u:: 0
`51
`52
`53
`54
`55
`56
`57
`58
`59
`60
`61
`62
`63
`64
`65
`66
`67
`68
`69
`70
`71
`72
`73
`74
`75
`76
`77
`78
`79
`80
`81
`82
`83
`84
`85
`86
`87
`88
`89
`90
`91
`92
`93
`94
`95
`96
`97
`98
`99
`100
`
`Date
`
`·•
`
`SYMBOLS
`./ ................................. Rejected
`= ................................. Allowed
`• (Through numberal) Canceled
`+ ................................. Restricted
`N ................................. Non-elected
`I ................................. Interference
`A ................................. Appeal
`· 0 ................................. Objected
`
`1)
`1
`1
`1B
`
`Ii q;r --
`
`». 15
`~ 16
`c.f 17
`&:, 18
`{.. 19
`1 20
`21
`ex-
`10 22
`/D 23
`II 24
`n.. 25
`/-z, 26
`ltli 27
`r: "'"28
`I (p 29
`, -, 30
`'X' 31
`f/q 32
`l7 t> 33
`L7
`34
`~). 35
`LJ; 36
`Q(/ 37
`.,.., 38
`,/() 39
`~g 41 -
`~ 40
`
`~ /h
`iy,
`K.3
`7.{J
`,.. '1 (.4~')
`u 45
`?
`46
`,,
`47
`9'. ~ 48
`... -..w.:
`50
`
`,v; ~
`)I II{!
`
`~
`
`~
`
`(LEFT INSIDE)
`
`Ex. 1002 / Page 3 of 281
`
`
`
`' f .
`
`--
`
`._, _ _:. __ ~.:.:. ... f..._,; _____
`
`-
`
`--·~-----
`
`M E j
`7
`vDEFKRfMENI
`PA‘TENTJAQ4D;'rRADEMAfl_KLOFE'
`
`Ex. 1002 I Page 4 of 281
`
`Ex. 1002 / Page 4 of 281
`
`
`
`SEARCHED
`
`Class
`
`Sub.
`
`Date
`
`Exmr.
`
`"""
`
`:
`
`_,..,-,-~
`
`SEARCH NOTES
`
`Date
`
`Exmr.
`
`3c?S" ~ .. 77 "b/Xi/1t1 ~
`
`JO-O .lJ l..
`"5 o I
`t~o
`·1a1
`Cf oz
`c.fo1
`lo
`
`Jtf<i
`
`i~/;.;1 Q_
`-
`
`Ptfe [)__
`
`-~
`
`v---- i-___.---..
`u~dibP
`
`i - - - -
`~
`
`~ --
`
`.
`
`INTERFERENCE SEARCHED
`Exmr.
`Sub.
`Date
`Class
`
`}'t'J
`
`)tr!J.77 >/uJt Q__
`-
`
`-
`
`\
`
`(RIGHT OUTSIDE)
`
`Ex. 1002 / Page 5 of 281
`
`
`
`nn 1·102!l 11
`PATENT APPLICATION SERIAL NO.
`-------
`
`U.S. DEPARTMENT OF COMMERCE
`PATENT AND TRADEMARK OFFICE
`FEE RECORD SHEET
`
`340 BA 19-1353 10/28/96 08702911
`34011 101
`188.00CH
`
`PT0-1556
`(5/87)
`
`Ex. 1002 / Page 6 of 281
`
`
`
`SGS·THOMSON
`"® ~O©OO©rn[brn©uOO©~O©~
`
`THE ASSISTANT COMMISSIONER FOR PATENTS
`Washington, D.C. 20231
`
`Re:
`
`lnventor(s): Raul Z. Diaz and Jefferson E. Owen
`
`For: Video and/or Audio Decompression and/or Compression Device that Shares a
`Memory Interface
`
`Our File No: 96-S-11
`
`Sir:
`
`Enclosed with this transmittal letter are:
`
`(1)
`(2)
`(3)
`(4)
`(5)
`(6)
`
`Subject patent application with Declaration and Power of Attorney;
`Five (5) sheets of informal drawings;
`Certificate of Express Mail;
`Assignment and Recordation Cover Sheet;
`Check in the amount of $1,396.00;
`Return postcard which we would appreciate your date stamping and returning to
`us upon receipt;
`
`The total filing fee has been calculated as follows:
`
`Basic fee
`Recordation of Assignment
`29 claims in excess of 20
`2 independent claim in excess of 3
`Total filing fee
`
`=
`
`$ 750.00
`=
`40.00
`=
`638.00
`=
`156.00
`= $1.584.00
`
`I authorize the Commissioner to charge any additional fees which may be required, or
`credit any overpayment to Account No. 19-1353. A duplicate copy of this sheet is
`enclosed.
`
`SGS.· THOMSON Microelectronics, Inc.• 1310 Electronics Drive• Carrollton, TX 75006-5039
`'
`Telephone (214) 466-6000, Telex 730643
`
`ST002-CAA
`
`Ex. 1002 / Page 7 of 281
`
`
`
`n~ 1~n?Q11
`
`In Re Application of:
`
`Raul Z. Diaz and Jefferson E. Owen
`
`Docket No. 96-S-011
`
`For: Video and/or Audio Decompression and/or Compression Device that Shares
`a Memory Interface
`
`CERTIFICATE OF EXPRESS MAIL
`
`"EXPRESS MAIL" NO. EG947362259US
`
`Date of Deposit: August 23, 1996
`
`I hereby certify that this paper or fee is being deposited with the United States
`Postal Service "Express Mail Post Office to Addressee" service under 37 CFR 1.1 O on
`the date indicated above and is addressed to the Assistant Commissioner for Patents,
`Box Patent Application, Washington, D .. C. 2.0231~
`
`~'
`
`Signature of person mailing paper or fee
`
`.
`
`Ex. 1002 / Page 8 of 281
`
`
`
`Cross-reference to Related Applications
`
`5
`
`This application contains some text and drawings in common with pending
`
`U.S. Patent Applications entitled: "Video and/or Audio Decompression and/or
`
`Compression Device that Shares a Memory" by Jefferson E. Owen, Raul Z. Diaz,
`$ J ..I 0 <}; f "J D;J 1 .::j J u
`O'is } ) b) C, fo
`f l l6 &_ O'"Y"""\..
`and Osvaldo Colavin _:::_i,'_'J_ (Marney's Docket Ne. 96-8-Q.12), and has the
`
`10
`
`same effective filing date and ownership as the present application, ·and to that
`
`extent is related to the present application, which is incorporated herein by
`
`reference.
`
`15
`
`";
`
`Background
`
`The present invention relates to the field of electronic systems having a video
`
`and/or audio decompression and/or compression device, and is more specifically
`
`directed to sharing a memory interface betWeen a video and/ or audio decompression
`
`20
`
`and/ or compression device and another device contained in the electronic system.
`
`The size of a digital representation of uncompressed video images is
`
`dependent on the resolution, and color depth of the image. A movie composed of
`
`a sequence of such images, and the audio signals that go along with them, quickly
`
`25
`
`becomes large enough so that uncompressed such a movie typically cannot fit
`
`entirely onto conventional recording medium, such as a CD.
`
`It is also typically
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 1
`
`Ex. 1002 / Page 9 of 281
`
`
`
`now prohibitively expensive to transmit such a movie uncompressed. ,
`
`It is therefore advantageous to compress video and audio sequences before
`
`they are transmitted or stored. A great deal of effort is being expanded to develop
`
`5
`
`systems to compress these sequences. There are several coding standards currently
`
`used that are based on the discrete cosine transfer algorithm including MPEG-1,
`
`MPEG-2, H.261, and H.263. (MPEG stands for "Motion Picture Expert Group",
`
`a committee of the International Organization for Standardization, ISO.) The
`
`MPEG-1, MPEG-2, H.261, and H.263 standards are decompression protocols that
`
`10
`
`describe how an encoded bitstream is to be decoded. The encoding can be done
`
`in any manner, as long as the resulting bitstream complies with the standard.
`
`Video and/or audio compression devices (hereinafter encoders) are used to
`
`encode the video and/or audio sequence before it is transmitted or stored. The
`
`15
`
`resulting bitstream is decoded by a video and/or audio decompression device
`
`(hereinafter decoder) before the video and/or audio sequence is displayed.
`
`However, a bitstream can only be decoded by a decoder if it complies to the
`
`standard used by the decoder. To be able to decode the bitstream on a large
`
`number of systems it is advantageous to encode the video and/or audio sequences
`
`20
`
`to comply to a well accepted decompression standard. The MPEG standards are
`
`currently well accepted standards for one way communication. H.261, and H.263
`
`are currently well accepted standards for video telephony.
`
`Once decoded the images can be displayed on an electronic system dedicated
`
`25
`
`to displaying video and audio,, such as television or digital video disk (DVD)
`
`player, or on electronic systems where image display is just one feature of the
`
`system, such as a computer. A decoder needs to be added to these systems to
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 2
`
`Ex. 1002 / Page 10 of 281
`
`
`
`allow them to display compressed sequences, such as received images and
`
`associated audio, or ones taken from a storage device. An encoder needs to be
`
`added to allow the system to compress video and/or audio sequences, to be
`
`transmitted or stored. Both need to be added for two way communication such as
`
`5
`
`video telephony.
`
`A typical decoder, such as an MPEG decoder 10 shown in Figure la,
`
`contains video decoding circuitry 12, audio decoding circuitry 14, a microcontroller
`
`16, and a memory interface 18. The decoder can also contain other circuitry
`
`10
`
`depending on the electronic system the decoder is designed to operate in. For
`
`example, when the decoder is designed to operate in a typical television the decoder
`
`will also contain an on screen display (OSD) circuit.
`
`Figure lb shows a better decoder architecture, used in the STi3520 and
`
`15
`
`STi3520A MPEG Audio/MPEG-2 Video Integrated Decoder manufactured by SGS(cid:173)
`
`THOMSON Microelectronics. The decoder has a register interface 20 instead of
`
`a microcontroller.
`
`The register
`
`interface 20
`
`is coupled to an external
`
`microcontroller 24. The use of a register interface 20 makes it possible to tailor the
`
`decoder 10 to the specific hardware the decoder 10 interfaces with or change its
`
`20
`
`operation without having to replace the decoder by just reprogramming the register
`
`interface. It also allows the user to replace the microcontroller 24, to upgrade or
`
`tailor the microcontroller 24 to a specific use, by just replacing the microcontroller
`
`and reprogramming the register interface 20, without having to replace the decoder
`
`10.
`
`25
`
`The memory interface 18 is coupled to a memory 22. A typical MPEG
`
`decoder 10 requires 16 Mb its of memory to operate in the main profile at main
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 3
`
`Ex. 1002 / Page 11 of 281
`
`
`
`level mode (MP at ML). This typically means that the decoder requires a 2Mbyte
`
`memory. Memory 22 is dedicated to the MPEG decoder 10 and increases the price
`
`of adding a decoder 10 to the electronic system. In current technology the cost of
`
`this additional dedicated memory 22 can be a significant percentage of the cost of
`
`5
`
`the decoder.
`
`An encoder also requires a memory interface 18 and dedicated memory.
`
`Adding the encoder to an electronic system again increases the price of the system
`
`by both the price of the encoder and its dedicated memory.
`
`10
`
`A goal in the semiconductor industry is to reduce the die area of an
`
`integrated circuit device for a given functionality. Some advantages of reducing the
`
`die area is the increase in the number of the die that can be manufactured on same
`
`size silicon wafer, and the reduction in price per die resulting therefrom. This
`
`15
`
`results in both an increase in volume and reduction in price of the device.
`
`Many of the functional circuits described above for Figure 1 a and Figure 1 b
`
`take up a lot of die space. However, each of them is needed to make the respective
`
`decoder operate.
`
`20
`
`Figure le shows a computer 25 containing a decoder 10, a main memory 168
`
`and other typical components such as a modem 199, and graphics accelerator 188.
`
`The decoder 10 and the rest of the components are coupled to the core logic chipset
`
`190 through a bus 170. The bus is typically a PCI (peripheral component interface)
`
`25
`
`or ISA (industry standard architecture) bus, and each component contains an
`
`appropriate interface for interfacing with the bus.
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 4
`
`Ex. 1002 / Page 12 of 281
`
`
`
`When any component needs access to the memory 168 either to read from
`
`or write to the main memory 168, it generates a request which is placed on the bus
`
`26. When the request is a write the data to be written is also placed on the bus 26.
`
`The request is processed in the core logic chipset 190 and the data is then either
`
`5
`
`written to or read from the main memory 168. When data is read from the main
`
`memory 168 the data is now placed on the bus and goes to the component that
`
`requested the read.
`
`There are typically many components in the computer systems that may
`
`10
`
`require access to the main memory 168, and they are typically all coupled to the
`l°I~
`same bus 174, or possibly several buses 170, ~connected together by a PCI
`_,,._
`bridge 192, if there are not enough connectors on one bus to accommodate all of
`
`the peripherals. However, the addition of each bus is very expensive. Each request
`
`is typically processed according to a priority scheme. The priority scheme is
`
`15
`
`typically based on the priority given to the device and the order in which the
`
`requests are received. Typically, the priority scheme is set up so no device
`
`monopolizes the bus, starving all of the other devices. Good practice suggests that
`.....
`no device on the bus require more than approximately 50% of the bus's bandwidth.
`
`20
`
`The minimum bandwidth required for the decoder 10 can be calculated based
`
`on the characteristics and desired operation of the decoder. These characteristics
`
`include the standard to which the bitstream is encoded to comply with, whether the
`
`decoder is to operate in real time, to what extent frames are dropped, and how the
`
`images are stored. Additiona11):7, the latency of the bus that couples the decoder to
`
`25
`
`the memory should be considered.
`
`If the decoder does not operate in real time the decoded movie would stop
`
`SGS-THOMSON Microelectronics Inc.
`96-S-ll
`Page 5
`
`Ex. 1002 / Page 13 of 281
`
`
`
`periodically between images until the decoder can get access to the memory to
`
`process the next image. The movie may stop quite often between images and wait.
`
`To reduce the minimum required bandwidth and still operate in real time, the
`-fr et.-,.... e....5
`decoder 10 may need to drop--fFame. If the decoder 10 regularly does not decode
`,A.-
`every frame then it may not need to stop between images. However, this produces
`
`very poor continuity in the images. This is problematic with an image encoded to
`
`the MPEG-1 or MPEG-2 standard/, or any standards that uses temporal
`
`compression.
`
`In temporal (interpicture) compression some of the images are
`
`10
`
`decoded based on previous images and some based on previous and future images. .
`
`Dropping an image on which the decoding of other images is based is unacceptable/
`
`and will result in many poor or even completely unrecognizable images.
`
`The computer can also contain both a decoder and encoder to allow for video
`
`15
`
`telephony, as described above. In this case not operating in real time would mean
`
`that the length of time between the occurrence of an event, such as speaking, at one
`
`end of the conversation until the event is displayed at the other end of the
`
`conversation is increased by the time both the encoder and then the decoder must
`
`wait to get access to the bus and the main memory. Not being able to operate in
`
`20
`
`real time means that there would be gaps in the conversation until the equipment
`
`can catch up. This increases the time needed to have a video conference, and
`
`makes the conference uncomfortable for the participants.
`
`One widely used solution to allow a component in a computer system to
`
`25
`
`operate in real time is to give t~e component its own dedicated memory. Thus, as
`
`shown in Figure le, the decoder 10 can be given its own dedicated memory 22,
`
`with a dedicated bus 26 to connect the decoder 10 to its memory 22. The
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 6
`
`Ex. 1002 / Page 14 of 281
`
`
`
`dedicated memory 22, its controller and the pms to control this memory
`
`significantly increase the cost of adding a decoder 10 to the computer.
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 7
`
`Ex. 1002 / Page 15 of 281
`
`
`
`Summary of the Invention
`
`The present application discloses an electronic system that contains a first
`
`device and video and/or audio decompression and/or compression device capable
`
`5
`
`of operating in real time. Both the first device and the video and/or audio
`
`decompression and/or compression device require a memory interface. The video
`
`and/or audio decompression and/or compression device shares a memory interface
`
`and the memory with the first device. In the preferred embodiment of the invention
`
`the shared memory interface contains an arbiter. The arbiter and DMA engines of
`
`10
`
`the video and/or audio decompression and/or compression device and of the first.
`
`device are configured to arbitrate between the two devices when one of them is
`
`requesting access to the memory. This allows the use of one memory interface to
`
`control the access of both the video and/or audio decompression and/or
`
`compression device and the first device to the memory.
`
`15
`
`When the video and/or audio decompression and/or compression device used
`
`in an electronic system, such as a computer, already containing a device that has
`
`a memory interface the video and/or audio decompression and/or compression
`
`device can share that memory interface and the memory of the device and the
`
`20
`
`memory interface and memory of the video and/or audio decompression and/or
`
`compression device can be eliminated. Eliminating this memory interface reduces
`
`the die area without changing the critical dimensions of the device. Therefore
`
`increasing the volume and reducing the cost of the decoder or encoder. Eliminating
`
`the memory greatly reduces the cost of adding the video and/or audio
`
`25
`
`decompression and/or compres.sion device to the electronic system while not
`
`requiring the video and/or audio decompression and/or compression device to be
`
`connected to the system bus, allowing the video and/or audio decompression and/or
`
`/
`'
`t
`
`>,
`
`)
`/l
`'
`
`\
`I
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 8
`
`Ex. 1002 / Page 16 of 281
`
`
`
`compression device to operate in real time.
`
`An advantage of the present invention is significant cost reduction due to the
`
`fact that the video and/or audio decompression and/or compression device does not
`
`5
`
`need its own dedicated memory but can share a memory with another device and
`
`still operate in real time.
`
`Another significant advantage of the present invention is that the die space
`
`needed for the video and/or audio decompression and/or compression device is,
`
`10
`
`smaller because the memory interface on the video and/or audio decompression
`
`and/or compression device is eliminated.
`
`A further advantage of the present invention is that the video and/or audio
`
`decompression and/or compression device can share the memory of the device with
`
`15
`
`which it is sharing the memory interface more efficiently.
`
`Another advantage of the present invention is that the cost of producing a
`
`video and/or audio decompression and/or compression device is reduced because
`
`the memory interface on the video and/or audio decompression and/or compression
`
`20
`
`device is eliminated.
`
`Another advantage of the present invention is that the video and/ or audio
`
`decompression and/or compression device can be monolithically integrated into the
`
`first device and no extra packaging or pins are needed for the video and/ or audio
`
`25
`
`decompression and/or compression device, and no pins are needed for the first
`
`device to connect to the video and/or audio decompression and/or compression
`
`device, saving pins on both devices and producing a better connection between the
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 9
`
`Ex. 1002 / Page 17 of 281
`
`
`
`two devices.
`
`Other advantages and objects of the invention will be apparent to those of
`
`ordinary skill in the art having reference to the following specification together with
`
`5
`
`the drawings.
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 10
`
`Ex. 1002 / Page 18 of 281
`
`
`
`Brief Description of the Drawings
`
`Figure la and lb are electrical diagrams, in block form, of prior art decoders.
`
`5
`
`Figure 1 c is an electrical diagram, in block form, of a computer system
`
`containing a decoder according to the prior art.
`
`10
`
`15
`
`Figure 2 is an electrical diagram, in block form, of an electronic system
`
`containing a device having a memory interface and an encoder and decoder.
`
`Figure 3 is an electrical diagram, in block form, of a computer system ·
`
`containing a core logic chipset designed for the CPU to share a memory interface
`
`with an encoder and decoder.
`
`Figure 4 is an electrical diagram, in block form, of a computer system
`
`containing a graphics accelerator designed to share a memory interface with an
`
`encoder and/or decoder.
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 11
`
`Ex. 1002 / Page 19 of 281
`
`
`
`Detailed Description of the Preferred Embodiment
`
`Figure 2 shows an electronic system 40 containing a first device 42 having
`
`access to a memory 50 through a memory interface 48, and a decoder 44 and
`
`5
`
`encoder 46, having access to the same memory 50 through the same memory
`
`interface 48. First device 42 can be a processor, a core logic chipset, a graphics
`
`accelerator, or any other device that requires access to the memory 50, and either
`
`contains or is coupled to a memory interface. Any parts common to Figures 1
`
`through 4 are indicated using the same numbering system.
`
`In the preferred
`
`10
`
`embodiment of the invention, electronic system 40 contains a first device 42, a
`
`decoder 44, an encoder 46, a memory interface 48, and a memory 50. Although,
`
`either the decoder 44 or encoder 46 can be used in the decoder/encoder 45 without
`
`the other. For ease of reference, a video and/or audio decompression and/or
`
`compression device 45 will hereinafter be referred to as decoder/encoder 45. The
`
`15
`
`decoder/encoder 45 may be a single device, or cell on an integrated circuit, or may
`
`be two separate devices, or cells in an integrated circuit.
`
`In the preferred
`
`embodiment of the invention, the first device 42, decoder/encoder 45, and memory
`
`interface 48 are on one integrated circuit, however, they can be on separate
`
`integrated circuits in any combination.
`
`20
`
`C"t (CLA-\t I :J,.
`The decoder 44 includes a video decoding ~ eireuit- and an audio decoding
`;;..L
`circuit 14, both coupled to a register interface 20. The decoder 44 can be either a
`video and audio decoder, just a video, or just an audio decoder. If the decoder 44
`
`is just a video decoder it does not contain the audio decoding circuitry 14. The
`
`25
`
`audio decoding can be performed by a separate audio codec coupled to the first
`
`device 42, or through software.
`
`In the preferred embodiment of the invention,
`
`when the decoder/encoder 45 is in a system containing a processor and is coupled
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 12
`
`Ex. 1002 / Page 20 of 281
`
`
`
`to the processor, the audio decoding is performed in software. This frees up space
`on the die without causing significant delay in the decoding. If the audio decoding
`
`is performed in software, the processor should preferably operate at a speed to
`
`allow the audio decoding to be performed in real time without starving other
`
`5
`
`components of the system that may need to utilize the processor. For example,
`
`currently software to perform AC-3 audio decoding takes up approximately 40%
`
`of the bandwidth of a 133 MHz Pentium. The encoder 46 includes a video
`
`encoding circuit 62 and an audio encoding circuit 64, both coupled to a register
`
`interface 20. The encoder 46 can be either a video and audio encoder, just a video,
`or just an audio encoder. If the encoder 46 is just a video encoder, it does not
`
`10
`
`contain the audio encoding circuitry 64. The audio encoding can be performed by
`
`a separate audio codec coupled to the first device 42, or through software. In the
`
`preferred embodiment of the invention, when the decoder/encoder 45 is in a system
`
`C\-15
`
`containing a processor and is coupled to the processor, the audio encoding is
`Sof +..uo..r~ pre.s-e->"+·,~
`performed in ~e. Presenting the same advantages of freeing up space on the
`"""'
`die without causing significant delay in the encoding. The register interfaces 20
`
`of the decoder 44 and encoder 46 are coupled to a processor.
`
`The decoder 44 and encoder 46 are coupled to the direct memory access
`
`20
`
`(DMA) engine 52. The decoder and encoder can be coupled to the same DMA
`
`engine as shown in Figure 2, or each can have its own DMA engine, or share a
`
`DMA engine with another device. When the decoder/encoder 45 are two separate
`
`devices or cells, decoder 44 and encoder 46 can still be coupled to one DMA
`
`engine 52. When the decoder/encoder is one device or is one cell on an integrated
`
`25
`
`circuit, the DMA engine 52 can be part of the decoder/encoder 45, as shown in
`
`Figure 2. The DMA engine 52 is coupled to the arbiter 54 of the memory interface
`
`48.
`! i
`'
`I
`!
`L-+-\~
`\ ,
`J
`\
`
`l
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 13
`
`Ex. 1002 / Page 21 of 281
`
`
`
`The first device 42 also contains a DMA engine 60. The DMA engine 60
`
`of the first device 42 is coupled to the arbiter 54 of the memory interface 48. The
`
`arbiter is also coupled to the refresh logic 58 and the memory controller 56. The
`
`(;:/ 5
`
`memory interface 48 is coupled to a memory 50. The memory controller 56 is the
`control logic that generates the address the memory interface 48~~Th_f
`"""--'
`memory 50 and the timing of the burst cycles.
`
`In current technology, memory 50 is typically a DRAM. However, other
`
`types of memory can be used. The refresh logic 58 is needed to refresh the
`
`10
`
`DRAM. However, as is known in the art, if a different memory is used, the refresh
`
`logic 58 may not be needed and can be eliminated.
`
`The decoder/encoder 45 is coupled to the memory 50 through devices,
`
`typically a bus 70, that have a bandwidth greater than the bandwidth required for
`
`15
`
`the decoder/encoder 45 to operate in real time. The minimum bandwidth required
`
`for the decoder/encoder 45 can be calculated based on the characteristics and
`
`desired operation of the decoder, including the standard to which the bitstream is
`
`encoded to comply with, whether the decoder/encoder 45 is to operate in real time,
`
`to what extent frames are dropped, and which images are stored. Additionally, the
`
`20
`
`latency of the bus 70 that couples the decoder/encoder 45 to the memory 50 should
`
`be considered.
`
`A goal is to have the decoder/encoder 45 operate in real time without
`
`dropping so many frames that it becomes noticeable to the human viewer of the
`
`25
`
`movie. To operate in real time the decoder/encoder 45 should decoder and/or
`
`encode images fast enough so that any delay in decoding and/or encoding cannot
`
`be detected by a human viewer. This means that the decoder/encoder 45 has a
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 14
`
`Ex. 1002 / Page 22 of 281
`
`
`
`required bandwidth that allows the decoder/encoder 45 to operate fast enough to
`
`decode the entire image in the time between screen refreshes, which is typically
`
`1/30 of a second, with the human viewer not being able to detect any delay in the
`
`decoding and/ or encoding. To operate in real time the required bandwidth should
`
`5
`
`be lower than the bandwidth of the bus.
`
`In order not to starve the other
`
`components on the bus, i.e. deny these components access to the memory for an
`
`amount of time that would interfere with their operation, this required bandwidth
`
`should be less the entire bandwidth of the bus. Therefore a fast bus 70 should be
`
`used. A fast bus 70 is any bus whose bandwidth is equal to or greater that the
`
`V"..- 10
`
`required bandwidth. There are busses,. in current technology, including the ISA ·
`
`bus, whose bandwidth is significantly below the bandwidth required for this.
`
`In the preferred embodiment of the invention the decoder/encoder 45 is
`coupled to the memory 50 through a fast bus 70 that has a bandwidth of at least
`
`15
`
`the bandwidth required for the decoder/encoder 45 to operate in real time, a
`
`threshold bandwidth. Preferably the fast bus 70 has a bandwidth of at least
`
`approximately twice the bandwidth required for the decoder/encoder 45 to operate
`
`in real time.
`
`In the preferred embodiment the fast bus 70 is a memory bus,
`
`however any bus having the required bandwidth can be used.
`
`20
`
`25
`
`The decoder/encoder 45 only requires access to the memory during operation.
`
`Therefore, when there is no need to decode or encode, the first device 42, and any
`
`other devices sharing the memory 50 have exclusive access to the memory7 and can
`use the entire bandwidth of the fast bus 70.
`
`In the preferred embodiment, even during decoding and encoding the
`
`decoder/encoder 45 does not always use the entire required bandwidth. Since the
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 15
`
`Ex. 1002 / Page 23 of 281
`
`
`
`fast bus 70 has a bandwidth a little less than twice the required bandwidth the
`
`decoder/encoder 45 uses at most 60% of the bandwidth of the fast bus 70.
`
`The required bandwidth is determined based on the size and resolution of the
`image, and the type of frame (I, P, or B).
`
`In the preferred embodiment the
`
`5
`
`decoder/encoder typically will be using less than 40% of the bandwidth of the fast
`
`bus 70. This frees up the remaining bandwidth to be used by the other devices the
`
`decoder/encoder 45 is sharing the memory 50 with.
`
`10
`
`The decoder/encoder 45 can decode a bitstream formatted according to any.
`
`one or a combination of standards. In the preferred embodiment of the invention
`
`the decoder/encoder 45 is a multi-standard decoder/encoder capable of decoding and
`
`encoding sequences formatted to comply to several well accepted standards. This
`
`allows the decoder/encoder 45 to be able to decode a large number of video and/or
`
`15
`
`audio sequences. The choice of which standards the decoder/encoder 45 is capable
`
`of decoding bitstreams formatted to and of encoding sequences to comply to is
`
`based on the desired cost, efficiency, and application of the decoder/encoder 45.
`
`In the preferred embodiment, these standards are capable of both intrapicture
`
`20
`
`compression and interpicture compress10n.
`
`In intrapicture compress10n the
`
`redundancy within the image is eliminated.
`
`In interpicture compress10n the
`
`redundancy between two images are eliminated and only the difference information
`
`is transferred. This requires the decoder/encoder 45 to have access to the previous
`
`or future image that contains information needed to decode or encode the current
`
`25
`
`image. These precious and/or fµture images need to be stored then used to decode
`
`the current image. This is one of the reasons the decoder/encoder 45 requires
`
`access to the memory, and requires a large bandwidth. The MPEG-1 and MPEG-2
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 16
`
`Ex. 1002 / Page 24 of 281
`
`
`
`standards allow for decoding based on both previous images and/or future images.
`
`Therefore for a decoder/encoder 45 capable of operating in real time to be able to
`
`comply with the MPEG-1 and MPEG-2 standards it should be able to access two
`
`images, a previous a