PATENT DATE
`
`mt ~':n29111
`35,; 22
`
`.....r.
`_......_'.._40_
`1.0.4.
`,
`.
`._.
`
`FILINODATEICLASS
`
`5'.
`
`5812789 I
`
`I‘
`EXAIIIIIER
`
`E . 1%.»? H. 1
`
`:_
`
`
`
`
`
`ISSUEI::LAssIF1c.ii1'IbN'
`
`UTILITY
`
`SERIAL
`NUMBER
`
`
`sanw.m.I||B:n
`
`Pflliianpriofllrainiulod
`‘IIIlDGI1I9olIC§MII|l|ll
`
`El:1
`
`I
`
`'
`
`PA!‘-US or APPLIcA11oN
`FILED SEPARATELY
`NOTICE OF ALLO INCE HMLED
`
`E.
`
`l
`—_'!~:.%te9‘§
`.
`_
`: P
`
`‘
`
`P
`
`WARNING: The irflormaflm cladoeed health may ha raflflmad. Unamharlzzad disclosure may be prohibited
`byihe Unilfld Stains C-ode ‘nfle 35.Sec1:1uns122,1B1arId365. Possesslunoutside IheU.S.
`Patent 8: Tradarrlaflt Offlce is les1I1cDed1o'auflmI1zpd emploqmos and caniraclnla only.
`
`
`
`
`
`Page 1 of 280
`
`Samsung Exhiltiitl ‘1--002
`Petitioners HTC & LG — Exhibit 1002, p. 1
`
`Petitioners HTC & LG - Exhibit 1002, p. 1
`
`

`
`;°E'I;Ei~.l1—'.—APP.|.lCA“i'l0N’\'- °*‘”P”°”"%’b:F°”%%”‘;'5"§E9gj51
`_
`_i
`. {?IrrIALs .:...._fag
`'
`
`L
`
`-
`
`4:-.,=_::
`
`'
`
`--
`
`..
`
`}
`
`03702911
`..
`
`_
`'
`
`Date
`
`1"
`
`at";
`
`_
`
`$’_
`
`I \
`
`--5..
`
`‘I
`
`1.
`
`
`
`I
`
`T 7%-’f#I‘FL!5'”/%'$'¥?I$?.??I?l
`
`.
`
`-:
`
`
`
`\._. ______.
`
`name
`Entered
`
`_
`
`-
`
`_
`
`.a-*;a=a=t=.
`
`'.. 5°
`
`:aaza.na:;:::2Vzsss:-=22
`
`
`
`‘F"°1'33titioners HTC & LG — Exhibit 100
`
`Petitioners HTC & LG - Exhibit 1002, p. 2
`
`

`
`_
`
`.'
`
`E
`
`‘\-
`
`.
`
`_
`
`Staple Issue Slip Here;
`
`POSITION
`
`DD}5
`
`5
`
`t1
`
`FmalQ'\9.jIngmal3
`
`Ha
`
`nnnugg\
`—~-*--_'....l—1—-I.~
`“SW
`azmfi||!fl!: L
`
`
`
`
`
`
`I
`
`Itu
`
`SYMBOLS
`
`Rejected
`IK
`: ‘'(‘:ftIrnugh numhavr-$t‘)V:!?;'r?\‘:’:2‘|ar‘ejd
`Restricted
`elected
`.
`cc
`o‘>2+
`. Apps
`._ Obtected
`
`80
`
`33
`
`*
`
`E
`A’
`8
`
`B
`
`It
`
`I
`mmAm
`(001
`
`II
`
` an0
`
`(LE7 lNSlDE)Petitioners HTC & LG — EXh1
`
`Petitioners HTC & LG - Exhibit 1002, p. 3
`
`

`
`
`
`Page 4 of 280
`
`Petitioners HTC & LG 4 Exhibit 100
`
`Petitioners HTC & LG - Exhibit 1002, p. 4
`
`

`
` SEARCH NOTES
`
`Page 5 o 280
`
`(F“G“T °”TSI55’titi0ners HTC & LG — EX
`
`Petitioners HTC & LG - Exhibit 1002, p. 5
`
`

`
`1.5.4
`
`an/'7n2911
`PATENT APPLICATION SERIAL NO.
`
`U.S. DEPARTMENT OF COMMERCE
`P$TENT AND TRADEMARK OFFICE
`
`FEE RECORD SHEET_
`
`.
`
`340 BA 19-1353 1D!2B/96 03702911
`133.com
`34011 101
`
`PTO-1556
`(5:37)
`
`Page 6 of 280
`
`Petitioners HTC & LG — Exhibit 100
`
`Petitioners HTC & LG - Exhibit 1002, p. 6
`
`

`
`tit./"3’i31:;’$"‘1 1
`
`ElE©'fifi3©llN|fl@§
`
`THE ASSISTANT COMMISSIONER FOR PATENTS
`
`Washington, D.C. 20231
`
`Re:
`
`lnventor(s): Raul Z. Diaz and Jefferson E. Owen
`
`For: Video and/or Audio Decompression and/or Compression Device that Shares a
`Memory Interface
`
`Our File No: 96-S-11
`
`Sir:
`
`Enclosed with this transmittal letter are:
`
`(1)
`(2)
`(3)
`(4)
`(5)
`(6)
`
`Subject patent application with Declaration and Power of Attorney;
`Five (5) sheets of informal drawings;
`Certificate of Express Mail;
`Assignment and Recordation Cover Sheet;
`Check in the amount of $1,396.00;
`Return postcard which we would appreciate your date stamping and returning to
`us upon receipt;
`
`The total filing fee has been calculated as follows:
`
`Basic fee
`Recordation of Assignment
`29 claims in excess of 20
`2 independent claim in excess of 3
`Total filing fee
`
`=
`
`$ 750.00
`40.00
`=
`638.00
`=
`156.00
`=
`= $1.584.00
`
`I authorize the Commissioner to charge any additional fees which may be required, or
`credit any overpayment to Account No. 19-1353. A duplicate copy of this sheet is
`enclosed.
`
`Irena Lager EL
`
`Reg. No. 39, 60
`
`SGSA-THOMSON Microelectronics, Inc. -1310 Electronics Drive - Carrolllon, TX 75006-5039
`Telephone (214) 466-6000, Telex 730643
`
`STKDZ-CAR
`
`Page 7 Jiéd
`
`00
`
`Petitioners HTC & LG — Exhibit 100
`
`Petitioners HTC & LG - Exhibit 1002, p. 7
`
`

`
`llii /"7l'I‘.3Q‘I 1 TATES PATENT AND TRADEIIARK OFFICE
`
`Docket No. 95-54311
`
`In Re Application of:
`
`Raul Z. Diaz and Jefferson E. Owen
`
`For: Video andlor Audio Decompression andior compression Device that Shares '
`a Memory Interface
`
`CERTIFICATE OF EXPRESS IlIA|I_._
`
`"EXPRESS MAIL" NO. EG9«‘-l73'62259US
`
`Date of Deposit: August 23, 1996
`
`I hereby certify that this paper or fee is being deposited with the United States
`Postal Service "Express Mail Post Offioe to Addressee" service under 37 CFR 1.10 on
`the date indicated above and is addressed to the Assistant Commissioner for Patents,
`
`Box Patent Application. Washington, DC. 20231.
`
`Signature of person mailing paper or fee
`
`Page 8 of
`
`Petitioners HTC & LG — Exhibit 100
`
`Petitioners HTC & LG - Exhibit 1002, p. 8
`
`

`
`
` ;
`
`ECOMPRESSION AND/OR COMPRESSION
`DEVICE THAT SHARES A MEMORY INTERFACE
`
`
`Cross-reference to Related Applications
`
`This application contains some text and drawings in common with pending
`
`U.S. Patent Applications entitled: "Video and/or Audio Decompression and/or
`
`Diaz,
`Compression Device that Shares a Memory" by Jefferson E. Owen, Raul
`_
`N 0g[7a;,¢;;o ”’z\¢L‘l wax
`0%)3é-/"1
`and Osvaldo Colavin
`l,
`( %$-M2), and has the
`
`same effective filing date and ownership as the present application, and to that
`
`extent
`
`is related to the present application, which is incorporated herein by
`
`reference.
`
`Babkground
`
`The present invention relates to the field of electronic systems having a video
`
`and/or audio decompression and/or compression device, and is more specifically
`
`directed to sharing a memory interface between a video and/or audio decompression
`
`and/or compression device and another device contained in the electronic system.
`
`The size of a digital representation of uncompressed video images is
`dependent on the resolution, and color depth of the image. A movie composed of
`
`a sequence of such images, and the audio signals that go along with them, quickly
`becomes large enough so thatiuncompressed such a movie typically cannot fit
`
`entirely onto conventional recording medium, such as a CD.
`
`It is also typically
`
`{/
`
`;
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 1
`
`Page 9 of 280
`
`Petitioners HTC & LG — Exhibit 100
`
`Petitioners HTC & LG - Exhibit 1002, p. 9
`
`

`
`now prohibitively expensive to transmit such a movie uncompressed. \
`
`It is therefore advantageous to compress video and audio sequences before
`
`they are transmitted or stored. A great deal of effort is being expanded to develop
`
`5
`
`systems to compress these sequences. There are several coding standards currently
`
`used that are based on the discrete cosine transfer algorithm including MPEG-1,
`
`MPEG-2, H.261, and H.263.
`
`(MPEG stands for "Motion Picture Expert Group",
`
`a committee of the International Organization for Standardization, ISO.) The
`
`MPEG-1, MPEG-2, H.261, and H.263 standards are decompression protocols that
`
`10
`
`describe how an encoded bitstream‘ is to be decoded. The encoding can be done
`
`in any manner, as long as the resulting bitstream complies With the standard.
`
`Video and/or audio compression devices (hereinafter encoders) are used to
`
`encode the video and/or audio sequence before it is transmitted or stored. The
`
`15
`
`resulting bitstream is decoded by a video and/or audio decompression device
`
`(hereinafter decoder) before the video and/or audio sequence is displayed.
`
`However, a bitstream can only be decoded by a decoder if it complies to the
`
`standard used by the decoder. To be able to decode the bitstream on a large
`
`number of systems it is advantageous to encode the video and/or audio sequences
`
`20
`
`to comply to a well accepted decompression standard. The MPEG standards are
`
`currently well accepted standards for one Way, communication. H.261, and H.263
`
`are currently well accepted standards for video telephony.
`
`Once decoded the images can be displayed on an electronic system dedicated
`
`25
`
`to displaying video and audio, such as television or digital video disk (DVD)
`
`player, or on electronic systems where image display is just one feature of the
`
`system, such as a computer. A decoder needs to be added to these systems to
`
`l,,/'‘;
`
`/’-\.
`
`1
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 2
`
`Page 10 of 280
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 10
`
`

`
`allow them to display compressed sequences, such as received images and
`
`associated audio, or ones taken from a storage device. An encoder needs to be
`
`added to allow the system to compress video and/or audio sequences,
`
`to be
`
`transmitted or stored. Both need to be added for two Way communication such as
`
`5
`
`video telephony.
`
`A typical decoder, such as an MPEG decoder 10 shown in Figure la,
`
`contains video decoding circuitry 12, audio decoding circuitry 14, a microcontroller
`
`16, and a memory interface 18. The decoder can also contain other circuitry
`
`10
`
`depending on the electronic system the decoder is designed to operate in. For ‘
`
`example, when the decoder is designed to operate in a typical television the decoder
`
`will also contain an on screen display (OSD) circuit.
`
`Figure 1b shows a better decoder architecture, used in the STi352O and
`
`15
`
`STi352OA MPEG Audio/MPEG-2 Video Integrated Decoder manufactured by SGS-
`
`THOMSON Microelectronics. The decoder has a register interface 20 instead of
`
`a rnicrocontroller.
`
`The
`
`register
`
`interface 20 is coupled to an external
`
`microcontroller 24. The use of a register interface 20 makes it possible to tailor the
`
`decoder 10 to the specific hardware the decoder 10 interfaces with or change its
`
`20
`
`operation without having to replace the decoder by just reprogramming the register
`
`interface.
`
`It also allows the user to replace the microcontroller 24, to upgrade or
`
`tailor the microcontroller 24 to a specific use, by just replacing the microcontroller
`
`and reprogramming the register interface 20, without having to replace the decoder
`
`10.
`
`25
`
`The memory interface 18 is coupled to a memory 22. A typical MPEG
`
`decoder 10 requires 16 Mbits of memory to operate in the main profile at main
`
`”
`
`-
`K
`PagL44+7>i280
`
`SGS-THOMSON Microelectronics Inc.
`
`96-S-1 1
`Page 3
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 11
`
`

`
`level mode (MP at ML). This typically means that the decoder requires a 2Mbyte
`
`memory. Memory 22 is dedicated to the MPEG decoder 10 and increases the price
`
`of adding a decoder 10 to the electronic system.
`
`In current technology the cost of
`
`this additional dedicated memory 22 can be a significant percentage of the cost of
`
`5
`
`the decoder.
`
`An encoder also requires a memory interface 18 and dedicated memory.
`
`Adding the encoder to an electronic system again increases the price of the system
`
`by both the price of the encoder and its dedicated memory.
`
`10
`
`A goal
`
`in the semiconductor industry is to reduce the die area of an
`
`integrated circuit device for a given functionality. Some advantages of reducing the
`
`die area is the increase in the number of the die that can be manufactured on same
`
`size silicon wafer, and the reduction in price per die resulting therefrom. This
`
`15
`
`results in both an increase in volume and reduction in price of the device.
`
`Many of the functional circuits described above for Figure la and Figure 1b
`
`take up a lot of die space. However, each of them is needed to make the respective
`
`decoder operate.
`
`20
`
`Figure 1c shows a computer 25 containing a decoder 10, a main memory 168
`
`and other typical components such as a modern 199, and graphics accelerator 188.
`
`The decoder 10 and the rest of the components are coupled to the core logic chipset
`
`190 through a bus 170. The bus is typically a PCI (peripheral component interface)
`
`25
`
`or ISA (industry standard architecture) bus, and each component contains an
`
`appropriate interface for interfacing with the bus.
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 4
`
`~.__\.‘
`
`Page 12 of
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 12
`
`

`
`When any component needs access to the memory 168 either to read from
`
`or write to the main memory 168, it generates a request which is placed on the bus
`
`26. When the request is a write the data to be written is also placed on the bus 26.
`
`The request is processed in the core logic chipset 190 and the data is then either
`
`written to or read from the main memory 168. When data is read from the main
`
`memory 168 the data is now placed on the bus and goes to the component that
`
`requested the read.
`
`There are typically many components in the computer systems that may
`require access to the main memory 168, and are typically all coupled to the
`same bus 174, or possibly several buses 1'?O,J_l~%8- connected together by a PCI
`bridge 192, if there are not enough connectors on one bus to accommodate all of
`
`‘
`
`the peripherals. However, the addition of each bus is very expensive. Each request
`
`is typically processed according to a priority scheme. The priority scheme is
`
`typically based on the priority given to the device and the order in which the
`
`requests are received. Typically,
`
`the priority scheme is set up so no device
`
`monopolizes the bus, starving all of the other devices. Good practice suggesgthat
`no device on the bus require more than approximately 50% of the bus's bandwidth.
`
`The minimum bandwidth required for the decoder 10 can be calculated based
`
`on the characteristics and desired operation of the decoder. These characteristics
`
`include the standard to which the bitstream is encoded to comply with, whether the
`
`decoder is to operate in real time, to what extent frames are dropped, and how the
`
`images are stored. Additionally, the latency of the bus that couples the decoder to
`
`the memory should be considered.
`
`F Ifthe decoder does not operate in real time the decoded movie would stop
`
`I
`
`'
`
`Page 13 9f 280
`
`SGS-THOMSON Microelectronics Inc.
`96-S-ll
`Page 5
`
`Petitioners HTC & LG — Exhibit 10
`
`7
`
`Petitioners HTC & LG - Exhibit 1002, p. 13
`
`

`
`periodically between images until the decoder can get access to the memory to
`
`process the next image. The movie may stop quite often between images and wait.
`
`To reduce the minimum required bandwidth and still operate in real time, the
`
`decoder 10 may need to dro‘g:£:’ah1a;eeTsIf the decoder 10 regularly does not decode
`every frame then it may not need to stop between images. However, this produces
`
`very poor continuity in the images. This is problematic with an image encoded to
`
`the MPEG-1 or MPEG-2 standards’, or any standards that uses temporal
`
`compression.
`
`In temporal (interpicture) compression some of the images are
`
`decoded based on previous images and some based on previous and future images. _
`
`Dropping an image on which the decoding of other images is based is unacceptable,
`
`and will result in many poor or even completely unrecognizable images.
`
`The computer can also contain both a decoder and encoder to allow for video
`
`telephony, as described above.
`
`In this case not operating in real time would mean
`
`that the length of time between the occurrence of an event, such as speaking, at one
`
`end of the conversation until the event
`
`is displayed at
`
`the other end of the
`
`conversation is increased by the time both the encoder and then the decoder must
`
`wait to get access to the bus and the main memory. Not being able to operate in
`
`real time means that there would be gaps in the conversation until the equipment
`
`can catch up. This increases the time needed to have a video conference, and
`
`makes the conference uncomfortable for the participants.
`
`One widely used solution to allow a component in a computer system to
`
`operate in real time is to give the component its own dedicated memory. Thus, as
`
`shown in Figure 1c, the decoder 10 can be given its own dedicated memory 22,
`
`with a dedicated bus 26 to connect the decoder 10 to its memory 22.
`
`The
`
`3‘
`
`3"
`
`l
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 6
`
`Page 14 of 280
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 14
`
`

`
`dedicated memory 22,
`
`its controller and the pins to control
`
`this memory
`
`significantly increase the cost of adding a decoder 10 to the computer.
`
`/I
`
`/ V
`K‘ /
`
`SGS-THOMSON Microelectronics Inc.
`96-8-11
`Page 7
`
`Page 15 of 280
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 15
`
`

`
`Summary of the Invention
`
`The present application discloses an electronic system that contains a first
`
`device and video and/or audio decompression and/or compression device capable
`
`5
`
`of operating in real
`
`time. Both the first device and the video and/or audio
`
`decompression and/or compression device require a memory interface. The video
`
`and/or audio decompression and/or compression device shares a memory interface
`
`and the memory with the first device. In the preferred embodiment of the invention
`
`the shared memory interface contains an arbiter. The arbiter and DMA engines of
`
`10
`
`the video and/or audio decompression and/or compression device and of the first .
`
`device are configured to arbitrate between the two devices when one of them is
`
`requesting access to the memory. This allows the use of one memory interface to
`
`control
`
`the access of both the video and/or audio decompression and/or
`
`compression device and the first device to the memory.
`
`15
`
`When the video and/or audio decompression and/or compression device used
`
`in an electronic system, such as a computer, already containing a device that has
`
`a memory interface the video and/or audio decompression and/or compression
`
`device can share that memory interface and the memory of the device and the
`
`20
`
`memory interface and memory of the video and/or audio decompression and/or
`
`compression device can be eliminated. Eliminating this memory interface reduces
`
`the die area without changing the critical dimensions of the device. Therefore
`
`increasing the volume and reducing the cost of the decoder or encoder. Eliminating
`
`the memory greatly reduces
`
`the cost of adding the video and/or audio
`
`25
`
`decompression and/or compression device to the electronic system while not
`
`requiring the video and/or audio decompression and/or compression device to be
`
`connected to the system bus, allowing the video and/or audio decompression and/or
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 8
`
`;/
`'
`
`T
`52
`‘.
`3
`I
`
`Page 16 of 280
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 16
`
`

`
`compression device to operate in real time.
`
`An advantage of the present invention is significant cost reduction due to the
`
`fact that the video and/or audio decompression and/or compression device does not
`
`5
`
`need its own dedicated memory but can share a memory with another device and
`
`still operate in real time.
`
`Another significant advantage of the present invention is that the die space
`
`needed for the video and/or audio decompression and/or compression device is_
`
`10
`
`smaller because the memory interface on the video and/or audio decompression
`
`and/or compression device is eliminated.
`
`A further advantage of the present invention is that the video and/or audio
`
`decompression and/or compression device can share the memory of the device with
`
`15
`
`which it is sharing the memory interface more efficiently.
`
`Another advantage of the present invention is that the cost of producing a
`
`video and/or audio decompression and/or compression device is reduced because
`
`the memory interface on the video and/or audio decompression and/or compression
`
`20
`
`device is eliminated.
`
`Another advantage of the present invention is that the video and/or audio
`
`decompression and/or compression device can be monolithically integrated into the
`
`first device and no extra packaging or pins are needed for the video and/or audio
`
`25
`
`decompression and/or compression device, and no pins are needed for the first
`
`device to connect to the video and/or audio decompression and/or compression
`
`device, saving pins on both devices and producing a better connection between the
`
`:,
`/‘
`
`f/
`
`,5
`I
`3
`l V
`Page_._Z__if 280
`
`SGS-THOMSON Microelectronics Inc.
`96-S—11
`Page9
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 17
`
`

`
`two devices.
`
`Other advantages and objects of the invention will be apparent to those of
`
`ordinary skill in the art having reference to the following specification together with
`
`5
`
`the drawings.
`
`I
`
`I
`E
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 10
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 18
`
`

`
`Brief Description of the Drawings
`
`Figire la and 1b are electrical diagrams, in block form, of prior art decoders.
`
`5
`
`Figure 1c is an electrical diagram,
`
`in block form, of a computer system
`
`containing a decoder according to the prior art.
`
`Figure 2 is an electrical diagram, in block form, of an electronic system
`
`containing a device having a memory interface and an encoder and decoder.
`
`10
`
`Figure 3 is an electrical diagram,
`
`in block form, of a computer system‘
`
`containing a core logic chipset designed for the CPU to share a memory interface
`
`with an encoder and decoder.
`
`15
`
`Figure 4 is an electrical diagram,
`
`in block form, of a computer system
`
`containing a graphics accelerator designed to share a memory interface with an
`
`encoder and/or decoder.
`
`,
`
`3
`
`f,,,._;__
`
`’
`
`N‘
`
`SGS-THOMSON Microelectronics Inc.
`
`96-S-11
`Page 11
`
`Page 19 of 280
`
`Petitioners HTC & LG — Exhibit 1002
`
`Petitioners HTC & LG - Exhibit 1002, p. 19
`
`

`
`Detailed Description of the Preferred Embodiment
`
`Figure 2 shows an electronic system 40 containing a first device 42 having
`
`access to a memory 50 through a memory interface 48, and a decoder 44 and
`
`5
`
`encoder 46, having access to the same memory 50 through the same memory
`
`interface 48. First device 42 can be a processor, a core logic chipset, a graphics
`
`accelerator, or any other device that requires access to the memory 50, and either
`
`contains or is coupled to a memory interface. Any parts common to Figures 1
`
`through 4 are indicated using the same numbering system.
`
`In the preferred
`
`10
`
`embodiment of the invention, electronic system 40 contains a first device 42, a
`decoder 44, an encoder 46, a memory interface 48, and a memory 50. Although, I
`
`either the decoder 44 or encoder 46 can be used in the decoder/encoder 45 without
`
`the other.
`
`For ease of reference, a video and/or audio decompression and/or
`
`compression device 45 will hereinafter be referred to as decoder/encoder 45. The
`
`15
`
`decoder/encoder 45 may be a single device, or cell on an integrated circuit, or may
`
`be two separate devices, or cells in an integrated circuit.
`
`In the preferred
`
`embodiment of the invention, the first device 42, decoder/encoder 45, and memory
`
`interface 48 are on one integrated circuit, however,
`
`they can be on separate
`
`integrated circuits in any combination.
`
`20
`a,_
`
`C'tFcuc'.+ IQ-
`The decoder 44 includes a video decodingk_12—-ei-reuif and an audio decoding
`circuit 14, both coupled to a register interface 20. The decoder 44 can be either a
`
`video and audio decoder, just a video, or just an audio decoder.
`
`If the decoder 44
`
`is just a video decoder it does not contain the audio decoding circuitry 14. The
`
`25
`
`audio decoding can be performed by a separate audio codec coupled to the first
`
`device 42, or through software‘.
`
`In the preferred embodiment of the invention,
`
`when the decoder/encoder 45 is in a system containing a processor and is coupled
`‘/————..
`
`)
`‘,/. J>-'H.
`2
`
`SGS-THOMSON Microelectronics Inc.
`96-S-11
`Page 12
`
`Page 20 of 280
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 20
`
`

`
`to the processor, the audio decoding is performed in software. This frees up space
`
`on the die without causing significant delay in the decoding. If the audio decoding
`
`is performed in sofiware, the processor should preferably operate at a speed to
`allow the audio decoding to be performed in real time without starving other
`
`components of the system that may need to utilize the processor. For example,
`
`currently software to perform AC-3 audio decoding takes up approximately 40%
`
`of the bandwidth of a 133 MHz Pentium. The encoder 46 includes a video
`
`encoding circuit 62 and an audio encoding circuit 64, both coupled to a register
`
`- interface 20. The encoder 46 can be either a video and audio encoder, just a video,
`
`or just an audio encoder.
`
`If the encoder 46 is just a video encoder, it does not
`
`contain the audio encoding circuitry 64. The audio encoding can be performed by
`a separate audio codec coupled to the first device 42, or through software.
`In the ‘
`
`preferred embodiment ofthe invention, when the decoderfencoder 45 is in a system
`
`containing aspgogtisspgiaid Fi’sr_'tp‘ the processor, the audio encoding is
`performed in software?-Presenting the sarfiétdvantages of freeing up space on the
`die without gausing significant delay in the encoding. The register interfaces 20
`
`of the decoder 44 and encoder 46 are coupled to a processor.
`
`The decoder 44 and encoder 46 are coupled to the direct memory access
`
`(DMA) engine 52. The decoder and encoder can be coupled to the same DMA
`
`engine as shown in Figure 2, or each can have its own DMA engine, or share a
`
`DMA engine with another device. When the decoderfencoder 45 are two separate
`
`devices or cells, decoder 44 and encoder 46 can still be coupled to one DMA
`
`engine 52. When the decoderfencoder is one device or is one cell on an integrated
`
`circuit, the DMA engine 52 can be part of the decoderfencoder 45, as shown in
`
`Figure 2. The DMA engine 52 is coupled to the arbiter 54 of the memory interface
`
`48.
`,
`
`.
`
`E /
`i
`I
`J‘
`
`,
`
`‘
`
`i
`
`I
`
`Page 21 of 280
`
`SGS—Tll0MSON.MiI:rnelectronlcsIne.
`95-5-11
`Page 13
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 21
`
`

`
`The first device 42 also contains a DMA engine 60. The DMA engine 60
`
`of the first device 42 is coupled to the arbiter 54 of the memory interface 48. The
`
`arbiter is also coupled to the refresh logic 58 and the memory controller 56. The
`
`memory interface 48 is coupled to a memory 50. The memory controller 56 is the
`access as
`control logic that generates the address the memory interface 48 access-in
`Au
`
`P 5
`
`memory 50 and the timing of the burst cycles.
`
`In current technology, memory 50 is typically a DRAM. However, other
`
`types of memory can be used. The refresh logic 58 is needed to refresh the
`
`10
`
`DRAM. However, as is known in the art, if a different memory is used, the refresh
`
`logic 58 may not be needed and can be eliminated.
`
`The decoder/encoder 45 is coupled to the memory 50 through devices,
`
`typically a bus 70, that have a bandwidth greater than the bandwidth required for
`
`15
`
`the decoder/encoder 45 to operate in real time. The minimum bandwidth required
`
`for the decoder/encoder 45 can be calculated based on the characteristics and
`
`desired operation of the decoder, including the standard to which the bitstream is
`
`encoded to comply with, whether the decoder/encoder 45 is to operate inreal time,
`
`to what extent frames are dropped, and which images are stored. Additionally, the
`
`20
`
`latency of the bus 70 that couples the decoder/encoder 45 to the memory 50 should
`
`be considered.
`
`A goal
`
`is to have the decoder/encoder 45 operate in real time without
`
`dropping so many frames that it becomes noticeable to the human viewer of the
`
`25
`
`movie. To operate in real time the decoder/encoder 45 should decoder and/or
`
`encode images fast enough so that any delay in decoding and/or encoding cannot
`
`be detected by a human viewer. This means that the decoder/encoder 45 has a
`
`SGS-THOMSON Microelectronics Inc.
`96-S-ll
`Page 14
`
`f /““*“"'
`
`“
`
`;
`
`l
`
`.
`
`Page 22 of 280
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 22
`
`

`
`required bandwidth that allows the decoder/encoder 45 to operate fast enough to
`
`decode the entire image in the time between screen refreshes, which is typically
`
`1/30 of a second, with the human viewer not being able to detect any delay in the
`
`decoding and/ or encoding. To operate in real time the required bandwidth should
`
`be lower than the bandwidth of the bus.
`
`In order not
`
`to starve the other
`
`components on the bus, i.e. deny these components access to the memory for an
`
`amount of time that would interfere with their operation, this required bandwidth
`
`should be less the entire bandwidth of the bus. Therefore a fast bus 70 should be
`
`used. A fast bus 70 is any bus whose bandwidth is equal to or greater that the
`
`required bandwidth. There are busses, in current technology, including the ISA ‘
`
`bus, whose bandwidth is significantly below the bandwidth required for this.
`
`In the preferred embodiment of the invention the decoder/encoder 45 is
`
`coupled to the memory 50 through a fast bus 70 that has a bandwidth of at least
`
`the bandwidth required for the decoder/encoder 45 to operate in real
`
`time, a
`
`threshold bandwidth.
`
`Preferably the fast bus 70 has a bandwidth of at
`
`least
`
`approximately twice the bandwidth required for the decoder/encoder 45 to operate
`
`in real time.
`
`In the preferred embodiment the fast bus 70 is a memory bus,
`
`however any bus having the required bandwidth can be used.
`
`The decoder/encoder 45 only requires access to the memory during operation.
`
`Therefore, when there is no need to decode or encode, the first device 42, and any
`
`other devices sharing the memory 50 have exclusive access to the memory; and can
`use the entire bandwidth of the fast bus 70.
`
`In the preferred embodiment, even during decoding and encoding the
`
`decoder/encoder 45 does not always use the entire required bandwidth. Since the
`
`I,’
`,'
`I
`
`/A»
`
`SGS-THOMSON Microelectronics Inc.
`96-8-11
`Page 15
`
`age23*of 280 s
`
`I
`
`J P
`
`_
`
`Petitioners HTC & LG — Exhibit 1002,
`
`Petitioners HTC & LG - Exhibit 1002, p. 23
`
`

`
`fast bus 70 has a bandwidth a little less than twice the required bandwidth the
`
`decoder/encoder 45 uses at most 60% of the bandwidth of the fast bus 70.
`
`The required bandwidth is determined based on the size and resolution of the
`
`5
`
`image, and the type of frame (I, P, or B).
`
`In the preferred embodiment the
`
`decoder/encoder typically will be using less than 40% of the bandwidth of the fast
`
`bus 70. This frees up the remaining bandwidth to be used by the other devices the
`
`decoder/encoder 45 is sharing the memory 50 with.
`
`10
`
`The decoder/encoder 45 can decode a bitstream formatted according to any _
`
`one or a combination of standards.
`
`In the preferred embodiment of the invention
`
`the decoder/encoder 45 is a multi-standard decoder/encoder capable of decoding and
`
`encoding sequences formatted to comply to several well accepted standards. This
`
`allows the decoder/encoder 45 to be able to decode a large number of video and/or
`
`15
`
`audio sequences. The choice of which standards the decoder/encoder 45 is capable
`
`of decoding bitstreams formatted to and of encoding sequences to comply to is
`
`based on the desired cost, efficiency, and application of the decoder/encoder 45.
`
`In the preferred embodiment, these standards are capable of both intrapicture
`
`20
`
`compression and interpicture compression.
`
`In intrapicture compression the
`
`redundancy within the image is eliminated.
`
`In interpicture compression the
`
`redundancy between two images are eliminated and only the difference information
`
`is transferred. This requires the decoder/encoder 45 to have access to the previous
`
`or fixture image that contains information needed to decode or encode the current
`
`25
`
`image. These precious and/or future images need to be stored then used to decode
`
`the current image. This is one of the reasons the decoder/encoder 45 requires
`
`access to the memory, and requires a large bandwidth. The MPEG-1 and MPEG-2
`
`SGS-THOMSON Microelectronics Inc.
`
`96-S-11
`Page 16
`
`5;
`i.
`
`II
`
`age of
`
`Petitioners HTC & LG — Exhibit 1002,
`
`.1
`
`I E
`
`Petitioners HTC & LG - Exhibit 1002, p. 24
`
`

`
`standards allow for decoding based on both previous images and/or fixture images.
`
`Therefore for a decoder/encoder 45 capable of operating in real time to be able to
`
`comply with the MPEG-1 and MPEG-2 standards it should be able to access two
`
`images, a previous and a fL1lZL1I'6 image, fast enough to decode the current image in
`
`the 1/30 of a second between screen refreshes.
`
`An MPEG environment is asymmetrical; there are much fewer encoders than
`
`decoders. The encoders are very difficult and expensive to manufacture and the
`
`decoders are comparatively easy and cheap. This encourages many more decoders
`
`than encoders, with the encoders in centralized locations, and decoders available
`
`such that every end user can have a decoder. Therefore, there are many receivers
`
`but few transmitters.
`
`For video telephony and teleconferencing each end user has to be able to
`
`both receive and transmit. H.261, and H.263 are currently well accepted standards
`
`for Video telephony. An encoder that can encode sequences to comply to the
`
`H.261 and H.263 standards is less complicated, having a lower resolution and lower
`_
`S'l'a;.nclGLr‘<:lS
`osss lo 1 )1
`frame rate than an encoder that complies to the MPEG-1 or MPEG-”
`-Possibly making the quality of the decoded images somewhat lower than those
`
`from an encoder that complies with the MPEG-1 or MPEG-2 standards. Such an
`
`encoder, since it should be inexpensive and operate in real time,
`
`is al