`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`HTC CORPORATION, HTC AMERICA, INC., and
`LG ELECTRONICS, INC.,
`Petitioners,
`
`v.
`
`
`
`PARTHENON UNIFIED MEMORY
`ARCHITECTURE LLC,
`Patent Owner
`
`Case IPR No.: IPR2015-01500, -1501, -01502
`U.S. Patent Nos. 7,321,368, 7,777,753, & 7,542,045
`
`REPLY DECLARATION OF HAROLD S. STONE, PH.D.
`
`
`
`
`
`
`
`
`
`
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. Cover
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`IPR2015-01500, -1501, -1502
`
`
`
`Reply Decl. of Dr. Stone
`
`TABLE OF CONTENTS
`
`INTRODUCTION ........................................................................................... 1
`I.
`SHARED MEMORY V. DEDICATED MEMORY ...................................... 2
`II.
`III. BOWES AND VIDEO DECODING .............................................................. 3
`IV. DSP’S FOR COMPRESSION AND DECOMPRESSION ............................ 6
`V. DEDICATED DSP MEMORY ..................................................................... 22
`VI. BLOCK READ AND BLOCK WRITE TO SHARED MEMORY ............. 32
`VII. ARBITER GRANTS BUS ACCESS AND GRANTS ACCESS TO
`SHARED MEMORY AT THE SAME TIME .............................................. 33
`A.
`The Bowes DSP does not have to write to main memory when
`it is granted access to the bus. ............................................................. 33
`The DSP cannot monopolize the bus when it is granted access
`to the bus. ............................................................................................ 38
`Arbiter Access to a Bus to Control Access to Memory ...................... 47
`C.
`VIII. COMPATIBILITY OF BOWES AND MPEG ............................................. 49
`A.
`Bowes’ disclosure of a video decoder. ................................................ 49
`B.
`Advantages of a Shared Memory. ....................................................... 50
`C.
`Bowes’ arbitration scheme is compatible with the MPEG
`standard. .............................................................................................. 51
`Bowes’ arbitration scheme can decode images in real time, even
`with the presence of a watchdog timer. ............................................... 54
`It Would Have Been Obvious To Combine Bowes With MPEG. ...... 56
`E.
`IDLE STATE (’753 PATENT) ..................................................................... 58
`
`B.
`
`D.
`
`IX.
`
`
`-ii-
`
`Petitioners HTC and LG - Exhibit 1032, p. ii
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`I, Harold S. Stone, Ph.D., declare as follows:
`
`I.
`
`INTRODUCTION
`1.
`
`I am the Harold S. Stone who has previously submitted declarations in
`
`these three proceedings (Exs. 1030 in each proceeding). The terms of my
`
`engagement, my background, qualifications and prior testimony, and the legal
`
`standards and claim constructions I am applying are set forth in my previous
`
`declarations. I offer this declaration in reply to the testimony of Prof. Thornton
`
`provided in each proceeding (Exs. 2009). Because Prof. Thornton’s testimony and
`
`the issues it raises are substantially identical between proceedings, I intend to reply
`
`to his testimony in each proceeding in parallel. In forming my opinion, I have
`
`considered the materials noted in my previous declarations in these proceedings, as
`
`well as the following additional materials:
`• Ex. 1033 — U.S. Patent No. 5,682,484 (“Lambrecht ’484”)
`• Ex. 1034 — U.S. Patent No. 5,375,068 (“Palmer”)
`• Ex. 1035 — U.S. Patent No. 5,557,538 (“Retter”)
`• Ex. 1036 — K. Konstantinides and V. Bhaskaran, “Recent
`Developements in the Design ofImage and Video Processing ICs,”
`Chapter 2 - VLSI Signal Processing Technololgy, Kluwer Academic
`Press, 1994
`• Ex. 1037 — Deposition Transcript of Dr. Mitchell A. Thornton, Ph.D.
`(June 17, 2016)
`
`1
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 1
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`• Ex. 1038 — Information technology – Generic Coding of Moving
`Pictures and Associated Audio Information: Systems, ISO/IEC 13818-
`1:1996 (1996) (“MPEG-2 Standard”)
`• Ex. 1039 — Srinath V. Ramaswamy and Gerald D. Miller, “Efficient
`Implementation of the Two Dimensional Discrete Cosine Transform
`for Image Coding applications on the DSP96002 Processor,” Proc. of
`the Midwest Conf. on Circuits and Systems, (IEEE 1993)
`
`II.
`
`SHARED MEMORY V. DEDICATED MEMORY
`2.
`
`Prof. Thornton writes:
`
`29. Typically, a decoder requires its own dedicated memory. For
`instance, traditional MPEG decoders require a 2 Mbyte dedicated
`memory which is utilized during the decoding process. This dedicated
`memory is necessary to allow the decoder to decode images in real-
`time without dropping frames which would result in a deterioration of
`the video quality at the receiver. This prior art implementation is
`shown, for example, in Figure 1c of the `368 Patent.
`
`[IPR2015-01500, Ex. 2009 at ¶29; see also IPR2015-01501, Ex. 2009
`at ¶29; IPR2015-01502, Ex. 2009 at ¶29].
`
`There is no support for this opinion. On the contrary, the Petitioners
`
`3.
`
`have cited to various prior art references that disclose MPEG decoders based on
`
`shared memory and which do not require dedicated memory. [See, e.g., S.
`
`Rathnam et al., “An Architectural Overview of the Programmable Multimedia
`
`Processor, TM-1,” IEEE Proceedings of COMPCON ’96, pp. 319-326 (1996)
`
`2
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 2
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`(“Rathnam”) (Ex. 1005); U.S. Patent No. 5,774,676, Figs 3 & 4 (“Stearns”) (Ex,
`
`1007); U.S. Patent No. 5,797,028 (“Gulick 028”) (Ex. 1023); U.S. Patent No.
`
`5,432,900 (“Rhodes”) (Ex. 1028); see also U.S. Patent No. 5,682,484 (“Lambrecht
`
`’484”) (Ex. 1032). ]
`
`III. BOWES AND VIDEO DECODING
`4.
`
`Prof. Thornton also writes:
`
`The word “video” is only mentioned four times in Bowes. [Bowes,
`1:34; 1:37; 1:41; 6:16]. The first three times the term “video” is used
`in conjunction with a description of related art and the fourth time, the
`term “video” is used in reference to a NuBus peripheral bus video
`controller and not in reference to a processing application. The words
`“decode” or “decoding” never appear in Bowes.
`
`44. Instead, Bowes specifically teaches that the DSP in the preferred
`embodiment is suitable for audio processing, image signal processing,
`speech processing, and modem emulation. [Bowes Pat., 1:48-49; 6:32-
`37]. Bowes does not state that the DSP is suitable for video
`compression and decompression applications such as the
`implementations of the MPEG Standard. A POSA would recognize
`that audio processing, speech processing and modem emulation are
`clearly distinct from video compression and decompression. The same
`is true with respect to “image processing.”
`
`
`3
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 3
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`[IPR2015-01500, Ex. 2009 at ¶¶43-44; ; see also IPR2015-01501 at
`¶46-47; IPR2015-01502 at ¶45-46]
`
`
`5.
`
`However, Bowes also discloses video conferencing in one of those
`
`references to “video.” In particular, in a section entitled “Description of Related
`
`Art,” Bowes states: “The emerging technologies will allow for collaboration at a
`
`distance such as by video conferencing.” [Ex. 1003 at 1:39-40] Collaborative
`
`video conferencing requires an exchange of video streams in real time, as a person
`
`of skill would understand. [See, e.g., U.S. Patent No. 5,375,068 (“Palmer”) (Ex.
`
`1034) at Abstract (“For full featured video teleconferencing, users require both an
`
`audio communications path and a real time visual communication path
`
`synchronized to the audio path.”)] Indeed, a person of ordinary skill would also
`
`understand that compression and decompression of such video streams would
`
`greatly enhance such a system, and that MPEG compression would be one obvious
`
`prior art approach. [See id. at 7:23-30 (“Video compression techniques can greatly
`
`enhance the effective frame rate of the video teleconference especially over lower
`
`data transfer rate services such as ISDN. Standard compression methods such as
`
`JPEG (Joint Photographic Experts Group), MPEG (Motion Picture Experts Group)
`
`and Px64 can be implemented using a dedicated hardware subsystem 37 which
`
`have recently become available.”)]
`
`4
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 4
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`6. Moreover, later in that same section Bowes refers to the previously
`
`identified technologies, including video conferencing, as “aspects of real-time
`
`information processing.” [Ex. 1003 at 1:42-44] He also states that his invention
`
`“relates to a computer architecture in which multiple processing units share a
`
`common memory bus and support real-time applications” [Ex. 1003 at 1:19-22],
`
`and that an object of his invention is to permit a scheme in which various system
`
`components receive “sufficient bandwidth over the memory bus to provide for
`
`real-time isochronous data processing,” id. at 2:67-3:2. He also explains that “[t]he
`
`scheme is implemented such that the DSP is provided with sufficient bandwidth to
`
`perform real-time digital signal processing using the system’s dynamic random
`
`access memory (DRAM) and not requiring the incorporation of an expensive block
`
`of static random access memory (SRAM).” Id. at 4:55-60.
`
`7.
`
`One of ordinary skill in the art in the mid-1990s would therefore
`
`understand, upon reading Bowes, that Bowes’s invention was directed to the field
`
`of real-time processing, that such processing included the processing of audio,
`
`image, and video data, and that therefore video conferencing was part of the
`
`technological context in which Bowes’ invention was useful and intended to be
`
`used. A person of skill would also have known that Bowes could be used with
`
`video encoding for transmission at the source of a stream and video decoding,
`
`5
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 5
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`including decompression, at the stream destination. Because of the 100 to 1
`
`compression in the data rate of the video data stream offered by technologies such
`
`as MPEG, a person of skill would understand that MPEG compression would
`
`facilitate real-time video conferencing and would enhance collaboration at a
`
`distance. The MPEG standard was a known standard on the priority dates of the
`
`’368, ’753, and ’045 patents.
`
`IV. DSP’S FOR COMPRESSION AND DECOMPRESSION
`8.
`
`Prof. Thornton claims that DSPs for image processing and those for
`
`compression/decompression are different. For support, Prof. Thornton suggests
`
`that floating-point operations and fixed point operations are distinguishing features
`
`that determine whether or not a DSP is suitable for one or the other application.
`
`[IPR2015-01500, Ex. 2009 at ¶¶46-47; see also IPR2015-01501, Ex. 2009 at ¶¶49-
`
`50; IPR2015-01502, Ex. 2009 at ¶¶48-49] Prof. Thornton’s opinion is that
`
`compression/decompression necessarily requires fixed point computation. My
`
`deposition points out that floating-point arithmetic will also be satisfactory for
`
`decoding. [Ex. 2006 at 203:8 to 205:7] I provide additional explanation below as
`
`to why Professor Thornton’s opinion in this regard is incorrect.
`
`6
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 6
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`9.
`
`Prof. Thornton asserts that the ATT DSP3210 is a floating-point DSP,
`
`and claims that floating-point arithmetic is unsuitable for decoding MPEG
`
`decoding.
`
`47. That the DSP (20) of Bowes is not suitable for video compression
`and decompression is further evident from the fact that Bowes states
`that in a preferred embodiment, the DSP (20) of Bowes is the AT&T
`DSP3210. [Bowes, 6:28-30].Such a DSP is not suitable for MPEG
`video decoding because it is a floating point DSP. [Ex. 2003, at 1].
`Specifically, the AT&T DSP3210 utilizes a floating-point Data
`Arithmetic Unit (DAU) that “is the primary execution unit for signal
`processing algorithms.
`
`[IPR2015-01500, Ex. 2009 at ¶47 (emphasis added); see also
`IPR2015-01501, Ex. 2009 at ¶50; IPR2015-01502, Ex. 2009 at ¶49]
`
`10. This opinion diverts attention unnecessarily to a particular DSP unit,
`
`the AT&T DSP3210. Bowes teaches that one can use any “off the shelf” DSP and
`
`merely identifies the AT&T chip as an example. “The DSP 20 may be an off-the-
`
`shelf DSP such as the AT&T DSP3210.” [Ex. 1003 at 2:22-23] The suitability of
`
`this particular device is secondary as compared to how one of skill in the art would
`
`understand the teaching of Bowes. Specifically, a person of skill would understand
`
`Bowes to be pointing out that any available DSP could potentially be used in the
`
`7
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 7
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`system of Bowes. In the specific context of the Bowes/MPEG combination, such a
`
`DSP would be an integrated circuit capable of decoding MPEG data streams, and
`
`therefore hardware and/or software that translates such data to audio and video.
`
`Several such DSP’s were available in the prior art, and a person of skill could have
`
`employed them in the system of Bowes, [Ex. 1006 at Fig. 1; Ex. 2008 at 2; U.S.
`
`Patent No. 5,557,538 (“Retter”) (Ex. 1035) at Fig. 1; K. Konstantinides and V.
`
`Bhaskaran, “Recent Developements in the Design ofImage and Video Processing
`
`ICs,” Chapter 2 - VLSI Signal Processing Technololgy, Kluwer Academic Press,
`
`1994 (Ex. 1036), at 25], since Bowes is using his DSP 20 in the usual, known
`
`manner in which DSPs were used. For example, one prior art DSP specifically
`
`designed for decompressing MPEG video data was the Texas Instruments MVP.
`
`See Robert J. Gove, “The MVP: A Highly-Integrated Video Compression Chip,”
`
`DCC ’94, Data Compression Conference, Mar. 29-31, 1994, pp. 215-224 (Ex.
`
`1006).
`
`11.
`
`In his deposition, Professor Thornton places great weight on the
`
`phrase “such as” in the sentence quoted above, suggesting that it necessarily limits
`
`the range of DSPs referred to by Bowes. [Ex. 1037 (Thornton Depo. Tr.) at 62:21-
`
`66:2] A person of ordinary skill in the art would not read the sentence in that
`
`manner. Bowes’ invention is not a novel DSP, but a system including a novel
`
`8
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 8
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`arbitration scheme that permits “off the shelf” DSPs to be used for real-time
`
`processing of audio and video data. [Ex. 1003 at 2:51-3:54] His identification of
`
`an “off the shelf” DSP commercially available from a company (AT&T) other than
`
`his employer (Apple) confirms that his claimed novelty lays in the arbitration
`
`scheme, not in any particular aspect of the DSP. Thus, a person of skill would read
`
`Bowes statement at 2:22-23 to mean that any DSP capable of performing the
`
`processing needed in the particular technological environment in which his
`
`invention was being used would be suitable.
`
`12. Prof. Thornton also opines, incorrectly, that the MPEG format is
`
`incompatible with floating-point and that a fixed-point DSP is necessary for MPEG
`
`decoding. The support offered is based on the need to convert from integer to
`
`floating-point representation in order to use a floating-point DSP.
`
`50. Moreover, floating-point processors are incompatible with
`MPEG decoding due to the format of the encoded and decoded
`MPEG video data as in the MPEG standard and the H.262
`specifications. The video data as per the standard and specification is
`not in the form of floating point values and would require
`conversions to floating-point prior to decoding and a conversion back
`to its initial format after decoding. A POSA would recognize that
`these conversions would incur additional processing delay that would
`be otherwise unnecessary if a fixed point DSP were used. Floating-
`
`9
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 9
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`point values require that the significand fall within a numerical range
`between 1 and 2, with the exponent field used to indicate the actual
`location of the binary-point. Thus, all input encoded MPEG values
`would need to be converted to this floating-point format before
`processing, and the resulting decoded values would have to be
`converted back from this range to appropriate pixel representations
`that are collections of fixed-point values. Such numerous
`conversions to and from floating point data could cause the real
`time decoding constraint to be unrealizable.
`
`[IPR2015-01500, Ex. 2009 at ¶50 (emphasis added); see also
`IPR2015-01501, Ex. 2009 at ¶53; IPR2015-01502, Ex. 2009 at ¶52]
`
`13. This ignores several important factors. (1) The MPEG data stream
`
`encodes data as signed magnitude integers of variable precision. [Ex. 1004 at 31,
`
`40-48] These data have to be converted to a scaled two’s complement format for
`
`fixed-point calculations or to a floating-point format for floating-point
`
`calculations, so that a format conversion is required whether the DSP uses
`
`floating-point or fixed-point representations. (2) Floating point DSPs provide fast
`
`conversion between integer and floating-point representations. These would be
`
`performed once when inputting integer data to the DSP and once again when
`
`storing final output data from the DSP. Intermediate data can be held in floating-
`
`point format during the computation. (3) Calculations for decoding MPEG
`10
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 10
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`necessarily involve computations for the inverse DCT, which involve cosine
`
`coefficients that have fractional values. When these computations are done in
`
`fixed point arithmetic, the intermediate results of the computations could lose
`
`precision and cause errors. It is common practice to use extended fixed-point
`
`precision and/or additional fixed-point scaling operations to maintain numerical
`
`accuracy when doing fixed-point inverse DCTs in order to maintain numerical
`
`accuracy. Such extended precision or additional scaling adds computational
`
`overhead for fixed-point calculations, but is absent in floating-point calculations.
`
`14. Floating-point calculations in a DSP maintain precision without the
`
`overhead of additional instructions to protect from loss of precision. The DSP3210
`
`floating point reduces overhead of scaling, normalization, and overflow, all of
`
`which have been ignored by Prof. Thornton. This is because floating point
`
`operations use normalized operands so that the operands are represented at full
`
`precision, and the results of floating-point operations are scaled automatically to
`
`full precision by a postnormalization. This reduces the likelihood of overflow, and
`
`simplifies code for handling overflow. “The use of floating-point arithmetic in the
`
`DSP3210 simplifies application development because it eliminates the effects of
`
`scaling, normalization, and overflow (which complicate development when using
`
`a fixed-point device) and facilitating computation by eliminating the scaling and
`
`11
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 11
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`sources of errors due to scaling and quantization.” [Ex. 2001 at 1-1, emphasis
`
`added] In view of the problems with maintaining precision associated with fixed-
`
`point DCT calculations, it is improper to conclude that a particular floating-point
`
`DSP device is less suitable for MPEG decoding than a fixed-point DSP device.
`
`Prof. Thornton cites alleged overhead in using the floating-point device while
`
`ignoring possibly greater overhead in using the fixed-point device to maintain
`
`precision.
`
`15. As evidence of the scaling overhead, Kitson [Ex. 2008] discloses that
`
`the authors modified the PA RISC processor to introduce instructions that combine
`
`scaling with addition in order to reduce overhead. These are the instructions that
`
`do both shift and add, where the shift is a scaling operation. “Note that the add,
`
`subtract, shift-left-and-add and shift-right-and-add are used intensively within the
`
`IDCT and thus led to additional speedup of the IDCT task due to architectural
`
`enhancements compared with the algorithmic enhancements performed on the
`
`IDCT as described earlier.” [Ex. 2008 at 5] Thus, the authors of Prof. Thornton’s
`
`Ex. 2008, two HP computer scientists, felt compelled to modify their RISC
`
`processor to specifically address the very fixed-point scaling issue Prof. Thornton
`
`ignores in his analysis.
`
`12
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 12
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`16. Prof. Thornton also ignores the overhead of converting MPEG data
`
`into fixed point format and back to integer format. MPEG input data uses signed
`
`magnitude representation, which has to be converted to two’s complement and
`
`scaled, such as to 16 bits for the 16-bit addition hardware of the HP RISC
`
`processor identified by Prof. Thornton. [Ex. 2008 at 5; see also MPEG Standard
`
`(Ex. 1004) at 31 (converting the variable “level” in signed magnitude
`
`representation from the bitstream to two’s complement representation for
`
`computation)] A person of ordinary skill in the art would understand that Prof.
`
`Thornton’s alleged overhead for converting to floating-point is comparable to the
`
`overhead for converting input data to fixed-point two’s complement representation
`
`at the precision required for calculation. For example, a person of skill in the art
`
`would know to combine the inverse quantization operation with the conversion to
`
`floating-point representation for the IDCT when executed on a floating-point DSP,
`
`just as Kitson discloses combining of the inverse quantization operation with the
`
`fixed-point implementation of the IDCT. [Ex. 2008 at 4] One of skill in the art
`
`could, for example, use a lookup-table for a one-instruction conversion of a signed
`
`magnitude integer directly into the floating-point representation for use by the
`
`DSP3210. The entries in the tables at 54 to 58 of Ex. 1004 can be used to create
`
`such lookup tables. Those entries show the coded input data and the corresponding
`
`13
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 13
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`signed magnitude representation. For each of the coded input data entries, a
`
`lookup table would hold the corresponding representation as a floating-point
`
`number instead of signed magnitude number, and thereby completely avoid the
`
`cost of conversion from coded input to a scaled 16-bit two’s complement fixed-
`
`point integer required for the hardware disclosed in Ex. 2008. For a floating-point
`
`IDCT algorithm, the cosine coefficients for the IDCT need never be converted to
`
`floating point because they can be initially stored in a floating-point format.
`
`17.
`
`In summary, the main point with respect to the issue of floating-point
`
`vs. fixed point is Prof. Thornton holds an incorrect belief that fixed point
`
`arithmetic has less overhead than floating-point arithmetic for purposes of
`
`decoding MPEG data. The belief, as indicated above, is based on an assertion
`
`about the alleged overhead for the floating-point algorithm that does not take into
`
`account consideration of the overhead incurred in the fixed-point algorithm.
`
`18. Prof. Thornton also incorrectly believes the DSP3210 is not suitable
`
`for decoding MPEG in the context of the Bowes/MPEG combination. [Ex. 2009 at
`
`¶46 “There are additional reasons why a POSA would recognize that a DSP used
`
`for image processing is not suitable for video compression and decompression.”]
`
`The DSP3210 is a high performance computation engine, suitable for numerically
`
`intensive algorithms like those required to decode MPEG. Within the DSP3210,
`
`14
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 14
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`the DAU is the primary execution unit for signal processing algorithms. This unit
`
`contains a 32-bit floating-point multiplier, a 40-bit floating-point adder, four 40-bit
`
`accumulators, and a control register (dauc). The multiplier and adder work in
`
`parallel to perform 16.7 million computations per second, yielding 33.4 MFLOP
`
`performance. [See, e. g., Ex. 2001 at 1-1 and 3-3] The multiplier and adder each
`
`produce one floating-point result per instruction cycle. The DAU contains a four-
`
`stage pipeline (fetch, multiply, accumulate, write). Thus, in any instruction cycle,
`
`the DAU may be processing four different instructions, each in a different stage of
`
`execution. All normalization is done automatically, so the result in the accumulator
`
`is always fully normalized.
`
`The DAU supports two floating-point formats: single precision (32-
`bit) and extended single precision (40-bit). Extended single precision
`provides eight additional mantissa guard bits. Postnormalization logic
`transparently shifts binary points and adjusts exponents to prevent
`inaccurate rounding of bits when the floating-point numbers are
`added or multiplied. This eliminates concerns such as scaling and
`quantization error.
`
`[Ex. 2001 at 3-3 (emphasis added)]
`
`19. Additionally, the DSP3210 supports fast conversion between floating-
`
`point and fixed-point representations.
`
`15
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 15
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`Single instruction, data type conversions are done in hardware in
`the DAU, reducing overhead required to do these conversions. The
`DAU performs data type conversions between the DSP3210 32-bit
`floating-point format and IEEE P754 standard 32-bit floating-point,
`16- and 32-bit integer, 8-bit unsigned, u-law and A-law formats.
`
`[Ex. 2001 at 3-3, emphasis added]
`
`This permits input data to be in integer or fixed-point format, and output data to be
`
`delivered in integer or fixed point format, whereas all intermediate data
`
`computations can be carried out by the DSP unit in floating-point format to take
`
`advantage of its high speed, parallelism of operation, and elimination of scaling
`
`and quantization errors. The DSP3210 is therefore perfectly suitable for decoding
`
`MPEG-1 data streams.
`
`20. One of skill in the art on reading Bowes would understand that
`
`Bowes’ disclosure is not tied to a particular DSP. “The DSP 20 may be an off-the-
`
`shelf DSP such as the AT&T® DSP321O.” [Ex. 1003 at 2:22-23] Nevertheless,
`
`even if one of skill in the art were influenced to use the DSP3210 instead of
`
`another off-the-shelf DSP, Prof. Thornton offers specious evidence as to why the
`
`DSP3210 is unsuitable for MPEG decoding.
`
`Specifically, a POSA would have been aware that 16.7 MIPS is
`insufficient for real time video decoding under the MPEG Standard
`16
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 16
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`which requires 524 MIPS (632 MIPS if YUV to RGB conversion is
`included) [Ex. 2008, at 8], more than 31 times the maximum amount
`of processing allowable by the DSP3210.
`
`[IPR2015-01500, Ex. 2009 at ¶51; see also IPR2015-01501, Ex. 2009
`at ¶54; IPR2015-01502, Ex. 2009 at ¶53]
`
`21. There are several errors in this statement.
`
`a.
`
`Prof. Thornton’s own Ex. 2008 discloses that the PA RISC
`
`processor operates at 80 MHz, performs four 16-bit operations at a time, and
`
`decodes MPEG1 at 33.1 frames per second [Ex. 2008 at 6]. If the PA RISC
`
`can decode MPEG1 with its technology, this contradicts Prof. Thornton’s
`
`belief that the 524 MIPS bandwidth is required to do MPEG decoding,
`
`because the PA RISC is far too slow to execute at a rate of 524 MIPS.
`
`Consequently, the DSP3210 does not to have to execute at 524 MIPS in
`
`order to decode an MPEG datastream at 30 frames per second. The actual
`
`MIPS requirement for MPEG1 is on the order of 30 MIPS, as indicated
`
`immediately below. Both the PA RISC and the DSP3210 can process at 30
`
`MIPS.
`
`b.
`
`The MPEG standard cited in Ex. 2008 (MPEG-2) is not the
`
`MPEG standard combined with Bowes. The alleged MIPS requirement of
`
`17
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 17
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`524 MIPS is for the MPEG-2 standard, [Ex. 2008 at 7 and 8] which includes
`
`various bitrates. An example of a “lower bitrate” is 27.1 Mbps for a 6 MHz
`
`CATV channel. [Information technology – Generic Coding of Moving
`
`Pictures and Associated Audio Information: Systems, ISO/IEC 13818-
`
`1:1996, ITU-T Rec. H.222.0 (1996) (“MPEG-2 Standard” (Ex. 1038) at 92.]
`
`Examples of systems with higher supported bitrates of 128.2 Mbps and 34.7
`
`Mbps are also cited in the MPEG-2 Standard (Ex. 1038) at 92. The MPEG
`
`standard combined with Bowes according to the Board’s institution decision
`
`is MPEG-1, ISO IEC/11172-2, 1992; see Paper 14 at 5, 23-24; which cites a
`
`maximum MPEG bitrate to be about 1.5 Mbps. [Ex. 1004 at iv “This part of
`
`ISO/IEC 11172 specifies a coded representation that can be used for
`
`compressing video sequences to bitrates around 1,5 Mbit/s.”] The ratio of
`
`the MPEG-2 bitrate of 27.1 to the MPEG bitrate of 1.5 is 18.1. When the
`
`524 MIPS requirement for MPEG-2 is reduced by that ratio to estimate the
`
`MPEG MIPS requirement, the result is 30.0 MIPS.
`
`c.
`
`The requirement of 524 MIPS in Ex. 2008 at 8 is specious
`
`because it is not itself explained in Ex. 2008. Rather, Ex. 2008 cites to
`
`Konstantinides (Ex. 1036). However, Konstantinides does not have the cited
`
`rate of 524 MIPS. Instead, Konstantinides has a MIPS requirement table
`
`18
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 18
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`similar to that of Ex. 2008 at 8, but with a MIPS requirement for MPEG-2 of
`
`198 MIPS. [Ex. 1036 at 35] Using 198 MIPS as the MPEG-2 requirement
`
`instead of the 524 MIPS requirement, and reducing by the MPEG-2 to
`
`MPEG-1 bitrate ratio of 18.1 yields an estimate for the MIPS requirement
`
`for MPEG 1 of 10.9 MIPS. This is easily within the performance capability
`
`of the HP PA RISC processor and the DSP3210 processor.
`
`d.
`
`Prof. Thornton misstates the MIPS capability of the DPS3210.
`
`It is 33.4 MIPS, not 16.7 MIPS. In particular, I note that the DSP3210 has a
`
`Data Arithmetic Unit (“DAU”), which can execute 16.7 million floating
`
`point additions and 16.7 million floating-point multiplications per second in
`
`parallel, thereby performing 33.4 million floating-point operations per
`
`second. [Ex. 2001 at 3-3]. Moreover, the DSP3210 also includes a Control
`
`Arithmetic Unit (“CAU”) that operates in parallel with the DAU and can
`
`itself execute 16.7 MIPS, for example for address calculation. [Ex. 2001 at
`
`3-3] The published numbers of the DSP3210 therefore exceed the
`
`processing power one would estimate is required for the decoding of MPEG-
`
`1 data based on Prof. Thornton’s exhibits.
`
`19
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 19
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`e.
`
`In sum, even taking Prof. Thornton’s numbers at face value, a
`
`person of skill would expect the Bowes system employing a DSP3210 to
`
`adequately decode MPEG-1 data streams.
`
`f.
`
`Indeed, Konstantinides (Ex. 1036) further confirms the errors of
`
`Prof. Thornton’s analysis, since it identifies various MPEG encoding and
`
`decoding devices commercially available in 1994, [Ex, 1036 at 36],
`
`demonstrating that a person of skill could have implemented the
`
`Bowes/MPEG combination before mid-1996 using “off the shelf” DSP
`
`components.
`
`g. Moreover, I note that the actual MIPs required to decode any
`
`particular video data stream would vary based on the resolution of the
`
`underlying video. The claims of the patent do not require decoding of any
`
`particular resolution. Prof. Thornton’s evidence is therefore both unreliable
`
`and irrelevant.
`
`h.
`
`Prof. Thornton’s Ex. 2008 provides an additional “off-the-
`
`shelf” device, the enhanced PA RISC processor, that one of the skill in the
`
`art would consider as a substitute for the DSP3210 when combining MPEG
`
`Standard with Bowes. This particular choice is known to be able to decode
`
`MPEG at 33.1 frames per second.
`
`20
`
`
`
`Petitioners HTC and LG - Exhibit 1032, p. 20
`HTC and LG v. PUMA, IPR2015-01501
`
`
`
`
`
`i.
`
`If there were any doubt that a floating-point DSP is a
`
`reasonable device to decode MPEG, in the prior art, Ramaswamy and
`
`Miller’s “Efficient Implementation of the Two Dimensional Discrete Cosine
`
`Transform for Image Coding applications on the DSP96002 Processor”
`
`(IEEE 1993) (Ex. 1039) discloses a fast floating-point implementation based
`
`on an off-the-shelf DSP96002 processor that can be used to perform MPEG
`
`inverse DCT computations. “These fast routines can be utilized in image
`
`coding applicat