`_______________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`_______________
`
`LG ELECTRONICS, INC.
`Petitioner
`
`v.
`
`ATI TECHNOLOGIES ULC
`Patent Owner
`_______________
`
`Case: IPR2015-00321
`
`Patent 7,095,945
`_______________
`
`DECLARATION OF DAN SCHONFELD, PHD
`
`
`
`
`
`LG Ex. 1004, pg 1
`
`
`
`I.
`
`II.
`
`Table of Contents
`A.
`Background And Qualifications ................................................................... 3
`B.
`Information Considered................................................................................ 6
`LEGAL STANDARDS...............................................................................7
`A.
`Person Of Ordinary Skill In The Art........................................................... 8
`B.
`Anticipation..................................................................................................... 8
`C.
`Obviousness.................................................................................................... 9
`D.
`Claim Construction ...................................................................................... 11
`TECHNOLOGY BACKGROUND ........................................................ 12
`A.
`Multiplexers and Demultiplexers................................................................ 12
`B.
`Packetized Data ............................................................................................ 14
`C.
`MPEG Streams............................................................................................. 14
`1.
`MPEG Program Stream (PS) .......................................................... 16
`2.
`MPEG-2 Transport Stream (TS) .................................................... 17
`Time Shifting................................................................................................. 19
`D.
`Clock Recovery And Timing Information................................................ 19
`E.
`Coded Multimedia Data And Timing Information.................................. 20
`F.
`Real-Time Applications ............................................................................... 21
`G.
`III. THE ’945 PATENT.................................................................................23
`A.
`Background Of The ’945 Patent................................................................. 23
`B.
`Prosecution History (Ex. 1002)................................................................. 25
`C.
`Challenged Claims ........................................................................................ 27
`D.
`Claim Construction ...................................................................................... 27
`PRIOR ART ANALYSIS .........................................................................28
`A.
`U.S. Patent No. 6,233,389 (“Barton”) ....................................................... 28
`1.
`Claims 18 and 21 Are Anticipated By Barton ............................... 28
`U.S. Patent No. 6,397,000 (“Hatanaka”)................................................... 52
`1.
`Claim 18 Is Obvious In View Of Hatanaka .................................. 54
`2.
`Claim 21 Is Anticipated By Hatanaka ............................................ 68
`U.S. Patent No. 6,591,058 (“O’Connor”) ................................................ 83
`1.
`Claim 18 Is Obvious Over Hatanaka In View of O’Connor...... 84
`U.S. Patent No. 5,521,922 (“Fujinama”) ................................................... 89
`
`IV.
`
`B.
`
`C.
`
`D.
`
`1
`
`LG Ex. 1004, pg 2
`
`
`
`Claim 21 Is Obvious Over Fujinami .............................................. 91
`1.
`SUPPLEMENTATION ........................................................................ 106
`
`V.
`
`2
`
`LG Ex. 1004, pg 3
`
`
`
`I.
`1.
`
`INTRODUCTION
`My name is Dan Schonfeld, and I have been retained by the law firm of Mayer
`
`Brown LLP on behalf of LG Electronics, Inc. as an expert in the relevant art.
`
`2.
`
`I have been asked to provide my opinions and views on the materials I have
`
`reviewed in this case related to Ex. 1001, U.S. Patent No. 7,095,945 (“the ’945
`
`Patent”) (“the patent-at-issue”), and the scientific and technical knowledge regarding
`
`the same subject matter before and for a period following the date of the first
`
`application for the patent-at-issue was filed.
`
`3.
`
`I am being compensated for my work on this case at my standard consulting
`
`rate of $500 per hour.
`
`I am also being reimbursed for all incurred expenses. My
`
`compensation has not influenced any of my opinions in this matter and does not
`
`depend on the outcome of this proceeding or any issue in it.
`
`4.
`
`5.
`
`My opinion and underlying reasoning for this opinion is set forth below.
`
`A.
`
`Background And Qualifications
`
`I am currently a Professor in the Department of Electrical and Computer
`
`Engineering at the University of Illinois at Chicago. I have been elected Fellow of the
`
`Institute of Electrical and Electronics Engineers (“IEEE”) as well as Fellow of the
`
`International Society for Optics and Photonics (“SPIE”). I have also been elected
`
`University Scholar of the University of Illinois. A complete list of my publications,
`
`professional activities, and honors that I have received is fully set forth in my
`
`curriculum vitae, attached hereto as Appendix A.
`
`3
`
`LG Ex. 1004, pg 4
`
`
`
`6.
`
`I received my B.S. degree in Electrical Engineering and Computer Science from
`
`the University of California, Berkeley, California, and my M.S. and Ph.D. degrees in
`
`Electrical and Computer Engineering from The Johns Hopkins University, Baltimore,
`
`Maryland, in 1986, 1988, and 1990, respectively.
`
`7.
`
`In August 1990, I joined the Department of Electrical Engineering and
`
`Computer Science at the University of Illinois, Chicago, Illinois, where I am currently
`
`a Professor in the Departments of Electrical and Computer Engineering, Computer
`
`Science,
`
`and Bioengineering.
`
`I
`
`serve as Co-Director of
`
`the Multimedia
`
`Communications Laboratory (“MCL”) and member of the Signal and Image Research
`
`Laboratory (“SIRL”).
`
`I have also served as Director of the University-Industry
`
`Engineering Research Center (“UIERC”), formerly known as the Manufacturing
`
`Research Center (“MRC”), in the College of Engineering.
`
`8.
`
`I currently serve as Editor-in-Chief of the IEEE Transactions on Circuits and
`
`Systems for Video Technology. I have previously served as Deputy Editor-in-Chief
`
`of the IEEE Transactions on Circuits and Systems for Video Technology and Area
`
`Editor for special issues of the IEEE Signal Processing Magazine. I have also served
`
`as Associate Editor of the IEEE Transactions on Circuits and Systems for Video
`
`Technology, IEEE Transactions on Image Processing, and IEEE Transactions on
`
`Signal Processing. I also served on the editorial board of the IEEE Signal Processing
`
`Magazine, EURASIP Journal of Image and Video Processing, and Research Letters in
`
`Signal Processing. I have served as guest editor of numerous special issues in various
`
`4
`
`LG Ex. 1004, pg 5
`
`
`
`journals in the area of multimedia systems.
`
`9.
`
`I currently serve as Technical Program Chair of the IEEE International
`
`Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2018.
`
`I have
`
`served as General Co-Chair of the IEEE International Conference on Multimedia and
`
`Expo (ICME) 2012. I have also served as Chair of the IEEE Workshop on Video
`
`Mining 2008 and the SPIE Conference on Visual Communication and Image
`
`Processing 2007. I have also served on the organizing committees of various
`
`conferences including the IEEE International Conference on Image Processing
`
`1998 and 2012.
`
`10.
`
`I have authored and co-authored over 200 technical papers for various journals
`
`and conferences. I was co-author (with Carlo Giulietti and Rashid Ansari) of a paper
`
`that won the Best Paper Award at the ACM Multimedia Workshop on Advanced
`
`Video Streaming Techniques for Peer-to-Peer Networks and Social Networking 2010.
`
`I was also co-author (with Junlan Yang) of a paper that won the Best Student
`
`Paper Award at the IEEE International Conference on Image Processing 2007. I was
`
`also co-author (with Wei Qu) of a paper that won the Best Student Paper Award at
`
`the IEEE International Conference on Image Processing 2006. I was also co-author
`
`(with Nidhal Bouaynaya) of a paper that won the Best Student Paper Award in Visual
`
`Communication and Image Processing 2006. My publications in the area of image
`
`and video processing and communications date back to 1988. A list of my
`
`publications within the past ten years is included in Appendix A.
`
`5
`
`LG Ex. 1004, pg 6
`
`
`
`11.
`
`I was invited as a Plenary Speaker and Keynote Speaker to the International
`
`Conference on Intelligent Control and Information Processing (ICICIP) 2013 and
`
`International Conference on Brain Inspired Cognitive Systems
`
`(BICS) 2013,
`
`IEEE/EIT International Conference on Audio, Language, and Image Processing
`
`2010, the IEEE International Conference on Advanced Video and Signal-Based
`
`Surveillance 2009, and the ASME International Conference on Communications,
`
`Signals and Systems 1995 and 2001.
`
`12.
`
`I have served as Region 1-6 (North America) representative on the Chapters
`
`Committee of the IEEE Signal Processing Society. I have also served as Chairman of
`
`the IEEE Signal Processing Chicago Chapter. I have also served on the IEEE Image,
`
`Video and Multidimensional Signal Processing Technical Committee as well as the
`
`IEEE Multimedia Communications Technical Committee.
`
`I also serve on the
`
`American National Standards
`
`Institute (ANSI)/Underwriters Laboratory (UL)
`
`Standards Technical Panel (“STP”) on Multimedia Systems.
`
`13.
`
`Additional details of my education and work experience, awards and honors,
`
`and publications that may be relevant to the opinions I have formed are set forth in
`
`my curriculum vitae (see Appendix A). Additionally, I have consulted for several
`
`companies in the area of multimedia systems. A list of cases in which I have testified
`
`as an expert at trial or by deposition is attached hereto as Appendix B.
`
`Information Considered
`B.
`In addition to my general knowledge gained as a result of my education and
`
`14.
`
`6
`
`LG Ex. 1004, pg 7
`
`
`
`experience in this field, I have reviewed and considered, among other things, the ’945
`
`Patent, the prosecution history of the ’945 Patent, and the prior art of record.
`
`15.
`
`The full list of information that I have considered in forming my opinions for
`
`this Declaration is set forth throughout the Declaration and listed in the attached
`
`Appendix C.
`
`I.
`16.
`
`LEGAL STANDARDS
`In forming my opinions and considering the patentability of the claims of the
`
`’945 Patent, I am relying upon certain legal principles that counsel has explained to
`
`me.
`
`17.
`
`I understand that for an invention claimed in a patent to be found patentable, it
`
`must be, among other things, new and not obvious in light of what came before it.
`
`Patents and publications which predated the invention are generally referred to as
`
`“prior art.”
`
`18.
`
`I understand that in this proceeding the burden is on the party asserting
`
`unpatentability to prove it by a preponderance of the evidence. I understand that “a
`
`preponderance of the evidence” is evidence sufficient to show that a fact is more
`
`likely than not.
`
`19.
`
`I understand that in this proceeding, the claims must be given their broadest
`
`reasonable interpretation consistent with the specification. The claims after being
`
`construed in this manner are then to be compared to information that was disclosed
`
`in the prior art.
`
`7
`
`LG Ex. 1004, pg 8
`
`
`
`Person Of Ordinary Skill In The Art
`A.
`I have been informed that the claims of a patent are judged from the
`
`20.
`
`perspective of a hypothetical construct involving “a person of ordinary skill in the
`
`art.” The “art” is the field of technology to which the patent is related. I understand
`
`that the purpose of using a person of ordinary skill in the art’s viewpoint is objectivity.
`
`Thus, I understand that the question of validity is viewed from the perspective of a
`
`person of ordinary skill in the art, and not from the perspective of (a) the inventor, (b)
`
`a layperson, or (c) a person of extraordinary skill in the art. I have been informed that
`
`the claims of the patent-at-issue are interpreted as a person of ordinary skill in the art
`
`would have understood them in the relevant time period (i.e., when the patent
`
`application was filed or the earliest effective filing date).
`
`21.
`
`It is my opinion that a person of ordinary skill in the art relevant to the ’945
`
`Patent at the time of the filing of the patent would have had a bachelor’s degree in
`
`electrical engineering, computer engineering, computer science, or the equivalent, and
`
`two (2) years of work experience in the area of multimedia systems, including time
`
`shifting of video data.
`
`22.
`
`I understand that a “person of ordinary skill
`
`is also a person of ordinary
`
`creativity, not an automaton” and that would be true of anyone developing video
`
`time-shifting systems.
`
`B.
`
`Anticipation
`
`23.
`
`I understand that the following standards govern the determination of whether
`
`8
`
`LG Ex. 1004, pg 9
`
`
`
`a patent claim is “anticipated” by the prior art. I have applied these standards in my
`
`analysis of whether claims of the ’945 Patent were anticipated at the time of the
`
`invention.
`
`24.
`
`I understand that a patent claim is “anticipated” by a single prior art reference
`
`if that reference discloses each element of the claim in a single embodiment. A prior
`
`art reference may anticipate a claim inherently if an element is not expressly stated,
`
`but only if the prior art necessarily includes the claim limitations.
`
`25.
`
`I understand that the test for anticipation is performed in two steps. First, the
`
`claims must be interpreted to determine their meaning. Second, a prior art reference
`
`is analyzed to determine whether every claim element, as interpreted in the first step,
`
`is present in the reference.
`
`If all the elements of a patent claim are present in the
`
`prior art reference, then that claim is anticipated and is invalid.
`
`26.
`
`I understand that it is acceptable to examine extrinsic evidence outside the
`
`prior art reference in determining whether a feature, while not expressly discussed in
`
`the reference, is necessarily present within that reference.
`
`Obviousness
`C.
`I understand that a claim can be invalid in view of prior art if the differences
`
`27.
`
`between the subject matter claimed and the prior art are such that the claimed subject
`
`matter as a whole would have been “obvious” at the time the invention was made to a
`
`person having ordinary skill in the art.
`
`28.
`
`I understand that the obviousness standard is defined at 35 U.S.C. § 103(a). I
`
`9
`
`LG Ex. 1004, pg 10
`
`
`
`understand that a claim is obvious over a prior art reference if that reference,
`
`combined with the knowledge of one skilled in the art or other prior art references
`
`disclose each and every element of the recited claim.
`
`29.
`
`I also understand that
`
`the relevant
`
`inquiry into obviousness
`
`requires
`
`consideration of four factors:
`
`a.
`
`b.
`
`c.
`
`d.
`
`The scope and content of the prior art;
`
`The differences between the prior art and the claims at issue;
`
`The knowledge of a person of ordinary sill in the pertinent art; and
`
`Objective factors indicating obviousness or non-obviousness may be
`
`present in any particular case, such factors including commercial success of products
`
`covered by the patent claims; a long-felt need for the invention; failed attempts by
`
`others to make the invention; copying of the invention by others in the field;
`
`unexpected results achieved by the invention; praise of the invention by the infringer
`
`or others in the field; the taking of licenses under the patent by others; expressions of
`
`surprise by experts and those skilled in the art at the making of the invention; and that
`
`the patentee proceeded contrary to the accepted wisdom of the prior art.
`
`30.
`
`I understand that when combining two or more references, one should
`
`consider whether a teaching, suggestion, or motivation to combine the references
`
`exists so as to avoid impermissible hindsight.
`
`I have been informed that the
`
`application of the teaching, suggestion or motivation test should not be rigidly
`
`applied, but rather is an expansive and flexible test. For example, I have been
`
`10
`
`LG Ex. 1004, pg 11
`
`
`
`informed that the common sense of a person of ordinary skill in the art can serve as
`
`motivation for combining references.
`
`31.
`
`I understand that the content of a patent or other printed publication (i.e., a
`
`reference) should be interpreted the way a person of ordinary skill in the art would
`
`have interpreted the reference as of the effective filing date of the patent application
`
`for the ’945 Patent. I have assumed that the person of ordinary skill is a hypothetical
`
`person who is presumed to be aware of all the pertinent information that qualifies as
`
`prior art.
`
`In addition, the person of ordinary skill in the art makes inferences and
`
`creative steps. He or she is not an automaton, but has ordinary creativity.
`
`32.
`
`I have been informed that the application that issued as the ’945 Patent was
`
`filed on November 6, 2000. Thus, the relevant time period for determining what one
`
`of ordinary skill in the art knew is November 6, 2000.
`
`Claim Construction
`D.
`I have been informed that a claim subject to Inter Partes Review is given its
`
`33.
`
`“broadest reasonable construction in light of the specification of the patent in which
`
`it appears.” I have been informed that this means that the words of the claim are
`
`given their plain meaning from the perspective of one of ordinary skill in the art
`
`unless that meaning is inconsistent with the specification. I understand that the “plain
`
`meaning” of a term means the ordinary and customary meaning given to the term by
`
`those of ordinary skill in the art at the time of the invention and that the ordinary and
`
`customary meaning of a term may be evidenced by a variety of sources, including the
`
`11
`
`LG Ex. 1004, pg 12
`
`
`
`words of the claims, the specification, drawings, and prior art.
`
`34.
`
`I understand that
`
`in construing claims “[a]ll words in a claim must be
`
`considered in judging the patentability of that claim against the prior art.” (MPEP §
`
`2143.03, citing In re Wilson, 424 F.2d 1382, 1385 (CCPA 1970)).
`
`35.
`
`I understand that extrinsic evidence may be consulted for the meaning of a
`
`claim term as long as it is not used to contradict claim meaning that is unambiguous in
`
`light of the intrinsic evidence. Phillips v. AWH Corp., 415 F.3d 1303, 1324 (Fed. Cir.
`
`2005) (citing Vitronics Corp. v. Conceptronic, Inc., 90 F.3d 1576, 1583-84 (Fed. Cir. 1996)).
`
`I also understand that in construing claim terms, the general meanings gleaned from
`
`reference sources must always be compared against the use of the terms in context,
`
`and the intrinsic record must always be consulted to identify which of the different
`
`possible dictionary meanings is most consistent with the use of the words by the
`
`inventor. See, e.g., Ferguson Beauregard/Logic Controls v. Mega Systems, 350 F.3d 1327,
`
`1338 (Fed. Cir. 2003) (citing Brookhill-Wilk 1, LLC v. Intuitive Surgical, Inc., 334 F.3d
`
`1294, 1300 (Fed. Cir. 2003)).
`
`II. TECHNOLOGY BACKGROUND
`A. Multiplexers and Demultiplexers
`A multiplexer combines a plurality of signals into a multiplexed signal. A
`
`36.
`
`demultiplexer performs the reverse operation by extracting the plurality of signals
`
`12
`
`LG Ex. 1004, pg 13
`
`
`
`from the multiplexed signal.
`
`Some well known1 examples of signal multiplexing
`
`include: (a) frequency division multiple access (FDMA), where individual signals
`
`correspond to distinct frequency bandwidths in the multiplexed signal (e.g., radio and
`
`television signals); (b) time division multiple access (TDMA), where individual signals
`
`correspond to distinct time slots in the multiplexed signal (e.g., telephone signals); and
`
`(b) code division multiple access (CDMA), where individual signals correspond to
`
`distinct code sequences in the multiplexed signal (e.g., cellular signals).
`
`37. Multiplexing and demultiplexing is essential in multimedia data communication
`
`for
`
`the efficient
`
`representation,
`
`transmission, processing, and presentation of
`
`correlated multimedia streams. For example, sending a movie or television program
`
`to a viewer requires the transmission of an audio and a video stream as well as related
`
`control data (e.g., subtitles).
`
`The audio, video, and control data streams are
`
`multiplexed together into a multiplexed signal, and thus the receiver can extract
`
`correlated data streams and synchronize the presentation of the audio, video, and
`
`control signals.
`
`38.
`
`Similarly, broadcasting television signals to multiple viewers requires the
`
`simultaneous transmission of a plurality of television programs in a single multiplexed
`
`1 To be clear, when I refer to an element or concept as “well-known” or “known in
`
`the prior art,” I particularly mean to say that the element or concept was well-known
`
`as of November 6, 2000.
`
`13
`
`LG Ex. 1004, pg 14
`
`
`
`signal by combining the individual television programs (each of which is already a
`
`multiplexed signal representing audio, video and control signals) into a multiplexed
`
`signal that is broadcast to all viewers. The receivers of individual viewers are then
`
`instructed by the viewers to extract a specific television program from the multiplexed
`
`broadcast signal for presentation on the television set.
`
`Packetized Data
`B.
`Traditional communication was performed over circuit-switched networks. In
`
`39.
`
`such networks, signals are transmitted as uninterrupted data streams from the sender
`
`to the receiver. Examples of circuit-switched networks include analog multimedia
`
`communications such as radio, television, and telephone systems.
`
`40. With the emergence of the Internet in the late 1960s, an alternative data
`
`communication method was introduced, known as packet-switched networks.
`
`In
`
`these networks, data is fragmented into a sequence of data chunks. The individual
`
`data chunks are then encapsulated in a packet. The packet consists of the data chunk,
`
`referred to as the packet payload, as well as additional information that is typically
`
`located in a packet header. The packet header consists of various parameters for
`
`efficient data communication (e.g., packet identifier, sequence number, timestamp,
`
`cyclic redundancy check, etc.).
`
`MPEG Streams
`C.
`There are many different multimedia coding standards. For simplicity, I shall
`
`41.
`
`limit this discussion here to compression known as the Moving Picture Expert Group
`
`14
`
`LG Ex. 1004, pg 15
`
`
`
`(MPEG) compression standards. In particular, I will focus exclusively on aspects of
`
`the two earliest MPEG compression standards, known as MPEG-1 and MPEG-2.
`
`42. MPEG-1 was aimed at compression and storage on a compact disc (CD) and
`
`was released in 1993.
`
`It is comprised of three parts: video compression, audio
`
`compression, and systems. The audio compression part portion includes the well
`
`known MPEG-1 Audio Layer III audio compression format, better known as MP3.
`
`43. MPEG-2 targeted broadcast-quality television signals and was released in 1996.
`
`It was adopted for the representation of video signals on a digital video disc (DVD).
`
`MPEG-2 eventually emerged as a ubiquitous format for the representation of video
`
`signals and was adopted by the Advanced Television Systems Committee (ATSC),
`
`Digital Video Broadcasting (DVB), Integrated Services Digital Broadcasting (ISDB),
`
`and Blu-ray discs for the representation of high definition television (HDTV). It is
`
`comprised of multiple parts including video compression, audio compression, and
`
`systems.
`
`44.
`
`The parts of the MPEG-1 and MPEG-2 standards germane to our discussion
`
`are MPEG-1 Systems and MPEG-2 Systems, formally known as ISO/IEC 11172-1
`
`and ISO/IEC 13818-1, respectively. The systems part provides a container for the
`
`representation, multiplexing, and synchronization of audio and video data. There are
`
`two approaches to systems: Program Stream (PS) and Transport Stream (TS).
`
`45.
`
`Both MPEG PS and MPEG TS are used to represent coded audio and video
`
`streams. The basic representation of MPEG data is known as an elementary stream
`
`15
`
`LG Ex. 1004, pg 16
`
`
`
`(ES) and contains the raw coded audio and video data.
`
`46.
`
`The elementary audio and video streams are divided into variable length
`
`packets known as packetized elementary streams (PES). The PES includes a header
`
`that contains various parameters such as a stream identifier (SID) used to identify the
`
`stream.
`
`47.
`
`The PES also includes optional timing information such as the presentation
`
`time stamp (PTS) and decoding time stamp (DTS). The PTS provides timing
`
`information that allows for synchronization of the multiple PESs. For example, PTS
`
`allows for synchronization between audio and video streams.
`
`It is thus used to
`
`determine when to display and then discard a program segment.
`
`48.
`
`The DTS is needed because video frames are displayed and coded in different
`
`orders. More particularly, some video frames depend on the present or past, whereas
`
`other video frames depend on the future. It is therefore necessary to include a DTS
`
`which provides timing information that specifies when to decode a program segment.
`
`MPEG Program Stream (PS)
`1.
`Program Stream (PS) is used by MPEG-1, and a very similar extension of the
`
`49.
`
`MPEG-1 PS is used by MPEG-2, in a container format for storage of audio and video
`
`data in storage devices such as CDs and DVDs. MPEG-1 and MPEG-2 PS allow for
`
`random access of audio and video data from storage devices, thus allowing efficient
`
`trick play (e.g., fast-forward, rewind, pause, and stop).
`
`50.
`
`The MPEG PS combines multiple PESs (typically a single audio PES with a
`
`16
`
`LG Ex. 1004, pg 17
`
`
`
`single video PES) from a single program that share a common time base into a PS
`
`structure known as a multiplex. The combination of the multiple PESs is performed
`
`by interleaving through a process known as time-division multiplexing. The MPEG
`
`PS also includes timing information such as a system clock reference (SCR).
`
`51.
`
`The ’945 Patent describes the multiplexing of several packetized elementary
`
`streams into a program stream: “an elementary stream (ES) is a set of data generally
`
`consisting of compressed data from a single source, such as a video or audio source,
`
`with some
`
`additional
`
`ancillary data
`
`for
`
`identification,
`
`characterization and
`
`synchronization. ES streams are first packetized into either constant length or variable
`
`length Packetized Elementary Stream packets (PES packets) consisting of header and
`
`payload. Each PES packet header starts with start code (ox000001) followed with the
`
`stream id byte identifying type of ES underneath. PES packets from various
`
`elementary streams are merged together to form a program (service) with its own
`
`system time clock (STC). All ES component streams within one program are
`
`synchronized have periodic PTS stamps corresponding to the STC counter to indicate
`
`the proper timing for each ES.” ’945 Patent, 1:64-2:12 (emphasis added).
`
`MPEG-2 Transport Stream (TS)
`2.
`Transport Stream (TS) was introduced in MPEG-2, and is a container format
`
`52.
`
`originally designed for transmission over ATM networks. It has since been adopted
`
`for communication over various networks including radio-frequency networks and
`
`cable networks. MPEG-2 TS has also been adopted by ATSC, DVB, and ISDB for
`
`17
`
`LG Ex. 1004, pg 18
`
`
`
`HDTV signal broadcasting.
`
`53. MPEG-2 TS encapsulates PESs from multiple programs (unlike program
`
`streams) into fixed-size TS packets. Each packet is identified by a packet identifier
`
`(PID) that is unique to each program.
`
`It also contains an optional program clock
`
`reference (PCR).
`
`54. MPEG-2 TS packets are multiplexed and transmitted over the network. The
`
`PID is used to extract the desired program from the multiplexed data stream.
`
`MPEG-2 TS that are associated with multiple programs are called Multi Program
`
`Transport Stream (MPTS); whereas MPEG-2 TS that correspond to a single program
`
`are referred to as Single Program Transport Stream (SPTS).
`
`55.
`
`The ’945 Patent describes the predefined packet identifier (PID) in MPEG-2
`
`TS packets: “Each TS packet consists of a TS Packet header with optional Adaptation
`
`Field followed by useful data payload containing portion of a PES packet. The TS
`
`header consists of a sync byte, flags, indicators information for error detection and
`
`timing and Packet_ID (PID) field used to identify elementary stream carried
`
`underneath of a PES packet. In addition to identifying specific elementary streams,
`
`one PID is used to identify a program specific Information (PSI) table data. Each TS
`
`PSI table is sent in sections, usually occupying one or more TS packets. Four types of
`
`PSI tables exist: 1) Program Association Table (PAT) listing unique program_number
`
`(as an identifier of each program in one multiplex) and PID of the PMT table; 2)
`
`Program Map Table (PMT) listing PIDs of all component streams making a given
`
`18
`
`LG Ex. 1004, pg 19
`
`
`
`program. PMT may be constructed for each program separately or be common for a
`
`group of programs; 3) Conditional Access Table (CAT)
`
`identifying PID of
`
`Entitlement Management Messages and ID of used conditional access system if any
`
`scrambling of TS or PES packets is done; 4) Private Table carrying Network
`
`Information Table (NIT) or private data.” Ex. 1001, 2:18-39 (emphasis added).
`
`Time Shifting
`D.
`56. Generally, time shifting allows a viewer to step away from a multimedia
`
`presentation, such as a television program, without missing any part of
`
`the
`
`presentation.
`
`57.
`
`Time shifting live programs requires receiving a multimedia data stream, such
`
`as an audio and/or video stream, and then storing the data stream in a storage device.
`
`When the viewer signals that he or she would like to pause the program, an indication
`
`of the location within the multimedia stream where the pause request had been made
`
`is recorded. Once the viewer signals that he or she wishes to resume the multimedia
`
`presentation, the stored data is retrieved and displayed from the location where the
`
`presentation was paused.
`
`Clock Recovery And Timing Information
`E.
`The location within the multimedia presentation where the data stream has
`
`58.
`
`been paused is typically identified by the time when the request has been made, and
`
`must be recorded in order to display the multimedia data stream from the correct
`
`location once the viewer wishes to resume the presentation.
`
`19
`
`LG Ex. 1004, pg 20
`
`
`
`59.
`
`As stated above, the received multimedia data stream typically includes timing
`
`information necessary for various tasks such as synchronization of distinct data
`
`streams; e.g., simultaneous presentation of audio and video streams. The time when
`
`the multimedia data stream has been paused must be compared to timing information
`
`embedded in the multimedia data stream to determine the location where the stream
`
`has been paused.
`
`60.
`
`In addition, once the viewer has indicated that he or she would like to resume
`
`the presentation of the multimedia stream, the recorded data must be sent to the
`
`decoder at the proper rate. This rate is typically also determined from the timing
`
`information embedded in the received multimedia data stream.
`
`61.
`
`In order to ensure that the timing information generated by the receiver and
`
`the embedded timing information are compatible, they must be compared to the same
`
`clock. This is typically accomplished by embedding clock information in the received
`
`multimedia data stream and comparing it to the clock at the receiver.
`
`62.
`
`Therefore, when recording the multimedia data stream, it is essential to also
`
`record the timing information embedded in the received data stream so that the
`
`location of the pause request and the rate at which the data should be sent to the
`
`decoder can be determined.
`
`Coded Multimedia Data And Timing Information
`F.
`The multimedia data stream is typically received and stored in coded form. As
`
`63.
`
`stated above, coded multimedia data streams generally arrange the data in a coding
`
`20
`
`LG Ex. 1004, pg 21
`
`
`
`order which allows for fast data decompression. For example, the order of frames in
`
`a coded video sequence is rearranged depending on whether the frames are
`
`compressed based on past or future frames. As a result, coded multimedia data
`
`streams will typically include timing information for both the decoding time and
`
`display time.
`
`In addition, both the decoding time and display time are generally
`
`recorded with respect to a reference clock which is usually included in the coded data
`
`stream.
`
`64.
`
`Therefore, time shifting of coded multimedia data streams requires additional
`
`timing information.
`
`In particular, when a viewer wishes to pause the program, the
`
`t