`
`
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`
`
`UNIFIED PATENTS, LLC,
`Petitioner,
`
`v.
`
`INTERDIGITAL VC HOLDINGS, INC.,
`Patent Owner.
`
`
`Case No. IPR2021-00102
`U.S. Patent No. 8,363,724
`
`
`
`DECLARATION OF ROBERT LOUIS STEVENSON, PH.D. REGARDING
`PATENT OWNER PRELIMINARY RESPONSE FOR INTER PARTES
`REVIEW OF U.S. PATENT NO. 8,363,724
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Interdigital Exhibit 2001
`Unified Patents v. Interdigital
`IPR2021-00102
`Page 1 of 19
`
`
`
`Case No. IPR2021-00102
`Patent No. 8,363,724
`
`
`TABLE OF CONTENTS
`
`I.
`
`II.
`
`Introduction ..................................................................................................... 1
`
`Qualifications .................................................................................................. 1
`
`III. Materials Considered ...................................................................................... 3
`
`IV. Overview of Petition ....................................................................................... 4
`
`V. Virtual Reference Pictures are Not Synonymous with Synthesized
`Pictures ............................................................................................................ 5
`
`1.
`
`Virtual Reference Pictures .......................................................... 6
`
`a.
`
`b.
`
`Golden Frames ................................................................. 6
`
`Detail-Obscured Frames ................................................. 10
`
`B.
`
`Synthesized Pictures ........................................................................... 11
`
`1. Multi-View Video ..................................................................... 11
`
`2.
`
`3.
`
`“Synthesized” Picture Frames Are Created To Provide
`Additional Viewpoints For Multi-View Videos ....................... 12
`
`Synthesized Reference Pictures Are Not Virtual
`Reference Pictures .................................................................... 14
`
`C.
`
`D.
`
`Petitioner Incorrectly Assumes Virtual Reference Pictures Are
`“Also Known As Synthesized Reference Pictures” ............................ 15
`
`Petitioner Incorrectly Asserts that Xin’s Synthesized Pictures
`Are Not And Cannot Be Used As Part of The Multi-Video
`Display ................................................................................................ 16
`
`
`
`i
`
`Interdigital Exhibit 2001, Page 2 of 19
`
`
`
`I, Dr. Robert Louis Stevenson, hereby declare as follows:
`
`Case No. IPR2021-00102
`Patent No. 8,363,724
`
`
`I.
`
`Introduction
`
`1. My name is Robert Louis Stevenson. I have been retained in the
`
`above-referenced inter partes review proceeding by InterDigital VC Holdings,
`
`Inc., to evaluate United States Patent No. 8,363,724 (the “’724 patent”) against
`
`certain references that are presented by the Petitioner. As detailed in this report, it
`
`is my opinion that the Petition does not establish that of the challenged claims are
`
`anticipated or rendered obvious by the references presented by the Petitioner. If
`
`requested by the Patent Trial and Appeal Board, I am prepared to testify at trial
`
`about my opinions expressed herein.
`
`II. Qualifications
`2.
`
`I have over thirty-five years of experience in the field of electrical
`
`engineering, spanning a variety of positions in academia and industry. For the past
`
`thirty years I have held a faculty position in the Department of Electrical
`
`Engineering at the University of Notre Dame. I am presently the Associate Chair
`
`of the department. In my work and research, I have researched and written about
`
`video coding, multimedia compression, video enhancement, and other topics in
`
`video and image processing.
`
`3.
`
`I studied electrical engineering at the University of Delaware and
`
`Purdue University. I received my B.S.E.E. in 1986, and my Ph.D. in 1990. In 1990
`
`1
`
`Interdigital Exhibit 2001, Page 3 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`I joined the faculty of the University of Notre Dame. From 1990 to present I held
`
`a number of positions there, beginning as an Assistant Professor and then being
`
`promoted to Associate Professor, Professor and ultimately Associate Chair of the
`
`Department. During this time I also held a Professorship in the Department of
`
`Computer Science and Engineering.
`
`4.
`
`I have authored or co-authored dozens of publications in the areas of
`
`multimedia coding and compression, multimedia communications, and video and
`
`image processing
`
`including articles
`
`in
`
`technical
`
`journals and referenced
`
`conference presentations. My publications and patents are listed on my curriculum
`
`vitae, which is attached hereto as EX2004.
`
`5.
`
`As a result of my background in multimedia coding and compression,
`
`I have extensive knowledge regarding the state of the technical art in this area at
`
`the time of the filing of the ‘724 patent. I wrote many papers on video and image
`
`coding and multimedia systems.
`
`6.
`
`One of my papers was cited and relied upon by Petitioner and its
`
`declarant LeGall. (Pet., 7-9; EX1005, 26-28) Specifically, I am a co-author of Ex.
`
`1008, an article titled “Error Resilient Video Coding Using Virtual Reference
`
`Picture” which was published in the Proceedings of the SPIE/IS&T Conference on
`
`Image and Video Communications and Processing 2005, San Jose, CA, at pp.
`
`896–903.
`
`2
`
`Interdigital Exhibit 2001, Page 4 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`
`
`
`III. Materials Considered
`7.
`
`In preparing this declaration, I have reviewed the specification and
`
`claims of U.S. Patent No. 8,363,724 (“’724 patent” (EX1001)) and the file history
`
`of the ’724 patent (EX1013). I understand that the ’724 patent was issued on
`
`January 29, 2013 from U.S. Patent Application No. 12/309,066, which is a
`
`national stage application of International Application No. PCT/US2007/015719,
`
`which claims priority to U.S. Provisional Application No. 60/830,195, which was
`
`filed on July 11, 2006. (EX1001).
`
`8.
`
`I have also reviewed the Petition, declaration of Didier J. LeGall
`
`(EX1005), and all publications and exhibits cited in the Petition and declaration of
`
`Didier J. LeGall including:
`
`• US. Patent Application Publication No. 2006/0146138 (“Xin” (EX1003));
`
`• Didier Legall, “MPEG: A Video Compression Standard for Multimedia
`
`Applications,” Communications of the ACM, vol. 34, no. 4, April 1991
`
`(“LeGall” (EX1004));
`
`• European Patent 2278816 (“Lim” (EX1006));
`
`• U.S. Patent Application Publication No. 2002/0071485
`
`(“Calgar”
`
`(EX1007));
`
`3
`
`Interdigital Exhibit 2001, Page 5 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`• Guanjun Zhang and Robert L. Stevenson, “Error resilient video coding using
`
`virtual
`
`reference picture,” Proc. SPIE 5685,
`
`Image and Video
`
`Communications and Processing 2005 (“Zhang” (EX1008));
`
`• U.S. Patent 7,292,636 (“Haskell” (EX1009)); and
`
`• K. Yamamoto et al., “Multiview Video Coding Using View Interpolation
`
`and Color Correction,” IEEE Transactions on Circuits and Systems for
`
`Video Technology, vol. 17, no. 11, pp. 1436-1449, Nov. 2007 (“Yamamoto”
`
`(EX1014)).
`
`9.
`
`I have also reviewed relevant parts of the publications and exhibits
`
`cited in this declaration, including the following publications:
`
`• Wien, Mathias, “High Efficiency Video Coding”, Signals and
`
`Communication Technology, Springer 2015 (“Wein” (EX2002));
`
`• Smolic,
`
`Interactive 3-D Video Representation and Coding
`
`Technologies, Proceedings of the IEEE, Vol. 93, No. 1, January 2005
`
`(“Smolic” (EX2003));
`
`
`
`IV. Overview of Petition
`10.
`
`I understand the Petition alleges asserts the following Grounds:
`
`Ground
`I
`
`References Basis
`Xin
`§ 102
`
`Challenged Claims
`1-3, 9, 15, 19-22, 28, 34, 38-41, 47, 53,
`56-59, 65, 71, and 74
`
`4
`
`Interdigital Exhibit 2001, Page 6 of 19
`
`
`
`II
`
`III
`
`Xin
`
`§ 103
`
`Xin and
`LeGall
`
`§ 103
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`1-3, 7, 8, 12, 15, 19-22, 26, 28, 31, 34,
`38-41, 45, 47, 50, 53, 56-59, 63, 65, 68,
`71, and 74
`4, 8, 10, 11, 23, 27, 29, 30, 42, 46, 48, 49,
`60, 64, 66, and 67
`
`11.
`
`In each Ground, the Petition relies solely on Xin (EX1003) as teaching
`
`the “virtual reference picture” limitations required in each challenged claim,
`
`including the claimed use of a “virtual reference picture” and the claimed storing
`
`of the “virtual reference picture … in a decoded picture buffer that also stores non-
`
`virtual reference pictures.” (E.g., Pet., 24-27, 63, 69, 71).
`
`12. The Petition asserts that a person of ordinary skill in the art (“POSA”)
`
`at the time of the invention of the ‘724 patent (July 11, 2006) would have had at
`
`least the equivalent of a Bachelor’s degree in electrical engineering, computer
`
`engineering, computer science, or a related subject and two or more years of
`
`experience in relevant fields. (Pet., 13). I meet and exceed the qualifications of the
`
`POSA.
`
`
`
`V. Virtual Reference Pictures are Not Synonymous with
`Synthesized Pictures
`
`13. The Petition raises two distinct concepts that can be used in creating a
`
`compressed video sequence: (1) the use of a virtual reference picture to compress
`
`and decompress digital video data, which is the subject of the challenged ’724
`
`5
`
`Interdigital Exhibit 2001, Page 7 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`Patent; and (2) the use of a synthetic picture and reference picture to create and
`
`transmit video images that were not actually captured by a video camera, which is
`
`the subject of the Petition’s primary reference, Xin. (Pet., 5-9, 15-22).
`
`1.
`
`Virtual Reference Pictures
`
`14. As I noted above, I meet and exceed the POSA, and I understand
`
`virtual reference pictures to be picture data that is not designed for use as a frame
`
`of a video, rather its only purpose is to serve as a reference frame (See also,
`
`EX1008, 897-898). Such picture frames are not frames of a displayed video
`
`sequence and often contain picture content that has undesirable artifacts by design
`
`which render them unsuitable for display purposes. Examples of virtual reference
`
`pictures include golden frames and detail-obscured frames, which are explained in
`
`the following sections.
`
`a. Golden Frames
`15. One example of a virtual reference picture is a “golden frame.” A
`
`golden frame is created solely as a reference picture; it includes high-quality pixel
`
`information, but is not itself a picture frame from the video and is not available for
`
`inclusion as a frame of the video. Rather, when the golden frame is created by the
`
`encoder and transmitted to the decoder, it includes an indication that the golden
`
`frame is not available for display in the output video.
`
`6
`
`Interdigital Exhibit 2001, Page 8 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`16. To better understand a “golden frame” and its benefits, first consider a
`
`compression approach that does not use a “golden frame.” In this example, a
`
`video scene has a static background and a dynamic foreground, such as a video of
`
`a ball (foreground) moving across a landscape (background). As the ball moves
`
`across the scene from frame-to-frame the ball covers a portion of the background
`
`and reveals another portion of the background that was previously covered. If a
`
`video compression scheme used adjacent frames as (non-virtual) reference frames
`
`(as in the foregoing eye blinking example), then the portion of the background that
`
`was previously covered would not be included in the reference frame and would
`
`therefore need to be included in the residual (along with motion information for
`
`the ball “block”). This concept is illustrated below. As shown, if the first picture
`
`serves as a reference picture for the second picture then the residual contains the
`
`information for the background that the ball was obscuring in the first frame.
`
`Depending on the specific compression technology used, the residual may also
`
`include some additional information, which is ignored here for the purposes of
`
`illustrating the concept of a golden frame.
`
`
`
`7
`
`Interdigital Exhibit 2001, Page 9 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`
`
`
`17. But instead of using the first picture as a reference frame, greater
`
`compression could be achieved by using a virtual reference frame. Specifically, a
`
`picture frame could be created comprising solely the static background of the
`
`scene, with no foreground. Because this “golden frame” contains the complete
`
`background information, the portion of the background that was previously
`
`covered by the ball from one frame to the next would be available in the golden
`
`frame. As a result, as illustrated below this information would not be required to
`
`be included in the residual, and the compression of the video using a golden frame
`
`is more efficient than using merely the adjacent frames for prediction.
`
`8
`
`Interdigital Exhibit 2001, Page 10 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`
`
`
`18. But although this “golden frame” is created for use as a reference
`
`picture, it is not available for inclusion as a video frame in the displayed video
`
`itself. Continuing our example from above, in the displayed video, the ball is
`
`always present in the foreground in the video; in contrast, the golden frame
`
`contains only background and cannot be displayed as part of the video because it
`
`is missing the required ball image. To this end, when an encoder transmits a
`
`golden frame to a decoder for use as a reference picture, the golden frame is
`
`designated as not available for display in the output video.
`
`19. Because this golden frame is created solely for purposes of serving as
`
`a reference picture and is not available for display as a video frame of the video, it
`
`is a virtual reference picture.
`
`9
`
`Interdigital Exhibit 2001, Page 11 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`
`b.
`20. A detail-obscured (or filtered) reference frame is another example of a
`
`Detail-Obscured Frames
`
`virtual reference picture, and is the subject of the article I co-authored with G.
`
`Zhang (the “Zhang” article) and relied on in the Petition to explain virtual
`
`reference pictures. (Pet., 7-8; EX1008).
`
`21. As I explained in my article, certain transmission methods (such as
`
`certain wireless transmissions) may be prone to introducing errors into the
`
`compressed video file. (EX1008, 896). And because compression using motion
`
`compensation encodes video information as changes or differences between
`
`frames, small errors in a video sequence can lead to very large and noticeable
`
`errors in the decoded output video. (EX1008, 896).
`
`22. As described in my article, I investigated using virtual reference
`
`pictures to address this problem, specifically the use of a detail-obscured or
`
`filtered frame as a virtual reference frame in order to reduce the impact of errors in
`
`the compressed data. (EX1008, 897-898). I described decoding several actual
`
`frames of the transmitted video, then applying a filter to those frames, which
`
`would in effect “obscure details” of content of the frames. (EX1008, 897-898).
`
`The resulting detail-obscured frame is used as a virtual reference to predict the
`
`next current frame. (EX1008, 897-898). The virtual reference picture appears to
`
`include undesirable artifacts and is not suitable for viewing in a video, unlike non-
`
`10
`
`Interdigital Exhibit 2001, Page 12 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`virtual reference frames, which are actual video frames containing all the image
`
`details to be displayed.
`
`Synthesized Pictures
`
`B.
`23. The Petition’s Xin reference does not concern or describe the use of
`
`virtual reference pictures, but instead concerns a different and distinct concept,
`
`synthesized pictures. Unlike virtual reference pictures, synthesized pictures (and
`
`synthesized pictures that are used as reference pictures) are created so that they
`
`may be displayed or made available for display as part of the video, specifically,
`
`a multi-view video.
`
`1. Multi-View Video
`24. Multi-view video refers to video sequences captured simultaneously
`
`from multiple camera angles and represented in a single video stream. (EX2003,
`
`99; EX1003, [0003]-[0005], [0036], [0067]-[0068]). Each camera angle or view
`
`can be considered as a separate single-view video. (EX2003, 99; EX1003, [0003]-
`
`[0005]). In other words, multi-view video is a collection of video formed from two
`
`or more camera views. (EX2003, 99; EX1003, [0003]-[0005]).
`
`25. Consider for example the below figure from Xin illustrating four
`
`cameras pointed towards a common scene. Each camera is generating a video
`
`from their respective viewing positions and angles. (EX1003, [0003]-[0005]; See
`
`also, EX2003, 105-107). The video from each camera, or view, can be encoded,
`
`11
`
`Interdigital Exhibit 2001, Page 13 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`and the collection of encoded video is called a multi-view video. (EX1003,
`
`[0003]-[0005]; See also, EX2003, 99, 105-107).
`
`
`
`26. Multi-view video can also be used for free viewpoint television
`
`(FTV), which is a system for allowing the user to interactively control the
`
`viewpoint and generate new views of a dynamic scene from any 3D position.
`
`(EX2003, 105-107; EX1003, [0003]).
`
`2.
`
` “Synthesized” Picture Frames Are Created To Provide
`Additional Viewpoints For Multi-View Videos
`
`27. A synthesized picture frame is a picture frame created to provide an
`
`additional camera angle for a multi-view video. (EX1003, [0036], [0067]). In
`
`other words, instead of limiting the available views and angles for a multi-view
`
`video to those created by actual cameras, synthesized picture frames are created to
`
`provide additional angles and views. (EX1003, [0036], [0067]). These additional
`
`12
`
`Interdigital Exhibit 2001, Page 14 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`angles and views correspond to a non-existent camera, sometimes referred to as
`
`virtual camera, and should not be confused with a virtual reference picture, which
`
`is a different concept entirely.
`
`28. Xin (EX1003) illustrates this in the figure below. Two videos (or
`
`views) are acquired by two cameras 2 and 3 at respective viewing locations and
`
`angles. (EX1003, [0067]; See also, EX2003, 99, 105-107). A third video can be
`
`synthesized from the two acquired videos by, for example, interpolation, to create
`
`a video appearing to be from the view of (non-existent) camera 800. (EX1003,
`
`[0067]; See also, EX2003, 99, 105-107). Although this synthesized video was not
`
`acquired from a camera, the synthesized video along with the two acquired videos
`
`may be shown as part of the multi-view video. (EX1003, [0003]-[0005], [0036],
`
`[0067]-[0068]; See also, EX2003, 99, 105-107).
`
`
`
`13
`
`Interdigital Exhibit 2001, Page 15 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`29. As mentioned, synthesizing can be performed by interpolation.
`
`(EX1003, [0068]; EX2003, 99, 105-107). For example, a synthesized picture view
`
`residing between a first view and a second view can be generated by interpolating
`
`the pixel values of the first view and the second view for each frame. (EX1003,
`
`[0068]; EX2003, 99, 105-107). This interpolation can generate synthesized video
`
`picture frames that appear to a viewer to have been acquired by a camera
`
`positioned between the camera of the first view and the camera of the second
`
`view. (EX1003, [0067]-[0068]; EX2003, 99, 105-107).
`
`3.
`
`Synthesized Reference Pictures Are Not Virtual Reference
`Pictures
`
`30.
`
`Just as an ordinary picture frame (one created by a camera) may be
`
`used as a reference picture, so may a synthesized picture frame. (EX1003, [0042],
`
`[0067]-[0068]). For example, just as an ordinary picture frame from a specific
`
`camera angle can be used as reference picture to predict the frame of other picture
`
`frames from that camera, a synthesized picture frame can be used to predict the
`
`contents of other synthesized picture frames. (EX1003, [0042], [0067]-[0068]). As
`
`Xin describes: “The synthesized frames serve as reference pictures from which a
`
`current synthesized frame can be predicted.” (EX1003, [0071]).
`
`31. But critically, a “synthesized reference picture” is not a “virtual
`
`reference picture.” Where a synthesized reference picture is itself a synthesized
`
`picture frame created so that it may be viewed as part of the available multi-view
`
`14
`
`Interdigital Exhibit 2001, Page 16 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`video, a virtual reference picture is not created to be viewed. (EX1003, [0042],
`
`[0067]-[0068]). Unlike a synthesized reference picture, a virtual reference picture
`
`is created for the sole purpose of serving as a reference, and it is not suitable,
`
`intended, or available for viewing. (EX1008, 897-898).
`
`32.
`
`In other words, virtual reference pictures are not synonymous
`
`with synthesized reference pictures.
`
`C.
`
`Petitioner Incorrectly Assumes Virtual Reference Pictures Are
`“Also Known As Synthesized Reference Pictures”
`
`33. Petitioner and its declarant LeGall both assert that virtual
`
`reference pictures (“VRPs”) are also known as “synthesized reference
`
`pictures”. (Pet., 8-9; EX1005, ¶28). This is incorrect.
`
`34. A synthesized video is one that is created or generated from
`
`another video using, for example, warping or interpolation. If one were to
`
`compress the synthesized video using compression that relies upon motion
`
`compensated prediction techniques, reference frames would be utilized as
`
`well, and these reference frames would be previously decoded frames, that
`
`is, frames that form a part of the video.
`
`35. A virtual reference picture, however, is a different concept from
`
`synthetic video or synthetic reference frames. A virtual reference picture
`
`can include a picture that is processed, for example, by applying a filter as I
`
`15
`
`Interdigital Exhibit 2001, Page 17 of 19
`
`
`
`Case No. IPR2021-00102
`
`Patent No. 8,363,724 Patent No. 8,363,724
`described in my article (EX1008). Importantly, the virtual reference picture
`
`is not a frame in the video itself, rather, it is virtual because it doesn’t form
`
`a part of a video. Indeed, the virtual reference pictures as I described in my
`
`article (EX1008) were not available for display. That is to say, you would
`
`not want to display such a frame nor were they designed to be displayed.
`
`Thus, a virtual reference picture can be used for prediction, but is not
`
`displayed.
`
`D.
`
`Petitioner Incorrectly Asserts that Xin’s Synthesized Pictures Are
`Not And Cannot Be Used As Part of The Multi-Video Display
`
`36. Petitioner and its declarant LeGall also assert that Xin’s
`
`synthesized pictures are virtual reference pictures because Xin supposedly
`
`discloses that its synthesized pictures are intended to be used solely as
`
`reference pictures and not available to be shown as part of a video: “because
`
`Xin’s synthesized reference pictures are used for prediction and not
`
`displayed, they are VRPs.” (Pet. 22, 24; EX1005, ¶¶58, 61). This is
`
`incorrect.
`
`37. Xin repeatedly describes that its synthesized pictures are
`
`intended to be used as video picture frames in the multi-view video, and not
`
`just as reference pictures. For example, Xin makes clear that a video is
`
`synthesized and the synthesized reference frames form part of the
`
`16
`
`Interdigital Exhibit 2001, Page 18 of 19
`
`
`
`Case No. IPR2021-00102
`Patent No. 8,363,724
`synthesized video at least at [0002], [0015], [0016], [0067], abstract, and
`
`claims 1. (EX1003). In other words, Xin’s synthesized reference pictures
`
`are created for more than serving as a reference frame, and they are
`
`available for use as frames of a video, specifically, a synthesized view of a
`
`multi-view video.
`
`I declare that all statements made herein of my knowledgeare true, and that all
`
`statements made on information andbelief are believed to be true, and that these
`
`statements were made with the knowledgethat willful false statements and the like
`
`so made are punishable by fine or imprisonment, or both, under section 1001 of
`
`Title 18 of the United States Code.
`
`Dated: March 8, 2021 A=
`Dr. Robert Louis Stevenson
`
`17
`
`Interdigital Exhibit 2001, Page 19 of 19
`
`Interdigital Exhibit 2001, Page 19 of 19
`
`