`IPR2016-01243
`
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`__________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`__________________
`
`
`
`
`Prime Focus Creative Services Canada Inc.,
`Petitioners,
`
`v.
`
`Legend3D, Inc.,
`Patent Owner.
`
`
`
`__________________
`
`Case IPR2016-01243
`
`U.S. Patent No. 7,907,793
`__________________
`
`
`
`DECLARATION OF CARLOS VAZQUEZ, PH.D.
`
`IN SUPPORT OF PATENT OWNER RESPONSE TO THE PETITION AND
`MOTION TO AMEND IN
`IPR2016-01243
`
`
`
`
`
`
`
`
`
`
`
`
`
`Legend3D, Inc. Ex. 2024-0001
`IPR2016-01243
`
`
`
`TABLE OF CONTENTS
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Page
`
`I.
`
`II.
`
`INTRODUCTION ................................................................................ 1
`
`EXPERT QUALIFICATIONS AND CREDENTIALS ....................... 1
`
`III. BASIS FOR OPINIONS, AND MATERIALS REVIEWED .............. 5
`
`IV. LEGAL STANDARDS ........................................................................ 8
`
`A.
`
`B.
`
`FOR PRIORITY AND WRITTEN DESCRIPTION ................. 8
`
`Claim Interpretation in Inter Partes Review .............................. 9
`
`V.
`
`THE PERSON OF ORDINARY SKILL IN THE ART ...................... 9
`
`VI. BACKGROUND ON RELEVANT TECHNOLOGY....................... 11
`
`A. Depth Cues in 2-Diminensional Images .................................. 11
`
`B.
`
`C.
`
`Color, Shading, Perceived Shape, and Perceived Depth .......... 19
`
`Depth Enhancement vs. 2D to 3D Conversion ........................ 21
`
`VII. THE ’793 PATENT AND CHALLENGED CLAIMS ...................... 21
`
`VIII. CLAIM CONSTRUCTION ............................................................... 23
`
`IX. THE ’081 AND ’670 PATENTS TEACH THE CLAIMED “DEPTH
`PARAMETER.” ................................................................................. 23
`
`X. DECLARATION ................................................................................ 28
`
`
`
`
`
`
`
`
`-i-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0002
`IPR2016-01243
`
`
`
`DECLARATION OF DR. CARLOS VAZQUEZ
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`I, Carlos Vazquez, declare as follows:
`
`I.
`
`INTRODUCTION
`
`1.
`
`I am making this Declaration at the request of Legend3D, Inc.
`
`(“Patent Owner”) in the matter of IPR2016-01243, an Inter Partes Review of U.S.
`
`Patent No. 7,709,793 to B. Sandrew (“the ’793 Patent”).
`
`2.
`
`I am being compensated for my work in this matter at a rate of $235
`
`per hour for travel and $470 per hour for my professional consulting services. I am
`
`also reimbursed for my reasonable expenses incurred in connection with my work
`
`in this proceeding. My compensation is not in any way contingent upon the
`
`outcome of this Inter Partes Review. I have no interest in the outcome of this
`
`proceeding or any related litigation.
`
`II. EXPERT QUALIFICATIONS AND CREDENTIALS
`
`3. My qualifications for presenting the opinions in this Declaration are
`
`set forth below and in my curriculum vitae, a copy of which is provided as Ex.
`
`2027.
`
`4.
`
`I have studied, taught, and practiced electrical engineering and
`
`software engineering for over 20 years. I attended the Higher Polytechnic
`
`Institute José Antonio Echeverría (ISPJAE) in Havana, Cuba, and earned a
`
`
`
`
`
`
`-1-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0003
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Bachelor’s (Eng) Degree in Electronics (Electronic Components and Devices)
`
`in 1992, where I graduated summa cum laude, and a Master’s (M.Sc.) of
`
`Science Degree in Applied Informatics (Signal and Image Processing) in 1997. I
`
`earned my Ph.D. in Telecommunications from the Institut National de la
`
`Recherche Scientifique (INRS), in Montréal, Canada, in 2003, where I was also a
`
`post-doctoral fellow from 2002-2004 conducting research on a project related to
`
`the application of Level Set Methods to image segmentation, motion estimation,
`
`and video super resolution.
`
`5.
`
`I am currently an Associate Professor of Software Engineering and
`
`Information Technology at the École de Technologie Supérieure (ETS) in
`
`Montréal, Canada. I have held this position since 2013, teaching courses on
`
`Parallel programming, multimedia systems, digital imaging, and computer
`
`graphics; 3D Vision Systems, and operating systems, and conducting research in
`
`3D video systems, 3D object extraction and reconstruction; medical imaging
`
`applications; 3D cinema; and 3D-TV applications.
`
`6.
`
`Prior to my employment at ETS, I was a research Scientist with the
`
`Communications Research Center (CRC) in Ottawa, Canada, in the Advanced
`
`Video Systems, where from 2005 through 2013 I worked in 3D-TV systems,
`
`namely developing algorithms and applications to help advance the 3D-TV
`
`industry, 2D-to-3D conversion, novel displays, S3D video processing; Multi-View
`
`
`
`
`
`
`-2-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0004
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Video Coding, namely, 4D wavelet-based multi-view video coding, 2D + Depth +
`
`Occlusion multi-view coding and representation, and view synthesis and
`
`occlusions removal; Depth-Image Based Rendering, namely, real time image based
`
`rendering techniques, parallel programming, GP-GPU, among other topics.
`
`7.
`
`I have also worked as a Research Associate in the Electrical and
`
`Computer Engineering Department of Concordia University, Montréal, Canada,
`
`from 2004 to 2005, and as a Research Assistant at the University of Quebec,
`
`Montréal, Canada, from 1998 to 2004 in the areas of software development
`
`and hardware installation and administration areas, namely, video equipment and
`
`stereoscopic video equipment.
`
`8.
`
`Upon graduation with my Bachelor’s degree and while obtaining
`
`my Master’s Degree, I was an Assistant Professor in the Telecommunications
`
`Department of the ISPJAE in Havana, Cuba, where I taught courses on
`
`programming languages, embedded systems, signal acquisition, and processing
`
`from 1992 through 1997.
`
`9. My twenty-plus years of industry experience includes authoring
`
`numerous peer-reviewed journal and conference articles, including attaining as a
`
`co-recipient the Scott Helt best paper award from IEEE Broadcast Technology
`
`Society in 2011. My research interests are in the area of 3D-TV; 2D-to-3D
`
`video conversion; stereo and multi-view vision systems; 3D object extraction
`
`
`
`
`
`
`-3-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0005
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`and reconstruction; image/video representation, sampling and interpolation; image
`
`and video coding; and motion/disparity/depth estimation and compensation.
`
`10.
`
`I have authored over 30 papers at various conferences and symposia
`
`over the past ten-plus years, such as the IEEE International Conference on Image
`
`Processing, the International Workshop on Computer Vision and Its Application to
`
`Image Media Processing, The SPIE Visual Communications and Image
`
`Processing, and SPIE Stereoscopic Displays and Applications. I also am co-
`
`inventor on three U.S. Patents disclosing Generation of a depth map from a
`
`monoscopic color image for rendering stereoscopic still and video images; Method
`
`and graphical interface for modifying depth maps; and Enhancing virtual presence
`
`using multiview and stereoscopic 3D displays with a grounded curvilinear screen.
`
`11.
`
`I became a Senior Member of the IEEE in 2012, and obtained the
`
`2012 Journal Certificate of Merit from SMPTE Motion Imaging Journal. In 2011,
`
`I received the NAB Technology Innovation Award as the co-inventor of one of
`
`the technologies supporting the award: 2D-to-3D video conversion.
`
`12.
`
`I have also authored book chapters on “2D to 3D video conversion-
`
`overview and perspectives” in the book “ Emerging Technologies for 3D
`
`Video: Creation, Coding, Transmission, and Rendering”; and “DIBR-based
`
`conversion from monoscopic to stereoscopic and multiview video” in “ 3D-TV
`
`System with Depth-Image Based Rendering: Architectures, Techniques and
`
`
`
`
`
`
`-4-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0006
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Challenges.”
`
`III. BASIS FOR OPINIONS, AND MATERIALS REVIEWED
`
`13. The opinions set forth in my Declaration are based on my personal
`
`knowledge gained from my education, on my personal experience, and on the
`
`review of the documents and information described in this Declaration.
`
`14.
`
`In preparation of this Declaration, I have considered:
`
` Exhibit 1001: U.S. Patent No. 7,907,793 (“the ‘793 Patent”).
`
` Exhibit 1002: File History for the ‘793 Patent.
`
` Exhibit 1003: U.S. Patent No. 7,181,081 (“the ‘081 Patent”).
`
` Exhibit 1004: U.S. Patent No. 7,333,670 (“the ‘670 Patent”).
`
` Exhibit 1005: U.S. Patent Application No. 12/241,992 (“Passmore”).
`
` Exhibit 1006: U.S. Patent No. 7,573,475 (“Sullivan”).
`
` Exhibit 1007: Declaration of Rob S. Schmitt from Business Wire, Inc.
`
`attaching: Exhibit 1: “Legend Films Successfully Raise $5 Million in Venture
`
`Capital Funding” (press release); and Exhibit 2: “‘Night of the Living Dead’ to
`
`Be Released in Color and 3D” (press release).
`
` Exhibit 1008: Curriculum Vitae of Dr. David Forsyth.
`
` Exhibit 1009: Corrected Expert Declaration of Dr. David Forsyth.
`
` Exhibit 1010: U.S. Patent No. 7,577,312.
`
`
`
`
`
`
`-5-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0007
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
` Exhibit 1011: Article: “How do 3D films work?” available at
`
`http//www.physics.org/article-questions.asp?id=56.
`
` Exhibit 1012: Book excerpt: Rendering Techniques 2001, edited by Steven J.
`
`Gortler and Karol Myszkowski (2001).
`
` Exhibit 1013: Article: Matt Sullivan, Trick and Treat: Behind the Scenes of the
`
`New Nightmare Before Christmas and the 3D Movie Revolution, Popular
`
`Mechanics, available at
`
`http://www.popularmechanics.com/technology/gadgets/a3607/4200796/.
`
` Exhibit 1014: Redline of specification from ‘081 Patent to ‘793 Patent.
`
` Exhibit 1015: Redline of specification from ‘670 Patent to ‘793 Patent.
`
` Exhibit 2005: U.S. Patent No. 8,922,628 (Petitioner’s ‘628 Patent).
`
` Exhibit 2006: Article: Jonathan Sachs, Digital Image Basics, Digital Light &
`
`Color, 1996-1999.
`
` Exhibit 2007: Article: Paul Bourke, HSL colour space, June 2000.
`
` Exhibit 2008: Archived web page: ColorModels.html, available at
`
` http://cs.brown.edu/courses/cs092/VA10/HTML/ColorModels.html, from
`
`Brown University course CS092, archived Jan 24, 2000.
`
` Exhibit 2009: Book: Lenny Lipton, Foundations of the Stereoscopic Cinema –
`
`A Study in Depth, 1982.
`
`
`
`
`
`
`-6-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0008
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
` Exhibit 2010: Article: Dorothy A. Kleffner and V. S. Ramachandran, On the
`
`Perception of Shape from Shading, Perception & Psychophysics 1992, 52 (1),
`
`18-36.
`
` Exhibit 2011: Version d.1 for the MS-DOS and PC-DOS operating System,
`
`Chapter 1, Borland International, 1992.
`
` Exhibit 2012: Archived web page: GIMP core_blend_tool.html, available at
`
` http://www.gimp.org/core_blend_tool.html, archived August 17, 2000.
`
` Exhibit 2013: Archived web page: GIMP transforms.html, available at
`
` http://manual.gimp.org/manual/GUM/transforms.html, archived March 5, 2001.
`
` Exhibit 2014: Book: Adobe Photoshop 6.0 User Guide, Adobe Systems
`
`Incorporated, 2000.
`
` Exhibit 2015: U.S. Patent No. 8,396,328 (First child Patent to the ‘793 Patent).
`
` Exhibit 2016: Internet Archive Affidavit for Exhibits 2008, 2012 and 2013.
`
` Exhibit 2018: Deposition of David Forsyth, Ph.D. taken March 9, 2017 for Case
`
`No. IPR2016-00806, re U.S. Patent No. 8,922,628.
`
` Exhibit 2019: Colored photos: Virgin of the Rocks.
`
` Exhibit 2020: Book: David A. Forsyth, Computer Vision – A Modern
`
`Approach, 2003 (excerpts).
`
`
`
`
`
`
`-7-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0009
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
` Exhibit 2021: Deposition of David Forsyth, Ph.D. taken March 20, 2017 for
`
`Case No. IPR2016-01243, re U.S. Patent No. 7,907,793.
`
` Exhibit 2022: U.S. Patent No. 4,925,294.
`
` Exhibit 2023: Article: Moving Cast Shadows and the Perception of Relative
`
`Depth, Technical Report No. 6, June 1994.
`
` Exhibit 2024: Declaration of Carlos Vazquez.
`
` Exhibit 2025: U.S. Patent No. 6,141,433.
`
` Exhibit 2026: U.S. Patent No. 5,940,528.
`
` Exhibit 2027: Curriculum Vitae of Carlos Vazquez.
`
`IV. LEGAL STANDARDS
`
`A.
`
`15.
`
`FOR PRIORITY AND WRITTEN DESCRIPTION
`
`I am not an attorney. In preparing and expressing my opinions and
`
`considering the subject matter of the ’793 Patent, I am relying on certain basic
`
`legal principles that have been explained to me. These principles are discussed
`
`below.
`
`16.
`
`I have been informed that in order for a claim to get priority to an
`
`earlier-filed application, it must be shown that any claim term that does not appear
`
`in the written description of the patent is necessarily comprehended in the
`
`description provided and would have been so understood at the time the patent
`
`application was filed. I have also been informed that the patent disclosure need
`
`
`
`
`
`
`-8-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0010
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`only reasonably convey to persons skilled in the art that the inventor had
`
`possession of the subject matter in question.
`
`B. Claim Interpretation in Inter Partes Review
`
`17.
`
`I understand that the first step in determining the validity of a claim is
`
`for the claim to be properly construed.
`
`18. For purposes of Inter Partes Review, I understand that each
`
`challenged claim must be given its broadest reasonable interpretation in light of the
`
`patent specification. I am informed that the broadest reasonable interpretation
`
`dictates that claim terms be given their ordinary and customary meaning, as would
`
`be understood by a POSITA in the context of the entire disclosure or specification
`
`of the patent, unless the inventor, as a lexicographer, has set forth a special
`
`meaning for a term. I understand that a patent’s “specification” includes all the
`
`figures, discussion, and claims within the patent document. I understand that the
`
`USPTO will look to the specification to see if there is a definition for a claim term,
`
`and if not, will apply the broadest reasonable interpretation from the perspective of
`
`a POSITA.
`
`19.
`
`I provide my opinions in this Declaration based on the guidelines set
`
`forth above.
`
`V. THE PERSON OF ORDINARY SKILL IN THE ART
`
`20.
`
`I have reviewed the ’793 Patent (Ex. 1001), the ’081 Patent (Ex.
`
`
`
`
`
`
`-9-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0011
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`1003), the ’670 Patent (Ex. 1004), the prior art documents and other printed
`
`publications referenced above, and the Corrected Declaration of David Forsyth
`
`(Ex. 1009). My opinion regarding the level of one of ordinary skill in the art is
`
`based on this review and my knowledge of image analysis and enhancement.
`
`21. The ’793 Patent describes the field of the invention as follows: “One
`
`or more embodiments of the invention are related to the field of image analysis and
`
`image enhancement . . . .” Ex. 1001 at 1:19-20. Based on the ’793 Patent’s
`
`description, my opinion is that a POSITA in the field of the ’793 Patent as well as
`
`the ’081 and ’670 Patents at the time of the invention would have had at least an
`
`undergraduate degree in electrical engineering or computer science (or an
`
`equivalent subject) and would have been someone with a good working knowledge
`
`of computer programming, data structures, and image processing. In addition to
`
`the undergraduate degree, the person would have gained this knowledge through
`
`several years of practical working experience. This description is approximate,
`
`and a higher level of education or skill might make up for less experience, and
`
`vice-versa.
`
`22.
`
`I believe that I would qualify as at least a POSITA at all relevant
`
`times for purposes of this proceeding, including at least as early as the priority date
`
`of the ’081 Patent. I have a sufficient level of knowledge, experience, and
`
`education to provide an opinion in the field of the ’793 Patent. Specifically, I work
`
`
`
`
`
`
`-10-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0012
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`with many people, including students, researchers that I supervise, and colleagues
`
`in the industry, who are POSITAs. I therefore understand what a POSITA is and
`
`would have been as of the priority date of the ’081 Patent. It is through this
`
`viewpoint that I make my conclusions in this matter.
`
`23. My opinion of the level of ordinary skill in the art is based on my
`
`extensive personal and professional experience working in the field of image
`
`processing, and on my knowledge of the level of education and experience of
`
`colleagues and others actively working in the field as of and for several years prior
`
`to the priority date of the ’081 Patent, for example.
`
`24.
`
`I understand that a POSITA in the art is not a specific, real individual,
`
`but rather a hypothetical individual presumed to have knowledge of the relevant art
`
`at the time of the invention.
`
`VI. BACKGROUND ON RELEVANT TECHNOLOGY
`
`A. Depth Cues in 2-Diminensional Images
`
`25.
`
`In two-dimensional images, it was well-known at the time of the ’081
`
`Patent (i.e., 2001-2002) that monocular depth cues were used in photographs and
`
`films in order to realistically depict scenes from our three-dimensional world on a
`
`two-dimensional plane (e.g., a movie screen or photograph). Ex. 2009 at p. 20.
`
`This was well-recognized in Lenny Lipton’s book entitled “Foundations of the
`
`Stereoscopic Cinema—A Study in Depth,” which was published in 1982. Ex.
`
`
`
`
`
`
`-11-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0013
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`2009.
`
`26. Monocular depth cues were known to Renaissance painters in the
`
`1500s, and codified by Leonardo for example. Ex. 2009 at p. 19. Almost all depth
`
`cues can be perceived on a flat screen. Ex. 2009 at p. 54. Depth cues include
`
`aerial perspective, as well as shadows and shading. Ex. 2009 at pp. 57-58.
`
`27. The aerial perspective depth cue necessarily causes hazy objects to
`
`appear to be farther away than objects for which there is no intervening haze. The
`
`color image below shows a lower saturation of colors for background objects. In
`
`other words, the haze depth cue is created in the photograph below using a
`
`modification to the saturation component of colors. It can be appreciated in the
`
`below photograph that the mountains in the background appear farther away than
`
`those of the foreground.
`
`
`
`
`
`
`
`
`-12-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0014
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`28. Lipton taught this principle as well. Figure 2.4 of his book shows an
`
`aerial perspective in which “haze contributes to making the background look far
`
`away.” Ex. 2009 at p. 57. As is clearly shown, the haze has the effect of making
`
`the trees in the background look farther away than the darker trees in the
`
`foreground.
`
`
`
`29. Shadings and shadows provide visual cues regarding the shape of
`
`objects and their relative distance from one another. Ex. 2009 at pp. 57, 58 (Fig.
`
`2.5). Lipton taught that “[c]ast shadows provide an effective depth cue, as does
`
`light coming from one or more directions.” 2009 at p. 57. Indeed, Lipton’s Figure
`
`
`
`
`
`
`-13-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0015
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`2.5 (reproduced below) uses light and shade as depth cues to create different
`
`perceived shapes. Ex. 2009 at p. 58. If the light is assumed to be coming down
`
`from above (top to bottom), then the shadow on the left shape causes it to appear to
`
`pop out whereas the shadow on the right shape causes it to appear to pop in.
`
`
`
`30. The creation of an impression of perceived depth in 2-dimensional
`
`images using only subtle variations in shading was well-known at the time of the
`
`’081 Patent. Ex. 2010 at p. 2. Shading was thus considered a monocular depth cue
`
`at the time of the ’081 Patent. Ex. 2010 at p. 17. For example, a 1992 publication
`
`by Kleffner and Ramachandran described that “Figure 1 [reproduced below]
`
`depicts a set of objects that conveys a strong impression of depth.” Ex. 2010 at p.
`
`
`
`
`
`
`-14-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0016
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`3. The objects in Figure 1 of Kleffner “convey an impression of depth based
`
`exclusively on subtle variations in luminance.” Ex. 2010 at p. 3. Thus, even
`
`absent other depth cues, the use of shading (or variation in luminance) to convey a
`
`perception of depth was understood at the time of the ’081 Patent. Ex. 2010 at p. 3
`
`(Figure 3 caption).
`
`
`
`31. A book co-authored by Petitioner’s expert Dr. Forsyth around the time
`
`of the ’081 Patent also demonstrates how shading serves as a depth cue. For
`
`
`
`
`
`
`-15-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0017
`IPR2016-01243
`
`
`
`example, in Figure 5.10 (reproduced below), Dr. Forsyth’s book shows five two-
`
`dimensional views of images of a sphere. Ex. 2020 at p. 4.
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`
`
`32. Each of the five views shows the sphere with self-shadows created
`
`from variations in brightness distributed over the sphere. Ex. 2020 at p. 4; Ex.
`
`
`
`
`
`
`-16-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0018
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`2021 at 176:19-177:2.1 The variations in brightness act as a cue from which a
`
`viewer recovers the shape in the depth dimension of the surface. Ex. 2020 at p. 4;
`
`Ex. 2021 at 177:3-8. Thus, color, in this example shading, is being used to create
`
`perceived shape. Perceived shape necessarily implies that different points on the
`
`object have different depths. And therefore, perceived shape implies perceived
`
`depth.
`
`33.
`
`In Figure 5.11 of Dr. Forsyth’s book, the representation of the same
`
`sphere without any shading appears to be flat or 2-dimensional. Ex. 2020 at p. 5.
`
`In other words, the uniform luminance applied across the image of the sphere in
`
`this case creates the perception that the object is something other than a sphere
`
`(e.g., a flat disk). Forsyth Tr. 180:19-23; Ex. 2020 at p. 5.
`
`
`
`1 All citations to Ex. 2021 are deposition transcript pages as opposed to exhibit
`
`pages.
`
`
`
`
`
`
`-17-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0019
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`
`
`34. Finally, Petitioner’s U.S. Patent No. 8,922,628 (“Bond”) also teaches
`
`that monocular depth cues include haze and shadows. Ex. 2005 at 35:35-37. The
`
`Bond patent also implies that these depth cues were well known at the time of the
`
`’081 Patent because they were and are part of the nature of 2D film-making. Id.
`
`35. As another example, a 1994 paper by Kersten et al. entitled “Moving
`
`Cast Shadows and the Perception of Relative Depth” described how increasing the
`
`displacement between shadows cast by three identical squares produces an
`
`impression of different depths for each square relative to the background. Ex.
`
`2023 at p. 2 (Figure 1 and accompanying text). Figure 1, reproduced below,
`
`illustrates this concept.
`
`
`
`
`
`
`-18-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0020
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`
`
`36. As shown in Figure 1, the placement of the shadows, which are
`
`created by varying the luminance (as described below) of the background image
`
`within the shadows, causes the left-most square to appear to be closer to the
`
`background than the right-most square. Thus, Kertsen’s paper teaches how
`
`shadows alter the perceived depth of an object. See also Ex. 2021 at 130:20-22
`
`(referring to Kersten paper).
`
`B. Color, Shading, Perceived Shape, and Perceived Depth
`
`37. The HSL (Hue, Saturation, Luminance) color space is one of a
`
`number of ways of defining color. Hue, Saturation, and Lightness (sometimes
`
`
`
`
`
`
`-19-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0021
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Luminance) was one of the major color spaces known at least as early as the time
`
`of the ’081 Patent. See, e.g., Ex. 2007 at pp. 1, 10; Ex. 2021 at p. 118:6-21
`
`(testifying that lightness and luminance are about the same). At a high level, hue
`
`generally identifies what is commonly referred to as “color.” Ex. 2006 at p. 7.
`
`Saturation of a color identifies how pure or intense the color is. Ex. 2006 at p. 7.
`
`As the saturation decreases, the color gets paler and more washed out. Id. The
`
`luminance of a color is a measure of its perceived brightness. Id. In the HSL color
`
`model, Hue, Saturation, and Luminance can each be represented as numeric values.
`
`See, e.g., Ex. 2007 at pp. 10-13 (e.g., saturation ranges from 0 to 1 and luminance
`
`takes discrete values from 0.0, 0.1, up to 1.0.
`
`38. Dr. Forsyth’s book from around the time of the ’081 Patent also
`
`shows that the HSL color model (or very similar HSV model as he describes it)
`
`was known in the art and understood by POSITAs at the time of the ’081 Patent.
`
`Ex. 2020 at p. 8 (discussing HSV model). Dr. Forsyth acknowledged this in his
`
`deposition as well. Ex. 2021 181:8-182:9. And I agree with him in this regard.
`
`See also Ex. 2007 at p. 1, 10 (discussing HSV color model).
`
`39. As mentioned above, the Saturation and Luminance aspects of color
`
`in the HSL color space are related to shading (luminance), perceived shape, and
`
`perceived depth. Color is thus an important component of several monocular depth
`
`cues that at the time of the ’081 Patent were used to indicate depth ordering of
`
`
`
`
`
`
`-20-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0022
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`objects, perceived relative depth of objects, and/or depth within an object (e.g.,
`
`shape). Ex. 2021 at 77:17-20 (testifying that “[m]onoscopic depth cues can create
`
`a perception of depth”). For example, Dr. Forsyth has testified in this proceeding
`
`that luminance can be used to create a sense of shape and that changes in saturation
`
`can act as cues to depth. Ex. 2021 at 79:12-14. Dr. Forsyth further testified that
`
`adjusting luminance can create the perception that the depth of one point on a
`
`sphere, for example, is different than the depth of another point on the sphere. Ex.
`
`2021 at 87:13-23. In other words, “those two points are perceived to have different
`
`depths.” Ex. 2021 at 88:6-8. I agree with Dr. Forsyth in this respect.
`
`C. Depth Enhancement vs. 2D to 3D Conversion
`
`40. At least as early as the ’081 Patent, color-related (pictorial) depth cues
`
`were used to enhance the sense of depth in a 2D image. See, e.g., Ex. 2009 at pp.
`
`57 (Figure 2.4), 58 (Figure 2.5). As the Bond patent acknowledges, using 2D
`
`depth cues is part of the nature of traditional 2D film-making. Ex. 2005 at 35:35-
`
`38. This was true at the time of the ’081 Patent as well. This type of depth
`
`enhancement can be done whether or not there is a stereoscopic conversion being
`
`done on a particular image.
`
`VII. THE ’793 PATENT AND CHALLENGED CLAIMS
`
`41. The ’793 Patent and its parent patents, the ’081 and ’670 Patents,
`
`teach that an artist can manipulate various parameters, including for example any
`
`
`
`
`
`
`-21-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0023
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`combination of hue, saturation, and luminance. See, e.g., Ex. 1003 at 9:2-8. As
`
`described above, it was well known at the time of the ’081 Patent that altering the
`
`saturation and/or luminance components of the HSL color model as taught by the
`
`’081 Patent creates depth cues that affect depth perception. See, e.g., Ex. 1003 at
`
`9:18-26. For example, the ’081 Patent teaches that the image designer can
`
`“correctly lighten or darken” the color (HSL) of an object as appropriate “with the
`
`introduction of shadows or bright light.” Ex. 1003 at 9:18-26.
`
`42. While this general concept was well-known and understood by
`
`POSITAs by the time of the’081 Patent (see, e.g., Ex. 2009 at p. 57 (Figure 2.4);
`
`Ex. 2007 at p. 10; Ex. 2021 121:14-122:7; Ex. 2023 at p. 2 (Figure 1)), the ’081
`
`Patent introduced the concept of automatically applying the created color effects
`
`for masks in a single key frame to subsequent frames, for example using mask
`
`fitting techniques. Ex. 1003 at Abstract, 1:55-2:6. The ’793 Patent claims the
`
`specific embodiments taught in the ’081 Patent that relate to altering color,
`
`including saturation and luminance, in a way that creates depth cues that affect the
`
`perceived depth of objects. The ’793 Patent also describes stereoscopic conversion
`
`of a two-dimensional image into a three-dimensional image, but nothing in the
`
`claims requires stereoscopic or 3D conversion. Ex. 1001 at 34:58-36:52; see also
`
`Ex. 2021 at 74:9-13, 76:5-12.
`
`
`
`
`
`
`
`
`-22-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0024
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`VIII. CLAIM CONSTRUCTION
`
`43.
`
`In view of the above teachings of the ’081 Patent, and the teachings of
`
`the ’793 Patent, in my opinion, the broadest reasonable interpretation of the term
`
`“depth parameter” is “a parameter used to create a perceived depth of an object or
`
`a region in an image.” My opinion in this regard is based on the intrinsic evidence
`
`from the ’081, ’670, and ’793 Patents, as well as how a POSITA would interpret
`
`this evidence, and on the state of the art at the time of the ’081 Patent. See
`
`Sections VI, VII, VIII, supra and Section X, infra.
`
`44. By way of example, the plain and ordinary meaning for the term
`
`“depth parameter” includes the saturation and luminance parameters of color
`
`taught by the ’081 Patent. As discussed throughout this Declaration (e.g., Sections
`
`VI, VII, VIII, supra and Section X), at the time of the ’081 Patent, the use of
`
`saturation and luminance to create perceived depth was well known in the art. The
`
`saturation and luminance of objects in an image thus “relate[] to the perceived
`
`distance of an object from the camera,” and hence provide examples of depth
`
`parameters under Petitioner’s description of that term. Paper 1 at p. 10; Ex. 1009
`
`at ¶ 39.
`
`IX. THE ’081 AND ’670 PATENTS TEACH THE CLAIMED “DEPTH
`
`PARAMETER.”
`
`45. At the time of the ’081 Patent, a POSITA—a person with an
`
`
`
`
`
`
`-23-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0025
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`undergraduate degree in electrical or computer engineering or equivalent and
`
`several years of experience—would have understood that color, including in
`
`particular saturation and/or luminance as taught by the ’081 Patent, was used to
`
`create depth cues that indicate perceived depths of objects in an image.
`
`46. My opinion is based on intrinsic evidence from the ’081 Patent, as
`
`well as how a POSITA would have interpreted this evidence given the state of the
`
`art. For example, the ’081 Patent teaches that objects in scenes can be classified
`
`into two separate categories: stationary background elements and motion elements
`
`that move through the scene. Ex. 1003 at 1:55-59. Within each of these
`
`categories, the ’081 Patent further teaches that a designer can assign color to a
`
`plurality of pixels in a frame using an HSL color space model. Ex. 1003 at 9:4-6.
`
`The designer can make this assignment “based on creative considerations and the
`
`grayscale and luminance distribution underlying each mask.” Ex. 1003:6-8. The
`
`’081 Patent goes on to explain that “[s]ince the color applies to the feature extends
`
`the entire range of potential grayscale values from dark to light the designer can
`
`insure that as the distribution of the gray-scale values representing the pattern
`
`change homogenously into dark or light regions within subsequent frames of the
`
`movie such as with the introduction of shadows or bright light, the color for each
`
`feature also remains consistently homogenous and correctly lighten or darken with
`
`the pattern upon which it is applied.” Ex. 1003 at 9:17-26.
`
`
`
`
`
`
`-24-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0026
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`47.
`
`In other words, according to the ’081 Patent, the color of the object,
`
`for example the luminance, can be varied to account for the introduction of
`
`shadows or bright light. Ex. 1003 at 9:17-26. In this regard, the ’081 Patent also
`
`teaches that a background mask overlay can be created where the mask overlay
`
`represents designer-selected “color lookup tables in which dynamic pixel colors
`
`automatically compensate or adjust for moving shadows and other changes in
`
`luminance.” Ex. 1003 at 3:47-51; see also id. at 26:66-27:20 (teaching that
`
`