VAZQUEZ DEC.
`IPR2016-01243
`
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`__________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`__________________
`
`
`
`
`Prime Focus Creative Services Canada Inc.,
`Petitioners,
`
`v.
`
`Legend3D, Inc.,
`Patent Owner.
`
`
`
`__________________
`
`Case IPR2016-01243
`
`U.S. Patent No. 7,907,793
`__________________
`
`
`
`DECLARATION OF CARLOS VAZQUEZ, PH.D.
`
`IN SUPPORT OF PATENT OWNER RESPONSE TO THE PETITION AND
`MOTION TO AMEND IN
`IPR2016-01243
`
`
`
`
`
`
`
`
`
`
`
`
`
`Legend3D, Inc. Ex. 2024-0001
`IPR2016-01243
`
`
`IPR2016-01243
`
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`__________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`__________________
`
`
`
`
`Prime Focus Creative Services Canada Inc.,
`Petitioners,
`
`v.
`
`Legend3D, Inc.,
`Patent Owner.
`
`
`
`__________________
`
`Case IPR2016-01243
`
`U.S. Patent No. 7,907,793
`__________________
`
`
`
`DECLARATION OF CARLOS VAZQUEZ, PH.D.
`
`IN SUPPORT OF PATENT OWNER RESPONSE TO THE PETITION AND
`MOTION TO AMEND IN
`IPR2016-01243
`
`
`
`
`
`
`
`
`
`
`
`
`
`Legend3D, Inc. Ex. 2024-0001
`IPR2016-01243
`
`
`
`TABLE OF CONTENTS
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Page
`
`I.
`
`II.
`
`INTRODUCTION ................................................................................ 1
`
`EXPERT QUALIFICATIONS AND CREDENTIALS ....................... 1
`
`III. BASIS FOR OPINIONS, AND MATERIALS REVIEWED .............. 5
`
`IV. LEGAL STANDARDS ........................................................................ 8
`
`A.
`
`B.
`
`FOR PRIORITY AND WRITTEN DESCRIPTION ................. 8
`
`Claim Interpretation in Inter Partes Review .............................. 9
`
`V.
`
`THE PERSON OF ORDINARY SKILL IN THE ART ...................... 9
`
`VI. BACKGROUND ON RELEVANT TECHNOLOGY....................... 11
`
`A. Depth Cues in 2-Diminensional Images .................................. 11
`
`B.
`
`C.
`
`Color, Shading, Perceived Shape, and Perceived Depth .......... 19
`
`Depth Enhancement vs. 2D to 3D Conversion ........................ 21
`
`VII. THE ’793 PATENT AND CHALLENGED CLAIMS ...................... 21
`
`VIII. CLAIM CONSTRUCTION ............................................................... 23
`
`IX. THE ’081 AND ’670 PATENTS TEACH THE CLAIMED “DEPTH
`PARAMETER.” ................................................................................. 23
`
`X. DECLARATION ................................................................................ 28
`
`
`
`
`
`
`
`
`-i-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0002
`IPR2016-01243
`
`
`
`DECLARATION OF DR. CARLOS VAZQUEZ
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`I, Carlos Vazquez, declare as follows:
`
`I.
`
`INTRODUCTION
`
`1.
`
`I am making this Declaration at the request of Legend3D, Inc.
`
`(“Patent Owner”) in the matter of IPR2016-01243, an Inter Partes Review of U.S.
`
`Patent No. 7,709,793 to B. Sandrew (“the ’793 Patent”).
`
`2.
`
`I am being compensated for my work in this matter at a rate of $235
`
`per hour for travel and $470 per hour for my professional consulting services. I am
`
`also reimbursed for my reasonable expenses incurred in connection with my work
`
`in this proceeding. My compensation is not in any way contingent upon the
`
`outcome of this Inter Partes Review. I have no interest in the outcome of this
`
`proceeding or any related litigation.
`
`II. EXPERT QUALIFICATIONS AND CREDENTIALS
`
`3. My qualifications for presenting the opinions in this Declaration are
`
`set forth below and in my curriculum vitae, a copy of which is provided as Ex.
`
`2027.
`
`4.
`
`I have studied, taught, and practiced electrical engineering and
`
`software engineering for over 20 years. I attended the Higher Polytechnic
`
`Institute José Antonio Echeverría (ISPJAE) in Havana, Cuba, and earned a
`
`
`
`
`
`
`-1-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0003
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Bachelor’s (Eng) Degree in Electronics (Electronic Components and Devices)
`
`in 1992, where I graduated summa cum laude, and a Master’s (M.Sc.) of
`
`Science Degree in Applied Informatics (Signal and Image Processing) in 1997. I
`
`earned my Ph.D. in Telecommunications from the Institut National de la
`
`Recherche Scientifique (INRS), in Montréal, Canada, in 2003, where I was also a
`
`post-doctoral fellow from 2002-2004 conducting research on a project related to
`
`the application of Level Set Methods to image segmentation, motion estimation,
`
`and video super resolution.
`
`5.
`
`I am currently an Associate Professor of Software Engineering and
`
`Information Technology at the École de Technologie Supérieure (ETS) in
`
`Montréal, Canada. I have held this position since 2013, teaching courses on
`
`Parallel programming, multimedia systems, digital imaging, and computer
`
`graphics; 3D Vision Systems, and operating systems, and conducting research in
`
`3D video systems, 3D object extraction and reconstruction; medical imaging
`
`applications; 3D cinema; and 3D-TV applications.
`
`6.
`
`Prior to my employment at ETS, I was a research Scientist with the
`
`Communications Research Center (CRC) in Ottawa, Canada, in the Advanced
`
`Video Systems, where from 2005 through 2013 I worked in 3D-TV systems,
`
`namely developing algorithms and applications to help advance the 3D-TV
`
`industry, 2D-to-3D conversion, novel displays, S3D video processing; Multi-View
`
`
`
`
`
`
`-2-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0004
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Video Coding, namely, 4D wavelet-based multi-view video coding, 2D + Depth +
`
`Occlusion multi-view coding and representation, and view synthesis and
`
`occlusions removal; Depth-Image Based Rendering, namely, real time image based
`
`rendering techniques, parallel programming, GP-GPU, among other topics.
`
`7.
`
`I have also worked as a Research Associate in the Electrical and
`
`Computer Engineering Department of Concordia University, Montréal, Canada,
`
`from 2004 to 2005, and as a Research Assistant at the University of Quebec,
`
`Montréal, Canada, from 1998 to 2004 in the areas of software development
`
`and hardware installation and administration areas, namely, video equipment and
`
`stereoscopic video equipment.
`
`8.
`
`Upon graduation with my Bachelor’s degree and while obtaining
`
`my Master’s Degree, I was an Assistant Professor in the Telecommunications
`
`Department of the ISPJAE in Havana, Cuba, where I taught courses on
`
`programming languages, embedded systems, signal acquisition, and processing
`
`from 1992 through 1997.
`
`9. My twenty-plus years of industry experience includes authoring
`
`numerous peer-reviewed journal and conference articles, including attaining as a
`
`co-recipient the Scott Helt best paper award from IEEE Broadcast Technology
`
`Society in 2011. My research interests are in the area of 3D-TV; 2D-to-3D
`
`video conversion; stereo and multi-view vision systems; 3D object extraction
`
`
`
`
`
`
`-3-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0005
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`and reconstruction; image/video representation, sampling and interpolation; image
`
`and video coding; and motion/disparity/depth estimation and compensation.
`
`10.
`
`I have authored over 30 papers at various conferences and symposia
`
`over the past ten-plus years, such as the IEEE International Conference on Image
`
`Processing, the International Workshop on Computer Vision and Its Application to
`
`Image Media Processing, The SPIE Visual Communications and Image
`
`Processing, and SPIE Stereoscopic Displays and Applications. I also am co-
`
`inventor on three U.S. Patents disclosing Generation of a depth map from a
`
`monoscopic color image for rendering stereoscopic still and video images; Method
`
`and graphical interface for modifying depth maps; and Enhancing virtual presence
`
`using multiview and stereoscopic 3D displays with a grounded curvilinear screen.
`
`11.
`
`I became a Senior Member of the IEEE in 2012, and obtained the
`
`2012 Journal Certificate of Merit from SMPTE Motion Imaging Journal. In 2011,
`
`I received the NAB Technology Innovation Award as the co-inventor of one of
`
`the technologies supporting the award: 2D-to-3D video conversion.
`
`12.
`
`I have also authored book chapters on “2D to 3D video conversion-
`
`overview and perspectives” in the book “ Emerging Technologies for 3D
`
`Video: Creation, Coding, Transmission, and Rendering”; and “DIBR-based
`
`conversion from monoscopic to stereoscopic and multiview video” in “ 3D-TV
`
`System with Depth-Image Based Rendering: Architectures, Techniques and
`
`
`
`
`
`
`-4-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0006
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Challenges.”
`
`III. BASIS FOR OPINIONS, AND MATERIALS REVIEWED
`
`13. The opinions set forth in my Declaration are based on my personal
`
`knowledge gained from my education, on my personal experience, and on the
`
`review of the documents and information described in this Declaration.
`
`14.
`
`In preparation of this Declaration, I have considered:
`
` Exhibit 1001: U.S. Patent No. 7,907,793 (“the ‘793 Patent”).
`
` Exhibit 1002: File History for the ‘793 Patent.
`
` Exhibit 1003: U.S. Patent No. 7,181,081 (“the ‘081 Patent”).
`
` Exhibit 1004: U.S. Patent No. 7,333,670 (“the ‘670 Patent”).
`
` Exhibit 1005: U.S. Patent Application No. 12/241,992 (“Passmore”).
`
` Exhibit 1006: U.S. Patent No. 7,573,475 (“Sullivan”).
`
` Exhibit 1007: Declaration of Rob S. Schmitt from Business Wire, Inc.
`
`attaching: Exhibit 1: “Legend Films Successfully Raise $5 Million in Venture
`
`Capital Funding” (press release); and Exhibit 2: “‘Night of the Living Dead’ to
`
`Be Released in Color and 3D” (press release).
`
` Exhibit 1008: Curriculum Vitae of Dr. David Forsyth.
`
` Exhibit 1009: Corrected Expert Declaration of Dr. David Forsyth.
`
` Exhibit 1010: U.S. Patent No. 7,577,312.
`
`
`
`
`
`
`-5-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0007
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
` Exhibit 1011: Article: “How do 3D films work?” available at
`
`http//www.physics.org/article-questions.asp?id=56.
`
` Exhibit 1012: Book excerpt: Rendering Techniques 2001, edited by Steven J.
`
`Gortler and Karol Myszkowski (2001).
`
` Exhibit 1013: Article: Matt Sullivan, Trick and Treat: Behind the Scenes of the
`
`New Nightmare Before Christmas and the 3D Movie Revolution, Popular
`
`Mechanics, available at
`
`http://www.popularmechanics.com/technology/gadgets/a3607/4200796/.
`
` Exhibit 1014: Redline of specification from ‘081 Patent to ‘793 Patent.
`
` Exhibit 1015: Redline of specification from ‘670 Patent to ‘793 Patent.
`
` Exhibit 2005: U.S. Patent No. 8,922,628 (Petitioner’s ‘628 Patent).
`
` Exhibit 2006: Article: Jonathan Sachs, Digital Image Basics, Digital Light &
`
`Color, 1996-1999.
`
` Exhibit 2007: Article: Paul Bourke, HSL colour space, June 2000.
`
` Exhibit 2008: Archived web page: ColorModels.html, available at
`
` http://cs.brown.edu/courses/cs092/VA10/HTML/ColorModels.html, from
`
`Brown University course CS092, archived Jan 24, 2000.
`
` Exhibit 2009: Book: Lenny Lipton, Foundations of the Stereoscopic Cinema –
`
`A Study in Depth, 1982.
`
`
`
`
`
`
`-6-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0008
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
` Exhibit 2010: Article: Dorothy A. Kleffner and V. S. Ramachandran, On the
`
`Perception of Shape from Shading, Perception & Psychophysics 1992, 52 (1),
`
`18-36.
`
` Exhibit 2011: Version d.1 for the MS-DOS and PC-DOS operating System,
`
`Chapter 1, Borland International, 1992.
`
` Exhibit 2012: Archived web page: GIMP core_blend_tool.html, available at
`
` http://www.gimp.org/core_blend_tool.html, archived August 17, 2000.
`
` Exhibit 2013: Archived web page: GIMP transforms.html, available at
`
` http://manual.gimp.org/manual/GUM/transforms.html, archived March 5, 2001.
`
` Exhibit 2014: Book: Adobe Photoshop 6.0 User Guide, Adobe Systems
`
`Incorporated, 2000.
`
` Exhibit 2015: U.S. Patent No. 8,396,328 (First child Patent to the ‘793 Patent).
`
` Exhibit 2016: Internet Archive Affidavit for Exhibits 2008, 2012 and 2013.
`
` Exhibit 2018: Deposition of David Forsyth, Ph.D. taken March 9, 2017 for Case
`
`No. IPR2016-00806, re U.S. Patent No. 8,922,628.
`
` Exhibit 2019: Colored photos: Virgin of the Rocks.
`
` Exhibit 2020: Book: David A. Forsyth, Computer Vision – A Modern
`
`Approach, 2003 (excerpts).
`
`
`
`
`
`
`-7-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0009
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
` Exhibit 2021: Deposition of David Forsyth, Ph.D. taken March 20, 2017 for
`
`Case No. IPR2016-01243, re U.S. Patent No. 7,907,793.
`
` Exhibit 2022: U.S. Patent No. 4,925,294.
`
` Exhibit 2023: Article: Moving Cast Shadows and the Perception of Relative
`
`Depth, Technical Report No. 6, June 1994.
`
` Exhibit 2024: Declaration of Carlos Vazquez.
`
` Exhibit 2025: U.S. Patent No. 6,141,433.
`
` Exhibit 2026: U.S. Patent No. 5,940,528.
`
` Exhibit 2027: Curriculum Vitae of Carlos Vazquez.
`
`IV. LEGAL STANDARDS
`
`A.
`
`15.
`
`FOR PRIORITY AND WRITTEN DESCRIPTION
`
`I am not an attorney. In preparing and expressing my opinions and
`
`considering the subject matter of the ’793 Patent, I am relying on certain basic
`
`legal principles that have been explained to me. These principles are discussed
`
`below.
`
`16.
`
`I have been informed that in order for a claim to get priority to an
`
`earlier-filed application, it must be shown that any claim term that does not appear
`
`in the written description of the patent is necessarily comprehended in the
`
`description provided and would have been so understood at the time the patent
`
`application was filed. I have also been informed that the patent disclosure need
`
`
`
`
`
`
`-8-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0010
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`only reasonably convey to persons skilled in the art that the inventor had
`
`possession of the subject matter in question.
`
`B. Claim Interpretation in Inter Partes Review
`
`17.
`
`I understand that the first step in determining the validity of a claim is
`
`for the claim to be properly construed.
`
`18. For purposes of Inter Partes Review, I understand that each
`
`challenged claim must be given its broadest reasonable interpretation in light of the
`
`patent specification. I am informed that the broadest reasonable interpretation
`
`dictates that claim terms be given their ordinary and customary meaning, as would
`
`be understood by a POSITA in the context of the entire disclosure or specification
`
`of the patent, unless the inventor, as a lexicographer, has set forth a special
`
`meaning for a term. I understand that a patent’s “specification” includes all the
`
`figures, discussion, and claims within the patent document. I understand that the
`
`USPTO will look to the specification to see if there is a definition for a claim term,
`
`and if not, will apply the broadest reasonable interpretation from the perspective of
`
`a POSITA.
`
`19.
`
`I provide my opinions in this Declaration based on the guidelines set
`
`forth above.
`
`V. THE PERSON OF ORDINARY SKILL IN THE ART
`
`20.
`
`I have reviewed the ’793 Patent (Ex. 1001), the ’081 Patent (Ex.
`
`
`
`
`
`
`-9-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0011
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`1003), the ’670 Patent (Ex. 1004), the prior art documents and other printed
`
`publications referenced above, and the Corrected Declaration of David Forsyth
`
`(Ex. 1009). My opinion regarding the level of one of ordinary skill in the art is
`
`based on this review and my knowledge of image analysis and enhancement.
`
`21. The ’793 Patent describes the field of the invention as follows: “One
`
`or more embodiments of the invention are related to the field of image analysis and
`
`image enhancement . . . .” Ex. 1001 at 1:19-20. Based on the ’793 Patent’s
`
`description, my opinion is that a POSITA in the field of the ’793 Patent as well as
`
`the ’081 and ’670 Patents at the time of the invention would have had at least an
`
`undergraduate degree in electrical engineering or computer science (or an
`
`equivalent subject) and would have been someone with a good working knowledge
`
`of computer programming, data structures, and image processing. In addition to
`
`the undergraduate degree, the person would have gained this knowledge through
`
`several years of practical working experience. This description is approximate,
`
`and a higher level of education or skill might make up for less experience, and
`
`vice-versa.
`
`22.
`
`I believe that I would qualify as at least a POSITA at all relevant
`
`times for purposes of this proceeding, including at least as early as the priority date
`
`of the ’081 Patent. I have a sufficient level of knowledge, experience, and
`
`education to provide an opinion in the field of the ’793 Patent. Specifically, I work
`
`
`
`
`
`
`-10-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0012
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`with many people, including students, researchers that I supervise, and colleagues
`
`in the industry, who are POSITAs. I therefore understand what a POSITA is and
`
`would have been as of the priority date of the ’081 Patent. It is through this
`
`viewpoint that I make my conclusions in this matter.
`
`23. My opinion of the level of ordinary skill in the art is based on my
`
`extensive personal and professional experience working in the field of image
`
`processing, and on my knowledge of the level of education and experience of
`
`colleagues and others actively working in the field as of and for several years prior
`
`to the priority date of the ’081 Patent, for example.
`
`24.
`
`I understand that a POSITA in the art is not a specific, real individual,
`
`but rather a hypothetical individual presumed to have knowledge of the relevant art
`
`at the time of the invention.
`
`VI. BACKGROUND ON RELEVANT TECHNOLOGY
`
`A. Depth Cues in 2-Diminensional Images
`
`25.
`
`In two-dimensional images, it was well-known at the time of the ’081
`
`Patent (i.e., 2001-2002) that monocular depth cues were used in photographs and
`
`films in order to realistically depict scenes from our three-dimensional world on a
`
`two-dimensional plane (e.g., a movie screen or photograph). Ex. 2009 at p. 20.
`
`This was well-recognized in Lenny Lipton’s book entitled “Foundations of the
`
`Stereoscopic Cinema—A Study in Depth,” which was published in 1982. Ex.
`
`
`
`
`
`
`-11-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0013
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`2009.
`
`26. Monocular depth cues were known to Renaissance painters in the
`
`1500s, and codified by Leonardo for example. Ex. 2009 at p. 19. Almost all depth
`
`cues can be perceived on a flat screen. Ex. 2009 at p. 54. Depth cues include
`
`aerial perspective, as well as shadows and shading. Ex. 2009 at pp. 57-58.
`
`27. The aerial perspective depth cue necessarily causes hazy objects to
`
`appear to be farther away than objects for which there is no intervening haze. The
`
`color image below shows a lower saturation of colors for background objects. In
`
`other words, the haze depth cue is created in the photograph below using a
`
`modification to the saturation component of colors. It can be appreciated in the
`
`below photograph that the mountains in the background appear farther away than
`
`those of the foreground.
`
`
`
`
`
`
`
`
`-12-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0014
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`28. Lipton taught this principle as well. Figure 2.4 of his book shows an
`
`aerial perspective in which “haze contributes to making the background look far
`
`away.” Ex. 2009 at p. 57. As is clearly shown, the haze has the effect of making
`
`the trees in the background look farther away than the darker trees in the
`
`foreground.
`
`
`
`29. Shadings and shadows provide visual cues regarding the shape of
`
`objects and their relative distance from one another. Ex. 2009 at pp. 57, 58 (Fig.
`
`2.5). Lipton taught that “[c]ast shadows provide an effective depth cue, as does
`
`light coming from one or more directions.” 2009 at p. 57. Indeed, Lipton’s Figure
`
`
`
`
`
`
`-13-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0015
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`2.5 (reproduced below) uses light and shade as depth cues to create different
`
`perceived shapes. Ex. 2009 at p. 58. If the light is assumed to be coming down
`
`from above (top to bottom), then the shadow on the left shape causes it to appear to
`
`pop out whereas the shadow on the right shape causes it to appear to pop in.
`
`
`
`30. The creation of an impression of perceived depth in 2-dimensional
`
`images using only subtle variations in shading was well-known at the time of the
`
`’081 Patent. Ex. 2010 at p. 2. Shading was thus considered a monocular depth cue
`
`at the time of the ’081 Patent. Ex. 2010 at p. 17. For example, a 1992 publication
`
`by Kleffner and Ramachandran described that “Figure 1 [reproduced below]
`
`depicts a set of objects that conveys a strong impression of depth.” Ex. 2010 at p.
`
`
`
`
`
`
`-14-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0016
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`3. The objects in Figure 1 of Kleffner “convey an impression of depth based
`
`exclusively on subtle variations in luminance.” Ex. 2010 at p. 3. Thus, even
`
`absent other depth cues, the use of shading (or variation in luminance) to convey a
`
`perception of depth was understood at the time of the ’081 Patent. Ex. 2010 at p. 3
`
`(Figure 3 caption).
`
`
`
`31. A book co-authored by Petitioner’s expert Dr. Forsyth around the time
`
`of the ’081 Patent also demonstrates how shading serves as a depth cue. For
`
`
`
`
`
`
`-15-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0017
`IPR2016-01243
`
`
`
`example, in Figure 5.10 (reproduced below), Dr. Forsyth’s book shows five two-
`
`dimensional views of images of a sphere. Ex. 2020 at p. 4.
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`
`
`32. Each of the five views shows the sphere with self-shadows created
`
`from variations in brightness distributed over the sphere. Ex. 2020 at p. 4; Ex.
`
`
`
`
`
`
`-16-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0018
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`2021 at 176:19-177:2.1 The variations in brightness act as a cue from which a
`
`viewer recovers the shape in the depth dimension of the surface. Ex. 2020 at p. 4;
`
`Ex. 2021 at 177:3-8. Thus, color, in this example shading, is being used to create
`
`perceived shape. Perceived shape necessarily implies that different points on the
`
`object have different depths. And therefore, perceived shape implies perceived
`
`depth.
`
`33.
`
`In Figure 5.11 of Dr. Forsyth’s book, the representation of the same
`
`sphere without any shading appears to be flat or 2-dimensional. Ex. 2020 at p. 5.
`
`In other words, the uniform luminance applied across the image of the sphere in
`
`this case creates the perception that the object is something other than a sphere
`
`(e.g., a flat disk). Forsyth Tr. 180:19-23; Ex. 2020 at p. 5.
`
`
`
`1 All citations to Ex. 2021 are deposition transcript pages as opposed to exhibit
`
`pages.
`
`
`
`
`
`
`-17-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0019
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`
`
`34. Finally, Petitioner’s U.S. Patent No. 8,922,628 (“Bond”) also teaches
`
`that monocular depth cues include haze and shadows. Ex. 2005 at 35:35-37. The
`
`Bond patent also implies that these depth cues were well known at the time of the
`
`’081 Patent because they were and are part of the nature of 2D film-making. Id.
`
`35. As another example, a 1994 paper by Kersten et al. entitled “Moving
`
`Cast Shadows and the Perception of Relative Depth” described how increasing the
`
`displacement between shadows cast by three identical squares produces an
`
`impression of different depths for each square relative to the background. Ex.
`
`2023 at p. 2 (Figure 1 and accompanying text). Figure 1, reproduced below,
`
`illustrates this concept.
`
`
`
`
`
`
`-18-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0020
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`
`
`36. As shown in Figure 1, the placement of the shadows, which are
`
`created by varying the luminance (as described below) of the background image
`
`within the shadows, causes the left-most square to appear to be closer to the
`
`background than the right-most square. Thus, Kertsen’s paper teaches how
`
`shadows alter the perceived depth of an object. See also Ex. 2021 at 130:20-22
`
`(referring to Kersten paper).
`
`B. Color, Shading, Perceived Shape, and Perceived Depth
`
`37. The HSL (Hue, Saturation, Luminance) color space is one of a
`
`number of ways of defining color. Hue, Saturation, and Lightness (sometimes
`
`
`
`
`
`
`-19-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0021
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`Luminance) was one of the major color spaces known at least as early as the time
`
`of the ’081 Patent. See, e.g., Ex. 2007 at pp. 1, 10; Ex. 2021 at p. 118:6-21
`
`(testifying that lightness and luminance are about the same). At a high level, hue
`
`generally identifies what is commonly referred to as “color.” Ex. 2006 at p. 7.
`
`Saturation of a color identifies how pure or intense the color is. Ex. 2006 at p. 7.
`
`As the saturation decreases, the color gets paler and more washed out. Id. The
`
`luminance of a color is a measure of its perceived brightness. Id. In the HSL color
`
`model, Hue, Saturation, and Luminance can each be represented as numeric values.
`
`See, e.g., Ex. 2007 at pp. 10-13 (e.g., saturation ranges from 0 to 1 and luminance
`
`takes discrete values from 0.0, 0.1, up to 1.0.
`
`38. Dr. Forsyth’s book from around the time of the ’081 Patent also
`
`shows that the HSL color model (or very similar HSV model as he describes it)
`
`was known in the art and understood by POSITAs at the time of the ’081 Patent.
`
`Ex. 2020 at p. 8 (discussing HSV model). Dr. Forsyth acknowledged this in his
`
`deposition as well. Ex. 2021 181:8-182:9. And I agree with him in this regard.
`
`See also Ex. 2007 at p. 1, 10 (discussing HSV color model).
`
`39. As mentioned above, the Saturation and Luminance aspects of color
`
`in the HSL color space are related to shading (luminance), perceived shape, and
`
`perceived depth. Color is thus an important component of several monocular depth
`
`cues that at the time of the ’081 Patent were used to indicate depth ordering of
`
`
`
`
`
`
`-20-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0022
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`objects, perceived relative depth of objects, and/or depth within an object (e.g.,
`
`shape). Ex. 2021 at 77:17-20 (testifying that “[m]onoscopic depth cues can create
`
`a perception of depth”). For example, Dr. Forsyth has testified in this proceeding
`
`that luminance can be used to create a sense of shape and that changes in saturation
`
`can act as cues to depth. Ex. 2021 at 79:12-14. Dr. Forsyth further testified that
`
`adjusting luminance can create the perception that the depth of one point on a
`
`sphere, for example, is different than the depth of another point on the sphere. Ex.
`
`2021 at 87:13-23. In other words, “those two points are perceived to have different
`
`depths.” Ex. 2021 at 88:6-8. I agree with Dr. Forsyth in this respect.
`
`C. Depth Enhancement vs. 2D to 3D Conversion
`
`40. At least as early as the ’081 Patent, color-related (pictorial) depth cues
`
`were used to enhance the sense of depth in a 2D image. See, e.g., Ex. 2009 at pp.
`
`57 (Figure 2.4), 58 (Figure 2.5). As the Bond patent acknowledges, using 2D
`
`depth cues is part of the nature of traditional 2D film-making. Ex. 2005 at 35:35-
`
`38. This was true at the time of the ’081 Patent as well. This type of depth
`
`enhancement can be done whether or not there is a stereoscopic conversion being
`
`done on a particular image.
`
`VII. THE ’793 PATENT AND CHALLENGED CLAIMS
`
`41. The ’793 Patent and its parent patents, the ’081 and ’670 Patents,
`
`teach that an artist can manipulate various parameters, including for example any
`
`
`
`
`
`
`-21-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0023
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`combination of hue, saturation, and luminance. See, e.g., Ex. 1003 at 9:2-8. As
`
`described above, it was well known at the time of the ’081 Patent that altering the
`
`saturation and/or luminance components of the HSL color model as taught by the
`
`’081 Patent creates depth cues that affect depth perception. See, e.g., Ex. 1003 at
`
`9:18-26. For example, the ’081 Patent teaches that the image designer can
`
`“correctly lighten or darken” the color (HSL) of an object as appropriate “with the
`
`introduction of shadows or bright light.” Ex. 1003 at 9:18-26.
`
`42. While this general concept was well-known and understood by
`
`POSITAs by the time of the’081 Patent (see, e.g., Ex. 2009 at p. 57 (Figure 2.4);
`
`Ex. 2007 at p. 10; Ex. 2021 121:14-122:7; Ex. 2023 at p. 2 (Figure 1)), the ’081
`
`Patent introduced the concept of automatically applying the created color effects
`
`for masks in a single key frame to subsequent frames, for example using mask
`
`fitting techniques. Ex. 1003 at Abstract, 1:55-2:6. The ’793 Patent claims the
`
`specific embodiments taught in the ’081 Patent that relate to altering color,
`
`including saturation and luminance, in a way that creates depth cues that affect the
`
`perceived depth of objects. The ’793 Patent also describes stereoscopic conversion
`
`of a two-dimensional image into a three-dimensional image, but nothing in the
`
`claims requires stereoscopic or 3D conversion. Ex. 1001 at 34:58-36:52; see also
`
`Ex. 2021 at 74:9-13, 76:5-12.
`
`
`
`
`
`
`
`
`-22-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0024
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`VIII. CLAIM CONSTRUCTION
`
`43.
`
`In view of the above teachings of the ’081 Patent, and the teachings of
`
`the ’793 Patent, in my opinion, the broadest reasonable interpretation of the term
`
`“depth parameter” is “a parameter used to create a perceived depth of an object or
`
`a region in an image.” My opinion in this regard is based on the intrinsic evidence
`
`from the ’081, ’670, and ’793 Patents, as well as how a POSITA would interpret
`
`this evidence, and on the state of the art at the time of the ’081 Patent. See
`
`Sections VI, VII, VIII, supra and Section X, infra.
`
`44. By way of example, the plain and ordinary meaning for the term
`
`“depth parameter” includes the saturation and luminance parameters of color
`
`taught by the ’081 Patent. As discussed throughout this Declaration (e.g., Sections
`
`VI, VII, VIII, supra and Section X), at the time of the ’081 Patent, the use of
`
`saturation and luminance to create perceived depth was well known in the art. The
`
`saturation and luminance of objects in an image thus “relate[] to the perceived
`
`distance of an object from the camera,” and hence provide examples of depth
`
`parameters under Petitioner’s description of that term. Paper 1 at p. 10; Ex. 1009
`
`at ¶ 39.
`
`IX. THE ’081 AND ’670 PATENTS TEACH THE CLAIMED “DEPTH
`
`PARAMETER.”
`
`45. At the time of the ’081 Patent, a POSITA—a person with an
`
`
`
`
`
`
`-23-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0025
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`undergraduate degree in electrical or computer engineering or equivalent and
`
`several years of experience—would have understood that color, including in
`
`particular saturation and/or luminance as taught by the ’081 Patent, was used to
`
`create depth cues that indicate perceived depths of objects in an image.
`
`46. My opinion is based on intrinsic evidence from the ’081 Patent, as
`
`well as how a POSITA would have interpreted this evidence given the state of the
`
`art. For example, the ’081 Patent teaches that objects in scenes can be classified
`
`into two separate categories: stationary background elements and motion elements
`
`that move through the scene. Ex. 1003 at 1:55-59. Within each of these
`
`categories, the ’081 Patent further teaches that a designer can assign color to a
`
`plurality of pixels in a frame using an HSL color space model. Ex. 1003 at 9:4-6.
`
`The designer can make this assignment “based on creative considerations and the
`
`grayscale and luminance distribution underlying each mask.” Ex. 1003:6-8. The
`
`’081 Patent goes on to explain that “[s]ince the color applies to the feature extends
`
`the entire range of potential grayscale values from dark to light the designer can
`
`insure that as the distribution of the gray-scale values representing the pattern
`
`change homogenously into dark or light regions within subsequent frames of the
`
`movie such as with the introduction of shadows or bright light, the color for each
`
`feature also remains consistently homogenous and correctly lighten or darken with
`
`the pattern upon which it is applied.” Ex. 1003 at 9:17-26.
`
`
`
`
`
`
`-24-
`
`
`
`
`Legend3D, Inc. Ex. 2024-0026
`IPR2016-01243
`
`
`
`VAZQUEZ DEC.
`IPR2016-01243
`
`
`47.
`
`In other words, according to the ’081 Patent, the color of the object,
`
`for example the luminance, can be varied to account for the introduction of
`
`shadows or bright light. Ex. 1003 at 9:17-26. In this regard, the ’081 Patent also
`
`teaches that a background mask overlay can be created where the mask overlay
`
`represents designer-selected “color lookup tables in which dynamic pixel colors
`
`automatically compensate or adjust for moving shadows and other changes in
`
`luminance.” Ex. 1003 at 3:47-51; see also id. at 26:66-27:20 (teaching that
`
`
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
