`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`
`)
`)
`)
`)
`)
`
`
`
`In re Inter Partes Review of:
`U.S. Patent No. 8,922,628
`Issued: December 30, 2014
`Application No.: 12/874,190
`Filing Date: September 1, 2010
`
`For: System and Process for Transforming Two-Dimensional Images Into
`Three-Dimensional Images
`
`FILED VIA PRPS
`
`
`
`
`
`
`DECLARATION OF KENNETH ZEGER, Ph.D. IN SUPPORT OF
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 8,922,628
`
`
`
`
`
`
`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`TABLE OF CONTENTS
`
`
`I. INTRODUCTION ............................................................................................. 1
`II. QUALIFICATIONS ......................................................................................... 3
`III. UNDERSTANDING OF THE GOVERNING LAW ................................... 5
`A. INVALIDITY BY ANTICIPATION OR OBVIOUSNESS ............................................. 6
`B. INTERPRETING CLAIMS BEFORE THE PATENT OFFICE ....................................... 7
`IV. LEVEL OF ORDINARY SKILL IN THE ART .......................................... 7
`V. OVERVIEW OF THE ‘628 PATENT ............................................................ 9
`A. DISCLOSURE OF THE ‘628 PATENT ................................................................... 9
`VI. TECHNOLOGY BACKGROUND CIRCA 1986 ....................................... 12
`VII. OVERVIEW OF THE PRIOR ART .......................................................... 17
`A. U.S. PATENT NO. 8,488,868 .......................................................................... 19
`B. U.S. PATENT NO. 8,213,711 .......................................................................... 21
`C. MOTIVATION TO COMBINE TAM ‘868 WITH TAM ‘711 ................................... 24
`VIII. SUMMARY OF OPINIONS ...................................................................... 25
`IX. CLAIM CONSTRUCTION .......................................................................... 25
`A. “DEPTH SELECTION MASK” ............................................................................. 25
`B. “IMAGE FEATURES” ........................................................................................ 28
`C. “CHARACTERISTIC OF THE WORKING IMAGE” ................................................. 29
`D. “CREATING A DEPTH SELECTION MASK ASSOCIATED WITH AT LEAST ONE
`CHARACTERISTIC OF THE WORKING IMAGE AND ASSOCIATED WITH AT LEAST A
`PORTION OF THE WORKING IMAGE” ....................................................................... 30
`E. “REAL VALUE IN A NORMALIZED RANGE” ....................................................... 31
`F. “VECTOR FIELD OF EACH PIXEL” ..................................................................... 33
`G. “WEIGHTED DISPLACEMENT MASK” ................................................................ 33
`H. “WEIGHTED DISTRIBUTION OF THE IMAGE CHARACTERISTIC” ......................... 34
`X. ‘628 PATENT PROSECUTION HISTORY ................................................ 35
`XI. ANTICIPATION ........................................................................................... 37
`A. TAM ‘868 ANTICIPATES CLAIMS 1-5 AND 7-11 OF THE ‘628 PATENT ............. 37
`XII. OBVIOUSNESS ............................................................................................ 59
`A. TAM ‘868 IN VIEW OF TAM ‘711 RENDERS CLAIMS 6 AND 12-18 OF THE ‘628
`PATENT OBVIOUS .................................................................................................. 59
`XIII. CONCLUSION ............................................................................................ 89
`
` i
`
`
`
`
`
`I.
`
`INTRODUCTION
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`I, Kenneth Zeger, declare as follows:
`
`1.
`
`I have been retained to provide expert testimony in support of
`
`Legend3D, Inc. (“Petitioner”) in its Petition for Inter Partes Review of U.S. Patent
`
`No. 8,922,628 (“Petition”).
`
`2.
`
`I understand this proceeding involves U.S. Patent No. 8,922,628 (the
`
`‘628 patent”), titled “System and Process for Transforming Two-Dimensional
`
`Images Into Three-Dimensional Images”. I further understand that the ‘628 patent
`
`issued Dec. 30, 2014, based on U.S. utility patent application no. 12/874,190 filed
`
`Sep. 1, 2010, which claims priority to U.S. provisional patent application
`
`61/239,049, (the ‘049 provisional), filed Sep. 1, 2009. (“Earliest Potential Filing
`
`Date”). A copy of the ‘628 patent is attached as Ex. 1001, its file history is attached
`
`as Ex. 1002 and the ‘049 provisional is attached as Ex. 1003.
`
`3.
`
`I understand that according to United States Patent and Trademark
`
`Office (“USPTO”) records, the '628 patent was assigned to Prime Focus VFX
`
`Services II, Inc., but there is a subsequent recorded document showing the owner
`
`(or former owner) as Prime Focus VFX USA, Inc. (“Patent Owner”).
`
`4.
`
`I understand that Petitioner challenges in its Petition the validity of
`
`claims 1-18 of the ‘628 patent (“challenged claims”). The challenged claims
`
`1
`
`
`
`
`
`include two independent claims, claims 1 and 12; and 16 dependent claims, claims
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`2-11 and 13-18.
`
`5.
`
`I have reviewed and am familiar with the ‘628 patent as well as its file
`
`history along with the provisional to which the ‘628 patent claims priority.
`
`6.
`
`I have also reviewed and am familiar with U.S. Patent No. 8,488,868,
`
`(“Tam ‘868”) (Ex. 1004) and U.S. Patent No. 8,213,711, (“Tam ‘711”) (Ex. 1006),
`
`the primary prior art references cited in the Petition, as well as Provisional Appl.
`
`Nos. 61/129,869, (“ ‘869 Provisional”), filed 7/25/2008 as well as 60/907,475, (“
`
`‘475 Provisional”), filed 4/3/2007 to which Tam ‘868 and Tam ‘711 claim priority.
`
`7.
`
`As set forth below, I am familiar with the technology at issue as of the
`
`Earliest Potential Filing Date.
`
`8.
`
`I have been asked to provide my technical analysis and opinions
`
`regarding the ‘628 patent which form the basis for the grounds of invalidity set
`
`forth in the Petition. In forming my opinions, I relied on my own experience and
`
`knowledge as well as the Tam ‘868, Tam ‘711, ‘869 Provisional and ‘475
`
`Provisional, and the ‘628 patent and its file history.
`
`9.
`
`I have also relied on additional data and facts of the type that experts
`
`in the field of invention would reasonably rely upon in forming an opinion on the
`
`subject. I have relied on such information in order to support and explain my
`
`opinion as to how a person of ordinary skill in the art at the time of the alleged
`
`2
`
`
`
`
`
`invention of the ‘628 patent would have interpreted the disclosures of the prior art
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`references set forth in the Petition. (See Corning Inc. v. DSM IP Assets B.V.,
`
`IPR2013-00046, Paper 12, at 23 (“It is not the disclosure of the prior art references
`
`that determines unpatentability, but how a person of ordinary skill would interpret
`
`that disclosure.”) Where relevant, I cite such references in this declaration.
`
`II. QUALIFICATIONS
`
`10.
`
`I have studied, taught, and practiced electrical and computer
`
`engineering for thirty years. I attended the Massachusetts Institute of Technology
`
`(“MIT”) and earned a Bachelors (SB) and Masters (SM) of Science Degrees in
`
`Electrical Engineering and Computer Science in 1984. I earned a Masters of Arts
`
`(MA) Degree in Mathematics in 1989 from the University of California, Santa
`
`Barbara. I also earned my Ph.D. in Electrical and Computer Engineering from the
`
`University of California, Santa Barbara in 1990.
`
`11.
`
`I am a Full Professor of Electrical and Computer Engineering at the
`
`University of California, San Diego (UCSD). I have held this position since 1998,
`
`having been promoted from Associated Professor after two years at UCSD. I teach
`
`courses full-time at UCSD in the fields of Electrical and Computer Engineering,
`
`and specifically in subfields including information theory, data compression,
`
`speech coding, and image coding, at the undergraduate and graduate levels. Prior
`
`to my employment at UCSD, I taught and conducted research as a faculty member
`
`3
`
`
`
`
`
`at the University of Illinois, Urbana-Champaign for four years, and at the
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`University of Hawaii for two years.
`
`12. My twenty-plus years of industry experience includes consulting work
`
`for the United States Department of Defense as well as for private companies such
`
`as Xerox, Nokia, MITRE, ADP, and Hewlett-Packard. The topics upon which I
`
`provide consulting expertise include image, video, and speech coding; data
`
`compression; networks; digital communications; pattern recognition; computer
`
`software; and mathematical analyses.
`
`13.
`
`I have authored approximately 70 peer-reviewed journal articles, the
`
`majority of which are on the topic of compression or signal processing. I have also
`
`authored over 100 papers at various conferences and symposia over the past
`
`twenty-plus years, such as the IEEE International Symposium on Information
`
`Theory, the International Conference on Image Processing, and the Data
`
`Compression Conference. I also am co-inventor on a U.S. patent disclosing a
`
`memory saving technique for image compression.
`
`14.
`
`I was elected a Fellow of the IEEE in 2000, an honor bestowed upon
`
`only a small percentage of IEEE members. I was awarded the National Science
`
`Foundation Presidential Young Investigator Award in 1991, which included
`
`$500,000 in research funding. I received this award one year after receiving my
`
`Ph.D.
`
`4
`
`
`
`
`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`15.
`
`I have served as an Associate Editor for the IEEE Transactions on
`
`Information Theory and have been an elected member of the IEEE Information
`
`Theory Board of Governors for three, three-year terms. I organized and have been
`
`on the technical advisory committees of numerous workshops and symposia in the
`
`areas of image coding, information theory, and data compression. I regularly
`
`review submitted journal manuscripts, government funding requests, conference
`
`proposals, student theses, and textbook proposals. I also have given many lectures
`
`at conferences, universities, and companies on topics in image coding, data
`
`compression, and information theory.
`
`16.
`
`I have extensive experience in electronics hardware and computer
`
`software, from academic studies, work experience, and supervising students. I
`
`personally program computers on an almost daily basis and have fluency in many
`
`different computer languages.
`
`17. My Curriculum Vitae is provided as Ex. 1010.
`
`18. Zunda LLC is being compensated for my work in this matter at a rate
`
`of $750 per hour, with reimbursement for necessary and reasonable expenses. This
`
`compensation is not in any way contingent upon the outcome of this Inter Partes
`
`Review. Neither I nor Zunda LLC has any interest in the outcome of this
`
`proceeding or any related litigation.
`
`III. UNDERSTANDING OF THE GOVERNING LAW
`
`5
`
`
`
`
`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`A.
`
`Invalidity by Anticipation or Obviousness
`
`19.
`
`I understand that a claim is invalid if it is anticipated or obvious. I
`
`understand that anticipation requires that every element of a claim is disclosed
`
`expressly or inherently in a single prior art reference, as arranged in the claim.
`
`20.
`
`I understand that a claim is invalid as obvious if it would have been
`
`obvious from the perspective of a person of ordinary skill in the relevant art, at the
`
`time the invention was made. In analyzing obviousness, I understand that it is
`
`important to understand the scope of the claims, the level of skill in the relevant
`
`art, the scope and content of the prior art, the differences between the prior art and
`
`the claims, and any secondary considerations.
`
`21.
`
`I also understand that if a technique has been used to improve one
`
`device, and a person of ordinary skill in the art would recognize that it would
`
`improve similar devices in the same way, using the technique is obvious unless its
`
`actual application is beyond his or her skill. There may also be a specific teaching,
`
`suggestion or motivation to combine any first prior art reference with a second
`
`prior art reference. Such teaching, suggestion, or motivation to combine can be
`
`explicit or implicit in the first or second prior art references as per KSR Int’l Co. v.
`
`Teleflex, Inc. It is my understanding that KSR confirms that any motivation that
`
`would have been known to a person of skill in the art, including common sense, or
`
`6
`
`
`
`
`
`derived from the nature of the problem to be solved, is sufficient to explain why
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`references would have been combined.
`
`22.
`
` I understand that the earliest filed patent application to which the
`
`‘628 patent claims priority was filed September 1, 2009. I have therefore analyzed
`
`the prior art references discussed herein from the perspective of a person of
`
`ordinary skill in the art in the 2009 time frame, understanding that as time passes,
`
`the knowledge of a person of ordinary skill in the art will increase.
`
`B.
`
`Interpreting Claims Before the Patent Office
`
`23.
`
`I understand that in an Inter Partes Review proceeding, claims are
`
`given their broadest reasonable interpretation that is consistent with the patent
`
`specification. I understand that a patent’s “specification” includes all the figures,
`
`discussion, and claims within the patent document. I understand that the USPTO
`
`will look to the specification to see if there is a definition for a claim term, and if
`
`not, will apply the broadest reasonable interpretation from the perspective of a
`
`person of ordinary skill in the art. I have reviewed the ‘628 file history in making
`
`my conclusions. I present a more detailed explanation of the interpretation of
`
`certain terms in the ‘628 patent in section IX below.
`
`IV. LEVEL OF ORDINARY SKILL IN THE ART
`
`24. The ‘628 patent describes the field of the invention as follows: “The
`
`field of the invention is generally related to three-dimensional film post-production
`
`7
`
`
`
`
`
`processes. In particular, the invention relates to a system and process for
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`converting two-dimensional images into three-dimensional images.” ‘628 patent at
`
`Col. 1, ll. 15-18. Based on this description as well as the ‘628 patent’s description,
`
`it is my opinion that the relevant art for the ‘628 patent is two-dimensional to
`
`three-dimensional image conversion including use of information contained in
`
`images to generate depth maps for rendering stereoscopic images.
`
`25.
`
`I believe that a person of ordinary skill in the art in the field of the
`
`‘628 patent is a person with an undergraduate degree in electrical engineering or
`
`computer science (or an equivalent subject) and would have been someone with a
`
`good working knowledge of computer programming, data structures, and image
`
`processing. The person would have gained this knowledge either through an
`
`undergraduate education in computer science or comparable field, in combination
`
`with training or several years of practical working experience. This description is
`
`approximate, and a higher level of education or skill might make up for less
`
`experience, and vice-versa.
`
`26.
`
`I believe that I would qualify as at least a person of ordinary skill in
`
`the art at that time, and that I have a sufficient level of knowledge, experience and
`
`education to provide an opinion in the field of the ‘628 patent. Specifically, I work
`
`with many people including students, researchers that I supervise and colleagues in
`
`the industry who are persons of ordinary skill in the art, so I have an understanding
`
`8
`
`
`
`
`
`of what a person of ordinary skill in the art is. It is through this viewpoint that I
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`make my conclusions in this matter.
`
`27.
`
`I understand that a person of ordinary skill in the art is not a specific,
`
`real individual, but rather a hypothetical individual presumed to have knowledge of
`
`the relevant art at the time of the invention.
`
`28. My opinion of the level of ordinary skill in the art is based on my
`
`extensive personal experience working in the field of image processing; my
`
`knowledge of the level of education and experience of colleagues and others
`
`actively working in the field as of and for several years prior to the Earliest
`
`Potential Filing Date; the types of problems encountered in the art at the time the
`
`subject matter was developed, and prior art solutions to those problems including,
`
`in particular, problems arising from creating
`
`three-dimensional (3D) or
`
`stereoscopic images; and the level of sophistication of the technology at issue in
`
`this proceeding.
`
`V. OVERVIEW OF THE ‘628 PATENT
`
`A.
`
`Disclosure of the ‘628 Patent
`
`29. According to its specification, the ‘628 patent relates to a system and
`
`method for the conversion of two-dimensional (2D) images into images providing
`
`the perception of three-dimensional (3D) images as per the Abstract and Field of
`
`Invention at Col. 1, ll. 15-20.
`
`9
`
`
`
`
`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`30. The ‘628 patent aims to convert 2D images to 3D images using the
`
`image’s own intrinsic qualities or “characteristics” (e.g., how bright a pixel is),
`
`for “segmenting” images and “remapping” portions of the image. Segmenting is
`
`also known as “selecting”, which is the process of defining “masks” for
`
`recognizable objects in images. “Masks” define areas in an image in which to
`
`perform some function, such as adding color or depth to that portion of image
`
`within the area defined by the mask. The ‘628 patent seeks to minimize manual
`
`labor through use of the image’s intrinsic characteristics, such as how bright a
`
`pixel is, to define the area to operate on. The ‘628 patent seeks to eliminate the
`
`manual labor related to traditional “rotoscoping”, which is manual drawing of
`
`borders around objects at Col. 4, ll. 58-59. The ‘628 patent “remapping” process
`
`also uses the image’s own intrinsic qualities (characteristics) to shift pixels within
`
`masked objects, (also referred to as “image elements”) in the image to produce a
`
`stereo depth effect at Col. 5, ll. 2-3.
`
`31. Remapping in the ‘628 patent relates to the use of a depth selection
`
`mask to displace or shift pixels more, in an object such as a face, if the respective
`
`characteristic (for example brightness) of the pixel is higher, since brighter
`
`portions of the object may be closer than darker portions of the objects, e.g., a
`
`shiny nose. The ‘628 patent teaches that characteristics that may be used to remap
`
`pixels in a particular object in an image include hue (a color irrespective of
`
`10
`
`
`
`
`
`brightness), luminance (how bright), saturation (how pure or gray a color is), and
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`color (the combination of hue, luminance, and saturation) at Col. 5, ll. 64-67.
`
`32. As taught in the ‘628 patent, characteristics such as the variations of
`
`color in an object, such as skin tones, may be used to segment the objects and
`
`remap pixels to vary the depth within that object. The ‘628 patent teaches adding
`
`depth corresponding to natural and dimension contours that exist in the image at
`
`Col. 16, ll. 1-3, so that the object does not appear to be flat and has detail at Col.
`
`15, ll. 65-66.
`
`33. The occurrence of objects in converted 3D images that appear flat,
`
`with no variation in depth within each object is known as the “card board cut-out”
`
`effect, a problem solved as early as 1986 in USPN 4,925,294 (‘294) to Geshwind,
`
`discussed in further detail below.
`
`34. The ‘628 patent also uses the characteristics in the image to add depth
`
`to objects to minimize manual labor in two areas of conversion, segmentation and
`
`remapping at Col. 4, l. 67 through Col. 5, ll. 1-3:
`
`it avoids rotoscoping by using weighted
`image selection
` “First,
`(segmentation) based upon a variety of visual image properties. This
`requires little interaction from a human operator as compared to rotoscoping.
`Second it avoids any costly virtual reconstruction step by vector-based
`remapping of image elements”
`
`35.
`
`‘628 Fig. 3 shows the 2D to 3D conversion system.
`
`11
`
`
`
`
`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`
`
`36. Thus, the gist of the ‘628 patent is to use the actual values of an
`
`image’s characteristics for an image element, for example each pixel’s hue,
`
`luminance, saturation or color to select or segment objects, and to shift pixels or
`
`remap pixels more or less based on depth to avoid manual labor and to avoid a
`
`scene having only flat objects, i.e., to avoid the “card board cut-out” problem.
`
`VI. TECHNOLOGY BACKGROUND CIRCA 1986
`
`37. Pictures and videos that are captured with standard two-dimensional
`
`(2D) cameras result in 2D images, as is commonly known. There is an immense
`
`12
`
`
`
`
`
`amount of standard 2D pictures and motion picture material that exists. The
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`motivation to convert existing 2D material for 3D viewing using digital computers
`
`is taught in USPN 4,925,294 (‘294) to Geshwind, filed in 1986, at least at Col. 2,
`
`ll. 66-68 through Col. 3, ll. 1-6. The ‘628 patent does not cite the Geshwind ‘294
`
`patent, which was issued and published in 1990 and was available to the public for
`
`two decades before the ‘628 patent was filed.
`
`38. Geshwind ‘294, in 1986, identified and solved many of the problems
`
`related to conversion of 2D images to 3D images including filling holes that result
`
`when foreground objects are remapped or otherwise shifted, resulting in missing
`
`background data, as well as the “card board cut-out” problem that the ‘628 patent
`
`purports to solve to avoid flat objects.
`
`39. With respect to remapping based on the image’s intrinsic qualities,
`
`Geshwind ‘294 in 1986 claims varying depth within a particular image element, in
`
`claim 1, “d. specifying three-dimensional information for at least one of said image
`
`elements” and in claim 26, “wherein at least some of said three-dimensional
`
`information specified in step d is derived from the measurement of at least one
`
`aspect of an image element.” and in claim 28, “wherein said aspect pertains to
`
`illumination.” Illumination is synonymous with luminance. Thus, remapping
`
`based on the image’s intrinsic qualities was claimed by Geshwind ‘294, 23 years
`
`before the ‘628 patent was filed. Someone of ordinary skill in the art would
`
`13
`
`
`
`
`
`recognize that remapping was known at the time of filing of the ’628 patent based
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`at least on Geshwind ‘294.
`
`40. The overall 2D to 3D conversion process as taught by Geshwind ‘294
`
`is detailed below. Geshwind ‘294 utilizes a 2D picture that is taken of a real world
`
`3D scene, as exemplified in Fig. 1 of the ‘294 patent below. The 2D image depicts
`
`a square in the foreground, in front of a circle in the mid-ground, which in turn is
`
`in front of a background.
`
`41. The 2D to 3D conversion process in Geshwind ‘294 begins with
`
`human identification of the image elements within the 2D image, which are user
`
`identifiable objects such as a square and circle shown in Fig. 1, or for example a
`
`car, face, or other object. Thus, segmented objects are assigned depth. The
`
`computer uses the depth assignment for each object to shift pixels in the image
`
`corresponding to each object (remapping). Specifically, the computer shifts pixels
`
`more to the left and right for objects or portions thereof that are assigned depths in
`
`the foreground and shifts pixels relatively less to the left and right for mid-ground
`
`objects. See annotated Fig. 1 below for the relative positions of the circle and
`
`square in left/right images and the anaglyph image. As shown, the square in the
`
`foreground is shifted more left and right as per the larger vector associated with the
`
`shift with respect to the smaller mid-ground shift associated with the circle. The
`
`14
`
`
`
`
`
`computer generates a pair of left and right viewpoint images with different relative
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`positions for the image elements and/or an anaglyph image.
`
`
`42. Geshwind ‘294 also teaches remapping, e.g., varying depth within a
`
`given image element or object, e.g., based on underlying image characteristics to
`
`avoid 3D renderings that appear as “card board cut-out” characters, i.e., where each
`
`object is flat although potentially having a different depth relative to another
`
`object. Specifically, Geshwind ‘294 teaches varying depth in each image element
`
`(for example the circle or square as shown in Fig. 1 of ‘628) at least at Col. 4, ll.
`
`67-68 through Col. 5, ll. 1-15:
`
`Each image element may be given a uniform depth designation which
`
`15
`
`
`
`
`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`may cause the perception of 'card board cut-out' characters. Alternately,
`different portions of a single image element may be given different depth
`designations with the computer interpolating depth coordinates over the
`entire element. For example, an image element positioned diagonally in a
`frame may have its right edge designated to be closer than its left edge.
`Alternately, one feature of an image element, say a person's nose in a
`close-up, might be designated as being closer than the rest of the image
`element. In such manner, the computer would be instructed to interpolate
`and process depth information over the entire image element. Such
`processing of the image element, in accordance with varying depth
`information, may result in the stretching, skewing or other distortion of
`the image element.
`
`43. Hence, by 1986, Geshwind had invented a fully digital 2D to 3D
`
`conversion system to convert 2D images to stereoscopic images for 3D viewing.
`
`Geshwind solved the problem associated with holes that may occur when shifting
`
`pixels that expose missing background data, in several ways including distortion of
`
`the object to cover holes. Geshwind also solved the card board cut-out problem by
`
`varying depth within an object, including based on the characteristic luminance, to
`
`avoid flat objects having the same depth across a given image element as
`
`purportedly invented in the ‘628 patent. Geshwind ‘294 accomplished this by
`
`creating realistic 3D non-uniform depth on the image elements using the image
`
`characteristic of illumination (luminance) within an image element, to specify
`
`depth information within that image element, more than two decades before the
`
`16
`
`
`
`
`
`‘628 patent was filed. Thus the main purported point of novelty in ‘628 was well
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`known long before the ‘628 patent was filed.
`
`44. To understand how 3D images are viewed, once converted, the viewer
`
`may view the left and right eye images using a stereoscope, or for example
`
`anaglyph glasses or other types of viewing apparatuses. A stereoscope shows only
`
`the left image to the left eye and only the right image to the right eye, (in use at
`
`least since the 1830’s as refined by Wheatstone and Brewster as described in
`
`Wikipedia under stereoscope). Anaglyph glasses show a single encoded image
`
`that each eye interprets respectively. This type of 2D encoded image enables a user
`
`to perceive left and right images in each eye respectively. Polarized lenses and
`
`other forms of 3D viewing apparatus also exist and enable 2D projection
`
`equipment to display images intended for the left and right eyes separately.
`
`Anaglyph glasses are inexpensive to produce and enable 2D projection systems to
`
`display both left and right images overlaid, using different colors that the left and
`
`right lenses filter.
`
`VII. OVERVIEW OF THE PRIOR ART
`
`
`
`
`
`
`
`45. As described above, Geshwind ‘294 teaches “remapping” by varying
`
`17
`
`
`
`
`
`depth within a given image element based on luminance, for example, to avoid the
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`“card board cut-out” effect for given image elements, in 1986. Geshwind ‘294
`
`segments objects in images by accepting user input for outlines of the boundaries
`
`of image elements, for example to manually “rotoscope” the image to define areas
`
`to apply depth at Col. 4, ll. 23-26.
`
`46. References relied upon below, along with other attached Exhibits
`
`teach the use of other image characteristics such as color, hue, luminance,
`
`saturation, optical flow, etc., to: 1) segment or define regions to work on, e.g.,
`
`create masks for, and 2) remap pixels to vary depth within image elements based
`
`on the characteristics in an image element or object within an image.
`
`47.
`
`In summary, the concept of segmenting, e.g., selecting an area to work
`
`on based on the underlying characteristics of that area, as well as the concept of
`
`remapping, or altering depth based on the underlying characteristics of that area,
`
`were well known by the Earliest Effective Filing date of the ‘628 patent and taught
`
`in many references. 1
`
`
`
`
`1 A person of ordinary skill in the art would also recognize that U.S. Patent
`7,558,420 to Era, (Era ‘420), with priority date 8/20/2002 teaches segmenting
`images based on pixel brightness, saturation or hue in order to select a portion of
`the image at Col. 24, ll. 6-15. Era ’420 also uses grayscale values as image
`characteristics to remap the image element to add depth at Col. 8, ll. 27-30, 7 years
`
`18
`
`
`
`
`
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`A.
`
`U.S. Patent No. 8,488,868
`
`48. Tam ‘868 was filed on 4/2/2008 and claims priority to provisional
`
`patent application 60/907,475 filed 4/3/2007. Tam ‘868 lists Wa James Tam and
`
`Carolos Vazquez as the inventors, Ex. 1003. Tam ‘868 is prior art to the ‘628
`
`patent at least under 35 U.S.C. §102(e). Tam ‘868 was not considered by the
`
`examiner during the prosecution of the ‘628 patent.
`
`49. Tam ‘868 relates to a method and apparatus for converting 2D images
`
`to 3D images, and segmenting, e.g., selecting objects based on characteristics of
`
`the image, and remapping, e.g., varying depth values of pixels of the image based
`
`on characteristics of the image, namely the color component of respective pixels,
`
`
`
`before the ‘628 patent. Ex. 1008.
`A person of ordinary skill in the art would also recognize that Baraldi 1989
`teaches remapping of ‘628 through disclosure of altering depth based on an image
`characteristic known as “Optical Flow”. Ex. 1012. The ‘628 patent teaches that
`optical flow is a type of image feature that may be utilized in the conversion
`process at least at Col. 25, ll. 8-10 and ll. 28-30.
`A person of ordinary skill in the art would also recognize that Japanese
`Patent 2002123842 having a priority date of 13 October 2000 also teaches
`remapping of ‘628 by remapping or altering depth in an image element based on
`image features that include saturation, brightness and hue within that image
`element. See machine translation Ex. 1013.
`
`
`19
`
`
`
`
`
`to create stereoscopic still and video images at Col. 3, ll. 63-66 and Abstract ll. 3-8.
`
`Declaration for Inter Partes Review of USPN 8,922,628
`
`50. Tam ‘868 aims to minimize manual labor just as the ‘628 patent does.
`
`Specifically, Tam ‘868 teaches that known approaches are labor intensive at Col. 2,
`
`ll. 43-45, “This type of approach requires trained technicians, and the task can be
`
`quite laborious and time-consuming for a full-length movie”. In addition, Tam
`
`‘868 teaches that “Another object of the present invention is to provide relatively
`
`simple and computationally inexpensive method and apparatus for rendering
`
`stereoscopic and multi-view video and still images from 2D video and still images
`
`utilizing color information contained in said 2D images.” at Col. 3, ll. 37-41.
`
`51. Specifically, Tam ‘868 discloses segmenting an image based on the
`
`image’s characteristics, specifically color, to create a depth selection mask for each
`
`object to later add depth to. Segmenting in Tam
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ ChargeStill Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site