`____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________
`
`APPLE INC.
`Petitioner
`
`v.
`
`GESTURE TECHNOLOGY PARTNERS LLC
`Patent Owner
`_________________
`
`Inter Partes Review Case No. IPR2021-00922
`U.S. Patent No. 8,553,079
`
`DECLARATION OF DR. BENJAMIN B. BEDERSON
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 1
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`
`I, Benjamin B. Bederson, hereby declare the following:
`
`I.
`
`BACKGROUND AND QUALIFICATIONS
`
`1. My name is Benjamin B. Bederson, Ph.D and I am over 21 years of age
`
`and otherwise competent to make this Declaration. I make this Declaration based on
`
`facts and matters within my own knowledge and on information provided to me by
`
`others.
`
`2.
`
`I have been retained by counsel for Petitioner as a technical expert in the
`
`above-captioned case. Specifically, I have been asked to render certain opinions in
`
`regard to the IPR petition with respect to U.S. Patent No. 8,553,079 (the “’079
`
`Patent”). I understand that the Challenged Claims are claims 1-30. My opinions are
`
`limited to those Challenged Claims.
`
`3. My compensation in this matter is not based on the substance of my
`
`opinions or the outcome of this matter. I have no financial interest in Petitioner. I am
`
`being compensated at an hourly rate of $600 for my analysis and testimony in this
`
`case.
`
`4.
`
`In writing this declaration, I have considered my own knowledge and
`
`experience, including my work experience in the field of electrical and computer
`
`engineering; my experience in teaching this area; and my experience working with
`
`others involved in this field, including in both the design and analysis of multimedia-
`
`
`
`2
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 2
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`focused communication systems and subsystems. In reaching my opinions in this
`
`matter, I have also reviewed the following references and materials:
`
`• The ’079 Patent (Ex. 1001)
`• The ’079 Patent File History (Ex. 1002)
`• U.S. Patent No. 6,144,366 (“Numazaki”) (Ex. 1004)
`• U.S. Patent No. 5,900,863 (“Numazaki ’863”) (Ex. 1005)
`• U.S. Patent No. 6,064,354 (“DeLuca”) (Ex. 1006)
`• U.S. Patent No. 6,008,018 (“DeLeeuw”) (Ex. 1007)
`• U.S. Patent No. 6,191,773 (“Maruno”) (Ex. 1008)
`• “CCD and CMOS Imaging Array Technologies,” Stuart Taylor, Xerox
`Research Centre Europe, 1998 (Ex. 1011)
`• Any additional background materials cited below
`
`A. Educational Background
`5.
`I received a Bachelor of Science degree in Computer Science with a
`
`
`
`minor in Electrical Engineering from Rensselaer Polytechnic Institute (“RPI”) in
`
`1986. I received a Master of Science degree and a Ph.D. in Computer Science from
`
`New York University (“NYU”) in 1989 and 1992, respectively.
`
`B.
`6.
`
`Professional Experience
`Since 1998, I have been a Professor of Computer Science at the
`
`University of Maryland (“UMD”), where I have joint appointments at the Institute
`
`for Advanced Computer Studies and the College of Information Studies (Maryland’s
`
`“iSchool”), and am currently Professor Emeritus. I was also Associate Provost of
`
`Learning Initiatives and Executive Director of the Teaching and Learning
`
`Transformation Center from 2014 to 2018. I am a member and previous director of
`
`the Human-Computer Interaction Lab (“HCIL”), the oldest and one of the best
`
`
`
`3
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 3
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`known Human-Computer Interaction research groups in the country. I was also co-
`
`founder and Chief Scientist of Zumobi, Inc. from 2006 to 2014, a Seattle-based
`
`startup that is a publisher of content applications and advertising platforms for
`
`smartphones. I am also co-founder and co-director of the International Children’s
`
`Digital Library (“ICDL”), a web site launched in 2002 that provides the world’s
`
`largest collection of freely available online children’s books from around the world
`
`with an interface aimed to make it easy for children and adults to search and read
`
`children’s books online. I am also co-founder and prior Chief Technology Officer of
`
`Hazel Analytics, a data analytics company whose product sends alerts in warranted
`
`circumstances. In addition, I have for more than 25 years consulted for numerous
`
`companies in the area of user interfaces, including Logitech, Microsoft, the Palo Alto
`
`Research Center, Sony, Lockheed Martin, Hillcrest Labs, and NASA Goddard Space
`
`Flight Center.
`
`7.
`
`For more than 30 years, I have studied, designed, and worked in the
`
`field of computer science and human-computer interaction. My experience includes
`
`30 years of teaching and research, with research interests in human-computer
`
`interaction and the software and technology underlying today’s interactive
`
`computing systems. This includes the design and implementation of image sensing
`
`and image processing systems as well as software applications on mobile devices,
`
`including smart phones and PDAs, such as my work on DateLens, LaunchTile, and
`
`
`
`4
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 4
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`StoryKit described below. My consulting included helping companies apply my
`
`work on “zoomable user interfaces” to their consumer-facing audio/video access
`
`software.
`
`8.
`
`In 1992, I completed my Ph.D. dissertation at New York University
`
`titled “A Miniature Space-Variant Active Vision System: Cortex-I” (Ex. 1015)1 in
`
`which I worked with both CMOS and CCD image sensors and wrote image
`
`processing software using those sensors. As depicted in the VLSI circuit layout
`
`image below, I designed a custom CMOS image sensor with a radial pixel layout.
`
`Figure 2.1 from my dissertation show the result of images taken with a camera I built
`
`using that image sensor. I then built a custom CCD-based camera by manufacturing
`
`a lens that I attached directly to a commercially available CCD sensor that is shown
`
`in Figure 2.2.
`
`
`1 Ex. 1015 is a pre-publication version of my thesis, which does not include the
`final chapter.
`
`
`
`5
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 5
`
`
`
`IPR2021-00922
`IPR2021-00922
`U.S. Patent No. 8,553,079
`
`US. Patent No. 8,553,079
`
`
`
`
`
`6
`
`IPR2022-00090 - LGE
`
`Ex. 1010 - Page 6
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 6
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`
`
`
`9.
`
`In 1995, I built an “audio augmented reality” system2 that identified
`
`which piece of art a person was standing in front of. This worked by installing
`
`infrared transmitters in the ceiling above each piece of art which was identified by
`
`an infrared receiver controlled by a microcontroller. As depicted in Figure 1 below,
`
`
`2 Benjamin B. Bederson. 1995. Audio augmented reality: a prototype automated tour
`guide. In Conference Companion on Human Factors in Computing Systems (CHI
`'95), I. Katz, R. Mack, and L. Marks (Eds.). ACM, New York, NY, USA, 210-211.
`DOI=http://dx.doi.org/10.1145/223355.223526 (Ex. 1016).
`
`
`
`7
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 7
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`a person wearing the receiver walked around, the microcontroller they were carrying
`
`would identify the code from the transmitter they were standing under.
`
`
`
`10. At UMD, my research is in the area of Human-Computer Interaction
`
`(“HCI”), a field that relates to the development and understanding of computing
`
`systems to serve users’ needs. Researchers in this field are focused on making
`
`universally usable, useful, efficient and appealing systems to support people in their
`
`wide range of activities. My approach is to balance the development of innovative
`
`technology that serves people’s practical needs. Example systems following this
`
`approach that I have built include PhotoMesa (software for end users to browse
`
`personal photos), DateLens (2002 software for end users to use their mobile devices
`
`to efficiently access their calendar information), LaunchTile (2005 “home screen”
`
`software for mobile devices to allow users to navigate apps in a zoomable
`
`
`
`8
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 8
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`environment), SpaceTree (2001 software for end users to efficiently browse very
`
`large hierarchies), ICDL (as described above), and StoryKit (a 2009 iPhone app for
`
`children to create stories).
`
`11. LaunchTile led to my creation of Zumobi in 2006, where I was
`
`responsible for investigating new software platforms and developing new user
`
`interface designs that provide efficient and engaging interfaces to permit end users
`
`to access a wide range of content on mobile platforms (including the iPhone and
`
`Android-based devices). For example, I designed and implemented software called
`
`“Ziibii,” a “river” of news for iPhone, software called “ZoomCanvas,” a zoomable
`
`user interface for several iPhone apps, and iPhone apps including “Inside Xbox” for
`
`Microsoft and Snow Report for REI. At the International Children’s Digital Library
`
`(ICDL), I have since 2002 been the technical director responsible for the design and
`
`implementation of
`
`the web site, www.childrenslibrary.org
`
`(originally at
`
`www.icdlbooks.org). In particular, I have been closely involved in designing the
`
`user interfaces as well as the software architecture for the web site since its inception
`
`in 2002.
`
`12. Beginning in the mid-1990s, I have been responsible for the design and
`
`implementation of numerous other web sites in addition to the ICDL. For example,
`
`I designed and built my own professional web site when I was an Assistant Professor
`
`of Computer Science at the University of New Mexico in 1995 and have continued
`
`
`
`9
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 9
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`to design, write the code for, and update both that site (which I moved to the
`
`University of Maryland in 1998, currently at http://www.cs.umd.edu/~bederson/) as
`
`well
`
`as
`
`numerous
`
`project
`
`web
`
`sites,
`
`such
`
`as
`
`Pad++,
`
`http://www.cs.umd.edu/hcil/pad++/. I received the Janet Fabri Memorial Award for
`
`Outstanding Doctoral Dissertation for my Ph.D. work in robotics and computer
`
`vision. I have combined my hardware and software skills throughout my career in
`
`Human-Computer Interaction research, building various interactive electrical and
`
`mechanical systems that couple with software to provide an innovative user
`
`experience.
`
`13. My work has been published extensively in more than 160 technical
`
`publications, and I have given about 100 invited talks, including 9 keynote lectures.
`
`I have won a number of awards including the Brian Shackel Award for “outstanding
`
`contribution with international impact in the field of HCI” in 2007, and the Social
`
`Impact Award in 2010 from Association for Computing Machinery’s (“ACM”)
`
`Special Interest Group on Computer Human Interaction (“SIGCHI”). ACM is the
`
`primary international professional community of computer scientists, and SIGCHI
`
`is the primary international professional HCI community. I have been honored by
`
`both professional organizations. I am an “ACM Distinguished Scientist,” which
`
`“recognizes those ACM members with at least 15 years of professional experience
`
`and 5 years of continuous Professional Membership who have achieved significant
`
`
`
`10
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 10
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`accomplishments or have made a significant impact on the computing field.” I am a
`
`member of the “CHI Academy,” which is described as follows: “The CHI Academy
`
`is an honorary group of individuals who have made substantial contributions to the
`
`field of human-computer interaction. These are the principal leaders of the field,
`
`whose efforts have shaped the disciplines and/or industry, and led the research
`
`and/or innovation in human-computer interaction.” The criteria for election to the
`
`CHI Academy are: cumulative contributions to the field; impact on the field through
`
`development of new research directions and/or innovations; and (3) influence on the
`
`work of others.
`
`14.
`
`I have appeared on radio shows numerous times to discuss issues
`
`relating to user interface design and people’s use and frustration with common
`
`technologies, web sites, and mobile devices. My work has been discussed and I have
`
`been quoted by mainstream media around the world over 120 times, including by
`
`the New York Times, the Wall Street Journal, the Washington Post, Newsweek, the
`
`Seattle Post-Intelligencer, the Independent, Le Monde, NPR’s All Things
`
`Considered, New Scientist Magazine, and MIT’s Technology Review.
`
`15.
`
`I have designed, programmed, and publicly deployed dozens of user-
`
`facing software products that have cumulatively been used by millions of users. My
`
`work is cited in patents by several major companies, including Amazon, Apple,
`
`Facebook, Google, and Microsoft. I am the co-inventor of 12 U.S. patents and 18
`
`
`
`11
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 11
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`U.S. patent applications. The patents are generally directed
`to user
`
`interfaces/experience with some directed to mobile devices, including U.S. Patent
`
`No. 9,778,810 (issued 2017), entitled “Techniques to modify content and view
`
`content on mobile devices.”
`
`16. My curriculum vitae, which includes a more detailed summary of my
`
`background, experience, and publications, is attached as Appendix A.
`
`II. LEGAL FRAMEWORK
`
`A. Obviousness
`17.
`I am a technical expert and do not offer any legal opinions. However,
`
`counsel has informed me as to certain legal principles regarding patentability and
`
`related matters under United States patent law, which I have applied in performing
`
`my analysis and arriving at my technical opinions in this matter.
`
`18.
`
`I have been informed that the Patent Trial and Appeal Board (“PTAB”)
`
`now applies the claim construction standard applied by Article III courts (i.e., the
`
`Phillips standard) regardless of whether a patent has expired. I have been informed
`
`that under the Phillips standard, claim terms are to be given the meaning they would
`
`have to a person having ordinary skill in the art at the time of the invention, taking
`
`into consideration the patent, its file history, and, secondarily, any applicable
`
`extrinsic evidence (e.g., dictionary definitions). In my analyses below, I have applied
`
`the plain and ordinary meaning pursuant to the Phillips standard.
`
`
`
`12
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 12
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`I have also been informed that a person cannot obtain a patent on an
`
`19.
`
`invention if the differences between the invention and the prior art are such that the
`
`subject matter as a whole would have been obvious at the time the invention was
`
`made to a person having ordinary skill in the art. I have been informed that a
`
`conclusion of obviousness may be founded upon more than a single item of prior art.
`
`I have been further informed that obviousness is determined by evaluating the
`
`following factors: (1) the scope and content of the prior art, (2) the differences
`
`between the prior art and the claim at issue, (3) the level of ordinary skill in the
`
`pertinent art, and (4) secondary considerations of non-obviousness. In addition, the
`
`obviousness inquiry should not be done in hindsight. Instead, the obviousness
`
`inquiry should be done through the eyes of a person having ordinary skill in the
`
`relevant art at the time the patent was filed.
`
`20.
`
`In considering whether certain prior art renders a particular patent claim
`
`obvious, counsel has informed me that I can consider the scope and content of the
`
`prior art, including the fact that one of skill in the art would regularly look to the
`
`disclosures in patents, trade publications, journal articles, industry standards,
`
`product literature and documentation, texts describing competitive technologies,
`
`requests for comment published by standard setting organizations, and materials
`
`from industry conferences, as examples. I have been informed that for a prior art
`
`reference to be proper for use in an obviousness analysis, the reference must be
`
`
`
`13
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 13
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`“analogous art” to the claimed invention. I have been informed that a reference is
`
`analogous art to the claimed invention if: (1) the reference is from the same field of
`
`endeavor as the claimed invention (even if it addresses a different problem); or (2)
`
`the reference is reasonably pertinent to the problem faced by the inventor (even if it
`
`is not in the same field of endeavor as the claimed invention). In order for a reference
`
`to be “reasonably pertinent” to the problem, it must logically have commended itself
`
`to an inventor’s attention in considering his problem. In determining whether a
`
`reference is reasonably pertinent, one should consider the problem faced by the
`
`inventor, as reflected either explicitly or implicitly, in the specification. I believe that
`
`all of the references that my opinions in this IPR are based upon are well within the
`
`range of references a person having ordinary skill in the art would consult to address
`
`the type of problems described in the Challenged Claims.
`
`21.
`
`I have been informed that, in order to establish that a claimed invention
`
`was obvious based on a combination of prior art elements, a clear articulation of the
`
`reason(s) why a claimed invention would have been obvious must be provided.
`
`Specifically, I am informed that, under the U.S. Supreme Court’s KSR decision, a
`
`combination of multiple items of prior art renders a patent claim obvious when there
`
`was an apparent reason for one of ordinary skill in the art, at the time of the invention,
`
`to combine the prior art, which can include, but is not limited to, any of the following
`
`rationales: (A) combining prior art methods according to known methods to yield
`
`
`
`14
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 14
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`predictable results; (B) substituting one known element for another to obtain
`
`predictable results; (C) using a known technique to improve a similar device in the
`
`same way; (D) applying a known technique to a known device ready for
`
`improvement to yield predictable results; (E) trying a finite number of identified,
`
`predictable potential solutions, with a reasonable expectation of success; (F)
`
`identifying that known work in one field of endeavor may prompt variations of it for
`
`use in either the same field or a different one based on design incentives or other
`
`market forces if the variations are predictable to one of ordinary skill in the art; or
`
`(G) identifying an explicit teaching, suggestion, or motivation in the prior art that
`
`would have led one of ordinary skill to modify the prior art reference or to combine
`
`the prior art references to arrive at the claimed invention.
`
`22.
`
`I am informed that the existence of an explicit teaching, suggestion, or
`
`motivation to combine known elements of the prior art is a sufficient, but not a
`
`necessary, condition to a finding of obviousness. This so-called “teaching
`
`suggestion-motivation” test is not the exclusive test and is not to be applied rigidly
`
`in an obviousness analysis. In determining whether the subject matter of a patent
`
`claim is obvious, neither the particular motivation nor the avowed purpose of the
`
`patentee controls. Instead, the important consideration is the objective reach of the
`
`claim. In other words, if the claim extends to what is obvious, then the claim is
`
`invalid. I am further informed that the obviousness analysis often necessitates
`
`
`
`15
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 15
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`consideration of the interrelated teachings of multiple patents, the effects of demands
`
`known to the technological community or present in the marketplace, and the
`
`background knowledge possessed by a person having ordinary skill in the art. All of
`
`these issues may be considered to determine whether there was an apparent reason
`
`to combine the known elements in the fashion claimed by the patent.
`
`23.
`
`I also am informed that in conducting an obviousness analysis, a precise
`
`teaching directed to the specific subject matter of the challenged claim need not be
`
`sought out because it is appropriate to take account of the inferences and creative
`
`steps that a person of ordinary skill in the art would employ. The prior art considered
`
`can be directed to any need or problem known in the field of endeavor at the time of
`
`invention and can provide a reason for combining the elements of the prior art in the
`
`manner claimed. In other words, the prior art need not be directed towards solving
`
`the same specific problem as the problem addressed by the patent. Further, the
`
`individual prior art references themselves need not all be directed towards solving
`
`the same problem. I am informed that, under the KSR obviousness standard, common
`
`sense is important and should be considered. Common sense teaches that familiar
`
`items may have obvious uses beyond their primary purposes.
`
`24.
`
`I also am informed that the fact that a particular combination of prior art
`
`elements was “obvious to try” may indicate that the combination was obvious even
`
`if no one attempted the combination. If the combination was obvious to try
`
`
`
`16
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 16
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`(regardless of whether it was actually tried) or leads to anticipated success, then it is
`
`likely the result of ordinary skill and common sense rather than innovation. I am
`
`further informed that in many fields it may be that there is little discussion of obvious
`
`techniques or combinations, and it often may be the case that market demand, rather
`
`than scientific literature or knowledge, will drive the design of an invention. I am
`
`informed that an invention that is a combination of prior art must do more than yield
`
`predictable results to be non-obvious.
`
`25.
`
`I am informed that for a patent claim to be obvious, the claim must be
`
`obvious to a person of ordinary skill in the art at the time of the invention. I am
`
`informed that the factors to consider in determining the level of ordinary skill in the
`
`art include (1) the educational level and experience of people working in the field at
`
`the time the invention was made, (2) the types of problems faced in the art and the
`
`solutions found to those problems, and (3) the sophistication of the technology in the
`
`field.
`
`26.
`
`I am informed that it is improper to combine references where the
`
`references teach away from their combination. I am informed that a reference may
`
`be said to teach away when a person of ordinary skill in the relevant art, upon reading
`
`the reference, would be discouraged from following the path set out in the reference,
`
`or would be led in a direction divergent from the path that was taken by the patent
`
`applicant. In general, a reference will teach away if it suggests that the line of
`
`
`
`17
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 17
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`development flowing from the reference’s disclosure is unlikely to be productive of
`
`the result sought by the patentee. I am informed that a reference teaches away, for
`
`example, if (1) the combination would produce a seemingly inoperative device, or
`
`(2) the references leave the impression that the product would not have the property
`
`sought by the patentee. I also am informed, however, that a reference does not teach
`
`away if it merely expresses a general preference for an alternative invention but does
`
`not criticize, discredit, or otherwise discourage investigation into the invention
`
`claimed.
`
`27.
`
`I am informed that even if a prima facie case of obviousness is
`
`established, the final determination of obviousness must also consider “secondary
`
`considerations” if presented. In most instances, the patentee raises these secondary
`
`considerations of non-obviousness. In that context, the patentee argues an invention
`
`would not have been obvious in view of these considerations, which include: (a)
`
`commercial success of a product due to the merits of the claimed invention; (b) a
`
`long-felt, but unsatisfied need for the invention; (c) failure of others to find the
`
`solution provided by the claimed invention; (d) deliberate copying of the invention
`
`by others; (e) unexpected results achieved by the invention; (f) praise of the
`
`invention by others skilled in the art; (g) lack of independent simultaneous invention
`
`within a comparatively short space of time; (h) teaching away from the invention in
`
`the prior art.
`
`
`
`18
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 18
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`I am further informed that secondary considerations evidence is only
`
`28.
`
`relevant if the offering party establishes a connection, or nexus, between the
`
`evidence and the claimed invention. The nexus cannot be based on prior art features.
`
`The establishment of a nexus is a question of fact. While I understand that the Patent
`
`Owner here has not offered any secondary considerations at this time, I will
`
`supplement my opinions in the event that the Patent Owner raises secondary
`
`considerations during the course of this proceeding.
`
`III. OPINION
`
`A. Level of a Person of Ordinary Skill in the Art
`
`29.
`
`I was asked to provide my opinion as to the level of skill of a person
`
`having ordinary skill in the art (“PHOSITA”) of the ’079 Patent at the time of the
`
`claimed invention, which counsel has told me to assume is November 9, 1998. In
`
`determining the characteristics of a hypothetical person of ordinary skill in the art of
`
`the ’079 Patent, I considered several factors, including the type of problems
`
`encountered in the art, the solutions to those problems, the rapidity with which
`
`innovations are made in the field, the sophistication of the technology, and the
`
`education level of active workers in the field. I also placed myself back in the time
`
`frame of the claimed invention and considered the colleagues with whom I had
`
`worked at that time.
`
`
`
`19
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 19
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`In my opinion, a PHOSITA at the time of the claimed invention of the
`
`30.
`
`’079 Patent would have had at least a bachelor’s degree in electrical engineering or
`
`equivalent with at least one year of experience in the field of human computer
`
`interaction. Additional education or experience might substitute for the above
`
`requirements. Such a PHOSITA would have been capable of understanding the ’079
`
`patent and the prior art references discussed herein.
`
`31. Based on my education, training, and professional experience in the field
`
`of the claimed invention, I am familiar with the level and abilities of a person of
`
`ordinary skill in the art at the time of the claimed invention. Additionally, I met at
`
`least these minimum qualifications to be a person having ordinary skill in the art as
`
`of the time of the claimed invention of the ’079 Patent.
`
`B. Description of the Alleged Invention of the ’079 Patent
`
`32. The ’079 Patent generally relates to allowing users to interact with a
`
`computer through gestures (e.g., specific hand movements) that are detected by one
`
`or more cameras and converted into commands by the computer. Fig. 2 depicts one
`
`arrangement described by the ’079 Patent in which a user’s hand is illuminated by
`
`light sources, its positioning and movements are detected by cameras, and the user
`
`is able to command action in a computer application through gestures:
`
`
`
`20
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 20
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`
`
`
`’079 Patent at Fig. 2. In this figure, light source 122 “can be used to illuminate the
`
`finger that typically looks bright under such illumination.” Id. at 3:1-3. A number of
`
`optional camera locations are depicted for capturing positioning and movement of
`
`the user’s finger, including 100, 101, 105, 106, 108, and 109. Id. at 2:39-53. The
`
`’079 Patent refers to the space within which the user performs gestures as “work
`
`volume 170.” Id. at 3:4-8.
`
`33. As depicted in Fig. 2, the described invention may “determine the
`
`pointing direction vector 160 of the user’s finger (for example pointing at an object
`
`
`
`21
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 21
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`displayed on screen 107), or the position and orientation of an object held by the
`
`user.” Id. at 2:54-58. Other gestures such as pinch or grip motions can also be used
`
`to interact with a computer application. Id. at 2:58-64. Finally, although the system
`
`can simply detect the user’s finger alone, objects such as ring 208 may also be used
`
`to assist in detecting the position and movement of a user’s finger. Id. at 2:65-3:3.
`
`C. Opinions Related to Numazaki
`
`i. Overview of Numazaki (Ex. 1004)
`
`34.
`
` Numazaki describes numerous hardware
`
`configurations
`
`and
`
`applications directed to users interacting with a computer via gestures. In the
`
`subsequent paragraphs, I address a few described embodiments that are particularly
`
`relevant to my opinions below.
`
`Numazaki’s First Embodiment—Two-Camera Gesture Detection
`
`35.
`
`In its first embodiment, Numazaki describes a two-camera structure for
`
`detecting a user’s gestures. Namely, Numazaki purports to have improved upon prior
`
`methods by using a controlled light source to illuminate the target object (e.g., the
`
`user’s hand), a first camera unit (referred to by Numazaki as a “photo-detection
`
`unit”), and a second camera unit. Numazaki at 11:9-23. This arrangement is
`
`illustrated in Fig. 2 below:
`
`
`
`22
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 22
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`
`
`Id. at Fig. 2. A timing control unit is used to turn lighting unit 101 on (i.e.,
`
`illuminating the target object) when the first camera unit is active and off when the
`
`second camera unit is active. Id. at 11:20-32. The result of this light control is the
`
`first camera unit captures an image of the target object illuminated by both natural
`
`light and the lighting unit 101 and the second camera unit captures an image of the
`
`target object illuminated by only natural light. Id. at 11:33-39. The difference
`
`between the two images—obtained by difference calculation unit 111—represents
`
`the “reflected light from the object resulting from the light emitted by the lighting
`
`unit 101.” Id. at 11:43-51. This information is then used by feature data generation
`
`unit 103 to determine gestures, pointing, etc. of the target object that may be
`
`converted into commands executed by a computer. Id. at 10:57-66.
`
`36. The “photo-detection units” depicted in Fig. 2 and described in
`
`Numazaki are “camera units.” Each performs the basic function of a “camera,” which
`
`
`
`23
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 23
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`is to capture image information based on light captured by the sensors’ pixels.
`
`Further, Numazaki notes that these units can be “CMOS sensors” or “CCD image
`
`sensors.” Id. at 15:24-16:19. These were two widely recognized technologies used
`
`to implement imaging sensors for cameras. See, e.g., “CCD and CMOS Imaging
`
`Array Technologies,” Stuart Taylor, Xerox Research Centre Europe, 1998 (Ex.
`
`1011) (discussing the history of CCD and CMOS imaging technologies, concluding
`
`that CMOS is likely to overtake CCD devices for future imaging).
`
`Numazaki’s Second Embodiment—Hand-Based Gesture Detection
`
`37. Numazaki’s second embodiment focuses on a user’s hand as the target
`
`object such that a user can control the computing device using gestures. Id. at 17:18-
`
`22 (“When the hand is used as the target object, it is possible to capture the
`
`information on a position and a shape of the hand without a contact, so that it is
`
`possible to utilize the present invention as a means for inputting information from a
`
`human being to a machine.”). In one example, Numazaki describes a process by
`
`which a user’s finger tip position is identified and tracked such that “the cursor on
`
`the screen can be controlled” by the user’s gestures. Id. at 26:8-67. Numazaki
`
`describes many implementation details in this section, some of which I discuss
`
`below where relevant to my specific opinions.
`
`
`
`
`
`
`
`
`
`24
`
`IPR2022-00090 - LGE
`Ex. 1010 - Page 24
`
`
`
`IPR2021-00922
`U.S. Patent No. 8,553,079
`Numazaki’s Third Embodiment—Converting Gestures into Commands
`
`38. Numazaki’s third