`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`____________________________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`____________________________________________
`
`
`META PLATFORMS, INC.
`Petitioner
`
`v.
`
`THALES VISIONIX, INC.
`Patent Owner
`
`U.S. Patent No. 6,922,632
`U.S. Patent No. 7,725,253
`
`Cases IPR2022-01304, IPR2022-01305, IPR2022-01308
`
`
`SUPPLEMENTAL DECLARATION OF DR. ULRICH NEUMANN
`IN SUPPORT OF PETITIONER’S REPLY TO PATENT OWNER’S
`RESPONSE TO PETITION FOR INTER PARTES REVIEW OF U.S.
`PATENT NOS. 6,922,632 AND 7,725,253
`
`
`
`
`
`
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`TABLE OF CONTENTS
`
`Page
`
`I.
`
`Introduction .................................................................................................... 1
`
`II.
`
`Background .................................................................................................... 2
`
`A. Qualifications ........................................................................................ 2
`
`B.
`
`C.
`
`Prior Testimony ..................................................................................... 3
`
`Level of Ordinary Skill In The Art ....................................................... 4
`
`III. Supplemental Opinions ................................................................................. 4
`
`A. Welch-2001 In Combination With Welch-1997 “Enumerat[es],”
`Or Obviously Would Have “Enumerat[ed],” The Sensing
`Elements Available To It ....................................................................... 4
`
`B.
`
`C.
`
`Horton Performs “Computations” ......................................................... 8
`
`Horton “Enumerat[es],” Or Obviously Would Have
`“Enumerat[ed],” The Sensing Elements Available To It ....................12
`
`IV. Conclusion ....................................................................................................14
`
`
`
`
`
`i
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`I, Dr. Ulrich Neumann, declare as follows:
`
`I.
`
`INTRODUCTION
`
`1. My name is Ulrich Neumann. I am a Professor of Computer Science at
`
`the University of Southern California in Los Angeles, California. I was previously
`
`retained on behalf of Petitioner Meta Platforms, Inc. (“Petitioner”) to provide my
`
`opinions regarding whether claims 1-9, 11-36, 44-45, 47-55, 57-61, and 66-69 of
`
`U.S. Patent No. 6,922,632 (“the ’632 patent”) and claims 1-9 of U.S. Patent No.
`
`7,725,253 (“the ’253 patent”) would have been obvious to one of ordinary skill in
`
`the art as of August 9, 2002. I previously provided declarations containing my
`
`opinions, which I understand from counsel were filed as EX1005 (the “Original
`
`Declarations”) in IPR2022-01304, IPR2022-01305, and IPR2022-01308 related to
`
`the ’632 patent and the ’253 patent.
`
`2.
`
`I have been asked by Petitioner to provide my opinion about certain
`
`issues that I understand from counsel relate to arguments presented in Patent
`
`Owner’s Responses (IPR2022-01304, Paper 30; IPR2022-01305, Paper 30;
`
`IPR2022-01308, Paper 29) and the accompanying opinions in the Declaration of
`
`Yohan Baillot (filed as EX2007 in each of those proceedings). My opinion regarding
`
`those certain issues is reflected in this supplemental declaration. I was not asked by
`
`Petitioner to provide my opinion about every argument, contention, opinion, or piece
`
`of evidence in the aforementioned Patent Owner’s Responses or Declaration of
`
`1
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`Yohan Baillot; rather, I was only asked by Petitioner to provide my opinion
`
`regarding certain limited issues, as explained in more detail below. Accordingly, I
`
`have not formed an opinion about any of the arguments, contentions, opinions, or
`
`pieces of evidence in the aforementioned Patent Owner’s Responses or Declaration
`
`of Yohan Baillot except as expressly addressed below (and except to the extent a
`
`relevant opinion may be found in my Original Declarations, which I offered before
`
`Patent Owner’s Responses or the Declaration of Yohan Baillot were filed).
`
`3.
`
`In forming the opinions expressed in this declaration, I have relied on
`
`my academic and professional experience, as well as all of the evidence expressly
`
`cited herein. I considered the viewpoint of a person having ordinary skill in the art
`
`as of the time of alleged invention of the ’632 and ’253 patents (August 9, 2002).
`
`4.
`
`For time spent in connection with this matter, I will be compensated at
`
`my standard billing rate of $775 per hour. I am being separately reimbursed for any
`
`out-of-pocket expenses. My compensation does not depend on the content of my
`
`opinions or the outcome of these proceedings.
`
`II. BACKGROUND
`
`A. QUALIFICATIONS
`
`5.
`
`I previously provided my relevant qualifications for these proceedings
`
`in Section II.A-C of my Original Declarations. I incorporate herein those sections
`
`of my Original Declarations.
`
`
`
`2
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`I previously provided my curriculum vitae (“CV”) for these
`
`6.
`
`proceedings, which I understand was filed as EX1006 in these proceedings. I
`
`incorporate herein my previously filed CV.
`
`B.
`
`7.
`
`PRIOR TESTIMONY
`
`The cases in which I have testified as an expert at trial or by deposition
`
`within the preceding five years are as follows:
`
` Science Applications International Corp. v. United States (for defendant);
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01294 (for
`petitioner);
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01298 (for
`petitioner);
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01301 (for
`petitioner);
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01302 (for
`petitioner);
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01303 (for
`petitioner);
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01304 (for
`petitioner);
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01305 (for
`petitioner); and
`
` Meta Platforms, Inc. v. Thales Visionix, Inc., Case IPR2022-01308 (for
`petitioner).
`
`
`
`3
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`C. LEVEL OF ORDINARY SKILL IN THE ART
`
`8.
`
`I previously opined that, in my opinion, a person of ordinary skill in the
`
`art at the time at the time of the alleged invention (August 9, 2002) described in the
`
`’632 patent and the ’253 patent would have had a Bachelor’s degree in Computer
`
`Science, or an equivalent field, and three to five years of experience working with
`
`computer implemented tracking systems, virtual reality systems, and/or augmented
`
`reality systems. Additional education might compensate for less experience, and vice
`
`versa.
`
`9.
`
`I understand from counsel that Patent Owner and its expert, Mr. Baillot,
`
`adopted my opinion regarding the level of ordinary skill in the art. Accordingly, my
`
`supplemental opinions reflected in this declaration apply that same level of ordinary
`
`skill in the art. However, my supplemental opinions would not change if a somewhat
`
`higher or lower level of ordinary skill in the art is ultimately adopted.
`
`III. SUPPLEMENTAL OPINIONS
`
`A. WELCH-2001 IN COMBINATION WITH WELCH-1997
`“ENUMERAT[ES],” OR OBVIOUSLY WOULD HAVE “ENUMERAT[ED],”
`THE SENSING ELEMENTS AVAILABLE TO IT
`
`10.
`
`I understand from counsel that Patent Owner’s expert (Mr. Baillot)
`
`provided his opinion that a person of ordinary skill in the art would have understood
`
`“enumerating a set of sensing elements available to a tracking system” to mean
`
`“specifying or listing each of the sensing elements available to a tracking system.”
`
`EX2007, ¶72.
`
`
`
`4
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`I have been asked by Petitioner whether, in my opinion, Welch-2001
`
`11.
`
`(EX1007) in combination with Welch-1997 (EX1008) “enumerates,” or would
`
`obviously have “enumerated,” the sensing elements available to its tracking system
`
`under Mr. Baillot’s construction of “enumerating” discussed above. In my opinion,
`
`the Welch-2001+Welch-1997 combination does “enumerate” the sensing elements
`
`available to its tracking system, or at least would obviously have done so.
`
`12. As background, Welch-2001’s Figure 6 embodiment shows only two
`
`HiBall units in the system, but Welch-2001 teaches that its system can accommodate
`
`up to four HiBall units. See EX1007, 9. Thus, a POSITA would understand that
`
`Welch-2001’s system can accommodate a varying number of sensing elements in its
`
`system.
`
`13. Welch-2001 teaches that “[a]fter each Hiball is assembled, we perform
`
`an offline calibration procedure to determine the correspondence between image-
`
`plan coordinates and rays in space,” as part of Welch-2001’s “Bench-Top (Offline)
`
`Hiball Calibration.” EX1007, 9. Welch-2001 further teaches a process “used to
`
`subtract out DC bias, low-frequency noise, and background light from the LED
`
`signal” as part of its “Online” calibration procedure. EX1007, 10.
`
`14. A person of ordinary skill in the art would understand that each HiBall
`
`unit would undergo this calibration process. Welch-1997 (which is combined with
`
`Welch-2001) related teaches that “each [] sensor [is] to be calibrated” (EX1008 §§
`
`
`
`5
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`3.2, 3.2.1), which was standard for tracking systems at the time because, as a person
`
`of ordinary skill in the art would have known, each sensor is slightly different in the
`
`real world as a consequence of natural variance in the manufacturing process.
`
`15. Since “each” HiBall unit needs to be calibrated, and different
`
`embodiments of Welch-2001’s system have different numbers of HiBall units
`
`attached to it, something must specify to the PC (which performs Welch-2001’s
`
`tracking calculations) how many sensing elements are available to it, so that the
`
`system knows how many calibration procedures to perform.
`
`16.
`
`In practice, a person of ordinary skill in the art would have recognized
`
`that this specifying process would be reflected in code as the instantiation of a
`
`module associated with each sensor. Even if each HiBall unit were instantiated with
`
`the same underlying code module (because they all use the same number and type
`
`of LEPD sensors), each code module would have to carry the calibration parameters
`
`associated with its respective HiBall unit resulting from the calibration process
`
`described above. In other words, a POSITA would recognize that, for example, a
`
`code module associated with HiBall #1 would have some calibration parameters
`
`associated with it, and the system would have to have specified a separate code
`
`module associated with HiBall #2 because that HiBall #2 would have different
`
`calibration parameters associated with it (although some or all of those calibration
`
`
`
`6
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`parameters might ultimately take the same value as the calibration parameters
`
`associated with HiBall #1).
`
`17. To the extent Welch-2001 in combination with Welch-1997 is not
`
`understood to have “enumerated” the sensing elements, in my opinion it would have
`
`been obvious to a person of ordinary skill for Welch-2001 in combination with
`
`Welch-1997 to have specified or listed the sensing elements available to the tracking
`
`system. As discussed above, Welch-2001 contemplates a varying number of HiBall
`
`units in its system, and Welch-2001 and Welch-1997 both teach that “each” sensing
`
`element in the HiBall units needs to be calibrated. See EX1007, 9; EX1008 § 3.2.
`
`A person of ordinary skill would recognize that Welch-2001 would reasonably list
`
`the sensing elements available to it for a number of reasons, including: (i) keeping
`
`track of which HiBall units had been calibrated and which had not yet been
`
`calibrated; (ii) keeping track of the calibration parameters for each HiBall unit that
`
`has been calibrated, as such parameters can be different for each HiBall in the
`
`system; and (iii) keeping track of which HiBall units have provided measurement
`
`information at any given time, versus which HiBall units the system is still waiting
`
`to receive measurements from.
`
`18.
`
`In all of my experience, I have never worked with a tracking system
`
`that did not, in some form or another, specify or list the sensing elements available
`
`to it. That is because, as a person of ordinary skill would have recognized, the
`
`
`
`7
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`tracking system needs to know how many and what types of sensors are available to
`
`it in order to properly use the collected data. Without such knowledge, the system
`
`might not collect data from all the sensors available to it, might not know how to use
`
`the data it receives, and might not know how to correct the received data based on
`
`calibration. All of those issues are easily avoidable by specifying or listing the
`
`sensing elements available to the tracking system, which a person of ordinary skill
`
`thus would obviously have done (if they did not always already do that, which in my
`
`experience, they do).
`
`B. HORTON PERFORMS “COMPUTATIONS”
`
`19. Horton’s (EX1010) Figure 1 shows a number of linear accelerometer
`
`elements that pass information to corresponding low-pass filters, which in turn pass
`
`information to a multiplexor, which in turn passes information to an A/D (analog-
`
`to-digital) converter, which in turn passes “acceleration data” on to the tracking
`
`processor:
`
`
`
`8
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`
`
`
`See also EX1010, 3:63-4:46 (Horton’s discussion of these components of the
`
`accelerometer package).
`
`20.
`
`I have been asked by Petitioner whether these low-pass filters,
`
`multiplexor, and A/D converter perform, or would obviously have performed,
`
`computations. In my opinion, they do perform computations.
`
`21. For example, a person of ordinary skill in the art would have known
`
`that the the multiplexer (such as an ADG508A, as taught by Horton, EX1010 at 4:11-
`
`13) is controlled by a digital value that selects one of the six inputs to be passed to
`
`the output. This selection process is a sequence whose timing needs to be
`
`synchronized to the A/D converter. The selection has to be made and held for a
`
`
`
`9
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`period of time sufficient for the A/D conversion process to complete for a given
`
`input. Then the selection is changed and the A/D process is repeated. Thus, this
`
`selection and timing process is part of a computation that computes time intervals
`
`and performs functions to advance the selection process. A person of ordinary skill
`
`would recognize that this was how a multiplexor and A/D converter worked
`
`together.
`
`22. A person of ordinary skill would also have recognized that the A/D
`
`converter is itself a small computing system because A/D conversion requires a
`
`sequence of computing steps. In an A/D converter, there is a D/A (digital-to-analog)
`
`converter that takes an internally generated digital value and converts it to an analog
`
`voltage. That voltage is compared to the input voltage (in this case, from the
`
`multiplexer) and a decision is made depending on which is the greater voltage.
`
`Ultimately, the D/A voltage is found that best matches the input voltage, and the
`
`digital value that creates that D/A voltage is the output of the A/D converter. Various
`
`computational algorithms may be used to find the digital value quickly, and that
`
`algorithm is embedded in the A/D converter and controls its operation.
`
`23. Once the A/D conversion is complete, the digital value that was
`
`computed for each accelerometer is stored in memory (via computing functions that
`
`store data) as acceleration data (35) so that it can be communicated to and/or
`
`accessed by the tracking processor (40) over some interface. That process also
`
`
`
`10
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`requires computing the proper timing and control so that the A/D output is written
`
`to memory when the output value is available from the A/D converter.
`
`24. As a person of ordinary skill would have recognized, all of these steps
`
`and processes are orchestrated and timed carefully so that the values in memory are
`
`the filtered and digitized accelerometer data. In my opinion, that orchestration is a
`
`form of programming that involves performing computations, including because the
`
`designer of these devices would program the sequence of steps into some computing
`
`engine such as a sequential state machine or a small dedicated computing element.
`
`As a person of ordinary skill would have recognized, the sequence of steps and their
`
`timing are usually expressed as a program that is stored in a read-only memory, as
`
`many other computer programs are.
`
`25. All of these computations by the multiplexor and A/D converter are a
`
`result of their underlying design, and are independent of how Horton’s tracking
`
`processor is implemented or uses these values. In other words, regardless of whether
`
`Horton’s tracking system used a Kalman filter or some other type of algorithm to
`
`track objects, regardless of whether Horton would correct the values based on
`
`calibration parameters or not, regardless of whether Horton’s tracking system used
`
`other sensors beyond the six linear accelerometer sensing elements shown in Figure
`
`1, and regardless of other options available to the system designer for implementing
`
`Horton’s tracking system, the computations performed by the multiplexor and A/D
`
`
`
`11
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`converter would still be the same computations they are designed to perform and
`
`they would be performed in the same way and the results of any particular
`
`computation would still be the same (given the same input).
`
`C. HORTON “ENUMERAT[ES],” OR OBVIOUSLY WOULD HAVE
`“ENUMERAT[ED],” THE SENSING ELEMENTS AVAILABLE TO IT
`
`26.
`
`I have been asked by Petitioner whether, in my opinion, Horton
`
`(EX1010) “enumerates,” or would obviously have “enumerated,” the sensing
`
`elements available to its tracking system under Mr. Baillot’s construction of
`
`“enumerating” discussed above. In my opinion, Horton does “enumerate” the
`
`sensing elements available to its tracking system, or at least would obviously have
`
`done so.
`
`27. Horton teaches setting a value “i” in its Table 1 code, which determines
`
`the number of accelerometers available to the system. See EX1010, 11:16-17. For
`
`example, Horton specifies that in one embodiment there are six accelerometers. See
`
`EX1010, Fig. 1, 3:41. Horton also specifies the make and model of each
`
`accelerometer in that embodiment. See EX1010, 3:64 (“ADXL-2 manufactured by
`
`Analog Devices Corp.”).
`
`28. Horton goes on teach that the system then initializes each of the six
`
`accelerometers in that embodiment. See EX1010, 5:64-6:3. A person of ordinary
`
`skill would understand that, as a result of specifying each of the six accelerometers
`
`available to the system during the process of initialization, Horton’s main loop 41 is
`
`
`
`12
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`Declaration of Ulrich Neumann, Ph.D.
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`able to go through and read data from each accelerometer 1 through 6, because it
`
`knows that there are six accelerometers, and further knows that each accelerometer
`
`is an accelerometer (as opposed to another type of sensor taught by Horton, such as
`
`a magnetic sensor (see EX1010, claim 8)), and further knows what the proper
`
`correction calibration factors are for each accelerometer based on their pre-specified
`
`bias (EX1010, 5:64-67). See EX1010, 12:44-56.
`
`29. A person of ordinary skill would therefore have understood that, for any
`
`particular embodiment of Horton, the system would need to specify all the sensors
`
`available to it, at least once during initialization, so that Horton’s system knows how
`
`many sensing elements to collect data from, what kind of data those sensing elements
`
`generate, and what their correction factors are. Failure to do so would mean that the
`
`system would not necessarily know how to use the data available to it such that the
`
`tracking system would not be expected to calculate an accurate tracking estimate (if
`
`it was capable of calculating anything at all).
`
`30. To the extent that Horton is not understood to have “enumerated” the
`
`sensing elements, in my opinion it would have been obvious to a person of ordinary
`
`skill for Horton to have specified or listed the sensing elements available to the
`
`tracking system. Horton teaches that many different types and numbers of sensing
`
`elements can be used (see EX1010, e.g., 14:44-49, 3:44-52), and the estimation
`
`subsystem needs to know what kind of and how much data it will be receiving.
`
`
`
`13
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`
`
`
`
`Again, as I noted above, I have never worked with a tracking system that did not
`
`Declaration of Ulrich Neumann, Ph.D.
`
`
`
`
`
`IPR2022-01304, IPR2022-01305, IPR2022-01308
`
`
`
`specify or list the sensing elements available to it in some capacity. That is because,
`
`
`
`
`
`
`
`
`
`
`
`as a person of ordinary skill would recognize, the tracking system needs to know
`
`
`
`
`
`how many and what types of sensors are available to it in order to properly use the
`
`
`
`
`
`
`
`
`
`collected data. Without such knowledge, the system might not collect data from all
`
`
`
`
`
`
`
`
`
`the sensors available to it, might not know how to use the data it receives, and might
`
`
`
`
`
`All of those
`not know how to correct the received data based on prior calibration.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`issues are easily avoidable by specifying or listing the sensing elements available to
`
`
`
`
`
`
`
`
`
`the tracking system, which a person of ordinary skill thus would obviously have
`
`
`
`
`
`
`
`
`
`done (if they did not always already do that, which in my experience, they do).
`
`IV.CONCLUSION
`
`
`
`31.I hereby declare under penalty of perjury under the laws of the United
`
`
`
`
`
`
`
`States of America that the foregoing is true and correct, and that all statements made
`
`
`
`
`
`
`
`
`
`
`
`
`
`of my own knowledge are true, and that all statements made on information and
`
`
`
`
`
`are that willful false statements belief are believed to be true. I understand
`
`
`
`
`
`
`
`
`
`
`
`punishable by fine or imprisonment or both. 18 U.S.C. § 1001.
`
`
`
`
`
`Date: September 20, 2023
`
`
`
`Respectfully submitted,
`
`
`
`Dr. Ulrich Neumann
`
`14
`
`META 1038
`IPR2022-01308
`META V. THALES
`
`