`
`
`
`Filed: April 25, 2019
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`____________________
`
`LG ELECTRONICS INC..
`Petitioner
`
`v.
`
`CYWEE GROUP LTD.
`Patent Owner
`
`____________________
`
`Case IPR2019-00560
`Patent No. 8,552,978
`____________________
`
`
`EXPERT DECLARATION OF DR. JOSEPH LAVIOLA, PH.D., IN
`SUPPORT OF PATENT OWNER PRELIMINARY RESPONSE
`
`
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`
`Table of Contents
`
`I. INTRODUCTION ............................................................................................. 1
`II. QUALIFICATIONS, PUBLICATIONS, AND PRIOR TESTIMONY ...... 2
`III. RELEVANT LEGAL PRINCIPLES ............................................................ 6
`IV. PERSON OF ORDINARY SKILL IN THE ART ........................................ 9
`V. U.S. PATENT 8,552,978 (THE “‘978 PATENT”) ....................................... 11
`A. Specification ................................................................................................. 11
`VI. Background of the Technology .................................................................... 18
`VII. CLAIM CONSTRUCTION ........................................................................ 23
`A. “3D pointing device” ................................................................................... 25
`B. “spatial reference frame”/ “spatial reference frame associated with the
`3D pointing device” ........................................................................................... 27
`C. “rotation output” ......................................................................................... 27
`VIII. REFERENCES RELIED ON BY THE PETITIONER .......................... 28
`A. U.S. Patent 7,089,148 to Bachmann (Bachmann, Exhibit 1004) ............. 28
`B. U.S. Patent Application Pub. No. 2004/0095317 to Zhang (Zhang,
`Exhibit 1005) ...................................................................................................... 32
`i. Zhang has been interpreted by the USPTO in various ex parte
`examinations against relevant parties .......................................................... 34
`C. U.S. Patent 7,158,118 to Liberty (Liberty, Exhibit 1006) ......................... 35
`i. Liberty has been interpreted by the USPTO in various ex parte
`examinations against relevant parties .......................................................... 37
`IX. BACHMANN DOES NOT QUALIFY AS ANALGOUS ART .................. 39
`X. THE REFERENCES DO NOT DISCLOSE ALL LIMITATION OF THE
`CHALLENGED INVENTIONS .......................................................................... 43
`A. Ground 1 –Zhang in view of Bachmann .................................................... 43
`i. Claim 1 ....................................................................................................... 45
`ii. Claim 12 .................................................................................................... 50
`B. Ground 2 –Liberty in view of Bachmann ................................................... 51
`i. Claim 10 ..................................................................................................... 53
`ii. Claim 12 .................................................................................................... 54
`XI. CONCLUSION .............................................................................................. 55
`
`
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`
`INTRODUCTION
`
`I have been retained by Patent Owner CyWee Group Ltd. (“CyWee” or
`
`I.
`
`1.
`
`“Patent Owner”) as an expert in the area of motion sensors and sensor fusion
`
`technology. I make this Declaration at the request of CyWee regarding my
`
`opinions as an independent expert regarding issues of validity of U.S. Patent No.
`
`8,552,978 (the “‘978 Patent”) raised in the matter of Inter Partes Review, Petition
`
`IPR2018-01257 (“Petition”).
`
`2.
`
`I am being compensated for this work at the rate of $400/hr, and my
`
`compensation is not dependent on the outcome of this matter.
`
`3.
`
`In preparation for this Declaration, I studied Exhibits 1001-1006 provided by
`
`Petitioner as well as the Petition. I have also studied several documents from
`
`various district court actions concerning the ‘978 Patent. These include the claim
`
`construction orders in CyWee Group Ltd. v. Samsung Electronics Co., Ltd., et al,
`
`No. 2:17-cv-00140-WCB-RSP (E.D. Tex.) (the “Samsung Suit”) and CyWee
`
`Group Ltd. V. Motorola Mobility LLC, No. 1:17-cv-00780-GMS (D. Del.) (the
`
`“Motorola Suit”).1 The claim construction orders from these cases are provided as
`
`Exhibits 2003 and 2006-2007.
`
`
`
`1 I have also reviewed the claim construction order in CyWee Group, Ltd. v.
`Huawei Device Co. Ltd., No. 2:17-cv-00495-WCB-RSP (E.D. Tex) (the “Huawei
`
` 1
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`In addition to the above Papers and other documents, my opinions herein are
`
`4.
`
`also based upon my personal knowledge, professional judgment, education and
`
`experience gained through my years as a computer scientist, professor, and
`
`consultant.
`
`II. QUALIFICATIONS, PUBLICATIONS, AND PRIOR TESTIMONY
`
`5. My Curriculum Vitae is provided as Exhibit 2005. The following is a
`
`summary of my education and relevant experience.
`
`6.
`
`I have almost 20 years of experience working in the virtual reality (“VR”)
`
`and augmented reality (“AR”) fields, as well as advancing three-dimensional
`
`(“3D”) interaction techniques for use in both VR and AR environments. More
`
`specifically, I have worked extensively on and with user and object motion
`
`tracking sensors, algorithms and systems.
`
`7.
`
`I am the Charles N. Millican Professor of Computer Science in the
`
`Department of Computer Science at the University of Central Florida, located in
`
`Orlando, Florida. I also serve as the Director of the Interactive Computing
`
`Experiences Research Cluster at the University of Central Florida. Through these
`
`functions, I supervise over fifteen graduate and undergraduate students working on
`
`various research projects in the area of human-computer interaction. In addition,
`
`
`
`Suit”) in which the court adopted the same constructions it adopted in the Samsung
`Suit.
`
` 2
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`since 2013, I have served as an Adjunct Associate Professor of Computer Science
`
`at Brown University, located in Providence, Rhode Island.
`
`8.
`
`I received my Bachelor of Science degree in Computer Science from Florida
`
`Atlantic University in 1996. I also hold two Masters degrees – an Sc.M. in
`
`Computer Science and an Sc.M. in Applied Mathematics from Brown University –
`
`which were awarded in 2000 and 2001 respectively. I received my Ph.D. in
`
`Computer Science from Brown University in 2005.
`
`9.
`
`I serve as Associate Editor for various publications in the area of human-
`
`computer interaction, including the International Journal of Human-Computer
`
`Studies and the Association for Computing Machinery’s Transactions on
`
`Interactive Intelligent Systems. I have also served as Program Chair for the IEEE
`
`Virtual Reality conference.
`
`10.
`
`I have contributed to more than 40 peer-reviewed journal publications and
`
`nearly 100 refereed conferences and workshops, the majority of which deal with
`
`virtual and augmented reality and user and object motion tracking. For instance, I
`
`am the lead author of the second edition of the most comprehensive textbook on
`
`3D user interaction, entitled “3D User Interfaces: Theory and Practice.” As part of
`
`that work, I analyzed many different types of input and output hardware, 3D user
`
` 3
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`interfaces and general topics related to virtual and augmented reality. I have also
`
`worked specifically with sensors and algorithms related to the ‘978 patent.
`
`11.
`
`I have been working with recursive-style estimators, including Kalman
`
`filters and Extended Kalman Filters (“EKFs”), for over 15 years. I have
`
`implemented Kalman filters and EKFs in practice and written several papers on
`
`them as part of my work in motion tracking and 3D interfaces, including the
`
`following:
`
`Julier, S., and LaViola, J., "On Kalman Filtering with Nonlinear Equality
`Constraints", IEEE Transactions on Signal Processing, 55(6):2774-2784,
`June 2007.
`
`LaViola, J., "A Comparison of Unscented and Extended Kalman Filtering
`for Estimating Quaternion Motion", In the Proceedings of
`the
`2003
`American Control Conference, IEEE Press, 2435-2440, June 2003.
`
`LaViola, J., "Double Exponential Smoothing: An Alternative to Kalman
`Filter-Based Predictive Tracking", In the Proceedings of Immersive
`Projection Technology and Virtual Environments 2003, ACM Press, 199-
`206, May 2003.
`
`I have also been teaching Kalman filters and EKFs to my students over the
`
`12.
`
`last 15 years. An example of work using EKFs that I published with one of my
`
`students follows:
`
`Williamson, B., and Wingrave, C., and LaViola, J., "RealNav: Exploring
`Natural User Interfaces for Locomotion in Video Games",
`Proceedings
`of the IEEE Symposium on 3D User Interfaces 2010, 3-10, March 2010.
`
`
` 4
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`I have done research exploring the function and operation of the Nintendo
`
`13.
`
`Wii Remote (Wiimote) device and written papers and given lectures on its
`
`operation. For example, I have explored how to use the Wiimote to support 3D
`
`gesture recognition interfaces:
`
`Hoffman, M., Varcholik, P., and LaViola, J., "Breaking the Status Quo:
`Improving 3D Gesture Recognition with Spatially Convenient Input
`Devices", Proceedings of IEEE Virtual Reality 2010, 59-66, March 2010.
`
`In another example, I wrote a tutorial paper with my students on how the
`
`Wiimote works, how to use it in 3D interfaces, and what its strengths and
`
`weaknesses are as well as established a set of novel 3D interface techniques that
`
`takes advantage of its functionality:
`
`Wingrave, C., Williamson, B., Varcholik, P., Rose, J., Miller, A.,
`Charbonneau, E., Bott, J. and LaViola, J. "Wii Remote and Beyond: Using
`Spatially Convenient Devices for 3DUIs", IEEE Computer Graphics and
`Applications, 30(2):71-85, March/April 2010.
`
`I have submitted declarations and deposition testimony related to claim
`
`
`14.
`
`construction in the Samsung Suit and in the Huawei Suit. I have also submitted
`
`several expert reports on various topics in the Samsung Suit, and provided a
`
`declaration in support of CyWee’s motion for partial summary judgment of
`
`infringement in that case. I have provided additional declarations related to claim
`
`construction in the Motorola Suit; CyWee Group Ltd. v. ZTE Corp. et al., No. 3:17-
`
`cv-02130-BN-RBB (S.D. Cal.); and CyWee Group v. HTC Corp., et al., No. 2:17-
`
` 5
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`cv-00932-JLR (W.D. Wa.). I have not otherwise testified in any other lawsuit in
`
`the last four years.
`
`III. RELEVANT LEGAL PRINCIPLES
`
`15.
`
`I have been asked to opine on whether claims 10 and 12 of the ‘978 Patent
`
`would have been nonobvious in light of the prior art to one skilled in the art at the
`
`time of the invention.
`
`16.
`
`I am not an attorney. In preparing and expressing my opinions and
`
`considering the subject matter of the ‘978 Patent, I relied on certain legal principles
`
`that have been explained to me by counsel.
`
`17.
`
`I understand that under 35 U.S.C. § 103, a patent may be invalid as obvious
`
`if the differences between the subject matter patented and the prior art are such
`
`that, at the time that the invention was made, the subject matter as a whole would
`
`have been obvious to a person having ordinary skill in the art (“PHOSITA”) to
`
`which said subject matter pertains. I understand that although the ultimate
`
`determination of obviousness is a question of law, there are factual issues
`
`underlying the ultimate obviousness decision. I understand that the obviousness
`
`analysis is based on four underlying factual inquiries: (1) the scope and content of
`
`the prior art;  (2) the differences between the claims and the prior art;  (3) the level
`
` 6
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`of ordinary skill in the pertinent art;  and (4) any secondary considerations of
`
`nonobviousness.
`
`18.
`
`I understand that to sustain an obviousness challenge against a claim(s), a
`
`petitioner must show that the asserted prior art references in combination must
`
`teach or suggest all the limitations of the challenged claim(s).
`
`19.
`
`I understand that to qualify as prior art for an obviousness challenge, the
`
`asserted references must be of the same type of art or of an analogous art as the
`
`challenged patent. I understand that a reference may be “analogous” if it is either
`
`(1) in the same field of endeavor as the challenged patent, or (2) if it is in a
`
`different field of endeavor, it is still reasonably pertinent to the particular problem
`
`addressed by the challenged patent.
`
`20.
`
`I understand that, because inventions often involve a combination of known
`
`elements that seem predictable in hindsight, to prevent hindsight invalidation of
`
`patent claims, the law requires some “teaching, suggestion or motivation” to
`
`combine cited references. I understand that motivation to combine references may
`
`be found in the cited references themselves, in the knowledge of a PHOSITA at the
`
`time of invention, and/or in the nature of the problem to be solved.
`
`21.
`
`I understand that the Supreme Court has recognized other rationales for
`
`combining references or modifying a reference to show obviousness of claimed
`
` 7
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`subject matter. Some of these other rationales are: (a) combining prior art
`
`elements according to known methods to yield predictable results; (b) simple
`
`substitution of one known element for another to obtain predictable results; (c) use
`
`of a known technique to improve a similar device (method, or product) in the same
`
`way; (d) applying a known technique to a known device (method, or product)
`
`ready for improvement to yield predictable results; (e) choosing from a finite
`
`number of identified, predictable solutions, with a reasonable expectation of
`
`success.
`
`22.
`
`I understand that obviousness does not require physical combination/bodily
`
`incorporation, but rather consideration of what the combined teachings would have
`
`suggested to a PHOSITA at the time of the invention.
`
`23.
`
`I further understand that an asserted motivation to combine references can be
`
`overcome based on a showing that those references together “teach away” from
`
`their combination. I understand that if the references taken in combination would
`
`produce a “seemingly inoperable device,” such references teach away from the
`
`combination and thus cannot serve as a basis for obviousness.
`
`24.
`
`I understand that within the framework of an obviousness analysis, it is
`
`impermissible to pick and choose from any one reference only so much of it as will
`
`support a given position to the exclusion of other parts necessary to the full
`
` 8
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`appreciation of what such reference fairly would have suggested to a PHOSITA.
`
`Thus, a reference must be considered in its entirety, including those portions of the
`
`reference that argue against obviousness.
`
`IV. PERSON OF ORDINARY SKILL IN THE ART
`
`25.
`
`I understand that the level of ordinary skill may be reflected by the prior art
`
`of record, and that a PHOSITA would have been capable of understanding the
`
`scientific and engineering principles applicable to the pertinent art. I understand
`
`that one of ordinary skill in the art is not an automaton and has ordinary creativity.
`
`26.
`
`I understand there are multiple factors relevant to determining the level of
`
`ordinary skill in the pertinent art, including (1) the education level of the inventor;
`
`(2) the types of problems regularly encountered in the field; (3) the solutions of the
`
`prior art to those problems; (4) the rapidity with which innovations are made in the
`
`field; (5) the sophistication of the technology in the field; and (6) the levels of
`
`education and experience of persons working in the field at the time of the
`
`invention.
`
`27.
`
`I am familiar with all of the subject matter of the ‘978 Patent (Ex. 1001). I
`
`am also aware of the state of the art at the time the application resulting in the ‘978
`
`Patent was filed. Based on the inventions disclosed in the ‘978 Patent and my
`
`experience with computer scientists at that time, I believe that a PHOSITA at the
`
` 9
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`time of the filing of the ’978 Patent would typically have at least a Bachelor’s
`
`Degree in Computer Science, Electrical Engineering, Mechanical Engineering, or
`
`Physics, or equivalent work experience, along with knowledge of sensors (such as
`
`accelerometers, gyroscopes, and magnetometers), and mobile computing
`
`technologies. In addition, a PHOSITA would be familiar with Kalman filters and
`
`EKFs, and with equations typically used with such filters.
`
`28. For purposes of this Declaration, unless otherwise noted, my statements and
`
`opinions, such as those regarding my experience and the understanding of a
`
`PHOSITA generally reflect the level of knowledge that existed in the field prior to
`
`the priority date of the ‘978 Patent.
`
`29. As will be discussed below, prior art pointing devices using motion sensors
`
`suffered from the inability to accurately detect movement and orientation along all
`
`three spatial axes. This was due in large part to the fact that each additional sensor
`
`would introduce added noise that was further amplified by the integral calculations
`
`used in determining the orientation of the pointing device. Additionally, each type
`
`of orientation sensor suffered from errors in their measurements that accumulate
`
`over time. The precise detection of actual deviation angles in three-dimensional
`
`space by pointing devices using inertial sensors was impossible before the specific
`
` 10
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`application of quaternion mathematics and the improved comparison and
`
`compensation method that are claimed by the ‘978 Patent.
`
`V. U.S. PATENT 8,552,978 (THE “‘978 PATENT”)
`
`30. The following is a discussion of the ‘978 Patent, its various aspects, and its
`
`prosecution history that are relevant to the claims subject to the Petition.
`
`A. Specification
`
`31. The ‘978 Patent discloses a 3D pointing device and a method for
`
`compensating the device’s movement and rotations. The pointing device utilizes a
`
`nine-axis sensor module that may include sensors known in the art. These sensors
`
`include sensors for detecting axial acceleration (i.e., accelerometers) along three
`
`spatial axes, sensors for detecting the angular velocity of rotation (i.e., gyroscopes)
`
`about those same three axes, and sensors for detecting magnetic field (i.e.,
`
`magnetometers) about the same three axes. Exhibit 1001, 4:15-32. The invention
`
`applies a novel “enhanced comparison method” to reduce and compensate for
`
`errors and noise in the sensor readings that normally accumulate over time in order
`
`to better map the movements of the device and have the capability to more
`
`precisely control a display. Id. at 4:33-57.
`
`32. The invention of the ‘978 Patent takes the readings of the axial accelerations
`
`along the three spatial axes from the accelerometers (the first signal set) and the
`
` 11
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`magnetic fields about the three spatial axes from the magnetometer (the second
`
`signal set) and the angular rotations about the three spatial axes from the rotational
`
`rate sensors (the rotation output), fuses the three signal sets by the method
`
`disclosed, and then is able to map the movements of the 3D pointing device onto
`
`the display frame of a screen. Id. at Claim 10, Fig. 10-11.
`
`.
`
` 12
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`33. The 3D pointing device of the ‘978 Patent is a handheld portable electronic
`
`device such as a controller, smartphone, or navigation equipment. Id. at 8:38-42,
`
`13:5-16, Figs. 3 and 6. The optional display on which the pointing device’s
`
`movements can be mapped may be attached or integrated with the pointing device
`
`itself—such as in a mobile gaming system or a smartphone. Id. at 13:5-16, Fig. 6.
`
`
`
`34. The 3D pointing device and enhanced comparison method of the ‘978 Patent
`
`utilizes three types of motion sensors: accelerometers, magnetometers, and rotation
`
`sensors such as gyroscopes. Accelerometers are used
`
`to measure axial
`
`accelerations. For example, car airbags use accelerometers to trigger release when
`
` 13
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`a sudden deceleration is detected. Accelerometers can also be used to measure the
`
`forces exerted by the acceleration due to gravity. Gyroscopes are used to measure
`
`angular velocity, the rate of rotation about an axis. Magnetometers measure
`
`magnetism such as the strength of a magnetic field along a particular direction.
`Each kind of sensor suffers from inaccuracies. For instance, gyroscopes are
`
`subject to a small, added bias. This bias is an offset from the true value that
`
`accumulates over time and eventually amasses to a potentially very large drift
`
`error. Magnetometers suffer from interference from magnetic fields generated by
`
`various natural and manmade sources (e.g., powered electronics). Each sensor
`
`typically only takes measurements along a single axial direction. In order to
`
`accurately measure motion and orientation in a three-dimensional reference frame,
`
`three sensors of a kind must be grouped together and arranged orthogonally to one
`
`another. A set of three sensors aligned at right angles like this is referred to as a
`
`three-axis sensor.
`
`35. The ‘978 Patent discloses and claims technology for combining different
`
`kinds of sensors to incorporate their data and to correct for the errors generated by
`
`the various sensors. The ‘978 Patent specifically discloses and claims an improved
`
`nine-axis sensor system and a method for measuring, calculating, and mapping
`
`orientation (including deviation angles along three axes) by using measurements
`
` 14
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`from a three-axis accelerometer, a three-axis magnetometer, and a three-axis
`
`rotation sensor such as a three-axis gyroscope. Id. at 4:15-32. The ‘978 Patent
`
`further discloses and claims an enhanced method for combining or fusing the
`
`various signal sets that is capable of reducing the errors and noise that accumulate
`
`over time. Id. at 4:33-57. This technology is capable of accurately representing the
`
`orientation and movement of a portable 3D pointing device in three-dimensional
`
`space on a two-dimensional display. Id. at 7:55-67. The invention of the ‘978
`
`Patent is capable of mapping deviation angles such as yaw, pitch, and roll of the
`
`pointing device in relation to its movement onto a display reference frame, such as
`
`that of a two-dimensional display screen. Id. at 5:12-45. In sum, the ‘978 Patent
`
`discloses and claims an improved system and method for capturing the motion of a
`
`pointing device in three-dimensional space, compensating or correcting for errors
`
`in the sensed movement, and outputting a movement pattern on a display frame to
`
`control actions on a display.
`
`36. The invention of the ‘978 Patent corrects for such errors using its novel
`
`enhanced comparison method. An embodiment of the method can be found in
`
`Figure 10 of the ‘978 Patent. Id. at Fig. 10. First, a quaternion is taken from a
`
`previous timestep t-1 (the first quaternion) and used in conjunction with the
`
`angular velocity from the rotation sensor (three-axis gyroscope) and the differential
`
` 15
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`equation described in Equation 1 to compute a current quaternion at time t (the
`
`second quaternion). Id. at 16:27-56, Fig. 10. This second quaternion is used to
`
`compute predicted axial accelerations with Equations 2-4 and predicted
`
`magnetisms with Equations 18-20 in the ‘978 Patent. Id. at 16:57-17:23, 22:34-
`
`23:8. The method then takes the current quaternion (second quaternion), the
`
`predicted axial accelerations, the measured axial accelerations (first signal set), the
`
`predicted magnetisms, and the measured magnetisms (second signal set) and uses
`
`Equations 5-11 to compute an updated quaternion (third quaternion). Id. at 17:24-
`
`18:33.
`
`37. Equations 5-11 represent a non-linear recursive estimator (as any PHOSITA
`
`would understand by examining the mathematical equations) which, in the case of
`
`this embodiment of the ‘978 Patent, combines elements of an extended Kalman
`
`Filter coupled with a weighted vector norm. Id. Equation 5 is the predicted state
`
`estimate. Id. at 17:47-54. The predicted covariance estimate is equivalent to the
`
`Equation above Equation 6 in the ’978 patent. Id. at 17:55-67. The state transition
`
`Jacobian matrix is equivalent to Equations 6 and 7 in the ‘978 Patent. Id. This state
`
`transition Jacobian matrix maps to both Equations 6 and 7 based on which variable
`
`is held constant during partial differentiation. Equation 8 describes the innovation
`
`or measured residual. Id. at 18:1-6. The innovation or measured residual
`
` 16
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`covariance matrix is Equation 9. Id. at 18:7-17. The observation Jacobian matrix
`
`described is equivalent to Equation 10. Id. Finally, the weighted vector norm is
`
`presented as Equation 11 which ultimately takes the information from the process
`
`and measurement models and finds an optimal recursive least squares estimate for
`
`the updated quaternion. Id. at 18:18-24. This procedure (Equations 5-11)
`
`effectively combines or fuses the second quaternion, the predicted axial
`
`acceleration, the measured axial accelerations, the predicted magnetism, and the
`
`measured magnetism to compute the third quaternion (updated state). Id. at 17:24-
`
`18:33, Fig. 10.
`
`38. The orientation information output by the claimed invention of the ‘978
`
`Patent has various potential applications, particularly when applied to mobile
`
`cellular devices. These applications can
`
`include navigation, gaming, and
`
`augmented and virtual reality simulation. Navigation applications can use
`
`orientation information from the mobile phone to determine the direction the user
`
`is facing and can then re-align the map to align with the proper cardinal directions.
`
`Mobile games and other applications can use the motion of the phone to trigger
`
`input commands; for instance, rotating the mobile phone to simulate turning the
`
`steering wheel of a vehicle in a racing game. Augmented and virtual reality
`
`simulators rely on the accurate determination of a device’s orientation to properly
`
` 17
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`render graphics and images at the correct location on the screen. Even small
`
`inconsistencies in tracking a device’s orientation and movement in virtual reality
`
`simulation can result in severe cybersickness for the user, rendering the simulator
`
`unusable.
`
`VI. Background of the Technology
`
`39. Before 2010, technological hurdles severely limited the applicability of
`
`motion sensors to portable electronic devices. For example, the sensors were often
`
`only accurate when static because they could not differentiate between different
`
`kinds of acceleration (e.g. linear, centrifugal, gravitational). Ex. 1001 at 3:13-19.
`
`Rapid, dynamic, and unexpected movements caused significant errors and
`
`inaccuracies. These hurdles were largely due to the fact that acceleration is
`
`inherently a “noisy” signal. Angular acceleration is calculated as the derivative of
`
`angular velocity with respect to time, and angular velocity is itself the derivative of
`
`orientation with respect to time. Thus, small magnitudes of noise caused by small –
`
`even imperceptible – changes in rotation can result in rather large derivatives. This
`
`means that even seemingly minor noise from outside sources such as vibration or
`
`electrical fluctuations will be considerably amplified when measured at the
`
`acceleration level. As a result, stationary devices will often still have notable noise
`
`when measured by accelerometers alone, and movement of the device will magnify
`
` 18
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`that noise. Since accelerometers measure linear and centripetal accelerations as
`
`well as the acceleration due to gravity, orientation estimates of certain movements
`
`of a device based on this type of sensor alone will not be accurate.
`
`40. Prior to the invention of the ‘978 Patent, these difficulties were exacerbated
`
`by the miniaturization of sensors. The development of micro-electromechanical
`
`systems (“MEMS”) allowed for miniaturized motion sensors to be manufactured
`
`and incorporated on semiconductor chips; the sensors could then be used in smaller
`
`devices such as portable electronic devices. MEMS sensors, however, also had
`
`significant limitations. For instance, a MEMS accelerometer could not distinguish
`
`between different types of acceleration. Because of this, an enhanced filter method
`
`is necessary to correct for extraneous readings of other types of acceleration as
`
`high-frequency noise to produce a more accurate reading. Another limitation of
`
`early MEMS sensors is the drift to which MEMS gyroscopes are prone that will
`
`accumulate over time without recalibration or correction by another sensor. Each
`
`additional sensor that is added to a device brings the challenge of added noise and
`
`errors that must be accounted for when calculating the true orientation of the
`
`device.
`
`41. Orientation cannot be properly calculated from a single type of MEMS
`
`sensor alone. For example, when using only a three-axis MEMS accelerometer to
`
` 19
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`determine orientation, only pitch and roll can be measured, but not yaw. When
`
`using only a MEMS gyroscope to measure angular velocity, only relative changes
`
`in orientation can be measured. When using these different sensors together in
`
`combination with a fusion algorithm, the different sensors can be used to account
`
`for the errors of the other.
`
`42. Without any orientation information, mobile device applications would be
`
`limited to static operations. This was the case with early smartphones and other
`
`mobile devices. Navigation programs on such devices, for example, could only
`
`render a map with the North axis oriented towards the top of the screen and could
`
`indicate location of the device by using GPS; however, these devices were
`
`incapable of sensing user direction or heading and appropriately and automatically
`
`orienting the map with the direction of the user towards the top of the screen. Users
`
`could manually rotate the image of the map in these earlier navigation programs
`
`with touch commands, but the map would not automatically rotate as the user
`
`changed directions. Those devices could not indicate what direction the device was
`
`facing.
`
`43. Modern games will frequently use the motion of a handheld device to
`
`control the game. One common motion-based control scheme is to rotate the
`
`device—such as a mobile phone or gaming controller—like a steering wheel to
`
` 20
`
`

`

`Case IPR2019-00560
`Patent No. 8,552,978
`Declaration of Joseph LaViola, Ph.D.
`
`simulate operating a vehicle in drivi