Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 1 of 18 PageID #: 120
`
`Case 1:18-cv-00571—GMS Document 1—5 Filed 04/16/18 Page 1 of 18 PageID #: 120
`
`EXHIBIT E
`
`EXHIBIT E
`
` LG 1011
`
`

`

`Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 2 of 18 PageID #: 121
`
`U.S. Patent No. 8,552,978
`
`Google Pixel
`
`SUBJECT TO CHANGE
`
`1
`
`

`

`Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 3 of 18 PageID #: 122
`U.S. Patent No. 8,552,978 – GOOGLE PIXEL
`
`Claim 10
`A method for compensating rotations of a 3D pointing device, comprising:
`
`Google Pixel
`
`SUBJECT TO CHANGE
`
`2
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GMSs, W815; 9Fsledm1fiimqme 4 of 18 PagelD #: 123
`
`generating an orientation output associated with an orientation of the 3D pointing device associated with three coordinate axes of a
`
`global reference frame associated with Earth;
`
`
`When the orientation sensor is software—based, the orientation output is the attitude of the device that can be represented by the
`azimuth, pitch, and roll angles relative to the magnetic North Pole associated with a global reference frame associated with Earth.
`
`Rotation vector
`
`Underlying physical sensors: Accelerometer, Magnetometer, and Gyroscope
`
`Reporting-mode: Continuous
`
`getDefaultSensor(SENSOR_TYPE_ROTATION_VECTOR) returns a non-wake-up sensor
`
`A rotation vector sensor reports the orientation of the device relative to the East-North-UB coordinates frame. It is usually
`obtained by integration of accelerometer, gyroscope, and magnetometer readings. The East-North-Up coordinate system
`
`coordinates (x,y,z).
`
`is defined as a direct orthonormal basis where:
`
`. X points east and is tangential to the ground.
`
`. Y points north and is tangential to the ground.
`
`. 2 points towards the sky and is perpendicular to the ground.
`
`TheW necessary to align theW with the
`
`phone's coordinates. That is, applying the rotation to the world frame (XIYIZI would align them with the phone
`
`Source: https://source.android.com/devices/sensors/sensor-types#rotation_vector
`
`SUBJECT TO CHANGE
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GMSs,mug 9Ffledmrfimfime 5 of 18 PagelD #: 124
`
`generating a first signal set comprising axial accelerations associated with movements and rotations of the 3D pointing device in the
`spatial reference frame;
`
`
`Accelerometer
`
`Reporting-mode: Continuous
`
`getDefaultSensor(SENSOR_TYPE_ACCELEROMETER) returns a non-wake-up sensor
`
`An accelerometer sensor reports the acceleration of the device along the 3 sensor axes. The measured acceleration
`includes both the physical acceleration (change of velocity) and the gravity. The measurement is reported in the x, y and
`2 fields of sensors_event_t.acceleration.
`
`All values are in SI units (m/s‘2) and measure the acceleration of the device minus the force of gravig along the 3
`W
`
`
`
`device) that's used by the Sensor API.
`
`Source: https://source.android.com/devices/sensors/sensor-types#accelerometer
`
`Sensor Coordinate System
`
`In general, the sensor framework uses a standard 3-axis coordinate system to express data values. For most
`sensors, the coordinate system is defined relative to the device's screen when the device is held in its default
`orientation (see figure I 2. When a device is held in its default orientation, the X axis is horizontal and points to
`the right, the Y axis is vertical and points up, and the Z axis points toward the outside of the screen face. In
`this system, coordinates behind the screen have negative 2 values. This coordinate system is used by the
`following sensors:
`
`- Acceleration sensor
`
`Gravity sensor
`
`Gyroscope
`Linear acceleration sensor
`.
`.
`Geomagnetlc field sensor
`
`
`
`Figure 1. Coordinate system (relative to a
`
`Source: http://developer.android.com/guide/topics/sensors/sensors_overview.html#sensors-coords
`
`SUBJECT TO CHANGE
`
`4
`
`

`

`Claim 10
`
`Case 1:18-cv-00571—GMSS. mnflpfilemlfi’gme 6 of 18 PagelD #: 125
`
`generating a second signal set associated with Earth's magnetism;
`
`
`The magnetometer (i.e., the compass) generates a second signal set associated with Earth’s magnetism.
`
`Magnetic field sensor
`
`Reporting-mode: Continuous
`
`getDefaultSensor ( SENSOR_TYPE_MAGNETIC_FIELD) returns a non-wake—up sensor
`
`SENSOR_TYPE_GEOMAGNETIC_FIELD == SENSOR_TYPE_MAGNETIC_FIELD
`
`Source: https://source.android.com/devices/sensors/sensor-types#magnetic_field_sensor
`
`A magnetic field sensor (also known as magnetometer) reports the ambient magnetic field, as measured along the 3
`sensor axes.
`
`The measurement is reported in the x, y and 2 fields of sensors_event-t .magnetic and all values are in micro-Tesla
`
`(uT).
`
`SUBJECT TO CHANGE
`
`

`

`Claim 10
`
`Case 1:18-cv-00571—GMSS.WhflgffleWMe 7 of 18 PagelD #: 126
`
`generating the orientation output based on the first signal set, the second signal set and the rotation output or based on the first
`signal set and the second signal set;
`
`The Android source code shows generating the orientation output based on the first signal set, the second signal set and the
`
`rotation output.
`
`flmction passes rotation output w to the predict ()
`The handleGyro ()
`orientation output, KO.
`
`
`fimction and the update ()
`
`flmction to calculate an
`
`313
`
`314
`
`315
`
`void Fusion::handleGyro(const vec3_t& w, float dT)
`
`{
`
`if (lcheckInitComplete(GYRO, w, dT))
`
`return;
`
`xlwa
`
`void Fusion::predict(const vec3_t& w, float dT)
`
`{
`
`‘12]
`
`85
`
`— 4
`
`6
`
`= x0;
`const vec4_t q
`
`
`x0 = 0*q;
`
`
`
`‘ PW)
`
`49b
`
`void Fusion::update(const vec3_t& 2, const vec3_t& Bi, float sigma)
`
`vec4_t q(x0);
`
`{
`
`(329
`
`53M
`
`const VeC3_t e(Z _ Bb);
`
`const vec3_t dq(K[0]*e);
`
`q +2 getF(q)*(0.5f*dq);
`
`x0 = normalize_quat(q);
`
`Source: https://android.googlesource.com/platform/frameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`SUBJECT TO CHANGE
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GMSs,meg 9ffledmflfimfime 8 of 18 PagelD #: 127
`
`generating the orientation output based on the first signal set, the second signal set and the rotation output or based on the first
`signal set and the second signal set;
`
`
`The handleAcc 0 function passes the accelerometer measurements (first signal set) a to the update ()
`
`function, which updates the
`
`orientation output x0.
`
`status_t Fusion::hand1eAcc(const vec3_t& a, float dT)
`
`{
`
`if (lcheckInitComplete(ACC, a, dT))
`
`return BAD_VALUE;
`
`x0 = normalize_quat(Q);
`
`void Fusion::update(const vec3_t& 2, const vec3_t& Bi, float sigma)
`
`{
`
`vec4_t q(x0);
`
`const vec3_t e(z - Bb);
`
`const vec3_t dq(K[0]*e);
`
`q += getF(q)*(0.5f*dQ);
`
`Source: httpsz/landroid.googlesource.com/p1atfonn/fiameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`SUBJECT TO CHANGE
`
`w
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GMSs,mug gfiedflfltfimfime 9 of 18 PagelD #: 128
`
`generating the orientation output based on the first signal set, the second signal set and the rotation output or based on the first
`signal set and the second signal set;
`
`
`The handleMag 0 fimction passes the magnetometer measurements (second signal set) In to the same update 0 , which also updates
`the orientation output x0.
`
`
`
`status_t Fusion::hand1eMag(const vec3_t& m)
`
`{
`
`if (lcheckInitCompIete(MAG, m))
`
`return BAD_VALUE;
`
`x0 = normalize_quat(Q);
`
`void Fusion::update(const vec3_t& 2, const vec3_t& Bi, float sigma)
`
`{
`
`vec4_t q(x0);
`
`const vec3_t e(z - Bb);
`
`const vec3_t dq(K[0]*e);
`
`q +2 getF(q)*(0.5f*dQ);
`
`Source: httpszllandroid.googlesource.com/p1atfonn/frameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`SUBJECT TO CHANGE
`
`S
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GM63Dammfl2WPEE? 10 of 18 PagelD #: 129
`
`generating a rotation output associated with a rotation of the 3D pointing device associated with three coordinate axes of a spatial
`reference frame associated with the 3D pointing device; and
`
`
`Gyroscope
`
`Reporting-mode: Continuous
`
`getDefaultSensor ( SENSOR_TYPE_GYROSCOPE) returns a non—wake-up sensor
`
`A gyroscope sensor reeorts the rate of rotation of the device around the 3 sensor axes.
`
`Rotation is positive in the counterclockwise direction (right-hand rule). That is, an observer looking from some positive
`
`location on the x, yor z axis at a device positioned on the origin would report positive rotation if the device appeared to
`
`be rotating counter clockwise. Note that this is the standard mathematical definition of positive rotation and does not
`
`agree with the aerospace definition of roll.
`
`Source: https://source.android.com/devices/sensors/sensor—types#gyroscope
`
`Sensor Coordinate System
`
`In general, the sensor framework uses a standard 3-axis coordinate system to express data values. For most
`sensors,
`.
`.
`.
`.
`.
`,
`.
`.
`.
`.
`
`WW. When a device is held in its default orientation, the X axis is horizontal and points to
`the right, the Y axis is vertical and points up, and the Z axis points toward the outside of the screen face. In
`this system, coordinates behind the screen have negative 2 values. This coordinate system is used by the
`following sensors:
`
`
`
`
`
`- Acceleration sensor
`
`Gravity sensor
`
`Gyroscope
`Linear acceleration sensor
`
`Geomagnetic field sensor
`
`Source: http://developer.android.com/guide/topics/sensors/sensors_overview.html#sensors-coords
`
`SUBJECT TO CHANGE
`
`Figure 1. Coordinate system (relative to a
`device) that's used by the Sensor API.
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GM83Damn1g552meg 11 of 18 PagelD #: 130
`
`using the orientation output and the rotation output to generate a transformed output associated with a fixed reference frame
`associated with a display device |Cou11’s construction: using the orientation output and the rotation output to generate a transformed
`output that corresp_onds to a two-dimensional movement in a plane that is parallel to the screen of a display device I,
`
`The fixed reference frame is defined by the horizontal and vertical axes of pixels on HTC 10 display device.
`
`y
`
`Figure 1. Coordinate system (relative to a
`
`device) that's used by the Sensor API.
`
`i
`
`Three Dimensional Sensors
`
`m b a v u
`.SensorKlnettcs S
`Mum-Sensor Recorder
`
`
`
`Source: http://developer.android.com/guide/topics/sensors/sensors_overview.html#sensors—coords
`
`SUBJECT TO CHANGE
`
`
`
`
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GM65Damummgg2meg 12 of 18 PagelD #: 131
`
`using the orientation output and the rotation output to generate a transformed output associated with a fixed reference frame
`associated with a display device |Couit’s construction: using the orientation output and the rotation output to generate a transformed
`output that corresp_onds to a two-dimensional movement in a plane that is parallel to the screen of a display device I,
`
`Google sells the Google Daydream View, which promotes and induces use of CyWee’s patents. For example, the screen below
`
`shows a transformed output that corresponds to movements of the Google Pixel via AR and/or VR apps.
`
`Source: https://vr.google.com/daydream/sma11phonevr/
`
`SUBJECT TO CHANGE
`
`11
`
`

`

`Case 1:18-cv-00571-GM83Dammfl2WPEE? 13 of 18 PagelD #: 132
`
`using the orientation output and the rotation output to generate a transformed output associated with a fixed reference frame associated with a
`display device |Court’s construction: using the orientation output and the rotation output to generate a transformed output that corresppnds to a
`
`two-dimensional movement in a plane that is parallel to the screen of a display device |,
`
`The remapCoordinateSystem () flmction transforms the orientation output (inR) to a transformed output (outR), associated with a
`two dimensional movement in a plane that is parallel to the screen of a display device.
`
`inR,
`
`int X.
`
`int V, float[: outR)
`
`{
`
`
`
`public static boolean remapCoordinateSystem(float[]
`if (inR == outR)
`{
`final float[] temp = sTempNatrix;
`synchronized (temp)
`{
`// we don't expect to have a lot of contention
`if (remapCoordinateSystemImpI(inR. X. Y.
`tempi)
`final int size = outR.1ength;
`for (int i = e;
`i
`< size:
`i++)
`outREi]
`= temp[i];
`
`{
`
`{
`
`} r
`
`eturn true;
`
`i r
`
`1
`
`eturn remapCoordinateSystemImp1(inR. X. Y, outR);
`
`Source: https://android.googlesource.com/platform/frameworks/base/+/master/core/javalandroid/hardware/SensorManager.java
`boolean remapCoordinateSystem (float[] inR,
`int XJ
`int Y,
`float[] outR)
`
`Rotates the supplied rotation matrix so it is expressed in a different coordinate system. This is typically used when an application needs to
`
`compute the three orientation angles of the device (see geturientationfiloatfl , float [ ] )) in a different coordinate system.
`
`When the rotation matrix is used for drawing (for instance with OpenGL ES), it usually doesn't need to be transformed by this function, unless
`
`the screen is physically rotated, in which case you can use Display.getRotation() to retrieve the current rotation of the screen. Note that
`
`because the user is generally free to rotate their screen, you often should consider the rotation in deciding the parameters to use here.
`
`Source: http://developer.android.com/reference/android/hardware/SensorManager.html
`
`SUBJECT TO CHANGE
`
`13
`
`

`

`Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 14 of 18 PageID #: 133
`U.S. Patent No. 8,552,978 – GOOGLE PIXEL
`
`Claim 10
`wherein the orientation output and the rotation output is generated by a nine-axis motion sensor module;
`
`The Google Pixel includes a 3-axis gyroscope, a 3-axis accelerometer, and a 3-axis magnetometer which form a nine-axis motion
`sensor module.
`
`SUBJECT TO CHANGE
`
`13
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GM8.sDamflIfl2,WflflflBng 15 of 18 PagelD #: 134
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx,
`My, M2 and a plurality ofpredicted magnetism Mx', My’ and Mz' for the second signal set.
`
`The measured magnetisms Mx, My, Mz are values [0] - [2].
`
`Sensor . TYPE_MAGNETIC_FIELD_UNCAI_IBRATED:
`
`Similar to TYPE_MAGNETIC_FIELD, but the hard iron calibration is reported separately instead of being
`included in the measurement. Factory calibration and temperature compensation will still be applied to
`the "uncalibrated" measurement. Assumptions that the magnetic field is due to the Earth's poles is
`avoided.
`
`The values array is shown below:
`
`device. Soft iron — These distortions arise due to the interaction with the earth's magentic field.
`
`
`. values|0| = x_uncalib
`. values] = _unca|ib
`
`. values[2]=z_unca|ib
`
`. values[3]=x_bias
`
`. values[4] =y_bias
`
`. values[5] = z_bias
`
`Luncalib, y_unca|ib, z_uncalib are the measured magnetic field in X, Y, Z axes. Soft iron and
`temperature calibrations are applied. But the hard iron calibration is not applied. The values are in
`micro—Tesla (uT).
`
`x_bias, y_bias, z_bias give the iron bias estimated in X, Y, Z axes. Each field is a component of the
`estimated hard iron calibration. The values are in micro-Tesla (uT).
`
`Hard iron - These distortions arise due to the magnetized iron, steel or permanenet magnets on the
`
`Source: http://developer.android.com/reference/android/hardware/SensorEvent.html#values
`
`SUBJECT TO CHANGE
`
`14
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GM6.S.Dammfl2,Wng 16 of 18 PagelD #: 135
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx,
`My, M1 and a plurality of predicted magnetism Mx', My' and Mz' for the second signal set.
`
`The measured magnetisms, z, and a predicted magnetism, Bb, are used to calculate a global variable x0 in quaternion form.
`
`measured magnetisms
`
`update(m, Bm, mParam.magStdev);|
`
`void Fusion::update(const vec3_t& 2, const vec3_t& Bi, float sigma)
`
`{
`
`Source: https://android.googlesource.com/platform/fiameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`const vec3_t Bb(A*Bi);
`
`const vec3_t e(z — Bb)+_______
`
`predicted magnetism
`
`const vec3_t dq(K[0]*e);
`
`q +: getF(q)*(0.5f*dQ);
`
`x0 = normalize_quat(Q);
`
`SUBJECT TO CHANGE
`
`15
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GM8.S.Demfllggz,mddflflng 17 of 18 PagelD #: 136
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx,
`My, M2 and a plurality ofpredicted magnetism Mx', My' and Mz' for the second signal set. .
`
`The global variable x0 is in quaternion form, and can easily be converted to resultant angles.
`
`According to Android’s developer library, the getOrientation 0 function “computes the device’s orientation based on the rotation
`matrix,” and returns deviation angles including the Azimuth, Pitch, and Roll angles.
`
`getOrientation
`
`float[] getOrientation (float[] R,
`float[] values)
`
`Computes the device's orientation based on the rotation matrix.
`
`When it returns, the array values are as follows:
`
`r AP. level 3
`
`
`
`
`0 values[O]: Azimuth, angle of rotation about the —z axis. This value represents the angle between the device‘s y axis and the magnetic north pole, When
`
`facing north, this angle is 0, when facing south, this angle is TI. Likewise, when facing east, this angle is n/2, and when facing west, this angle is —rI/2.
`
`The range of values is -rr to rt.
`
`. values[l]: Pitch, angle of rotation about the x axis. This value represents the angle between a plane parallel to the device's screen and a plane
`
`parallel to the ground. Assuming that the bottom edge of the device faces the user and that the screen is face-up, tilting the top edge of the device
`
`toward the ground creates a positive pitch angle. The range of values is -n to rr.
`
`. values[2]: Roll, angle of rotation about the y axis. This value represents the angle between a plane perpendicular to the device's screen and a plane
`
`perpendicular to the ground. Assuming that the bottom edge of the device faces the user and that the screen is face-up, tilting the left edge of the
`
`device toward the ground creates a positive roll angle. The range of values is 41/210 rt/2.
`
`The getRotationMatrixFromVector () ftmction “convert[s] a rotation vector to a rotation matrix,” and the
`getQuaternionFromVector ()
`function “convert[s] a rotation vector to a normalized quaternion.” Therefore, the quaternion, x0, can
`be easily converted to its mathematically equivalent form, rotation matrix, and used by getOrientation ()
`function to compute the
`orientation in its angular form.
`
`Source: https://developer.android.com/reference/android/hardware/SensorManager.html#get0rientation(float[],%20float[])
`
`SUBJECT TO CHANGE
`
`16
`
`

`

`Claim 10
`
`Case 1:18-cv-00571-GM63Damatfl2mee 18 of 18 PagelD #: 137
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx,
`My, M2 and a plurality ofpredicted magnetism Mx', My’ and M2' for the second signal set. .
`
`
`
`
`‘Rmanonvecmr
`
`getRotationMatrixFromVector
`
`WWW
`
`_
`.
`Underlying physrcal sensors: Accelerometer. Magnetometer, and Gyroscope
`
`void getRotationMatrixFromVector (float[] R,
`float” rotationvfltor)
`
`
`
`Reporting-mode: Continuous
`
`getDefaultSensor(SENSOR_TYPE_ROTATION_VECTOR) returns a non-wakeup sensor
`Source: hflpszllmce_mdroidcmn/devioeslsmsmslsensm4ypes#totafim_vecmr
`
`ge men a IOl'l
`
`float[] getOrientation (float[] R;
`float[] values)
`
`Computes the device's orientation based on the rotation matrix.
`
`When it returns. the array values are as follows:
`
`e -z axrs.
`
`Helper function to convert a rotation vector to a rotation matrix. Given a rotation vector
`
`(presumably from a ROTAT|0N_VECTOR sensor), returns a 9 or 16 element rotation matrix in the
`array R. R must have length 9 or 16. If R.length == 9, the following matrix is returned:
`Source: https://developer.android.com/reference/androidlhardware/
`SensorManager hunl#getRotafionMauimemVector(float[].%20float[])
`
`addedinAPllevelS
`
`facing north. this angle is 0, when facing south. this angle is n. Likewise. when facing east, this angle is n/Z, and when facing west. this angle is «112.
`The range of values is -n to n.
`
`. values[11:Pitch. angle of rotation about the x axis. This value represents the angle between a plane parallel to the device's screen and a plane parallel
`to the ground. Assuming that the bottom edge of the device faces the user and that the screen is face-up. tilting the top edge of the device toward
`the ground creates a positive pitch angle. The range of values is -n to Tt.
`
`. values[2]: Roll. angle of rotation about the y axis. This value represents the angle between a plane perpendicular to the device's screen and a plane
`perpendicular to the ground. Assuming that the bottom edge of the device faces the user and that the screen is face-up, tilting the left edge of the
`'l|.l|'II|l ".IO '0 In-
`I
`[0"
`'
`'I
`ll
`
`Source: hmj/develwaandmidcmn/mfaumdmdtoid/hudwmdSmsmMmager_hunl#gd(himtafim(flmtfl,%20floafl)
`
`SUBJECT TO CHANGE
`
`17
`
`