Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 1 of 18 PageID #: 120
`
`Case 1:18-cv-00571-GMS Document1-5 Filed 04/16/18 Page 1 of 18 PagelD #: 120
`
`EXHIBIT E
`
`EXHIBIT E
`
` LG 1011
`
`

`

`Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 2 of 18 PageID #: 121
`
`U.S. Patent No. 8,552,978
`
`Google Pixel
`
`SUBJECT TO CHANGE
`
`1
`
`

`

`Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 3 of 18 PageID #: 122
`U.S. Patent No. 8,552,978 – GOOGLE PIXEL
`
`Claim 10
`A method for compensating rotations of a 3D pointing device, comprising:
`
`Google Pixel
`
`SUBJECT TO CHANGE
`
`2
`
`

`

`Case 1:18-cv-00571-GM&s. Daeniman’ 53 sfsle@Oura/EPreape 4 of 18 PagelD #: 123
`
`
`generating an orientation output associated with an orientation of the 3D pointing device associated with three coordinate axes of a
`global reference frame associated with Earth;
`
`
`Whenthe orientation sensoris software-based, the orientation outputis the attitude of the device that can be represented by the
`azimuth, pitch, and roll angles relative to the magnetic North Pole associated with a global reference frame associated with Earth.
`
`Rotation vector
`
`Underlying physical sensors: Accelerometer, Magnetometer, and Gyroscope
`
`Reporting-mode: Continuous
`
`getDefaultSensor (SENSOR_TYPE_ROTATION_VECTOR) returns a non-wake-up sensor
`
`coordinates(x,y,z). Source: https://source.android.com/devices/sensors/sensor-types#rotationvector
`
`A rotation vector sensorreports the orientation of the device relative to the East-North-Up coordinates frame.It is usually
`obtained by integration of accelerometer, gyroscope, and magnetometer readings. The East-North-Up coordinate system
`is defined as a direct orthonormal basis where:
`
`e X points east and is tangential to the ground.
`
`e Y points north andis tangential to the ground.
`
`e Z points towardsthe sky andis perpendicular to the ground.
`
`The orientationofthephonerepresentedbytherotationis necessary to align the East-North-Upcoordinates with the
`
`
`
`
`
`phone's coordinates. That is, applying the rotation to the world frame (X,Y,Z) would align them with the phone
`
`SUBJECT TO CHANGE
`
`

`

`Case 1:18-cv-00571-GMS§s. Dacian’53 oFsle@OUra/ErRepe 5 of 18 PagelD #: 124
`
`
`generating a first signal set comprising axial accelerations associated with movementsandrotations of the 3D pointing device in the
`spatial reference frame;
`
`
`Accelerometer
`
`Reporting-mode: Continuous
`
`getDefaultSensor (SENSOR_TYPE_ACCELEROMETER) returns a non-wake-up sensor
`
` All values are in SI units (m/s*2) and measure the acceleration of the device minusthe force of gravity along the 3
`
`device) that’s used by the SensorAPI. Source: http://developer.android.com/guide/topics/sensors/sensors_overview.html#sensors-coords
`
`An accelerometer sensorreports the acceleration of the device along the 3 sensor axes. The measuredacceleration
`includes both the physical acceleration (changeof velocity) and the gravity. The measurementis reported in the x, y and
`z fields of sensors_event_t.acceleration.
`
`=SEDSOLSXES,
`Source: https://source.android.com/devices/sensors/sensor-types#accelerometer
`
`Sensor Coordinate System
`
`In general, the sensor framework uses a standard 3-axis coordinate system to express data values. For most
`sensors, the coordinate system is defined relative to the device's screen whenthe device is held in its default
`orientation (see figure 1). When a deviceis held in its default orientation, the X axis is horizontal and points to
`the right, the Y axis is vertical and points up, and the Z axis points toward the outside of the screenface.In
`this system, coordinates behind the screen have negative Z values. This coordinate system is used by the
`following sensors:
`
`e Acceleration sensor
`
`e Gravity sensor
`e Gyroscope
`e Linear acceleration sensor
`¢ Geomagnetic field sensor
`
`Figure 1. Coordinate system (relative to a
`
`SUBJECT TO CHANGE
`
`
`
`

`

`Magnetic field sensor
`
`Reporting-mode: Continuous
`
`getDefaultSensor(SENSOR_TYPE_MAGNETIC_FIELD) returns a non-wake-up sensor
`
`SENSOR_TYPE_GEOMAGNETIC_FIELD == SENSOR_TYPE_MAGNETIC_FIELD
`
`A magnetic field sensor (also known as magnetometer) reports the ambient magnetic field, as measured along the 3
`Sensor axes.——
`
`The measurementis reportedin the x, y and z fields of sensors_event_t.magnetic and all values are in micro-Tesla
`(uT).
`
`Case 1:18-cv-00571-GM&s. Dacniman’53oFsle@OuUra/Errepe 6 of 18 PagelD #: 125
`
`generating a second signalset associated with Earth's magnetism;
`
`
`The magnetometer(i.e., the compass) generates a second signal set associated with Earth’s magnetism.
`
`Source: https://source.android.com/devices/sensors/sensor-types#magneticfieldsensor
`
`SUBJECT TO CHANGE
`
`5
`
`

`

`;
`
`The Android source code showsgenerating the orientation output based onthefirst signal set, the second signal set and the
`rotation output.
`The handleGyro() function passes rotation output w to the predict ()
`orientation output, x0.
`
`313
`void Fusion: :handleGyro(const vec3_t& w, float dT)
`{
`314
`if (!checkInitComplete(GYRO, w, dT))
`
`Case 1:18-cv-00571-GM&s. Paeniman’53oFsle@OuUra/Eereepe 7 of 18 PagelD #: 126
`Claim 10
`
`generating the orientation outputbased onthefirst signal set, the second signal set and the rotation output or based onthefirst
`signalset and the secondsignalset;
`
`function and the update ()
`
`functionto calculate an
`
`315
`
`return;
`
`void Fusion: :predict(const vec3_t& w, float dT)
`
`{
`
`= x0;
`const vec4_t q
`
`BBee—C:iCSCSCSC‘CCCNNHRY(UYCH#«é}HNW
`xO = Oxq;
`
`
`
`” void Fusion: :update(const vec3_t& z, const vec3_t& Bi, float sigma)
`
`vec4_t q(x0);
`
`496
`
`{
`
`529
`530
`
`531
`
`const vec3_t e(z - Bb);
`const vec3_t dq(K[0]*e) ;
`
`getF(q)*(0.5fx*dq) ;
`q +=
`x@ = normalize_quat(q);
`
`Source: https://android.googlesource.com/platform/frameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`SUBJECT TO CHANGE
`
`

`

`Case 1:18-cv-00571-GM&s. Dacniman’53 oFsle@OuUra/Errepe 8 of 18 PagelD #: 127
`
`|Claim10_|
`
`generating the orientation output based onthe first signal set, the second signal set and the rotation output or based onthe first
`signal set and the secondsignal set;
`
`
`The handleAcc() function passes the accelerometer measurements(first signal set) a to the update ()
`orientation output x0.
`
`function, which updates the
`
`status_t Fusion: :handleAcc(const vec3_t& a, float dT)
`if (!checkInitComplete(ACC, a, dT))
`
`{
`
`return BAD_VALUE;
`
`x® = normalize_quat(q) ; Source:https://android.googlesource.com/platform/frameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`void Fusion: :update(const vec3_t& z, const vec3_t& Bi, float sigma)
`vec4_t q(x@);
`
`{
`
`const vec3_t e(z - Bb);
`
`const vec3_t dq(K[@]*e);
`
`q += getF(q)*(0.5f*dq) ;
`
`SUBJECT TO CHANGE
`
`7
`
`

`

`Case 1:18-cv-00571-GM&s. Dacniman’53 oFsle@OUra/Errepe 9 of 18 PagelD #: 128
`
`|Claim10_|
`
`generating the orientation output based onthe first signal set, the second signal set and the rotation output or based onthe first
`signal set and the secondsignalset;
`
`
`function passes the magnetometer measurements (second signal set) m to the same update() , which also updates
`The handleMag()
`the orientation outputx0.
`
`
`status_t Fusion: :handleMag(const vec3_t& m)
`{
`if (!checkInitComplete(MAG, m))
`
`return BAD_VALUE;
`
`x® = normalize_quat(q) ; Source:https://android.googlesource.com/platform/frameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`void Fusion: :update(const vec3_t& z, const vec3_t& Bi, float sigma)
`vec4_t q(x@);
`
`{
`
`const vec3_t e(z - Bb);
`
`const vec3_t dq(K[@]*e);
`
`q += getF(q)*(@.5fxdq) ;
`
`SUBJECT TO CHANGE
`
`:
`
`

`

`Case 1:18-cv-00571-GM6.s.Dacauiiisat552 FisedOGUEi8piRage 10 of 18 PagelD #: 129
`
`
`generating a rotation output associated with a rotation of the 3D pointing device associated with three coordinate axesof a spatial
`reference frame associated with the 3D pointing device; and
`
`
`Gyroscope
`
`Reporting-mode: Continuous
`
`getDefaultSensor(SENSOR_TYPE_GYROSCOPE) returns a non-wake-up sensor
`
`A gyroscope sensor reports the rate of rotation of the device around the 3 sensor axes.
`
`Rotationis positive in the counterclockwise direction (right-handrule). That is, an observer looking from somepositive
`location on the x, y or z axis at a device positioned on the origin would report positive rotation if the device appeared to
`be rotating counter clockwise. Note thatthis is the standard mathematical definition of positive rotation and does not
`agree with the aerospacedefinition ofroll.
`
`Source: https://source.android.com/devices/sensors/sensor-types#gyroscope
`
`Sensor Coordinate System
`
`device) that’s used by the SensorAPI. Source: http://developer.android.com/guide/topics/sensors/sensors_overview.html#sensors-coords
`
`In general, the sensor framework uses a standard 3-axis coordinate system to express data values. For most
`sensors,
`:
`.
`:
`.
`-
`.
`i
`i
`. When a deviceis held in its default orientation, the X axis is horizontal and points to
`the right, the Y axis is vertical and points up, and the Z axis points toward the outside of the screenface.In
`this system, coordinates behind the screen have negative Z values. This coordinate system is used by the
`following sensors:
`
`
`
`e Acceleration sensor
`
`e Gravity sensor
`e Gyroscope
`e Linear acceleration sensor
`
`e Geomagnetic field sensor
`
`Figure 1. Coordinate system(relative to a
`
`SUBJECT TO CHANGE
`
`

`

`
`
`Claim 10
`
`Case 1:18-cv-00571-GM6.s.Daguinisat552 FisedOGU6i8piRage 11 of 18 PagelD #: 130
`
`using the orientation output and the rotation output to generate a transformed output associated with a fixed reference frame
`associated with a display device [Court’s construction: using the orientation output and the rotation output to generate a transformed
`output that corresponds to a two-dimensional movementin a planethat is parallel to the screen of a display device],
`
`y
`
`@rare
`BEd Sensor Kinetics }
`Multi-Sensor Recorder
`
`Three Dimensional Sensors
`
`;
`
`The fixed reference frameis defined by the horizontal and vertical axes of pixels on HTC 10 display device.
`
`Figure 1. Coordinate system(relative to a
`device) that's used bythe Sensor API.
`
`Source: http://developer.android.com/guide/topics/sensors/sensors_overview.html#sensors-coords
`
`SUBJECT TO CHANGE
`
`
`
`

`

`Case 1:18-cv-00571-GM6.s.Daguinisat552 PisedOG618piRage 12 of 18 PagelD #: 131
`
`showsa transformed output that corresponds to movements of the Google Pixel via AR and/or VR apps.
`
`6 OX
`
`using the orientation output and the rotation output to generate a transformed outputassociated with a fixed reference frame
`associated with a display device [Court’s construction: using the orientation output and the rotation output to generate a transformed
`
`output that corresponds to a two-dimensional movementin a planethat is parallel to the screen of a display device], Google sells the Google Daydream View, which promotes and induces use of CyWee’s patents. For example, the screen below
`
`Source: https://vr.google.com/daydream/smartphonevr/
`
`SUBJECT TO CHANGE
`
`

`

`The remapCoordinateSystem() function transforms the orientation output (inR) to a transformed output (outR), associated with a
`two dimensional movement in a planethatis parallel to the screen of a display device.
`
`inR,
`
`int X,
`
`int Y, float[]
`
`Case 1:18-cv-00571-GM6.s.Dacauiisat552 FisedO5UEi8piRage 13 of 18 PagelD #: 132
`
`using the orientation output and the rotation output to generate a transformed outputassociated with a fixed reference frameassociated with a
`display device [Court’s construction: using the orientation output and the rotation output to generate a transformed outputthat corresponds to a
`
`two-dimensional movementin a planethat is parallel to the screen of a display device],
`
`becausethe useris generally free to rotate their screen, you often should considertherotation in deciding the parametersto use here.
`
`public static boolean remapCoordinateSystem(float[]
`if (inR == outR)
`f{
`final float[] temp = sTempMatrix;
`synchronized (temp)
`{
`// we don't expect to have a lot of contention
`if (remapCoordinateSystemImpl(inR, X, Y,
`temp))
`final int size = outR.length;
`for (int i = 0;
`4
`< size;
`i++)
`outR[i] = temp[i];
`
`{
`
`{
`
`} r
`
`eturn true;
`
`} r
`
`1
`
`eturn remapCoordinateSystemImpl(inR, X, Y, outR);
`
`Source:https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/hardware/SensorManager.java
`boolean remapCoordinateSystem (float[] inR,
`int X,
`int Y,
`float[] outR)
`
`Rotates the supplied rotation matrix so it is expressed in a different coordinate system. This is typically used when an application needs to
`compute the three orientation angles of the device (see getOrientation(float[], float[{])) ina different coordinate system.
`
`Whenthe rotation matrix is used for drawing (for instance with OpenGL ES), it usually doesn't need to be transformed bythis function, unless
`the screenis physically rotated, in which case you can use Display. getRotation() to retrieve the current rotation of the screen. Note that
`
`
`Source:http://developer.android.com/reference/android/hardware/SensorManager.html
`SUBJECT TO CHANGE
`
`1”
`
`

`

`Case 1:18-cv-00571-GMS Document 1-5 Filed 04/16/18 Page 14 of 18 PageID #: 133
`U.S. Patent No. 8,552,978 – GOOGLE PIXEL
`
`Claim 10
`wherein the orientation output and the rotation output is generated by a nine-axis motion sensor module;
`
`The Google Pixel includes a 3-axis gyroscope, a 3-axis accelerometer, and a 3-axis magnetometer which form a nine-axis motion
`sensor module.
`
`SUBJECT TO CHANGE
`
`13
`
`

`

`Case 1:18-cv-00571-GM6.s.Dacauiiisat552,FisedOGUGEINSpiRage 15 of 18 PagelD #: 134
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx,
`My, Mz anda plurality ofpredicted magnetism Mx’, My’ and Mz’for the secondsignal set.
`
`The measured magnetisms Mx, My, Mz are values[0]-[2].
`
`Sensor. TYPEMAGNETICFIELDUNCALIBRATED.
`
`Similar to ryPE_MAGNETICFIELD, but the hard iron calibration is reported separately instead of being
`included in the measurement. Factory calibration and temperature compensationwill still be applied to
`the “uncalibrated” measurement. Assumptionsthat the magneticfield is due to the Earth's polesis
`avoided.
`The valuesarray is shownbelow:
`
`device. Soft iron - These distortions arise due to the interaction with the earth's magenticfield. Source: http://developer.android.com/reference/android/hardware/SensorEvent.html#values
` Hardiron - These distortions arise due to the magnetized iron, steel or permanenet magnets on the
`
`° values[0] = x_uncalib
`e values|1] = y_uncalib
`e values[2] = z_uncalib
`e values[3] = x_bias
`e values[4] = y_bias
`e values[5] = z_bias
`
`x_uncalib, y_uncalib, z_uncalib are the measured magneticfield in X, Y, Z axes. Soft iron and
`temperature calibrations are applied. But the hard iron calibration is not applied. The valuesare in
`micro-Tesla (uT).
`
`x_bias, y_bias, z_bias give the iron bias estimated in X, Y, Z axes. Each field is a componentof the
`estimatedhard iron calibration. The values are in micro-Tesla (uT).
`
`SUBJECT TO CHANGE
`
`14
`
`

`

`Case 1:18-cv-00571-GM6.s.Dacaiiisats552,FisdOGUGEINSpiRage 16 of 18 PagelD #: 135
`|Claim10_|
`
`The measured magnetisms, z, and a predicted magnetism, Bb, are used to calculate a global variable x0 in quaternion form.
`
`measured magnetisms
`
`update(m, Bm, mParam.magStdev) ; |
`
`void Fusion: :update(const vec3_t& z, const vec3_t& Bi, float sigma)
`
`{
`
`const vec3_t Bb(AxBi);
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx, My, Mz and a plurality of predicted magnetism Mx’, My’ and Mz’for the secondsignalset.
`
`.
`const vec3_t e(z - Bb .
`predicted magnetism
`const vec3_t dq(K[O0]*e) ;
`
`q += getF(q)*(0.5fxdq) ;
`x® = normalize_quat(q) ;
`
`Source: https://android.googlesource.com/platform/frameworks/native/+/master/services/sensorservice/Fusion.cpp
`
`SUBJECT TO CHANGE
`
`

`

`Case 1:18-cv-00571-GM6.s.Dacaiisats552,FIsdOGGEISpiRage 17 of 18 PagelD #: 136
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx,
`My, Mz and a plurality ofpredicted magnetism Mx’, My’ and Mz’for the secondsignal set..
`
`The global variable x0 is in quaternion form, and can easily be converted to resultant angles.
`According to Android’s developerlibrary, the getOrientation() function “computes the device’s orientation based on the rotation
`matrix,” and returns deviation angles including the Azimuth, Pitch, and Roll angles.
`
`getOrientation
`
`Ffloat[] getOrientation (float[] R,
`float[] values)
`
`Computesthe device's orientation based on the rotation matrix.
`
`Whenit returns,the array valuesare as follows:
`
`Addedin API level 3
`
`
`
`
`e values[0]: Azimuth, angle of rotation aboutthe -z axis. This value represents the angle betweenthe device's y axis and the magnetic north pole. When
`facing north, this angle is 0, when facing south,this angle is 11. Likewise, when facing east, this angle is 1/2, and whenfacing west, this angle is -11/2.
`
`The rangeof valuesis -T1 to Tl.
`
`e values[1]: Pitch, angle of rotation about the x axis. This value represents the angle betweena planeparallel to the device's screen and a plane
`parallel to the ground. Assuming that the bottom edge of the device faces the user and that the screenis face-up,tilting the top edge of the device
`toward the ground creates a positive pitch angle. The range of valuesis -1 to T1.
`
`e values[2]: Roll, angle of rotation aboutthe y axis. This value represents the angle between a plane perpendicularto the device's screen and a plane
`perpendicular to the ground. Assuming that the bottom edge of the device faces the user and that the screen is face-up,tilting the left edge of the
`
`device toward the ground createsa positive roll angle. The range of valuesis -11/2 to n/2.
`The getRotationMatrixFromVector() function “convert[s] a rotation vector to a rotation matrix,” and the
`getQuaternionFromVector() function “convert[s] a rotation vector to a normalized quaternion.” Therefore, the quaternion, x0, can
`be easily converted to its mathematically equivalent form, rotation matrix, and used by getOrientation() function to compute the
`orientation in its angular form.
`
`Source:https://developer.android.com/reference/android/hardware/SensorManager.html#getOrientation(float[],%20float|])
`
`SUBJECT TO CHANGE
`
`16
`
`

`

`Case 1:18-cv-00571-GM6.s.Dacauiisat552 FisedOGUGEi8piRage 18 of 18 PagelD #: 137
`
`obtaining one or more resultant deviation including a plurality of deviation angles using a plurality of measured magnetisms Mx,
`
`
`Rotation vector
`
`getRotationMatrixFromVector
`
`added in APlevel9
`
`.
`.
`Underlying physical sensors: Accelerometer, Magnetometer, and Gyroscope
`
`void getRotationMatrixFromVector (float[] R,
`float[] rotationVector)
`
`My, Mz anda plurality ofpredicted magnetism Mx’, My’ and Mz’ for the secondsignal set. .
`
`Reporting-mode: Continuous
`
`getDefaultSensor(SENSOR_TYPE_ROTATIONVECTOR) retums a non-wake-up sensor
`Source: https://source.android.com/devices/sensors/sensor-types#rotation_vector
`
`getUrientation
`
`float[] getOrientation (float[] R,
`float[] values)
`
`Computes the device's orientation based on the rotation matrix.
`
`Whenit returns,the array values are as follows:
`
`Helperfunction to convert a rotation vectorto a rotation matrix. Givena rotation vector
`(presumably from a ROTATION_VECTOR sensor), returns a 9 or 16 elementrotation matrix in the
`array R. R must have length 9 or16. If R.length == 9, the following matrix is returned:
`Source: https://developer.android.com/reference/android/hardware/
`SensorManager html#getRotationMatrixFromVector(float[].%20float[])
`added in API level 3
`
`facing north, this angle is 0, when facing south, this angle is m. Likewise, when facing east, this angle is 1/2, and when facing west, this angle is -m/2.
`The rangeof valuesis -m to n.
`
`* values[1): Pitch, angle of rotation about the x axis. This value represents the angle between a planeparallel to the device's screen and a planeparallel
`to the ground. Assumingthat the bottom edgeof the device faces the user andthatthe screenis face-up,tilting the top edge of the device toward
`the ground creates a positive pitch angle. The rangeof valuesis -m to T.
`
`* values[2]: Roll, angle of rotation about the y axis. This value represents the angle between a plane perpendicular to the device's screen and a plane
`perpendicular to the ground. Assumingthat the bottom edge of the device faces the user andthat the screen is face-up,tilting the left edge of the
`ice
`toward
`the ground
`create
`positive roll
`a
`
`Source: https://developer.android.c
`
`SUBJECT TO CHANGE
`
`7
`
`