`
`Technical Presentation:
`Motion Technology and Gaming Applications
`
`September 2010
`
`Page 1
`
`cywee.com
`
`confidential
`
`
`
`Overview
`
` What is motion gaming?
` Motion Gaming Evolution.
` Popular motion gaming solutions.
` How to use SDK for motion game development.
`
`Page 2
`
`cywee.com
`
`confidential
`
`
`
`Motion Gaming Introduction
`
`Page 3
`
`cywee.com
`
`confidential
`
`
`
`Motion Gaming Introduction
`
`<x,y,z>: linear movement trajectory
`
`<p,q,r>: rotation movement
`(Roll, Pitch & Yaw)
`
`These are translated into
`motions for gaming.
`
`Movement : 1:1
`real-time motion
`tracking
`
`“Motion Gaming" captures human body movements (translation +
`
`rotation) through a variety of sensors -- enabling 1:1 game play with
`
`real-time motion response.
`
`Page 4
`
`cywee.com
`
`confidential
`
`
`
`Motion Gaming Types
`
`Non-motion Gaming
`
`Motion Gaming
`
`Use buttons to steer car
`
`Use motion to steer
`
`Motion gaming types:
`
` Racing (3D racing,
`motorcycle racing,
`etc.)
`
` Flying (airplane
`shooting, flight
`simulation, etc.)
`
` Sporting (tennis,
`baseball, bowling,
`table tennis, etc.)
`
` Action (fishing,
`cooking, etc.)
`
` Shooting (first person
`shooting games, etc.)
`
`Use buttons to shoot
`
`Use motion to shoot
`
`Page 5
`
`cywee.com
`
`confidential
`
`
`
`Motion Gaming Evolution
`
`Page 6
`
`cywee.com
`
`confidential
`
`
`
`Motion Sensing Gaming Evolution
`
`2006
`
`Logitech
`Mx air
`
`2009
`
`Apple
`iPhone
`3Gs
`
`2010
`
`CyWee
`Sensor
`Fusion
`
`Inertia sensors (G + Gyro
` Only a mouse function
`
`Inertia sensors (G + M)
`
`Inertia sensors(G+M+Gyro)
` 9-axis sensor fusion
`
`Sony Eyetoy
`
`Cameras
`
`Nintendo Wii
`
`MicroSoft
`Kinect
`
` infrared transmitter/cable
`VGA camera group
`Array Microphone
`
`Sony
`PlayStation
`Move
`
`Nintendo Wii
`
`Motion plus
`
`Inertial sensors
`IR (Infrared ray)
`Launch rod + CMOS Receiver
`
`Inertial sensors (+ Gyro)
`IR (Infrared ray)
`Launch rod +
`CMOS Receiver
`
`Inertia sensors
`Camera
`
`Page 7
`
`cywee.com
`
`confidential
`
`Inertia
`Sensor
`
`Optical
`sensor
`
`Inertia +
`Optical
`Sensor
`
`Other
`
`Motion Sensing Technology
`
`
`
`Motion Sensing Technology Evolution
`Time Evolution
`
`Inertia
`Sensor
`
`3-axis G sensor
`
`3-axis G sensor
`+
`3-axis M sensor
`
`3-axis G sensor
`+
`2-axis Gyro sensor
`
`3-axis G sensor
`+
`3-axis Gyro sensor
`+
`3-axis M sensor
`
`CyWee 9-axis Sensor Fusion
`Algorithm
`
`Optical
`sensor
`
`Inertia +
`Optical
`Sensor
`
`Other
`
`2D Planar images
`Action Recognition
`
`3D Planar images
`Multi-point action recognition
`(Depth accessibility information)
`
`3-axis G sensor
`+
`2D Infrared plane
`positioning
`
`3-axis G sensor
`3-axis Gyro sensor
`+
`2D Infrared plane
`positioning
`
`3–axis G sensor
`3-axis Gyro sensor
`3-axis M sensor
`+
`Camera capture
`
`Page 8
`
`cywee.com
`
`confidential
`
`Motion Sensing Technology
`
`
`
`9-Axis Motion Module on Mobile Phones
`
`9-Axis Key Benefits:
` Does not require fixed
`optical sensor bar, making
`9-axis technology a
`convenient and portable
`solution.
` Motion sensor ensures that
`there are no line of sight
`issues.
` 9-axis sensor fusion
`algorithm enables real-time
`1:1 motion, 360°motion
`detection
`
`Page 9
`
`cywee.com
`
`confidential
`
`
`
`Motion Sensing Technology Comparison
`
`G Sensor only
`
`G Sensor + Gyro
`
`Wii (1st), iPhone
`(2G/3G)
`
`Current Air Mouse,
`Wii motion plus
`
`G Sensor +
`Magnetic Sensor
`Gphone, iPhone
`(3GS)
`
`G Sensor + Magnetic
`Sensor + Gyro
`
`CyWee solution
`
`Delay issue
`
`Temperature drift
`control issue
`
`Delay issue
`
`Force interfusion
`
`Pitch singularity
`
`Accumulated error
`
`Relative position
`issue
`Requires optical bar
`and provides no
`depth information
`
`Magnetic
`disturbance
`
`• Provide 9-axis physical
`outputs with 9-axis input
`solution (3-axis G-sensor, 3-
`axis Gyro and 3-axis
`Magnetic sensor).
`• Real time 1:1 motion
`sensing.
`• Provide absolutely position
`solution.
`•No singularity issue
`•Provides depth information
`
`Page 10
`
`cywee.com
`
`confidential
`
`
`
`Accelerometer Problem
`
`When rotating an accelerometer with angular motion (*r), there exists
`inherent problems
`
`Fixed finger
`
`C1
`
`Anchor
`
`K
`
`Mass
`
`C2
`
`Movable
`finger
`
`D
`
`Damper
`
`Accelerometer Problems:
` Decouple centrifugal force
` Decouple gravity
` Physical singular point
`
`No Acceleration
`C1 = C2
`
`Linear Acceleration
`C1 > C2
`
`x
`
`y
`
`Cent.
`Force
`
`Gravity
`
`z
`*r = radius
`
`Page 11
`
`Rotation Acceleration:
`coupled with
`centrifugal force and
`gravity
`
`cywee.com
`
`confidential
`
`U
`
`w
`
`tq
`
`twq
`
`q
`
`
`
`
`U(
`
`a
`
`x
`
`
`
`
`
`
`
`
`Wq
`
`)
`
`
`
`g
`
`
`
`
`x
`
`
`
`Example: Accelerometer Only
`
`2) Force interfusion issue
`
`Capability:
`
` Can detect 3 axes acceleration
`
` Can detect X and Y axis tilt angle
`
`Shortcomings:
`
`1) Delay issue
`
`2) Force interfusion issue
`
`3) Physical singular point issue
`
`3) Physical singular point issue
`
`1) Delay issue
`
`Page 12
`
`cywee.com
`
`confidential
`
`
`
`MEMS Gyro Working Principle
`
`Key points:
`1. Mass vibrates at resonant frequency
`Wc in up and down direction
`2. Mass rotates at angular speed Ωy
`3. Coriolis force will be generated
`towards left direction (Fc = 2mv x Ωy)
`4. Angular speed Ωy is determined by
`measuring Wheatstone Bridge
`voltage
`
`Drive mode
`
`Roll mode
`
`Yaw mode
`
`Page 13
`
`cywee.com
`
`confidential
`
`
`
`Gyroscope Drifting
`
`Gyroscope drifting mainly results from temperature changes that cause
`electric charges (Q) to change in the system.
`
`Domino Effect:
`
`1. When temperature changes,
`Q changes
`2. Cx1~Cx4 and system intrinsic
`capacitance changes as a
`result of Q
`3. Changes in Cx1~Cx4 cause
`measuring circuit with drift
`signal
`4. Drift signal directly impacts
`gyro outputs
`
`Page 14
`
`cywee.com
`
`confidential
`
`
`
`Gyro Accumulated Error
`
`The sum of digital and analog conversion error, and noise results in
`accumulated error.
`
`Red = Ideal Case Blue = Actual Case Shaded = Accumulated Error
`
`Ideal Case:
`
`Actual Case:
`
`noise
`
`( : drift, noise… )
`
`Page 15
`
`cywee.com
`
`confidential
`
`
`
`
` zyx ,,
`
`
`
`
`
`
`zyx, ,
`
`
`
`dt
`
`v
`
`
`
`
`
` zyx ,,
`
`
`
`a
`
`dt
`
`
`
`
`
` zyx ,,
`
`x
`
`
`
`
`
` zyx ,,
`
`
`
`v
`
`dt
`
`
`
`
`
` zyx ,,
`
`
`
`
` zyx ,,
`
`
`
`
`
`(
`
`
`
` zyx ,,
`
`
`
`
`dt)
`
`
`x
`
`
`
`
`
` zyx ,,
`
`
`
`([
`
`
`a
`
`
`
`
`
` zyx ,,
`
`
`
`)
`
`
`dt
`
`
`
`dt]
`
`
`
`
`
`Example: Gyro Only
`
`Capability:
`
`Shortcomings:
`
` Detect 3 axes dynamic angular speed
`
`1) Temperature drift issue
`
` Quick angular speed response
`
`2) Relative position only
`
`3) Accumulation error issue
`
`1) Temperature drift issue
`
`2) Relative position only & 3) Accumulation error issue
`
`Page 16
`
`cywee.com
`
`confidential
`
`
`
`Example: MR Only
`
`Capability:
`
` Can detect 3 axes magnetic field
`
`Shortcomings:
`
`1) Delay issue
`
`2)
`
`Unpredictable magnetism
`disturbance
`
`1) Delay Issue
`
`2) Unpredictable magnetism disturb.
`
`Page 17
`
`cywee.com
`
`confidential
`
`
`
`CyWee 9-axes Sensor Fusion Solution
`
`Euler Angle
`Calculation
`
`Gravity
`Decouple
`
`Coordinate
`Transformation
`
`Sensor Fusion
`Core
`
`Data
`Association
`
`Entire 3D-location based
`system
`
`Page 18
`
`cywee.com
`
`confidential
`
`BBB
`zyx
`
`
`
`
`
` )..( zyx
`
`t
` . .vvv
`y
`x
`
`
`
`
`
`(
`
`1
`
`
`)
`
`z
`
`t
`
`
`
`1
`
`t
`
`1
`
`
` ..( )
`
`
`
`
`
`t
`
`1
`eeee
`)3.2.1.0(
`
`
`
`CyWee Sensor Fusion Solution
`
`Integrated CyWee Motion Module Solution:
`
` Integrated 3-axis accelerometer, 3-axis gyro, and 3-axis MR
`
` Sensor fusion motion recognition development
`
` Provides 9-axis output (3-axis acceleration, 3-axis rotational rate, and 3-
`axis orientation) and real time 1:1 motion recognition
`
`Page 19
`
`cywee.com
`
`confidential
`
`
`
`Pointing Function Performance Comparison
`
`In laboratory testing, CyWee performed significantly better than Nintendo
`and Sony.
`
`CyWee
`
`(Nintendo)
`
`(Sony)
`
`Sensing Type
`
`Inertial sensing
`
`Optical sensing
`
`Optical + Inertial
`sensing
`
`Response
`Time
`
`67 ms
`(60 FPS camera)
`
`116 ms
`(60 FPS camera)
`
`133 ms
`(60 FPS camera)
`
`Page 20
`
`cywee.com
`
`confidential
`
`
`
`Motion Sensing Applications
`
`Page 21
`
`cywee.com
`
`confidential
`
`
`
`Endorsed by the Four Major Operators
`
`Designated motion sensor provider for JIL gaming
`handsets
`
`CyWee Motion Module
`
`(on-board chips)
`
`Wx, Wy, Wz
`axis gyro
`
`MR Sensor
`
`CyWee’s motion module supports various types
`of games, allows for 9-axis motion recognition,
`and 1:1 motion capture
`
`Key advantages:
`
`- Proprietary algorithms provide accurate motion
`recognition
`
`Processing
`chip
`
`X, Y, Z-axis
`G sensor
`
`- Provides absolute position (industry first)
`
`- Full 3D depth sensing for next generation motion
`gaming
`
`Page 22
`
`cywee.com
`
`confidential
`
`
`
`Relative Position vs. Absolute Position
`
`CyWee: after rotating 360 degrees, the blocks will return to its original location
`
`iPhone 4: Due to accumulated error, the blocks will not return to the original location
`Not capable of supporting advanced gaming experience.
`Absolute position is critical for FPS shooting games
`
`Start
`
`360°rotation
`
`Finish
`
`Page 23
`
`cywee.com
`
`confidential
`
`
`
`The Perfect Motion Gaming Experience
`
`FPS Shooting Games
`- 9 axis output
`- Absolute position
`- 1:1 real-time motion capture
`- No delay
`- Freedom to move in 3D space
`
`Archery Games
`- 9 axis output
`- Absolute position
`- 1:1 real-time motion capture
`- No delay
`- 3D depth information
`
`Sword Games
`- 9 axis output
`- Absolute position
`- 1:1 real-time motion capture
`- No delay
`- 3D depth information
`
`Page 24
`
`cywee.com
`
`confidential
`
`
`
`Application of CyWee SDK and
`Game Development
`
`Page 25
`
`cywee.com
`
`confidential
`
`
`
`CyWee Motion Module on Android
`
`Application
`
`JNI
`
`CyWee Sensor
`Fusion Core
`
`Middleware
`
`Kernel
`
`Sensor
`
`Page 26
`
`cywee.com
`
`confidential
`
`
`
`CyWee 9 Axis Algorithm
`
`9-axis Sensor output – Calibrated Sensor Data
`
`• Three-axis accelerometer: Offset Static Correction
`• Three-axis gyroscope: Baseline Dynamic Correction
`• Three-axis magnetic sensors: Offset Dynamic Correction
`
`Euler angle (Euler angle) - Output real-time, absolute angles Roll (Ф), Yaw (Ψ), Pitch (θ)
`
`z
`
`y
`
`x
`
`Page 27
`
`cywee.com
`
`confidential
`
`
`
`Sensor Fusion Output
`
` (Rotation matrix) – Output Rotation matrix can be directly applied to Open GL
`functions to rotate 3D objects
`
`Y
`
`X
`
`Rotation
`
` (Quaternions) – If (Pitch = 90) there is no singular point.
`Output can be directly applied to Open GL functions to rotate 3D objects
`
`Z
`
`Y
`
`X
`
`<λ1, λ2, λ3>
`
`u
`
`Page 28
`
`cywee.com
`
`confidential
`
`
`ZYX
`
`
`
`
`
`
`
`
`
`
`Rotation
`
`Matrix
`
`
`
`
`
`
`ZYX
`
`
`
`
`
`
`
`
`
`Roll
`
`
`Yaw
`
`
`
`
`Pitch
`
`
`
`
`'
`
`
`
`
`''
`
`ZYX
`
`
`
`
`
`)2/
`sin(
`1
`
`)2/
`sin(
`2
`
`)2/
`sin(
`3
`
`)2/
`cos(
`
`
`
`qqqq
`
`1
`
`2
`
`3
`
`4
`
`
`
`CyWee 9 Axis Algorithm - Sensor Fusion Output
`
`Distribution of Gravity outputs to three axis- gx, gy and gz
`
`g
`
`g value of the change on the axis can be solved without
`the need to use
`Low pass filter, hence get a faster dynamic response
`
`Game developers can call the motion functions located in
`SDK
`gx, gy and gz three output
`
`<Ex>
`
`gx
`
`g
`
`Page 29
`
`cywee.com
`
`confidential
`
`
`
`g
`
`sin
`
`
`
`
`
`
`
` sincos
`
`
`
`
`
` coscos
`
`g
`
`g
`
`
`
`yx
`
`g
`
`g
`
`g
`
`z
`
`
`
`CyWee 9 Axis Algorithm - Sensor Fusion Output
`
`Linear acceleration can be derived from removing gravity and
`centrifugal force
`
`[X, y, z axis of linear acceleration = Accelerometer value - Centrifugal Force –Gravity]
`
`The direction of linear acceleration
`
`Accurately detects every motion
`
`Page 30
`
`cywee.com
`
`confidential
`
`
`
`A New Motion Experience in Gaming
`
`Before: a very basic game using G
`sensor
`
`After Sensor Fusion: 360 ° real-time
`motion detection, absolute position
`
`CyWee Sensor Fusion
`(Real-time, absolute
`position: Roll, yaw, pitch)
`
`Before: using touch screen to throw toilet
`paper
`
`After Sensor Fusion: Calculate
`trajectory of throw using motion
`without touch
`
`Using CyWee Sensor Fusion
`(360 ° angle sensing)
`
`Page 31
`
`cywee.com
`
`confidential
`
`
`
`CyWee Algorithm for Sensor Calibration
`
`• Calibrated sensor raw data
`• Roll, Pitch angle (Yaw = 0)
`• Actions determine the output
`
`G
`
`G + M
`
`G + Gyro
`
`• Calibrated sensor raw data
`• Roll (real time), Pitch (real time),
`Yaw (real-time, relative position)
`• Rotation matrix (Yaw relative)
`• Quaternions output (Yaw relative)
`• Gravity changes
`•
`Linear acceleration
`• Actions determine the output
`
`G +Gyro
`+ M
`
`• Calibrated sensor raw data
`• Roll, Pitch, Yaw (delay)
`• Rotation matrix (Yaw delay)
`• Quaternions output (Yaw delay)
`• Gravity changes (delay)
`•
`Linear acceleration (delay)
`• Actions determine the output
`
`• Calibrated sensor raw data
`• Roll (real time), Pitch (real time),
`Yaw (real time) - absolute
`position
`• Rotation matrix
`• Quaternions output
`• Gravity changes
`• Linear acceleration
`• Actions determine the output
`•
`(360 °motion detection,
`rotation)
`
`Page 32
`
`cywee.com
`
`confidential
`
`
`
`CyWee Motion Database is Turnkey Solution
`
`CyWee
`Motion API
`
`CyWee
`Motion Modeling
`(critical step)
`
`CyWee
`Motion Database
`(world’s largest)
`
`Common Motion API
`Commands:
`
`• Roll, Yaw, Pitch
`• Rotation Matrix
`• Quaternions
`• Gravity
`• Linear Acceleration
`
`CyWee calculates
`thousands of motions:
`z
`
`(x,y,z), (r,,)
`
`
`
`r
`
`z
`
`y
`
`x
`
`
`
`rsin
`
`(x,y,0)
`
`Create Perfectly
`Mapped Motion Games
`
`Page 33
`
`cywee.com
`
`confidential
`
`
`xy
`
`
`
`z
`
`
`
`2
`
`x
`
`2
`
`
`
`y
`
`2
`
`
`
`z
`
`r
`
`
`
`2
`
`x
`
`2
`
`
`
`y
`
`2
`
`
`
`z
`
`
`
`,
`
`
`
`tan
`
`
`
`1
`
`
`
`
`
`1
`
`cos
`
`
`rz
`
`
`
`
`
`
`
`
`
`cos
`
`
`
`1
`
`
`
`Create User Generated Content
`
` Easy to use mapping tool for developers and users
` Turn existing games into motion-sensing games
`
`Take Existing Game
`
`Select Mapping
`Key Requirements
`
`Instantly Mapped
`Motion Game
`
`Page 34
`
`cywee.com
`
`confidential
`
`
`
`New Motion Experience: Game Mapping
`
`Before: Use touch screen to
`control shooter
`
`After Sensor Fusion: Using roll, yaw, pitch
`
`CyWee Sensor Fusion
`(Real-time, absolute position of
`roll, yaw, pitch)
`
`• Pitch angles must be provided in
`order to map vertical movements
`• Provides real-time information and
`absolute position
`• Gamers experience a life-like
`combat simulation
`
`Page 35
`
`cywee.com
`
`confidential
`
`
`
`Map Existing Games to Support Motion
`
`Before: Using touch screen to control FPS
`
`After mapping: target enemies by
`rotating the handset
`
`Mapping of touch screen
`
`Before: Using only touch to throw
`paper balls
`
`After mapping: use trajectory and speed to
`calculate a precise throw with motion
`
`Mapping of touch screen
`
`Page 36
`
`cywee.com
`
`confidential
`
`