US007450002B2
`
`a2) United States Patent
`US 7,450,002 B2
`(0) Patent No.:
`Nov.11, 2008
`(45) Date of Patent:
`Choiet al.
`
`(54) METHOD AND APPARATUS FOR
`MONITORING HUMAN ACTIVITY PATTERN
`
`(75)
`
`Inventors: Ji-hyun Choi, Seoul (KR); Kun-soo
`Shin, Seongnam-si (KR); Jin-sang
`Hwang, Suwon-si (KR); Hyun-tai
`Hwang, Yongin-si (KR); Wan-taek
`Han, Hwasgong-si (KR)
`
`(73) Assignee: Samsung Electronics Co., Ltd.,
`Suwon-si (KR)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 284 days.
`
`(21) Appl. No.: 11/332,586
`
`(22)
`
`Filed:
`
`Jan. 17, 2006
`
`(65)
`
`(30)
`
`Prior Publication Data
`
`US 2006/0161079 Al
`
`Jul. 20, 2006
`
`Foreign Application Priority Data
`
`Jan. 14,2005)
`
`(KR) eee 10-2005-0003635
`
`(51)
`
`Int. Cl.
`(2006.01)
`GOSB 1/08
`(2006.01)
`H04Q 7/00
`(2006.01)
`AGIB 5/103
`(2006.01)
`AGIB 5/117
`(52) U.S.C cc ceeeneeeeeee 340/539.11; 340/573.1;
`340/573.4; 340/686.1; 600/595
`
`(58) Field of Classification Search............ 340/539.11,
`340/573.1, 539.12, 539.13, 539.15, 539.26,
`340/573.4, 686.1, 689; 600/587-595
`See application file for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`........0.... 73/495
`4/2006 Shiratori et al.
`7,028,547 B2*
`7,127,370 B2* 10/2006 Kellyetal. 0.000... 702/151
`
`.........00..0. 600/595
`2006/0052727 Al*
`3/2006 Palestrant
`7/2006 Choietal. 0... 600/595
`2006/0161079 Al*
`2006/0255955 Al* 11/2006 O’Connoretal. ........ 340/573.1
`
`* cited by examiner
`
`Primary Examiner—Toan N Pham
`(74) Attorney, Agent, or Firm—Sughrue Mion, PLLC
`
`(57)
`
`ABSTRACT
`
`A method and apparatus for monitoring a humanactivity
`pattern irrespective of the wearing position of the sensor unit
`by a user and a direction of the sensor unit are provided. The
`method for monitoring an inertia movementsignal according
`to a movementof a user using a sensor unit attached to the
`user; detecting a direction of the sensor unit from the inertia
`movement signal; detecting a wearing location of the sensor
`unit by using acceleration and direction; determining the
`activity pattern ofthe user from inertia sensors; and delivering
`physical activity data corresponding to at least one caloric
`consumption, numberof steps, and movementdistance.
`
`17 Claims, 8 Drawing Sheets
`
`
`
`
`
`
`DETECT ACCELERATION
`
`DETECT WEARING LOCATION
`
`DETERMINE WEARING MODE
`
`ALYZE WEARING MODE
`
`23
`
`
`
`
`
`
`NOTIFY USER
`
`
`
`Page | of 14
`
`SAMSUNG EXHIBIT1013
`
`Page 1 of 14
`
`SAMSUNG EXHIBIT 1013
`
`

`

`U.S. Patent
`
`Nov.11, 2008
`
`Sheet 1 of 8
`
`US 7,450,002 B2
`
`1
`
`ACCELERATION
`SENSOR
`
`
`
`INTERFACE
`DATA PROCESSING
`UNIT
`UNIT
`
`
`
`TERRESTRIAL
`MAGNETISM
`
`SENSOR
`MOBILE
`
`TERMINAL
`
`
`FIG.
`
`FIG. 2
`
`DETECT ACCELERATION
`
`DETECT WEARING LOCATION
`
`
`
`DETERMINE WEARING MODE
`
`
`
`
`
`
`
`
`
`
`Page 2 of 14
`
`: ANALYZEWEARINGMODE
`
`
`
`
`
`NOTIFY USER
`
`
`
`
`
`Page 2 of 14
`
`

`

`U.S. Patent
`
`Nov.11, 2008
`
`Sheet 2 of 8
`
`US 7,450,002 B2
`
`FIG. 3
`
`DETECT YAW ANGLE FROM
`TERRESTRIAL SENSOR
`
`DETECT PITCH ANGLE AND
`ROLL ANGLE FROM DC COMPONENT
`OF ACCELERATION SENSOR
`
`PITCH ANGLE, AND ROLL ANGLE
`
`30
`
`31
`
`CALCULATE ROTATIONAL TRANSFORM
`
`MATRIX WITH RESPECT TO YAW ANGLE,|—32
`
`TRANSFORM COORDINATES BY USING
`ROTATIONAL TRANSFORM MATRIX
`
`33
`
`Page 3 of 14
`
`Page 3 of 14
`
`

`

`US 7,450,002 B2
`
`Sheet 3 of 8
`
`Nov.11, 2008
`
`FIG. 4A
`
`U.S. Patent
`
`Page 4 of 14
`
`Page 4 of 14
`
`

`

`U.S. Patent
`
`Nov.11, 2008
`
`Sheet 4 of 8
`
`US 7,450,002 B2
`
`FIG. 6A
`
`2
`
`lwi ©|----------------}----------------
`a’ of- -l wr lyws sin (21we/w4)
`
`aY g[--------------4--------------
`
`—lwit
`—lwi
`
`ax
`
`lw
`
`FIG. 6B
`
`lyw? + lyw5sin (21wo /w4)
`
`\
`
`-Lyw? - lyw3cos (2nwe /w4)
`
`0
`
`ax
`
`2
`2
`Lywi -lw2 cos (mwe/w)
`
`Page 5 of 14
`
`Page 5 of 14
`
`

`

`U.S. Patent
`
`Nov.11, 2008
`
`Sheet 5 of 8
`
`US 7,450,002 B2
`
`FIG. 7A
`
`4
`
`2,5
`(
`2 2
`| ie
`
`0
`
`0
`
`0.5
`
`1
`
`INTENSITY
`
`FIG. 7B
`
`
`
`FREQUENCY(Hz)
`
`INTENSITY
`
`Page 6 of 14
`
`Page 6 of 14
`
`

`

`U.S. Patent
`
`Nov.11, 2008
`
`Sheet 6 of 8
`
`US 7,450,002 B2
`
`FIG. 8A
`
`0.9
`
`0.8
`0.7
`0.6
`0.5
`
`0.4
`
`0.3
`
`V(km/h)
`
`0.2
`
`E
`0 a pataM
`
`uN
`go et 82
`f
`.
`
`a
`
`sal
`
`_ stil
`
`WALKING
`
`RUNNING
`
`0
`
`5
`
`10
`
`15
`
`20
`
`25
`
`°&30
`
`TIME (SECOND)
`
`FIG. 8B
`
`
`
`
`
`CALORICCONSUMPTION
`
`
`
` seeeeneheeemene
`
`veel
`
`AMOUNT OF PHYSICAL ACTIVITY
`
`Page 7 of 14
`
`Page 7 of 14
`
`

`

`U.S. Patent
`
`Nov.11, 2008
`
`Sheet 7 of 8
`
`US 7,450,002 B2
`
`FIG. 9A
`
` 43
`TIME (SECOND)
` |
`
`
`42
`
`Page 8 of 14
`
`TIME (SECOND)
`
`FIG. 9B
`
`
`
`Page 8 of 14
`
`

`

`U.S. Patent
`
`Nov.11, 2008
`
`Sheet 8 of 8
`
`US 7,450,002 B2
`
`FIG. 10
`
`p(n/&)
`
`Page 9 of 14
`
`Page 9 of 14
`
`

`

`US 7,450,002 B2
`
`1
`METHOD AND APPARATUS FOR
`MONITORING HUMAN ACTIVITY PATTERN
`
`CROSS-REFERENCE TO RELATED PATENT
`APPLICATIONS
`
`This application claimsthe benefit of Korean Patent Appli-
`cation No. 10-2005-0003635, filed on Jan. 14, 2005, in the
`Korean Intellectual Property Office, the disclosure ofwhich is
`incorporated hereinin its entirety by reference.
`
`10
`
`BACKGROUND OF THE INVENTION
`
`20
`
`40
`
`45
`
`50
`
`1. Field of the Invention
`
`Thepresent invention relates to a method and apparatus for
`monitoring human activity, and more particularly,
`to a
`method and apparatus for monitoring a human activity pattern
`to provide information on the amountofphysicalactivity ofa
`user by monitoring the caloric consumption ofthe user during
`daily activities.
`2. Description of the Related Art
`In order to maintain the healthy life of an individual, there
`is aneed to continuously measure the amountofdaily activity
`and caloric consumption withoutlimiting the daily activities.
`Among the technologies for monitoring the amount of
`daily activity are those disclosed in WO 96-30080 and U.S.
`Pat. No. 6,165,143. These patents disclose technologies for
`finding the activity pattern of an individual by using a variety
`of sensors, and measuring the amount of physical activity.
`However, these conventional technologies have restrictions
`such thatin orderto findthe activity pattern of an individual,
`the direction and the location of a sensor mustbe fixed.
`
`For example, in the WO 96-30080,a sensoris implanted in
`the heart, and the direction and location of the sensor are
`required to be fixed, and in U.S. Pat. No. 6,165,143 sensors
`are required to be attachedat the waist, the upperleg, and the
`frontal points of knee joints.
`
`SUMMARYOF THE INVENTION
`
`The present invention provides a method and apparatus for
`monitoring a humanactivity pattern in which by using a
`3-axis acceleration sensor and a terrestrial magnetism sensor,
`movementin the direction of gravity and movement in the
`horizontal direction by a user are separated. Further, by using
`the signal characteristics with respect to the locations of the
`sensor, the attached locations of the sensor can be recognized
`regardless of the directions of the sensor, and the activity
`pattern of the user can be determined.
`According to an aspect of the present invention, there is
`provided a method for monitoring a humanactivity pattern
`including: sensing an inertia movementsignal according to a
`movementof a user using a sensor unit attachedto the user;
`detecting a direction of the sensor unit from acceleration; by
`using the inertia movementsignal and direction, detecting a
`wearing location ofthe sensor unit; and determining theactiv-
`ity pattern of the user from the inertia movement signal by
`reflecting the wearing location.
`According to another aspect of the present invention, there
`is provided an apparatus for monitoring a human activity
`pattern including: a sensor unit attached to a user, which
`senses an inertia movement signal according to a movement
`of the user; and a data processing unit which detects an
`acceleration signal and a direction signal ofthe sensor unit by
`using the inertia movementsignal, detects a wearing location
`of the sensor unit by using the inertia movement signal and
`
`2
`the direction, and determines the activity pattern of the user
`from the inertia movement signal by reflecting the wearing
`location.
`
`Accordingto still another aspect of the present invention,
`there is provided a computer readable recording medium
`having embodied thereon a computer program for executing
`the method for monitoring an activity pattern.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The aboveand other features and advantagesof the present
`invention will become more apparent by describing in detail
`exemplary embodiments thereof with reference to the
`attached drawings in which:
`FIG. 1 is a block diagram of an apparatus for monitoring a
`human activity pattern according to an exemplary embodi-
`mentof the present invention;
`FIG. 2 is a flowchart of the operations performed by a
`method for monitoring a humanactivity pattern according to
`an exemplary embodimentofthe present invention;
`FIG. 3 is a detailed flowchart illustrating a process for
`detecting a direction;
`FIG. 4A illustrates a principle of measuring a yaw angle
`using a terrestrial magnetism sensor;
`FIG. 4B illustrates a principle of obtaining a pitch angle
`and a roll angle using a DC componentofacceleration;
`FIG.5 illustrates a process for modeling a sensor attached
`to the body as a pendulum;
`FIG. 6A isa phase diagram for the gravity direction and the
`horizontal direction components of acceleration, which is
`symmetric aboutthe axis of the gravity direction component;
`FIG.6B is a phase diagram forthe gravity direction and the
`horizontal direction components of acceleration, which is
`symmetric aboutthe axis of the gravity direction component;
`FIGS. 7A and 7B illustrate frequency distributions of the
`gravity direction and the horizontal direction components of
`an acceleration signal, respectively, with respect to intensity
`according to the pattern ofactivity;
`FIG.8A illustrates values output from an acceleration sen-
`sor when speed increases over time;
`FIG.
`8Billustrates the amount of caloric consumption
`measured with respect to an amountofphysicalactivity for 24
`individual users; and
`FIGS. 9A and 9Billustrate acceleration componentsin the
`gravity direction when a user movesat speeds of3.0 km/h and
`8.5 km/h, respectively.
`FIG.10 illustrates an example of conditional probabilities
`of activity pattern.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`The present invention will now be described more fully
`with reference to the accompanying drawings,
`in which
`exemplary embodiments of the invention are shown.
`Referring to FIG. 1, an apparatus for monitoring a human
`activity pattern includes a sensorunit 10 and a data processing
`umt 11.
`
`Also, the apparatus for monitoring a human activity pattern
`mayfurther include an interface unit 12 for providing results
`processed in the data processing unit to a user, or receiving
`required inputs from the user, and a mobile terminal 13 which
`operates in the same mannerasthe interface unit 12 does, but
`is wirelessly connected. In addition, according to another
`exemplary embodiment,
`the apparatus for monitoring a
`human activity pattern may be implemented as a separate
`apparatus, or may be embeddedin the mobile terminal 13. In
`
`Page 10 of 14
`
`Page 10 of 14
`
`

`

`US 7,450,002 B2
`
`3
`the latter case, the interface unit 12 can be implemented as a
`display panel or a keypad located on the mobile terminal 13.
`The mobile terminal, may be capable of wireless commu-
`nication, and may be a personal digital assistant (PDA), a
`portable computer, or a mobile phone. The mobile terminal
`may communicate in a short range wireless communication
`schemesuch as Bluetooth, and/or may communicate through
`a cable such as a USBport or RS232C.
`The sensor unit 10 includes a 3-axis acceleration sensor
`101 for measuring an inertia movement, more specifically,
`acceleration in x, y, and z directions, and may further include
`a terrestrial magnetism sensor 102 or gyroscope (not shown)
`for detecting the orientation ofthe sensor unit 10 with respect
`to a planarsurfaceparallel to the sensor unit 10. Also, in order
`to sense the orientation of the sensor unit 10 with respect to
`the planar surface, a tilt sensor for measuring the tilt from a
`reference vertical axis may further be included.
`The data processing unit 11 processes an acceleration sig-
`nal output from the acceleration sensor 101 to measure an
`acceleration value in relation to vibration in the 3-axis direc-
`tions or an external acceleration value such as gravity, and
`processesthe direction signal output from the terrestrial mag-
`netism sensor 102. While detailed descriptions pertaining to
`the signal processedby the data processing unit 11 are limited
`to the acceleration signal, one of ordinary skill will appreciate
`that
`in other exemplary embodiments the signal can be
`extended more broadly to the inertia movementsignal.
`Also, the data processing unit 11 transforms the measured
`acceleration values and direction signal in the body frame of
`the sensor unit 10 into the ones ofthe space fixed coordinates.
`FIG. 2 is a flowchart of the operations performed by a
`method for monitoring a humanactivity pattern according to
`the present invention.
`First, the data processing unit 11 detects the direction ofthe
`sensor unit 10 from the DC componentof the acceleration
`sensor 101, and after compensating the acceleration ofthe AC
`component output from the acceleration sensor 101 for the
`direction ofthe sensor unit 10, outputs the compensated result
`in operation 24.
`FIG. 3 is a flowchart of the operations performed in an
`exemplary process for detecting direction.
`First, a yaw angle W is detected by using the terrestrial
`magnetism sensor 102 in operation 30. The yaw angle is not
`necessarily needed for detecting the direction of the sensor
`unit 10, but is a useful component. FIG. 4A illustrates the
`principle of measuring a yaw angle by using theterrestrial
`magnetism sensor 102. Referring to FIG. 4A, whentheter-
`restrial magnetism sensor 102 is tilted with respect to the
`planar surface 40, the yaw angle w indicates the angle that the
`terrestrial magnetism sensor 102 sweeps the planar surface 40
`from the reference line 41 indicating the E-direction of the
`planar surface 40. When § denotes the gravity acceleration
`>
`and z denotes the vector of the orientation of the terrestrial
`magnetism sensor 102, a vector Zz, obtained by projecting Z
`onto the planar surface 40 and the yaw angle w can be
`obtained through the following equation 1:
`
`4
`Thepitch angle 0 andtheroll angle ® can be obtained from
`the DC componentof an acceleration signal output from the
`acceleration sensor 101 orthetilt sensor in operation 31. FIG.
`4Billustratesthe principle of obtaining a pitch angle and a roll
`angle by using the DC componentofacceleration.
`Referring to FIG. 4B, the pitch angle @ indicates an angle
`from the reference line 42 to the Y-axis of the acceleration
`
`sensor 101, and the roll angle ® indicates an angle from the
`reference line 41 to the X-axis of the acceleration sensor 101.
`
`The pitch angle 0 and roll angle ® can be obtained by the
`following equation 2:
`
`x
`cos(= -¢) =
`°
`
` 2.3
`xg
`
`Vex
`
`cos(= - 8) = v8
`
`(2)
`
`If the yaw angle, pitch angle, and roll angle are obtained as
`shown in FIGS. 4A and4B, arotational transform matrix with
`respect to the yaw angle, pitch angle, and roll angle is
`obtained in operation 32. The rotational transform matrix
`maybe obtained with respect to only the pitch angle androll
`angle, or with respect to the yaw angle, pitch angle, androll
`angle. The rotational transform matrix is multiplied by the AC
`componentof the acceleration value output from the accel-
`eration sensor 101. Thus, acceleration components in x,y, and
`z direction in the body frame of the acceleration sensor 101
`are transformed into acceleration values in the space fixed
`coordinates in operation 33. Consequently, the acceleration
`values output from the data processing unit 11 are compen-
`sated for the direction of the sensor unit 10 to be output. At
`this time, if the acceleration is compensated for by using the
`rotational transform matrix containing the yaw angle, pitch
`angle, and roll angle, more accurate compensation can be
`performed than when using the rotational transform matrix
`containing only the pitch angle androll angle,
`Usingthe acceleration value in the spacefixed coordinates,
`the wearing location of the sensor unit 10 is detected in
`operation 21. The wearing location can be detected by a
`kinematics approach to human walking and pendulum mod-
`eling.
`The kinematics approach focuses on the fact that when a
`person moves, a trajectory of a signal output from the sensor
`unit 10 varies depending on the wearing location. The pen-
`dulum modeling regards the sensoras attached to the human
`body as a pendulum, and models the movementtrace of the
`sensor as shown in FIG.5, to determine the characteristics of
`a signal which differ depending on the wearing location. That
`is, whenthe waist or the body is regarded as a fixed point, and
`the sensor unit 10 is located on the arm, hand,or leg, or ina
`pocket or handbag, the movement of the sensor unit 10 is
`modeledas a single or as a double pendulum movement.
`Referring to FIG. 5, reference number 50 indicates the
`bodyofthe fixed point, and reference number51 indicates the
`sensor unit 10 modeled as the single pendulum when the
`sensor unit 10 is held in the hand or the pocket. Reference
`number52 indicatesthe sensor unit 10 modeled as the second
`
`40
`
`55
`
`60
`
`zy =2- (2-28
`
`()
`
`pendulum connected to the first pendulum while the arm is
`modeledas the first pendulum whenthe sensor unit 10 is put
`fy
`cos(W) =
`in the handbag.
`Va,ayZH Si
`65
`If (X,, y,) denotes the location ofthe first pendulum 51, and
`(X>, Y>) denotes the location of the second pendulum 52, then
`whenthe user moves, it can be regardedthatthe fixed point 50
`
`Page 11 of 14
`
`Page 11 of 14
`
`

`

`US 7,450,002 B2
`
`5
`moveshorizontally at a speed ofv. At this time, the location of
`each pendulum 51 and 52 can be obtained by the following
`equation 3:
`x,=vi+l, sin 0,
`
`y=-1, cos 0,
`
`X2=vit+l, sin 0,+/> sin 05
`
`y2=-l, cos 8,-b cos 05
`
`10
`
`(3)
`
`Here, 1, denotes the distance betweenthefixed point 50 and
`the first pendulum 51, and 1, denotes the distance between the
`first pendulum 51 and the second pendulum 52.
`When the movement trajectory is modeled as a single
`pendulum, the acceleration signals in the gravity and hori-
`zontal directions show a phase diagram in the form ofa circle
`as shownin FIG. 6A.At this time, with respectto the radius
`hp of the circle, it is determined whetherthe sensorunit 10 is
`on the arm orleg, or in the pocket. That is, by referring to the
`distance betweenthe fixed point 50 to the first pendulum 51 as
`the distance from the reference point of the bodyto the wear-
`ing location of the sensor unit 10, the location of the sensor
`unit 10 is determined from the radiusof the circle.
`
`Atthis time, the data processing unit can store in advance
`the distances, input through the interface unit 12, from the
`reference point to all the wearing locations at which the
`sensor unit 10 can be located, such as the arm, leg, pocket, and
`hand.
`
`6
`FIGS. 7A and 7B illustrate the frequency distribution ofthe
`components of an acceleration signal in the gravity direction
`and the horizontal direction, respectively, with respect to
`signal intensity according to the pattern of activity. In a case
`wherein the movementspeed of a leg is measured, the sensor
`unit 10 is attached on the thigh. Referring to FIGS. 7A and 7B,
`each unique frequency and intensity area is divided in the
`gravity direction and horizontal direction, in FIGS. 7A and
`7B, respectively, for each type of activity. Similar distribution
`plots or phase diagrams can be made byreplacing the inten-
`sity axis with other dynamic parameters such as mean,
`median, peak, standard deviation, skew, or kurtosis of accel-
`eration for each direction, and a correlation coefficient
`between each pair of accelerations can be usedto classify the
`physical activity more specifically.
`In case there is overlap ofmore than twoactivities for given
`dynamic parameters, a sum of conditional probabilities of
`dynamic parameters given that an activity occurs will deter-
`minethe activity pattern such as
`
`y= » pan l€ppe)
`i
`wherein,
`
`= » pin| €))
`i
`
`ni: pattern (e.g, rest, walk, jog, run),
`
`&;: dynamic parameter(e.g, 7, etc),
`
`4)
`
`35
`
`45
`
`According to equation (4), the activity can be classified by
`When the movement trajectory is modeled as a double
`finding the maximum %,.
`pendulum, the acceleration signals in the gravity and hori-
`zontal directions show a phase diagram that is asymmetrical
`FIG.10 illustrates an example of conditional probabilities
`with respect to the acceleration axis for the gravity direction
`of activity pattern. In FIG. 10, the horizontal axis is of stan-
`dard deviation of €. As shownin FIG.10, each ofthe activity
`component as shown in FIG. 6B.
`patterns is distinguished from each other.
`Accordingly, from the phase diagram ofthe acceleration in
`Theactivity pattern andits duration can be providedto the
`the gravity direction and the horizontal direction, it can be
`user through the mobile terminal 13. Thus, the user can learn
`determined whether the movementtrajectory is modeled as a
`which activity pattern was performed, when it was_per-
`single pendulum or a double pendulum, and the wearing
`40
`location of the sensor unit 10 can be also determined. Thatis,
`formed, and for how long.
`it can be determinedto whichpart, such as the handorleg, the
`If the activity pattern is walking or running,the data pro-
`sensor unit 10 is attached, or whether the sensor unit 10 is
`cessing unit 11 again detects the current wearing location of
`carried in a handbag apart from the human body.
`the sensor unit 10. This is to determine whether the wearing
`location of the sensor unit 10 is changed during the activity.
`Ifthe wearing location ofthe sensor unit 10 is detected, the
`For example, if the location of the sensor unit 10 is changed
`wearing mode is determinedat that location by using accel-
`from the user’s handin the pocket,the acceleration anddirec-
`eration values on the space fixed coordinates in operation 22
`tion detected by the sensor unit 10 also change, and therefore
`(FIG.2). Here, the wearing modeindicates an activity pattern
`the operation 21 is performed again to detect the wearing
`such as walking, running or cycling. The determination is
`location.
`madebyreferring to the frequency andintensity of the accel-
`eration signal with respect to the wearing location. This is
`because even though activity patterns may be identical, the
`acceleration signals vary according to the wearing location of
`the sensor unit 10. Thatis, the detected acceleration signals of
`the sensor unit 10 held in the hand and put in the pocket may
`be different. Also, preferably, the data processing unit stores
`acceleration ranges for each activity pattern with respect to
`the wearing location in order to determine the wearing mode.
`Whenthe activity being performed, or wearing mode, is
`determined, the presence or absence of periodicity in an
`acceleration signal is determined. Periodicity is determined
`because the signal of walking, running, or cycling shows
`periodicity in the gravity direction or in the horizontal direc-
`tion according to the wearing location of the sensor unit 10.
`Morespecifically, the determination of the activity pattern
`can be performed by calculating the dynamic parameters of
`the gravity direction component and horizontal direction
`componentof an acceleration signal.
`
`If the activity pattern is determined, the analysis of the
`determinedactivity pattern can be performedin operation 23.
`The analysis of the activity pattern includes calculation of
`calories consumed by the activity pattern, the number of
`steps, and the moving distance. In addition, if the gravity
`direction component of the acceleration value sharply
`changes while the changein the horizontal direction compo-
`nentis negligible, it is determinedthat the user hasfallen over,
`and an alarm can be sent through the mobile terminal 13. Ifit
`is determined from personal information that the user is
`advancedin age, an emergencycenter can be informedof the
`fall by the mobile terminal 13.
`As an example of the analysis of the activity pattern, the
`process for measuring the consumedcalories will now be
`explained in more detail.
`FIG. 8A illustrates values output from the acceleration
`sensor 101 when speed is increased over time. Reference
`number80 indicates the speed gradually increasing over time,
`
`Page 12 of 14
`
`Page 12 of 14
`
`

`

`US 7,450,002 B2
`
`8
`The present invention can also be embodied as computer
`readable code on a computer readable recording medium. The
`computer readable recording medium is any data storage
`device that can store data which can be thereafter read by a
`computer system. Examples ofthe computer readable record-
`ing medium include read-only memory (ROM), random-ac-
`cess memory (RAM), CD-ROMs, magnetic tapes, floppy
`disks, optical data storage devices, and carrier waves (such as
`data transmission through the internet). The computer read-
`able recording medium canalso be distributed over network
`coupled computer systemsso that the computer readable code
`is stored and executed in a distributed fashion. Also, func-
`tional programs, code, and code segments for accomplishing
`the present invention can be easily construed by programmers
`skilled in the art to which the present invention pertains.
`According to the present invention, by detecting the loca-
`tion on which the sensor is attached, and monitoring the
`activity pattern of the user with reference to the detected
`location, the activity pattern of the user can be monitored
`withoutlimiting the wearing location of the sensor.
`Also, by measuring the activity pattern, the elapsed time,
`the caloric consumption of the activity, the numberof steps,
`or the moving distance, information on the amountof physi-
`cal activity can be providedto the user.
`25
`VM=|> fade
`Furthermore, if the user falls over, then if necessary, the
`Vf ie
`mobile terminal can notify an emergency center.
`While the present invention has been particularly shown
`and described with reference to exemplary embodiments
`thereof, it will be understood by those of ordinary skill in the
`art that various changes in form and details may be made
`therein without departing from the spirit and scope of the
`present invention as defined by the following claims. The
`exemplary embodiments should be considered in a descrip-
`tive sense only andnotfor purposesoflimitation. Therefore,
`the scope of the invention is defined not by the detailed
`description of the invention but by the appended claims, and
`all differences within their scope will be construed as being
`includedin the present invention.
`
`10
`
`20
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Whatis claimedis:
`
`1.A method for monitoring an activity pattern comprising:
`sensing an inertia movementsignal according to a move-
`mentof a user using a sensor unit attachedto the user;
`detecting a direction of the sensor unit from the inertia
`movementsignal;
`detecting a wearing location ofthe sensor unit, by using the
`inertia movementsignal and the direction; and
`determining the activity pattern of the user from the inertia
`movementsignal by reflecting the wearing location.
`2. The method of claim 1, wherein the direction is detected
`from a DC componentof the inertia movement signal by
`obtaining a roll angle anda pitch angle ofthe sensor unit with
`respect to a planar surface of the sensor unit.
`3. The method of claim 2, wherein the inertia movement
`signal is compensatedfor the direction by multiplying an AC
`component of the inertia movement signal by a rotational
`transform matrix defined as the pitch angle andthe roll angle.
`4. The method of claim 2, wherein the sensor unit com-
`prises a tilt sensor and the roll angle and the pitch angle are
`detected by thetilt sensor.
`5. The method of claim 2, wherein the sensor unit com-
`prises a terrestrial magnetism sensor, and the direction is
`detected by further obtaining a yaw angle sweptby theter-
`restrial magnetism sensor from a reference line on a planar
`surface.
`
`7
`and reference numbers 81 and 82 show acceleration sensed by
`different acceleration sensors. Referring to FIG. 8A,
`the
`speed of 0.7 km/h or moreis regardedas that of running, and
`it can be seen that the values output from the acceleration
`sensor 101 change abruptly from those output whenthe speed
`is 0.6 km/h.FIG.8Billustrates the caloric consumption of 24
`users, measured with respect to the amountof physical activ-
`ity. Referring to FIG. 8B,
`it can be seen that the caloric
`consumption amount for walking is clearly distinguished
`from that for running. Also, it can be seen that even in the area
`for running or walking, the measured amounts have a wide
`distribution. This distribution occurs because the physical
`condition of users varies. Accordingly,
`in an exemplary
`embodimentof the present invention, the consumed calories
`are measured with reference to the personal information of
`the user. The personal information includesat least one of the
`sex, age, height, and weightofthe user. The caloric consump-
`tion hasa linear relation with respect to the amountof physi-
`cal activity measured by the acceleration sensor 101, as
`described by the following equation 4:
`
`Calorie=bxVM +c
`
`(5)
`
`Here, b and c are constants anda, is an acceleration signal.
`In the equation 4, constants b and c are determined accord-
`ing to the personal information of an individual, and in the
`present invention, are obtained by applying a known multiple
`regression analysis method.
`As another example of the activity mode, measuring the
`numberof steps will now be explained. Generally, the number
`of steps is measured by counting the number oftimes the
`gravity direction component of the acceleration exceeds a
`certain value. The numberof steps is inclined to be over-
`counted when the user walks fast while inclined to be under-
`
`counted whenthe use walks slowly. Also, shock noises such
`as random shocks can be measuredincorrectly as steps.
`Accordingly,
`in the present exemplary embodiment,
`accordingto the changing rangeofthe gravity direction com-
`ponentofthe acceleration, the measuring time and threshold
`value are adjusted, and after measuring the steps, a locking
`periodis set so that the shock noises are not measured.
`FIGS. 9A and 9Billustrate the gravity direction compo-
`nents of the acceleration when a user movesat speeds of 3.0
`km/h and 8.5 km/h, respectively. Reference numbers 90 and
`92 each indicate a time for beginning to count the numberof
`steps in FIGS. 9A and 9B, respectively. While reference num-
`bers 91 and 93 indicate threshold speed levels counted by
`steps, in FIGS. 9A and 9B, respectively.
`Referring to FIGS. 9A and 9B, in order to measure the
`number of steps, it is preferable that with the increasing
`activity intensity, the counting timeinterval is shorter and the
`threshold value is higher. If the numberofsteps is measured,
`the moving distance can also be calculated. According to
`sports medicine, the length of a step of an ordinary personis
`(height-100 cm), so if the length is multiplied by the number
`of steps, the moving distance can be calculated.
`If the analysis of the activity pattern is performed, the
`analysis result can be providedto the user through the mobile
`terminal 13 in operation 24. The result includes current
`caloric consumption, numberof steps, and/or moving dis-
`tance.
`
`Page 13 of 14
`
`Page 13 of 14
`
`

`

`US 7,450,002 B2
`
`9
`6. The method of claim 2, wherein the sensor unit further
`includes a gyroscope andthe direction is detected by further
`obtaining an integration of angular speed in a horizontal
`plane.
`7. The methodof claim 1, wherein detecting the wearing
`location comprises:
`by using the direction, transforming the inertia movement
`signal output from the sensor unit into an inertia move-
`ment signal in space fixed coordinates; and
`obtaining the wearing location of the sensor unit from
`differences between signal patterns of the transformed
`inertia movement signal according to locations of the
`sensor unit.
`8. The methodof claim 1, wherein determiningthe activity
`pattern comprises:
`extracting dynamic parameters of inertia movement at a
`predetermined sensor unit location; and
`determining the activity pattern from a distribution of the
`values of extracted dynamic parameters.
`9. The methodof claim 1, further comprising:
`analyzing the activity pattern according to the determined
`activity pattern.
`10. The method of claim 9, further comprising:
`if the activity pattern is determined to be walking or run-
`ning, detecting a second wearing location of the sensor
`unit; and
`if the second detected wearing location is changed from a
`previous wearing location, analyzing again the activity
`pattern at the second detected wearing location.
`11. The method of claim 10, wherein in analyzing the
`activity pattern,at least one of a caloric consumption during
`the activity, a number of steps, and a moving distance is
`measured by using the inertia movementsignal.
`12. The method of claim 11, wherein the caloric consump-
`tion is measured by obtaining