as) United States
`a2) Patent Application Publication co) Pub. No.: US 2011/0081634 Al
`
`(43) Pub. Date: Apr. 7, 2011
`Kurataet al.
`
`US 20110081634A1
`
`(54) BEHAVIOUR PATTERN ANALYSIS SYSTEM,
`MOBILE TERMINAL, BEHAVIOUR PATTERN
`ANALYSIS METHOD, AND PROGRAM
`
`(76)
`
`Inventors:
`
`Masatomo Kurata, Tokyo (JP);
`Makoto Murata, Tokyo (JP);
`Takashi Ogata, Tokyo (JP);
`Yasutaka Fukumoto, Tokyo (JP);
`Masanori Katsu, Tokyo (JP)
`
`(21) Appl. No.:
`
`12/868,302
`
`(22)
`
`Filed:
`
`Aug. 25, 2010
`
`(30)
`
`Foreign Application Priority Data
`
`Oct. 2, 2009
`
`(IP) oes eteeenees P2009-230579
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`GO9B 19/00
`
`(2006.01)
`
`(52) U.S. C1. cece cccccceecsececneceeneeneenecaeneens 434/236
`
`(57)
`
`ABSTRACT
`
`Provided is a mobile terminal including a movement sensor
`that detects a movement of a user and outputs movement
`information, acquires information on a building existing at a
`current location or information on buildings existing in a
`vicinity of the current location, analyses the movementinfor-
`mation output from the movementsensor, and detects a first
`behaviour pattern corresponding to the movement informa-
`tion from multiple first behaviour patterns obtained by clas-
`sifying behaviours performed by the user over a relatively
`short period of time, and analyses the information on a build-
`ing or buildings andthefirst behaviour pattern, and detects a
`second behaviour pattern corresponding to the information
`ona building or buildings andthefirst behaviour pattern from
`multiple second behaviour patterns obtained by classifying
`behaviours performed by the user over a relatively long
`period of time.
`
`10: BEHAVIOUR/ SITUATION ANALYSIS SYSTEM
`
`=i||||||4
`
`106
`
`112
`
`
`TIME/ CALENDAR
`
`INFORMATION
`ACQUISITION UNIT
`
`
`BEHAVIOQUR/
`SITUATION
`
`
`RECOGNITION
`
`UNIT
`
`
`
`
`
`RECOMMENDED
`SERVICEIN
`
`
`ACCORDANCE WITH
`CATEGORISATION
`UNIT
`
`|BEHAVIOUR/ SITUATION
`
`
`APPLICATIONIN
`“MAP INFORMATION MP
`|
`“GEOCATEGORY
`ACCORDANCE WITH |
`INFORMATIONGC
`|
`_BEHAVIOURYSITUATION|
`
`«PERSONAL PROFILE INFORMATION PR
`*BEHAVIOUR/ SITUATION FEEDBACK FB
`
`1
`
`APPLE 1014
`
`APPLE 1014
`
`1
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 1 of 53
`
`US 2011/0081634 Al
`
`
`
`ch
`
`FHNO|IAWH38
`
`NOILVALIS
`
`LSid
`
`801
`
`Olt
`
`901
`
`
`
`YVONIVO/AWIL
`
`NOILVAYOSNI
`
`
`
`LINNNOILISINDOV
`
`aJeMpleH| |YOSNASNOILOW|201!|SlOsuag
`
`|I|||I
`
` I74I|'NOLLWNLIS/HNOIAWH3E|HLIMZONVGYOOON||NIFOINNSS|_G3QNSNWOO3Y|__030
`
`panosMsLINN
`FTEq
`
`
`
`
`31W4O0udTWNOSYAd=ddMOVESNOILVALISHNOIAVH9-YdNOILVWYOSNI
`NOILVNIAYSLAC
`OO7|
`
`NOILVALISHNOIAVHAE|HLIMSJONVQXOOOV
`NINOWWVONMdd¥
`NOLLINDO03Y
`
`
`
`
`WALSASSISATYNYNOILVALISHNOIAVH3d-O1
`
`dWNOILYWHYOSNIdV:
`LINNNOILINSO93Y
`
`
`ALVLS/LNAW3SA0WN
`09NOILVNYOSNI
`AYO93ALVO059=
`
`
`
`—-—-——-J
`
`LINN
`
`“0459
`
`vO0l
`
`NOILVSIYOSSLV9
`
`YOSNASNOILV9DO71
`
`2
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 2 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`NOILINSOOSYALVLS/LNAWSAOW
`
`
`
`
`
`})SIdWYX4NOILVENDISNOOWALSAS
`
`¢Old
`
`YsaAdsS
`
`IN3IT9
`
`fYNOIAVH34
`
`NOILVALIS
`
`YVONATVO
`
`NOILVAINOSNI
`
`
`
`LINANOLLISINDOV
`
`cOl
`
`
`
`SJOSUASaJeMpIeH
`
`/SANIL
`
`
`
`||I||||
`
`—
`
`QSQNAWWOO0SHY
`
`NISOIANSS
`
`
`
`NOILVALISRENOIAWHAE|HLIMAONVQYOOOV
`
`a
`
`LINN
`
`NOILVSINOSALVO
`
`YOSNSNOILVD01
`
`vol
`
`|NOLLYNLISPHNOIAVHSE|99NOLLVAMYOSNI
`
`|||
`
`HLIMSONVOYOOOV
`AYOSALVD039=
`
`NINOILVOMddV
`
`dWNOILVWHYOSNIdvW-
`
`LINNNOILINDO934
`
`
`YOSNASNOILOW
`
`3
`
`
`

`

`Apr. 7, 2011 Sheet 3 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`¢)TIdWVX4NOILVYNDIANODWALSAS
`
`Old
`
`L
`
`|||||II
`
`NOILVALIS
`
`
`
`NOILYNLIS201
`
`LN3IN9
`
`
`
`YVONSFTV9/AWIL
`
`NOLLVWHOZNI
`
`
`
`LINNNOILISINDOV
`
`4
`
`NOLLWONiddWdy------—-4------—
`
`
`NI
`dWNOLLVAYO-NIdv
`
`I||
`
`Patent Application Publication INOLLWNLIS
`PolGS0NSWW003Y“0359
`
`
`
`
`
`NOILVSIYODALYSYOSNSSNOILVOOT PHNOIAVH3E|HLMSONVOYOIOVLINNNISOIANSS
`
`
`
`NOILVNINYSL30sJosuasalemple}PANOIAVH3
`
`
`
`
`NOILINDOO3HLVS/LN3W3\0WHOSNASNOILON
`
`
`LINNLINNNOILINDOOSY
`
`PANOIAVH3E|99NOLLWWHYO-NI
`
`
`|
`
`
`
`HLIMSONYOYODOVAYODALYO059»
`
`4
`
`
`

`

`/SWIL YSANsS
`/YNOIAVHAd|LINA|HLIMSONVQYOOOV!NIFOIANSSNOILVSIYOSALVO
`|GS0NSWNOO3Y“039
`
`
`
`O09NOILVWYOANIAYODSLVOO39:
` |NOLLWNLLISHNOIAVHGE|HLM3ONVCYOOOY|NINOLLWOMddy|
`
`RANOIAVH38
`
`NOILVNLIS
`
`
`
`
`
`NOILINDOO3YALVLS/LNSWSAON
`
`
`
`YVONF1VO
`
`NOILVAOSNI
`
`
`
`LINANOILISINDOV
`
`Patent Application Publication
`
`Apr. 7, 2011 Sheet 4 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`LIND)LINANOILINDOO3Y
`
`|NOILYNLIS
`
`
`
`dWNOILVWYOANIdVW-
`
`
`
`
`
`
`
`€)SIdWVX4NOILYYNDIANODWALSAS
`
`vSla
`
`INAITO
`
`5
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 5 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`b)SIdWVX3NOILVYNDIANODWALSAS
`
`oSlA
`
`
`PUNOIAVHEE|LINN!|HLIMSONVGYOSOY|NOLLWSIYOOSLV9+HOSNASNOILV9OT|_NISOIANSS
`“039Pol|__GSONAWNNOOTE!
`INOLLWNLIS
`As~7|
`
`
`dW__-_--——4
` "NOLLWNLISUNOIAWHAS||HLIMSONWGHOOOV|I
`
`NINOLLWOIIdd¥
`dWNOLLVWHYOSNIdV:
`J9NOILVAYONI
`AYODALVSO59:
`
`RNOIAVHAG
`
`NOILVNLIS
`
`80lcol
`
`
`
`|SIOSUBSaJEMPIEH
`
`
`
`LINNNOILISINDDY
`
`LNANM9
`
`
`
`YVGNATV9/AWIL
`
`NOLLVWYOSNI
`
`a
`
`LINA
`
`NOILINDSOO3Y
`LINANOILINDOO3Y
`
`
`ALVLS/LNSAWSAOW
`
`YOSNASNOILOW
`
`6
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`
`
`
`
`G)TIdWVX3NOILYYNDISNOOWALSAS
`
`9°Did
`
`INAIT9
`
`
`
`YVONSTVO/ANIL
`
`NOILVWYOSNI
`
`
`
`LINNNOLLISINDOV
`
`Apr. 7, 2011 Sheet 6 of 53
`
`US 2011/0081634 Al
`
`
`
`NOILVNINYSL30RENOIAVHAE|SJOSUASBIEMPIE}H
`
`
`NOLLYNLIS501|20!
`
`
`NOLLINDOOSY31VS/LNAW3AOWr|_BOSNASNOILOW
`
`
`LINNLINNNOILINDODSY
`
`
`
`'NOLLVNLIS
`
`
`
`PYNOIAWH3E||HLIMSONVQYOOOV|JNINOWLVONddy|dvLoaay
`(NOLLWNLIS
`dWNOILVWHYOSNIdV-
`09NOILLVWHOSNI
`AYODALVO039=
`
`J
`
`vol
`
`
`
`
` /YNOIAVHAG||HLIMONVGYOOOY|LINN!|NISOIAYRSNOLLVSIHOD3LVOHYOSNASNOILWIO1
`
`
`___GSQNINNOOTY039
`
`
`
`7
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 7 of 53
`
`US 2011/0081634 Al
`
`SWIL
`
`SWIL
`
`
`
`
`
`(YOSNASNOILOWWO¥SLNdLNOVLVdWYOSSAVM)NOILVYSTS00V
`
`
`
`
`(ALILNYNOSYNLVAsSLVLS/LNAWSAOW)ALILNYND
`
`
`(NYSLLVdSLVLS/LNAWSA0W)
`
`Buea=|WS|eBueygaunjsog
`
`
`
`801S™LINNNOILINDO93¥YSLVLS/LNSWSAOW
`
`2’Sld
`
`BuryleAA
`
`JuNnLVs4
`
`8
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 8 of 53
`
`US 2011/0081634 Al
`
`FIG.8
`
`
`EXAMPLES OF MOVEMENT/ STATE PATTERN
`
`(INCLUDING POSTURE)
`
`(VEHICLE)
`
`ee
`ee
`
`
`
`
`
`
`
`
`
`
`
`[nefe
`ee
`
`
`[ace
`
`TT
`
`
`
`9
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 9 of 53
`
`US 2011/0081634 Al
`
`FIG.9
`
`X-acc
`
`y-acc
`
`Z-acc
`
`I
`
`FIFO
`
`|
`
`CALCULATE VARIANCES
`
`$1000
`
`$1002
`
`CALCULATE LARGEST VALUE
`
`31004
`
`det
`
`51006'
`
`NO
`
`YES
`
`.
`$1008
`
`
`
`
` CONTINUE
`
`
`FOR LONGER THAN
`
`TIME T?
`
`NO
`
`PAUSING
`
`$1012
`
`YES
`
`STAYING STILL
`
`31010
`
`10
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 10 of 53
`
`US 2011/0081634 Al
`
`FIG.10
`
`Rfee
`
`|
`
`BPF
`
`|
`
`py «81102
`
`
`sttt2
`
`S1114
`
`$1128
`
`CALCULATE: NUMBER
`OF STEPS+ freq * time
`
`NUMBER OF STEPS
`
`$1130
`
`tT nsioa
`ply
`CALCULATE
`CALCULATE
`VARIANCES
`AUTOCORRELATION
`
`FUNCTION
`+
`
`CALCULATE LARGEST
`VALUE
`
`SACF
`
`$1106
`
`var
`
`CALCULATE ENHANCED
`AUTOCORRELATION
`FUNCTION
`
`CALCULATE: freq x var
`
`$1116
`
`$1120
`YES
`
`D2<det<D3?
`
`NO
`
`81194
`
`WALKING
`
`$1122
`
`$1126
`
`NO
`
`RUNNING
`
`NEITHER WALKING
`NOT RUNNING
`
`$1128
`
`11
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 11 of 53
`
`US 2011/0081634 Al
`
`FIG.11
`
`CALCULATE MAGNITUDE
`
`$1200
`
`BPF
`
`$1202
`
`CALCULATE ABSOLUTE VALUE
`
`$1204
`
`$1206
`
`LPF
`
`det
`
`$1208<a
`
`YES
`
`$1210
`
`$1212
`
`JUMPING
`
`NOT JUMPING
`
`12
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 12 of 53
`
`US 2011/0081634 Al
`
`FIG.12
`
`LPF
`
`$1300
`
`CALCULATE CHANGESIN
`DIRECTIONS OF GRAVITY
`
`$1302
`
`CALCULATE MAGNITUDE
`
`31304
`
`$1306
`
`LPF
`
`det
`
`$1308
`
`NO
`
`$1316
`
`NO CHANGE
`IN STATE
`
`YES
`
`$1310
`
` CURRENT STATE?
`
`
`SITTING
`
`STANDING
`
`SITTING
`
`$1312
`
`STANDING
`
`$1314
`
`13
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 13 of 53
`
`US 2011/0081634 Al
`
`FIG.13
`
`Kf
`
`7
`
`$1400
`
`CALCULATE MAGNITUDE
`
`LPF
`|
`aee
`
`$1404
`
`41406
`
`$1408
`
`acc
`
`
`CALCULATE
`ACCELERATIONIN
`$1410
`
`GRAVITY DIRECTION
`No
`
`
`DOES NOT
`_S1414
`UPDATE GRAVITY
`NO
`$1416
`
`
`
`
`$1412
`
`VES
`<A?
`
`
`UPDATE GRAVITY
`
`DOES NOT
`
`si4i8
`
`Ai <avg
`
`$1420
`
`$1422
`
`
`
`$1424
`
`NO
`
`FIRST
`OBSERVATION?
`
`NO
`
`$1426.
`YES
`
`$1428
`ASCENDINGIN
`ELEVATOR
`
`$1432
`
`DESCENDINGIN
`ELEVATOR HAS ENDED
`
`$1430
`
`NO
`
`$1434
`YES
`
`FIRST
`OBSERVATION?
`
`$1436
`DESCENDINGIN
`ELEVATOR
`
`NO
`ASCENDINGIN
`ELEVATOR HAS ENDED
`IN ELEVATOR
`
`$1440
`CURRENT
`STATE?
`
`S1438
`
`S1442
`NO CHANGEIN
`STATE
`
`NOT IN ELEVATOR
`
`s1444
`
`14
`
`14
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 14 of 53
`
`US 2011/0081634 Al
`
`FIG.14
`
`[aes[ae[ae
`td
`
`$1502
`
`IN HORIZONTAL DIRECTION
`
`IN VERTICAL DIRECTION
`
`$1506
`
`$1510
`
`$1504
`
`$1508
`
`$1512
`
`
`
`
`
`CALCULATE VARIANCE IN
`HORIZONTZL DIRECTION
`
`CALCULATE VARIANCE IN
`VERTICAL DIRECTION
`
`h-var
`
`v-Var
`
`$1514
`NO
`
`YES
`
`SET TO ZERO
`
`$1528
`
`$1516
`NO
`
`$1520
`
`
`CALCULATE:
`INTEGRATION OF v-var
`
`
`
`CALCULATE:
`INTEGRATION OF h-var
`
`$1522
`NO
`
`YES
`
`NOT ON TRAIN
`
`$1526
`
`ON TRAIN
`
`$1524
`
`15
`
`15
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 15 of 53
`
`US 2011/0081634 Al
`
`FIG.15
`
`CALCULATE ANGULAR
`VELOCITY IN GRAVITY
`DIRECTION
`
`$1602
`
`BPF
`
`$1604
`
`:ooh a © a
`
`YES
`
`YES
`
`$1608
`
`TURNING RIGHT
`
`$1612
`
`TURNING LEFT
`
`NO
`
`Zz oO
`
`NO TURNING
`
`$1614
`
`16
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 16 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`
`
`JCNLISNOVaACN.LLV]AgGALVOIGNINOILYOOT
`
`
`
`SISV@AdAL-YddNOWVYSOLSIHOLNIGANYNL
`
`
`
`
`
`‘LNSWSAOWJOLNNOWY“SGNLIONOVAGNLILY1(ea)
`
`‘O14
`
`
`
`4OALNIOIASHLNISONICTING4OMASHINN
`
`
`
`
`
`JAGNLIONOVACNLILVTA@GALVOIGNINOILYDO1
`
`
`
`
`
`LVONIGINE40AdALONILVOIGNIACO9AdAL
`
`
`
`3d09ANODALVOO39(18)
`
`9b'Sls
`
`
`WVYSOLSIHANODSLV)039(Z9)
`NOLLYSINOD31LV9-039\SOL
`JINNcSanIIONO1
`
`‘ONIGTING40AdVHSNONOILYWYO-NI)
`
`(AVOYYOSLIS
`
`
`
`
`
`
`
`VLVGINSW313SdVHS(Z¥)
`
`ASVaVLVddv
`
`
`
`NOILVAYOSNIdv!(LV)
`
`Ol}
`
`aee
`
`\)/
`
`17
`
`
`
`‘O13‘(AYODALYOTWNOILYdNDD0)SYHO_LS(EY)
`
`ae eeEE
`
`
`
`
`
`
`
`
`
`ALISYOONIGTINGHLIMGSYSLSIOIY
`
`eeeeeeeeeeee
`
`—
`
`\
`
`17
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 17 of 53
`
`US 2011/0081634 Al
`
`FIG.17
`
`
`
`
`
`
`
`
`
`
`
`OFECanaT
`
`RALWAY
`
`
`
`pFttteofTwonor
`
`_
`
`ROPEWAY
`
`oo
`
`ae
`
`e
`
`BASEBALLFIELD
`BATTING CENTRE
`
`——
`
`PC
`PS
`p
`po
`
`
`
`PT
`i
`
`1405000
`
`18
`
`
`
`CHESS ROOM
`
`18
`
`

`

`Patent Application
`
`Publ
`
`ication
`
`Apr. 7, 2011 Sheet 18 of 53
`
`US 2011/0081634 Al
`
`(WYYDOLSIHAHODALYOO49)
`
`
`
`
`
`AYO93LV9
`
`81
`dis
`
`NOILNGIYLSIG
`
`19
`
`19
`
`

`

`Patent Application
`
`Publication
`
`Apr.7, 2011
`
`Sheet 19 of 53
`
`US 2011/0081634 Al
`
`
`
`\__Auoaivo039P|iS-F\icoI|aCWVYOOLSIHMoodyoiOW.2.NETeit
`_X‘NeSSSOR‘ossaYq*
`(NOILW1S)xx0Z3000\\\“Ss
`
`
`61'Sia
`
`iN,
`
`20
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 20 of 53
`
`US 2011/0081634 Al
`
`ovateLINN~-----------------5NOILINDOOSYVVNOSNSS
`
` NOILVNLIS!MNOIAYHSSNOILLWNLUS‘ALILNWNOSYNLVAS!MNOIAVHAELNNYsllVdSLVLS/LNANSAOWNJ
`
`
`
`
`
`
`
`|NYSLLVd
`
`
`
`
`
`WOWEdS34NOILVNLISNOIAVHSE-
`
`
`
`31140YdTWNOSYaAd>
`
`l|||||||||
`
`NOILVNINYSLAG
`
`TAGOW
`
`bo
`
`meeeeaeaaaeee4
`
`poorcoeaeeeeeeeeeeeee
`
`dvINAWSAOW40LNNOWY
`
`
`
`NOILVONdd¥LNAIMNO
`
`
`
`FAQNLILV1WVYSOLSIH
`
`
`
`ALIDOTSA‘SGN.LIONO
`
`21
`
`
`
`___3009AYODALVO039_
`
`|||)
`
`Gth
`
`02°Sis
`
`MVONSTWO/AWL|
`
`NOLLVWMOSNI
`
`21
`
`
`
`
`

`

` eeeeeee
`
`
`
`/TIVHo00¢0steeeeePooftefeefee]ppfTTIVHATEWSSSY
`poteTaeeeeeeeeeeeee
`eeeeeeeeee
`potpeperces
`
`TWYSN39)ftpepeepeeange)mover
`SLVAIHdfe(WW1ldSOH
`
`
`
`ONIdFFIS}ONIMAIA|HYOM|AONLSWAW|conasnon|SYSHLOFPayoisnwcave|wuvanMIVM|LYOdSsSWYNTAAGIN
`
`
` TOOHOS
` ‘NALYVOUSONIN
`
`PoteetpepeteEete|Tseng
`
`
`3HOHOSONVAHOSHAdAYODALYS
`
`
`‘JOOHDSONIYOLNL
`OOHOSANOLWHWdaudoooLoez
`
`
`
`AYASYAN
`
`000z0eZ
`
`o0oeore
`
`00OPOE?
`
`22
`
`Patent Appl
`
`icat
`
`ion
`
`Publ
`
`icat
`
`ion
`
`Apr
`
`7;
`
`2011 Sheet 21 of 53
`
`US 2011/0081634 Al
`
`
`
`Le’Sis
`
`
`
`‘AMATIVDLYY‘ANSSNIN000105}
`
`22
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 22 of 53
`
`US 2011/0081634 Al
`
`FIG.22
`
`SELECT COMBINATIONS OF
`SCORES IN ACCORDANCE
`WITH GEO CATEGORY
`
`SELECT SCORE MAPS IN
`ACCORDANCE WITH BEHAVIOUR/
`SITUATION PATTERN
`
`
`
`SELECT SCORE MAPIN
`ACCORDANCE WITH TIME/
`CALENDAR INFORMATION
`
`23
`
`510
`
`912
`
`514
`
`23
`
`

`

`Apr. 7, 2011 Sheet 23 of 53
`
`US 2011/0081634 Al
`
`Buyoq||
`
`Patent Application Publication
`
`
`
`
`
`LNNOWY/G33d$LNAWSAON)
`
`(LNAWSAOW40
`
`o's
`
`NYSLLVdNOILVALISYNOIAWH3
`
`
`
`
`NOILLVNIWYSLAGNYSLLVdNOILVALIS
`
`MNOIAVHA40AIdNVX>
`
`Tome,ktot|
`
`TTYr-
`~|WLMIEMPEL!|poadg||ins5wean!L
`
`pL>HIeEM>eL//(ZL<)Nas/(LL<)
`lEM/SN/ule/4e9/aoAaIgéZL>INS>pLPue
`
`(LA<)peeds
`
`Lo
`
`Siofoig|
`
`24
`
`
`
`
`
`
`
`
`
`3AMFHLAOAV/AWILOLONIGNOdSAYHOD6GNIS-LWSSdNOUDdVWSYHOOS‘6d-ld3X
`
`24
`
`
`
`
`
`

`

`
` |>ONILHOISM+|N8€dd_olpd=FY[£ vounsusia{~~frounesusoef||[elefel[ef[ee]foodina
`
` ‘ON)ai|re/(ox)|:eo/ONULHOISM||"<'iulitsettHoude=|PPT.11.1.1,
`
`ASIIVWYON(00020913yoos3409S“U_d¥N3HOOS=\;mt
`
`
`
`AYODSLVD|AMODSLVO|ANODALVO|ANODSLWO|ANODSLWO|AYODSLWO|AYODSLVO|ANODALVO|ANODALVO|ANODALWO|“Gosiv5 039039039039039
`m=T[eyeyyeyys
`
`
`-—_f\_||[woureruso&—-Nvourmsoa—a[+][elelel[+][+[[e[woketn
`qYOOSI]d¥WSYOoS
`
`
`JHOOSVaASTIVWYON(000S06|‘ON)
`1dif\Lpd
`
`
`
`
`
`
`
`(699)Nonnanisia(999)(299)(999)(s99)(p09)(199)
`039
`
`
`JSITWWYON(000€0€Z‘ON)
`
`
`
`veSis
`
`
`
`
`
`Patent Application Publication
`
`ly
`
`
`
` ///\ONILHOISM
`
`
`
`WVEDOLSIHO59ONISNGOHLSWNOILVINOWONOILNEINISIGSYOOS
`
`US 2011/0081634 Al
`
`tr(oex)
`
`
`
`25
`
`25
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 25 of 53
`
`US 2011/0081634 Al
`
`NusllVd,NOILVNLISNOIAVHAEONIHOLVW|‘
`
`
`NOILYNINYS130 ——
`‘[8]812d=LNJey=aauNoSONINYVS1SNIHOVN$09‘UIs‘UpZ
`
`
`
`
`
`
`‘[b]By2d=ZNJeOHaBeq=ONJeoNOISSSUOSYFYNLV3SSLVLS/LNAWSAOWYVAN
`
`
`
`
`:ALILNYND“N+ON=SNFeO}NOISS3YOSYUVSNMINONGHLJOY
`
`
`
`
`
`
`TL]Bled=@Ne0y(auiysey40}98,qYoddns)WAS
`[eounos]WLVOWaSM-40-AVO/SWIL
`
`
`
`
`
`
`YOLOFASYNLVAs40LANI
`
`TSGOW
`
`
`
`ViVdYSHOV3L
`
`
`
`YOLOSASYNLV34
`
`‘[6]BPC=ENJeOHWHLIMODTY
`
`
`
`
`dz‘NUSLLVdSLVLS/LNSWSA0W
`
`-YN+EN=ZNJE0+
`
`‘8N+SN=PNJeOyBujsoog
`
`4¥60617'0-=+ON
`
`
`
`4672500'0-=.ONGOHLSWHOYVSS
`
`‘CN“LN=ONJE0}ALLLNWNOSYNLVAd
`
`
`SILSNS9AdNOILOAISS
`
`
`‘JON3Y3ISIGAGNLIONOT/ACGALILV]
`
`eeee eeee
`
`
`
`WVYSOLSIHANOD3SLVOO39
`
`
`
`4d09AYO93LV9039
`
`aul)xGE
`
`
`
`
`
`(2H9L)C9‘VLVGYOSNAS
`
`
`
`PANOIAVHASLOFYYOO
`
`
`
`J1EVINVASSNOdS3Y
`
`
`
`NusllvdNOILVNLIS
`
`eeeeeeeEE Ee
`
`26
`
`So'Dls
`
`
`
`
`
`ONINYVS71SNIHOVWONISNGOHLAWNOILVINOTVSNOILNEIYLSIdSYOOS
`
`
`
`
`
`
`
`
`
`26
`
`

`

`Patent Appl
`
`icat
`
`ion
`
`Publ
`
`icat
`
`ion
`
`Apr
`
`7;
`
`2011 Sheet 26 of 53
`
`US 2011/0081634 Al
`
`ONIMSIA|
`
`3IAOW408
`
`oISnW208
`
`NOISIATSL€08
`
`SYSHLO=.
`
`SYSHLO
`
`S$1uOd$
`
`NOHLVYVN
`
`4109
`
`TWaasvd
`
`SUSHLO669|SUSHLO662 SYSHLO
`
` ANIHOVW
`1004NOba13dHIMWMJOSSONSINSANOOhoINSW3AONmiSM|e|ONIddOHS4|9¢'DIs
`
`ONIGNSA0|AYOLSVOINse
`
`27
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 27 of 53
`
`US 2011/0081634 Al
`
`FIG.27
`
`BEHAVIOUR/ SITUATION RECOGNITION FLOW
`
`BEHAVIOUR OF USER
`
`520
`
`ACQUISITION OF SENSOR
`DATA BY SENSOR
`
`S22
`
`MOVEMENT/ STATE DETECTION,
`GEO-CATEGORISATION
`
`SoA
`
`|
`
`DETECTION OF BEHAVIOUR/
`SITUATION PATTERN
`
`Le
`
`“ ND D>
`
`NOTIFICATION TO APPLICATION
`
`S28
`
`NO oO
`
`28
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 28 of 53
`
`US 2011/0081634 Al
`
`FIG.28
`
`(A: RULE-BASED DETERMINATION)
`
`$26: DETECTION OF BEHAVIOUR/
`SITUATION PATTERN
`
`USER PROFILE IS
`REGISTERED?
`
`
`
`
`LATITUDE/
`LONGITUDE IS HOME OR
`
`
`
`WORKPLACE?
`
`
` SELECT SCORE MAPS
`
`CORRESPONDING TO
`SELECT, TAKING PROFILE||_OME/ WORKPLACE
`
`INTO CONSIDERATION,
`SELECT COMBINATION OF
`
`COMBINATION OF SCORES
`SCORES IN ACCORDANCE
`
`
`IN ACCORDANCE
`WITH GEO CATEGORY
`
`WITH GEO CATEGORY
` NARROW DOWNTYPES
`
`
`
`OF SCORE MAPS BASED
`ON MOVEMENT/ STATE
`PATTERN AND TIME/
`
`
`CALENDAR INFORMATION
`
` SELECT BEHAVIOUR/
`
`SITUATION PATTERN
`CORRESPONDING TO
`HIGHEST SCORE FROM
`COMBINATION OF SCORES
`OF TYPE OBTAINED BY
`NARROWING DOWN
`
`
`
`
`
`
`
`29
`
`29
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 29 of 53
`
`US 2011/0081634 Al
`
`FIG.29
`
`(B: MACHINE LEARNING-BASED DETERMINATION)
`
`$26: DETECTION OF BEHAVIOUR/
`SITUATION PATTERN
`
`USER PROFILE IS
`REGISTERED?
`
`
`
`
`
`SELECT DETERMINATION
`
`MODEL OBTAINED WITH
`USER PROFILE TAKEN INTO
`
`CONSIDERATION
`SELECT
`
`GENERAL-PURPOSE
`DETERMINATION MODEL
`
`INPUT INFORMATION TO
`
`SELECTED DETERMINATION
`
`
`MODEL AND DETECT
`MATCHING BEHAVIOUR/
`
`SITUATION PATTERN
`
`
`
`OUTPUT DETECTED
`BEHAVIOUR/ SITUATION
`PATTERN
`
`30
`
`30
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 30 of 53
`
`US 2011/0081634 Al
`
`FIG.30A
`
`(C: COMBINATION OF RULE/ MACHINE LEARNING)
`
`$26: DETECTION OF BEHAVIOUR/
`SITUATION PATTERN
`
`USER PROFILE IS
`REGISTERED?
`
`
`LATITUDE/
`LONGITUDE IS HOME OR
`
`WORKPLACE?
`
`
`
`
`
`
`SELECT SCORE MAPS
`
`
`CORRESPONDING TO
`
`
`
`HOME/ WORKPLACE
`SELECT, TAKING PROFILE
`
`SELECT COMBINATION OF
`INTO CONSIDERATION,
`
`
`
`SCORES IN ACCORDANCE
`COMBINATION OF SCORES
`
`
`WITH GEO CATEGORY
`IN ACCORDANCE
`
`
`
` WITH GEO CATEGORY
`NARROW DOWN TYPES
`
`OF SCORE MAPS BASED
`
`
`ON MOVEMENT/ STATE
`
`
`PATTERN AND TIME/
`
`
`CALENDAR INFORMATION
`
`
`
`SELECT BEHAVIOUR/
`SITUATION PATTERN
`
`
`CORRESPONDING TO
`HIGHEST SCORE FROM
`
`
`COMBINATION OF SCORES
`OF TYPE OBTAINED BY
`NARROWING DOWN
`
`
`
`31
`
`31
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 31 of 53
`
`US 2011/0081634 Al
`
`FIG.30B
`
`
`SCORE OF
`
`
`SELECTED BEHAVIOUR/
`SITUATION PATTERN IS A
`
`SPECIFIC VALUE
`
` USER PROFILEIS
`
`MODEL OBTAINED WITH
`USER PROFILE TAKEN INTO
`
`
`CONSIDERATION
`
`SELECT
`GENERAL-PURPOSE
`
`
`
`YES
`REGISTERED? SELECT DETERMINATION
`
`DETERMINATION MODEL
`
`
`
`INPUT INFORMATION TO
`
`SELECTED DETERMINATION
`
`MODEL AND DETECT
`S164
`MATCHING BEHAVIOUR/
`SITUATION PATTERN
`
`OUTPUTSELECTED OR
`DETECTED BEHAVIOUR/|~S166
`
`
`
` SITUATION PATTERN
`
`32
`
`32
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 32 of 53
`
`US 2011/0081634 Al
`
`(NOLLVWHOSNINOLLYOISILON
`
`4OSLNSLNOO)
`
`NOILYNLISYNOIAWH38)
`
`(41NS3YNOILOSL3q
`
`(ONIWILNOILWOISILON)
`
`JindNIVYL
`
`ONDIIWM
`
`
`
`YOJONIGVSH
`
`NOILLVLS
`
`ISIHNOLJONOILVINASSYd
`
`
`
`NOILVWHOINILOdS
`
`NOILLVWYOSNI40NOLLVLNSS3ud
`
`
`
`dOHSYINAANOSNO
`
`NOILVWHOSNI30NOILVINSS3Yd
`
`
`
`
`
`LOdSLSIHNOLLXANNO
`
`LAWYNOS4ONOILVLINASSYd
`
`NOILVWHOSNI
`
`ATEAVLAWIL40NOILWLNSSSud
`
`
`
`LNSWSAOWLXANHOS
`
`ONIMTVM
`
`ONIddOHS
`
`NIVYL
`
`ONINIVM
`
`WAN
`
`
`
`ONIATHYVLSOWTV
`
`
`
`LOdS$LSINOLLV
`
`
`
`LSMNOLGNNOYV
`
`LOdS
`
`QN3SHLGNNOYV
`
`ONISSSLHOIS40
`
`SWOHAVM
`
`
`
`(vuNODANNOY)
`
`ONISSSLHOIS
`
`Qo
`2
`N
`—_
`
`
`
`(WHNODGNNONY)
`
`WAW
`
`Qo
`°oO—
`
`Qo
`2S
`a
`—_
`
`ON(WHVHNYOONSS)
`lddSLOH
`
`snd
`
`OLOWNA*-VEVHMHOONSS
`
`(NYAPNIHS
`
`—OLOWNA-SNONVH)NIWaL
`
`Q
`oe
`oO
`_
`
`(OLOWNA
`-JNOWVH
`
`<—NMNENIHS)NWel
`
`(OLOWNA-SNOWVH
`
`GNNOUY)
`
`ONISASLHOIS
`
`Oo
`2S
`oO
`—
`
`©
`2
`—
`_
`
`AWOH
`
`4IVM
`
`00:8
`
`(A
`
`INGSHOS)
`
`33
`
`
`
`
`
`
`
`NOILVOMddV¥SINGSHOSHLIMNOILVTSYYOONOGSSV8GOHLAWNOILVINASSYdNOILVAWYOANISAIL9035443
`
`
`
`
`
`
`
`LeSid
`
`33
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 33 of 53
`
`US 2011/0081634 Al
`
`
`
`WO.wees|ll
`
`
`
`
`
`(UNOIAVHAWNLOVONYYNOIAVHAdGAINGSHOS
`
`
`
`NASML3dNOSINVdNOO40AV1dSIQ)
`
`celd
`
`(OLOWNA-SNOVH
`
`(\NPNIHS)
`
`(OLOWNA-ANOMYH
`
`SNISSSLHOIS
`
`annowy)
`
`(31NGSHOS)
`
`7
`
`7
`
`7
`
`)
`
`(1NSAZGS1NdSHOS
`
`
`(vunodGNNOUy)
`
`ONISSSLHDIS
`
`7
`
`ONINdSLOH
`
`
`
`(YuNOdGNNOUY)
`
`Waw
`
`(ALMV3YNI)
`VeVHNWOONAS*+-VeNOS
`
`snd
`
`2TWAWONIAVH
`
`OLOWNA*-YHVHNYOONAS
`
`sng
`
`ONIYdSLOH
`
`(WHVHNHOONSS)
`
`34
`
`NOILVALIS/UNOIAVHAS
`
`NOWOSL30NYALLVd
`
`—OLOWNA-SNOXVH)
`
`NIVEL
`
`(NYNPNIHS
`
`34
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 34 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`
`
`
`
`ceOla
`
`JINNNOILDIdSed
`(AYOLSIHNOILVOO71Olt
`
`LINNSADVYOLSAYOLSIH
`
`
`/AMOLSIHYNOIAVHA®)
`
`NOILVAOSNILANYNOD-
`NOILVWHOANILNSAS=NOLLVOMNddV
`
`
`cll901
`NOILINSOOSY
`dNNOILVWHOSNIdv:
`09NOILVAOSNI
`AYODIALVSO39>
`/LNANSAON||801|GOL
`ALVIS
`LINANOLLVOISIMSA
`
`LINNNOLLINDOO3Y
`NOILVSIHODALVO
`LINNNOILISINDOV
`
` ||||||YOSNASNOILOW|;||||
`
`YVONATVO/AWIL
`
`
`
`
`
`WALSASDNIGIAOUdNOLLWWYOSNI)WALSASSISATWNYNOILVALISHNOIAVH3E‘02
`
`
`
`
`
`
`
`NOILVAMOSNINOLLVLYOdSNVYLO118Ad:LINNAVIdsId
`
`WS
`
`[idVW3xO090S
`
`YNOIAVHAg
`
`
`
`206
`
`RENOIAVH38
`
`NOLLVALIS
`
`LINN
`
`Ol2
`
`YNOIAVH3A8
`
`vOd
`
`“049
`
`LINA
`
`||
`
`|
`
`|
`
`|vOl
`
`35
`
`NOILVAOSNI
`
`AINGAHOS
`
`LINNSDVYOLS
`
`35
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 35 of 53
`
`US 2011/0081634 Al
`
`
`
`(NHBLLYdNOILYALIS
`
`MALSN1O¥NIHLIM
`
`=
`
`
`
`SHOLOSALNAWSAOW40
`
`
`
`HOSUBISNTONSSML39
`
`peSis
`
`NOILY907)
`
`
`RINOIAVHAGJOAMOLSIH):
`AWIL|snaONDNIVELTLSSNM
`~~ALnlavaoudSrNOLLISNVELL
`yoNOILWINOWO
`MSEWNNAGSLVINOW)(NOUWMOANI
`
`
`
`yualsnio@¥aLsnioALVLS
`
`duaisnigAVESSOHLONST
`UNOIAVHAENIV‘SNL
`
`NAMYL3AVWSLNSLNOO
`OLNI
`
`NOILVHACISNOD
`
`36
`
`NOILY9O1
`
`LVONINSISNTO
`
`4OSINIOd
`
`
`
`TLS/ONINTWM
`
`‘NOLLVO07
`
`QNIMALSN19
`
`36
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 36 of 53
`
`US 2011/0081634 Al
`
`FIG.35A
`
`PRE-PROCESSINGFOR
`BEHAVIOUR VERIFICATION UNIT
`
`DETECT MOVEMENT/
`STATE PATTERN FROM
`SENSOR DATA
`
`“7
`OO
`|
`SENSORDATA,
`\.__ |
`| LOCATON INFORMATION
`
`
`$202
`
`INFORMATION
`
`CALCULATION GEO
`CATEGORY INFORMATION|~$204
`(CODE, HISTOGRAM) FROM
`LOCATION INFORMATION
`
`DETECT BEHAVIOUR’
`SITUATION PATTERN FROM
`MOVEMENT/ STATE PATTERN
`AND GEO CATEGORY
`
`$206
`
`$208
` SCHEDULE
`
`NO
`
`
`INFORMATION
`EXISTS?
`
`$212
`
`
`
`
`CHECK CURRENT
`
`PREDICT NEXT BEHAVIOUR
`
`SCHEDULED EVENT AND
`BASED ON HISTORY OF
`
`
`NEXT SCHEDULED EVENT
`
`
`BASED ON SCHEDULE
`BEHAVIOUR/ SITUATION
`
`
`
`
`
`PATTERNS
`
`INFORMATION
`
`
`
`
`TO PROCESSING BY BEHAVIOUR
`VERIFICATION UNIT
`
`37
`
`37
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 37 of 53
`
`US 2011/0081634 Al
`
`FIG.35B
`
`PROCESSING BY BEHAVIOUR
`VERIFICATION UNIT
`
`|
`GURRENTLOCATON?
`|
`____
`BEHAVIOUR, NEXT
`Hoooo—-|
`| SCHEDULEDEVENT _ |
`
`$224
`
`
`
`
`
`$222
`
`
`
`CURRENT
`NO
`
`[OCATON/ BEHAVIOUR AND
`
`SCHEDULED EVENT
`
`
`S228
`MATCH?
`
`
`
`
`
`MATCH NEXT
`NO
`SCHEDULED EVENT?
`
`
`$232
` MATCH PAST
`
`
`NO
`CURRENT
`
`SCHEDULED EVENT?
`
`SCHEDULED BEHAVIOURIS
`
`YET TO BE OVER?
`
`
`
`ACQUIRE NEXT
`SCHEDULED EVENT
`
`
`
`ACQUIRE CURRENT
`SCHEDULED EVENT
`
`
`
`ACQUIRE BEHAVIOUR
`
`
`PREDICTION DATA FOR
`
`VICINITY OF CURRENT
`LOCATION
`
`
`
`ACQUIRE BEHAVIOUR
`PREDICTION DATA FOR
`
`PLACE LIKELY TO BE
`VISITED NEXT
`
`
`
`CHECK TYPE OF
`SCHEDULED BEHAVIOUR
`
`
`
`
`(GOURMET,
`ENTERTAINMENT,VEHICLE)
`
`
`
`ACQUIRE PAST
`SCHEDULED EVENT
`
`
`
`S240
`
`ACQUIRE INFORMATIONIN
`ACCORDANCEWITH TYPE
`
`OF SCHEDULED BEHAVIOUR
`OF APPLICATION
`
`DISPLAY ACQUIRED
`INFORMATION ON SCREEN
`
`CASE WHERE THEREIS
`
`SCHEDULE INFORMATION
`
`S242
`
`S244
`
`38
`
`38
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 38 of 53
`
`US 2011/0081634 Al
`
`FIG.35C
`
`CASE WHERE THERE !S NO SCHEDULE INFORMATION
`
`PROCESSING BY BEHAVIOUR
`VERIFICATION UNIT
`
`OT 7
`
`|
`\, = —_s,: CURRENT LOCATON/
`|
`|
`BEHAVIOUR, NEXT
`| SCHEDULED EVENT.|
`
`ACQUIRE BEHAVIOUR
`PREDICTION DATA FORVICINITY|~S252
`OF CURRENT LOCATION
`
`OF APPLICATION
`
`ACQUIRE BEHAVIOUR
`PREDICTION DATA FOR PLACE
`LIKELY TO BE VISITED NEXT
`
`CHECK TYPE OF SCHEDULED
`BEHAVIOUR (GOURMET,
`ENTERTAINMENT, VEHICLE)
`
`ACQUIRE INFORMATION IN
`ACCORDANCE WITH TYPE
`OF SCHEDULED BEHAVIOUR
`
`DISPLAY ACQUIRED
`INFORMATION ON SCREEN
`
`$254
`
`S256
`
`$258
`
`S260
`
`End
`
`39
`
`39
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 39 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`VYSWAINSOLST-0deLVSWAW40LSIdol NASYO0SAV1dSIGN3aYOSNOILVYLSIOSY
`
`AYOLSHOOsOLOD#
`AWOHAVMNOdhWoldOLSONIHL® ‘SINALNOO
`
`
`SSOLVWOL
`
`
`
`AWOHAVMNO‘LSISNOIAVHAS
`
`‘LSAONIddOHS
`
`Le/8/6002
`
`-INNGV30
`
`9e°DIs
`
`‘$993“MIN‘SLNALNOOAV1dSId
`
`
`
`JWOHAVMNO-LSI1YNOIAVHad
`
`
`NOILYNLIS/YNOIAVH38)
`
`(L7NS3YNOILOSLIA
`ja]asménoio|[alanoiaviaa|
`
`ja]asricro||]unoyivuas
`oqo)40YNOIAVH3Ad
`AWv4ONDHYOM
`
`anidONILVA
`AVidSId
`
`
`
`
`
` 4008¥YACHO‘SLNALNOOAV1dSIC
`
`ONIYVHSdO3dAL
`
`WV3lTIV8ASV8
`
`ONIddOHS
`
`GaLV1su-yYOM
`MIOHASNOH
`
`SAYOHO
`
`ALVWAYOM
`
`ONIMSIA
`
`40
`
`40
`
`
`
`
`
`
`
`

`

`2011 Sheet 40 of 53
`
`US 2011/0081634 Al
`
`Apr. 7
`
`’
`
`Patent Application Publication
`
`SOL Ze'Sls
`
`
`
`
`
`ONIddOHS‘AWOHAVMNO‘(310A01d“YV9‘SN‘NIVEL)SNIAOWLSI]SNIiddOHS
`
`
`
`
`NdsitVdNOILVALISAYNOIAVH3SSLNALNOOAVIdSId
`ONIMSIAAL‘SSYOHOGIOHSSNOH|ONIMSIAWVYDOudCAdHOOIY
`
`
`
`
`
`
`ONINTWM‘(LOONON)SNIAOWSi]S9VSSsW
`
`
`
`
`ONIddOHSYALSV‘ONIddOHSSLNNOODVCIOHASNOH
`
`
`
`
`(SNE‘NIVYL)SNIAOWONINALSI]HSITONS
`
`TWAWYSL4V“IVAWMOAHOAO
`
`A1
`
`41
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 41 of 53
`
`US 2011/0081634 Al
`
`BESls
`
`
`
`
`
`
`
`WALSASONIGIAONdNOILVAYOANI)WALSASSISATYNYNOILVNLISYNOIAVH3E“Of
`
`
`
`
`
`
`
`
`
`
`
`dW!NOILVWYOSNIdV
`
`39NOLLVAYONI
`
`
`
`||!
`
`
`
`(NOLLVYLSIDTYOCOL
`
`NOLLVOLSLLON
`
`NOILVOIdd
`
`roea208AMODALVO039-
`
`WS
`
`dVW3xOO0S
`
`cll
`
`L------~-~---~-----4LINN
`NOLLINDO93MLINNNOILINSOO3Y
`
`NOLLWNLISJLVLS/LNSWSA0W
`
`O01NIHLIMOl}
`
`
`
`YSAYSSLNAWSOVNVA|“049
`
`908|NOILVSINODSLVO
`LINNSDVHOLS|ga!<<|LNA
`
`MNOIAVH3801
`
`90}
`
`
`
`YVONS1VO/AWLL
`
`NOILVAYOSNI
`
`LINNOILISINDODY
`
`v0}
`
`c0}
`
`
`
`YOSNASNOILOW
`
`42
`
`42
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 42 of 53
`
`US 2011/0081634 Al
`
`ee ae eee ere
`
`
`
`|I
`
`6e°Sis
`
`GGFLVISUNOIAWHaaSuasN||—(NOILVOISILON)
` 1|gaINSWS9VNYWdnoUd/Yasn||
`
`
`90€vOe
`gaJ9VSS3WCC0L|
`
`NOONIGNSdadLDULNOILVOISILONSNINYSLSCCNAIH4OLASILON
`
`
`
`NOILVYadONOILVOISILON/NOILVYLSISSYOGOL
`|LINN000140||
`
`4aSNYAHLOdNO¥9OLASILON
`
`
`INAWS9DVNVW00°(NOILVYLSIDY)NOILVYLSIDAY
`
`
`
`4ONOLLYNLISNOIAWHa
`
`
`
`41ASUH/4TASWIH
`
`4ASNOLASILON
`
`43
`
`43
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 43 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`
`
`
`
`
`
`
`NOLLVusdONOILVOISILONNOILATdNOD/NOILVYLSIOSYNOILAIdUWIOD9COL
`
`ondis
`
`90€vOE
`
`||
`
`||
`
`
`
`LNAWSASVNYAOgoL
`
`NOIL31dWO9)
`
`
`
`(NOILVYLSIDSYNOILIIdNOO
`
`|!LINN0q014ONOILVYLSIO’Y
`
`‘\
`
`
`
`(NOILVOISILONNOLLS1dW09)
`
`Y3SNOLASILON
`
`
`
`4TASUYSH/STSSWIH
`
`44
`
`
`
`YOJNOILVOISILONNOLLA1dWO9GNAIY4OLASILON
`
`
`
`
`
`
`
`NOILVYLSIDSYWOHMOLYaSn
`
`INASN339SVHNOLLVOIALLONdNOYSOLAJILON
`
`44
`
`
`

`

`
`
`gdLNSWS9VNVWdNOud/H3asnal:
`Patent Application Publication
`GdJLVLSYNOIAVH3dS.uasnNOLLWOI4ILON°C
`
`
`
`
`
`
`ddS9VSS3W°doLNOILVSINONHONASOC°L
`
`ZIEYaSnaaSSayaav:
`
`
`LINNNOILVYHLSIOSYALAIdWOOLON/ALAIdWO9-
`
`
`
`NOILVYLSIOSYOCOLYNOIAVH3dUSDOL-
`
`NOLLATdNODCOOLdl-
`
`LINN3L31dNOOLON/ALFIdNOO-
`LINNJOVSS3N:
`
`
`
`
`vOE
`
`LpSis
`
`
`
`LINNLNAWSSDVNVYWOCOL
`
`NOILILSd4u-NON/NOILILSdSe-
`
`Apr. 7, 2011 Sheet 44 of 53
`
`US 2011/0081634 Al
`
`LIND
`
`
`
`NOILILAdSu-NON/NOILILAdS4«
`
`
`
`YNOIAVHSdHS99IML*
`
`AOVSSAN:
`
`45
`
`45
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 45 of 53
`
`US 2011/0081634 Al
`
`FIG.42A
`
`| ID FROM WHICH REGISTRATIONI OPERATION OF ToDo
`| NOTIFICATION IS RECEIVED |
`REGISTRATION UNIT
`
`
`
`$302
`
`USER EXISTS
`
`REGISTER DB
`
`5304
`
`TO OPERATION OF ToDo
`SYNCHRONISATION UNIT
`
`
`
`
`
`
`
`
`USER/ GROUP
`MANAGEMENT DB
`
`ToDo MESSAGE DB
`
`46
`
`46
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 46 of 53
`
`US 2011/0081634 Al
`
`FIG.42B
`
`ID FOR WHICH COMPLETION!
`NOTIFICATION IS RECEIVED |
`
`
`
`
`COMPLETION
`REGISTRATION UNIT
`
`
`
`
`
` OPERATION OF ToDo
`
`
`
`REGISTERED ID?
`
`
`ToDo MESSAGE DB
`
`
`SET STATUS OF MESSAGE
`
`
`
`TO "COMPLETE"
`
`TO OPERATION OF ToDo
`SYNCHRONISATION UNIT
`
`47
`
`47
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 47 of 53
`
`US 2011/0081634 Al
`
`FIG.42C
`
`eeeee =
`ToDOo RECENTLY |
`UPDATED ToDo
`|
`
`_—_—_——
`
`~ ||
`
`IIu
`
`USER/ GROUP
`MANAGEMENT DB
`
` THEREIS ToDo
`FOR WHICH NOTIFICATION
`PROCESS IS NOT
`PERFORMED?
`
`w
`
`
`ToDo MESSAGE DB
`
`TO OPERATION OF ToDo
`NOTIFICATION UNIT
`
`48
`
`48
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 48 of 53
`
`US 2011/0081634 Al
`
`FIG.42D
`
`ToDo THAT |
`
`IS NOT NOTIFIED |
`
`
`
`USER'S BEHAVIOUR
`STATE DB
`
` BEHAVIOUR
`STATE OF USER MATCH
`TRIGGER OF ToDo?
`
`USER/ GROUP
`MANAGEMENT DB
`
`NOTIFY USER OF ToDo
`
`49
`
`49
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 49 of 53
`
`US 2011/0081634 Al
`
`
`
`
`
`
`
`
`NYAVdNOLLVALISHNOIAVHSdONIHOLVWHLIMY3S/NOL991SNIASILON403SVO
`
`erdis
`
`LVNOILWAYSSSY-ASILONJ
`
`INVYNVLSSY
`
`LoaiasONIAOW
`
`ONIMYOM
`
`INVYNVLS3SYLVNOILWAYSS3Y
`
`
`
`‘SLNALNOONOILVOISILON
`
`
`
`SNIAOW:NOLLIGNOS
`
`50
`
`50
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 50 of 53
`
`US 2011/0081634 Al
`
`ONIASILON40SSVO
`YSENNNNOONIGNAdSGSYASNSHLTIVOLGOL
`
`
`prdis
`
`
`
`
`
`SNYSALLVdNOLLWNLISYNOIAVH3dONIHOLWAHLIMSY3SN40
`
`
`
`
`
`
`
`
`
`(JTaISSOdOSTVJUV‘O14‘SYSEWAWTV
`
`
`
`
`
`dOONIHOLVW'YSEWAWSNOJOONIHOLYW3X)
`
`ANOAYSAFOLAJILON
`
`AdGAHOLVA
`
`
`
`JIWHNVHLSHOW
`
`
`
`ONIAOW
`
`ONIMYOM
`
`
`
`ONIAOWONIAOW:NOILIGNOO
`
`
`
`‘SLNALNOONOILVOISILON
`
`YANNIGYO4LNOODS.L31
`
`51
`
`51
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 51 of 53
`
`US 2011/0081634 Al
`
`(YDWaseeo—||l
`ébNOWVOMdd¥|=|NOLVONddY
`
`ONIddOHSONiddOHS
`
`eNOILVYOMIddV
`
`SNIddOHS
`
`GP’I4
`
`NIddOHS
`
`
`
`
`
`VLVGYOSNAS‘NOILVWHO-NI
`
`
`
`NOILVY9O1JYINDOV
`
`
`
`RNOIAVHALOALAG
`
`
`
`NYsllVdNOILVALIS
`
`52
`
`
`
`
`
`SNOILVOIIddV¥SONVYYVSY
`
`52
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 52 of 53
`
`US 2011/0081634 Al
`
`9pDIS
`
`fidVWSY¥O9S
`
`
`
`YVONS1V9/SWIL
`
`NOILVWHOSNI
`
`
`
`LINNNOLLISINDOV
`
`aJeMple}}
`
`ObAYO93LV9O59-
`LINNAV1dSIGAV1dSI0
`WSLSASTOYLNODAV1dSIC)WALSASSISATWNVNOILYNLISHNOIAVH3E“OF
`
`
`
`
`
`
`Z0¢09NOLLVWHOANI
`
`
`
`
`NOILINDODSY|.!NOILINDOO3YFIVLS/INSWSAOW[+1]YOSNSSNOLLON
`PNOIAVHaE801|!||SJOSUaS
`NOLLWALIS|ZOl
`oll901
`NOILVO07||-|1039'401!Ol|!
`LINN
`NOILVSINODSLVDYOSNAS
`
`LINN|
`
`
`
`LINNJONLNOD
`
`
`
`dWNOILVAIMOANIdV:
`
`LINN
`
`53
`
`53
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 7, 2011 Sheet 53 of 53
`
`US 2011/0081634 Al
`
`NOILVOINNWKIOD
`NOILOANNOO
`
`LYOd
`
`SOVAYNSLNI
`
`ATIWNYALX3
`WAOWSY
`
`CaLOSNNOO
`ONIGYOOAY
`
`JOIASG
`
`WnIdaw
`
`06vO6069
`
`026816916
`
`LINNLINNLINN
`
`SNIWNYSLXS
`
`ANE||soyuois||indinoLNdNI
`
`806
`
`OL6A990
`
`ZpDis
`
`é
`
`54
`
`54
`
`
`
`
`
`
`
`

`

`US 2011/0081634 Al
`
`Apr. 7, 2011
`
`BEHAVIOUR PATTERN ANALYSIS SYSTEM,
`MOBILE TERMINAL, BEHAVIOUR PATTERN
`ANALYSIS METHOD, AND PROGRAM
`
`BACKGROUNDOF THE INVENTION
`
`1. Field of the Invention
`[0001]
`[0002] The present invention relates to a behaviour pattern
`analysis system, a mobile terminal, a behaviourpattern analy-
`sis method, and a program.
`[0003]
`2. Description of the Related Art
`[0004] A technology for installing a motion sensor on a
`mobile terminal such as a mobile phone and for enabling
`automatic detection and recording of a user’s behaviour his-
`tory is gaining attention. For example, JP-A-2008-3655 dis-
`closes a technology for detecting a walking movement, a
`running movement, a movementof turningleft or right and a
`still state by using a motion sensor such as an acceleration
`sensor and a gyro sensor. This patent document describes a
`methodof calculating a walking speed, a walking power and
`a rotation angle around a gravity axis from output data of the
`motion sensor, and detecting the walking movement,the run-
`ning movement, the movementofturning left or right and the
`still state by using the calculation result. Furthermore, this
`patent document describes a method of detecting a user’s
`behaviour pattern by statistical processing which has, as
`inputs, the pattern of the movementor the state such as the
`type of the movementor the state, the duration of the move-
`mentor the state and the numberof times of the movement.
`
`SUMMARYOF THE INVENTION
`
`[0005] Using the method described above, a behaviourpat-
`tern such as “slow walking” and “busy movement” can be
`obtained as time series data. However, the behaviour pattern
`obtained by this method mainly expressesa relatively short-
`term movementorstate of'a user. Accordingly,it is difficult to
`estimate, from a behaviour pattern history, a specific content
`of a behavioursuchas “today, went shopping at a department
`store”or “yesterday, had a mealat a restaurantin a hotel.” The
`behaviourpattern obtained by using the method described in
`the patent documentdescribed above is an accumulation of
`behaviours performed in a relatively short period of time.
`Also, each of the behaviours constituting the behaviourpat-
`tern is not purposefully performed by a user. On the other
`hand,a specific content of a behaviour is, in many cases, that
`of a behaviour that is purposefully performed by a user and
`that is performed for entertainment over a relatively long
`period of time. Accordingly,it is difficult to know such spe-
`cific content of a behaviour from an accumulation of behav-
`iours performedin a short period of time.
`[0006]
`In light of the foregoing,it is desirable to provide a
`behaviour pattern analysis system, a mobile terminal, a
`behaviour pattern analysis method, and a program, which are
`newand improved, and which are capable of detecting, from
`arelatively short-term behaviour pattern obtained by using a
`motion sensor, a relatively long-term highly-entertaining
`behaviour pattern.
`[0007] According to an embodimentofthe present inven-
`tion, there is provided a behaviour pattern analysis system
`which includes a mobile terminal including a movementsen-
`sor that detects a movementof a user and outputs movement
`information, a current location information acquisition unit
`that acquires information on a current location, a building
`information acquisition unit that acquires information on a
`
`building existing at a location indicated by the information
`acquired by the current location information acquisition unit
`or information on buildings existing at the current location
`and in a vicinity of the current location, a first behaviour
`pattern detection unit that analyses the movement informa-
`tion output from the movement sensor, and detects a first
`behaviour pattern corresponding to the movement informa-
`tion from multiple first behaviour patterns obtained by clas-
`sifying behaviours performed by the user over a relatively
`short period of time, and a transmission unitthat transmits, to
`a server, the information on a building or buildings acquired
`by the building information acquisition unit and the first
`behaviour pattern detected by the first behaviour pattern
`detection unit, and a server including a reception unit that
`receives, from the mobile terminal, the information on a
`building or buildings and the first behaviour pattern, and a
`second behaviour pattern detection unit that analyses the
`information on a building or buildings andthe first behaviour
`pattern received by the reception unit, and detects a second
`behaviour pattern corresponding to the information on a
`building or buildings and the first behaviour pattern from
`mu