(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2011/0122153 A1
`(43) Pub. Date:
`May 26, 2011
`OKAMURA et al.
`
`US 2011 0122153A1
`
`(54) INFORMATION PROCESSINGAPPARATUS,
`INFORMATION PROCESSING METHOD,
`AND PROGRAM
`
`(76) Inventors:
`
`Yuki OKAMURA, Saitama (JP);
`Daisuke Mochizuki, Chiba (JP);
`Akiko Terayama, Tokyo (JP);
`Takuro Noda, Tokyo (JP); Eijiro
`Mori, Tokyo (JP); Takuo Ikeda,
`Tokyo (JP)
`
`(21) Appl. No.:
`
`12/908,779
`
`(22) Filed:
`
`Oct. 20, 2010
`
`(30)
`
`Foreign Application Priority Data
`
`Nov. 26, 2009 (JP) ............................... P2009-268661
`
`
`
`Publication Classification
`
`(51) Int. Cl.
`(2006.01)
`G09G 5/00
`(52) U.S. Cl. ........................................................ 345/629
`(57)
`ABSTRACT
`An information processing apparatus includes a transformed
`coordinate calculating section that calculates transformed
`coordinates for each of a plurality of Superimposed images
`associated with coordinates in a background image, by trans
`forming coordinates of other Superimposed images with
`respect to one Superimposed image as a reference image in
`Such a way that coordinate intervals within a predetermined
`area with respect to the reference image become denser with
`increasing distance from the reference image toward the
`boundary, a coordinate setting section that sets coordinates of
`the reference image on the basis of a mean value obtained by
`calculating a mean of the calculated coordinates of the other
`Superimposed images with respect to the reference image,
`and a display control section that displays the background
`image and the plurality of Superimposed images on a display
`section in Such a way that the reference image is placed at the
`set coordinates in the background image.
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 1 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 1 of 66
`
`US 2011/0122153 A1
`
`
`
`FACE
`CLUSTER
`GENERATING
`SECTION
`
`ATRIBUTE
`INFORMATION
`ACQUIRING
`SECTION
`
`TREE
`GENERATING
`SECTION
`
`HIERARCHY
`DETERMINING
`SECTION
`
`TREE
`RESTRUCTURING
`SECTION
`
`CONTENT
`STORING
`SECTION
`
`EVENT
`CLUSTER
`GENERATING
`SECTION
`
`MAP
`INFORMATION
`STORING
`SECTION
`
`CLUSTER
`INFORMATION
`GENERATING
`SECTION
`
`CONDITION
`SETTING
`SECTION
`
`ADDRESS
`INFORMATION
`STORING
`SECTION
`
`CLUSTER
`INFORMATION
`STORING
`SECTION
`
`OPERATION
`ACCEPTING
`SECTION
`
`DISPLAY
`SECTION
`
`DISPLAY
`CONTROL
`SECTION
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 2 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 2 of 66
`
`US 2011/0122153 A1
`
`
`
`ATTACHED
`INFORMATION
`
`IMAGE
`INFORMATION
`
`213
`
`ATTRIBUTE
`INFORMATION
`GPS
`INFORMATION
`DATE ANDTIME
`OF SHOOTING
`UPDATE
`PICTURE SIZE
`COLOR SPACE
`INFORMATION
`MAKERNAME
`
`214
`
`MAKER NOTE
`AREA
`
`( ExENSON)
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 3 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 3 of 66
`
`US 2011/O122153 A1
`
`FIG. 3
`
`230
`ar
`
`231
`
`
`
`N SE
`LATITUDE: OOOO'
`LONGITUDE: OOOO'
`
`232
`
`ADDRESS INFORMATION
`
`TOKYO-PREFECTURE SHINAGAWAWARD
`KITASHINAGAWA3-CHOME
`
`LATITUDE: OOOO'
`LONGITUDE: OOOO'
`
`TOKYO-PREFECTURE SHINAGAWAWARD
`KITASHINAGAWA3-CHOME
`OOCITY SHINAGAWA
`
`LATITUDE: OOOO'
`LONGITUDE: OOOO'
`
`TOKYO-PREFECTURE SHINAGAWAWARD
`OSAK 1-CHOME OOCITY OSAK WT
`
`LATITUDE: OOOO'
`LONGITUDE: OOOO'
`
`TOKYO-PREFECTURE SHINAGAWAWARD
`OSAK 1-CHOME GOTENYAMAOOTOWER
`
`LATITUDE: OOOO'
`LONGITUDE: OOOO'
`
`TOKYO-PREFECTURE SHINAGAWAWARD
`OSAK 1-CHOME OO COMPANY
`
`LATITUDE: OOOO'
`LONGITUDE: OOOO'
`
`TOKYO-PREFECTURE SHINAGAWAWARD
`OSAK 1-CHOME
`OO COMPANYTECHNICAL CENTER
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 4 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 4 of 66
`
`US 2011/O122153 A1
`
`251
`
`FIG. 4
`252
`
`253
`
`
`
`PREFECTURE
`MUNICIPALTY
`CHOME/BANCH
`TOKYOPREFECTURESHINAGAWAWARDKASHINAGAWA3CHOME
`2 TOKYOPREFECTUREESHINAGAWAWARDKITASHINAGAWA3CHOME
`TOKYOPREFECTURESHINAGAWAWARD
`TOKYOPREFECTUREESHINAGAWAWARDKITASHINAGAWA3CHOME
`FOSAK1-CHOME
`
`7
`
`254
`
`BUILDINGNAMEETC,
`
`(OOCTYOSAKW-258
`
`9 TOKYOPREFECTURESHINAGAWAWARD
`10 TOKYOPREFECTURESHINAGAWAWARD
`
`OSAKCHOME
`OSAKCHOME
`
`OOCTYOSAKWT
`OOCTYOSAKWT-258
`
`
`
`
`
`
`
`
`
`
`
`
`
`QOCITYOSAKIWI
`OOCTYOSAKWI-258
`
`OOCTYOSAKW-258
`OOCOMPANY
`OOCTYOSAK
`
`OSAKCHOME
`OSAKCHOME
`OSAKCHOME
`OSAKCHOME
`OSAKCHOME
`OSAKCHOME
`
`OSAKCHOME IOOCOMPANYTECHNICALCENTER
`
`14 TOKYOPREFECTURE
`
`18 TOKYOPREFECTUREESHINAGAWAWARD
`19TOKYOPREFECTURESHINAGAWAWARD
`20 TOKYOPREFECTURESHINAGAWAWARD
`21 TOKYOPREFECTURESHINAGAWAWARD
`23 TOKYOPREFECTUREESHINAGAWAWARD
`24ETOKYOPREFECTUREESHINAGAWAWARD
`25ETOKYOPREFECTURE
`26 TOKYOPREFECTURE
`27TOKYOPREFECTURE;
`28 TOKYOPREFECTURESHINAGAWAWARD
`
`31 TOKYOPREFECTURESHINAGAWAWARD
`
`
`
`
`
`- - - - - - - - - - - us--- ---------- us--
`
`--------- v---
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 5 of 109
`
`

`

`Patent Application Publ
`ion
`icat
`
`May 26
`, 2011 Sheet 5 of 66
`
`US 2011/0122153 A1
`
`
`
`872
`
`}}ELSTATO
`LNERHWd
`
`OO OO:EGn|LIONOT
`,OO OO (BODLJIVT
`
`NO||WW?JOHN||
`(NOLLISOd HEINEO)
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 6 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 6 of 66
`
`US 2011/O122153 A1
`
`F.G. 6A
`
`(x2, y2)
`312
`
`F.G. 6B
`
`313
`
`
`
`
`
`y/ 304
`
`306
`d2
`
`305
`
`314
`
`308
`
`FIG. 6C 316
`
`315
`O d3EO
`
`FIG. 6D
`318
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 7 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 7 of 66
`
`US 2011/O122153 A1
`
`FIG. 7
`
`381
`
`382
`
`383
`
`384
`
`385
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 8 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 8 of 66
`
`US 2011/O122153 A1
`
`
`
`FIG.8
`
`essessssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 9 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 9 of 66
`
`US 2011/0122153 A1
`
`FIG. 9
`
`an
`
`A-...-------------------ear-----------------re-resear-ass---------
`re-sous
`resea----------
`--------------
`
`n-a-sa-----issor assoss anot
`
`s
`
`"----------------------------------" "
`sonosaess-eeeee-eares--"
`
`8-|--
`-
`---,
`32 Fr.
`325
`"324,
`322 "323
`321
`is
`is
`is
`La La
`at
`(2) () (2) (G10 (9) (3) (5) (12) (G) () (13) () (6) (8)
`-y- N-N- Y
`Y
`386
`387
`388
`389
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 10 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 10 of 66
`
`US 2011/0122153 A1
`
`F.G. 10
`
`---------------------------ass-"
`
`()() ()() ()() ()() ()() () (2 ge
`tly t; td3 to
`tis tle ti, td8 tg tdio td11 to 12 tdia
`\ y \-y-N-y-N-y-/-y-
`381, id1
`382, id2
`383, id3
`384, id4
`385, id5
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 11 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 11 of 66
`
`US 2011/0122153 A1
`
`FIG. 11A
`
`FIG. 11D
`
`NUMBER OF
`CONTENTS
`
`NUMBER OF
`CONTENTS
`
`id1
`
`d2 id3 id4 id5 EVENT
`ID
`
`id1
`
`d2 id3 id4 id5 EVENT
`ID
`
`FIG. 11B
`
`FIG. 11E
`
`NUMBER OF
`CONTENTS
`
`NUMBER OF
`CONTENTS
`
`id 1 id2 id3 id4 id5 EVENT
`ID
`
`id 1 id2 id3 id4 id5 EVENT
`ID
`
`FIG. 11C
`
`FIG. 11F
`
`NUMBER OF
`CONTENTS
`
`
`
`NUMBER OF
`CONTENTS
`
`id 1 id2 id3 id4 id5 EVENT
`
`id 1 id2 id3 id4 id5 EVENT
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 12 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 12 of 66
`
`US 2011/0122153 A1
`
`FIG. 12A
`
`NUMBER OF
`CONTENTS
`
`NUMBER OF
`CONTENTS
`
`id1 id2 id3 id4 d5 EVENT
`ID
`
`id1 id2 id3 id4 id5 EVENT
`ID
`
`FIG. 12B
`
`NUMBER OF
`CONTENTS
`
`
`
`NUMBER OF
`CONTENTS
`
`id1 id2 id3 id4 id5 EVENT
`
`id1 id2 id3 id4 id5 EVENT
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 13 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 13 of 66
`
`US 2011/0122153 A1
`
`FIG. 13A
`
`386
`
`387
`
`388
`
`389
`
`RESTRUCTURING OF TREE
`
`FIG. 13B
`
`
`
`386
`
`387
`
`388
`
`389
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 14 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 14 of 66
`
`US 2011/O122153 A1
`
`F.G. 14
`
`171
`
`172
`
`CLUSTERDIAMETER (km)
`
`MAP SCALE
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 15 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 15 of 66
`
`US 2011/0122153 A1
`
`
`
`Sys
`
`FIG. 15A
`
`NSCAX
`
`1.
`
`s:
`
`/
`
`{
`
`N s
`
`-
`N1i
`/2S K /SZ AN
`
`1.
`
`X
`
`RN a
`
`y
`
`2
`
`M y Q
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 16 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 16 of 66
`
`US 2011/O122153 A1
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 17 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 17 of 66
`
`US 2011/O122153 A1
`
`FIG. 17
`
`401
`
`INDEX SCREEN
`
`MODE
`SWITCHING
`
`MODE v 7
`swen
`
`
`
`402
`
`CONTENT PLAYBACK
`SCREEN
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 18 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 18 of 66
`
`US 2011/O122153 A1
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 19 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 19 of 66
`
`US 2011/O122153 A1
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 20 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 20 of 66
`
`US 2011/0122153 A1
`
`
`
`S.s
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 21 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 21 of 66
`
`US 2011/O122153 A1
`
`
`
`S
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 22 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 22 of 66
`
`US 2011/O122153 A1
`
`FIG. 22
`
`
`
`442
`
`441
`
`443
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 23 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 23 of 66
`
`US 2011/O122153 A1
`
`FIG. 23
`
`451
`
`
`
`452
`
`450
`
`453
`
`FACE DATE:29.07.2008 TIME: 19:23 - 22:49
`
`442
`
`441
`
`443
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 24 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 24 of 66
`
`US 2011/O122153 A1
`
`FIG. 24
`
`451
`
`
`
`452
`
`460
`
`453
`
`442
`
`441
`
`443
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 25 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 25 of 66
`
`US 2011/O122153 A1
`
`FIG. 25
`
`451
`
`
`
`452
`
`465
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 26 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 26 of 66
`
`US 2011/0122153 A1
`
`FIG. 26
`
`451
`
`
`
`452
`
`470 453
`
`454
`
`442
`
`441
`
`472
`
`419 43
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 27 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 27 of 66
`
`US 2011/0122153 A1
`
`
`
`480
`
`FIG. 27A
`
`/\
`
`482
`/
`
`LOCATION
`YUKIGAYAOTSUKASTATION
`ANSWERE"" 106 photos VSC
`DOWNTOW
`WALK
`:
`:
`D
`OTASTATION
`()
`g
`AND WICINITY
`TOKYO OOLAND
`AND WICINITY
`NARITAAIRPORT
`AND WICINITY
`KRYUGAOKAZOO
`/25 3.
`AND WICINITY
`y /
`() A
`JOMOSANZAN PANORAMA
`SWAWARS WRNW" 7photos v X //\ ZIAN
`
`547 photos
`8 photos
`
`FIG. 27B
`
`481
`
`480
`
`483
`
`
`
`
`
`A
`
`24 photos
`- 09/01/2008
`18 photos
`N - 09/02/2009
`KRYUGAOKAZOO
`AND WICINITY
`JOMOSANZAN PANORAMA
`HIGHWAY AND WICINITY
`
`
`
`487
`
`YY2SS(7//
`
`SXSh
`
`:
`
`NSW
`
`
`
`AN. W.
`2 C; VW
`A. (2 N
`1. V
`/ /X I
`y
`28 //NZ AN
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 28 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 28 of 66
`
`US 2011/O122153 A1
`
`FIG. 28
`
`START
`
`S901
`
`
`
`
`
`HAS INSTRUCTING
`OPERATION FOR GENERATING
`CLUSTER INFORMATION BEEN
`PERFORMED?
`
`AcquireATTRIBUTEINFORMATIONs'
`
`YES
`
`OF CONTENTS
`
`TREE GENERATION PROCESS
`
`S910
`
`EVENT CLUSTERGENERATION is
`PROCESS
`
`HIERARCHY DETERMINATION PROCESS
`
`TREE RESTRUCTURINGPROCESS
`
`S970
`
`S990
`
`GENERATE CLUSTER MAPS
`CORRESPONDING TO
`GENERATED CLUSTERS
`
`STORE GENERATED
`CLUSTER INFORMATION
`
`S904
`
`S905
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 29 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 29 of 66
`
`US 2011/O122153 A1
`
`FIG. 29
`
`so
`er 1
`HIERARCHY DETERMINATION PROCESS
`
`SET EVENTID'S WITH RESPECT TO
`INDIVIDUAL CONTENTS
`
`S971
`
`CALCULATE FREQUENCYDISTRIBUTION is S972
`OF EVENTID'S WITH RESPECT TO
`EACH OF NODES
`
`CALCULATE LINKAGE SCORES OF
`EACH PARENT NODEON BASIS OF
`FREQUENCYDISTRIBUTION OF CHILD NODES
`
`S973
`
`SELECT ONE NODE BEGINNING WITH
`NODESATUPPERLEVELS, AND
`SET THIS NODEASTARGET NODE
`
`S974
`
`S975
`
`LINKAGE SCORES
`CLINKAGETHRESHOLD
`th92
`
`S977
`EXCLUDE
`TARGETNODE | DETERMINETARGETNODEAS EXTRACTION NODE,
`FROMNODES
`AND EXCLUDETARGET NODE AND
`TO BESELECTED ITS CHILD NODES FROMNODESTO BE SELECTED
`
`S978
`
`DOES ANOTHER
`NODE TO BE SELECTED
`EXIST?
`
`
`
`
`
`NO
`
`S979
`
`GENERATE TREE FORMED BY EXTRACTION NODES
`
`RETURN
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 30 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 30 of 66
`
`US 2011/O122153 A1
`
`FIG. 30
`
`S990
`
`
`
`TREE RESTRUCTURING PROCESS
`
`S991
`
`
`
`IS NUMBER OF
`CHILD NODES BELONGING TO
`TARGET NODE EQUAL TO OR
`SMALLERTHAN 12
`
`YES
`
`EXTRACT PAR WITHSMALLEST
`CLUSTER-TO-CLUSTER DISTANCE
`FROMAMONG PARS OF CHILD NODES
`BELONGING TO TARGET NODE
`
`
`
`DOES EXTRACTED
`PARSATISFY CONSTRAINT
`
`
`
`MERGE THAT PAR
`INTO SINGLE NODE
`
`RETURN
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 31 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 31 of 66
`
`US 2011/O122153 A1
`
`FIG. 31
`
`
`
`S1001
`
`HASCONTENT
`PLAYBACKINSTRUCTING OPERATION
`BEENPERFORMED?
`YES
`DISPLAY INDEX SCREEN OF CLUSTERMAPS
`
`S1002
`
`S1003
`
`SWITCH INDEX SCREENS1004
`N EEE
`SWITCHINGOP
`
`NO
`
`HASSWITCHING
`OPERATION OF INDEX SCREEN BEEN
`PERFORMED
`NO
`
`S1005
`
`HASSCROLLOPERATION
`BEEN PERFORMED
`YES
`SWITCH INDEX SCREEN INACCORDANCE
`WITHSCROLLOPERATION
`
`S1006
`
`S1007
`
`HASSELECTING
`OPERATION OF INDEXIMAGE BEEN
`PERFORMED
`S1008
`YES
`DISPLAYINFORMATION RELATED TO CLUSTER
`CORRESPONDING TO THAT INDEX IMAGE
`S1009
`
`HAS DETERMINING OPERATION
`BEEN PERFORMED
`S1020
`YES
`CONTENT PLAYBACKSCREEN DISPLAY PROCESS
`S1010
`
`HASCONTENT
`PLAYBACKENDING OPERATION BEEN
`PERFORMED2
`YES
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 32 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 32 of 66
`
`US 2011/O122153 A1
`
`FIG. 32
`
`S1020
`er
`CONTENTPLAYBACKSCREEN DISPLAY PROCESS
`
`S1021
`
`HAS OPERATIONAL
`INPUT BEEN MADE
`
`S1022
`
`ATACHFACEBOXES
`TOFACES
`
`
`
`
`
`
`
`S1023
`
`DISPLAY CONTENT
`INFORMATIONAND
`OPERATIONASSISTANCE
`INFORMATION
`
`S1024
`
`YES
`
`HAS DISPLAY
`SWITCHING OPERATION
`TOINDEX SCREENBEEN
`PERFORMED
`
`
`
`RETURN
`
`ARE CONTENT
`INFORMATIONAND
`OPERATIONASSISTANCE
`INFORMATION
`DISPLAYED
`
`HASNO
`OPERATIONAL
`INPUT BEEN MADE WITHIN
`PREDETERMINED
`
`YES
`
`DISPLAYNEXTCONTENT
`
`OPERATIONAL
`INPUT BEEN MADE WITHIN
`PREDETERMINED
`TIME PERIOD)
`
`
`
`NO
`ERASE FACEBOXES
`
`
`
`ERASE CONTENTINFORMATIONAND
`OPERATIONASSISTANCE INFORMATION
`
`NO
`
`S1032
`
`DISPLAYEVENICONS
`
`S1031
`
`
`
`PLAYBACK
`SCREEN FOREVENT
`CLUSTER
`
`S1033
`
`
`
`PLAYBACK
`SCREEN FOR POSITION
`CLUSTER
`
`NO
`
`S1034
`
`DISPLAYPOSITION CONS
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 33 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 33 of 66
`
`US 2011/O122153 A1
`
`FIG. 33
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`HASSELECTING
`OPERATION ON
`EVENTCON BEEN
`PERFORMED
`
`
`
`S1041
`YES
`DISPLAYINFORMATION
`RELATED TO EVENT CLUSTER
`S1042
`CHANGEEVENTICONTO EVENTS
`REPRESENTATIVE IMAGE AND
`DATE AND TIME INFORMATION
`
`
`
`
`
`()
`
`S1035
`
`HASSELECTING
`OPERATION ON FACE BEEN
`PERFORMED?
`YES
`DISPLAYINFORMATION ON
`FACECLUSTERRELATED TO THATFACE
`
`DISPLAYIMAGE OF WICINITY OF
`THATFACE IN MAGNIFIED FORM
`S1038
`
`S1036
`
`S1037
`
`HASDETERMINING
`OPERATION BEEN
`PERFORMED
`YES
`DISPLAYPLAYBACKSCREEN FOR
`FACE CLUSTERRELATED TODETERMINED FACE
`
`S1039
`
`
`
`HASSELECTING
`OPERATION ON POSITIONICON
`BEEN PERFORMED
`YES
`DISPLAYINFORMATION
`RELATED TO POSITION CLUSTER
`
`CHANGE POSITIONICON
`TO CLUSTERMAP
`
`
`
`
`
`HASDETERMINING
`OPERATION BEEN
`PERFORMED?
`
`
`
`DISPLAYPLAYBACKSCREEN
`FORDETERMINEDEVENT CLUSTER
`
`HASDETERMINING
`OPERATION BEEN
`PERFORMED?
`
`DISPLAYPLAYBACKSCREEN FOR
`DETERMINED POSITION CLUSTER
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 34 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 34 of 66
`
`US 2011/O122153 A1
`
`FIG. 34
`
`600
`
`MAP
`INFORMATION
`STORING
`SECTION
`
`BACKGROUND
`MAP
`GENERATING
`SECTION
`
`CONTENT
`STORING
`SECTION
`
`BACKGROUND
`MAP
`INFORMATION
`STORING SECTION
`
`
`
`CLUSTER
`INFORMATION
`STORING
`SECTION
`
`DISPLAY
`CONTROL
`SECTION
`
`MAGNIFICATION/
`SHRINKAGE
`PROCESSING
`SECTION
`
`
`
`
`
`COORDINATE
`CALCULATING
`SECTION
`
`DISPLAY
`SECTION
`
`RELOCATION
`PROCESSING
`SECTION
`
`
`
`
`
`
`
`NON-LINEAR
`ZOOM
`PROCESSING
`SECTION
`
`OPERATION
`ACCEPTING
`SECTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 35 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 35 of 66
`
`US 2011/O122153 A1
`
`N
`
`
`
`DIDIIDILLINS-ZIZDILIITTTTTTTTTTTT
`-L
`EEHHHHHHHHEEEEEEEEEE|
`
`PAHHHHHHHHHHP2=#EEEEEE|
`
`LZILaT| | | | |
`eue KolHHH
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 36 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 36 of 66
`
`US 2011/O122153 A1
`
`FIG. 36
`
`763
`v
`
`Utsunomiy
`
`Hitachi
`
`Hitachiomiya
`Mito
`
`m
`
`Maebashi Kiryu
`
`-
`
`SanO
`
`Saku
`
`Takasaki
`
`Ota
`
`Chichibu
`
`Tsukuba
`
`Saitama
`
`... OKamisu
`Kashiwa NaritaKator
`764
`
`Sanmu
`
`s
`CyOshi
`
`Mobara
`
`Izumi
`
`MinamibOSO Kamogawa-
`
`| | | |
`
`Tokyo
`Sagamihara %c?
`Yokosuk
`Skitsu
`
`Kofu
`O
`
`Hachioji
`
`Fujinomiya Gotenba
`Full
`ul
`Numazu Ito
`Shizuoka
`
`Odawara
`
`Shimoda
`
`-
`
`--
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 37 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 37 of 66
`
`US 2011/0122153 A1
`
`FIG. 37
`
`763
`t/
`
`Utsunomiy
`Hitachiomiya
`Mito
`
`Hitachi
`
`.
`Maebashi-Kyu
`SanO
`Takasaki IOta
`
`Saku
`
`----
`
`Chichibu
`
`Tsukuba
`
`765
`
`Kofu
`
`Hachioji
`
`r
`Saitama
`
`LKator
`Kashiwa.
`N
`Narita
`764
`
`Kamisu
`
`Samy
`
`Cyoshi
`
`O
`
`Mobara
`.
`Izumi
`
`Kamogawa
`
`H
`
`-
`
`-
`
`Tokyo
`s soka
`Yokosuka1.
`Fujinomiya Gotenbaab.Odawara Ny) Kimitsu
`e- Fuji
`A E
`|
`-
`HNInazuEto
`Shizuoka
`
`Minamibo
`
`-
`
`H H
`Shimodal
`H -
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 38 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 38 of 66
`
`US 2011/O122153 A1
`
`
`
`FIG
`
`38
`
`-------------
`
`------
`---.
`
`--
`-------.
`
`-----+-----+------ i.
`
`--#-----+-----+-----+-----
`
`;-----+-----
`++
`
`3;
`
`i
`... 1...i.e.
`---. -----
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 39 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 39 of 66
`
`US 2011/0122153 A1
`
`
`
`FIG
`. 39
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 40 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 40 of 66
`
`US 2011/0122153 A1
`
`ITT
`||
`|
`|
`|
`|
`II ITTTTTTTTTTTTTTTTTTTT
`2VSNW
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`al
`DNA-
`S.
`NEN
`NRIN
`N
`
`
`
`
`
`CN NIII
`
`N
`
`
`
`ESH
`A.
`Ni
`N2N
`INI L II
`NY H
`g
`Sa N
`2N ANN TH
`sia?
`2
`iNSA
`cities
`Yg
`SiiW
`is:
`
`s
`
`
`
`TA
`4
`VH
`A\|
`VIII
`a
`NHANS NNN
`NNY and
`II
`NN
`NN N\,
`H
`S. His
`Sis
`HHNiSAS4222ll
`S.
`CD HHGHH
`
`V
`
`re
`
`
`
`
`
`
`
`
`
`
`
`
`
`TI ITT I ITT/TTTT
`
`VIN
`TVYN
`.
`A
`aul
`H
`HA
`WN AI-Hi
`RN/
`QUIEH
`H
`s(HH
`HE
`H
`| YH
`.../
`H H
`H RS-LH
`HE
`SYH1
`N H
`
`v
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TT
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 41 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 41 of 66
`
`US 2011/O122153 A1
`
`3
`
`
`
`O
`d
`
`c
`2
`
`N
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 42 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 42 of 66
`
`US 2011/O122153 A1
`
`FIG. 42
`
`
`
`733
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 43 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 43 of 66
`
`US 2011/O122153 A1
`
`
`
`FIG. 43A
`
`
`
`DW1
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 44 of 109
`
`

`

`FG, 44A
`
`FIG. 4.4B
`
`
`
`\S(C)
`N s
`(K
`MC/U
`y
`Na1
`YY S1,
`(e 2. e
`2 SC3. 803
`O23 VA ( /\\
`Y-Azér
`-------
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 45 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 45 of 66
`
`US 2011/O122153 A1
`
`FIG. 45
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 46 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 46 of 66
`
`US 2011/0122153 A1
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 47 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 47 of 66
`
`US 2011/0122153 A1
`
`
`
`sis as:
`
`a
`
`wn
`
`:
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 48 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 48 of 66
`
`US 2011/O122153 A1
`
`
`
`FIG. 48A
`
`833
`
`830
`
`842
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 49 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 49 of 66
`
`US 2011/0122153 Al
`
`918
`
`MalA
`
`
`
`AvidNOLLWusdO
`
`N3SYOS
`ONITISONVO
`
`NOLLVYadO
`
`ONINIWY3130
`
`'''''‘1''''‘'''‘'1!'’'’'''''''!'(‘''|'''l''''fi111!1''’!'I'1''''
`
`E18
`
`salietlaeicaice6r‘SIs
`
`AMEE,
`Sb8LB
`Ie118
`
`YSLIVOSYALIVOS
`
`N33YOSN33Y4OS
`N3au9SNaHS
`
`MalANalA
`
`MaIAMat
`
`NOLLVH3dO_ONITTIONYOONIHOLIMS
`ONLoaTas|300MVV
`
`
`NOILWYAdOONIDNVHOSGNLIONOT/SGNLILVT
`
`
`NOILWY3dOONIDNVHO31VOS
`
`''('tIt’'
`
`dV
`
`NOILVaadO
`ONIHOLIMS
`
`COW
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 50 of 109
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 50 of 109
`
`
`
`
`
`

`

`Patent Application Publication May 26, 2011 Sheet 50 of 66
`
`US 2011/O122153 A1
`
`FIG. 50
`
`
`
`890
`
`wa.
`
`sessessessessessessess
`
`II B
`
`E.
`
`abs
`
`-------
`893
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 51 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 51 of 66
`
`US 2011/0122153 A1
`
`FIG. 51
`
`
`
`ACQUIRE
`CLUSTER INFORMATION
`
`S1101
`
`GENERATE
`CLUSTER WIDEAREA MAP
`ON BASIS OF ACQUIRED
`CLUSTER INFORMATION
`
`S1102
`
`S1103
`
`HAS GENERATION OF
`CLUSTER WIDEAREA MAP
`BEENFINISHED FOR
`EVERY CLUSTER
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 52 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 52 of 66
`
`US 2011/O122153 A1
`
`FIG. 52
`
`
`
`START
`
`- HASCONTENT PLAYBACK
`
`INSTRUCTING OPERATION BEEN
`PERFORMED?
`
`
`
`YES
`DISPLAY MAPW
`IEWSCRE
`EN
`
`S1112
`
`
`
`
`
`S1113
`
`HAS MODE
`SWITCHING OPERATION BEEN
`PERFORMED?
`
`YES
`
`S1114
`
`
`
`ISMAPVIEWSCREEN
`DISPLAYED?
`WEs
`S1116
`DISPLAYSCATTERVIEWSCREEN
`SCATTERVIEWPROCESS
`S1117
`
`ISOPERATIONMODE
`SWITCHING
`
`s1118
`
`ISOPERATION
`DETERMINING OPERATION
`ON CLUSTER MAP
`
`S1124
`
`HASCONTENT
`PLAYBACKENDING OPERATION
`BEEN PERFORMED
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DISPLAYMAPVIEWSCREEN
`
`MAP VIEWPROCESS
`
`DISPLAYPLAYVIEWSCREEN
`
`PLAY VIEW PROCESS
`
`HASCANCELLING
`OPERATION OF PLAY
`VIEWSCREEN BEEN
`PERFORMED?
`
`DISPLAYSCREEN DISPLAYED
`ATIMMEDIATELY PREVIOUS
`DETERMINING OPERATION
`
`ISMAP
`VIEWSCREEN
`DISPLAYED?
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 53 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 53 of 66
`
`US 2011/O122153 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Yn
`MAP VIEW PROCESS
`
`S1131
`
`GENERATE BACKGROUND MAPON BASIS OF PIECES OF CLUSTERINFORMATION
`
`CALCULATE COORDINATES OF CLUSTER MAPS
`CORRESPONDING TO GENERATED BACKGROUND MAP
`
`NON-LINEARZOOMPROCESS
`
`DISPLAY CLUSTER MAPS IN OVERLAD MANNER ON BACKGROUND MAP
`
`S1132
`
`S1150
`
`S1133
`
`HAS MOVE/
`SCALE-CHANGE
`OPERATION ON MAPBEEN
`PERFORMED?
`GENERATE BACKGROUND MAP
`NACCORDANCE WITH
`OPERATION PERFORMED
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`YES
`
`
`
`
`
`HASSELECTING
`OPERATION ON CLUSTERMAP
`BEEN PERFORMED?
`
`
`
`HAS DESELECTING
`OPERATION ON CLUSTERMAP
`BEEN PERFORMED?
`
`NO
`
`S1140
`
`HASDETERMINING
`OPERATION ON CLUSTER MAP
`BEEN PERFORMED?
`
`HAS MODE
`SWITCHING OPERATION BEEN
`PERFORMED?
`YES
`RETURN
`
`DISPLAY CONTENT
`LISTINGDISPLAY AREA
`
`DELETE CONTENT
`LISTINGDISPLAY AREA
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 54 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 54 of 66
`
`US 2011/O122153 A1
`
`F.G. 54
`
`S1150
`er-1
`
`NON-LINEARZOOMPROCESS
`
`SELECT ONE CLUSTER MAPAND
`SET THIS CLUSTER MAPAS CLUSTER MAP
`
`S1151
`
`SET CENTERPOSITION OF CLUSTER MAPAS FOCUS
`CALCUAETRANSFORMED COORDINAIESPEij
`is S5
`WITH RESPECT TO CLUSTER MAPEXISTING
`INTRANSFORMATION TARGET AREA
`
`S1152
`
`HAS CALCULATION OF
`TRANSFORMED COORDINATES BEEN
`FINISHED WITH RESPECT TO EVERY CLUSTER MAP
`EXISTING INTRANSFORMATION
`TARGET AREA?
`
`S1154
`
`YES
`
`SELECT ONE CLUSTER MAP AND
`SET THIS CLUSTER MAPAS CLUSTER MAP
`
`CALCULATE MEAN OF
`CALCULATEDTRANSFORMED COORDINATESPEij
`
`SET COORDINATES OF CLUSTER MAP
`ON BASIS OF CALCULATED MEAN OF PEij
`
`S1155
`
`S1156
`
`S1157
`
`S1158
`
`
`
`HAS SETTING OF
`COORDINATES BEENFINISHED FOR
`EVERY CLUSTER MAP?
`YES
`RETURN
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 55 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 55 of 66
`
`US 2011/O122153 A1
`
`FIG.55 s
`
`Y^
`SCATTERVIEW PROCESS
`
`CALCULATE COORDNATES OF
`CLUSTER MAPSON BASIS OF
`PIECES OF CLUSTER INFORMATION
`NONLINEARZOOMPROCESS
`FORCE-DIRECTED
`RELOCATION PROCESS
`
`MAGNIFICATION/SHRINKAGE PROCESS
`
`DISPLAYCLUSTER MAPS
`
`S1161
`
`S1150
`
`S1170
`
`S1162
`
`S1163
`
`
`
`
`
`
`
`S1136
`
`HASSELECTING
`OPERATION ON CLUSTER MAP
`BEEN PERFORMED?
`
`YES
`
`S1137
`
`DISPLAY CONTENT
`LISTINGDISPLAY AREA
`
`
`
`S1138
`
`HAS DESELECTING
`OPERATION ON CLUSTERMAP
`BEEN PERFORMED?
`
`YES
`
`S1139
`
`ERASE CONTENT
`LISTING DISPLAY AREA
`
`
`
`
`
`
`
`
`
`
`
`YES
`
`
`
`HAS DETERMINING
`OPERATION ON CLUSTER MAP
`BEENPERFORMED?
`
`HAS MODE
`SWITCHING OPERATION BEEN
`PERFORMED?
`
`YES
`RETURN
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 56 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 56 of 66
`
`US 2011/0122153 A1
`
`FIG. 56
`
`S1170
`er 1
`
`FORCE-DIRECTED RELOCATION PROCESS
`
`SELECT ONE CLUSTER MAPAND
`SET THIS CLUSTER MAPAS CLUSTER MAP
`CALCULATEREPULSIVE FORCEVECTORSF
`EXERTED ON CLUSTER MAPiFROMCLUSTER MAPSj
`
`CALCULATE MEAN OFREPULSIVE FORCE VECTORSF
`AS REPULSIVE FORCE VECTORF ON CLUSTERMAPi
`
`S1171
`
`sist172
`
`S1173
`
`S1175
`
`SUBSTITUTE REPULSIVE FORCE
`VECTORFBYK(FMF)
`
`HAS EVERY
`CLUSTERMAPBEEN
`SELECTED2
`YES
`SELECT ONE CLUSTER MAPAND
`SET THIS CLUSTER MAPAS CLUSTER MAP
`
`ADDREPULSIVE FORCE VECTORFTO
`CALCULATED COORDINATES OF CLUSTER MAP
`
`S1177
`
`S1178
`
`S1179
`
`S1180
`
`HAS EVERY
`CLUSTER MAPBEEN
`SELECTED?
`YES
`ISIFCth 11
`WITH RESPECT TO EVERY
`REPULSIVE FORCE
`VECTORF?
`YES
`RETURN
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 57 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 57 of 66
`
`US 2011/0122153 Al
`
`FIG. 57A
`
` toonat#ir“a|evsedevssnccreeeeiiPCaJehoei1I!.iFATTrrreternotober11'robot111|'a|1aeaiIi1dITTTrrrprspete
`
`toccor1111i1I11logiIt|Iii1tI'1Iory1rastersatS@Onto==rii11I1
`-4----4----4----4--4bei'1{11'111111!1{1'!I1I1111111-d4e---d----4--~-4----4---pg!!11t!«>(-).1i1i|I1'A|rok!!TAaanf4tttt1oof|toy|!!'Io|1#y''|1-|1yt1|i1fo||'!KeAger
`4tcomnttomo
` aNKe)OLL
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 58 of 109
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 58 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 58 of 66
`
`US 2011/O122153 A1
`
`FIG. 58A
`501 -G).
`
`504
`
`502
`
`(b)
`
`503
`
`(c)
`
`FIG. 58B
`
`
`
`
`
`
`
`
`
`
`
`DobAND cle SC
`INCLUDED INCLUDED
`OVERLAP"A:" "A
`false
`false
`true
`false
`true
`false
`true
`false
`false
`true
`false
`true
`true
`true
`false
`true
`true
`true
`
`CASE
`
`
`
`2
`4
`5
`6
`7
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 59 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 59 of 66
`
`US 2011/O122153 A1
`
`FIG. 59A
`CASEO
`
`FIG. 59B
`CASE 1
`
`b
`Gn)
`
`(n)
`
`FIG. 59C
`CASE 2
`
`FIG. 59E
`CASE 4
`250
`
`250
`
`O
`
`(n)
`FIG. 59G
`CASE 6
`
`O
`
`FIG. 59D
`CASE 3
`
`undefined
`
`FIG. 59F
`CASE 5
`
`FIG. 59H
`CASE 7
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 60 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 60 of 66
`
`US 2011/O122153 A1
`
`FIG. 60A
`
`FIG. 6OB
`
`SET a->m AND END
`
`5,553, C)
`
`(a)
`
`SETae-b
`OTHER
`CASES C) AND CONTINUE
`
`FIG. 6OC
`
`5,555
`
`(a)
`
`SETae-C
`OTHER
`CASES O) AND CONTINUE
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 61 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 61 of 66
`
`US 2011/O122153 A1
`
`FIG. 61A
`
`ROOTNODE
`
`FIG. 61B
`
`
`
`/ ROOT \
`
`y a v\b b11 bi2
`
`v
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 62 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 62 of 66
`
`US 201 1/O122153 A1
`
`FIG. 62
`
`
`
`
`
`INITIAL GROUPNG
`PROCESS
`
`TREE GENERATION
`PROCESS
`
`
`
`
`
`
`
`S920
`
`S940
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 63 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 63 of 66
`
`US 2011/0122153 A1
`
`FIG. 63
`
`so
`r1
`INITIAL GROUPNGPROCESS
`
`SET ni INSETS
`
`CALCULATE d(head (S), ni)
`S925
`d(head (S), ni) < th202
`NO
`TREE GENERATION
`PROCESS WITH RESPECT TO
`CONTENTS INCLUDED INSETS
`
`S921
`
`S922
`
`S923
`
`S924
`
`S940
`
`HOLD RESULTS OF
`TREE GENERATION PROCESS
`
`S927
`
`DELETE CONTENTS IN SETS
`
`SET ni INSETS
`
`S930
`
`S928
`
`S929
`
`NO
`USE HELD RESULTS OF
`TREE GENERATION PROCESS
`ASNODESTO BE PROCESSED
`
`S931
`
`RETURN
`
`
`
`ADD ni TO SETS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 64 of 109
`
`

`

`Patent Application Publication
`
`May 26, 2011 Sheet 64 of 66
`
`US 2011/0122153 A1
`
`
`
`
`
`
`FIG. 64
`
`S940
`
`TREE GENERATION PROCESS
`
`NODE INSERTION
`PROCESS
`
`TREE UPDATING PROCESS
`AFTER NODE INSERTION
`
`S950
`
`S980
`
`S941
`
`
`
`
`
`
`
`
`
`HAS PROCESSING OF
`NODESTO BE PROCESSED
`BEEN FINISHED2
`
`
`
`RETURN
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 65 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 65 of 66
`
`US 2011/0122153 A1
`
`FIG. 65 sq50
`N
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Sb EAF
`ORISRADIUS OF b
`EQUAL TO 0.
`
`YES
`
`GENERATE
`NEW NODEm
`(left(m)=b, right(m)=n,
`left(a)=m)
`
`
`
`SCEAF
`ORISRADIUS OFC
`EQUAL TO O2
`
`
`
`GENERATE
`NEW NODEm
`(left(m)=C, right(m)=n,
`left(a)=m)
`
`NODE INSERTION PROCESS
`
`S951
`
`ACQUIRE ADDITION NODEn
`AND ROOTNODE a
`CASEANALYSIS PROCESS (S952
`ON BASIS OF
`RELATIONSHIPBETWEEN
`b=left(a), C=right(a), ANDn
`
`S953
`
`YES
`
`DOES THAT
`RELATIONSHIP
`CORRESPOND TO
`CASE 4 OR7?
`
`TREE GENERATION
`PROCESS WITH RESPECT TO
`(left(b), right(b), left(c), right(c),n}
`
`
`
`SET RESULTS OF
`TREE GENERATION PROCESS
`WITH RESPECT TO ROOTNODE a
`
`CALCULATED STANCES
`d(b, n), d(c, n), AND d(b,c)
`
`ISd(b, n)
`SMALLEST?
`
`GENERATE NEW NODE m
`(left(m)=a, right(m)=n, a=m)
`
`RETURN
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 66 of 109
`
`

`

`Patent Application Publication May 26, 2011 Sheet 66 of 66
`
`US 2011/0122153 A1
`
`FIG. 66
`
`S980
`a 1
`TREE UPDATINGPROCESSAFTERNODE INSERTION
`
`SETS={a}, Sb = {}, and p=a
`
`S981
`
`ISpROOTNODE'.
`
`NO
`
`Sb= (brother (p)}
`
`S982
`
`YES
`
`S989
`
`TREE GENERATION PROCESS
`WITH RESPECT TOELEMENTS INS
`S983
`
`RETURN
`
`S986
`
`S=S+head (Sb),
`Sb=Sb-head (Sb)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`S984
`
`DO head(Sb)
`AND a COINCIDE WITH
`EACH OTHER
`
`
`
`NO
`
`tmp=head (Sb),
`Sb=Sb-(imp),
`Sb={left(tmp), right (tmp)}+(Sb)
`
`
`
`S987
`
`YES
`p=parent (p)
`
`S988
`
`UNIFIED PATENTS EXHIBIT 1004
`Page 67 of 109
`
`

`

`US 2011/O 122153 A1
`
`May 26, 2011
`
`INFORMATION PROCESSINGAPPARATUS,
`INFORMATION PROCESSING METHOD,
`AND PROGRAM
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`0001
`0002 The present invention relates to an information pro
`cessing apparatus, in particular, an information processing
`apparatus which displays contents such as image files, an
`information processing method, and a program for causing a
`computer to execute the information processing method.
`0003 2. Description of the Related Art
`0004. In recent years, there has been a proliferation of
`image capturing apparatuses such as a digital still camera and
`a digital video camera (for example, an integrated camera
`recorder) which capture a subject such as a landscape or a
`person to generate an image, and record the generated image
`as an image file (content). Also, there are image capturing
`apparatuses which can record a generated image in associa
`tion with positional information on the position where the
`image is captured. There have been proposed information
`processing apparatuses with which, when displaying contents
`generated in this way, the generated positions of the contents
`identified by their positional information are displayed in
`association with the contents.
`0005 For example, there has been proposed an informa
`tion processing apparatus which arranges thumbnail icons of
`images side by side in time series and displays the thumbnail
`icons in a film window, displays position icons indicating the
`shooting locations of these images in a map window, and
`displays these icons in association with each other (see, for
`example, Japanese Unexamined Patent Application Publica
`tion No. 2001-160058 (FIG. 12)). This information process
`ing apparatus is configured Such that, for example, when a
`click operation on a thumbnail icon is performed by the user,
`a position icon indicating the shooting location of an image
`corresponding to the clicked thumbnail icon is displayed at
`the center of the map window.
`0006. Also, there has been proposed an information pro
`cessing s