1111111111111111 IIIIII IIIII 11111 1111111111 lllll lllll lllll 111111111111111 1111111111 11111111
`US 20070263984Al
`
`c19) United States
`c12) Patent Application Publication
`Sterner et al.
`
`c10) Pub. No.: US 2007 /0263984 Al
`Nov. 15, 2007
`(43) Pub. Date:
`
`(54) METHODS AND SYSTEMS FOR HANDLING
`MONTAGE VIDEO DATA
`
`(75)
`
`Inventors: Christian A. Sterner, Boulder, CO
`(US); Haider Zainy, Denver, CO (US);
`Philip Thomas DiGuilio, New York,
`NY (US)
`
`Correspondence Address:
`TOWNSEND AND TOWNSEND AND CREW,
`LLP
`TWO EMBARCADERO CENTER
`EIGHTH FLOOR
`SAN FRANCISCO, CA 94111-3834 (US)
`
`(73) Assignee: WellcomeMat, LLC, Boulder, CO (US)
`
`(21) Appl. No.:
`
`11/744,764
`
`(22) Filed:
`
`May 4, 2007
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/746,645, filed on May
`7, 2006.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`H04N 7100
`(2006.01)
`(52) U.S. Cl. ................................................................ 386/95
`
`(57)
`
`ABSTRACT
`
`Video data may be extremely useful in online property
`commerce. However, the current methods of handling this
`video data fail to provide users of the data with effective and
`convenient ways of consolidating footage from multiple
`videos or portions thereof and conveying their particular
`viewpoints and other property features. Among other things,
`embodiments of the invention address these failures by
`providing methods and systems for generating and handling
`montage video data.
`
`,
`
`500
`
`________ T _______ _
`
`504----,_ --------
`5o4-t~ ________________ -- - PROVIDE LOGIN - - - - -
`:
`-
`INFORMATID~-----
`
`i
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`NOT
`VERIFIED
`
`502-::--:i_ i ----GOTOLOGIN _____ :
`'
`--~
`VERIFIED 5o4 -2
`
`?
`
`510
`
`512-1
`
`'
`ENTER VEE
`
`I
`
`512-2
`
`Y~N
`
`514-1
`
`514-2
`
`UPLOAD NEW VIDEO
`
`LOAD EXISTING VIDEO
`
`530-1
`
`N
`
`520
`
`524
`
`526
`
`I
`I
`I
`I
`I
`I
`
`I
`
`532
`
`530
`
`~y
`
`530-2
`
`VIEW VIDEO IN FRAME
`
`522
`
`N 522-1
`
`y 522-2
`
`PROVIDE CUE POINT DATA
`
`SUBMIT CUE POINT DATA
`
`528-1
`
`N
`
`Exhibit 1008
`MG Freesites v. Scorpcast
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 1 of 28
`
`US 2007 /0263984 Al
`
`f
`
`100
`
`,...--140
`r-··················································································································~··~·~····················L .................. .
`
`!
`
`APPLICATION
`SERVER
`
`Iii ~--7-----+l
`
`CUE POINT
`SERVER
`
`120
`
`118
`
`122
`
`i
`
`AUTHORIZATION
`SERVER
`
`!
`~
`!
`1
`
`I
`'························································································· ... ·······•····•··•····················••··••·••··••·························
`I
`---•
`
`-
`
`----~,
`I
`I
`I
`I
`- ~ I VAE I I - -
`LL _LJ
`1367
`
`138
`
`NETWORK
`
`130
`
`132
`
`D
`
`VIEWERS
`
`134
`
`D
`
`EDITORS
`
`Fig. 1A
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 2 of 28
`
`US 2007 /0263984 Al
`
`150
`
`••••••••••••••••••n•••••••••••••••••••••••••••••••••••••••••••••••n••••••••••••••••••••••.t•••••••••••••n••i
`
`,,,,--190
`
`APPLICATION
`SERVER
`
`166
`
`<"?n
`1/U
`
`168
`
`180
`
`182
`
`0
`
`VIEWERS
`
`184
`
`0
`
`EDITORS
`
`Fig. 1 B
`
`l~I Ill /
`
`f
`
`1'
`
`' ,
`.._ -
`
`-
`
`-
`
`-
`
`1
`,,,,VIDEO
`.,, SERVERS
`
`\,
`
`\
`
`\
`162-1 \
`
`162~,,
`/
`
`I
`I
`I
`f
`I
`I
`I
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 3 of 28
`f
`
`200
`
`US 2007 /0263984 Al
`
`202
`
`204
`
`REQUEST LOGIN/CLIENT
`INFORMATION
`
`ASSIGN CLIENT ID
`
`206-1
`
`EDITOR-----<
`
`220
`
`y
`
`224
`
`210
`
`212
`
`>-----N-~-1(cid:141)~0ooumrr )..,. .. __ _
`
`230
`
`212-;:7
`
`y
`214 ~
`~
`
`Fig. 2
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 4 of 28
`
`US 2007 /0263984 Al
`
`300
`
`:
`PROCESS LOGIN
`1
`________ T _______ _
`
`304 ~ ,, ,, " ' -.... ..._
`
`f
`302 ~---------------
`I '
`3~~-~~ - - - - - - - - - - - - - - - - -·<. ;~~~~:~ENE;'o': '>
`:
`~
`y 304-2
`
`I
`I
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`N
`
`' ,
`...., '
`
`310
`
`.,....,,..,,,,
`
`.,,,.
`
`LOAD VEE
`
`312-1
`
`312
`
`312-2
`
`314-1
`
`316
`
`318
`
`320
`
`324
`
`326
`
`330-1
`
`N
`
`GENERATE VEE GUI
`
`GENERATE VEE VIDEO FRAME
`
`POLL FOR CUE POINT MODE
`
`322
`
`N 322-1
`
`RECEIVE CUE POINT DATA
`
`328-1
`
`N
`
`332 ~ 330
`
`~y
`
`330-2
`
`Fig. 3A
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 5 of 28
`
`US 2007 /0263984 Al
`
`342
`
`344
`
`346
`
`348
`
`350
`
`352
`
`354
`
`f
`
`340
`
`GENERATE UPLOAD FORM
`
`PROCESS UPLOAD FORM
`
`UPLOAD VIDEO FILE TO SERVER
`
`REDIRECT USER TO VEE
`
`BEGIN DYNAMIC LOADING OF
`VIDEO INTO FRAME
`
`LOAD ASSOCIATED CUE POINT
`DATA
`
`364-1
`POLL FOR INTERACTIVITY 14 - - - - - - -~ -N - - - - - ,
`
`356
`
`358
`
`y
`
`356-1
`362
`
`PLAY VIDEO
`
`y
`
`360-1
`
`y
`
`364-2
`
`366
`
`PAUSE VIDEO
`AND RUN CUE
`POINT HANDLER
`
`N
`
`y
`
`366-1
`
`366-2
`
`SAVE AND
`EXIT VEE
`
`EXIT VEE
`
`368-
`
`Fig. 3B
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 6 of 28
`
`US 2007 /0263984 Al
`
`f
`
`370
`
`372
`
`LOAD CUE POINT EDITOR
`
`374-1
`..,,_ ______ ---1... __ DELETE
`
`/376-1
`">-.__-NONE
`
`380-1
`
`FULL
`
`382
`
`I
`I
`I
`I
`I
`I
`
`,---
`
`1
`I
`I
`I
`I
`I
`I
`
`i'
`
`396-1
`
`I
`y
`
`386
`
`388
`
`GENERATE AND SAVE CUE
`POINT ON SUBMIT
`
`390
`
`DELETE CUE POINT
`
`--
`
`L -
`
`-
`
`-
`
`- ~._:- -
`
`I _t
`--
`- CONTINUE? - - -;;>
`396~- - -r- - - - -
`; - ·N- -
`396-Y
`
`I
`
`-
`
`-
`
`-
`
`EXIT CUE POINT EDITOR
`
`398
`
`Fig. 3C
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 7 of 28
`
`US 2007 /0263984 Al
`
`f
`
`400
`
`440
`
`VIDEO EDITING
`ENVIRONMENT
`
`430
`
`,•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• •••••••••••••••••••••••••Hoou•••••••n••••••n•••••••••••n•••••••••••••••n
`
`410
`j
`\.j
`
`422
`
`NETWORK
`DEVICE
`
`424
`
`420
`
`428
`
`INPUT
`DEVICE
`
`----PROCESSOR-------
`
`DATA
`STORE
`
`426
`
`DISPLAY
`DEVICE
`
`Fig. 4
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 8 of 28
`
`US 2007 /0263984 Al
`
`500
`
`:
`GO TO LOGIN
`1
`________ T _______ _
`
`--
`
`f
`502 ~---------------
`'
`-~
`VERIFIED 5o4-2
`J
`ENTER VEE
`
`504
`,,,,..- .,,,,. - .........
`---
`~ - -
`504-1
`-~ - - __ __ __ _ ____ _____
`PROVIDE LOGIN
`I
`INFORMATION
`-
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`NOT
`VERIFIED
`'
`
`I
`
`510~
`I
`
`512-1
`
`512~
`
`512-2
`
`Y - - -<~> - -
`
`514-1
`
`514-
`
`530-1
`
`N
`
`520
`
`524
`
`526
`
`I
`I
`I
`I
`I
`
`532 ~ 530
`
`~y
`
`530-2
`
`522
`
`N 522-1
`
`y 522-2
`
`PROVIDE CUE POINT DATA
`
`528-1
`
`N
`
`Fig. 5
`
`

`

`~ ....
`z 0
`.... 0 =
`"'O = O" -....
`.... 0 =
`t "e -....
`
`.i;.,.
`QO
`1,0
`~
`O'I
`N
`0
`~
`0
`0
`N
`rJJ
`
`> ....
`
`c
`
`1,0
`
`('D
`('D
`
`0 ....
`.....
`rJJ =(cid:173)
`
`QO
`N
`
`-....J
`0
`0
`N
`~Ul
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`('D = .....
`~ .....
`"'O
`
`Fig. 6
`
`638
`
`656
`
`654
`
`652
`
`640
`
`~
`
`:SA VI:
`
`CANCEL I I
`
`GO TO VIEWER MODE
`
`GO TO CUE POINT BANK
`
`ADVANCED OPTIONS
`
`FORMAT TEXT BOX
`
`ASSOCIATE CUE POINT
`
`LOOKUP NAME FROM DB
`
`MANICURED FRONT
`A LARGE,
`PROPERTY BOASTS
`EXTERIOR -THIS
`
`0:00:34.03
`EXTER_
`
`I HIERARCHY I CURRENT: NO
`I SORT
`I INSERT I I MODIFY 11 DELETE I
`
`I CURRENT: BY TIME
`
`CUE POINT TOOLS:
`
`I
`I
`EXTERIOR-THIS PROPERTY BOASTS A LARGE, MANICURED
`
`FRONT YARb AND A lWO-CAR GARAGE.
`
`\
`
`630
`
`636
`634
`632
`
`608
`
`606
`
`)
`JI~ 626
`
`:vt--t--LI__,.,, 624
`::":liill I ,. 622
`
`622
`
`VI '
`
`( LIVING ROOM
`
`(FOYER
`
`( EXTER_
`
`CUE POINTS:
`
`II
`I ( NEIGHBORHOOD
`I
`
`1_ -_1
`
`604,,.----1~
`
`602 ~ _J
`
`600
`
`,
`
`

`

`.i;...
`QO
`\0
`(,H
`O'I
`N
`0
`~
`0
`0
`N
`rJ'1
`c
`
`> ....
`
`Fig. 7
`
`~ ....
`z 0
`.... 0 =
`""O = O" -....
`.... 0 =
`t "e -... . (')
`
`~ .....
`
`(')
`
`~ .....
`
`('D = .....
`~ .....
`""O
`
`0 ....
`0
`....
`.....
`rJ'1 =(cid:173)
`
`('D
`('D
`
`QO
`N
`
`-....J
`0
`0
`N
`~Ul
`
`. ..
`
`...
`
`...
`
`...
`
`...
`
`...
`
`...
`
`. ..
`
`. ..
`
`. . .
`
`. ..
`
`KITCHEN
`
`MAIN STAIRS DINING ROOM
`
`EXTERIOR
`
`FOYER
`
`GARAGE
`
`FRONT PORCH
`
`OUTSIDE
`
`BATHROOM
`
`OFFICE
`
`DINING ROOM
`
`ENTRY
`
`. . .
`
`VID_0005
`
`VID_0004
`
`VID_0003
`
`LIVING ROOM
`
`ENTRY
`
`V
`
`EXTERIOR
`
`STREET
`
`'...__,,,, VI D _ 0002
`
`LIVING ROOM
`
`FOYER
`
`EXTERIOR
`
`NEIGHBORHCX)[)
`
`,...__,,,, VID_0001
`
`/
`
`71
`
`71
`
`71
`
`71
`
`760
`
`730
`
`CP1
`
`VIDEO_ID
`
`71
`
`720-2~
`
`720-1~
`
`CP4
`
`720~
`
`CP2 )\. CP3
`
`( ( 720-~
`740 750
`
`700
`
`f
`
`)
`
`

`

`>---------------
`•
`--
`----~
`
`,,. ,,. ' ._
`804 ---...._
`\.-,,.
`,,. ,,. PERMISSION T'a'-..
`,,. >- - - - - - - - - - - - - - -
`~ ._
`ENTER VPE?
`--
`,,.
`
`,,.
`
`--
`
`802
`
`810
`
`812
`
`814
`
`816
`
`818
`
`Patent Application Publication Nov. 15, 2007 Sheet 11 of 28
`
`US 2007 /0263984 Al
`
`t
`
`800
`
`:
`PROCESS LOGIN
`1
`________ T _______ _
`
`~04-1
`- - - - -
`
`y 804-2
`
`GENERATE VPE GUI
`
`GENERATE VPE VIDEO FRAME
`
`I
`N
`
`POLL FOR INTERACTIVITY
`
`32-2
`--~------N
`
`832
`
`y 820-1
`
`y 824-1
`
`828-1
`
`y 832-1
`
`826
`
`834
`
`PLAY VIDEO
`
`JUMP TO
`[CUE POINT]
`LOCATION
`
`y
`
`I
`
`INTERACT PER
`COMMAND 1 - - - -~
`
`EXITVEE }----_!
`c;
`
`830
`
`Fig. 8
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 12 of 28
`
`US 2007 /0263984 Al
`
`900
`
`VIDEO PLAYBACK
`ENVIRONMENT
`
`930
`
`910
`1··························································································································································
`
`!
`
`INPUT
`DEVICE
`
`(928
`
`DATA
`STORE
`
`Fig.9A
`
`950
`
`980
`
`960
`'-.._f·············································································· ·····························································•·•·············
`
`990
`
`INPUT
`DEVICE
`
`DATA
`STORE
`
`Fig. 98
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 13 of 28
`
`US 2007 /0263984 Al
`
`f
`
`1000
`
`1002\
`r----------------
`:
`GO TO LOGIN
`1
`________ T _______ _
`
`----'--- ....
`1004---....._
`1004-
`.._
`, ,..
`1
`(
`____ ,... PROVIDE',.._
`>- - - - - -- - - - - - - -- - - - - - - -
`~ VERIFICATION
`'-.. !_NFORMATIOf't,,.,..
`,..
`....
`- -~ 1004-
`2
`VERIFIED
`1010~r--'----'-'--~
`ENTERVPE
`
`1018
`
`VIEW VIDEO IN FRAME
`
`1032-
`---~-~2-----N
`
`NOT
`VERIFIED
`
`032
`
`1028-
`1
`
`026
`
`1032-
`1
`034
`
`INTERACT PER
`COMMAND
`
`t - - -~
`
`PLAY VIDEO
`
`JUMP TO
`[CUE POINT]
`LOCATION
`
`y
`
`I
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`(cid:141) c EXITVEE } - - - - - - :
`
`\_
`
`1030
`
`Fig. 10
`
`

`

`~ ....
`z 0
`.... 0 =
`"-= = O" -....
`.... 0 =
`t "e -....
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`('D = .....
`"-= ~ .....
`
`.i;...
`QO
`1,0
`~
`O'I
`N
`0
`~
`0
`0
`N
`rJJ
`c
`
`> ....
`
`0 ....
`....
`.....
`rJJ =(cid:173)
`
`.i;...
`
`('D
`('D
`
`QO
`N
`
`-....J
`0
`0
`N
`~Ul
`
`Fig. 11 B
`
`Fig. 11A
`
`1170
`
`1172-5
`
`1172-4
`
`1120
`
`~ f O
`~~ ~la)
`
`Dl ~ 1172-3
`
`1172-2
`
`1 o:341s:13
`
`II I IIIIIIIK'.:
`
`I
`I
`EXTERIOR -THIS PROPERTY BOASTS A LARGE, MANICURED
`
`FRONT YAHb AND A TWO-CAR GARAGE.
`
`\
`
`1172-1
`
`GUEST BATHROOM
`
`LIVING ROOM
`
`GUEST BEDROOM
`
`MASTER SUITE
`
`KITCHEN
`
`111
`
`FOYER
`
`EXTERIOR
`
`NEIGHBORHOOD
`
`I
`I
`EXTERIOR -THIS PROPERTY BOASTS A LARGE, MANICURED
`
`FRONT YARb AND A lWO-CAR GARAGE.
`
`\
`
`'I
`
`1104
`
`1150
`
`,
`
`,-1100
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 15 of 28
`
`US 2007 /0263984 Al
`
`, - 1180
`
`1182
`
`1184
`
`1186
`
`1188
`
`EXTERIOR- THIS PROPERTY BOASTS
`A LARGE, MANICURED FRONT YARD
`AND A TWO-CAR GARAGE.
`ADD COMMENT
`
`''-----....:::::::~~~--...J
`
`I
`
`0:34/6:13
`
`1192-5
`
`)
`)
`
`1190
`
`[.._N_E_1G_H_so_R_H_o_oo ____ 1 (FOYER
`
`[.._EX_TE_R_1o_R ______ )
`
`( LIVING ROOM
`
`Fig. 11C
`
`

`

`.i;...
`QO
`1,0
`~
`O'I
`N
`0
`~
`0
`0
`N
`rJJ
`c
`
`> ....
`
`0 ....
`O'I
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`QO
`N
`
`Fig. 12
`
`1250
`
`PROPERTY INFORMATION FIELD
`
`)
`
`)
`
`( GUEST BATHROOM
`
`( GUESTBEDROOM
`
`( MASTER SUITE
`
`(KITCHEN
`
`)
`
`)
`
`)
`
`)
`
`j 034/613
`
`~ ....
`z 0
`.... 0 =
`""O = O" -....
`.... 0 =
`t "e -....
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`('D = .....
`~ .....
`""O
`
`--....J
`0
`0
`N
`~Ul
`
`1230
`
`[ OD]
`[ m l [ I~ l
`r~~~~ ,==,
`
`$580,000
`
`LIST PRICE:
`
`I
`I
`I
`
`1210
`
`(LIVING ROOM
`
`(FOYER
`
`C EXTERIOR
`(NEIGHBORHOOD
`II I ~
`
`~
`~
`~
`~
`
`r-------------------------~
`
`_____________ 1
`
`BOULDER, CO 80301
`
`1 LUCKY STREET
`
`LISTINGS INC.
`PROPERTY
`
`1220
`
`1200
`
`f
`
`

`

`~ ....
`z 0
`.... 0 =
`""O = O" -....
`.... 0 =
`t "e -....
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`('D = .....
`~ .....
`""O
`
`.i;...
`QO
`1,0
`~
`O'I
`N
`0
`~
`0
`0
`N
`rJJ
`c
`
`> ....
`
`0 ....
`-....J
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`QO
`N
`
`-....J
`0
`0
`N
`~Ul
`
`Fig. 13B
`
`,,.,'
`
`---
`
`TERMINALS',_
`
`1350
`
`\\~II II)
`,ro
`Y
`lg ,
`/
`..... "
`
`/.,,,.
`
`I ____________________ I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`SERVERS
`
`VIDEO
`
`Fig. 13A
`
`1350
`
`,~
`
`'
`
`/
`
`____ .,,,.
`\\~II II)
`'D
`lg',
`/
`,,,..----
`
`,/
`
`TERMINALS',
`
`SERVERS
`
`VIDEO
`
`/
`
`I
`
`I
`I
`
`SERVERS
`,, CUE POINT
`
`-
`
`-
`
`SERVERS
`CUE POINT
`
`/
`
`---------,
`
`:
`~------------------------
`
`1~0
`
`,· 1360
`
`1300
`
`f
`
`~
`
`1314
`
`I I !
`
`:
`
`HANDLER
`;uE POINT
`
`\____j
`[
`
`1310
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 18 of 28
`
`US 2007 /0263984 Al
`
`✓- -
`
`... ,
`
`1340
`
`// ~ '~
`( I ~i i
`=
`__ .,,
`
`\
`
`\
`'
`'
`
`I
`I
`
`/
`
`/
`
`CUE POINT
`SERVERS
`
`-- -,
`
`1330
`
`I
`
`// il '~
`El)
`\
`( i ~- ,i
`\
`=
`\
`' ..... __ .........
`'
`
`I
`I
`
`/
`
`VIDEO
`SERVERS
`
`~ 1380
`
`,
`
`Fig. 13C
`
`

`

`.i;...
`QO
`1,0
`~
`O'I
`N
`0
`~
`0
`0
`N
`rJJ
`c
`
`> ....
`
`Fig. 15
`
`Fig. 14
`
`~ ....
`z 0
`.... 0 =
`""O = O" -....
`.... 0 =
`t "e -....
`
`1,0
`
`('D
`('D
`
`0 ....
`....
`.....
`rJJ =(cid:173)
`
`QO
`N
`
`-....J
`0
`0
`N
`~Ul
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`('D = .....
`~ .....
`""O
`
`I
`:
`
`'---i NAVIGATE VIDEO MONTAGE FILE
`
`________ J
`
`I
`
`1510
`
`I USING CUE POINT CONTROLS
`
`________ T _______ _
`I
`I
`:
`
`VIDEO PLAYER
`
`\__J VIEW VIDEO MONTAGE FILE ON
`
`r ________ J
`I
`
`1508
`
`AND CUE POINT INFORMATION
`RECEIVE VIDEO MONTAGE FILE
`
`TRANSMIT MONTAGE REQUEST
`
`GENERATE MONTAGE REQUEST
`
`1504
`
`1502
`
`1500
`
`,
`
`-----------------
`I
`I
`I AND CUE POINT INFORMATION TO I
`O\.___J TRANSMIT VIDE~ MONTAGE FILE ;
`
`VIDEO PLAYER
`
`141
`
`I
`
`BASED ON RELATIONAL DATASET
`
`BUILD VIDEO MONTAGE FILE
`
`GENERATE RELATIONAL DATASET
`
`BASED ON MONTAGE REQUEST
`
`ANALYZE MONTAGE REQUEST
`
`RECEIVE MONTAGE REQUEST
`
`1404
`
`1402
`
`1400
`
`t
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 20 of 28
`
`US 2007 /0263984 Al
`
`1600
`
`(
`
`VIDEO_ID
`
`CP1
`
`CP2
`
`CP3
`
`CP4
`
`VID_0001
`
`t-EG-Ernl-KX)D EXTERIOR
`
`FOYER
`
`LIVING ROOM
`
`VID_0002
`
`STREET
`
`EXTERIOR
`
`ENTRY
`
`LIVING ROOM
`
`166 0
`
`ViD_0003
`
`ENTRY
`
`UININl.;i KVVIVI
`
`OFFICE
`
`~AIHKVVM
`
`~ VID_0004
`
`OUTSIDE
`
`FRONT PORCH GARAGE
`
`FOYER
`
`VID_0005
`
`EXTERIOR MAIN STAIRS DINING ROOM
`
`KITCHEN
`
`.. .
`
`...
`
`...
`
`' ' '
`
`...
`
`...
`
`...
`
`...
`
`...
`
`...
`
`...
`
`' ' '
`
`1640
`
`1
`
`BUILD: "EXTERIOR" I I
`
`, '
`
`1650
`
`(
`
`1652
`
`(
`
`1654
`
`(
`
`1656
`
`(
`
`( - 1658
`
`VIDEO_ID
`
`CUE_LOCATION
`
`CUE_TITLE
`
`TIME_LOCATION
`
`VID_0001
`
`VID_0002
`
`VID_0005
`
`CP2
`
`CP2
`
`CP1
`
`SHAPPY LANE
`
`00:00:51.29
`
`1 LUCKY STREET
`
`00:00:34.03
`
`1554 VIOLET AVE
`
`00:00: 11. 78
`
`' ' '
`
`' ' '
`
`. ..
`
`...
`
`Fig. 16
`
`

`

`0 ....
`N ....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`QO
`N
`
`~ ....
`z 0
`~ ..... ... 0 =
`"-= = O" -... (')
`~ ..... ... 0 =
`t "e -... (')
`
`('D = .....
`"-= ~ .....
`
`-....J
`0
`0
`N
`~Ul
`
`'i
`
`I
`
`.....
`
`$175/YR
`
`0 . 2 5 ACRE
`
`HOA:
`TAXES: $1971
`LOT:
`BUILT: 1992
`1850 SQ FT
`3BR/2.5BA
`LIST:
`
`$580,000
`
`BOULDER, CO 80301
`1 LUCKY STRlilET
`
`CURRENT PROPERTY INFO:
`
`TYPE: SINGLE-FAMILY, CONDO
`LOCATION: BOULDER COUNTY
`LIST PRICE: $500,000 -$750,000
`
`SEARCH CRITERIA:
`
`J o:341s:13
`
`I
`I
`EXTERIOR -THIS PROPERTY BOASTS A LARGE, MANICURED
`
`FRONT YARb AND A TWO-CAR GARAGE.
`
`\
`
`E:>
`~
`<S>
`~
`
`(cid:141) I 111111(:
`
`,i;...
`QO
`1,0
`~
`O'I
`N
`0
`~
`0
`0
`N
`rJJ
`c
`
`> ....
`
`MODIFY SEARCH CRITERIA
`
`MAP SEARCH RESULTS
`
`SHOW SEAR.CH RESULT DETAILS
`
`( 1094 MOUNTAIN TRACE
`I
`,----------~ I
`
`)
`
`1 LUCKY STREET
`
`:
`I
`
`I-----------.. ~~---_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_~~ I
`
`Fig.17A
`
`1--------------------------------~
`: ( 1983 VIOLET AVE
`:
`I ,----------~ ~--------------I
`
`( 861 FRONT RANGE RD
`
`]
`
`)
`
`:
`
`)
`
`(1110MESATRAIL
`
`]
`
`H1554VIOLETAVE
`
`GO TO PROPERTY LISTING
`
`1 [ 5 HAPPY LANE
`:
`r--------------------------------,
`
`)
`
`( 12 FRONTAGE ROAD
`
`]
`
`1712
`
`1710
`
`EXTERIORS
`
`LISTINGS INC.
`
`PROPERTY
`
`1700
`
`f
`
`

`

`~ ....
`z 0
`.... 0 =
`"'O = O" -....
`.... 0 =
`t "e -....
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`('D = .....
`~ .....
`"'O
`
`.i;.,.
`QO
`1,0
`~
`O'I
`N
`0
`~
`0
`0
`N
`rJJ
`
`> ....
`
`c
`
`0 ....
`
`QO
`N
`
`N
`N
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`0
`0
`N
`~Ul
`
`.Y.
`
`~ I
`
`j I MODIFY:SEARCH CRITERIA
`I I
`
`MAP Sl:ARCH RESULTS
`
`Fig. 178
`
`L--------------------------------~
`: ~] [~
`
`GO TO PROPERTY LISTING
`
`J o:34/6:13
`
`1760
`
`~
`~
`~
`~
`
`EXTERIORS
`
`LISTINGS INC.
`PROPERTY
`
`1750
`
`f
`
`: (4aj ~ Im I i I SHOWSeARCHRESUC'DmLS
`i --------------------------------~ I
`(cid:141)I~
`
`0.25 ACRE
`
`$580,000
`
`BOULDER, CO 80301
`1 LUCKY STRJ!ET
`
`CURRENT PROPERTY INFO:
`
`TYPE: SINGLE-FAMILY, CONDO
`LOCATION: BOULDER COUNTY
`LIST PRICE: $500,000 -$750,000
`
`SEARCH CRITERIA:
`
`HOA:
`TAXES: $1971
`LOT:
`BUILT: 1992
`1850 SQ FT
`3 BR / 2. 5 BA
`LIST:
`
`$175/YR
`
`I
`I
`EXTERIOR -THIS PROPERTY BOASTS A LARGE, MANICURED
`
`FRONT YARb AND A TWO-CAR GARAGE.
`
`\
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 23 of 28
`
`US 2007 /0263984 Al
`
`1800
`
`(
`
`1810-1
`\_
`1810-2
`\_
`1810-3
`
`1810-4\_
`
`1810-~
`\_
`
`1802
`
`(
`
`1804-1 (
`
`1804-2 (
`
`1804-3( 1804-4(~ 1804-5
`
`(
`
`VIDEO_ID
`
`CP1
`
`CP2
`
`CP3
`
`CP4
`
`...
`
`VID_0001
`
`HOTEL
`
`DOWNTOWN
`
`SWIMMING
`POOL
`
`ROOFTOP
`
`HOT TUB
`
`VID_0002
`
`SPORTS
`
`BASKETBALL
`
`VID_0003
`
`HOTEL
`
`AIRPORT GUEST ROOM
`
`VID_0004
`
`HOTEL
`
`DOWNTOWN
`
`SWIMMING
`POOL
`
`ROOFTOP
`
`VID_0005
`
`NEWS
`
`WORLD
`
`ASIA
`
`CHINA
`
`04/02/2006
`
`...
`
`...
`
`. ..
`
`. ..
`
`...
`
`. ..
`
`1lr
`
`1840
`
`(
`
`184z___
`
`-H
`
`PIVOT: CP1 + CP2 I
`
`1850
`
`(
`( - 1852
`
`1860
`
`(
`
`.
`
`.
`
`VID_0001
`VID_0004
`
`(
`
`1854-1
`
`(
`
`1854-2
`
`CP1
`
`CP2
`
`VIDEO_ID
`
`HOTEL
`
`DOWNTOWN
`
`VID_0001
`
`HOTEL
`
`HOTEL
`
`NEWS
`
`DOWNTOWN
`
`VID_0004
`
`AIRPORT
`
`VID_0003
`
`WORLD
`
`VID_0005
`
`SPORTS
`
`BASKETBALL
`
`VID_0002
`
`...
`
`...
`
`...
`
`Fig. 18
`
`

`

`.... 0 = z 0
`.... 0 = ""O = O" -....
`t "e -....
`('D = .....
`~ .....
`""O
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`.i;...
`QO
`1,0
`~
`O'I
`N
`
`> ....
`
`0
`0
`N
`rJJ
`c
`
`-....J --0
`
`0 ....
`
`QO
`N
`
`.i;...
`N
`.....
`rJJ =- ('D
`
`('D
`
`~ ....
`
`-....J
`0
`0
`N
`~Ul
`
`Fig. 19A
`
`1904
`
`MODIFY SEARCH CRITERIA
`
`CONTACT PROPERTY
`
`MAP SEARCH RESULTS
`
`MAP HOTEL LISTING
`
`SHOW RESULTS DETAILS
`
`SHOW RA TINGS
`
`RESORT RESULTS
`
`GO TO HOTEL LISTING
`
`1920-4
`
`1920-3
`
`1920-2
`
`1920-1
`
`CONFERENCE CENT
`
`-----~
`Q~
`J."' 1924-2
`-----~ Q9 ,,,--1922-2
`
`£' 1924-1
`1922-1
`
`SEARCH RESULTS:
`
`JI
`
`1904
`
`1900
`
`,
`
`1902
`
`

`

`,i;...
`QO
`1,0
`~
`O'I
`N
`
`> ....
`
`0
`0
`N
`rJJ
`c
`
`-....J --- 0
`
`0 ....
`
`QO
`N
`
`Ul
`N
`.....
`rJJ =- ('D
`
`('D
`
`.... 0 = z
`.... 0 = ""O = O" -....
`t "e -....
`('D = .....
`~ .....
`""O
`
`~ ....
`
`-....J
`0
`0
`N
`~Ul
`
`0
`
`~ .....
`
`(')
`
`~ .....
`
`(')
`
`1940-2
`
`TODAY'S MiOST POPULAR FEEDS:
`
`1940-1
`
`3RESULTS
`NCAA AND "FINAL FOUR"
`
`CURRENTLY WATCHING:
`
`1944
`
`SPORTNEWS.COM
`
`?
`
`?
`
`?
`
`?
`
`1940-3
`
`WHO MOVES ON
`
`FINAL FOUFt-CAST
`
`RELATED HEADLINES:
`
`OLYMPIC CURLING
`
`BASEBALL BLOOPERS
`
`CHRIS JAME:S DUNKS
`
`FANS TODAY IS: "WHO WILL BE IN THE FINAL FOUR?"
`
`THE QUESTION ON THE MINDS OF ALL COLLEGE BASKETBALL
`
`\
`
`TODAY'S SPORTS HIGHLIGHTS
`
`1930
`
`f
`
`1932
`
`~
`~
`~
`~
`1942 ~
`~
`~
`~
`~
`1~
`
`DOWNLOAD FEED
`
`-- 1936
`
`SAVE FEED
`
`MODIFY SEARCH CRITERIA
`
`FOLL.OW THE FEEDER
`
`4 SCORE AND 2 ROUNDS TO GO
`
`3:1616:13
`
`Fig. 198
`
`'--1934
`
`J----\ 1950-3
`
`h 1950-1
`
`1938
`
`l{;z
`
`[HIGHLIGHTS.COM
`
`[ SPORTVIDS.COM
`
`(cid:141)
`
`~ 1950-2
`
`

`

`~ ....
`z 0
`.... 0 =
`~ .....
`1970-1 = (')
`0 = ""O = O"
`t "e -....
`('D = .....
`~ .....
`""O
`
`.i;...
`QO
`1,0
`~
`O'I
`N
`
`> ....
`
`0
`0
`N
`rJJ
`c
`
`--:i --- 0
`
`0 ....
`
`QO
`N
`
`O'I
`N
`.....
`rJJ =- ('D
`
`('D
`
`--:i
`0
`0
`N
`~Ul
`
`Q;
`~
`(')
`
`1970-3
`
`1970-2
`
`SMALL BOXES
`
`DIAMOND JEWELRY
`
`NECKLACES
`
`RELATED PRODUCT SEARCHES:
`
`___,J
`3:1616:13
`
`$159.99
`
`$119.99
`
`$138.00
`
`HANDICRAFTS ONLINE
`
`JEWELRYSTORE.COM
`
`SHOP4STUFF.NET
`
`LOWEST PRICES:
`
`1972
`
`... 6RESULTS
`ON "ALL NETWORKS"
`"JEWELRY BOXES"
`
`SET AUCTION ALERTS
`
`MODIFY SEARl:H CRITERIA
`
`MORE FROM VENDOR
`
`GO TO PRODUCT LISTING
`
`)
`I
`)
`)
`I
`
`I
`
`I
`
`(cid:144)
`
`(
`
`) ( ~
`) (
`
`0
`
`I:::.
`0
`0
`
`14k GOLD-PLATED, 4 STORAGE LEVELS
`HANDCRAFTED PYRAMID JEWELRY BOX
`
`CURRENTLY WATCHING:
`
`SHOP4STUFF.NET
`
`1!~74
`
`AUCTION CLEARINGHOUSE 4\
`
`1960
`
`t
`
`1966
`
`Fig. 19C
`
`1964
`
`1980
`
`4
`
`1962
`
`~
`~
`~
`~
`~
`~
`I Ownlt.net I
`
`1968
`
`(cid:141)
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 27 of 28
`
`US 2007 /0263984 Al
`
`, - 2000
`
`2002-1
`
`2002-2
`
`2002-3
`
`PROCESS CLIENT 1
`LOGIN
`
`PROCESS CLIENT 2
`LOGIN
`
`PROCESS CLIENT 3
`LOGIN
`
`2010-1~
`-
`--------1
`: GENERATE CLIENT 1 I
`I
`RESULTS LIST
`I
`
`2010-2~
`
`2010-3~
`
`: GENERATE CLIENT 2 :
`I
`RESULTS LIST
`1
`
`: GENERATE CLIENT 3 :
`I
`RESULTS LIST
`I
`
`----- ____ _.
`
`2022
`
`I
`I
`:- - - -- -- - - - - - --.
`I
`I
`I
`I
`I
`
`I
`USE RES UL TS TO
`:
`: GENERATE MONTAGE
`I
`I
`VIDEO FILE
`______ T _____ _
`
`f _____ J ____ c
`
`2040
`
`I
`
`2030 ~---'-----1 2042 ~---'-----1
`
`: TRANSMIT SYNERGY I
`I
`LIST
`I
`
`: TRANSMIT MONTAGE I
`I
`VIDEO FILE
`I
`
`Fig. 20
`
`

`

`Patent Application Publication Nov. 15, 2007 Sheet 28 of 28
`
`US 2007 /0263984 Al
`
`f
`
`2100
`
`EXTERIORS
`
`2102
`
`EXTERIOR- THIS PROPERTY BOASTS A LARGE, MANICURED
`/
`FRONT YARb AND A TWO-CAR GARAGE.
`I
`\
`
`2140-1
`
`1 o:34/e:13
`
`RATING
`
`2106
`
`2120-1
`
`2120-2
`
`2120-3
`
`2120-4
`
`SHAPPY LANE
`
`1 LUCKY STREET
`
`( 1554 VIOLET AVE
`
`( 1983 VIOLET AVE
`
`i ~ ,
`~)
`
`2150
`
`2152
`
`VIEW COMMENTS
`
`2104
`
`Fig. 21
`
`

`

`US 2007/0263984 Al
`
`Nov. 15, 2007
`
`1
`
`METHODS AND SYSTEMS FOR HANDLING
`MONTAGE VIDEO DATA
`
`CROSS-REFERENCES TO RELATED
`APPLICATIONS
`[0001] This application is a nonprovisional claiming the
`benefits of U.S. Prov. App. No. 60/746,645, entitled "SYS(cid:173)
`TEM FOR ON-LINE VIDEO-BASED PROPERTY SALES
`AND MARKETING," filed May 7, 2006 by Christian A.
`Sterner et al. This application is also related to U.S. App. No.
`??, entitled "METHODS AND SYSTEMS FOR ONLINE
`VIDEO-BASED PROPERTY COMMERCE" (Attorney
`Docket No. 026819-00011 0US), filed on the same date as
`the current application by Christian A. Sterner et al. The
`entire disclosures of both related applications are herein
`incorporated by reference for all purposes.
`
`BACKGROUND OF THE INVENTION
`[0002] This invention relates generally to online video;
`and more particularly to methods and systems for handling
`online video for property commerce over the Internet.
`
`[0003] The past decades have seen the tremendous growth
`of Internet traffic, and with it, the birth of a new world of
`global electronic commerce (e-commerce). E-commerce
`allows sellers to share their wares with consumers around
`the world in virtual marketplaces. Through these virtual
`marketplaces, sellers can open virtual storefronts with little
`to no capital, quickly and inexpensively distribute informa(cid:173)
`tion about products and services to a global customer base,
`and collect payment through electronic fund transfers.
`
`[0004]
`In exchange for these benefits, however, e-com(cid:173)
`merce drastically changes the relationships between buyers
`and sellers. One primary manifestation of this change is the
`difficulty with current technology to give consumers a feel
`for goods and services before they buy. In a virtual market(cid:173)
`place, a consumer has limited ability to experience a prod(cid:173)
`uct-to try on clothing, to smell a chefs creation, or to walk
`through a neighborhood.
`
`[0005] Still, as consumer high-speed data capability
`becomes increasingly ubiquitous, sellers are finding ways to
`combat those difficulties. The vast majority of e-commerce
`websites, for example, now use photos along with product
`descriptions to provide more information to consumers.
`Some progressive online retailers have even begun to use
`video, virtual reality applications, three-dimensional mod(cid:173)
`eling, and other tools to create a greater sense of virtual
`interactivity between consumers and products.
`
`[0006] One industry which has experienced this trend has
`been the real property marketing industry. For years, real
`estate marketers have experienced the difficulties inherent in
`dealing with large amounts of data from large numbers of
`sources. Services, such as the Multiple Listing Service
`(MLS) help provide some real property listings, but finding
`unlisted sellers and potential buyers often requires mass
`advertising and marketing. Traditionally,
`this
`includes
`expensive methods, such as creating full-color print adver(cid:173)
`tisements, purchasing advertising space in multiple loca(cid:173)
`tions, planting "For Sale" signs and information at locations
`for potential drive-by buyers, holding open houses, and
`guiding personal walk-throughs.
`
`[0007] The rise of the Internet has expanded the reach of
`advertising and has allowed for enhanced marketing capa-
`
`bilities at much lower cost. Still, real estate marketers and
`agents have found it difficult to communicate the feel of real
`property to prospective buyers in a virtual space. For
`example, while it is well accepted in the industry that
`property listings which include photos tend to be more
`successful at luring prospective buyers, it is often still
`difficult to relate the multiple image viewpoints together to
`convey a cohesive sense of the real property to a potential
`buyer.
`
`[0008] To remedy that and other problems, the industry
`has begun to see benefits to video marketing. Through video
`media, potential buyers can take virtual tours of properties,
`and even listen to audio associated with the video. While this
`type of information used to be very expensive to distribute
`(e.g. through shipping video cassette tapes), the Internet
`provides an inexpensive and efficient video distribution
`medium. Further, newer video compression technologies
`and the prevalence of home broadband connections have
`made it practical to distribute audiovisual advertisements to
`potential buyers around the world.
`
`[0009] There remain, however, significant disadvantages
`to video marketing. One disadvantage is the cost of produc(cid:173)
`ing a quality result. Though most home computers today
`come with user-friendly photo editing software, much of the
`video editing software and hardware is cumbersome, expen(cid:173)
`sive, and difficult to learn. The result is that the average
`property seller or marketer must either spend a significant
`amount of money to hire a videographer to shoot and process
`video, or spend a significant amount of time learning to use
`a video editing system.
`
`[0010] Another disadvantage is that, where photos often
`provide too little information to consumers, videos may
`provide too much to be useful. For example, say a consumer
`would like to find a house with a large kitchen. It may be
`easy to find photos of a kitchen, but difficult to really
`experience the size and utility of the kitchen from those
`photos. On the other hand, video of the kitchen may provide
`a much better representation of the space, but may be
`difficult to find within the footage of the entire house.
`Further, while it may be simple to compare photos of many
`kitchens, it may be cumbersome and time-consuming to find
`and compare video footage.
`
`[0011] Similar difficulties arise in the vast majority of
`property marketing industries, whether the property is real
`estate, consumer goods, services, or any other community of
`buyers and sellers. Thus, though these industries understand
`the vast advantages of video marketing over photos or plain
`text, the transition has been slow. The industry has remained
`in need of simple methods and systems for handling online
`video for property marketing.
`
`BRIEF SUMMARY OF THE INVENTION
`
`[0012] Embodiments of the invention can address this
`condition in the art by providing methods and systems for
`handling montage video data using cue point information.
`
`[0013] A first set of embodiments provides a system for
`generating video using cue point information. The system
`comprises a video server configured to serve a set of base
`video files indexable by a set of video identifiers; a cue point
`server configured to serve a set of cue point information; a
`cue point handler configured to generate a relational cue
`
`

`

`US 2007/0263984 Al
`
`Nov. 15, 2007
`
`2
`
`point dataset comprising a set of associations between at
`least a portion of the set of cue point information and at least
`a portion of the set of video identifiers; and a video generator
`configured to generate a video montage file based at least in
`part on the relational cue point dataset.
`
`[0014]
`In other embodiments, the system further com(cid:173)
`prises a montage cue point generator configured to generate
`a set of montage cue points based at least in part on the
`relational cue point dataset. In certain of these embodiments,
`the system further comprises a video playback environment
`configured to transmit data readable by a client terminal to
`display the video montage file, wherein the video playback
`environment is configured to transmit data readable by a
`client terminal to display the set of montage cue points.
`
`[0015]
`In some embodiments, the system further com(cid:173)
`prises a video playback environment configured to transmit
`data readable by a client terminal to display the video
`montage file. In other embodiments, the cue point informa(cid:173)
`tion substantially defines at least one of a metatag, a cue
`point name, a cue point time location, or a cue point
`hierarchy. In still other embodiments, the video generator is
`configured to concatenate a plurality of sets of base video
`data to generate at least a portion of the montage video file.
`
`[0016]
`In yet other embodiments, the video generator is
`configured to extract a set of base video data beginning at a
`time location within one of the set of base video files. In
`certain of these embodiments, the time location is based at
`least in part on a portion of the set of cue point information.
`
`[0017]
`In certain embodiments, the video server is in
`operative communication with at least one of a network or
`the video generator. In other embodiments, the cue point
`server is in operative communication with at least one of the
`network, the video server, or the video generator. In still
`other embodiments, the set of associations comprises a
`representation of a subset of the set of video base files
`relating to the subset of cue point information substantially
`matching an association criterion. And in certain of these
`embodiments, the set of cue point information comprises a
`set of cue point names, and the association criterion com(cid:173)
`prises at least one of the set of cue point names.
`
`[0018]
`In some embodiments, at least one base video file
`comprises footage relating to at least one of real property,
`personal property, or information property. And in other
`embodiments, at least a portion of the set of cue point
`information defines a set of metatags, and at least a portion
`of the set of metatags represent a subordinate category of a
`different portion of the set of metatags.
`
`[0019] A second set of embodiments provides a method
`for generating video using cue point information. The
`method provides a computer system having at least one
`computer-readable medium comprising a video playback
`environment, and the computer system is in operative com(cid:173)
`munication with a network. The method further receives
`over the network, using the computer system, a video
`montage request; analyzes the video montage request; gen(cid:173)
`erates a relational dataset based at least in part on the video
`montage request, a set of cue point information, and a set of
`video information; and generates a video montage file based
`at least in part on the relational dataset.
`
`[0020]
`In some embodiments, the method further trans(cid:173)
`mits data readable by a client terminal to display the video
`
`montage file. In other embodiments, the cue point informa(cid:173)
`tion substantially defines at least one of a metatag, a cue
`point name, a cue point time location, or a cue point
`hierarchy.
`
`[0021] Also, in certain embodiments, the at least one
`computer-readable medium further comprises a montage cue
`point generator configured to generate a set of montage cue
`points based at least in part on the relational dataset. In
`certain of these embodiments, the method further generates
`a set of montage cue points. Also in certain of these
`embodiments at least a portion of the set of montage cue
`points represents a set of metatags corresponding to a
`hierarchical categorization of at least a portion of the set of
`video information. Additionally, in certain of these embodi(cid:173)
`