I 1111111111111111 11111 1111111111 111111111111111 1111111111 lll111111111111111
`US009612126B2
`
`c12) United States Patent
`Beletski et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 9,612,126 B2
`Apr. 4, 2017
`
`(54) VISUAL TRAVEL GUIDE
`
`(75)
`
`Inventors: Oleg Beletski, Espoo (FI); Sampo
`Tuomas Vaittinen, Helsinki (FI);
`Timo-Pekka Olavi Viljamaa, Helsinki
`(FI); Tommi Juhani Lahti, Tampere
`(FI)
`
`(73) Assignee: NOKIA TECHNOLOGIES OY,
`Espoo (FI)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 2352 days.
`
`(21) Appl. No.: 11/949,290
`
`(22) Filed:
`
`Dec. 3, 2007
`
`(65)
`
`Prior Publication Data
`
`US 2009/0143977 Al
`
`Jun. 4, 2009
`
`(51)
`
`Int. Cl.
`G0lC 21100
`G0lC 21134
`G0lC 21136
`(52) U.S. Cl.
`CPC ......... G0lC 211343 (2013.01); G0lC 211362
`(2013.01)
`
`(2006.01)
`(2006.01)
`(2006.01)
`
`( 58) Field of Classification Search
`CPC combination set(s) only.
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,559,707 A * 9/1996 DeLorme .
`5,948,040 A * 9/1999 DeLorme .
`
`G0lC 21/20
`340/990
`G0lC 21/36
`340/990
`
`~VSG story 05 Microsoft Internet Ellplorer
`
`Add""'
`
`CiOoo,ment,andSelhng,1m.,,11ns,,,oesUopWSG,tory05n1ml
`
`6,401,034 Bl*
`
`6/2002 Kaplan
`
`6,826,472 Bl * 11/2004 Kamei
`
`6,845,321 Bl *
`
`1/2005 Kerns
`
`6,983,203 Bl *
`
`1/2006 Wako.
`
`(Continued)
`
`G0lC 21/3682
`340/988
`G0lC 21/3605
`340/995.19
`G0lC 21/3679
`340/988
`G0lC 21/3682
`340/988
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`1 193 626 Al
`
`4/2002
`
`OTHER PUBLICATIONS
`
`TomTom, portable GPS car navigation systems, <http://web.ar(cid:173)
`chive.org/web/20051231184850/http://www.tomtom.com/>, 2005,
`3 pages.
`
`(Continued)
`
`Primary Examiner - Bhavesh V Amin
`(74) Attorney, Agent, or Firm - Ditthavong & Steiner,
`P.C.
`
`ABSTRACT
`(57)
`A novel method, apparatus, and system are disclosed that
`provide a mobile terminal user with up to date information
`regarding a point/location of interest. A user may identify a
`destination, and subsequently select points of interest at the
`destination from photos, audio or other media displayed to
`the user. Each media item may be geo-tagged or otherwise
`associated with a particular point of interest, tourist location,
`museum, etc. For example, when visiting Washington, D.C.,
`the user may be shown photos of the Washington Monu(cid:173)
`ment, Smithsonian Museums, Lincoln Memorial, and the
`White House. The user may select one or more of the points
`of interest, and the system then generates an itinerary and/or
`a walking/driving/biking travel route responsive to the
`selected media items.
`
`19 Claims, 13 Drawing Sheets
`
`212
`
`~®!®re
`(Clf'®~~®
`w~i~
`
`Petitioner Apple Inc. - Ex. 1007, p. 1
`
`

`

`US 9,612,126 B2
`Page 2
`
`(56)
`
`References Cited
`
`2009/0143977 Al *
`
`6/2009 Beletski
`
`U.S. PATENT DOCUMENTS
`
`G0lC 21/362
`701/533
`
`7,321,826
`
`2002/0138196
`
`2003/0036848
`
`2003/0158658
`
`2003/0191578
`
`2004/0030678
`2004/0243306
`
`2005/0049765
`
`2005/0107949
`
`2006/0229807
`
`2006/0253247
`
`2007/0118281
`
`2007/0276596
`
`2008/0082262
`
`2008/0268876
`
`B2 *
`
`1/2008 Sheha
`
`Al*
`
`9/2002 Polidi
`
`Al*
`
`2/2003 Sheha
`
`Al*
`
`8/2003 Roever
`
`Al* 10/2003 Paulauskas
`
`G0lC 21/3632
`340/995.24
`G0lC 21/3682
`701/409
`G0lC 21/3679
`701/468
`G0lC 21/3605
`701/533
`G0lC 21/3682
`701/438
`Al*
`2/2004 Tu ............................. G06F 7/22
`Al* 12/2004 Han
`G0lC 21/3682
`701/438
`G06Q 30/02
`701/31.4
`G0lC 21/3611
`701/532
`G0lC 21/3679
`701/468
`G0lC 21/3611
`701/426
`G0lC 21/3647
`701/431
`G0lC 21/3632
`701/431
`G0lC 21/3682
`701/431
`G06Q 30/02
`455/457
`
`Al*
`
`3/2005 Chetia .
`
`Al*
`
`5/2005 Yokota.
`
`Al* 10/2006 Sheha
`
`Al* 11/2006 de Silva .
`
`Al*
`
`5/2007 Adam
`
`Al* 11/2007 Solomon.
`
`Al*
`
`4/2008 Silva.
`
`Al* 10/2008 Gelfand
`
`OTHER PUBLICATIONS
`
`XTech 2006: "Building Web 2.0", <http://2006.xtech.org/schedule/
`paper/195>, May 16-19, 2006, Amsterdam, The Netherlands, 6
`pages.
`<http://web.archive.org/web/
`Sharing-Places.com,
`20070206122435/preview.sharing-places.corn/>, Feb. 6, 2007, 1
`page.
`Welcome to Flickr-Photo Sharing, <http://web.archive.org/web/
`20051231152551/http://www.flickr.com/>, 2005, 1 page .
`Panoramio-Photos of the World, <http://web.archive.org/web/
`2006120 l 100603/http://www.panoramio.com/>, Dec. 1, 2006, 2
`pages.
`<http://web.archive.org/web/20050923183613/http://
`Picasa,
`picasa.google.corn/>, 2005, 1 page.
`Google Earth-Explore, Search and Discover, <http://web.archive.
`org/web/20051231043407/http://earth.google.com/>, 2005, 1 page.
`Placeopedia.com-Connecting Wikipedia articles with their loca-
`<http://web.archive.org/web/2006 l l 071427 57 /http://www.
`tions,
`placeopedia.corn/, Nov. 7, 2006, 2 pages.
`Extended European Search Report for EP 08 17 0172 dated Jun. 26,
`2012, pp. 1-5.
`Office Action for corresponding European Patent Application No.
`08170172.4-1557, dated Jan. 15, 2016, 5 Pages.
`Dffice Action for corresponding European Patent Application No.
`08 170 1724-1557, dated Sep. 21, 2016, 4 pages.
`
`* cited by examiner
`
`Petitioner Apple Inc. - Ex. 1007, p. 2
`
`

`

`U.S. Patent
`
`Apr. 4, 2017
`
`Sheet 1 of 13
`
`US 9,612,126 B2
`
`100
`\
`
`140
`
`DEV2
`
`130
`
`110
`120 .--------'----,
`DEV1
`
`I
`I
`I
`
`I
`I
`
`160a
`: :
`: :
`:
`,'
`I
`1 ________ L ________ _
`I
`----------r-----------
`,
`I 170a
`
`I
`
`I
`I
`
`I
`
`FIG. 1
`
`:
`170b
`: __ ---------~--------'_:
`
`I
`I
`I
`
`I
`I
`
`I
`/
`
`I
`I
`
`160b
`
`Petitioner Apple Inc. - Ex. 1007, p. 3
`
`

`

`U.S. Patent
`
`Apr. 4, 2017
`
`Sheet 2 of 13
`
`US 9,612,126 B2
`
`212
`
`254
`\
`
`254
`\
`
`254
`\
`
`254
`\
`
`252
`
`228
`
`234
`
`240
`
`242
`
`244
`
`RADIO
`RECEIVER
`
`DVB
`RECEIVER
`
`WLAN
`TRANSCEIVER
`
`TELECOM
`TRANSCEIVER
`
`USER INTERFACE
`
`BATTERY
`
`230
`
`250
`
`FIG. 2
`
`Petitioner Apple Inc. - Ex. 1007, p. 4
`
`

`

`U.S. Patent
`
`Apr. 4, 2017
`
`Sheet 3 of 13
`
`US 9,612,126 B2
`
`300
`\
`
`----------------~
`: CREATE/UPDATE
`:.,..-302
`:
`USER PROFILE
`:
`~~~~~~~~t~~~~~~~~
`SELECT
`:,,-306
`DESTINATION
`:
`--------7--------
`________ t _______ ~
`: STARTINGAND
`:.,..-308
`: DURATIONAL DATA :
`~~~~~~~~t~~~~~~~~
`:
`SEND INFO.
`[.,.--310
`:
`TO USER
`
`I
`
`I
`
`I
`
`I
`I
`I
`I
`I
`I
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`I
`
`316
`
`320
`
`326
`
`332
`
`USER MARKS
`INFO. OF INTEREST
`
`GENERATE
`ROUTE
`
`SEND
`SUPPLEMENTARY
`INFO. TO USER
`
`RECEIVE USER
`FEEDBACK
`
`FIG. 3
`
`Petitioner Apple Inc. - Ex. 1007, p. 5
`
`

`

`0--, = N
`'" """' N
`"""' N
`0--,
`_."-0
`
`d r.,;_
`
`~
`
`....
`0 ....
`..... ...
`rJJ =(cid:173)
`
`('D
`('D
`
`0 ....
`'" ...
`t :-:
`
`N
`
`-....J
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`FIG. 4
`
`HOME USER
`
`IMAGES AND MAPS 8~ 420
`
`DESTINATION
`
`MOBILE USER
`
`SYSTEM 8~
`
`TRAVEL GUIDE
`USER OF VISUAL
`
`414
`
`GUIDE SERVER
`VISUAL TRAVEL
`
`POSITION
`
`DB
`
`SERVER
`IMAGE
`
`~
`
`408a
`
`408
`
`GEO
`PHOTOS I
`I
`
`& UPLOAD
`
`TAGGED
`
`402
`
`INTERNET SERVER SITES
`
`CONTENT CREATOR
`
`Petitioner Apple Inc. - Ex. 1007, p. 6
`
`

`

`~
`
`....
`0 ....
`Ul
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`0 ....
`'" ...
`t :-:
`
`N
`
`--.J
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`~ Rhodes,
`
`August, 2005
`
`June 16, 2006
`Moscow
`
`June 16, 2006
`Moscow
`
`212
`
`llinks » j '@ v
`
`vi~ Go
`
`I 83
`[;)~~
`
`514
`
`508b
`
`508a
`
`502 --
`
`JS,
`
`lg] • D ca
`
`A.Q.dress f~ C:\Documents and Setlings\mwilinski\Desktop\VSG story 05. html
`@Back. @· @l i © I psea,ch *Fa,ontes ® I@·~
`
`File Edit View
`~ VSG story 05 Microsoft Internet Explorer
`
`Toob H!:!lp
`
`Favur11es
`
`l:9I~11~1
`tim
`
`0--, = N
`
`""'"' N
`'"
`""'"' N
`0--,
`_."-0
`
`d r,r;_
`
`I I I I I l@1nternet
`
`532
`
`FIG. 5
`
`1111
`
`~Done
`
`New Trip
`
`526
`J
`
`January 20, 2007
`New York
`
`520
`September 25, 2006 J
`Paris
`
`Petitioner Apple Inc. - Ex. 1007, p. 7
`
`

`

`U.S. Patent
`U.S. Patent
`
`Apr. 4, 2017
`Apr. 4, 2017
`
`Sheet 6 of 13
`Sheet 6 of 13
`
`US 9,612,126 B2
`US 9,612,126 B2
`
`~ fii
`~
`[]
`
`:::a
`
`0
`'-'
`[fl
`>
`
`~
`
`@l
`D
`
`IN]
`
`C'-J
`
`~
`
`1111
`
`
`
`
`
`
`
`pueyss]‘wigng
`
`‘uoleul]seq
`
`~
`® -
`~
`
`C'-J
`
`(
`>
`@
`
`YOSOIIIWGOA1O}sOSA
`
`
`
`
`
`
` SDO-8BB]Swohomef|BE@-O-© djaHSlOOL
`Jwyy“egA108QSA\doyysaqyysulMUysOunjagpuespuawnaoqg\.>it]ssosppy
`
`
`
`
`
`·~
`
`u,
`
`~
`
`~
`
`<(
`
`joulalyl@||
`
`(,0
`
`9Old
`
`(9
`LL
`
`
`
`Petitioner Apple Inc. - Ex. 1007, p. 8
`
`
`
`C
`0
`
`~
`{:r =
`
`~
`
`., ~ t
`
`9
`0
`~
`a. !?c
`>< @
`w ~
`j,
`1i>
`C: "' ~
`I 0
`-= ml
`~ -
`0
`t; @
`~
`:g ~
`9
`i::' >
`u
`~ ;'I
`a, ~
`Cl @
`Cf) >
`-
`ti.,]
`
`
`
`JaJojdxgyale}
`
`~
`
`0
`
`SayJOAeyMaiAUPAang
`
`Petitioner Apple Inc. - Ex. 1007, p. 8
`
`

`

`0--, = N
`
`"'""' N
`"'
`"'""' N
`O'I
`\0
`d r.,;_
`
`I I I I I © Internet
`
`l!!Joone
`
`FIG. 7
`
`~
`
`....
`0 ....
`-...J
`.....
`rJJ =- ('D
`
`('D
`
`~~~
`~~;~·
`
`.
`
`. . ~ /
`.··. ~ f':::::--.. .
`
`u n
`
`. L.
`
`.
`
`0 ....
`"' ...
`t :-:
`
`N
`
`-...J
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`212
`
`~
`
`llmks"l<iIDv
`
`Vl~Go
`
`I BJ
`[;J@[g)
`
`fill · Q ~ fS
`
`(E} I@•®
`
`@sack• @ • @] ~ (@ I J)seacch *Favantes
`
`F le Edit View
`l!JJ VSG story 05 Microsoft Internet Explorer
`
`Favo1ites Tuols Help
`
`Petitioner Apple Inc. - Ex. 1007, p. 9
`
`

`

`U.S. Patent
`U.S. Patent
`
`Apr. 4, 2017
`Apr.4, 2017
`
`Sheet 8 of 13
`Sheet 8 of 13
`
`US 9,612,126 B2
`US 9,612,126 B2
`
`ous&||||||oaL[x]mMIg[|
`
`
`
`Kee)Jasojdxqyouaju|YosossyyGOAlo}sHSA
`
`~®|«aun9(SJa1WId“SpAlnsOSA\dONsaq\ysuyMUASBUNAspuesjuaunaoq\:9ip!ssaJppy
`
`
`
`
`eal|djaH=$}0c,«SaOARY«MBIA.YIDQ_HE
`
`
`
`
`payaayasgjoQ<AlamaiqssouingS
`Wwwy(]aE|&“|QBsayoregIRwieasey|ch@|“@aee)
`
`
`
`
`
`
`V8Ols
`
`<(
`co
`(9
`LL
`
`■
`
`Petitioner Apple Inc. - Ex. 1007, p. 10
`
`lv]
`(
`
`>
`
`
`lt-----11
`
`..,___--II
`
`°;;
`u
`"'
`~
`"' o
`
`C>
`
`~
`"' ;;::
`"'
`
`!~I~=;;,
`
`
`
`
`~
`
`{Z i
`
`
`
`
`
`
`
`
`
`-~
`
`Petitioner Apple Inc. - Ex. 1007, p. 10
`
`

`

`0--, = N
`
`"'""' N
`'"
`"'""' N
`0--,
`'"'-.0
`
`d r.,;_
`
`I® Internet
`
`FIG. 88
`
`1111
`
`~Done
`
`~
`
`808
`
`....
`0 ....
`.....
`rJJ =(cid:173)
`
`1,0
`
`('D
`('D
`
`(,H
`
`0 ....
`'" ...
`t :-:
`
`N
`
`-....J
`
`d ~
`[ 11° /] 802
`
`0 of 6 selected
`
`< ~ .§
`
`mix being added during the serving process. It is
`The thick creamy head is the result of a nitrogen
`Guinness has refused to confirm if this still occurs.
`the beer was aged to give a sharp lactic flavour, but
`remains unfermented. For many years a portion of
`feature in the flavour is the roasted barley which
`world, being exported world wide. The distinctive
`is one of the most successful beer brands in the
`style that originated in London in the early 1700s. It
`Dublin, Ireland. The beer is based upon the porter
`Arthur Guinness's St. James's Gate Brewery in
`Guinness ['g?nas] is a dry stout that originated in
`
`For other uses, see Guinness (disambiguation).
`
`From Wikipedia, the free encyclopedia
`Guinness
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`212
`
`iunks »J ~ v
`
`vi~ Go
`
`I l:8
`Q~IR]
`
`<J Guiness brewery [>
`
`iS
`
`<eJ) I@•® ~·D ITi
`
`Address I~ C:\Documonts and Se_!!,ngs\mwilinski\Oesklop\VSG slory 05. html
`@3ack • @ • @l ~ E@ I psearch *Favor;tes
`
`File Edit V ew
`~ VSG story 05 Microsoft Internet Explorer
`
`foo,s Help
`
`Favorites
`
`Petitioner Apple Inc. - Ex. 1007, p. 11
`
`

`

`U.S. Patent
`U.S. Patent
`
`Apr. 4, 2017
`Apr. 4, 2017
`
`Sheet 10 of 13
`Sheet 10 of 13
`
`US 9,612,126 B2
`US 9,612,126 B2
`
`ssauing> tary“SOAloysOSA\doysagyysulauUyssaspue
` sve]
`
` eSHO-fB&-3G
`omey|BEB-O©-G
`noskh
`sjuauinsog\:9@Ssauppy
`
`
`Ja10|dxqJeusazU]YOSOIDIW|GQAIO}SNSALH
`djaHsfOOLSaOARJ=MBIAPJBily
`
`
`&)[s}E)
`
`(gJ EB
`Ae|
`~
`[J]
`
`@
`
`-BI«
`
`0
`C)
`
`2iA]
`
`[fl
`
`
`
`;'::'
`a,
`
`<]Alamaig
`3: !~
`
`~~I ; @
`I~~ ~ e
`
`
`
`C'-1
`
`C'-1
`
`
`
`~ , _ __.
`p3}2a[as9JO0
`
`-;:;
`a,
`a,
`
`"' <D
`
`0 - -0
`
`emerTT1
`
`|
`
`0)
`■ CJ
`LL
`
`co
`c::,
`0-,
`
`C'-1
`c::,
`0-,
`
`Petitioner Apple Inc. - Ex. 1007, p. 12
`
`Petitioner Apple Inc. - Ex. 1007, p. 12
`
`

`

`U.S. Patent
`
`Apr. 4, 2017
`
`Sheet 11 of 13
`
`US 9,612,126 B2
`
`<(
`0
`~ .
`(.9
`LL
`
`..
`
`C'J
`
`~
`
`C'J
`
`nu
`
`~ E[j
`~
`[]
`
`>
`
`(
`
`~ -
`
`~
`
`:_-:;
`
`0
`CJ
`
`[fl
`
`., .
`..
`••
`., ~-.
`
`D
`
`I
`
`'
`
`. · : 0
`
`• '
`
`6
`~ -;;;
`"' u
`-~
`:0
`=>
`0
`
`v
`
`~
`
`@]
`D
`
`[i:i]
`
`c® .
`<&
`)
`
`- @
`
`c
`0
`
`¢
`
`~
`
`~
`
`~
`0
`
`0
`
`~
`?
`
`m
`9
`
`~
`
`~
`
`°"
`~
`0
`a.
`:i; @
`><
`w
`Q) 1
`E 0 ~
`2l c°
`-=
`.
`m Lill ~
`;z::
`0
`en !
`e
`u @
`c:' > . i..i
`!
`:E:
`.,.,
`~
`
`0
`
`0
`ui
`~
`(!) @
`en >
`~ u:
`
`~
`
`.;/
`
`8
`V ~
`
`Petitioner Apple Inc. - Ex. 1007, p. 13
`
`

`

`0--, = N
`
`""'"' N
`'"
`""'"' N
`0--,
`_."-0
`
`d r.,;_
`
`I® Internet
`
`FIG. 10B
`
`1111
`
`908
`
`902
`
`1008
`
`~Done
`
`~
`
`....
`0 ....
`N
`....
`.....
`rJJ =- ('D
`
`('D
`
`0 ....
`'" ...
`t :-:
`
`N
`
`-....J
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`"'I
`
`212
`
`,:o
`
`QC
`
`ll1i!l ~
`
`-
`
`,d ••
`
`~-
`= -
`
`CC>
`
`~
`
`ing_s
`Marty: Rather small place surrounded by dirty build(cid:173)
`
`of one tower
`Alex: pretty boring place, nothing much to see except
`
`Gloria: I like it a lot, maybe because of good weather
`
`1 of 6 selected
`
`<J Dublin castle C>
`
`1014
`
`IL:nks » 1 <@ ,
`
`vi [f] Go
`
`Q~~
`
`IES
`
`lg] • D n !£1
`
`Mdress ~ C:\Documents and Setlmgs\mw1l1,1ski\Desktop\VSG story 05. html
`
`@sac,. @. ~ ~ © I J)search *Favorites ~ I@·®
`
`File Edit View
`~ VSG story 05 Microsoft Internet Explorer
`
`Fa\.'o"iles Too,s Help
`
`Petitioner Apple Inc. - Ex. 1007, p. 14
`
`

`

`U.S. Patent
`
`Apr. 4, 2017
`
`Sheet 13 of 13
`
`US 9,612,126 B2
`
`~ EB
`~
`[]
`
`>
`~
`-
`'
`
`::J
`
`0
`<!)
`
`~
`>
`
`=
`
`C"J
`r--
`
`C"J
`C"J
`
`~
`
`C"J
`
`=
`
`IJJI
`
`2
`
`g
`00
`00 ¢:::!)
`~
`00 =
`=
`00 00 00
`<ii
`~ E
`:§
`(!fjlJ
`b
`g
`@
`-
`(!fjlJ
`(Q)
`b
`""
`=
`¢:::!)
`(Q)
`c=
`w ~ c= c=
`0 ~ ¢:::!) ~ 00 [ill:,
`b g
`c= b
`(Q) =
`~ (Q)
`~~
`00 00
`00 ~
`~ (Q)
`b
`
`¢:::!)
`
`¢:::!)
`
`"'
`
`C"J = 00
`
`...---
`...---
`(9
`LL
`
`■
`
`Petitioner Apple Inc. - Ex. 1007, p. 15
`
`

`

`US 9,612,126 B2
`
`1
`VISUAL TRAVEL GUIDE
`
`FIELD
`
`2
`Museum annex near Dulles International Airport, and might
`miss out on the opportunity to visit it based on his lack of
`knowledge.
`
`BRIEF SUMMARY
`
`Aspects of the invention generally relate to computing 5
`technologies used to visually create a trip. More specifically,
`an apparatus, method and system are described for planning
`and obtaining an itinerary and a travel route based on visual
`information regarding a destination or point of interest and
`a user's personal preferences, past activities, and/or collab- 10
`orative filtering.
`
`BACKGROUND
`
`The following presents a simplified summary of aspects
`of the invention in order to provide a basic understanding of
`some aspects of the invention. This summary is not an
`extensive overview, and is not intended to identify key or
`critical elements or to delineate the scope of the claims. The
`following summary merely presents some concepts and
`aspects of the invention in a simplified form as a prelude to
`the more detailed description provided below.
`To overcome limitations in the prior art described above,
`and to overcome other limitations that will be apparent upon
`reading and understanding the present specification, aspects
`of the present invention are directed to a novel apparatus,
`20 method and system for providing access to information in a
`visual, fast, and easy way via one or more computer plat(cid:173)
`forms. More specifically, a user may select one or more
`visual elements from a community pool of information, a
`( commercial) content provider, or the like, and travel a route
`25 based on the selected one or more visual elements. A user
`may obtain additional, supplementary information while
`traveling the route, and the system may infer points of
`interest based on a person's past activities.
`Various aspects of the invention may, alone or in combi-
`3o nation with each other, provide a pool of information based
`on a point of interest within a proximal distance of a
`specified location, enable the specification of a temporal
`characteristic related to when a user intends to visit the
`specified location, provide the user with elements from the
`35 pool of information, and accept input from the user in
`response to the elements from the pool of information
`provided to the user, wherein the input specifies those
`elements of greatest interest to the user. Other various
`40 aspects of the invention may, alone or in combination with
`each other, provide supplementary information to the user
`related to those elements of greatest interest to the user, and
`provide a travel route to the user based on the elements of
`greatest interest to the user.
`These and other aspects of the invention generally relate
`to a user indicating an interest in a destination and/or one or
`more points of interest, either explicitly or implicitly. A user
`may enter a destination and a time that he intends to arrive
`at the destination. A pool of information may be maintained
`50 amongst a user community related to various destinations.
`One or more computer platforms may present elements from
`the pool of information to the user that relate to the user's
`entered destination. A user may select one or more elements
`from the pool of information, and travel a route based on the
`55 selected one or more elements. A user may obtain additional,
`supplementary information while traveling the route.
`
`15
`
`Improvements in computing technologies have changed
`the way people accomplish various tasks. For example, a
`resident of Long Island, N.Y. may plan a road trip to visit
`Washington, D.C. Traditional techniques would require the
`tourist, who is unfamiliar with the details of the trip, to
`obtain printed materials, such as a map, paper book, or travel
`guide in order to chart out the trip. Printed materials are
`frequently outdated, however, and careful planning may be
`frustrated by construction, closed roadways and the like.
`Additional techniques have since been developed in an
`effort to mitigate the short-comings associated with printed
`travel guides. Global Positioning System (GPS) navigational
`devices enable a user to obtain information about points of
`interest with accompanying directions. Geo-tagging of pho(cid:173)
`tographs and videos, and presenting the photographs and
`videos on an accompanying map has enabled users to share
`their experiences with one another regarding points of
`interest. For example, online sharing services like Flick(cid:173)
`r.com, locr.com, and twango.com provide opportunities to
`view a photo on a map representative of where the photo was
`taken. Also Lonely Planet has launched a lonelyplanet.tv
`service, where users can upload their travel videos and see
`them placed on a map. All of these techniques/devices have
`enhanced the state of the art in comparison to the aforemen(cid:173)
`tioned use of traditional print materials, but these tech(cid:173)
`niques/devices have their own deficiencies. The amount of
`information available to a user is often expansive, and the
`user often has to undertake a time-consuming search process
`to pin-point the information that is of greatest interest.
`For example, in relation to the preceding example, the
`tourist from New York may have heard through a friend that,
`in close proximity to Washington, D.C., inventors bring their
`inventions to be reviewed and evaluated for patentability.
`The tourist may have an interest in visiting the location
`referred to by his friend, but may be unaware that the
`location referred to is the United States Patent and Trade(cid:173)
`mark Office (PTO). If the tourist's friend is not available to
`discuss the matter, the tourist will have to undertake a
`time-consuming search ( e.g., reading articles, biogs, web(cid:173)
`postings and the like) to pin-point the PTO as a place of
`interest. Thereafter, in order to visit the PTO, the tourist will
`need to learn where the PTO is located. The tourist, however,
`may acquire information via a web search directing him to
`the previous location of the PTO, Crystal City, Va., as 60
`opposed to the PTO's current location in Alexandria, Va.
`Moreover, the tourist may have an interest in airplanes,
`having viewed an airplane stunt show five years ago. Thus,
`the tourist may have an interest in visiting the Smithsonian's
`National Air and Space Museum while visiting Washington, 65
`D.C. The tourist, however, might not even be aware that
`such an exhibit exists, or that there exists an Air & Space
`
`45
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`A more complete understanding of the present invention
`and the advantages thereof may be acquired by referring to
`the following description in consideration of the accompa(cid:173)
`nying drawings, in which like reference numbers indicate
`like features, and wherein:
`FIG. 1 illustrates a network computing environment suit(cid:173)
`able for carrying out one or more illustrative aspects of the
`invention.
`
`Petitioner Apple Inc. - Ex. 1007, p. 16
`
`

`

`US 9,612,126 B2
`
`3
`FIG. 2 illustrates a data processing architecture suitable
`for carrying out one or more illustrative aspects of the
`invention.
`FIG. 3 illustrates a flow chart depicting a method suitable
`for carrying out one or more aspects of the invention.
`FIG. 4 illustrates a use case scene wherein one or more
`illustrative aspects of the invention may be practiced.
`FIGS. 5 through 11 illustrate a use case scenario wherein
`one or more illustrative aspects of the invention may be
`practiced.
`
`DETAILED DESCRIPTION
`
`In the following description of the various embodiments,
`reference is made to the accompanying drawings, which
`form a part hereof, and in which is shown by way of
`illustration various embodiments in which one or more
`aspects of the invention may be practiced. It is to be
`understood that other embodiments may be utilized and
`structural and functional modifications may be made without 20
`departing from the scope of the present invention.
`Conventional current web services and guide books are
`based on written descriptions and small amounts of com(cid:173)
`mercially created photos and/or videos. The services do not
`know where a user has been traveling and what he likes to
`do while on vacation. Thus, as demonstrated herein, one or
`more aspects of the invention provide for the ability to plan
`a trip in a visual, fast and easy to use way while obtaining
`the latest information available about a travel destination.
`One or more aspects of the invention provide an ability to 30
`match a person's interests to activities located at the desti-
`nation.
`FIG. 1 illustrates a network computing environment 100
`suitable for carrying out one or more aspects of the present
`invention. For example, FIG. 1 illustrates a first device
`DEVl 110 (e.g., device 212, FIG. 2) connected to a network
`130 via a connection 120. Network 130 may include the
`Internet, an intranet, wired or wireless networks, or any
`other mechanism suitable for facilitating communication
`between computing platforms in general. FIG. 1 also depicts
`a second device DEV2 140 (e.g., a server) connected to
`network 130 via a connection 150. By virtue of the connec(cid:173)
`tivity as shown, DEVl 110 and DEV2 140 may communi(cid:173)
`cate with one another. Such communications may enable the
`exchange of various types of information. For example, the
`communications may include data to be exchanged between
`DEVl 110 and DEV2 140. Such data may include images,
`files, and the like. The communications may further include
`additional information such as control information.
`Connections 120 and 150 illustrate interconnections for 50
`communication purposes. The actual connections repre(cid:173)
`sented by connections 120 and 150 may be embodied in
`various forms. For example, connections 120 and 150 may
`be hardwired/wireline connections. Alternatively, connec(cid:173)
`tions 120 and 150 may be wireless connections. Connections
`120 and 150 are shown in FIG. 1 as supporting bi-directional
`communications (via the dual arrow heads on each of
`connections 120 and 150). Alternatively, or additionally,
`computing environment 100 may be structured to support
`separate forward (160a and 160b) and reverse (170a and
`170b) channel connections to facilitate the communication.
`Computing environment 100 may be carried out as part of
`a larger network consisting of more than two devices. For
`example, DEV2 140 may exchange communications with a
`plurality of other devices (not shown) in addition to DEVl
`110. The communications may be conducted using one or
`more communication protocols. Furthermore, computing
`
`4
`environment 100 may include one or more intermediary
`nodes (not shown) that may buffer, store, or route commu(cid:173)
`nications between the various devices.
`FIG. 2 illustrates a generic computing device 212, e.g., a
`5 desktop computer, laptop computer, notebook computer,
`network server, portable computing device, personal digital
`assistant, smart phone, mobile telephone, cellular telephone
`( cell phone),
`terminal, distributed computing network
`device, mobile media device, or any other device having the
`10 requisite components or abilities to operate as described
`herein. As shown in FIG. 2, device 212 may include pro(cid:173)
`cessor 228 connected to user interface 230, memory 234
`and/or other storage, and display 236. Device 212 may also
`include battery 250, speaker 252 and antennas 254. User
`15 interface 230 may further include a keypad, touch screen,
`voice interface, four arrow keys, joy-stick, stylus, data
`glove, mouse, roller ball, touch screen, or the like. In
`addition, user interface 230 may include the entirety of or
`portion of display 236.
`Computer executable instructions and data used by pro-
`cessor 228 and other components within device 212 may be
`stored in a computer readable memory 234. The memory
`may be implemented with any combination of read only
`memory modules or random access memory modules,
`25 optionally including both volatile and nonvolatile memory.
`Software 240 may be stored within memory 234 and/or
`storage to provide instructions to processor 228 for enabling
`device 212 to perform various functions. Alternatively, some
`or all of the computer executable instructions may be
`embodied in hardware or firmware (not shown).
`Furthermore, the computing device 212 may include
`additional hardware, software and/or firmware to support
`one or more aspects of the invention as described herein. For
`example, computing device 212 may include a camera (not
`35 shown) and/or audiovisual (e.g., movie/film) support soft(cid:173)
`ware/firmware. Device 212 may be configured to receive,
`decode and process digital broadband broadcast transmis(cid:173)
`sions that are based, for example, on the Digital Video
`Broadcast (DVB) standard, such as DVB-H, DVB-T or
`40 DVB-MHP, through a specific DVB receiver 241. Digital
`Audio Broadcasting/Digital Multimedia Broadcasting
`(DAB/DMB) may also be used to convey television, video,
`radio, and data. The mobile device may also include other
`types of receivers for digital broadband broadcast transmis-
`45 sions. Additionally, device 212 may also be configured to
`receive, decode and process transmissions through FM/ AM
`Radio receiver 242, WLAN transceiver 243, and telecom(cid:173)
`munications transceiver 244. In at least one embodiment of
`the invention, device 212 may receive radio data stream
`(RDS) messages.
`Device 212 may use computer program product imple(cid:173)
`mentations including a series of computer instructions fixed
`either on a tangible medium, such as a computer readable
`storage medium (e.g., a diskette, CD-ROM, ROM, DVD,
`55 fixed disk, etc.) or transmittable to computer device 212, via
`a modem or other interface device, such as a communica(cid:173)
`tions adapter connected to a network over a medium, which
`is either tangible ( e.g., optical or analog communication
`lines) or implemented wirelessly ( e.g., microwave, infrared,
`60 radio, or other transmission techniques). The series of com(cid:173)
`puter instructions may embody all or part of the function(cid:173)
`ality with respect to the computer system, and can be written
`in a number of programming languages for use with many
`different computer architectures and/or operating systems,
`65 as would be readily appreciated by one of ordinary skill. The
`computer instructions may be stored in any memory device
`( e.g., memory 234), such as a semiconductor, magnetic,
`
`Petitioner Apple Inc. - Ex. 1007, p. 17
`
`

`

`US 9,612,126 B2
`
`5
`
`5
`optical, or other memory device, and may be transmitted
`using any communications technology, such as optical infra(cid:173)
`red, microwave, or other transmission technology. Such a
`computer program product may be distributed as a remov(cid:173)
`able storage medium with accompanying printed or elec-
`tronic documentation (e.g., shrink wrapped software), pre(cid:173)
`loaded with a computer system (e.g., on system ROM or
`fixed disk), or distributed from a server or electronic bulletin
`board over a network ( e.g., the Internet or World Wide Web).
`Various embodiments of the invention may also be imple- 10
`mented as hardware, firmware or any combination of soft(cid:173)
`ware (e.g., a computer program product), hardware and
`firmware. Moreover, the functionality as depicted may be
`located on a single physical computing entity, or may be
`divided between multiple computing entities.
`In at least one embodiment, device 212 may include a
`mobile client implemented in a C-based, Java-based,
`Python-based, Flash-based or any other progranmiing lan(cid:173)
`guage for the Nokia® S60/S40 platform, or in Linux for the
`Nokia® Internet Tablets, such as NS00 and N810, and/or 20
`other implementations. Device 212 may communicate with
`one or more servers over Wi-Fi, GSM, 3G, or other types of
`wired and/or wireless connections. Mobile and non-mobile
`operating systems (OS) may be used, such as Windows
`Mobile®, Palm® OS, Windows Vista® and the like. Other 25
`mobile and non-mobile devices and/or operating systems
`may also be used.
`By way of introduction, aspects of the invention provide
`a traveler ( e.g., a tourist) the ability to obtain the latest
`information available regarding a destination he ( or she) 30
`intends to visit. The traveler may enter a destination he
`intends to visit into a computing platform (e.g., device 212).
`After inputting the destination, a start date and the duration
`of the trip may be requested. Thereafter, the computing
`platform may display photos, images, videos and/or any 35
`other media item types ( e.g., text, audio, multimedia, graph(cid:173)
`ics, animation, interactive video and the like) from the
`entered destination. Thereafter, the traveler may view pho(cid:173)
`tos, images, videos, and/or any other media item types and
`the like on his device 212 depicting various locations of 40
`interest at the entered destination, e.g., notable points of
`interest, museums, architecture, amusement parks, historical
`landmarks, monuments, etc. The material may be organized
`or ranked based on history, habits, or interests of the traveler,
`or based on collaborative filtering. The traveler may also 45
`access additional information in the form of textual and
`visual descriptions, visitor commentaries, reviews, and the
`like corresponding to the photos, images, videos, and/or any
`other media item types. The photos, images, videos, any
`other media item types and information may be filtered to 50
`(only) provide the traveler with points of interest that will be
`open when the user is scheduled to be located in proximity
`of the selected destination. The traveler may select from the
`photos, images, videos, and/or any other media item types
`those points of interest that he wants to see while visiting the 55
`destination. Each individual photo, image, video, or any
`other media item type may relate to a single candidate point
`of interest. Alternatively, or additionally, each individual
`photo, image, video, or any other media item type may
`contain multiple candidate points of interest, and a traveler 60
`may have an opportunity to select from the multiple candi(cid:173)
`date points of interest. The selection may, e.g., be based on
`the tourist's interest or schedule in visiting each depicted
`location. Aspects of the invention may then generate a route
`of travel for the tourist in the destination, whereby when the 65
`tourist follows the generated route of travel the tourist visits
`each location identified as being of interest to the tourist.
`
`6
`FIG. 3 depicts a flow chart describing a method 300
`suitable for carrying out one or more aspects of the invention
`as described herein. Method 300 may be executed on any
`suitable computing platform (e.g., computing device 212).
`More specifically, method 300 may be executed in conjunc(cid:173)
`tion within a (web) browser or the like, such as via a
`client/server, Java, Java Script, AJAX, applet, Flash®, Sil(cid:173)
`verlight™, or other applications, operating systems, devices
`and the like.
`In step 302, a user profile is created and/or updated. The
`use