Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 1 of 61 PageID #: 693
`Case 2:17-cv—00513-JRG Document 37-3 Filed 11/14/17 Page 1 of 61 PageID #: 693
`
`E X H IB IT 21
`EXHIBIT 2 1
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 2 of 61 PageID #: 694
`
`l|||||||||||||ll||l||||||||l|||||||||||||||||||||||||||||||||||||||||l|||l||||||||||||||||
`
`US 20040054428A1
`
`(19) United States
`(12) Patent Application Publication (10) Pub. N0.: US 2004/0054428 A1
`Sheha et al.
`(43) Pub. Date:
`Mar. 18, 2004
`
`(54) METHOD AND APPARATUS FOR SENDING,
`RETRIEVING AND PLANNING LOCATION
`RELEVANT INFORMATION
`
`(76) Inventors: Michael A. Sheha, Laguna Niguel, CA
`(US); Angie Sheha, Laguna Niguel, CA
`(US); Stephen Petilli, Laguna Niguel,
`CA (US)
`Correspondence Address:
`David T_ Yang
`Morrison & Foerster LLP
`35th Floor
`555 W_ 5th Street
`L05 Angeles’ CA 90013 (Us)
`
`(21) APPL NO;
`
`10/378,980
`
`(22) Filed;
`
`Man 3, 2003
`
`Related US, Application Data
`
`(60) Provisional application No. 60/360,737, ?led on Mar.
`1, 2002.
`
`Publication Classi?cation
`
`(51) Int. C1.7 ................................................... .. G05B 19/18
`(52) us. Cl. .............................................................. .. 700/56
`
`(57)
`
`ABSTRACT
`
`The present invention is directed to a method and apparatus
`for sending and retrieving location relevant information to a
`user by selecting and designating a point of interest that is
`displayed on a graphical user interface and sending the
`location information associated With that point of interest to
`a receiver that is also selected using the graphical user
`interface. The location relevant information may also
`include mapped routes, Waypoints, geo-fenced areas, mov
`ing vehicles etc. Updated location relevant information may
`also be continuously sent to the user While generating
`updated mapping information on the graphical user inter
`face. The present invention may be practiced by using
`communication devices such as a personal computer, a
`personal digital assistance, in-vehicle navigation systems, or
`a mobile telephone.
`
`_F_ile
`
`gdit Miew Boute Messenger Iools _|-_lelp
`
`‘X
`l a‘ 211
`\
`
`\
`\
`
`i
`
`a
`
`Messen erVWndow
`9
`User A a 213
`
`212
`
`Group 1 F 214
`‘_ User1 <— 215
`;— User 2 +— 216
`’ 1- User 3 P" 217
`'— User 4 +— 218
`
`—Group 2 P— 219
`
`Vehicle1 R220
`Vehicle 2 ~Q221
`Vehicle 3 F 222
`Vehicle 4 <—223
`
`‘ 20o
`
`_
`\201
`
`\’\
`\224
`
`\
`
`Burlington
`
`f
`
`202
`\
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 3 of 61 PageID #: 695
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 3 of 61 PageID #: 695
`
`Patent Application Publication
`
`Mar. 18, 2004 Sheet 1 0f 36
`
`US 2004/0054428 A1
`
`um<mwww.mm—B
`
`zo_._.<hm
`
`«NF
`
`
`
`
`
`Numo_>m_o.61mm:5525...!
`
`._o...mz<m._.z_
`
`hm2<mhxm
`
`on»QNFXMNF
`
`«2
`
`m2.
`
`2:.
`
`Stew€3.32
`
`
`
`an“
`
`
`
`
`
`33mm:oraoiaEEoowErzumm
`
`
`
`
`
`{0332
`
`33.85
`
`.8332“.
`
`519
`
`uaucmas.
`
`me
`
`w>_._o
`
`Amoutficc
`
`.33»
`
`tom
`
`$852:
`
`
`
`£23mmchEmom.320
`
`$38fine.
`
`
`
`.wmzos..uhmon>mx
`
`
`
`520m:25...Evinfiw
`
`
`
`.moEmEz.mmma
`
`Awuutuacs
`
` .,in.830Eu:
`
`
`xmfim>_._n_
`
`
`
`Amoutwucs
`
`52.
`
`h:
`
`boEoE2:033.
`
`moi.2205
`
`
`
`mmmuo<Eoucam
`
`E<522:22
`
`5wm0_>mn_.5«mm:
`
`EBmxwact—Ego.
`
`
`
`
`
`
`
`
`
`:oumficaEEooamains:
`
`nus.-EEmoE
`
`San.
`
`we.
`
`
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 4 of 61 PageID #: 696
`
`Patent Application Publication Mar. 18, 2004 Sheet 2 0f 36
`
`US 2004/0054428 A1
`
`. /
`
`Nu mO_>mD 50 1mm:
`
`A
`
`02.
`
`9:
`
`/
`/
`
`/ 2:
`
`vNv
`
`V
`
`N2.
`
`.1. .UE
`
`
`
`,/\, ‘ 32cm wings- . ‘
`
`/7
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 5 of 61 PageID #: 697
`
`Patent Application Publication Mar. 18, 2004 Sheet 3 0f 36
`
`US 2004/0054428 Al
`
`:BmEEm
`
`m3 IV :3:
`
`EN l r 965
`
`EN it v 5w:
`2N It u Em:
`Cu ii 0 an:
`
`m3 Ill u 955
`
`cal _. 22:;
`
`El ~ 202;
`Na i n 22;;
`muwlv w o_uEm> N
`
`\soucghwmcwwwwz N
`
`2N Ill 4. 3m: |
`
`NE
`
`N .QE
`
`
`
`
`
`Em? w_ooH hmmcowmw? 950M 265 Eu 2E.
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 6 of 61 PageID #: 698
`
`Patent Application Publication Mar. 18, 2004 Sheet 4 0f 36
`
`US 2004/0054428 A1
`
`8/
`
`
`
`
`
`SN/ Eo? 28H 30:335. $50M 32H 5.“ 2D
`
`
`
`
`
`
`
`
`
`/ .. B0950
`
`/ / 8m z
`
`/ t... E\ I
`
`m .OI
`
`/ Sm /
`
`/ / E0350 1
`/ 8,... /
`
`/ 2N lit N 9.6.6!
`
`S... LEN Yum: Ir
`WA 2N Ill u 3m: |
`El N 22;;
`5i * 22%; 5 l n 202;
`2" i F n:o._0||
`cal _. 20Eo> i
`
`an .. . M F 3850 l
`
`\W_ / C0250 |
`
`2» 223.5» |
`
`/ 2~l|¢ < 3 NE
`\ Low II
`
`302259332 R
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 7 of 61 PageID #: 699
`
`Patent Application Publication Mar. 18, 2004 Sheet 5 0f 36
`
`US 2004/0054428 A1
`
`new
`
`wow
`
`02mm
`
`2mm
`
`homcowmo? 32H
`
`Eulml 2m
`
`M.
`
`new /
`
`
`
`com-anon 3m: “mus-5i
`
`Q22 :0 2304
`
`/
`
`L
`
`“a.
`
`iiiii.-.
`
`v 6E
`
`m;
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 8 of 61 PageID #: 700
`
`Patent Application Publication Mar. 18, 2004 Sheet 6 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`Ban 23H .amcmmmo? Baum 32H gum. 2U
`
`
`
`
`
`
`
`
`
`\ Al N - cozmu=aa<
`
`
`
`
`
`Eur 28H hwmcommms :59 “EM 2m
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 9 of 61 PageID #: 701
`
`Patent Application Publication Mar. 18, 2004 Sheet 7 0f 36
`
`US 2004/0054428 A1
`
`02mm
`
`
`
`mmw Wm‘
`
`
`
`c0560.; 32.. 325mm E22 :0 2.33
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 10 of 61 PageID #: 702
`
`Patent Application Publication Mar. 18, 2004 Sheet 8 0f 36
`
`US 2004/0054428 A1
`
`com
`
`BN/
`
`F mmwD
`
`2% 28H
`
`
`
`050M $2M
`
`
`
`tulwl 2E.
`
`2.“ i F 9.95
`
`2~ IV < :5:
`
`mg l N am:
`“E Ii 0 how:
`2N , _. how:
`2~ l v 3m: W
`03 [IV N 95514
`
`
`
`Q?i _. 20Em>
`
`El N 22%;
`5i v 22%;
`a“ i n 22%;
`
`Na
`
`m .UE
`
`IJ
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 11 of 61 PageID #: 703
`
`Patent Application Publication Mar. 18, 2004 Sheet 9 0f 36
`
`US 2004/0054428 A1
`
`\ 1/ E." l n 3m: I
`
`
`
`
`WNN/ \ so ‘LEN ‘111v whom: I
`
`@ .QE
`
`
`
`SN \ \_ D3 l ~ .52.. |
`
`2~ .l w Law: |
`
`5i Q 2255 l 5 l n 22%; 1
`5i N 22%; i
`5i F 22%; l
`
`2w Ii N 955111
`
`l *QZOLOI
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 12 of 61 PageID #: 704
`
`Patent Application Publication Mar. 18, 2004 Sheet 10 0f 36 US 2004/0054428 A1
`
`BN/
`
`w- in i _. nsohw||
`
`z
`
`
`
`
`
`i4 ENIIV NE 4233.!
`
`
`
`>>ouE>> $96322 R
`
`66 .QE
`
`a“ l 0 202g l
`
`Mai * 22;; I
`
`5i N 22%“; 1
`
`5i 2.2%; 1
`
`in i u .EEOII
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 13 of 61 PageID #: 705
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 13 of 61 PageID #: 705
`
`Patent Application Publication Mar. 18, 2004 Sheet 11 0f 36
`
`US 2004/0054428 A1
`
`“o
`
`VNN
`
`
`
`nonllv
`
`I:|i:3:
`
`2:lN3w:
`
`2:11n3m:
`
`2:lvEm:
`
`:520
`
`
`
`afilvro_o_—_m>
`
`SiN22:;
`
`Elv202;Ein22;;
`
`2:lN53.0
`
`
`
`\
`
`afilv
`
`NE||rw<.3:m
`
`n.GE
`
`
`
`
`
`8NEmflw_ooH3333.2:
`
`
`
`ogsofi
`
`
`
`
`
`meni:3:
`
`Sui«hum:
`ENL«.5»:
`2NIiv3m:
`omnirm_u_:w>
`2“in22;;
`amid,v22;;
`32H:UM2a
`EuIll.1320
`
`:NiN22:;
`3NlN33.0
`«E|IIIV<.3:
`2625)595322\
`
`
`
`
`NFN
`
`
`
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 14 of 61 PageID #: 706
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 14 of 61 PageID #: 706
`
`Patent Application Publication Mar. 18, 2004 Sheet 12 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`
`
`Emfl
`
`,
`
`2:lwa»:
`
`m2|¢F5350
`
`n2|nw<.3:
`
`E:luhem...
`
`2:|Ivm:5:
`
`SN|¢v5w:
`
`871.202;
`
`Eliq22;;Ein2qu>ElN22.25
`
`2:iN9810
`
`
`
`
`
`
`/,jmwnllllv45mm:
`
`
`
` m_ooH3956me330M32H:uM2m
`
`
`
`sou
`
`ENIiNhow...
`
`EN[im3m:
`
`2mlv3m:
`
`InIll_.3.20
`
`2m|Vtom:3
`
`
`
`1galF2255
`
`.mwiNw_u_:w>
`
`«aim22%;
`
`31,e22;;
`
`2wlN1320
`
`
`
`EonE>>533meK
`
`
`
`
`
`NFN
`
`m
`
`GE
`
`_||Illlcillnll|
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 15 of 61 PageID #: 707
`
`Patent Application Publication Mar. 18, 2004 Sheet 13 0f 36 US 2004/0054428 A1
`
`2N /
`
`SN/
`
`fo> 262
`
`/
`
`/ Q9820 / /
`
`/ /
`
`IQ
`
`0 .UI
`
`
`
`8.20:8". cum
`
`mom
`
`2N I e an: I
`EN I m 3m: I
`2.“ I u 3m: I
`ma I F 3m: I
`
`‘R I _. 2.0.5..
`
`2N I N 2:90
`
`$50M
`>59. Ev
`
`7
`
`Na
`
`\_ El N 22%; I L QNNI F2£€>F
`
`a" I n 22;? l
`"ml Q 22%; I
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 16 of 61 PageID #: 708
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 16 of 61 PageID #: 708
`
`Patent Application Publication Mar. 18, 2004 Sheet 14 0f 36
`
`US 2004/0054428 A1
`
`
`
`fo>262
`
`/
`
`
`
`8~/.Ewflm_ooH59.335.£30m32»“EM2m
`
`
`
`
`
`
`
`mgInhum...—I
`“EIv3m:I\82.
`
`
`x:I:8:Iizmn<w<m
`2:IN.52...II///
`
`
`
`Nam
`
`ooflocflu.552wIv5m:I
`EmIr0.320
`38E;59.382x
`
`
`
`
`
`
`
`
`4/«RIw2qu>I
`fixImw_u_:u>I//EIN2%.;,I
`
`
`//aEI_.20Eu>I//\_m3IN95.5:\
`
`
`
`m?I—.239.0:
`
`
`
`VNN///O“OF—UthIrhowDI
`
`
`E~/_\j/«EI<.3:Ima
`
`EmgIc22;;I
`twIm3m:I/m2wIN3m:I
`«aIn22;;I/ENIN20Ew>I
`32RNI3.2;;I/2wIN3.20
`
`Oh.OF.‘
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 17 of 61 PageID #: 709
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 17 of 61 PageID #: 709
`
`Patent Application Publication Mar. 18, 2004 Sheet 15 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`«8 .omcommofi
`
`3...l:3:
`
`8...lN3m...
`
`8...im5m:
`
`3.”lv.3:
`
`8mIii,_‘9.9.0
`
`«on'11<.3:
`
`/
`
`/No:
`
`
`
`«81¢.uu_o_,._..u>
`
`SuiNm_oEm>
`
`eelv22;;Elnu_u_..o>
`
`
`momiN3.0.0
`
`
`
`
`
`
`
`
`
`<zmo<w<m
`
`/
`
`
`
`2.GI
`
`
`
`Eoflw_ooH
`
`33m32».EM2m
`
`
`
`2..|¢_..3:
`
`EN.IllvNnew:
`
`inii.nhow:
`
`En|.eEm:
`
`V;lv9.9.0
`
`
`
`Sui922.35
`
`.2iNm._u_§>
`
`anin2%.;
`
`«Niv20E0>
`
`Eu1N9.0.6
`
`2wl<.3:
`
`
`
`32.5359.3mem
`
`mum
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 18 of 61 PageID #: 710
`
`Patent Application Publication Mar. 18, 2004 Sheet 16 0f 36 US 2004/0054428 Al
`
`N“ .OE
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 19 of 61 PageID #: 711
`
`Patent Application Publication Mar. 18, 2004 Sheet 17 0f 36 US 2004/0054428 A1
`
`8~ /
`
`EN/ .
`
`050M.
`
`32H tum 2m
`
`
`
`>>oucS> 696322 R
`
`Ye:
`
`
`
`an i NhmmD I
`
`2w {it :3: 1L
`
`2“ i F “320T
`
`EN i n how: .|
`m? ..
`
`“K ‘V N 9.20
`
`Emit u 2oEo>
`5i F 22%;
`El Ema? J 2" i m 22%;
`
`NR
`
`2.:
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 20 of 61 PageID #: 712
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 20 of 61 PageID #: 712
`
`Patent Application Publication Mar. 18, 2004 Sheet 18 0f 36
`
`US 2004/0054428 A1
`
`
`
`3.0E
`
`
`
`
`
`calF22;;\ElNnacho
`
`
`
`
`
`
`
`Ewflm_ooHEmcwmmwfi
`
`
`23m2,2»EM2m
`
`
`
`
`
`Bou:_>>.mmcmwmmE
`
` ,WE||w<.3:
`
`male.1mel1
`INluv_,239.0
`2wll_‘Law:I
`haIlln3m:.|
`«ElN.3:I
`«aln202%L
`
`ENIVNo_uEm>l
`
`Maivm_u_._o>l
`
`NFN
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 21 of 61 PageID #: 713
`
`Patent Application Publication Mar. 18, 2004 Sheet 19 0f 36 US 2004/0054428 A1
`
`new
`
`3
`
`Mom“
`
`
`
`20$ 28H
`
`2.5M
`
`
`
`32» “EM 2E
`
`
`
`2N ‘It <._mm3
`
`
`
`7 ohm Iv who»:
`
`in l 0 Hum:
`EN ‘lit w 3m:
`
`3w til F 9.05
`
`
`
`m3 ‘it tow:
`
`2. GE
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 22 of 61 PageID #: 714
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 22 of 61 PageID #: 714
`
`Patent Application Publication Mar. 18, 2004 Sheet 20 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`
`
`
`
`8~/Eoflw_ooH.wmcowmwls.330m32»EM.2m
`
`
`
`
`
`
`
`
`2..OE
`
`
`
`
`
`
`26c£>>Lomcmwwos.N
`
`haIm.5»:I
`SNIFo_oEm>I
`«aIm202.;I
`E...ItownA
`3.“II.22.1%I
`2NI<33I, 3.4.IFn30.0L
`RNIN22;;I
`
`newIN.am:I
`,
`2NIv3m:I1
`2mIN93.54
`
`NE
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 23 of 61 PageID #: 715
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 23 of 61 PageID #: 715
`
`Patent Application Publication Mar. 18, 2004 Sheet 21 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`
`
`.2.GE
`
`«E“/f
`
`/
`
`Eafl£00...”59333
`
`
`930M32H:UM2m
`_l./2N
`8:/
`
`EN|v«hum:
`ohm|vN50»:
`2wlv3m:
`Sui259.0
`
`2m'1N9.9.01
`cal2.02:5|,
`
`«alm22;;|
`2695559885.N
`
`.ala22;;l
`3iw23;;|
` <0.mm..m02<mo.—E5591
`
`
`
`ms..>._._0w<mz<¥
`
`930m
`>2.xmo>>>mzE
`
`,«:33H
`
`
`
`
`
`
`
`
`
`
`
`NFN
`
`0:“
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 24 of 61 PageID #: 716
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 24 of 61 PageID #: 716
`
`Patent Application Publication Mar. 18, 2004 Sheet 22 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`
`
`
`3..0E .
`
`
`
`
`
`Emflm_ooHhomcawmmfi£30m303gumo=m
`
`
`
`
`
` \
`
`
`
`2635;EmcmmmmfiN
`
`2”||¢nhow:I
`
`2Nl.vLow:1
`
`newIqi.N9.9.0
`
`
`
`cal'23:;I
`
`
`
` awina_o=_o>fElNo_o_._¢>I
`
`amie22%;J
`
`230m/rllll/
`
`.mccmi
`
`0—:
`
`m<wz<¥W—E:/<0.mm._m02<8..E1|I|||J
`
`ms..>._._U
`
`>z£mo>§mzESt
`
`:\
`
`ma.ll;«3m:|.
`
`
`
`
`
`ENIiNnumbI
`
`2Nll_.96.5
`
`
`
`
`
`2m11¢«Co»:«E
`
`
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 25 of 61 PageID #: 717
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 25 of 61 PageID #: 717
`
`PAtnetaP
`
`1
`
`S
`
`3
`
`0
`
`1An
`
`ma
`
`..Eow<mz<zE.82/6,/.xk.m<._._<oE31pm/,St/¢<0.34358..E
`4ImmFGE
`Lu/
`.mZBouE>>mecmmmmERaKCIIIIl'-
`r..m3l|iN95.0a/M,2mlv:5:JnEIVa5m:
`pm\4/ma‘l<5m:na[fl
`4«Ele2255W«ain22;;2HuiNu_o__._m>
`89RNIIVFu_uEu>
`.m/,2NxtlN3m:imStm:ll.15»:WSun/J,inlF3.20
`
`>256>252E2,//,W/gt1.ms.
`
`
`
`fl23m...252.5,,LOCCNE930””—e€2.50:253m“3:./r||||i
`
`
`.“lw8N/..Emflm_ooH.5933:320M32>gum2E
`
`2couscou—\I‘ll-I.—
`WflJ/82/7
`
`
`
`
`
`
`
`
`
`
`NFN
`
`//
`
`awn“
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 26 of 61 PageID #: 718
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 26 of 61 PageID #: 718
`
`Patent Application Publication Mar. 18, 2004 Sheet 24 0f 36
`
`US 2004/0054428 Al
`
`
`
`flooH .mmcmmmmfi
`330M32Htum2m
`
`
`mE.C_om<mz<xE
`
`
`
`NFN
`
`/mam
`
`on.OE
`
`
`
`Ewfl
`
`BouE>>1mmcmwmmEw
`
`050m/ilIJ
`
` wall:520H_J/d«Elllv‘.<5m:
`
`EmIiN2:20
`<o.wm..mwz<mo;E.6585
`
`maulVm_u_.,_m>ll
`Na1ngains5
`cal_.2qu>II
`ENIllm3m:L
`n3ll:3:|
`2wlclvN.3:II
`EmilN20Eu>II
`amlv.3:1
`
`
`
`82/
`
`82/
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 27 of 61 PageID #: 719
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 27 of 61 PageID #: 719
`
`Patent Application Publication Mar. 18, 2004 Sheet 25 0f 36
`
`5
`
`1A
`
`
`
`
`
`
`
` 53382NJ22..202.39.03223:0132.»EH.SE,
`
`26955
`
`
`
`//w
`
`
` 088W//>z{10>252E0/0«an/2<>.ozammxofimS«SN//UI
`
`
`,awIN202$I
`maIto»:I EIv:90
`tnIInLaw:I
`SNIN5m:I
`2NIw.3:I
`SNI_‘m_u_cw>I
`«aIn22;;I
`2wIINa:20-I
`3iv22;;l
`
`930m/II/
`
`BEBE
`
`«Few
`
`R.MNat
`
`
`2<mE,/
` BE/7ms_t6w<wz§E36w<0.mmO...
`
`
`<o.mm_._moz<mo;ETIIIIIIIIIIIIIIIIIIIIIII4
`
`
`
`emI3m32fl/<3\«S
`
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 28 of 61 PageID #: 720
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 28 of 61 PageID #: 720
`
`Patent Application Publication Mar. 18, 2004 Sheet 26 0f 36
`
`US 2004/0054428 A1
`
`
`
` Ewfl
`Lomcmmmofi
`
`
`NNGE
`
`w_ooH
`8-\1\L 7
` 330m32»EH2m
`
`,AWN!»+2053,.
`m31whum:
`“.3llNLow:
`CN.|vM.”Law:
`2wI|Vv.62..
`Emlv9.9.0
`«.3Ill4.3m:
`animo_o_fi>
`
`26vE>>59385.J
`mewIiN“.320
`Elu22;;
`Elvc_u_fi>I,
`
`L
`
`
`Nun
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 29 of 61 PageID #: 721
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 29 of 61 PageID #: 721
`
`Patent Application Publication Mar. 18, 2004 Sheet 27 0f 36
`
`US 2004/0054428 A1
`
`
`
`330M263tumozm Eoflm_ooH.wmcommmfl.
`
`
`
`
`
`
`
`
`
`
`///\,EdemtEE,//32“@92596
`/NQMN
`ohmiNhum://hai_,5m:I,
`/EN'i_.2390'
`
`
`.i,Fm._o_:m_>E
`82z///556:;
`mtnIlln.meA
`3:3”./1|..II,
`2N11|w.5m7/r<3«E
`
`2NIllv.3:
`onwlv_.o_o__._o>42m1N0—320
`269:;Emcwmwms.R
`
`
`
`MNGE
`
`ENIVNm_oEo>
`
`«ain22;;
`
`Elvo_oEo>
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 30 of 61 PageID #: 722
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 30 of 61 PageID #: 722
`
`Patent Application Publication Mar. 18, 2004 Sheet 28 0f 36
`
`US 2004/0054428 A1
`
`
`
`3%
`
`
`
`VN.GE
`
`m_ooH 39.3me
`
`
`
`
`
`
`
`
`3wIllNumeIn,Ewlti7.33.!,
`
`
`
`ewwlv“0.3:;
`Em|w_.3.20.
`ENim3m:|
`«3ll<3.0.:
`Male22%;1Naln22;;I
`
`Boo£>>59335.N
`ElN22:?I
`2wllvN33th
`
`
`
` 330M...32H36%EM
`
`
`
`
`
`
`
`NFN
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 31 of 61 PageID #: 723
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 31 of 61 PageID #: 723
`
`Patent Application Publication Mar. 18, 2004 Sheet 29 0f 36
`
`US 2004/0054428 A1
`
`8N/
`
`
`
`8/[
`
`~NN/\\K///
`
`\/\,
`
`QOVN
`
`Venn
`
`/
`
`moVN
`
`\
`
`Toi
`(kl;
`
`mm.OE
`
` fil/
`
`///
`
`Fl
`
`
`
`
`
`
`
`
`
`Ewflm_ooH.mmcommmfi330M253
`
`:UM2m
`
`
`4//noun,//>33/F
`
`
`2N
`
`
`
`m..0=..m>
`
`EEi <.3:
`EnllF.520
`an“IV:3:1:
`tn44¢.nLow:11
`2w[:1Nhow:I
`
`2n|VN9.95
`EuIi78m:I
`
`Sui_.o_o__._w>J
`”#1w202%IEin202;|
`KatiNu_oEu>i
`
`.mccmi230m
`
`
`
`26v£>>.6913meA_\
`
`
`
`
`NFN
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 32 of 61 PageID #: 724
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 32 of 61 PageID #: 724
`
`Patent Application Publication Mar. 18, 2004 Sheet 30 0f 36
`
`US 2004/0054428 A1
`
`VNN
`
`ecu
`
`uzw/
`
`
`
`.mmcmwwwz 32.»to“2a
`
`
`26n£>>
`
`
`
`2~|v.152...
`
`3miN5m:
`
`ininhum:
`
`2m|vv.33
`
`Emil2390
`
`Sui_.o_oEm>
`
`“uniN20Ew>
`
`2“ln22:;
`
`9311v20Em>
`
`2”iN3.9.0
`
`2ai<Em:
`
`
`
`.mccmE230m
`
`NFN
`
`/Goon
`
`
`
`Emflm_ooH
`
`.mmcommofi
` 830M
`
`0N.OE
`
`SowIv<Zmn<w<n_
`
`
`
`
`\
`
`\/\\magi>2.on>Bmz
`
`83/
`
`Sew//
`
`<0.mm..m02<moi.E1
`
`ms..>:om<mz<xE
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 33 of 61 PageID #: 725
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 33 of 61 PageID #: 725
`
`Patent Application Publication Mar. 18, 2004 Sheet 31 0f 36
`
`US 2004/0054428 A1
`
`he
`
`VNN
`
`
`
`
`
`
`
`Eaflw_ooHBmcmmwmfi
`
`
`330M32H«EM2E.
`
`:nw
`
`
`
`\ISal«anal
`/2mlvEm:l
`Law:II
`Em|vF9.20
`wfim|v_..
`AFN'1nhow:I
`cal3.2;;t
`
`
`RwlvKama?Jin23%;|
`EmlcvN22;;I
`2....|VN«.5201!
`
` mE|v<.3:1‘
`
`Boucgeomcmmmmi\
`
`NFN
`
`RN.OE
`
`
`
`
`
`ER/_/:9:
`
`20.5.596
`
`
`
`«EN/_/,
`
`Emma/Fm._o_:m>
`
`
`
`EccmE230m
`
`
`
`
`EEE
`
`
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 34 of 61 PageID #: 726
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 34 of 61 PageID #: 726
`
`Patent Application Publication Mar. 18, 2004 Sheet 32 0f 36
`
`US 2004/0054428 A1
`
`
`
`mN.GE
`
`/W/,
`
`//
`
`20:5»30
`
`
`
`
`3mm,:0“:
`
`3mm2mlvLow:|\/NR'1mEm:I:
`
`//x82/_/
`
`20.5.5962~.w:3:I
`
`
`
`
` Emflm_ooH.mmcwmwofi230M32H“EM2m
`,Naim22;;l\Eluu_u_;o>|
`($11v22;;l
`E._o__._m>calF22;;I
`
`
`j/«EIll«Cum:
`/_II|III|1
`maiaLaw:l
`
`Bouc_>>.wmcmmmm_>_r
`2NIiiN9.0.5
`
`in|vF0.39.0i_|
`
`/
`
`Foam,
`/
`
`
`
`553m230m
`
`3%/_/
`,
`vm40_:m>
`
`FEu:
`
`
`
`
`BEE
`
`
`NFN
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 35 of 61 PageID #: 727
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 35 of 61 PageID #: 727
`
`Patent Application Publication Mar. 18, 2004 Sheet 33 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`
`
`
`
`Emflm_ooH39.33%330”.SEN«Elm.«EM
`
`
`
`
`
`
`
`J-eE'1new»:I
`
`2mIiN9.9.0
`calF22;;|
`«ain22%;I
`
`“uniNm_uEw>l
`«21e22:;l
`
` Sun262:;50:3meN
`
`mm.GE 1‘8380a63.80H.
`«63:00IF83.80
`awlN.8:I3ll23.:l EwIlvv9.9.0
` New1,11%:
`
`floficoo
`
`
`
`
`
`
`
`
`NFN
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 36 of 61 PageID #: 728
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 36 of 61 PageID #: 728
`
`Patent Application Publication Mar. 18, 2004 Sheet 34 0f 36
`
`US 2004/0054428 A1
`
`3/
`
`_|
`
`j
`
`
`
`on.OE
`
`
`
`
`
` Eml1_.«59.0
`2NIll_.new:II
`
`2N|$Nhow:I.
`«Einhow:I
`2wIvN3.9.0
`2NlvEn:l
`87¢3.2;;l
`«alm23;;IElN202%L
`Male22;;l
`
`
`
`EHm_ooH.mmcwmwwfi
`
`2.6m32»EM2m
`
`
`
`>>H0255590322F
`
`_<.ow1pr
`«.ij
`
`NFN
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 37 of 61 PageID #: 729
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 37 of 61 PageID #: 729
`
`Patent Application Publication Mar. 18, 2004 Sheet 35 0f 36
`
`US 2004/0054428 A1
`
`
`
`VNN/T\\E950
`\common...
`
`w.<mum:
`
`
`
`pm.GE
`
`
`
`
`
`
`/\no;/
`
`20.5593a//3:...fN,
`/2leF1390
`
`/_J/«E[i<Em:2n
`
`
`
`
`
`BouE>>596822N
`
` 98/,Eoflw_ooH.mmcmmmmfi930M32H26Mmzm
`
`
`
`/
`
`1kmlm3m:1|
`
`2NIieEm:II
`
`new|v_‘5m:I!
`
`
`
`2NIl|vNhum:|
`
`2NIIVN365
`
`SuiF22.35J,
`
`EmilN20Em>[J
`
`«aia22;;|
`
`aniv22;;l
`
`
`
`/83//5:52;9:3.
`
`/<mum:m
`
`son:
`
`
`
`
`
`
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 38 of 61 PageID #: 730
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 38 of 61 PageID #: 730
`
`Patent Application Publication Mar. 18, 2004 Sheet 36 0f 36
`
`US 2004/0054428 A1
`
`
`
`
`
`eon
`
`w_ooH 59332
`
`
`
`
`
`
`
`
`NFN
`
`NM.GE
`
`Noun
`/
`
`
`330mBEN:UM0:“
` 8%IfFm§zm>F
`fi/2wlli,4.3m:
`920m
`m3ii_.3w:in Ewl¢_.9.9.0
`
`Cwi|vnhow:I
`m31ivam:i
`
`2NIIIVNhow:I
`or“IiN9.20
`an},3.2;;l
`«aim22%5|
`xnlu22;;i
`Malv22;;I
`2635;$9385.1\
`
`.mccmE
`>256>5.2E
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 39 of 61 PageID #: 731
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 39 of 61 PageID #: 731
`
`US 2004/0054428 A1
`
`Mar. 18, 2004
`
`METHOD AND APPARATUS FOR SENDING,
`RETRIEVING AND PLANNING LOCATION
`RELEVANT INFORMATION
`
`BACKGROUND
`
`[0001]
`
`1. Field of Invention
`
`[0002] This present invention relates to a computerized
`mapping and real-time communication software program,
`and more specifically, to integrating or coupling computer-
`ized mapping and real-time communication software for the
`purpose of transferring location-related information using a
`real-time communication system.
`
`[0003]
`
`2. Description of the Related Art
`
`[0004] Computerized mapping and real-time communica-
`tion software are independently achieving widespread use
`today. Such mapping programs are commonly used to auto-
`mate tasks of calculating routes, viewing location-specific
`geographical areas
`for
`their
`spatial content,
`such as
`addresses, roadways, rivers, etc., and for the purpose of
`being used with Global Positioning System (GPS) devices
`for various applications, such as a personal navigation
`application. Mapping software programs apply to a wide
`variety of uses, such as personal navigation,
`telematics,
`thematic mapping, resource planning, routing, fleet tracking,
`safety dispatching (i.e., Police, Fire, and Rescue organiza-
`tions), and a wide variety of specialized Geographic Infor-
`mation System (GIS) applications, all of which are well
`known to people skilled in the art.
`
`[0005] Real-time communication software applications
`are also being used today in various applications, like Instant
`Messaging (IM) applications such as American Online’s
`(AOL) IM (AIM), Yahoo’s IM, and Microsoft’s IM, all of
`which are well known to people skilled in the art. None of
`these prior art IM software applications contain mapping
`capabilities. These applications provide presence informa-
`tion about other users on a user’s roster or buddy list, such
`as online, busy, away, on the phone, offline, etc., and are
`primarily used for noncommercial applications, such as for
`conversing with friends or buddies that are online.
`
`[0006] Prior art applications provide various features, such
`as displaying driving directions (i.e., routes), Points Of
`Interest (POI), waypoints (such as personalized, user-spe-
`cific, points on a route or along a track), etc., but do not
`enable the transfer of such information to other users in
`
`real-time. Auser will typically copy an image of a map from
`a standard mapping program, usually with a highlighted
`route, and e-mail the bitmap image and/or directions to
`another user or group of users for the purpose of meeting at
`a specific location or POI, such as a restaurant. Alternatively,
`with the adoption of IM programs, users can transfer these
`images and directions, typically by using an integrated file
`transfer program (FTP) connection,
`in real-time to other
`users based on their presence, and obtain real-time feedback
`from their buddies about the destination POI or location and
`
`specific route used to get to the destination.
`
`[0007] Current applications that integrate both mapping
`and real-time messaging are well known in the art, such as
`the Automatic Vehicle Location (AVL) or Fleet Tracking
`industry, where vehicles that have position devices, such as
`GPS, report their position to a centralized computer for the
`mapping and display of the vehicles’ locations. Some of
`
`these prior art systems may incorporate real-time messaging
`for the transfer of logistical information, such as pickup and
`drop-off status messages. However, these eXisting applica-
`tions do not provide a method for dynamically and graphi-
`cally transferring location-relevant
`information coupled
`with a spatial map. Additionally, these applications typically
`provide only one-way transfer of position information, from
`the mobile vehicle to the dispatcher application, either on a
`web-based or desktop-based program. Usually, there is no
`need to transfer the dispatcher’s location to the mobile
`vehicle since the dispatcher’s location is always stationary.
`Mobile devices typically use location telemetry devices to
`transmit their location in a pre-defined manner or by request,
`where the dispatcher’s location request is usually initiated
`by clicking on a Graphical User Interface (GUI) or by using
`a set of preferences to automatically request position
`updates. These preferences are based on various parameters,
`such as reporting location updates based on the distance
`traveled by the vehicle or by using various time intervals to
`trigger position updates either by a push or pull method
`relative to the telemetry device.
`[0008] Another problem with eXisting AVL software solu-
`tions are that most applications are web-based applications
`that only allow for static image-based mapping, such as
`those provided by various online mapping companies like
`MapQuest. Also, the mapping and communication systems
`are disjointed from each other, as is the case with various
`companies, such as Televoke, Inc. These static image-based
`mapping applications do not enable real-time graphical
`manipulation of POIs on the map, nor do they provide a
`graphical connection between the map and vehicle roster
`listing. Some AVL software solutions provide the ability to
`display moving vehicles on dynamically viewable maps.
`However, these solutions do not enable the user to select a
`vehicle on the map, nor a stationary representation of a
`vehicle in a roster list, in real-time for the purpose of sending
`the vehicle’s location to other users, and thus do not allow
`the creation of ad-hoc position transfers between various
`parties. Some dynamic mapping applications, such as
`Microsoft’s MapPoint application, allow users to select
`Points Of Interest (POI) generally for the purpose of pro-
`viding additional information about the POI or enabling the
`user to add the POI to a route planner as a route start, end,
`or stop point. This POI is selected by ‘right-clicking’ on the
`object after it has been selected and then choosing the
`specific route option. However, prior art fails to provide
`real-time communication capability with location-relevant
`information (i.e., POIs) for the purpose of graphically send-
`ing location-relevant information in established or ad-hoc
`networks to other users or location-enabled devices.
`
`[0009] Another problem with prior art, such as in the case
`of AVL software solutions, is that vehicles or other mobile
`devices that a user wishes to map must first be selected from
`a list of available position-enabled vehicles. These vehicles,
`however, must already be configured for mapping on a
`dispatcher’s mapping application and do not enable position
`requests in an ad-hoc environment. Prior art AVL mapping
`and tracking systems, such as At Road Inc., only allow users
`to select from a list of pre-configured location-updating
`vehicles, and then require the user to press a button in order
`to map the location of the selected vehicle(s). A much better
`solution, as people skilled in the art will appreciate, is to
`select a user, device, or group of users and devices in a roster
`list and graphically drag-and-drop the selection onto an
`
`

`

`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 40 of 61 PageID #: 732
`Case 2:17-cv-00513-JRG Document 37-3 Filed 11/14/17 Page 40 of 61 PageID #: 732
`
`US 2004/0054428 A1
`
`Mar. 18, 2004
`
`active map. This method significantly simplifies the process
`of identifying a single or group of user(s)/device(s) and
`mapping their location appropriately. Additionally, prior art
`AVL systems do not allow for the case of users or devices
`to disallow their position from being mapped on the current
`mapping application.
`
`[0010] There also exists a need for the consideration of
`permissions in such a case of privacy concerns, where a
`real-time location request be sent across the real-time com-
`munication connection to the user, vehicle, or device, whose
`location information is being requested. The user, vehicle, or
`device can select the resolution of position information they
`want to communicate (i.e., latitude and longitude, or city, or
`state, or etc.) to control the level of accuracy to which they
`can be mapped. Once approved,
`this ad-hoc transfer of
`position information occurs and the graphical mapping of
`the received position information is completed on the
`requestor’s mapping application. Thus, allowing users to
`initiate position requests graphically and in real-time, and
`providing the capability of ad-hoc position requests to other
`users not pre-configured to allow their location information
`to be mapped, provides an extremely efficient method and
`system when compared to prior art systems.
`
`[0011] Another drawback of prior art is that integrated
`mapping and communication programs, such as AVL appli-
`cations, provide the ability for the receiving of position
`information for mapping purposes only. These prior art
`systems do not provide the capability of sending, or pushing,
`location-relevant information, such as POIs, to other map-
`ping programs or textual devices, such as Personal Digital
`Assistants (PDA), pagers, cell phones, etc. For instance,
`prior art mapping systems, such as Microsoft’s MapPoint,
`allow the user to select POIs, such as restaurants and gas
`stations, but does not allow the user to transfer these POIs
`to other users, and more specifically does not allow users to
`graphically drag-and-drop these selections (i.e., POIs) for
`various purposes, such as to dynamically add them to a route
`planner for inclusion in an undefined route or pre-calculated
`route.
`
`[0012] The integration of a highly dynamic mapping
`application and a real-time communication system enables
`users to select POIs, such as houses, theaters, city names,
`roads, etc., or icon representations of other users on a
`mapping program for the purpose of graphically sending
`location-relevant graphical information, such as the selected
`POIs, to a specific user on a roster listing of available online
`users in real-time. As people skilled in the art will appreci-
`ate, graphical location-relevant information is not limited to
`only POIs, but also includes mapped routes, waypoints,
`geo-fenced areas, planes, etc. A valuable feature that prior
`art fails to provide is the transfer mechanism that allows the
`ability to drag-and-drop this location-relevant map informa-
`tion (i.e., routes, geo-fenced areas, etc.)
`to the current
`application’s roster list for such transfers.
`
`of another vehicle in the user’s roster list. Before the
`
`completion of the transfer of one vehicle’s location infor-
`mation to another vehicle, where the user or dispatcher acts
`as the location-transfer hub, each user sets the appropriate
`permissions to allow the transfer. Thus, each of the vehicles’
`roster lists do not need to be included in the other vehicle’s
`
`roster list, since the user or dispatcher has both vehicles on
`its roster list and acts as the hub for the transfer of the
`
`position information. This creates a dynamic environment
`for ad-hoc position transfers that are not available in prior art
`systems.
`
`[0014] As an additional drawback of prior art systems,
`there is no way to provide real-time route planning of a
`system consisting of a real-time communication system
`integrated with a mapping and real-time communication
`program. In other words, it is not currently possible for a
`roster icon representation of a vehicle or user to be graphi-
`cally selected into, or dragged-and-dropped onto, a route
`planner for the purpose of setting a user’s current position as
`a route’s destination points, where the term ‘destination’
`refers to a point or location on a map that the user indicates
`as a start of a trip, end of a trip, or stop or waypoint along
`a trip. Origin also is used to refer to the start of a trip. This
`route planning operation also applies to POI locations. For
`instance, prior art systems, such as Microsoft’s MapPoint
`allows users to graphically alter a pre-calculated route, such
`as graphically indicating the portion of the route to alter.
`
`[0015] However, current art systems do not allow the
`capability of selecting real-time location-enabled or static
`POIs (such as vehicles, restaurants, people, gas stations,
`houses, etc.) for the purpose of graphically adding to, or
`updating, a route’s destination points in an undefined or
`pre-calculated route. Additionally,
`this prior art system
`application only allows the alteration of a route to a new
`destination by dragging the selected portion of the route to
`that new location. A more useful method, which can incor-
`porate the integrated real-time communication system, is by
`allowing the user to drag a graphical represent