[MIWWWMWWWWMMWMMWMWMWMH
`
`USiH16724403131
`
`(12) United States Patent
`(1(1) Patent 1%.:
`US 6,724,403 B1
`
`Santoro et al.
`(45) Date of Patent:
`Apr. 20, 2004
`
`(54) SYSTEM AND METHOD FOR
`SIMULTANEOUS DISPLAY OF MULTIPLE
`INFORMATION SOURCES
`..
`.
`fl
`,
`,
`Inventors. End Santomél.ondhon ((rlnglaus
`“gemann’ ”1"“ 33°“
`1
`.
`,
`(73) Assrgnee: burfeasl, Inc., Palo Alto, CA (US)
`
`-
`(73)
`
`( * ) Notice:
`
`Subject to an)r disclaimer, the term ofthis
`patent 15 extended or adjusted under 35
`U.S.C. 154(1)) by 259E13y5.
`
`_
`.
`..
`,
`(71) Appl No - 091702 325
`9’}
`..
`.
`(__)
`liled.
`
`Oct. 30, 2000
`
`(6(1)
`
`Related U.S.Applieation Data
`Provisional application No. 111.111.52.522. tiled on Oct, 29.
`199:1
`'
`'
`'
`1. 7
`31;; (£311 .347ng :51?!)
`(2:)
`1:
`'
`'
`5";
`5’ 34‘ I?”
`(1‘)
`(38) he“ or8831;121:1114"En'g'migiiM-iéé"6:15;??? :32
`)5
`5
`“"3
`‘
`‘
`j
`‘J
`H 3
`9
`9
`799
`
`(56)
`
`References Cited
`U S 1’AI'EN’I‘ DOCUMENTS
`4 ass “1'13 A
`111198; Pike
`iggfifil :
`1:13;; She-“Elk“ 3‘ “L
`.‘
`.‘
`If
`'
`enna
`4.831556 A 3
`511989 00m
`€187,384 A
`1”“qu (”mum cl :1]
`335134.321 A
`3,1915 Henderson, Jr.- at 211.
`5,479“): A
`12,1995 Baccker cl at.
`5,550,958 A *
`811996 Miller et al.
`5,140,430 A
`411098 Rosenberg el al.
`3,140,549 A
`411998 Reilly et al.
`{57511371 A *
`311993 OFI" Cl 31-
`zit-11:21:51:
`$33: 31d”? 31 ‘11-
`3:
`3-
`-
`Eff
`3,196,383 A
`811998 Ilcnshaw et at.
`5,190,41'11 A
`811998 Wine:
`5.312.123 A "‘
`911998 Rowe Cl 511.
`5,813,001 A
`011998 Nielsen
`
`3451786
`
`3451741
`
`_
`3451779
`
`3451433
`725143
`
`725181
`5,831,664 A "‘ 1111998 Wharton et a1.
`.. 3451115
`53,838,326 A * 1111998 (Ta-rd cl at.
`
`3,841,418 A
`1111998 Brtl el al.
`34513
`011098 Doughcrly el al.
`3,843,352 A
`5,905,492 A
`51191.19 Strauh et at.
`5,918,231 A
`{”1999 analbam
`5,929,854 A *
`711999 Ross
`6,003,041 A
`1211999 “111 ofski
`6,011,537 A
`112000 Sloaniek
`(1,025.83? A
`212000 Matthews. 111 el al.
`
`3451783
`
`2,200“ Weidcnt‘cller el al.
`6,028,602 A
`1212111111 RoherLson et al.
`6,1611553 A *
`6,106,138 A " 12121“! Roberlson et al.
`fi,|88,4t]5 Ht
`1‘
`212110]
`(Tzcrwinski et al.
`
`345115?
`3451839
`34517154
`
`3451156
`
`Iledherg
`151211112
`5,411,275 81 *
`OTHER PUBi.lC/U'IONS
`.
`, a
`)
`'
`J
`1
`~
`'
`Martin 5 Matthews and Erik B. loulsen, Frontl age -001).
`TI“: “'1'me Refmncc’ May " [999’ MCGraw'H']
`Osborne Media, Chpater 1‘), pp. 1—12.*
`a
`s
`u
`John Ross ABCS of Internet Explore 4 Copyright 1997
`Sybex, Chapter 13, pp.1—3.*
`Paul Mcl-‘edries, The Complete ldiol's Guide to Window 95,
`Mar. 1997, 2nd Edition, pp. 3:1, 97. 101, 1(15-107,379.*
`(List continued on next page.)
`
`Primary Exmnr'ner—Cao (Kevin) Nguyen
`(74) Attorney, Agent, or It'irrn—I’ennie & Edmond-s. LLP
`(57)
`ABSTRACT
`A computerized method of presenting information from a
`in
`.
`il-
`-
`I
`.
`)
`i.)
`‘
`.u‘
`1
`variety of sources on a dtsplay demo... Speetfically the
`present
`inventlon descrtbes a graphical user interface for
`organizing the simultaneous display of information from a
`multitude of intorrnatton sources. In particular, the present
`invention comprises a graphical user interface which orga-
`nizes content from a variety of information sources into :1
`grid of tiles, each of which can refresh its content indepen-
`denlly of the others. The grid functionality manages the
`refresh rates of the multiple information sources. The
`..t
`‘
`u
`‘
`n
`t
`-
`.r.‘ -
`-
`'
`."
`presgntltnventton 1s intended to operate in a platform lnilL
`13"“ ”'1 mam“
`
`52 Claims, 27 Drawing Sheets
`
`'
`.
`‘uH»- : ""31 r r
`
`"“"
`>3-
`
`r
`.1,»
`
`Cine Wuafd‘
`
`-9'- cube-- you find "
`-j_.ut..m.m rm
`' 31.an
`
`at -., as
`
`.
`_.
`no
`
`am we..-
`I
`4-: I”
`-—.—1-
`'arv.
`
`
`
`J'erw.
`
`1.. n
`.n».
`..l
`.-
`i“ a»
`
`
`
`Mam
`-
`WWW-m a.
`' u);
`
`EXHIBI I
`Pelitioner - Kyocera
`PX 1040
`
`PX 1040
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)
`Kyocera PX 1040_1
`
`

`

`US 6,724,403 B1
`
`Page 2
`
`OTHER PUBLICATIONS
`
`l’Cl‘ International Search Report, Application No. PCT;J
`0800:29850, dated Jun. 25, 2001. 3 sheets.
`Available Web Site: Wdodotscom Accessed on: May 9.
`2001.
`Available Web Site: wvtwvsnippelscom Accessed on: May
`9. 2001.
`Available Web Site: wwwububucom Accessed on: May 9,
`2001.
`Available Web Site: www.chatbcom Accessed on: Nov T,
`2000.
`ireless
`.
`.
`.
`Duplex Multiplexer ,Scnscrmatic, Samsung,
`communications,hand heldsmaxon Available Web Site:
`www.mindspringcoml—attancomfmulti.html Accessed on:
`Nov. 7, 2000.
`
`push tech nology. Available Web Site: wwwwhaliscoml
`Whatls_Definition_Pagef0,4152,2l3345,00.html
`Last
`Update: Jul. 7, 2000 Accessed on Nov. 7, 2000.
`
`Clyman, John. Web Integration/Internet Explorer 4.0 Availw
`able Web Site: www.zdrtet.cnmfpcmagr’featuresitmemphisf
`memphislltlm Accessed on Nov. 7, 2000.
`
`Oct. 2000, Product Spotlight: Non~br0wser based portal
`solution from Snippets Software, Inc, Corporate Portals
`Letter [Online] 1(10), 1—3. Available Web Site: wwwsnip—
`petscomklownloadlCorporaze Portal Arliclepdf
`Accessed on May 9, 2001.
`
`‘ cited by examiner
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)
`Kyocera PX 1040_2
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 1 0f 27
`
`US 6,724,403 B1
`
`10
`
`20
`
`30
`
`§+. -£.j BUDkSIG: ]
`
`[BfldexfiSIngnwnnne- flgx
`_'|—
`5 1999
`
`Tred’mologg small; data.
`,..._..___.__....___... ,
`
`40
`
`50
`
`' .fltagu'figs
`.‘LJ'I‘I'IQM‘ :33!
`FilIII'Lx 'I'llll'villiq‘ hhlkh
`:23“:
`cm: ‘5; Em:m45
`(ImuflflITHl-JN‘L:
`(Jim-purer; 04;:
`TilIITlt. F;“lll m .'n \u
`
`HIE Sentltemx
`fife? Deleted Items [
`
`_ 'DHGEH I DHG
`DHNSKTUPPEN
`DET HGL. TEHTER
`DR-BUTIKHEH
`DR MULTIMEDIE
`DR2TELEFDHER
`DR 2: SHTELLIT
`DRIUNDERUISNIHG
`nHRTNn
`
`60
`
`'etiistncké?”
`..
`rtEfii‘I'Li'I'i-zlex
`;
`zedhenme. _'
`W011d: .busines:
`.Immfl ‘h: 14mm":
`
`g My Computer
`"'£33 31% Floppy Exit: ]
`
`3s .9 Bambarn[C:]
`[+3. Q33 Fatbelly [Di]
`ii"!- :_,3 Bigboss [E:]
`
`{£1 £3 Hulk [F:]
`
`70
`
`80
`
`Fig. 1
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:22)
`Kyocera PX 1040_3
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 2 0f 27
`
`US 6,724,403 B1
`
`100
`
`
`
`Operating System
`
`File System
`
`5
`
`122
`
`124
`
`-
`Application Program
`
`128-1
`
`Application Program
`
`128—N
`
`110
`
`i
`
`Display
`
`“5
`
`CPU
`
`104
`
`
`
`Network Interface
`
`134
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:23)
`Kyocera PX 1040_4
`
`Wireless Signal
`
`Cable Network
`
`130
`
`132
`
`Fig. 2
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 3 0f 27
`
`US 6,724,403 B1
`
`320
`
`”L“
`
`$2
`
`
`
`324
`
`340
`
`1‘
`
`. .
`
`348
`
`Ill
`
`Menu1 Menu2 Menu3 Menu4
`
`I??? lllir‘ééfiiéééliiilmlfllHIEIIIIIfiI_
`
`
`
`352
`
`346
`
`344
`
`342
`
`354
`
`Display area
`
`350
`
`356
`
`Fig.3
`(Prior Art)
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:24)
`Kyocera PX 1040_5
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 4 of 27
`
`US 6,724,403 B1
`
`402
`
`\
`
`
`
`404
`
`
`
`Portion of
`
`
`
`
`
`
`text docu
`
`
`
`on display
`within tile
`
`Fig. 4
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:25)
`Kyocera PX 1040_6
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 5 0f27
`
`US 6,724,403 B1
`
`500
`
`’3‘
`
`
`
`
`
`
`
`
`
`
`
`502
`
`504
`
`506
`
`508
`
`510
`
`512
`
`514
`
`516
`
`518
`
`520
`
`522
`
`
`
`
`
`
`
`
`
`
`Toolbar function
`
`Fig. 5
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:26)
`Kyocera PX 1040_7
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 6 0f 27
`
`US 6,724,403 B1
`
`FD><A HREF=“ti162_target.htm“
`TARG ET=“ncw”
`
`ON MOUS EOVER=actionl (arg3)
`ONMOUSEOUTr-‘action2(arg4)
`REFRESIFtimeouKSOO)
`SOIRCEWWWW
`C LICKMAP=
`{,0 0,50 50 cfickfunclbnl(clickargumenll),
`5 1, 0,5050, c lickfunction2( C lbkargumcnll),
`0,51,50,10, clickfunction3( clickargumentB) }
`TOOLBAR: {“bcal/too Ibars/radiojoo” PLACEMENT=‘T)0ttom" }
`ALA R M—' ‘ ‘alarmsfic 0nd itio n_rain=TRUE, c 0nd ition_weekend=F ALS E,
`alarmaction=b Iow_the_h0 m">
`
`/TD>
`
`Fig. 6
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:27)
`Kyocera PX 1040_8
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 7 0f 27
`
`US 6,724,403 B1
`
`rid 700
`
`N 0j
`
`704
`
`706
`
`708
`
`Web-page Viewer 712
`
`Streaming Video 714
`
`Audio Pia er 716
`
`File viewer 718
`
`710
`
`llllllll Grid 720
`
`Fig. 7
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:28)
`Kyocera PX 1040_9
`
`

`

`US. Patent
`
`Apr. 20,2004
`
`Sheet 8 of 27
`
`US 6,724,403 B1
`
`GUI124
`
`BUZ-1-N
`
`GridTDD
`
`8024-3
`
`8024-2
`
`SUE-14
`
`8024+!
`
`8132-2-23
`
` Ill
`
`BUE-M-‘l
`
`802-2-1
`
`802-2-3
`
`Fig. 8
`
`
`
`EUE-M-N
`
`80240-23240-3
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:19)
`Kyocera PX 1040_10
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 9 0f 27
`
`US 6,724,403 B1
`
`EDE-M-N
`
`802
`
`[01-3
`
`Fig. 9
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:20)
`Kyocera PX 1040_11
`
`

`

`US. Patent
`
`Apr. 20,2004
`
`Sheet 10 am
`
`US 6,724,403 B1
`
`
`
`Grid 700
`
`
`
`
`I-II-IIII1!IIIIII
`
` I‘ll-IIIIIJIIIII
`
`
`IIIIIII'IIIIIIHIIIIIII
`
`
`
`
`
`802-M-1-1-Y
`
`
`
`
`'02-M-1-X-Y
`
`
`Grid 1 000
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:21)
`Kyocera PX 1040_12
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 11 of 27
`
`US 6,724,403 B1
`
`
`
`mamapm?
`
`Do...“U:
`
`@555me
`
`:.E
`
`3E}
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:22)
`Kyocera PX 1040_13
`
`

`

`US. Patent
`
`Apr. 20,2004
`
`Sheet 12 01‘2"!
`
`US 6,724,403 B1
`
`1200
`
`1202
`
`1204
`
`1206
`
`1208
`
`1210
`
`1212
`
`1214
`
`1216
`
`Parent Grid (address)
`
`Address of self
`
`Configuration Wizard
`
`ile creation function
`
`ile annihilation function
`
`
`
`
`
`ile list Address Refresh rate Position Priority Password
`
`.
`
`_
`
`.
`
`Address Refresh rate
`
`Password
`
`.
`
`.
`
`.
`
`Fig. 12
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:23)
`Kyocera PX 1040_14
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 13 0f 27
`
`US 6,724,403 B1
`
`(HTML)
`(HEAD)
`
`<TITLE>Surfcast Grid ExampleC/TITLE>
`(META NAME=”Resource Manager” CONTENT=”RESMGR.EXE”>
`<META NAME=”Auth0r” CONTENT=”SurfcaSt,
`Inc.”>
`<fHEAD>
`
`<BODY BACKGROUND="SurfbaC k. gi f”)
`<TABLE>
`<TR>
`
`<TD><A HREF=”http://www.somewhere.com/sometarget.html”
`TARGET=”new”
`
`ONMOUSEOVER=acticn1(argl)
`ONMOUSEOUT=action2(arq2)
`RHFRESH=timeout£200J
`
`.OURCE=”http://www.surfcast.com/tilelibrary/tile2.til”>
`/TD>
`
`TD><A
`
`fl
`
`REF=”http://www.somewhereelse.com/someothertarget.html
`
`TARGET=”new”
`
`ONMOUSEOVER=acti0n3(arg3)
`ONMOUSEOUT=aCtion4(arg4}
`REFRESH=time0ut(SOO}
`
`
`
`uOURCE=”http://www.camelot.castle/swords/excalibur.til”
`
`/TD>
`
`TD><A HREF=”local/document.htm”
`TARGET="new”
`
`ONMOUSEOVERZactionl{arg3}
`ONMOUSEOUT=aCtion2£arg4}
`REFRESH=timeOUt(C)
`SOURCE=”local/documents/somedocument.til”>
`
`/TD>
`
`/TR>
`
`/TABLE>
`/BODY>
`/HTML>
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:24)
`Kyocera PX 1040_15
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 14 0f 27
`
`US 6,724,403 B1
`
`Customized Grid
`
`Grid Wizard
`
`
`0 Minimum Grid
`
`
`0 Standard Grid
`
`1408
`
`1402
`
`Grid Wizard
`
`
`
`Tile
`
`Settings
`
`1404 5
`
`1410
`
`
`
`
`
`“I420
`
`Store Grid
`
`Back
`
`Finish
`
`' 9 Y
`Launch Gnd.
`
`BS
`
`N
`
`0
`1422
`
`1412
`
`.
`Flg. 14
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:25)
`Kyocera PX 1040_16
`
`

`

`US. Patent
`
`Apr. 20,2004
`
`Sheet 15 0f 27
`
`Us 6,724,403 Bl
`
`1500
`
`1502
`
`
`
`Su rfcast
`User Tiles
`Application
`
`Program
`
`DEC]
`
`
`1504 Core
`
`Meta Base
`
`
`
`URL
`
`1510
`
`Loader
`
`Connection
`
`Manager/
`Bandwidth
`
`
`
`
`
`System Library
`
`Operating System Library
`
`
`
`1516
`
`
`
`
`
`Fig. 15
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:26)
`Kyocera PX 1040_17
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 16 0f 27
`
`US 6,724,403 B1
`
`1500
`
`SurfCast
`
`r1618
`
`Launcher
`
`r1620
`
`Framework
`
`1502
`
`User Tiles
`
`502
`
`504
`
`1606
`
`608
`
`(1610
`
`1612
`
`614
`
`1616
`
`WordTile
`
`ExcelTile
`
`RegTile
`
`LayoutTiie
`
`
`
`Events
`
`1504
`
`1622
`
`1624
`
`1626
`
`1628
`
`XP Core
`
`TileBase
`
`TileBaseView
`
`TileController
`
`Canvas
`
`1630
`
`Fig- 15
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:27)
`Kyocera PX 1040_18
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 17 of 27
`
`US 6,724,403 B1
`
`MetaBase
`
`{-1732
`
`1734
`
`(1736
`
`TileRegistry
`
`ContentStore
`
`
`
`
`1508
`
`Widget Set
`
`Button
`
`1738
`
`1740
`
`1742
`
`17 44
`
`1510
`
`URL Loader
`1746
`
`Cache
`
`1748
`
`FileManage
`
`1750
`
`Renderer
`
`[-1516
`08 Library
`
`1752
`
`1754
`
`1755
`
`1760
`
`1762
`
`Fig. 17
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:20)(cid:28)
`Kyocera PX 1040_19
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 18 0f 27
`
`US 6,724,403 B1
`
`1508
`
`1809
`
`
`Widget Set
`
`
`1810
`
`1802
`
`ButtonView
`
`
`
`
`
`
`ButtonEventConsumer
`
`ontroller
`
`1622
`
`1624
`
`Fig. 18
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:19)
`Kyocera PX 1040_20
`
`K-
`
`
`
`
`
`
`
`(/1804
`1806
`
`
`iButtonEvent l.......\ ButtonController
`
`
`
`
`
`
`1812
`
`1814
`
`
`
`
`1816
`
`User Tile
`
`1626
`
`r C
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 19 0f 27
`
`US 6,724,403 B1
`
`1506
`
`MetaBase
`
`
`
`
`
`
`
`
`
`
`TileTypeRegislry
`
`ile and Grid Store
`
`MetaBaseTile
`
`1906
`
`XML
`
`Library
`
`1622
`
`TileFactory
`
`Tile Creaior
`
`
`
`
`Content Store
`
`1918
`
`Link Database
`
`Content and Tile
`
`
`
`
`1912
`
`Inquiry
`Utility
`
`
`
`Fig. 19
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:20)
`Kyocera PX 1040_21
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 20 01'2"!
`
`US 6,724,403 B1
`
`XPCore
`
`1628
`
`2002
`
`08 Library
`
`2032
`
`GfxContextFactory
`
`EventSystem
`
`162
`
`2008
`
`-1Ti1eBaseVie-w
`
`HeBaseController
`
`Win32
`
`GfxContext
`
`201!
`
`2012
`
`162.
`
`GfxContext
`
`2015
`
`1812
`
`2020
`
`WidgetBase WidgetView
`
`WidgetController
`
`2022
`
`Fig. 20
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:21)
`Kyocera PX 1040_22
`
`

`

`US. Patent
`
`Apr. 20,2004
`
`Sheet 21 01'2”]
`
`US 6,724,403 B1
`
`2008 \
`
`TileBaseController
`
`Event System
`
`2104
`
`WW
`
`Fig. 21
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:22)
`Kyocera PX 1040_23
`
`

`

`US. Patent
`
`Apr. 20,2004
`
`Sheet 22 01'2"!
`
`US 6,724,403 Bl
`
`2200
`
`SurfWidget
`
`Controller
`
`Control Layer
`
`SurfWidget
`
`
`
`URL
`
`Pre~fetch
`Manager
`
`Manager
`
`
`
`URL
`
`2210
`
`2212
`
`2201
`
`2205
`
`2209
`
`221 5
`
`221g
`
`Connection Layer
`
`Connection
`Manager
`
`Connectlon
`Manager
`
`Cache
`Protocol
`
`
`
`FTP
`Protocol
`
`Surfcast
`
`Socket Layer
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:23)
`Kyocera PX 1040_24
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 23 0f 27
`
`US 6,724,403 B1
`
`Bandwidth Controller
`
`GetStatus J
`
`2302
`
`AddURL
`
`2304
`RemoveURL /
`
`GetURLStatus
`
`2306
`
`2308
`
`Fig. 23
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:24)
`Kyocera PX 1040_25
`
`

`

`US. Patent
`
`Apr. 20,2004
`
`Sheet 24 am
`
`US 6,724,403 B1
`
`Server 702
`
`2400
`
`2404
`
`Grid
`
`
`
`
`
`
`
`
`Generator
`
`
`
`2406
`
`Fig. 24
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:25)
`Kyocera PX 1040_26
`
`
`,4 2410-01
`
`lll‘lll
`
`User-Specific
`Content
`
`Tile Creator
`
`
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 25 of 27
`
`US 6,724,403 B1
`
`Fig. 25 2400 2402
`
`
`
`
`
`Client Device
`
`2500
`
`2512
`
`2518
`
`252!
`
`2522
`
`2526
`
`Verify User Identity
`
`2502
`
`Acknowledge Log-in
`
`2506
`
`Identify Device Type
`
`Retrieve Grid Settings
`
`2508
`
`2510
`
`Obtain
`
`User-Specific
`Advertising
`
`K
`2514
`
`Verify and Calculate
`Stream Parameters
`
`2516
`
`Request Update
`
`2524
`
`Render Grid
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:26)
`Kyocera PX 1040_27
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 26 of 27
`
`US 6,724,403 B1
`
`K 2602
`2600
`
`
` User/
`Client Device
`
`
`
`2606
`
`2604
`
`3rd party Web-Site
`
`
`
`Fig. 26
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:27)
`Kyocera PX 1040_28
`
`

`

`US. Patent
`
`Apr. 20, 2004
`
`Sheet 27 0f 27
`
`US 6,724,403 B1
`
`2714
`
`2700
`PDA
`
`
`
`Dial-in Bank
`
`Surfcast
`
`Internet Service
`
`Provider
`
`
`
`2710
`
`
`2706
`
`Su rfcast
`
`Server Farm
`
`Firewall
`
`2708
`
`Fig. 27
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:21)(cid:28)
`Kyocera PX 1040_29
`
`

`

`US 6,724,403 B1
`
`1
`SYSTEM AND METHOD FOR
`SIMULTANEOUS DISPLAY OF MULTIPLE
`INFORMATION SOURCES
`
`This application claims priority to provisional patent
`application entitled “System and Method For Simultaneous
`Display oi'Multiple Datastreams”, Ser. No. 60f162,522, tiled
`Oct. 29, 1999.
`
`FIELD OF THE INVENTION
`
`invention relates to methods of presenting
`The present
`information from a variety of sources on a display device.
`Specifically the present invention describes a graphical user
`interface for organizing simultaneous display of information
`from a multitude of sources.
`
`BACKGROUND OF THE INVENTION
`
`The scope of the global communications capacity, com-
`prising fixed link and wireless networks, continues to
`expand rapidly. The variety and complexity of communica—
`tion devices proliferates and the number of users escalates.
`As a result, users are faced with increasingly complex
`systems and interfaces with which to manage multiple
`sources of information. At
`the same time, society has
`increased its demands on time and productivity so that users
`no longer have the luxury of focusing their attention on a
`single source of information or means of communication.
`Instead, the norm today is for people to carry out many tasks
`simultaneously.
`these demands have exposed
`As might be expected,
`substantial problems in current communications technology.
`In particular, users are faced with insuflicient resources to
`manage and access the volume and variety of information
`available to them in an efficient and productive manner.
`While a variety of totals designed to assist in accessing and
`managing these resources have been created,
`these tools
`remain unsatisfactory. Consequently, users are impeded by
`the myriad ofinformation sou rees, each with its own method
`of use and often with its own login and password
`requirements, as well as by slow retrieval times to access the
`information. The result is an unacceptable delay for many
`operations.
`Under the present art, for example, it is usually the case
`that a user lacks the bandwidth resources to receive multiple
`video signals simultaneously. If an individual were receiving
`one video signal, it is usually impractical to receive a second
`at the same time due to bandwidth constraints. Thus, the user
`could not, for example, monitor multiple video data streams
`of sporting or news events; instead, the user could monitor
`only one video data stream at a time.
`To address such bandwidth resource limitations, the cur-
`rent an only accesses information when the user requests it.
`As a result, there is an inevitable delay between the user‘s
`request for information and the communications device’s
`presentation of it. For example, if a user wants to monitor
`sources of news information on the Internet using current
`browser technology, the user must continuously and manu-
`ally request
`the news data from its source to determine
`whether the data has been updated. Prior to requesting and
`subsequently receiving the data,
`the user has no way of
`knowing whether the data has been updated. In any case, the
`user
`is unlikely to want
`to refresh the status of each
`application by manual intervention himself at the frequency
`necessary to ensure that
`the information is up to date.
`Additionally, if a user wishes to view two or more webpages
`simultaneously, he must run two or more copies of the
`
`2
`web-browser program. The act of manually refreshing the
`content of alternate programs in order to ascertain which
`have any new material to offer is fundamentally inefficient.
`Similarly, the user’s access to such data is not in real-time
`or even near real-time because each time the user wants to
`view the information, he must request it from its source and
`wait for the source to transmit it to him. Thereafter, he must
`wait until his communications device has received and
`processed the information before it is presented. For com-
`plex information such as a video signal, this can take longer
`than a minute to occur; and, even for simple information,
`this process can take many seconds. Thins, the user is denied
`real-time or near real-time access to the information.
`
`Present technology that locally stores or “caches” previ~
`othly accessed information to make it available to the user
`more rapidly does not solve this problem, because the
`cached information is necessarily old. The user’s commu-
`nications device must still verify the accuracy of the infor-
`mation with the source before the system displays the
`cached information. As a result, the user is denied real-time
`or near real-time access to updated information.
`Similarly, if a user wishes to make two or more simultav
`ncous downloads there is no control over the relative rates
`at which the respective downloads would occur. So-called
`"push technologies” attempt
`to address this problem by
`organizing information from a number of related sources
`and sending it periodically to a user. While this arrangement
`frees a user from actively participating in the download, the
`price is that the user has little control over the organization
`of the information and can only practically handle a small
`number of such transmissions at any one time. Each trans
`mission is subject to the bandwidth available.
`Of course, not all tasks require the same allocation of
`resources and, correspondingly, not all
`tasks have equal
`priorities for a given user. In particular, a user may wish to
`customize the information environment in such a way that
`many processes are occurring synchronously, yet each is
`communicating with the user at a rate that is acceptable. For
`example, a television viewer may wish to know what is
`being broadcast on several channels at the same time but
`only care to watch one of them closely. An Internet user may
`wish to be continually in touch with sources of data from
`audio, video, chat—room, videoconferencing and e—mail
`checker utilities, but not wish all of them to update at the
`same frequency; the user would be satisfied merely to see at
`a glance a recent status of each. Some of these processes,
`such as chat-room activities entail very little data transmis-
`sion and can, indeed, be effectively updated on a continuous
`basis, whereas others require a great deal of bandwidth but
`could usefully be sampled at a lower rate. The current art
`lacks any technology for controlling the respective refresh
`rates of several simultaneous information sources.
`
`the same time that users are limited by system
`At
`resources, they are also finding that they have no effective
`way of managing the multiplicity of available data types and
`information sources. It is difficult both to conduct two or
`more different types of computing activities at the same time
`or to monitor two or more different
`information sources
`simultaneously because the tools available are confusing,
`inflexible, antUor otherwise difficult
`to implement. Users
`require immediate access to a wide variety of up to date
`content presented in a flexible, easily customized interface.
`In addition to restrictions in the capacity of today’s
`networks, there is very little conformity amongst the infor-
`mation content. A typical communication device, such as a
`personal computer,
`television or mobile telephone, com-
`
`5
`
`ill
`
`15
`
`3o
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:22)(cid:19)
`Kyocera PX 1040_30
`
`

`

`US 6,724,403 B1
`
`3
`prises a display unit connected to a processing unit that can
`accept
`information from many different sources. As
`described above, the signals, data andior datastreams that are
`available to such a device are diverse,
`including,
`for
`example. HTML content, e—mail, or streaming audio and
`video. Correspondingly, the software toolsthat interpret and
`process the dill’erent information sources present each in a
`different way to the user. From a user’s perspective, distinc~
`lions between the different types of information could use-
`fully be removed so that each is vieWed in a similar way and
`such that the current presentation associated with any infor-
`mation source gives an immediate indication of its current
`content. The present reality is different, however. The user
`must contend with a wide range of icons and program
`windows that may occupy space on a user’s display screen.
`Another lack of conformity is the diIIerent mode of behavior
`for programs that address different types of information. An
`efl‘ort to standardize the ways in which different types of
`information are presented to the user would be advanta-
`geous. Equally, unification of the way in which those types
`of information are managed would save time and increase
`user productivity, for productivity is reduced when users
`must cope with different attributes of dilIerent programs and
`learn distinct paradigms for different types of information.
`line nature of the application program windows and their
`respective icons predominantly found on today’s computer
`displays is restrictive. The application window typically
`displays the current content or output of only a single
`program and program icons convey nothing of the pro-
`gram‘s current state or content. Often, an icon is a static
`image which is merely characteristic of the program or data
`represented thereby rather than the program’s current state
`or its information content. In the present art,
`there is no
`intermediate between a window or an icon.
`
`Thus, while a window may be resized as appropriate, it
`will
`frequently occupy the full display area, effectively
`limiting the user to a view of a single program. It may have
`active areas around its borders such as menu bars, scroll
`bars, or tool bars designed to allow the user to control
`aspects of the window's appearance or to set parameters
`specific to the operation of the program controlling it. Icons,
`in contrast, ofl‘er ease of display when multiple programs are
`active, but
`they do not permit viewing or control of the
`underlying program or data represented thereby. Instead,
`icons require user intervention, typically in the form of a
`mouseelick on an icon of interest, to view or control the
`program or information. Consequently, the user’s viewing
`options are limited to a choice between one presenting very
`limited information about a multitude of programs and
`information and one presenting full information, but ofonly
`a single program or data source.
`The fact
`that
`the GUI’s of the present art are largely
`restricted to icons and windows diminishes the capacity to
`organize, manage, and access available information. With
`the Internet representing an ever expanding view of cur-
`rently accessible global information, the need for flexible
`information management
`tools has become crucial.
`Similarly, with the current expansion of television program-
`ming available, for example, through cable television and
`satellite broadcasting, the need to manage this audiovisual
`content becomes cute. The convergence of television pro-
`gramming and computers increases these management
`needs all the more.
`
`Current computer operating system software utilizes
`bookmarking schemes for managing Internet locations and
`complex database technologies for managing specialist
`information. Neither provides visual immediacy or case of
`
`ill
`
`15
`
`an
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`layout. Bookmark hierarchies are presented as cascading
`textual menus and database technologies arrange informa-
`tion into rigidly defined structures. The missing capability is
`a visual categorization in which an area of the display unit
`itself becomes the bookmark and the arrangement on the
`display becomes the categorization, independent of the type
`of content.
`
`While the most common way of accessing information
`sources is via a personal computer, present day technology
`exists to communicate via a television, handheld computing
`device, or even mobile telephones,
`in which case Internet
`content and other data can be displayed as some portion of
`the screen. There is a growing convergence of technologies:
`televisions are beginning to find application as viewers of
`non—television data, (for example through use of “Vertical
`Blanking lnterVal” technology in which a signal is inserted
`into the main video signal or through set-top boxes provid-
`ing limited computer and communications functionality);
`computers are already finding application for the display of
`movies, real-time data streams, and the playing of audio
`data; handheld computing devices and mobile telephones are
`also being enabled to access the Internet and other informa-
`tion sources.
`
`To summarize the current state of the art, display tech-
`nologies currently lack an interface which is capable of
`organizing any type of information, presenting such infor-
`mation to the user in a consistent manner and in such a way
`that all currently open channels are able to indicate their
`activity on a continual basis and which could run on any
`device.
`
`SUMMARY OF THE INVEN‘I'ION
`
`Accordingly. the present invention provides an easy to use
`graphical interface that facilitates the organization and man—
`agement of multiple data sources corresponding to a user’s
`needs and interests. The present invention comprises a grid
`of tiles that resides on the user's computer desktop. The grid
`of tiles provides a uniform, graphical environment in which
`a user can access, operate, andr’or control multiple data
`sources on electronic devices. The graphical environment is
`uniform with respect to the source of accessed information
`and can manage multiple streams of content. entirely of the
`user’s choice. For example,
`the invention presents video
`clips. e-mail messages,
`television shows, Internet sites.
`application programs, data files and folders,
`live video
`streams, music, radio shouts, and any other form of analog
`signal, digital data or electronically stored information, to
`the user uniformly and simultaneously, regardless of
`whether the information is stored locally or available via
`modem, T1 line, infrared. or any other form of communi-
`cation. The user’s impression of the interface is also inde—
`pendent of the type of electronic device upon which it is
`implemented.
`The present invention comprises a method executed by a
`computer under the control of a program stored in computer
`memory, said method comprising the steps of: partitioning a
`visual display of a computer into an array of tiles in a
`non-overlapping configuration; assigning a first refresh rate
`to a first tile of said array of tiles and a second refresh rate
`to a second tile of said array of tiles; updating information
`presented to said first
`tile in accordance with said first
`refresh rate; and updating information presented to said
`second tile in accordance with said second refresh rate.
`
`The present invention additionally includes an electronic
`readable memory to direct an electronic device to function
`in a specified manner, comprising: a first set of instructions
`
`(cid:46)(cid:92)(cid:82)(cid:70)(cid:72)(cid:85)(cid:68)(cid:3)(cid:51)(cid:59)(cid:3)(cid:20)(cid:19)(cid:23)(cid:19)(cid:66)(cid:22)(cid:20)
`Kyocera PX 1040_31
`
`

`

`US 6,724,403 B1
`
`6
`FIG. 15 shows an example of the architecture of the
`computer program in a preferred embodiment of the present
`invention.
`
`FIG. 16 shows the architecture of the application program
`and its components in a preferred embodiment of the present
`invention.
`
`FIG. 17 shows the architecture of components of the
`computer program in a preferred embodiment of the present
`invention.
`
`it]
`
`5
`to control simultaneous communication with a plurality of
`datastreams: a second set of instructions