USOO7461353B2
`
`(12) United States Patent
`(10) Patent No.:
`US 7,461,353 B2
`
`Rohrabaugh et al.
`(45) Date of Patent:
`*Dec. 2, 2008
`
`(54) SCALABLE DISPLAY OF INTERNET
`CONTENT ON MOBILE DEVICES
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`(75)
`
`Inventors: Gary B. Rohrabaugh, Bellingham, WA
`(US); Scott A. Sherman, Bellingham,
`WA (US)
`
`(73) Assignee: Gary Rohrabaugh, Bellingham, WA
`(US)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 376 days.
`
`This patent is subject to a terminal dis-
`claimer.
`
`(21) Appl.No.: 11/045,757
`
`(22)
`
`Filed:
`
`Jan. 28, 2005
`
`(65)
`
`Prior Publication Data
`
`US 2005/0131887 A1
`
`Jun. 16,2005
`
`Related US. Application Data
`
`(60) Division ofapplication No. 09/878,097, filed on Jun. 8,
`2001, now Pat. No. 7,210,099, which is a continuation-
`in-part of application No. 09/828,511, filed on Apr. 7,
`2001, now abandoned.
`
`(60) Provisional application No. 60/217,345, filed on Jul.
`11, 2000, provisional application No. 60/211,019,
`filed on Jun. 12, 2000.
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F 17/00
`(52) US. Cl.
`....................... 715/815; 715/760; 715/234;
`715/239; 715/249
`(58) Field of Classification Search ................. 715/517,
`715/70(L864
`
`5,966,135 A
`
`10/1999 Roy et al.
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`
`WO
`
`PCT/US01/40920
`
`12/2001
`
`(Continued)
`OTHER PUBLICATIONS
`
`11,
`http://www.w3.0rg/TIU1999/WD-SVG-l999021l/iPub Feb.
`1999 by W3C pp. 1-7 (A complet details can be down load at by the
`provides URL above).*
`
`(Continued)
`
`Primary ExamineriDoug Hutton
`Assistant ExaminergQuoc A Tran
`(74) Attorney, Agent, or FirmiLaw Office of R. Alan
`Burnett; R. Alan Burnett
`
`(57)
`
`ABSTRACT
`
`Mobile devices enabled to support resolution-independent
`scalable display of Internet (Web) content to allow Web pages
`to be scaled (zoomed) and panned for better viewing on
`smaller screen sizes. The mobile devices employ software-
`based processing of original Web content, including HTML-
`based content, XML, cascade style sheets, etc. to generate
`scalable content. The scalable content and/or data derived
`
`therefrom are then employed to enable the Web content to be
`rapidly rendered, zoomed, and panned. Moreover, the ren-
`dered displays provide substantially the same or identical
`layout as the original Web page, enabling users to easily
`navigate to selected content and features on familiar Web
`pages. Display lists may also be employed to provide further
`enhancements in rendering speed. Additionally, hardware-
`based programmed logic may be employed to facilitate vari-
`ous operations.
`
`See application file for complete search history.
`
`319 Claims, 22 Drawing Sheets
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT
`Petitioner - Motorola
`
`PX 1001
`
`Motorola PX 1001_1
`
`Motorola PX 1001_1
`
`

`

`US 7,461,353 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`1/2000 Huttenlocher et al.
`6,011,905 A
`5/2000 Davis, Jr. et al.
`6,057,854 A
`2/2001 Tso et a1.
`6,185,625 B1
`10/2001 Kanevsky
`6,300,947 B1
`7/2002 Tso et a1.
`6,421,733 B1
`10/2002 Van Ee
`6,466,203 B2
`4/2003 Rempell
`6,546,397 B1
`9/2003 Dutta et a1.
`6,615,212 B1
`................... 345/427
`6,642,925 B2* 11/2003 Roy et al.
`6,674,445 B1*
`1/2004 Chithambaram et al.
`345/619
`6,857,102 B1
`2/2005 Bickmore et al.
`6,886,034 B2 *
`4/2005 Blumberg ................... 709/217
`7,219,309 B2
`5/2007 Kaasila et al.
`2001/0047428 A1
`11/2001 Hunter
`2002/0112237 A1
`8/2002 Ketts
`2004/0049598 A1
`3/2004 Tucker et al.
`2005/0144256 A1
`6/2005 Blumberg
`
`FOREIGN PATENT DOCUMENTS
`
`WO
`
`PCT/US01/40920
`
`6/2002
`
`OTHER PUBLICATIONS
`
`Combined Printout of HTML Help Files for Opera 3.60 (HTML
`pages printed to PDF docs and combined), Available on May 12,
`1999.
`Screenshots compilations ofWeb pages using Opera 3 .60, including:
`WWW.Altavista.comiMay 1, 1999 www.cnetcomiMay 8, 1999
`www.excite.comiMay 8, 1999 www.nytimes.comiApr. 22, 1999
`WWW.softsource.comiApr. 17, 1999 www.utexas.eduiApr. 27,
`1999 www.uspto.goviMay 5, 1999 www.yahoo.comiMay 8,
`1999.
`Benjamin B. Bederson et al., Pad++: A Zoomable Graphical
`Sketchpad For Exploring Alternate Interface Physics, Sep. 19, 1995,
`http://WWW.cs.unm.edu/pad++.
`Benjamin B. Bederson et al., A Zooming Web Browser, SPIE 1996,
`http://Www.cs.umd.edu/hcil/jazz/learn/papers/spie-96-Webbrowser.
`pdf.
`Specification for Simple Vector Format (SVF) v1.1 Jan. 16, 1995.
`Specification for Simple Vector Format (SVF) V2.0 Dec. 6, 2000,
`http://WWW. svf. org/spec .htrnl.
`Changes to SVF, (Date unknown).
`Scalable Vector Graphics (SVG) Specification, W3C Working Draft
`Feb. 11, 1999 WD-SVG-19990211, http://www.w3.org/TIU1999/
`WD-SVG-19990211/ (HTML formatiinitial page).
`Scalable Vector Graphics (SVG) Specification, W3C Working Draft
`Apr. 12, 1999 WD-SVG-19990412, http://www.w3.org/TIU1999/
`WD-SVG-19990412/ (HTML formatiinitial page).
`Scalable Vector Graphics (SVG) 1.0 Specification, W3C Candidate
`Recommendation Nov. 2, 2000.
`
`Introduction to SVG, part of WD-SVG-1999021 1, http://WWW.W3.
`org/TIU1999/WD-SVG-19990211/intro. html#Document .
`.
`.
`.
`Steve Mulder, Sneak Peak at SVG, Mar. 4, 1999, http://WWW.
`webmonkey.com/99/10/index3a.html?tw:eg1999102.
`Janus Boye, SVG Brings Fast Vector Graphics to Web, Jul. 29, 1999
`http://WWW.irt.org/articles/js176/.
`Vector Markup Language (VML), World Wide Web Consortium
`Note May 13, 1998, NOTE-VML-19980513, http://WWW.W3.org/
`TIU1998/NOTE-VML719980513.
`Precision Graphics Markup Language (PGML), World Wide Web
`Consortium NOTE Apr. 10, 1998, NOTE-PGML-19980410, http://
`www.w3.org/TIU1998/NOTE-PGML-19980410.
`Rick Graham, Mobile SVG at BitFlash Inc., May 2001, http://WWW.
`W3 .org/Talks/2002/1007-DI-Helsinki/bitflash/indexhtml.
`.8. Appl. No. 60/296,327 Available on PAIR.
`.8. Appl.
`\Io. 60/288,287,
`filed May 2, 2001. Available on
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,275,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,237,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,274,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,284,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,231,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,224,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,426,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,273,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,283,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`filed Jun. 5, 2001. Available on
`\Io. 60/296,281,
`.8. Appl.
`AIRgelectronic copy previously filed via IDS submission of Sep.
`8, 2007.
`F‘
`filed Jun. 5, 2001. Available on
`\Io. 60/296,327,
`.8. Appl.
`PAIRgelectronic copy previously filed via IDS submission of Sep.
`18, 2007.
`
`
`
`("H’VF‘H’VF‘H’UF‘H’VF‘H’VF‘F‘
`("*—"'U(_‘*—"'U(">-"'U(_‘>-"'U
`
`>—"'U(“>—"'U
`
`* cited by examiner
`
`Motorola PX 1001_2
`
`Motorola PX 1001_2
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 1 of 22
`
`US 7,461,353 132
`
`mm>mww_
`mm>>_1.5::mm
`
`
`mumm1JmD“N.UNKwpzmso
`Eu.__>::E_|||||||mm
`
`>m<s=ENVmm35%.;09
`
`<5:_3$0
`"owningE.0?ONon...
`
`wok/32$:>on8Iu8mo<§ammmDD#50mm
`mm>mmmNN
`
`88at.__u_Fzmkzoo
`E5nEpzmkzooE2..m<sEmmmDEaN6
`on__n_.......1omNEoBmS"fl"E255.-
`
`_n_........#5>on
`
`
`
`..........L._a.E:595mm3
`
`mm
`
`5
`
`mobjmzékomB.35mmwhzmzo
`#58mmzsdzfi
`
`
`
`
`
`
`«Nmmpzmo<20E8$559.;o>m
`
`.05m58
`
`<9.
`
`Motorola PX 1001 3
`
`Motorola PX 1001_3
`
`
`

`

`
`
`
`
`
`
`
`LAND-LINE
`
`O
`5
`\\V\
`
`Dim
`LIJ
`9”;
`>2
`OLIJ
`0:0
`a.
`off
`>5
`‘0
`m
`"
`
`+—
`”2
`o:
`Lu
`I-
`E
`
`l—
`8|—
`a: a
`u a
`[I
`o; E5
`'50 50
`go
`0
`
`oN
`
`LIENT
`
`THIN
`
`é
`
`8
`
`US. Patent
`
`Dec. 2, 2008
`
`Sheet 2 of 22
`
`US 7,461,353 B2
`
`FTPSITE
`
`FIG.13
`
`n:
`o:
`2
`2
`_.|
`LIJ
`5%
`8205,
`2:2
`22
`<
`—<
`0:
`o:
`l—
`|_
`— —
`m
`o
`LO
`(.0
`
`‘1
`
`03
`:2 e
`Z
`mm
`E3
`:0
`
`
`
`12
`
`I"
`
`EH]
`|
`I‘
`
`R:
`
`CLIENTS
`
`Motorola PX 1001_4
`
`Motorola PX 1001_4
`
`

`

`U.S. Patent
`
`000022,
`
`mhS
`
`22f0
`
`164,7SU
`
`2B35
`
`amobjmz<ED8
`
`
`3_
`
`
`
`.05m6_.mew.n=>_m__
`
`85985
`
`Emfizoo
`
`00—.
`
`UN65%
`
`3,a
`
`km
`
`
`
`Dwmmozmmm._m<._<om
`
`
`
`”.meQZ<._.zw._.zOO
`
`w0<n_mw._.z_
`
`mm
`
`Motorola PX 1001 5
`
`Motorola PX 1001_5
`
`
`
`
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 4 of 22
`
`US 7,461,353 B2
`
`CLIENT SENDS CONTENT REQUEST
`
`INCLUDING LOCATION INDICIA (E.G., URL)
`
`100 TO PROXY SERVER
`PROXY SERVER RECEIVES REQUEST, CHECKS
`
`G.
`
`CACHE, AND SENDS REQUEST TO WEB SERVER
`CORRESPONDING TO URL ADDRESS
`
`102
`
`WEB SERVER TRANSMITS PARENT HTML
`DOCUMENT BACK TO PROXY SERVER
`
`104
`
`PROXY SERVER PARSES HTML SEARCHING
`FOR REFERENCES TO EXTERNAL OBJECTS
`
`106
`
`
`REFERENCED
` 44, 46, 48, 50
`OBJECT: HTML
`
`
`
`DOC OR
`IMAGE FILE
`
`
`
`108
`
`YES
`
`PROXY SERVER REQUESTS REFERENCED
`OBJECT FROM APPROPRIATE SERVER
`
`110 NO
`
`
`
`
`
`HTML, XML, CSS CONTENT IS TRANSLATED
`INTO SCALABLE VECTOR REPRESENTATION
`
`114
`
`
`
`116
`
`
`
`
`IMAGE CONTENT TRANSLATED INTO
`COMPRESSED BITMAP FORMAT
`
`
`
`
`
`
`VECTORIZED CONTENT AND COMPRESSED
`BITMAPS ARE STREAMED FROM PROXY
`
`SERVER TO CLIENT
`
` VECTORIZED CONTENT AND BITMAP CONTENT
`
` 120
`PROCESSED AND SCALED USING THIN CLIENT
`
`TO RENDER CONTENT ON CLIENT DEVICE
`
`
`118
`
`Motorola PX 1001_6
`
`Motorola PX 1001_6
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 5 of 22
`
`US 7,461,353 B2
`
`CLIENT SENDS CONTENT REQUEST
`TO NETWORK SITE (E.G., WEB URL)
`
`FIG. 23
`
`HTTP NEGOTIATION PERFORMED TO
`DETERMINE FORMAT CONTENT IS TO BE
`
`
`
`
`
`DELIVERED IN; SERVER CACHE CHECKED
`
`WEB SERVER RETRIEVES
`PARENT HTML DOCUMENT
`
`101
`
`103
`
`105
`
`WEB SERVER PARSES HTML SEARCHING FOR
`REFERENCES TO EXTERNAL OBJECTS
`
`107
`
`44, 45, 43, 50
`
`REFERENCED
`OBJECT: HTML
`DOC OR
`IMAGE FILE
`
`.
`
`YES
`
`109
`
`
`
`
`
`
`
`
`WEB SERVER RETRIEVES REFERENCED
`OBJECT LOCALLY OR RETRIEVES REFERENCED
`OBJECT FROM
`AN APPROPRIATE SERVER
`
`HTML, XML, CSS CONTENT IS TRANSLATED
`INTO SCALABLE VECTOR REPRESENTATION
`
`IMAGE CONTENT TRANSLATED INTO
`COMPRESSED BITMAP FORMAT
`
`116
`
`
`
`VECTORIZED CONTENT AND COMPRESSED
`BITMAPS ARE STREAMED FROM
`
`WEB SERVER TO CLIENT
`
` VECTORIZED CONTENT AND BITMAP CONTENT
`
` 120
`PROCESSED AND SCALED USING THIN CLIENT
`
`TO RENDER CONTENT ON CLIENT DEVICE
`
`
`119
`
`Motorola PX 1001_7
`
`Motorola PX 1001_7
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 6 of 22
`
`US 7,461,353 B2
`
`
`
`CLIENT SENDS CONTENT REQUEST
`TO NETWORK SITE (E.G., WEB URL) TO
`
`RETRIEVE CONTENT
`
`FIG. 2C
`
`NETWORK SITE SENDS
`PARENT HTML DOCUMENT
`
`CLIENT PARSES HTML SEARCHING FOR
`REFERENCES TO EXTERNAL OBJECTS
`
`113
`
`115
`
`117
`
`44' 46’ 48' 5°
`
`REFERENCED
`
`OBJECT: HTML
`DOC OR
`
`IMAGE FILE
`
`119
`
`YES
`
`CLIENT RETRIEVES REFERENCED OBJECT
`FROM AN APPROPRIATE SERVER
`
`121
`
`NO
`
`HTML. XML, CSS CONTENT Is TRANSLATED
`INTO SCALABLE VECTOR REPRESENTATION
`
`IMAGE CONTENT TRANSLATED INTO
`COMPRESSED BITMAP FORMAT
`
`TO RENDER CONTENT ON CLIENT DEVICE
`
`VECTORIZED CONTENT AND BITMAP CONTENT
`PROCESSED AND SCALED USING THIN CLIENT
`
`114
`
`1 16
`
`120
`
`Motorola PX 1001_8
`
`Motorola PX 1001_8
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 7 of 22
`
`US 7,461,353 B2
`
`CONTENT
`
`ARBWRARY
`INTERNET
`
`Illll I
`
`:l
`
`I I :I
`
`Il
`
`lIIIIlIIIlII
`
`MAIN
`MEMORY
`
`READ ONLY
`MEMORY
`
`$832!;
`DEVICE
`
`504
`
`‘ 506
`
`t
`
`PROCESSOR 502
`
`Motorola PX 1001_9
`
`I E
`
`:
`:I
`
`III
`
`
`
`
`IIIIIII :
`
`IIIII
`
`KEYBOARD
`
`522
`
`CURSOR
`CONTROLDEWCE
`
`523
`
`COMMUNBANON
`DEWCE
`
`
`
`Motorola PX 1001_9
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 8 of 22
`
`US 7,461,353 B2
`
`MARKET HOME
`
`EILE
`EDIT MIEW
`Q0 FAVORITES HELP
`63
`anionic/m
`BACK FORWARD STOP REFRESH HOME SEARCH FAVORITES HISTORY
`
`:
`
`--
`
`Dow Jones 38 lncpstrlais (54min Chart)
`‘- I083!
`lmélx
`_
`I!!!‘I
`'
`‘Ir'fllnnwtfimhi
`-- "3799
`—- was?
`_ - ions
`-- I670:
`
`ACCOUNT#
`255A
`ACCESS CODE
`256A
`
`where you want to go. Learn more
`
`page
`*—’- 449
`' 3"“..33
`~d‘22
`-- 4|“
`—am
`-— 4074
`—4059
`
`NetVest's Mutual Fund Mart gives you «1258A
`access to over 250 fund companies and
`funds. including 1,265 no load funds.
`32A
`
`Our no-fee lRA* combined with Fund «1 260A
`
`Mart is the vehicle that will get you
`
`228A
`(‘ 220A
`INVES TMEN TS
`230A
`
`MUTUAL
`FUNDS
`
`IRA, 401 K
`OPTIONS
`
`TAX
`INFORMATION
`
`Motorola PX 1001_10
`
`Motorola PX 1001_10
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 9 of 22
`
`US 7,461,353 B2
`
`MARKET HOME
`
`EILE
`EDIT
`\_/|EW
`GO FAVORITES HELP
`anal-0’36?
`BACK FORWARD STOP REFRESH HOME SEARCH FAVORITES HISTORY
`
`ADDRESS HTTP://<ADDRESS PATH>
`
`2383
`
`2403
`
`TEXT OBJECT #1
`
`ADD BANNER
`GRAPHIC OBJECT #9 \
`
`HEADLINE OBJECT
`
`248B
`
`GRAPHIC OBJECT #10
`
`.G_RAPH'C
`
`GRAPHIC OBJECT #12
`
`LOGO GRAPHIC
`OBJECT #1
`
`I GRAPHIC #2;
`LINK #12223
`
`GRAPHIC #3;
`LINK #222 33
`
`GRAPHIC #4;
`
`LINK #32263
`
`GRAPHIC #5;
`
`LINK #4
`
`GRAPHIC #6;
`LINK #5230A
`
`GRAPHIC #7;
`
`LINK #62323
`
`GRAPHIC # 8;
`
`LINK #72343
`
`212 /
`2103
`
`FIG. 43
`
`214
`
`Motorola PX 1001_11
`
`Motorola PX 1001_11
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 10 of 22
`
`US 7,461,353 B2
`
`MARKET HOME
`
`
`
`Motorola PX 1001_12
`
`Motorola PX 1001_12
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 11 of 22
`
`US 7,461,353 B2
`
`MARKET HOME
`
`
`
`2” 215‘
`
`FIG. 4D
`
`Motorola PX 1001_13
`
`Motorola PX 1001_13
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 12 0f 22
`
`US 7,461,353 B2
`
`MARKET HOME
`
`ED T
`
`\_/IEW
`
`(_3_O
`
`FAVORITES
`
`_E P
`
`1:].-
`e
`
`Ilo’
`9
`(321:?
`BACK FORWARD STOP REFRESH HOME SEARCH
`ADDRESS
`HTTP://<ADDRESS PATH>
`
`53
`HISTORY
`
`flLlNKS
`
`NEWS SPARKS MARKET
`
`the vehicle that will get you where you want to
`
`Il,""‘t""""mm'i.
`
`— 16831
`
`“- 1076;?
`
`3pm
`
`km Cowosite index (5mm Chart)
`.3
`-§!76
`“J.“ at...“ ,.._."~u..«m
`—4154
`- 4138
`”4122
`- m”4186
`-- 4890
`—4974
`“. 4958
`
`NetVest's Mutual Fund Mart gives you access
`to over 250 fund companies and funds,
`including 1,265 no load funds.
`
`Our no-fee IRA* combined with Fund Mart is
`
`Motorola PX 1001_14
`
`Motorola PX 1001_14
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 13 of 22
`
`US 7,461,353 B2
`
`
`
`21°F
`
`FIG. 4F
`
`Motorola PX 1001_15
`
`Motorola PX 1001_15
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 14 of 22
`
`US 7,461,353 B2
`
`H1'=180*SF
`
`210G
`
`FIG. 4G
`
`Motorola PX 1001_16
`
`Motorola PX 1001_16
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 15 of 22
`
`US 7,461,353 B2
`
`PARSE HTML CONTENT TO IDENTIFY LAYOUT
`INFORMATION TAGS
`
`150
`
`SEPARATE CONTENT INTO OBJECTS AND
`DEFINE BOUNDING BOX FOR EACH OBJECT
`
`152
`
`DEFINE PAGE LAYOUT
`BASED ON BOUNDING BOXES
`
`DEFINE DATUM POINT FOR PAGE AND FOR
`EACH OBJECT BOUNDING BOX
`
`154
`
`156
`
`GENERATE VECTOR FROM PRIMARY DATUM TO
`BOUNDING BOX DATUM FOR EACH OBJECT
`
`158
`
`CREATING A REFERENCE THAT LINKS EACH
`OBJECT TO ITS VECTOR AND BOUNDING Box
`
`150
`
`FIG. 5
`
`Motorola PX 1001_17
`
`Motorola PX 1001_17
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 16 of 22
`
`US 7,461,353 B2
`
`GATHER VECTOR REPRESENTATION DATA
`CORRESPONDING TO WEB PAGE
`
`160
`
`BUILD DISPLAY LIST OF VECTORS
`
`162
`
`DETERMINE USER-SELECTABLE
`ZOOM SCALE AND OFFSET
`
`PROCESS VECTORS AND BOUNDARY BOXES
`
`TO DETERMINE VIRTUAL LAYOUT OF DISPLAY;
`DETERMINE DISPLAY LIMIT BOUNDING BOX
`
`RETRIEVE CONTENT CORRESPONDING TO
`
`OBJECTS HAVING AT LEAST A PORTION OF
`THEIR BOUNDING BOXES FALL WITHIN
`
`THE DISPLAY LIMIT BOUNDING BOX
`
`SEPARATOR LINES AND BORDERS
`
`SCALE CONTENT; SCALE BITMAPS, RETRIEVE/
`BUILD APPROPRIATE FONTS; SCALE
`
`RENDER THOSE PORTIONS OF SCALED
`CONTENT FALLING WITHIN
`
`THE DISPLAY LIMIT BOUNDING BOX
`
`164
`
`166
`
`168
`
`170
`
`172
`
`FIG. 6
`
`Motorola PX 1001_18
`
`Motorola PX 1001_18
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 17 of 22
`
`US 7,461,353 B2
`
`Palm |||c
`
`IOCIIIOI
`I'lm-lfllfihwafi “it"m/
`II._II;-nnnvu III-I...
`hr‘li “Mann-my
`nu III-«VIIAJII-III
`In sud-sun. tn.”
`
`III, ._.I_Isa.II.§I.qInfamy.-
`
`a“,_ 9‘5“
`‘Iildllp PM“ V.
`liklrfl'bl-lihl-hr.
`--
`'
`- “z\‘-A-r’-»'
`-W ”W .IQ.&:I—ban—IL
`haul-raw...» Ind-III‘HI-
`.I.'—;.Ia III." ,
`IIaal'
`‘
`‘fi""' _
`
`u
`
`hl "Hf-Nil”!
`
`In.“
`II.III.
`I-jI-Iiihm-_ Inn-gunman...
`“Ian-mi
`_iII.iI
`,
`I
`myus-i.-n.i_'—I.s4II-u.p.ll;_i
`
`,
`,
`thug-l. III-“l
`
`;-g§..5.vu
`
`FIG. 7A
`
`Motorola PX 1001_19
`
`Motorola PX 1001_19
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 18 of 22
`
`US 7,461,353 B2
`
`Palm lllc
`
`mom->5]
`
`; Ydhooflagpirect fsend and"
`receiire {niche}: Oniih'e
`
`- Geti ourown-Web domain
`
`. YiTramel - buy tickets, check
`arrival times
`
`on
`
`-?pm ET: Manx-3.762” .
`
`- 8pm: Penguins #5. Stars
`
`FIG. 73
`
`Motorola PX 1001 _20
`
`Motorola PX 1001_20
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 19 0f 22
`
`US 7,461,353 B2
`
`Palm |||c
`
`"m"' '
`
`' nu! Iihh'
`5 5“" “U
`
`u- III—I
`
`III-'1 a- a nuf_ IWEHflt’mWi—ru ‘v
`
`- mum.- um.
`:wu. dfl--lmhfidufi-Rfi
`" mag-we...
`’ =:*.i'g'.‘.'“'v" Whflflh
`A
`d-Ivld-i—Db-L'
`4 my a.“
`|
`. I
`h
`I_
`“‘4"
`rag-“W
`‘ mu
`7
`_
`inn;
`
`“ I’
`. :w‘ fififif“
`-
`-_-
`III“
`‘ o
`a
`'
`.h!‘ ‘
`.. ,mm_-_...m 1'
`
`' limb!
`
`.IQ".
`
`FIG. 8A
`
`Motorola PX 1001_21
`
`Motorola PX 1001_21
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 20 of 22
`
`US 7,461,353 B2
`
`Palm |||c
`
`
`
`FIG. 83
`
`Motorola PX 1001 _22
`
`Motorola PX 1001_22
`
`

`

`US. Patent
`
`Dec. 2, 2008
`
`Sheet 21 of 22
`
`US 7,461,353 B2
`
`Palm Illc
`
`'W'r
`. JIII-‘I'HIIII‘
`I-Jlluolcl w -
`“pin My"
`'w‘
`mm"
`'Wth ._f
`u-. .Wnun hum-nu
`
`
`,
`9-
`Ill- !
`7
`I!
`'uuéu tin-h.
`7
`.6..." “all Mia
`>
`
`uh,
`
`vx-w...»........,..«....2W..._........2...H.2.2....._..._._.
`
`“Gianni.
`
`. {Ia-I
`'
`‘ hmh '
`,1: 2...“. an" ,
`‘flh...’ ’.
`.
`
`and! .g-gl Ivy—1.? ifilhflndniifin—Udu- M ‘7a...
`
`FIG. 9A
`
`Motorola PX 1001 _23
`
`Motorola PX 1001_23
`
`

`

`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 22 of 22
`
`US 7,461,353 B2
`
`Palm Illc
`
`j TeUt
`; May Resign {PIP}- President Joseph "
`Estroda'5 hold on power fellapartas
`militaryandpolitical allies abandoned
`‘ him and hundreds ofthousands of
`Filipinos threatened to march on the
`
`i
`
`‘ corruption scandai (FIFE)
`
`presidentialpalateatdawnSaturday1
`
`to force himout. The former movie
`‘ star, beleaguered for months by
`‘
`'corrLipt‘ion allegations was
`3 reportedly negotiating over terms
`i for his resignation Morey
`* ChI’OI‘IOIOQ)’ of Philippine
`
`FIG. 93
`
`Motorola PX 1001 _24
`
`Motorola PX 1001_24
`
`

`

`US 7,461,353 B2
`
`1
`SCALABLE DISPLAY OF INTERNET
`CONTENT ON MOBILE DEVICES
`
`RELATED APPLICATIONS
`
`The present application is a Divisional Application of US.
`application Ser. No. 09/878,097, filed Jun. 8, 2001, now US.
`Pat. No. 7,210,099, which is a Continuation-in-Part of US.
`Non-Provisional application Ser. No. 09/828,511, filed Apr.
`7, 2001, now abandoned, entitled “RESOLUTION INDE-
`PENDENT VECTOR DISPLAY OF INTERNET CON-
`
`TENT,” the benefit ofthe filing date ofwhich is claimed under
`35 U.S.C. § 120. This application further claims the benefit of
`the filing dates of US. Provisional Application No. 60/211,
`019, filed Jun. 12, 2000, entitled “METHOD AND SYSTEM
`FOR RESOLUTION INDEPENDENT DISPLAY OF
`
`HTML AND XML CONTENT” and US. Provisional Appli-
`cation No. 60/217,345,
`filed Jul.
`11, 2000,
`entitled
`“METHOD AND
`SYSTEM FOR
`SELECTION,
`RETRIEVAL, AND CONVERSION OF COMPUTER
`CONTENT TO VECTOR FORMAT FOR RESOLUTION
`
`INDEPENDENT DISPLAY,” under 35 U.S.C. § 119(e).
`The present application is also related to US. application
`Ser. No. 11/735,477 filed on Apr. 15, 2007, US. application
`Ser. No. 11/735,482 filed on Apr. 15, 2007, US. application
`Ser. No. 11/738,486 filed onApr. 21, 2007, and US. applica-
`tion Ser. No. 11/738,932 filed onApr. 23, 2007, each ofwhich
`are continuations of US. application Ser. No. 09/878,097,
`now US. Pat. No. 7,210,099.
`
`COPYRIGHT NOTICE
`
`Contained herein is material that is subject to copyright
`protection. The copyright owner has no objection to the fac-
`simile reproduction of the patent disclosure by any person as
`it appears in the Patent and Trademark Office patent files or
`records, but otherwise reserves all rights to the copyright
`whatsoever.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The invention relates generally to translation of Internet
`and World Wide Web content to scalable vector representa-
`tion. More particularly, the invention relates to apparatus and
`methods for zoom enabling the display of content in an Inter-
`net information browser by retrieving and translating Hyper-
`Text Markup Language (HTML), extensible Markup Lan-
`guage OiML), and other
`Internet content
`to vector
`representations of that content.
`2. Description of the Related Art
`Text only Internet information browsers began as a project
`at the CERN, European Organization for Nuclear Research,
`facility in Geneva Switzerland. From its inception the intent
`was to provide a mesh or web ofaccess to data with a common
`user interface. Browsers moved from the academic environ-
`
`ment when NCSA, the National Center for Supercomputing
`Applications at the University of Illinois in Urbana-Cham-
`paign developed Mosaic, an Internet information browser and
`World Wide Web client.
`
`Internet content is stored in multiple file formats. These
`formats include HTML (Hyper Text Markup Language) and
`XML (extended Markup Language) as well as graphic file
`format GIF (Graphics Interchange Format) and JPEG (Joint
`Photographic Experts Group). These four file formats consti-
`tute the majority of Internet content. Font size and resizing
`display area for content can alter the size of the display of
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`Internet content in existing browsers. The majority of Internet
`content displays as a flat single resolution with no browser
`support for zoom.
`Much ofthe Internet content has been designed for display
`on desktop computers with a single target resolution. Even
`though HTML has the ability to adapt to changes in screen
`resolution, major Internet content providers have chosen to
`create their Web pages using fixed resolution structures, such
`as tables. This gives them the ability to control the look and
`feel of their Web sites. This fixed resolution approach has
`evolved to the point that the fixed resolution layout of Web
`pages has become the most common method to brand or
`uniquely identify Web sites. While this fixed resolution
`approach is good for site branding and product differentiation
`it does present a daunting technical problem for display of
`Internet content (designed for desktop computers) on small
`screen, low resolution, or different aspect ratio devices, such
`as cell phones and hand held computers.
`
`BRIEF SUMMARY OF THE INVENTION
`
`In accordance with aspects ofthe invention, mobile devices
`enabled to support resolution-independent scalable display of
`Internet (Web) content to allow Web pages to be scaled
`(zoomed) and panned for better viewing on smaller screen
`sizes are disclosed. The mobile devices employ novel pro-
`cessing of original Web content, including HTML-based con-
`tent, XML, cascade style sheets, etc. to generate scalable
`content. The scalable content and/or data derived therefrom
`
`are then employed to enable the Web content to be rapidly
`rendered, zoomed, and panned. Moreover, the rendered dis-
`plays provide substantially the same or identical layout as the
`original Web page, enabling users to easily navigate to
`selected content and features on familiar Web pages. Display
`lists may also be employed to provide further enhancements
`in rendering speed. Additionally, hardware-based pro-
`grammed logic may also be employed to facilitate various
`operations.
`According to further aspects, some mobile devices may
`employ touch-sensitive display screens that enable users to
`provide various inputs to control display of content within
`Web pages. Exemplary user inputs include tap-based inputs
`to selectively zoom in on columns, images, and paragraphs.
`Users can also define a window to zoom in on via the touch-
`
`sensitive display.
`According to additional aspects of the invention, methods
`and software for enabling support for resolution-independent
`scalable display of Web content are provided. The methods
`and software enable users of various devices, from handheld
`devices with small screens, to desktop PCs and laptops, to
`very large screen devices, to view and interact with Web pages
`in a manner independent of the screen resolution of such
`device’s built-in or associated display, while maintaining the
`look and feel of browsing such pages with a conventional
`desktop browser. Thus, users of various devices having dif-
`ferent screen resolutions are enabled to browse Web pages
`from among literally billions of Web pages while providing a
`full Web browsing experience.
`Other features of the present invention will be apparent
`from the accompanying drawings and from the detailed
`description that follows.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The appended claims set forth the features of the invention
`with particularity. The invention, together with its advan-
`
`Motorola PX 1001 _25
`
`Motorola PX 1001_25
`
`

`

`US 7,461,353 B2
`
`3
`tages, may be best understood from the following detailed
`description taken in conjunction with the accompanying
`drawings of which:
`FIG. 1A is a block schematic diagram illustrating a first
`exemplary system infrastructure in accordance with the
`present invention in which content translation services are
`performed by a third-party proxy service that translates con-
`tent requested from a client that is retrieved from one or more
`network resources into a scalable vector representation and
`delivers the translated content to the client;
`FIG. 1B is a block schematic diagram illustrating a second
`exemplary system infrastructure in which the translation of
`content is performed at a content provider’s web site and
`delivered directly to the requesting client;
`FIG. 1C is a block schematic diagram illustrating a third
`exemplary system infrastructure in which content received
`from one or more network sources is translated into a scalable
`
`vector representation at the client;
`FIG. 2A is a flowchart illustrating how data is retrieved,
`processed and transferred in accordance with the system
`infrastructure of FIG. 1A;
`FIG. 2B is a flowchart illustrating how data is retrieved,
`processed and transferred in accordance with the system
`infrastructure of FIG. 1B;
`FIG. 2C is a flowchart illustrating how data is retrieved,
`processed and transferred in accordance with the system
`infrastructure of FIG. 1C;
`FIG. 3 is a block schematic diagram illustrating an exem-
`plary architecture corresponding to the proxy server of FIG.
`1A;
`FIG. 4A is a representation of an exemplary web page has
`displayed on a conventional browser;
`FIG. 4B is a schematic diagram illustrates various objects
`that are generated based on the HTML code of the web page
`of FIG. 4A;
`FIG. 4C is a schematic diagram illustrating a set of vectors
`and bounding boxes corresponding to the objects generated in
`FIG. 4B;
`FIG. 4D is a schematic diagram illustrating how various
`vectors and bounding boxes may be defined in accordance
`with the invention;
`FIG. 4E is a representation ofthe web page of FIG. 4A after
`it has been offset and scaled in accordance with the invention;
`FIG. 4F is a schematic diagram illustrating new datum
`points and bounding boxes corresponding to the scaled and
`offset web page;
`FIG. 4G is a schematic diagram illustrating new vectors
`and bounding box parameters for a pair of objects in the
`scaled and offset web page;
`FIG. 5 is a flowchart illustrating the logic used by the
`invention when translating content into a scalable vector rep-
`resentation of that content;
`FIG. 6 is a flowchart illustrating client-side operations that
`are performed to create a rendered display page based on the
`translated content the client receives and user-input;
`FIGS. 7A and 7B are representations of a nominal and a
`zoomed in column view of an exemplary web page as they
`might appear on a Palm device;
`FIGS. 8A and 8B are representation of nominal and
`zoomed in view of an exemplary graphic image as they might
`appear on the Palm device;
`FIGS. 9A and 9B are representations of a nominal and
`zoomed in view of a text portion of a web page as they might
`appear on the Palm device; and
`FIG. 10 illustrates an exemplary computer system that may
`be used for implementing various aspects of embodiments of
`the invention.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`DETAILED DESCRIPTION OF THE INVENTION
`
`Apparatus and methods are described for creating resolu-
`tion independent vector display of Internet content to allow it
`to be scaled (zoomed) larger and smaller for better viewing or
`to fit any resolution or screen size. In addition, infrastructure
`and methods are provided for delivering such content to cli-
`ents.
`
`In the following description, for the purposes of explana-
`tion, numerous specific details are set forth in order to provide
`a thorough understanding of the present invention. It will be
`apparent, however, to one skilled in the art that the present
`invention may be practiced without some of these specific
`details. In other instances, well-known structures and devices
`are shown in block diagram form.
`The present invention includes various operations, which
`will be described below. The operations of the present inven-
`tion may be performed by hardware components or may be
`embodied in machine-executable instructions, which may be
`used to cause a general-purpose or special-purpose processor
`or logic circuits programmed with the instructions to perform
`the operations. Alternatively,
`the operations may be per-
`formed by a combination of hardware and software.
`The present invention may be provided as a computer
`program product that may include one or more machine-
`readable mediums having stored thereon instructions, which
`may be used to program a computer (or other electronic
`devices) to perform a process according to the present inven-
`tion. The machine-readable medium may include, but is not
`limited to, floppy diskettes, optical disks, CD-ROMs, and
`magneto-optical
`disks,
`ROMs,
`RAMs,
`EPROMs,
`EEPROMs, magnetic or optical cards, flash memory, or other
`type of media/machine-readable medium suitable for storing
`electronic instructions. Moreover, the present invention may
`also be downloaded as a computer program product, wherein
`the program may be transferred from a remote computer (e. g.,
`a server) to a requesting computer (e.g., a client) by way of
`data signals embodied in a carrier wave or other propagation
`medium via a communication link (e.g., a modem or network
`connection). Accordingly, herein, a carrier wave shall be
`regarded as comprising a machine-readable medium.
`Client Overview
`
`According to one embodiment, an ultra-thin client-side
`viewer provides the graphics, linking, caching, and function
`handling capabilities necessary for extending the web to
`almost any platform. It is designed as a lightweight browser
`(micro-browser) running directly on device operating sys-
`tems. In alternative embodiments, the client-side viewer may
`be deployed as a standard browser plug-in, or Java applet for
`extending browser functionality. In one embodiment, the cli-
`ent-side viewer attains its small size and efficiency by taking
`advantage of the power of SVF (Simple Vector Format) to
`describe almost any current web content. SVF files can be
`handled with a tiny fraction of the client code required by
`normal web browsers because current browsers must inter-
`
`pret a large and growing number of file types and their idio-
`syncrasies. SVF was originally designed to handle a superset
`ofthe most commonly used file formats in the complex world
`of CAD. It can accommodate not only new graphical func-
`tions, but the storage and transfer of almost any foreseeable
`new functional capability. SVF has been under consideration
`by the W3C (World Wide Web Consortium) for adoption as a
`standard for vector content on the World Wide Web.
`
`By working tightly with a server-side content translator,
`web content and functionality can be passed seamlessly to the
`end user platform without any degradation in the look or feel
`of the output. In addition, because the resulting file graphics
`
`Motorola PX 1001 _26
`
`Motorola PX 1001_26
`
`

`

`US 7,461,353 B2
`
`5
`are handled as vectors, the end user can control real time
`changes in the size oftext and graphics as well as what portion
`of the file is viewable in the display. This “zoom and pan”
`capability, familiar to CAD and other vector content software
`users, adds dramatically to the usability of non-standard dis-
`play sizes. For very small displays, real time zooming and
`panning allows the user to see graphics and text at sizes that
`make them easily readable, and then “back up” to view an
`entire page for context or pan in any direction for navigation.
`Because the client-side viewer manipulates vectors, there is
`no loss in quality as the display is zoomed. The graphics
`rendering engine within the client is so efficient that file
`manipulation happens in a fraction of a second. There is no
`perceptible wait for the user as the file is resized, or the
`window is repositioned. Content created for one display reso-
`lution now can be sized, real time, for any other display
`without degradation. Besides making small displays emi-
`nently usable, this technology extends web content into some
`sur