(12) United States Patent
`Van Ee
`
`US006466203B2
`US 6,466,203 B2
`*Oct. 15, 2002
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`(54) HAND-HELD WITH AUTO-ZOOM FOR
`GRAPHICAL DISPLAY OF WEB PAGE
`
`(75) Inventor: Jan Van Ee, San Jose, CA (US)
`
`(73) Assignee: Koninklijke Philips Electronics N.V.,
`Eindhoven (NL)
`
`(*) Notice:
`
`This patent issued on a continued pros
`ecution application ?led under 37 CFR
`1.53(d), and is subject to the tWenty year
`patent term provisions of 35 U.S.C.
`154(a)(2).
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis
`claimer.
`
`(21) Appl. No.: 09/619,426
`(22) Filed:
`Jul. 19, 2000
`(65)
`Prior Publication Data
`
`US 2002/0030699 A1 Mar. 14, 2002
`
`Related US. Application Data
`
`(63) Continuation-in-part of application No. 09/062,364, ?led on
`Apr. 17, 1998.
`
`(52) US. Cl. ...................................... .. 345/173; 345/835
`(58) Field of Search ............................... .. 345/169, 172,
`345/173, 179, 348, 666, 864, 698, 783,
`784, 670, 671, 156, 157, 835
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,406,307 A * 4/1995 Hirayama et al. ........ .. 345/120
`5,596,346 A * 1/1997 Leone et al. .............. .. 345/127
`5,854,624 A * 12/1998 Grant ....................... .. 345/169
`5,969,706 A * 10/1999 Tanimoto et al. ......... .. 345/130
`6,104,334 A * 4/2000 Allport ..................... .. 341/175
`6,211,856 B1 * 4/2001 Choi et al. ................ .. 345/666
`
`* cited by examiner
`
`Primary Examiner—Bipin ShalWala
`Assistant Examiner—Mansour M. Said
`(74) Attorney, Agent, or Firm—Jeroen Heuvelman
`(57)
`ABSTRACT
`
`A mobile phone has a display With a touch screen. The
`device has a broWser and is capable of retrieving a Web page
`from the Internet. The page is ?rst displayed in its entirety.
`The user can recognize the page’s general lay-out and
`presence of hyperlinks. When the user touches a particular
`location on the touch screen that corresponds to a portion of
`the page’s image, the portion gets displayed so as to ?ll the
`display’s area. Thus, the user can broWse the Web With a
`display of limited siZe.
`
`(51) Int. Cl.7 ................................................ .. G09G 5/00
`
`6 Claims, 1 Drawing Sheet
`
`@
`
`Log
`
`HANDHELD
`
`LCD
`1%
`
`TOUCH
`SCREEN
`
`‘
`
`1% RAM
`11_2
`
`FRAME
`BUFFER
`
`<
`
`1%
`PROCESSOR
`A
`
`‘
`
`1.1.4
`WIRELESS
`MODEM
`\
`
`11_0
`ROM
`PROGRAMS
`BROWSER
`FONTS
`
`SETTINGS
`AUTO-ZOOM
`SOFTWARE
`
`‘ '
`
`,
`
`1 j
`‘ r
`
`11
`
`INTERNET
`
`GOOGLE Ex. 1009
`Google v. Philips
`
`

`
`U.S. Patent
`
`Oct. 15,2002
`
`US 6,466,203 B2
`
`w
`
`T_Og
`
`HANDHELD
`
`LCD
`1g
`TOOOR
`SOREEN
`
`‘
`
`1_0§ RAM
`1g
`FRAME
`BUFFER
`
`‘
`
`’
`
`<
`
`LO
`PROOESSOR
`
`L13
`WIRELESS
`MOOEM
`
`»
`
`1g
`ROM
`PROGRAMS
`BROWSER
`FONTS <—*
`SETTINGS
`AUTO-ZOOM
`SOFTWARE
`
`11
`
`INTERNET
`
`FIG. 1
`
`GOOGLE Ex. 1009
`Google v. Philips
`
`

`
`US 6,466,203 B2
`
`1
`HAND-HELD WITH AUTO-ZOOM FOR
`GRAPHICAL DISPLAY OF WEB PAGE
`
`RELATED APPLICATION
`This application is a continuation-in-part under 37 CFR
`§1.53(b).2 of co-pending U.S. Ser. No. 09/062,364 ?led Apr.
`17, 1198 for Sung Choi and Jan van Ee for GRAPHICAL
`USER INTERFACE TOUCH SCREEN WITH AUTO
`ZOOM FEATURE.
`
`FIELD OF THE INVENTION
`The invention relates to graphical user interfaces (GUI),
`in particular to GUI’s for devices With a relatively small
`screen real estate, such as handheld information appliances
`(palmtops, mobile phones, Web pads, PDA’s or notebook
`computers, etc.)
`
`15
`
`BACKGROUND ART
`Current trends indicate that by 2002 there Will be 1B
`subscribers WorldWide to mobile phones. By 2004 there Will
`be more Mobile phones in use than PCs. Mobile phones
`have become personal, trusted devices. Mobile phones, such
`a the Genie manufactured by Philips Electronics, typically
`have an LCD as part of the user interface and there is a trend
`to provide an ever larger number of onboard softWare
`services.
`Hand-held computing devices, such as personal digital
`assistants (PDA), e.g., the PalmIIIx manufactured by 3COM
`or a WindoWs CE-based handheld, can be connected to the
`Internet via a Wireless modem. As a result, ubiquitous
`information access via the Internet has started to become
`reality. Examples of a Wireless modems are, for example, the
`Minstrel marketed by Novatel Wireless, and the Ricochet
`from Metricom. The Minstrel is a tWo-Way Wireless modem
`for a PDA that lets the user broWse the Web and receive
`email, among other things. In a more general sense, a
`Wireless modem like the Minstrel or Ricochet provides the
`handheld device With an IP address that can be used by any
`type of application that uses the Internet for communication
`(Within limitations of throughput, latency and coverage).
`The Minstrel uses a technology referred to as Cellular
`Digital Packet Data (CDPD) that is supported by the cellular
`service providers. Web sur?ng is limited to a CDPD speed
`cap of 19.2-kbps. The Ricochet has a faster connect rate, in
`the 28.8K-bps range, but it is supported in only three
`metropolitan centers (the San Francisco Bay area, Seattle
`and Wash.).
`Studies further indicate that the functionalities of PDA’s
`and mobile phones have started to converge, and that a
`mobile information society is developing. There Will be an
`emerging of dedicated devices. PDA’s are noW Work
`related. In the near future PDA’s Will be personaliZed
`computers that stay With the user all the time. PDA’s Will get
`more poWer and smaller siZe and accommodate more, and
`more versatile, functionalities.
`BandWidth and display siZe are believed to be the factors
`that limit the usability and practicality of the handheld
`device, be it a mobile phone, a palmtop or a hybrid. In
`particular, the GUI and the services accessible to such
`handhelds are critical factors for the consumers’ acceptabil
`ity of such services. In particular, e-commerce or electronic
`shopping may bene?t from the ubiquity of handhelds if the
`implementing technology addresses the consumers’ needs in
`terms of user-friendliness of the handheld devices.
`
`25
`
`35
`
`45
`
`55
`
`SUMMARY OF THE INVENTION
`Co-pending U.S. Ser. No. 09/062,364 (PHA 23,387) men
`tioned above and incorporated herein by reference, corre
`
`65
`
`2
`sponds to published International Application WO9954807.
`This document relates to a graphical user interface touch
`screen for displaying controllable functions of an electronic
`device. The function is displayed as an icon and at a scale
`siZe in Which the function is recogniZable by a user but too
`small to easily access features of the function. A magni?ed
`version of at least an area of the icon is provided upon the
`user touching the area of the icon. For example, the device
`has a Virtual alphanumeric keyboard. The softkeys dis
`played are too small for the user to select an individual one
`of them. NoW, When the user touches the keyboard in a
`speci?c section that accommodates the desired key, the
`device’s GUI magni?es that speci?c section so that the
`intended key re-appears at a larger scale and can be selected.
`The inventor has realiZed that this feature, referred to as
`“auto-Zoom”, is not only useful Within the context of a
`user-interface for control functionalities represented graphi
`cally by icons. The auto-Zoom is also relevant to the
`rendering of any kind of graphical information on a display
`too small for the total information content, given the dis
`play’s resolution and siZe. For example, handheld informa
`tion processing devices With Internet access (broWsers) and
`displays, such as PDA’s, palmtops, Web pads, mobile
`phones using, e.g., the WAP (Wireless application protocol)
`technology, etc., can be given broWsers for retrieving and
`navigating Web pages from the Internet, but they cannot
`render a page in its entirety Without losing information. The
`lay-out and general appearance of the image of the page,
`hoWever, indicate Whether portions may or may not be of
`interest to the user, e.g., as containing hyperlinks.
`Such handheld devices provided With the auto-Zoom
`feature let the user retrieve graphical information, e.g., a
`Web page or streamed video that is stored, e.g., as a bitmap,
`in the display’s framebuffer or another cache.
`Accordingly, the invention relates to an information pro
`cessing apparatus that has an input for receiving data from
`an external resource, eg the Internet, a display, and a data
`processing system. The system is connected to the input and
`to the display. The system processes the data upon receipt
`and renders on the display an image corresponding to the
`data received. The apparatus has a touch screen for enabling
`a user to interact With the apparatus. The system is operative
`to enable the user to select via the touch screen a portion of
`the image When displayed at a ?rst scale. Upon the portion
`being selected the system renders the selected portion on the
`display at a second scale larger than the ?rst scale (Zoom-in).
`The portion selected corresponds to a location on the touch
`screen. The invention thus alloWs the user to perceive the
`graphical information of the image regardless of the display
`siZe. The invention is especially interesting to handhelds,
`such as PDA’s, palmtops,. mobile phones, Web pads (thin
`clients With broWsing capabilities), etc., because the siZe of
`a handheld’s display is necessarily small due to the required
`form factor and Weight limitation. The ubiquitous informa
`tion access via a broWser is a great asset for Internet-enabled
`handhelds (comprising a Wireless modem), as not only text
`pages but also, e.g., still pictures (e.g., jpeg), streaming
`video, Web page With hyperlinks (e.g., HTML) and java
`animation are noW Within reach of these devices Whose
`screen real estate needs not be the limiting factor anymore.
`The apparatus in the invention can have one or more Zoom
`levels.
`The term “texting” is currently being used to refer to the
`sending and receiving of short text messages using mobile
`phones. Many service providers alloW users to key in and
`send short text messages (SMS: short messaging service).
`
`GOOGLE Ex. 1009
`Google v. Philips
`
`

`
`US 6,466,203 B2
`
`3
`Teenagers in Europe and East Asia seem to have embraced
`this technology, just as young people in the US. have gone
`for live chat and instant messaging on the Internet. The
`invention in US. Ser. No. 09/062,364 (PHA 23,387)
`improves the user-friendliness of the mobile phone’s GUI.
`The invention of the auto-Zoom extended to cover the
`rendering of arbitrary graphical information further broad
`ens the scope of applicability of data processing handhelds,
`especially in the ?eld of consumer electronics
`Screenphones may also bene?t from the current inven
`tion. A screenphone is typically a Wired terminal that has a
`small display monitor and a keyboard. The Internet screen
`phone market is expanding rapidly. Analysts predict rev
`enues in this market segment as over $2 B by 2002. The
`success of the Minitel system in France has triggered the
`introduction of screenphones elseWhere by banks to promote
`home banking and shopping.
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`The invention is explained beloW in further detail, by Way
`of example and With reference to the accompanying draWing
`With a block diagram of a handheld device in the invention.
`
`DETAILED EMBODIMENTS
`
`As speci?ed above, the invention relates to, among other
`things, extending the autoZoom keyboard idea in US. Ser.
`No. 09/062,364 to any kind of graphical or displayable
`information, When the information cannot be displayed in its
`entirety. For example, displaying a Web page With hyper
`links on a mobile phone, Which has a small LCD With touch
`screen functionality, may render the user-interaction With the
`links cumbersome if possible at all, due to their siZe. The
`LCD can nevertheless display the Web page in its entirety by
`scaling it doWn. This initial rendering upon retrieval of the
`page from the Internet may very Well cause the page
`illegible. HoWever, proper ?ltering of the scaled doWn image
`ensures that the user can still recogniZe screen areas of
`potential interest and the hyperlinks therein. When the user
`touches the screen, the portion of the image underneath the
`touched location is enlarged and displayed so that hyperlinks
`can be individually be selected, possibly after a next, similar
`Zoom-in process.
`FIG. 1 is a block diagram With main components of an
`apparatus 100 in the invention. Apparatus 100 is, for
`example, a mobile phone or a palmtop PC With Internet
`access. Apparatus 100 comprises a display 102 for display of
`a graphical information, and a touch screen 104 for user
`interaction With the apparatus. Display 102 comprises, e.g.,
`an LCD. Touch screen 104 is, for example, a resistive tablet.
`For more background on such input devices, see, for
`example, US. Pat. Nos. 5,402,151; 5,231,381; 5,777,607
`and 5,767,458 of Philips Electronics, all incorporated herein
`by reference. LCD 102 and touch screen 104 are physically
`integrated. Apparatus 100 comprises a random-access
`memory 106, a microprocessor 108, and a program memory
`110 (e., an EEPROM). Aportion of memory 106 comprises
`a frame buffer 112. Frame buffer 112 is coupled to display
`102 and stores the information content shoWn on display
`102. Memory 106 stores bitmaps that are mapped onto
`display 102 via frame buffer 106 under control of a softWare
`bit-blitter run on microprocessor 108. Microprocessor 108
`receives user-input via touch screen 104 and translates the
`input into associated GUI actions via frame buffer 112 under
`control of the program memory 110. Handheld 100 com
`prises a Wireless modem 114 for connecting to Internet 116.
`Program memory 110 stores, among other things, a browser
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`4
`application for enabling the user to navigate the Web, and
`the softWare for processing the graphical data as explained
`herein.
`When the user has retrieved a Web page via modem 114
`the page gets displayed on LCD 102 in its entirety. Due to
`the scale siZe of LCD 102 individual hyperlinks or text
`fragments may not be Well discernible, although the lay-out
`of the page conveys sufficient information to the user to
`determine What portion of the page may be relevant. Assume
`that the portion is the upper right hand corner of the page.
`When the user noW touches screen 104 in the associated
`location or area, this action gets translated by processor 108
`and under control of program memory 110 into a Zooming-in
`on that part of the page image that is centered around the
`touch location. This can be implemented, e.g., by a pre
`determined segmentation of the display area into, say, four
`areas that each can be selected for the Zooming-in.
`Alternatively, that part of the original image gets magni?ed
`that has a center coinciding With the touch location.
`User-interaction With touch screen 104 causes display 102
`to undergo a change in appearance. The change is preferably
`effected through animation. Animation is the simulation of
`movement created by displaying a series of pictures, or
`frames, e.g., bitmaps. For example, an image of the original
`page gradually develops in the Zoomed in version of the area
`selected. Through the animation, the user perceives the
`development from one image to the other as a continuous
`transition. The impression is created of a gradually changing
`lay-out. Thus, the animation avoids the impression of an
`abrupt confrontation With a neW lay-out that requires the
`user to re-orientate him/herself. The animation is effected
`through proper processing of the bitmaps in memory 106
`and frame buffer 112.
`Incorporated by reference herein are the following:
`US. Ser. No. 09/062,364 (attorney docket PHA 23,387)
`?led Apr. 17, 1998 for Sung Choi and Jan van Ee for
`GRAPHICAL USER INTERFACE TOUCH SCREEN
`WITH AUTO ZOOM FEATURE. This document relates to
`a graphical user interface touch screen for displaying con
`trollable functions of an electronic device. The function is
`displayed as an icon and at a scale siZe in Which the function
`is recogniZable by a user but too small to easily access
`features of the function. A magni?ed version of at least an
`area of the icon is provided upon the user touching the area
`of the icon.
`U.S. Ser. No. 09/128,839 (attorney docket PHA 23,469)
`?led Aug. 4, 1998 for Jan van Ee for REMOTE CONTROL
`HAS ANIMATED GUI. This document relates to a remote
`control device for remote control of equipment such as a
`home theater. The remote has a display for display of a GUI
`that enables a user to interact With the device. User
`interaction With the device causes the GUI to undergo a
`change in appearance. The change is effected through ani
`mation. Animation is the simulation of movement created by
`displaying a series of pictures, or frames, e.g., bitmaps. For
`example, a panel With clustered control options slides out of
`vieW and a next one slides into vieW, or displayed icons slide
`to neW positions While neW icons appear, etc. Through the
`animation, the user perceives the development from one
`panel to the other as a continuous transition. The impression
`is created of a gradually changing lay-out, of scrolling
`panels, of sliding, rotating, expanding or contracting icons,
`etc. Thus, the animation avoids the impression of an abrupt
`confrontation With a neW lay-out.
`U.S. Ser. No. 09/427,821 (attorney docket PHA 23,786)
`?led Oct. 27, 1999 for Joost Kemink and Richard Sagar for
`
`GOOGLE Ex. 1009
`Google v. Philips
`
`

`
`US 6,466,203 B2
`
`5
`FDA HAS WIRELESS MODEM FOR REMOTE CON
`TROL VIA THE INTERNET. This document relates to a
`PDA combined With a Wireless modem to enable remote
`control of CE equipment via the Internet and a local home
`server. More speci?cally, The Wireless modem enables com
`munication With a server via a data netWork such as the
`Internet. A control netWork is coupled betWeen the server
`and controllable equipment. The handheld is noW capable of
`functioning as a Wireless remote control device for the
`equipment via the Internet and the server. The system may
`comprise a video camera together With hardWare and soft
`Ware to create a formatted still image suitable for being
`displayed on the handheld device. The user can noW instruct
`retrieval of a still image from the server via the Internet. This
`application serves as, e.g., a security system that enables the
`remote user to monitor his/her front porch, or to monitor a
`child by Way of a remote (or fall-back) baby-sit. The
`user-accessibility of equipment is guaranteed by the ubiquity
`of the Internet, thus enabling to expand the range of control
`and monitoring capabilities for a mobile user.
`What is claimed is:
`1. A handheld communication device comprising:
`a Wireless modem for receiving data;
`a display that has a substantially small siZe suitable for in
`the handheld communication device;
`a data processing system connected to the modem and to
`the display for processing the received data and for
`rendering an image corresponding to the data received;
`a touch screen for enabling a user to interact With the
`device;
`Wherein:
`the system is operative to enable the user to select through
`a touch location on the touch screen a portion of the
`image, When displayed at a ?rst scale, for rendering the
`
`15
`
`6
`selected portion on the display at a second scale larger
`than the ?rst scale thereby facilitating a selection of a
`feature; and
`the selected portion When rendered at the ?rst scale is a
`Zoomed-in version of part of the image at the ?rst scale
`substantially centered around the touch screen.
`2. The device of claim 1, Wherein a position of the touch
`location is arbitrary With respect to the touch screen.
`3. The device of claim 1, comprising a broWser.
`4. The device of claim 1, having Wireless Internet access.
`5. The device of claim 1, Wherein the data comprises
`streaming video.
`6. SoftWare for being installed on a handheld communi
`cation device comprising:
`a Wireless modem for receiving data;
`a display that has a substantially small siZe suitable for in
`the handheld communication device;
`a data processing system connected to the input and to the
`display for processing the received data and for ren
`dering an image corresponding to the data received;
`a touch screen for enabling a user to interact With the
`device;
`Wherein:
`the softWare is operative to enable the user to select
`through a touch location on the touch screen a portion
`of the image, When displayed at a ?rst scale, for
`rendering the selected portion on the display at a
`second scale larger than the “?rst scale;” and, thereby
`facilitating a selection of a feature,
`the selected portion When rendered at the ?rst scale is a
`Zoomed-in version of part of the image at the ?rst scale
`substantially centered around the touch screen.
`
`*
`
`*
`
`*
`
`*
`
`*
`
`GOOGLE Ex. 1009
`Google v. Philips