US007577910B1
`
`a2) United States Patent
`Husemann etal.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,577,910 B1
`Aug. 18, 2009
`
`(54) METHOD AND APPARATUS FOR
`PROVIDING A MORE POWERFUL
`USER-INTERFACE TO DEVICE WITH A
`LIMITED USER-INTERFACE
`
`(75)
`
`Inventors: Dirk Husemann, Adliswill (CH);
`Michael Moser, Zurich (CH)
`
`(73) Assignee:
`
`International Business Machines
`Corporation, Armonk, NY (US)
`
`(*) Notice:
`
`Subject to any disclaimer, the termof this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1343 days.
`
`(21) Appl. No.: 09/613,113
`
`(22)
`
`Filed:
`
`Jul. 10, 2000
`
`(30)
`
`Foreign Application Priority Data
`
`6,792,605 Bl *
`
`9/2004 Roberts et al.
`
`.............. 719/313
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`WO
`WO
`
`07-254920
`10-162060
`L0-240833
`11-027290
`11-031114
`11-161321
`2002-509669
`WO 95/28673
`WO 98/59284
`
`10/1995
`6/1998
`9/1998
`1/1999
`2/1999
`6/1999
`3/2002
`10/1995
`12/1998
`
`OTHER PUBLICATIONS
`
`European Search Report, Application No. EP 99 111 3414, Mar. 21,
`2000.
`Jaap Haartsen, “Bluetooth—The universalradio interface for ad hoc,
`wireless connectivity”, Ericsson Review No. 3, 1998.
`Christer Erlandson and Per Ocklind, “WAP—Thewireless applica-
`tion protocol”, Ericsson Review No. 4, 1998.
`
`Jul. 12, 1999
`
`(EP)
`
`wetonnce DOTA TS
`
`* cited by examiner
`
`(51)
`
`Int. Cl.
`(2006.01)
`G06F3/00
`(52) U.S.C. occ 715/744; 715/745; 715/746;
`715/747
`
`(58) Field of Classification Search .................. 345/716,
`345/719, 720, 721, 723, 730; 715/744-749
`See application file for complete searchhistory.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`............. 709/218
`1/2001 De Booretal.
`6,173,316 BI*™
`4/2001 Luoetal.
`........c.... 709/217
`6,216,158 BL*
`9/2002 Holland etal. .............. 7IS/513
`6,446,096 B1*
`9/2002 Dara-Abramsetal. ........ 700/83
`6,456,892 Bl*
`6.466.971 B1* 10/2002 Humpleman etal. ........ 709/220
`6,502,000 BL”
`12/2002 Arnoldetal.
`sees 700/83
`6,509,913 B2™ 1/2003 Martin et al.
`wee 345/762
`6,560,640 B2*
`
`5/2003 Smethers ..........0.060. 709/219
`
`Primary Examiner—Ba Huynh
`(74) Attorney, Agent, or Firm—Brian P. Verminski, Esq.
`
`(57)
`
`ABSTRACT
`
`A method and apparatus for controlling a computer device
`with a limited user-interface via a remote computer device
`having a more powerful user interface. Both computer
`devices are interconnected via a wireless communication
`channel and both computer devices support a common com-
`munications protocol. User-interface informationis sent from
`the computer device with a limited user-interface to the
`remote computer device to provide a user-interface at the
`remote computer device for receiving user input at the remote
`computerdevice. The user input controls the computer device
`with a limited user-interface to execute commands therein
`corresponding to the user input.
`
`26 Claims, 10 Drawing Sheets
`
` send Ul-deseription
`
`
`
`
`render Ul-description
`(e.g. display or read to user)
`
`
`
`
`
`send user inpalto device!
`
`UNIFIED 1006
`
`UNIFIED 1006
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet 1 of 10
`
`US 7,577,910 B1
`
`Me
`
`<command>;<parameters>
`
`16
`
`
`
`FIG. 1
`
`
`
` brought
`into vicinily
`
`/detautt-fonts | 3
`
`
`
`~ Pstyle|
`Ysize|
`
`|
`|
`
`— /
`|
`
`Ystyle
`
`-
`
`Vinterface | _
`|
`Vserial|
`|
`|
`oaud-rate|
`|
`|
`Vparity
`.
`
`-
`
`
`
`
`
`
`
`CEny
`| sats
`|
`j
`|
`formats
`
`FIG. 2
`
`| JCL[=3
`
`
`[PostSeript
`]
`|
`|
`[ad
`|
`
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet2 of 10
`
`US 7,577,910 B1
`
`erineeefilesiSeieeeeolBS
`URL: foluetoothvisony_cdp_990Ximain_menu.wml
`
`Play mode
`
`Welcome to
`SONY CD-Player 990X
`Please select the function you wantto
`configure:
`
`CD labels
`
`Generalsettings
`
`[|_cO.namesJ}
`play_mode
`{
`
`FIG. 3
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet3 of 10
`
`US 7,577,910 B1
`
`STEaecendear
`nsfsony_cdp_990Ximain_menu.wml#CD_names
`
`CD names:
`
`Enter a namefor the currently inserted |
`
`CO:
`
`|
`
`t
`
`General settings
`
`CO_names
`|_play.mede
`
`FIG. 4
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet4 of 10
`
`US 7,577,910 B1
`
`lSeymoaes
`
`‘Select one of the following play
`
`i| ©: User-Defined
`
`© Normal
`
`( Random
`
`© Shufle
`
`FIG. 5
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet 5 of 10
`
`US 7,577,910 B1
`
`
`
`64
`
`chitp-request>
`
`Sy)
`
`60
`
`<command>;<paramotors>
`
`66
`
`
`
` brought
`into vicinity
`
`
`FIG. 6
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet6 of 10
`
`US 7,577,910 B1
`
`73
`
`transmitter
`driver
`
`protocol
`
`device specific
`
`user interface
`
`manager
`
`
` 70
`
`UserInterfaceVE
`HWInterface
`
`receiver
`
`
`FIG.7D
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet7 of 10
`
`US 7,577,910 B1
`
`MAC
`protocol
`
`user interface
`
`manager
`
`FIG. 7B 700
`
`transrmutter
`
`receiver
`
`
`
`UserInterface
`
`FIG. 7C
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet8 of 10
`
`US 7,577,910 B1
`
`ona oi
`
`
`Feri ods cal |
`
`send service
`information
`
`84
`
`
`
`B44.
`
`background
`
`
`
`update own list
`with services
`
`ee ee)
`
`
`
`send feedback
`
`FIG. 8
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet9 of 10
`
`US 7,577,910 B1
`
`(02 +2:
`;
`
`receive service
`information
`
`103
`
`background
`
`
`
`user requests (List of) devices
`
`display list of controllable devices
`
`
`
`user selects device
`
`94
`
`send request to send Ul-description|
`7
`
`receive Ul-description
`
`NO
`
`95
`~ ‘Addiera|
`UL Infeemata:
`
`render Ul-description
`(e.g. display or read to user)
`
`receive user input
`
`97
`
`NO
`
`ES 98
`
`joo?
`
`send user input to device
`
`99
`
`FIG. 9
`
`

`

`U.S. Patent
`
`Aug. 18, 2009
`
`Sheet 10 of 10
`
`US 7,577,910 B1
`
`<?xml version="1.0"?>
`<!DOCTYPE WML SYSTEM ‘*wml.dtd">
`<WML>
`<TEMPLATE>
`<DO TYPE="HELP" LABEL="help"> <GO URL="#help"/> </DO>
`</TEMPLATE>
`<CARD NAME="General settings" NEWCONTEXT="TRUE">
`<BR ALIGN="CENTER*/>
`Welcome to
`<BR ALIGN="CENTER®*/>
`<BIG><B>SONY CD-Player 990X</B></BIG>
`axe
`<BR/> <BR/> <BR/>
`Please select the function you want to configure:<BR/>
`<A TITLE=“set/edit CD labels*>CD labels<Go
`URL=*#CD_names*/></A>
`
`<BR/>
`<A TITLE="select play mode">Play mode<GO
`URL="#play_mode*/></A>
`
`</CARD>
`<CARD NAME=*CD_names°*>
`<BIG><BoCD names:</B></BIG>
`<BR/>
`Enter a name for the currently inserted CD:
`<BR/>
`<INPUT TYPE=*TEXT* KEY="CD_NAME* FORMAT="MMMMMM"
`EMPTYOK=°TRUE* />
`<DO TYPE="*ACCEPT” LABEL2="ok*>
`<GO
`
`URL="bluetooth://sony_cdp_990/cd_titles/add;$({CD_NAME) */>
`</DO>
`</CARD>
`<CARD NAME=“play_mode*>
`<BIG><B>Play-modes: </B></BIiG>
`<BR/>
`Select one of the following play modes:
`<BR/>
`<SELECT TITLE="play-modes:* KEY=*PLAYMODE"
`DEFAULT="Normal *>
`<OPTION VALUE="Normal* TITLE="Normal"/>
`<OPTION VALUE="Random* TITLE=*Random*/>
`<OPTION VALUE="Shuffle* TITLE="Shuffle*/>
`<OPTION VALUE="UserDef* TITLE="User-Defined’ />
`</SELECT>
`<DO TYPE="“ACCEPT" LABEL="*ok*>
`<GO
`URL="bluetooth: //sony_cdp_990/playmode/select;§ (PLAYMODE) "/>
`</DO>
`</CARD>
`<CARD NAME="*help*>
`Some help text here...
`</CARD>
`
`</WML>
`
`FIG. 10
`
`

`

`US 7,577,910 B1
`
`1
`METHOD AND APPARATUS FOR
`PROVIDING A MORE POWERFUL
`USER-INTERFACE TO DEVICE WITHA
`LIMITED USER-INTERFACE
`
`TECHNICAL FIELD
`
`The present invention relates to computer systems which
`have a limited user-interface, such as hand-held pervasive
`computing devices, and in particular to a design for allowing
`easy interaction with such computer systems.
`
`BACKGROUND OF THE INVENTION
`
`ay
`
`0
`
`2
`cation feature (such as the Casio PC Unite Data Bank Watch,
`HBX-100B-1) used to connect to a PC.
`There are many other examples of user-interlaces that are
`severely lacking for various reasons, the most prominent of
`which are size and cost constraints. Often suchuser-interface
`restrictions make the respective devices less useful for their
`owners than they could be.
`It would thus be useful to have a way to unleash the full
`potential of all these devices and to program and configure
`them much more conveniently, thereby making them more
`useful to their users.
`There is growing demandin the industry to offer devices
`that are ‘open’ in the sense that a user has access via an
`interface to the device’s processor or other components, An
`ideal ‘open’ device would be fully controlled by the user,
`preferably within well-defined rules to prevent misuse or
`destructionof the deviceitself.
`
`In addition, there are a growing numberof devices that are
`network enabled, which means that they can communicate
`with one or more other devices via a network. This can be
`achieved using physical connections, suchas cables orfibers,
`for example. As these devices get smaller, however,
`it
`becomes desirable to replace the physical connections with
`wireless connections (e.g. body networks, radio frequency
`connections, or infrared connections), since physically con-
`necting the devices by means of cables or fibers severely
`reduces the efficiency gained by making the units smaller.
`Ad-hoc wireless connections are required where devices
`movearound,enter an area and exit the area, The term ad-hoc
`refers to the need for frequent network reorganization.
`In addition, there are many different known communica-
`tions protocols or standards that have been developed and
`designed (and continue to be developed and designed)
`directed at communication between devices or subsystems.
`Hereinbelow, some wireless communications protocols or
`standards will be mentioned. There are many fiber or cable-
`based, standardized approaches that are suited for such com-
`munication as well.
`
`GTE Corporation has developed a short-range radio-fre-
`quency (RF) technique which is aimed at giving mobile
`devices such as cellular phones, pagers and hand-held per-
`sonal computers (PCs) a smart way to interact with one
`another. GTE’s technique is tentatively named Body LAN
`(local area network). The original development of Body LAN
`was via a wired vest with which various devices were con-
`
`nected (hence the name Body LAN). This then developed into
`to an RF connection.
`Xerox Corporation has developed a hand-held computing
`device called PARC TAB. The PARC TABis portable yet
`connected to the office workstation through base stations
`which have known locations. The PARC TAB basestations
`are placed around the building, and wired into a fixed wired
`network. The PARC TAB system uses a preset knowledge of
`the building layout and the identifiers of the various base
`stations to determine the strongest base station signal for a
`PARC TAB portable device.A PARC TAB portable device
`has a wireless interface to the base stations. The PARC TAB
`
`systemassumes that the PARC TAB portable device is always
`connectedto the networkinfrastructure. The location of each
`
`portable PARC TAB device is always known to the system
`software. The base stations establish regions and are con-
`nected to power supplies. PARC TAB communication sys-
`tems havea star topology.
`In an attempt to standardize data communication between
`disparate PC devices, several companies, including Ericsson,
`IBM, Intel, Nokia, and Toshiba have established a consortium
`to create a global standard for wireless RF-based connectivity
`
`bh a
`
`La
`
`4
`
`45
`
`Through relatively recent technological innovations, com-
`puter systems have become dramatically smaller and more
`portable. Even very powerful personal computers (PCs), for
`example, are small enoughto sit on the desk at work. Smaller
`still are lap top computers and notebook computers. There are
`J
`computer terminals which are small enough to be mounted in 2
`avehicle, such as a delivery truck. Still smaller are hand held
`terminals, which are typically used for their portability fea-
`tures, allowing a user to carry the terminal in one hand and
`operate it with the other.
`In addition, there is a trend toward offering consumers
`electronic devicesthat include some sort of computer system,
`e.g., a microprocessor. Usually, these computer systems not
`only controlthe operation or function of the consumerdevice,
`but also provide someinterface for a user or operator to
`control certain functions or parameters according to actual
`needs. It is in the nature of these consumerdevices that they
`do not have a full user-interface like a computer withdisplay
`and keyboard. It is not likely that a dishwasher, for example,
`will ever have such a full user-interface. In some cases the
`interface is limited due to space constraints(a typical example
`for this is a wrist watch), while in other cases the interface is
`limited to keep the cost of manufacturing low, while still in
`other cases the processing power of the computer system, or
`the constrained memory space,limits the interaction between
`the user and system.
`Manyof today’s devices have an inadequateuser-interface.
`A typical example is a compact disk (CD) player which
`allows programming ofCD titles using a small four button-
`control. Programming of such a CD playeris very cumber-
`some because one needs to use the buttons to move through
`the entire alphabet to select letters and/or numbers. Another
`example is a wrist watch that allows the user to enter phone
`book entries, appointments, and to-do items. Typically, the
`keyboard includes a very limited number of keys. Further-
`more, the display is small and its resolution limited. Certain s
`keys have to be pressed several times to reach special char-
`acters, or to activate special functions. Yet another example is
`a personal digital assistant (PDA) with a touch sensitive
`screen. In this case the screen occupies most of the device’s
`surface and there are very fewbuttons, if any. Some functions 5
`are easily accessible using a pointing device, but other func-
`tions have to be selected or activated flipping through several
`layers of menus, for example. Other examplesare telephones,
`vending machines, microwave ovens, mobile phones, etc. For
`the purposes of the present description these devices are
`referred to hereinafter as user-interface limited devices.
`Currently there are a few approaches using a personal
`computer (PC) to run better user-interfaces, e.g. the “Nokia
`Cellular Data Suite” for mobile phones allows the entry of
`phone book data. The Cellular Data Suite is a hardware and
`software package from Nokia designed for cellular phones.
`Another example is a wrist-watch that has an IR-communi-
`
`60
`
`65
`
`

`

`US 7,577,910 B1
`
`ay
`
`3
`betweenfixed, portable and mobile devices. There are many
`other companies adopting the proposed standard. The pro-
`posed standard is called Bluetooth and comprises architecture
`and protocol specifications ranging from the physical layer up
`to the applicationlayer. The Bluetooth standard contemplates
`allowing users to connect a wide range ofdevices easily and
`quickly, without the need for cables, expanding communica-
`tions capabilities for mobile computers, mobile phones and
`other mobile devices. The Bluetooth operating environment
`is not yet fully defined, but similarities are expected with the
`IrDA (Infrared Data Association) specification and the
`Advanced Infrared (Alr) specification. It is not unreasonable
`to expect that the Bluetooth standard will eventually incorpo-
`rate aspects of the IEFE standard 802.11 and/or HIPERLAN,
`as promulgated by the European Telecommunications Stan-
`dards Institute (ETSI).
`Bluetooth radio technology provides a standard protocol
`suitable for forming small private ad-hoc groupings of con-
`nected devices away from fixed network infrastructures.
`J
`Bluetooth makesa distinction between a master unit—which 2
`is a device whose clock and hopping sequence are used to
`synchronize all other devices—andslave units in the same
`network segment. In other words, the Bluetooth approachis
`centralized. A query-based discovery design is used for find-
`ing Bluetooth devices with an unknownaddress. Queries are
`also centralized at a registry server. Further details can be
`found in Haartsen, Allen, Inouye, Joeressen, and Naghshineh,
`“Bluetooth: Vision, Goals, and Architecture” in the Mobile
`Computing and Communications Review, Vol. 1, No. 2.
`Mobile Computing and Communications Review is a publi-
`cation of the ACM SIGMOBILE.
`
`30
`
`HomeRF (based on Shared Wireless Access Protocol
`(SWAP))is another example of a prospective operating envi-
`ronment protocol which can be used to connect devices. A
`HomeRF Working Group was formed to provide the founda-
`tion for a broad range of interoperable consumer devices by
`establishing an open industry specification for wireless digi-
`tal communication between PCs and consumer electronic
`
`devices anywhere in and around the home. The working
`group, which includes the leading companies fromthe per-
`sonal computer, consumerelectronics, peripherals, commu-
`nications, software, and semiconductor industries,
`is cur-
`rently
`developing
`a
`specification
`for
`wireless
`communications in the homecalled the SWAP. The HomeRF
`SWAPsystemis designed to carry both voice and data traffic
`and to interoperate with the Public Switched Telephone Net-
`work (PSTN)and the Internet. It operates in the 2400 MHz
`band and uses a digital frequency hopping spread spectrum
`radio. The SWAPtechnology protocol being developed is
`being derived to some extent from extensions of existing
`cordless telephone (DECT) and wireless LAN technology to
`enable a new class of home cordless services. It envisions
`
`supporting both a time division multiple access (TDMA)
`service to provide delivery of interactive voice and other
`time-critical services, and a carrier sense multiple access/
`collision avoidance (CSMA/CA)service for delivery of high
`speed packet data. The SWAP system is contemplated to
`operate either as an ad-hoc network or as a managed network
`underthe control ofa connection point. In an ad-hoc network,
`where only data communicationis supported, all stations will
`be equal and control of the network will be distributed
`betweenstations. For time critical communications such as
`interactive voice, the connection point—whichprovides the
`gateway to the PSTN—will be required to coordinate the
`system. Stationswill use the CSMA/CAto communicate with
`a connection point and otherstations. Further details about
`HomeRFcan be found at the Home Radio Frequency Work-
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`ing Group’s web site http://www.homerf.org. The SWAP
`specification 1.0 is incorporated by reference inits entirety.
`There are several other knownprotocols and techniques
`that allow communication between two or more devices. The
`above described Bluetooth radio technology and HomeRF
`approachare prominent wireless examples.
`
`SUMMARYOF THE INVENTION
`
`It is therefore an object of the present inventionto provide
`a method and apparatus for providing a more powerful user-
`interface to a device with a limited user-interface.
`It is another object of the present invention to provide a
`method and apparatus for simplified and improveduserinter-
`action with a device with a limited user-interface, also
`referred to herein as an “interface limited device.”
`To achieve the above objects, a method and apparatus in
`accordance with the present invention includes a standard
`wireless communications protocol which allows a user to
`interact with or control aninterface limited device wirelessly
`using a second (independent) device.
`In other words, the present invention provides a more pow-
`erful user-interface to an interface limited device by interfac-
`ing it with a more powerful device in its vicinity. The more
`extensive input capabilities of the more powerful device (sec-
`ond device) are employed to control certain aspects of the
`limited user-interface device(first device).
`invention
`The method in accordance with the present
`includes the steps oftransmitting user-interface information
`from the first device to the second device; providing a user-
`interface at the second device corresponding to the user-
`interface information; receiving user input via the user-inter-
`face at
`the second device;
`transmitting user command
`information correspondingto the user input fromthe second
`devicetothe first; and executing the corresponding user com-
`mandsat the first device.
`
`DESCRIPTION OF THE DRAWINGS
`
`The above and other objects, features and advantages ofthe
`present invention will become more apparentin light of the
`following detailed description of an exemplary embodiment
`thereof taken in conjunction with the attached drawings in
`which:
`
`FIG, Lisa block diagramillustrating a typical environment
`whereinthe present inventionis utilized, in accordance with
`an embodimentof the present invention:
`FIG, 2 is anillustrative representation of an exemplary
`execution tree in accordance with an embodiment ofthe
`present invention;
`FIG, 3 illustrates a first user-interface windowin accor-
`dance with an embodimentofthe present invention;
`FIG, 4 illustrates a second user-interface windowin accor-
`dance with an embodimentof the present invention;
`FIG. 5 illustrates a third user-interface windowin accor-
`
`dance with an embodimentofthe present invention;
`FIG. 6 is a block diagram illustrating another embodiment
`of a typical environment in which the present invention is
`utilized, in accordance with an embodiment ofthe present
`invention;
`FIG. 7A is a block diagramillustrating a first device (with
`a limited user-interface) in accordance with an embodiment
`of the present invention;
`FIG, 7B is a block diagramillustrating a second device
`with a more powerful user-interface) in accordance with an
`embodimentof the present invention;
`
`

`

`US 7,577,910 B1
`
`5
`FIG, 7C is a block diagramillustrating the hardware layer
`ofa second device in accordance with an embodimentof the
`
`present invention;
`FIG. 7D is a block diagramillustrating the hardware layer
`of a first device in accordance with an embodiment of the
`present invention;
`FIG. 8 is a flowchart illustrating the method ofthe present
`inventionina first device, in accordance with an embodiment
`ofthe present invention;
`FIG. 9 is flowchart illustrating the method of the present
`invention in a second device to control a first device in accor-
`
`ay
`
`0
`
`dance with an embodiment of the present invention; and
`FIG. 10 is a Wireless Markup Language (WML) document
`describing the user-interlaces of FIGS. 3-5.
`
`DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`For the purpose ofthe present description, a network can be
`anything that allowsa first device (the limited user-interface
`device) to communicate with a second device (which has a
`more powerful user-interface). A simple point-to-pointlink, a
`local area network (LAN), aGSM telephonelink, an ethernet
`link or any other kind oflink is hereinafter referred to as
`network. This network can either be a physical network or a
`wireless network (e.g., infrared (IR), radio-frequency (RF),
`such as HomeRF), The network may be completely isolated
`from any other network, or it might comprise one or more
`access points which provide the devices with access to
`another network.
`
`ba
`
`La
`
`6
`addressing design is also required in a GSM-based imple-
`mentation of the present invention.
`It is understood by those skilled in the art that at the present
`time manyofthe protocols thatare suited for use in wireless
`communications systemsarestill in draft status. The present
`design is independent of any one particular protocol and can
`be used in connection with many such protocols. Accord-
`ingly, someone having ordinary skill in the art is able to
`implement the present design in existing protocol environ-
`ments as well as in protocol environments under development
`or yet to be developed.
`Thepresent design can be used anywhereinside,i.e. ware-
`houses, on manufacturing floors, in offices, on trading floors,
`in private homes, and outside ofbuildings, in cars and trucks,
`in airplanes, just to mention a few examples.
`‘Two devices can be connected using a 1:1 connection.
`Possible media are infrared and magnetic fields. The proce-
`dure to setup such a 1:1 connection can be similar to today’s
`setup of a connection between two IrDA enabled devices, i.e.
`the devices must be positionedsuchthat their communication
`subsystems (transceivers) can “see” each other. Then both
`systemsare triggered to start a connection setup procedure
`until a wireless communication channelis established.
`
`Likewise, two devices can be connected using a shared
`medium. A possible shared medium is RF (Radio Frequency).
`Possible systems could be based on technology and protocols
`like BlueTooth, DECT, and HummingBird.
`Details about HummingBird transceivers are given in
`“Hummingbird Spread Spectrum Transceiver Operator's
`Manual”, Rev. 24 Jun. 1998, XETRON Corp., Cincinnati,
`Ohio, USA
`Details concerning the basic problems regarding identifi-
`cation and addressing,initial (resource) discovery, matching
`and selection of communication partners, etc. depend on the
`medium used and the communications protocol employed.
`Whenreferring to a device, any kind ofdevice is meantthat
`can establish a network connection to another device.
`Examples of devices are: laptop computers, workpads, node-
`pads, personal digital assistants (PDAs), notebook computers
`and other wearable computers, desktop computers, computer
`terminals, networked computers, internet terminals and other
`computing systems, set-top boxes, cash registers, bar code
`scanners, point of sales terminals, kiosk systems, cellular
`phones, pagers, wrist watches, digital watches, badges, and
`smart cards. Other contemplated devices include: headsets,
`Human Interface Device (HID) compliant peripherals, data
`and voice access points, cameras, printers,
`fax machines,
`keyboards, joysticks, HiFi systems, audio (sound) cards,
`loudspeakers, amplifiers, video cards, kitchen appliances,
`tools, sensors such as smoke and/orfire detectors, and virtu-
`ally any other digital device.
`Other examples of devices that can be used in connection
`with the present
`invention are, personal effects being
`equipped with computer-like hardware, such as a “smart wal-
`let” computer, jewelry, or articles of clothing. In addition to a
`“smart wallet” computer, there are a numberof other varia-
`tions of wearable computers. A “belt” computer is such a
`variation whichallows the user to surf, dictate, and edit docu-
`ments while they are moving around. Yet another example is
`achild’s computer which is comparable to a personal digital
`assistant for grade-school children. The child’s computer
`might hold assignments, perform calculations, and help kids
`manage their homework.It can interface with other children’s
`computers to facilitate collaboration, and it can access a
`teacher’s computer to download assignments or feedback.
`Any wearable or portable device, any office tool or equip-
`ment, hometool or equipment, systemfor use in vehicles, or
`
`The specific range that constitutes a wireless network in
`accordance with the present invention depends on actual
`implementation details. Generally, a wireless network can be
`described as having a coverage area between a few square
`meters and several thousands of square kilometers (e.g., in
`case of a GSM network). Under certain circumstances the
`
`communication range may go even further. The two
`devices—which communicate with each other haveto be “in
`vicinity,” which meansthat they have to be sufficiently close
`or otherwise interconnected in order to be able to exchange
`information with each other.
`The devices need to be able to transmit and/or receive
`information via the network. Accordingly, two devices that
`communicate with each other must support the same commu-
`nication protocol,
`Well suited to support communication between devicesis
`the Bluetooth communications design, whichis described in
`the Haartsen, Allen, Inouye, Joeressen, and Naghshineh,
`“Bluetooth: Vision, Goals, and Architecture” in the Mobile ;
`Computing and Communications Review, Vol.
`1, No. 2.
`Mobile Computing and Communications Review is a publi-
`cation of theACM SIGMOBILE. This reference is incorpo-
`rated herein by referenceinits entirety.
`It is assumed, that once the devices are in vicinity of each s
`other, a wireless communication path between these devices
`canbe established—e.g. using magnetic field (near field/5-30
`em), infrared (IR), e.g., IrDA (0.5-2 m) or Alr (1-10 m), or
`low-powerradio-frequency (RF) communication, e.g. Blue-
`‘Tooth (“1-10 m), or HomeRF (“1-50 m), just to list some
`examples of wireless schemesthat are suited.
`Of course such a vicinity requirement may also be accom-
`plished via some global addressing design and a local proxy
`(e.g. an IR-beacon oneach ceiling or a BlueTooth “relay”in
`each room or home) connected to a network (e.g. the Inter-
`net), thus effectively allowing remote control of a variety of
`devices from virtually anywhere in the world. Sucha global
`
`4
`
`45
`
`60
`
`65
`
`

`

`US 7,577,910 B1
`
`7
`systems for use in the public (i.e. vending machines, ticketing
`machines, automated teller machines, etc.) may include the
`present invention.
`It is also assumed that a device, as used in connection with
`the present invention, includes a minimum amount of pro-
`cessing powertoenable it to participate in the design accord-
`ing to the present invention. These devices are thus also
`referred to as computer devices. Most, if not all, of the above
`listed devices may be viewed as being devices with limited
`user-interfaces. This may even be the case for a personal
`computer which has a display and a keyboard. Thereis still
`roomfor improvement in such a computer’s interface, e.g., by
`adding speechinput. There are no absolute criteria which can
`be used to decide whether a particular device is a device with
`a limited user-interface or not. There is always room for
`improvement and thus any computer deviceperse is assumed
`to bea device with a limited user-interface. The present inven-
`tion may be implemented where there is a second computer
`device that has a more powerful user-interface, more
`adequate, more convenient, or superior user-interface capa-
`bilities. Not all aspects ofthe user-interface have to be supe-
`rior or more powerful, It is sufficient, for example,ifthere is
`a first device which has no speech input(i.e. it has a limited
`user-interface) and a second device whichhas a speech input.
`The present invention may also be implemented, as selected
`by a user, to control the second computerdevice that has a
`more powerful user-interface via the device with the limited
`user-interface, if so desired as a users convenience dictates.
`Some of the above-mentioned devices can be regarded
`hereinafter as the device (controller) whoseinterfaceis used
`to interact with the user-interface limited device (controlled
`device).
`A computer device is generally referred to as a computer
`device witha limited user-interface where, for example, one
`or more ofthe following applies: the user-interface is inad-
`equate for the tasks required; the user-interface is small and
`difficult to read, understand, or hear; the user-interface pre-
`sents an inconvenience to the user;
`there is no graphics
`capable display (e.g. a text-only display); there is a restricted
`numberofinput keys, or input keys which are too small; there
`are too many functions which are mappedtoa limited number
`of buttons and thus imposes complicated control structures
`that makeit difficult to operate the device without prior exten-
`sive study ofa user’s manual, especially for seldom used or
`advanced functions;
`the user-interface is not powerful
`enough, making its use to slow, or has lowresolution, or the
`like.
`Devices with a more powerful user-interface capabilities
`generally fulfill, for example, one or more ofthe following
`criteria: there is a larger screen; there is a screen with graphics
`capability; there is a full keyboard; there is a pointing device:
`there is a voice-input feature and soforth.
`The user-interface may be any kind of interface used for
`interaction betweena user and the device, such as a display,
`keyboard, mouse, track point, audio output, speech recogni-
`tion input, tactile input, ete.
`A typical environment where the present inventionis used
`is illustrated in FIG, 1. There is a first computer device 10
`whichhas a limited user-interface 11 (in the present example
`the user-interface comprises a simple display and a fewbut-
`tons). Whenthere is a second computer device 12 invicinity
`of the first device 10 that has superior user-interface capabili-
`ties (keyboard 13 and display 14) than thatof the first device
`10 and which may be controlled or configured, and assuming
`that the two devices 10 and 12 communicate with eachother,
`the superior user-interface capabilities of the second device
`12 may be employed to facilitate and speed-up the use of the
`
`ay
`
`J
`2
`
`30
`
`4
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`first device 10 by transmitting a more powerful user-interface
`(user-interface description 15) storedinthefirst device 10 to
`the second device 12. Thefirst device 10 and second device 12
`communicate via a wireless communications channel 16. A
`more powerful user-interface is a user-interfacethat is easier
`to use,i.e. “richer” (in that it has more features), more intui-
`tive, faster, or the like. The user-interface description 15 is
`then processed by the second device 12 such that the more
`powerful user-interface 19 is then displayed and operated via
`the second device 12. Then user-inputs and/or commands
`and/or parameters are sent back to the first device for execu-
`tion. In the present example, commands 17 (<command>)
`and parameters 18 (<parameters>) are