|||||||||||||||||||||||||||||||||l||l||||||||||||||||||||||||||||||||||||||
`
`USOOTST'7910BI
`
`(12) United States Patent
`Husemann et at.
`
`(in) Patent No.:
`
`(45} Date of Patent:
`
`US 7,577,910 Bl
`Aug. 18,2009
`
`(54) METHOD AND APPARATUS FOR
`PROVIDINGA MORE POWERFUL
`USER-INTERFACE TO DEVICE WITH A
`LIMITED USER-INTERFACE
`
`(75}
`
`Inventors: Dirk Husemann.Adliswill (CH);
`Michael Maser. Zurich [Cl-l)
`
`(73) Assignee:
`
`International Business Machines
`Corporation, Armonk. NY (US)
`
`6.7192605 Bl "‘
`
`912004 Roberts ctal.
`
`T193313
`
`FOREIGN PA'l‘IiN'I‘ DOCUMENTS
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`W0
`W0
`
`[ET-254920
`10- 162060
`10-240333
`| 1 4127290
`11-031114
`11-161321
`2002-5096691
`W0 95.328613
`W0 9859284
`
`10.91995
`6-"1998
`9-" 1998
`|.-" 1999
`2-1999
`6-"1999
`3.32002
`10-"1995
`1231998
`
`( ‘9 ) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adiusted under 35
`U.S.C. 154(b) by 1343 days.
`
`(21) Appl.No.: 091613JI3
`
`(22)
`
`Filed:
`
`Jul. 10. 2000
`
`(30)
`
`Foreign Application Priority Data
`
`OTHER PUBLICATIONS
`
`European Search Report. Application No. [EP 99 l l
`2000.
`Jaap Hamisen. “Bluetooth—The universal radio interface for ad hoc.
`wireless connectivity". Fricsson Review No. 3, 1998.
`Christcr Iii'landson and Per Ucklind. “WA? The wireless applica-
`tion protocol". Ei’icsson Review No. 4. 1998.
`
`1 3414. Mar. 21.
`
`Jul. 12. 1999
`
`(EP)
`
`99113414
`
`* cited by examiner
`
`(51)
`
`Int. (:1.
`(£061? 3/00
`[52) U.S. (fl.
`
`(2006.01)
`715N461; 715.9745: 715K746;
`715F147
`
`343716.
`(58) Field ofClassifieation Search
`3451’719. 720. 721. T23. 730; 715f744—i’49
`See application file for complete search history.
`
`(56}
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`6.173.316 Bl“
`6.216.158 151*
`6.446.096 Bl “‘
`6.456.892 B1 ‘1‘
`6.466.971 131*
`6.502.000 Bl "‘
`6.509.913 32"
`6.560.640 132*
`
`1.5200} De Boorel a1.
`452001 Luo et a1.
`952002 Holland etal.
`932002 Dam-Abrmns et al.
`1032002 lIumpletnnn eta].
`12-"2002 Arnold et a1.
`152003 Martin et a1.
`552003 Smethcrs
`
`T093213
`7095217
`'i‘lfis'filit
`700583
`TUE-220
`"£00383
`3453762
`709-"219
`
`-Ba l-Iuynh
`Primary Examiner-
`(74) Attorney. Agent. or Firm—Brian 1“. Vem'iinslii. Esq.
`
`[57)
`
`ABSTRACT
`
`A method and apparatus for controlling a computer device
`with a limited user-interihce via a remote computer device
`having a more powerfill user interface. Both computer
`devices are intercomiected via a wireless communication
`channel and both computer devices support a common com-
`munications protocol. U tier-interface information is sent from
`the computer device with a limited user-interface to the
`remote computer device to provide a user-interface at the
`remote computer device for receiving user input at the remote
`computer device. The user input controls the computer device
`with a limited user-interface to execute conunands therein
`
`corresponding to the user input.
`
`26 Claims, 10 Drawing Sheets
`
`r0215
`
`user requests [Lin acclaim
`
`iiiiptly I'm «00th devices
`
`mid request In
`
`
`
`
`
`
`
`
`
`
`render Ut-dm'pliun
`(u. display or m to user)
`
`
`
`
`and user input to device
`
`UNIFIED 1006
`
`UNIFIED 1006
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 1 0f10
`
`US 7,577,910 B1
`
`
`
`<command>:<parame£ers>
`
` L—,—__/
`
`15
`
`
`
`brought
`inlo vicinity
`
`
`
`J
`_ _-._
`Pdafaull-fonlsl
`
`|__._._
`Fdraii. _._-:; 1.11”:
`
`l
`|
`.
`._EV_S!Y.I.6..
`.J
`I—I
`;I75ize
`I
`r—Vl‘"q_'1¥r—I_
`r———[
`ngtyle
`‘
`1
`l—l
`Vsize.
`.J
`.. ..
`.
`
`Fmriace
`[
`C_._.]
`I
`pseria1 -
`l
`i
`[——I
`PEaud-ratel
`r—l
`'Pparily
`.
`,l
`
`I
`_
`..
`.
`I
`Plormats
`|
`
`l
`PnarausL.
`_
`1
`[
`...._.- ...._._.{‘;..-......._.
`I
`.
`DJCL
`[—j
`
`I
`
`pPostScn'pt
`
`H.
`
`
`
`.
`
`-
`
`
`
`I
`
`I
`
`
`
`...........____
`
`
`
`
`FIG. 1
`
`FIG. 2
`
`
`
`
`
`
`
`
`
`
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 2 of 10
`
`US 7,577,910 B1
`
`EIWJL Brgwser- file.:EJJavaDeWclassesthEle
`
`Welcome to
`Bengal settings 1
`SONY CID-Player 990x
`Please select the function you want to I
`
`Play mode
`
`configure:
`
`CD labels
`
`FIG. 3
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 3 of 10
`
`US 7,577,910 B1
`
`«pH—n.- “n.” ,_
`Lam; firowsé;-file'iEIJavaDemrclassesfleglflm
`[h'flsonLcdp__990)Umain_menu .wml#CD__names
`
`i—l
`
`CD names:
`
`General settings}
`Enler a name for the currently insened i—
`
`1E
`
`:
`
`FIG. 4
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 4 of 10
`
`US 7,577,910 B1
`
`' C: User-Defined
`
`C's Random
`
`0 Shuttle
`
`FIG. 5
`
`E3§ML1_I_.___Browser-fieIJavaDeWclassesfleEEfEIE
`é:JPlayhmodes:
`
`{ E
`
`Selecl one ofthe following play
`modes:
`3 rpIayLmodes:
`G) Normal
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 5 of 10
`
`US 7,577,910 B1
`
`
`
`54
`
`NH
`chap-muss»
`
`6
`
`n
`
`E
`
`"7"}.
`
`hr—wa—F-ya—u—l-‘r-‘r-Kr‘fi
`v—u v—HI—n—qHHHI—‘r‘fi
`f-H—H—Imc—nr—ur—w—u—fl
`!—'I C: F—‘r-——u-'-u'—\r—n=;
`
`13
`
`60
`
`\. _._—.______._,
`66
`
`
`
`
` «ecommandxcparamolars :
`
` brought
`into vicinity
`
`
`FIG. 6
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 6 of 10
`
`US 7,577,910 B1
`
`pro locol
`
`device specific
`
`user interface
`
`manager
`
`
` 70
`
`UserInterface3.
`HWInterface
`memory
`rcccwer
`
`M .—
`
`FIG. 70
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 7 of 10
`
`US 7,577,910 B1
`
`M AC
`
`protocol
`
`transmitter
`
`driver
`
`receiver
`
`driver
`
`
`
`user interface
`
`manager
`
`user interface driver
`
`70°
`
`FIG. 7B
`
`700
`
`Lu
`
`n. FIG. 7C
`
`0 2
`
`‘.u
`
`uoI
`
`lransrmuer
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 8 of 10
`
`US 7,577,910 B1
`
`
`
`
`“Pa". 04103:! '3
`
`send scrvzcc
`
`
`
`84
`
`infonnation
`
`request to
`
`send UI
`
`86
`
`
`
`send feedback
`
`FIG. 8
`
`
`
`
`Z handlclcxccutc
`
`3 user input
`........
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 9 of 10
`
`US 7,577,910 B1
`
`I02 13
`'.
`
`receive service
`information
`
`103
`
`backgreend
`
`user requests (list of) devices
`
`diSplay list of controllable devices
`
`94
`
`
`
`
`send request to send {ll-description
`
`NO
`
`95
`
`, JAI’WI
`UI Infimm'm.
`
`receive UI~description
`
`render UI-descdplion
`(e. 3. display or read to user)
`
`97
`
`ES
`
`
`
`“’0 j
`
`98
`
`99
`
`FIG. 9
`
`

`

`US. Patent
`
`Aug. 18, 2009
`
`Sheet 10 0f10
`
`US 7,577,910 B1
`
`<?xm1 version='1.0'?>
`<!DOCTYPE NHL SYSTEM 'wm1.dtd'>
`(NHL)
`<TEHPLATE>
`
`<00 T?PE='HELP' LABEL='he1p'> <60 URL='#help'/> <fno>
`<fTEHPLATE>
`<CARD NAHE='Genera1 settings' NEWCONTEXT='TRUE'>
`<BR ALIGN:'CENTER'I>
`Welcome to
`<BR ALIGN='CENTER'I>
`<BIG><B>SONY CD-Player 990X<IB><IBIG>
`._-
`<BR/> <BRI> <BRI>
`Please select the function you want to configure:<BR/>
`<A TITLE='set/edit CD 1abe15'>CD labels<GO
`URL=JICD_names'/)<!A>
`
`<BRI:
`
`<A TITLE='select play mode'>Play mode<GO
`URL='lplay_mode‘/></A>
`
`<ICARD;
`<CARD NAHE='CD_names°>
`<3I6><B>CD nomes:<lB></BIG>
`(BR!)
`
`Enter a name for the currently inserted CD:
`<8RI>
`
`<INPUT TYPE='TEKT' KEY='CD_NAME' FORMATa’MMMMMM'
`EHPTYOK='TRUE'I>
`<00 TYPE=‘ACCEPT' LABELa'ok'>
`€60
`
`URLs'bluetooth://sony_cdp_990lcd_titlee/add;S{CD_NAME)"(a
`<IDO>
`<ICARD>
`<CARD NAHE:‘p1ay_mode‘>
`<BIG><B>P1ay—modes:<IB><IBIG>
`<3Rl>
`
`Select one of the following play modes:
`<BRI>
`
`<SELECT TITLE='p1ay-modes:' KEY='PLAYHODE'
`DEFAULTz'Norma1'>
`<OPTION VALUE='Norm41' TITLE='Normal'/>
`<OPTION VALUE='Handom’ TITLE=’Random‘/>
`cOPTION VALUE='Shuffle' TITLE='ShuEf1e'/>
`(OPTION VALUE='UserDef' TITLE='User-Defined'lb
`</SELECT>
`(DO TYPE='ACCEPT' LABEL='Ok'>
`{GO
`
`URL='b1uetooth:llsony_cdp_990/playmodefselect:SlPLAYHODE)‘/>
`<IDO>
`<ICARD)
`
`<CARD NAME=‘help'>
`Some help text here...
`</CARD>
`
`<IWHL)
`
`FIG. 10
`
`

`

`US ?,577,910 B]
`
`1
`METHOD AND APPARATUS FOR
`PROVIDING A MORE POWERFUL
`USER-INTERFAC E To DEVICE WITH A
`LIMITED USER-INTERFAC E
`
`'l‘l'iCI INICAI.. FIELD
`
`The present invention relates to computer systems which
`have a limited user-interface. such as hand-held pervasive
`computing devices. and in particularto a design for allowing
`easy interaction with such computer systems.
`
`1t)
`
`BACKGROUND OI" Tl 1E INVENTION
`
`Through relatively recent technological innovations, com—
`puter systems have become dramatically smaller and more
`portable. liven very powerful personal computers (PCs). for
`example. are small enough to sit on the desk at work. Smaller
`still are lap top computers and notebook computers. There are
`computer terminals which are small enough to be mounted in
`a vehicle. such as a delivery truck. Still smaller are hand held
`terminals. which are typically used for their portability fea—
`tures, allowing a user to carry the terminal in one hand and
`operate it with the other.
`In addition, there is a trend toward offering consumers
`electronic devices that include solue sort ol‘computer system.
`cg. a microprocessor. Usually. these computer systems not
`only control the operation or function of the consumer device.
`but also provide sortie interface for a user or operator to
`control certain functions or parameters according to actual
`needs. It is in the nature of these consumer devices that they
`do not have a full user-interface like a computer with display
`and keyboard. It is not likely that a dishwasher. for exatnple.
`will ever have such a full user-interface. In some cases the
`interface is limited due to space constraints (a typical example
`for this is a wrist watch), while in other cases the interface is
`limited to keep the cost of manufacturing low, while still in
`other cases the processing power of the computer system. or
`the constrained memory space. limits the interaction between
`the user and system.
`Many o f today’ 5 devices have an inadequate user-interface.
`A typical example is a compact disk (CD) player which
`allows programming of CD titles using a small four button—
`control. Programming of such a CD player is very cumber-
`some because one needs to use the buttons to move through
`the entire alphabet to select letters andtor numbers. Another
`example is a wrist watch that allows the user to enter phone
`book entries. appointments. and to-do items. Typically. the
`keyboard includes a very limited number of keys. Further—
`more. the display is small and its resolution limited. Certain
`keys have to be pressed several times to reach special char-
`acters. or to activate special functionsYet another example is
`a personal digital assistant [PDA} with a totteh sensitive
`screen. in this case the screen occupies most of the device‘s
`surface and there are very lew buttons. ifany. Some functions
`are easily accessible using a pointing device. but other func~
`tions have to be selected or activated flipping through several
`layers of menus. for example. Other examples are telephones.
`vending machines, microwave ovens. mobile phones, etc. For
`the purposes of the present description these devices are
`referred to hereinafter as user-interface limited devices.
`
`Currently there are a few approaches using a personal
`computer (PC) to run better uservinterfaces. e.g. the “Nokia
`Cellular Data Suite” for mobile phones allows the entry of
`phone book data. The Cellular Data Suite is a hardware and
`software package from Nokia designed for cellular phones.
`Another example is a wrist-watch that has an lR-commurti-
`
`3t]
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`cation feature (such as the Casio PC Unite Data Bank Watch,
`l-lBX-IOUB- 1} used to connect to a PC.
`There are many other examples of user-interfaces that are
`severely lacking for various reasons. the most prominent of
`which are size and cost constraints. Often such user-interface
`restrictions make the respective devices less useful for their
`owners than they could be.
`It would thus be useful to have a way to unleash the full
`potential of all these devices and to program and configure
`them much more conveniently. thereby making them more
`useful to their users.
`
`There is growing demand in the industry to offer devices
`that are ‘open’ in the sense that a user has access via an
`interface to the device’s processor or other components. An
`ideal ‘open’ device would be fully controlled by the user.
`preferably within well-defined rules to prevent misuse or
`destruction of the device itself.
`
`in addition. there are a growing number of devices that are
`network enabled. which means that they can communicate
`with one or more other devices via a network. This can be
`achieved using physical connections, such as cables or fibers,
`for example. As these devices get smaller. however.
`it
`becomes desirable to replace the physical connections with
`wireless connections [e.g. body networks. radio frequency
`comiections. or infrared connections), since physically cort-
`necting the devices by means of cables or fibers severely
`reduces the efficiency gained by making the units smaller.
`Ad—hoc wireless connections are required where devices
`move around. enter an area and exit the area. The term ad—hoc
`refers to the need for frequent network reorganization.
`ln addition. there are many different known communica-
`tions protocols or standards that have been developed and
`designed (and continue to be developed and designed)
`directed at communication between devices or subsystems.
`Hereinbelow. sortie wireless communications protocols or
`standards will be mentioned. There are many fiber or cable—
`based. standardized approaches that are suited for such com—
`munication as well.
`
`GTE Corporation has developed a short-range radio-fre-
`quency (RF) tcclmique which is aimed at giving mobile
`devices such as cellular phones. pagers and hand-held per-
`sonal computers (PCs) a smart way to interact with one
`another. GTE’s technique is tentatively named Body LAN
`(local area network). The original development of Body LAN
`was via a wired vest with which various devices were con-
`
`nected {hence the name Body LAN }. This then developed into
`to an RF connection.
`
`Xerox Corporation has developed a hand-held computing
`device called PARC TAB. The PARC TAB is portable yet
`connected to the ofiice workstation through base stations
`which have known locations. The PARC TAB base stations
`are placed around the building. and wired into a fixed wired
`network. The PARC TAB system uses a preset knowledge of
`the building layout and the identifiers of the various base
`stations to determine the strongest base station signal for a
`PARC TAB portable device. A PARC TAB portable device
`has a wireless interface to the base stations. The PARC TAB
`
`system assumes that the PARC TAB portable device is always
`connected to the netvvork infrastructure. The location of each
`
`portable PARC. TAB device is always known to the system
`software. The base stations establish regions and are con-
`nected to power supplies. PARC TAB communication sys-
`tems have a star topology.
`In an attempt to standardize data communication between
`disparate PC devices. several companies. including Ericsson,
`IBM. Intel, Nokia. and Toshiba have established a consonium
`to create a global standard for wireless RF-based connectivity
`
`

`

`3
`
`4
`
`US ?,577,910 B]
`
`between fixed. portable and mobile devices. There are many
`other companies adopting the proposed standard. The pro-
`posed standard is called Bluetooth and comprises architecture
`and protocol specifications ranging from the physical layer up
`to the application layer. The Bluetooth standard contemplates
`allowing users to connect a wide range of devices easily and
`quickly. without the need for cables. expanding eonununica—
`tions capabilities for mobile computers. mobile phones and
`other mobile devices. The Bluetooth operating environment
`is not yet fully defined. but similarities are expected with the
`IrDA (Infrared Data Association) specification and the
`Advanced Infrared (Air) specification. It is not unreasonable
`to expect that the Bluetoolh standard will eventually incorpo-
`rate aspects of the IEEE standard 802.1 1 andfor HIPERLAN.
`as promulgated by the European Telecommunications Stan«
`dards Institute (ETSI).
`Bluetooth radio technology provides a standard protocol
`suitable for forming small private ad-hoc groupings ofcon-
`nected devices away from fixed network infrastructures.
`Bluetooth makes a distinction between a master unit- which
`is a device wltose clock and hopping sequence are used to
`synchronize all other devices—and slave units in the same
`network segment. In other words. the Bluetooth approach is
`centralized. A query-based discovery design is used for tind-
`ing Bluetooth devices with an unknown address. Queries are
`also centralized at a registry server. Further details can be
`found in liaartsen. Allen. Inouye. Joeressen. and Naghshineh.
`“Bluetooth: Vision, Goals, and Architecture” in the Mobile
`Computing and Communications Review, Vol.
`I. No. 2.
`Mobile Computing and Communications Review is a publi-
`cation of the ACM SIGMOBIIE.
`
`HorneRF (based on Shared Wireless Access Protocol
`(SWAPD is another example of a prospective operating envi-
`ronment protocol which can he used to connect devices. A
`HomeRF Working Group was formed to provide the founda—
`tion for a broad range of interoperable consumer devices by
`establishing an open industry specification for wireless digi-
`tal communication between PC‘s and consumer electronic
`
`devices anywhere in and around the home. The working
`group. which includes the leading companies from the per-
`sonal computer, consumer electronics. peripherals. commu-
`nications. software. and semiconductor industries.
`is cur—
`rently
`developing
`a
`specification
`for
`wireless
`communications in the home called the SWAP The HomeRF
`SWAP system is designed to carry both voice and data traffic
`and to interoperate with the Public Switched 'l'elephone Net-
`work (PS'IN) and the Internet. It operates in the 2400 MHz
`band and uses a digital Frequency hopping spread spectrum
`radio. The SWAP technology protocol being developed is
`being derived to some extent front extensions of existing
`cordless telephone (DECT) and wireless LAN technology to
`enable a new class of home cordless services. It envisions
`
`supporting both a time division multiple access (TDMA)
`service to provide delivery of interactive voice and other
`time-critical services, and a carrier sense multiple access!
`collision avoidance (CSMAKCA) service for delivery of high
`speed packet data. The SWAP system is contemplated to
`operate either as an ad—hoc network or as a managed network
`under the control ofa connection point. In an ad-hoc network.
`where only data communication is supported, all stations will
`be equal and control of the network will be distributed
`between stations. For time critical communications such as
`interactive voice. the connection point—which provides the
`gateway to the PSTN—will be required to coordinate the
`system. Stations will use the CSMA/CA to communicate with
`a connection point and other stations. Further details about
`HomeRl2 can be found at the Home Radio Frequency Work-
`
`10
`
`3t]
`
`4t]
`
`45
`
`50
`
`55
`
`60
`
`65
`
`ing Group‘s web site httpflwwwhomertiorg. The SWAP
`specification [.0 is incorporated by reference in its entirety.
`There are several other known protocols and techniques
`that allow communication between two or more devices. The
`
`above described Bluetooth radio technology and HomeRF
`approach are prominent wireless examples.
`
`SUMMARY OF THE INVENTION
`
`It is therefore an object of the present invention to provide
`a method and apparatus for providing a more powerful user-
`interface to a device with a limited user-interface.
`
`It is another object of the present invention to provide a
`method and apparatus for simplified and improved user inter—
`action with a device with a limited user-interface. also
`referred to herein as an “interface limited device.“
`To achieve the above objects, a method and apparatus in
`accordance with the present invention includes a standard
`wireless communications protocol which allows a user to
`interact with or control an interface limited device wirelessly
`using a second (independent) device.
`In other words. the present invention provides a more pow—
`erful user-interface to an interface limited device by interfac-
`ing it with a more powerful device in its vicinity. The more
`extensive input capabilities of the more powerful device (sec-
`ond device) are employed to control certain aspects of the
`limited user-interface device (first device).
`invention
`The method in accordance with the present
`includes the steps of transmitting user—interface information
`from the first device to the second device: providing a user-
`interface at the second device corresponding to the user-
`interface information; receiving user input via the user-inter-
`face at
`the second device;
`transmitting user command
`information corresponding to the user input from the second
`device to the first; and executing the corresponding user com-
`mands at the first device.
`
`DESCRIPTION OF THE DRAWINGS
`
`The above and other objects. features and advantages ofthe
`present invention will become more apparent in light ofthe
`following detailed description of an exemplary embodiment
`thereof taken in conjunction with the attached drawings in
`which:
`
`FIG. I is a block diagram illustrating atypical environment
`wherein the present invention is utilized. in accordance with
`an embodiment of the present invention;
`FIG. 2 is an illustrative representation of an exemplary
`execution tree in accordance with an embodiment of the
`present invention;
`FIG. 3 illustrates a first user—interface window in accor-
`
`dance with an embodiment of the present invention:
`FIG. 4 illustrates a second user-interface window in accor-
`dance with an embodiment of the present invention:
`FIG. 5 illustrates a third user-interface window in accor-
`
`dance with an embodiment of the present invention:
`FIG. 6 is a block diagram illustrating another embodiment
`of a typical environment in which the present invention is
`utilized. in accordance with an embodiment of the present
`invention:
`
`FIG. 7A is a block diagram illustrating a first device (with
`a limited user-interface) in accordance with an embodiment
`of the present invention;
`FIG. 7B is a block diagram illustrating a second device
`with a more powerful user-interface) in accordance with an
`embodiment of the present invention:
`
`

`

`5
`
`6
`
`US ?,577,910 Bl
`
`FIG. 7C is a block diagram illustrating the hardware layer
`of a second device in accordance with an embodiment of the
`
`present invention:
`FIG. 7D is a block diagram illustrating the hardware layer
`of a first device in accordance with an embodiment of the
`present invention;
`1"IG. 8 is a llowchan illustrating the method of the present
`invention in a first device. in accordance with an embodiment
`of the present invention:
`FIG. 9 is flowchart illustrating the method of the present
`invention in a second device to control a first device in accor-
`
`.3
`
`10
`
`dance with an embodiment of the present invention; and
`FIG. 10 is a Wireless Markup Language (WMI..) document
`describing the user-interfaces of FIGS. 3-5.
`
`DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`For the purpose ofthe present description. a network can be
`anything that allows a first device (the limited user—interface
`device) to communicate with a second device (which has a
`more powerful user-interface). A simple point-to-point link, a
`local area network (LAN). a GSM telephone link, an ethernet
`link or any other kind of link is hereinafter referred to as
`network. This network can either be a physical network or a
`wireless network (e.g._. infrared (IR). radio—frequency (RF).
`such as HotneRF]. The network may be completely isolated
`from any other network. or it might comprise one or more
`access points which provide the devices with acceSs to
`another network.
`
`The specific range that constitutes a wireless network in
`accordance with the present invention depends on actual
`implementation details. Generally. a wireless network can be
`described as having a coverage area between a few square
`meters and several thousands of square kilometers (e.g.. in
`case of a GSM network). Under certain circumstances the
`communication range may go even further. The two
`devices—which communicate with each other have to be “in
`
`vicinity.“ which means that they have to be sufficiently close
`or otherwise interconnected in order to be able to exchange
`information with each other.
`The devices need to be able to transmit and/”or receive
`infim‘nation via the network. Accordingly. two devices that
`commtmicate with each other must suppon the same commu-
`nication protocol.
`Well suited to support communication between devices is
`the Bluetooth communications design. which is described in
`the I-Iaartsen. Allen. Inouye. Joeressen. and Naghshineh.
`“Bluetooth: Vision Goals. and Architecture“ in the Mobile
`Computing and Communications Review. Vol.
`I. No. 2.
`Mobile Computing and Communications Review is a publi-
`cation of the ACM SIGMOBILE. This reference is incorpo-
`rated herein by reference in its entirety.
`[t is assumed. that once the devices are in vicinity of each
`other. a wireless conununication path between these devices
`can be established—cg. using magnetic field (near fieldf5—30
`cm}. infrared (IR). c.g.. Irl')A [0.5-2 rn) or Air (1-10 In). or
`low-power radio-frequency (RF) communication, e.g. Blue-
`Tooth (“l-10 m). or l-lomeRF (“l-50 m), just to list some
`examples of wireless schemes that are suited.
`Of course such a vicinity requirement may also be accom-
`plished via some global addressing design and a local proxy
`(e.g. an IR—beacon on each ceiling or a BlueTooth “relay” in
`each room or home) connected to a network (cg. the Inter—
`net). thus elfectively allowing remote control of a variety of
`devices from virtually anywhere in the world. Such a global
`
`3t]
`
`4t]
`
`45
`
`50
`
`55
`
`60
`
`65
`
`addressing design is also required in a GSM»based imple»
`mentation of the present invention.
`It is understood by those skilled in the art that at the present
`time many of the protocols that are suited for use in wireless
`communications systems are still in draft status. The present
`design is independent of any one particular protocol and can
`be used in connection with many such protocols. Accord—
`ingly. someone having ordinary skill in the art is able to
`implement the present design in existing protocol environ-
`ments as well as in protocol environments under development
`or yet to be developed.
`The present design can be used anywhere inside. i.e. ware-
`houses. on manufacturing floors. in offices. on trading floors.
`in private homes, and outside of buildings. in cars and trucks,
`in airplanes. just to mention a few examples.
`'l‘wo devices can be connected using a lrl connection.
`Possible media are infrared and magnetic fields. The proce-
`dure to setup such a 1:1 connection can be similar to today‘s
`setup of a connection between two IrDA enabled devices. Le.
`the devices must be positioned such that their communication
`subsystems (transceivers) can “see” each other. Then both
`systems are triggered to start a connection setup procedure
`until a wireless communication channel is established.
`
`Likewise. two devices can be connected using a shared
`medium . A possible shared medium is RF (Radio Frequency].
`Possible systems could be based on technology and protocols
`like BlueTooth. DUCT. and I-IummingBird.
`Details about HummingBird transceivers are given in
`“Hummingbird Spread Spectrum Transceiver Operator‘s
`Manual". Rev. 24 Jun.
`[998. XIi‘l‘RON (.‘orp.. Cincinnati.
`Ohio. USA
`Details conceming the basic problems regarding identifi-
`cation and addressing, initial (resource) discovery, matching
`and selection of communication partners. etc. depend on the
`medium used and the communications protocol employed.
`When referring to a device. any kind ofdevice is meant that
`can establish a network connection to another device.
`Examples of devices are: laptop computers. workpads, node-
`pads. personal digital assistants (PDAs). notebook computers
`and other wearable computers, desktop computers. computer
`terminals. networked computers, interact terminals and other
`computing systems. set—top boxes. cash registers. bar code
`scanners. point of sales terminals. kiosk systems. cellular
`phones. pagers. wrist watches. digital watches. badges. and
`smart cards. Other contemplated devices include: headsets.
`Human Interface Device [I-IID) compliant peripherals. data
`and voice access points. cameras. printers,
`fax machines.
`keyboards. joysticks, Hilii systems. audio (sound) cards.
`loudspeakers. amplifiers. video cards. kitchen appliances.
`tools. sensors such as smoke andtor fire detectors. and virtu—
`ally any other digital device.
`Other examples of devices that can be used in coiniection
`with the present
`invention are, personal effects being
`equipped with computer-like hardware. such as a “smart wal-
`let" computer, jewelry. or articles ofclothing. In addition to a
`“smart wallet“ computer. there are a number of other varia-
`tions of wearable computers. A “belt” computer is such a
`variation which allows the user to surf, dictate. and edit docu—
`ments while they are moving around. Yet another example is
`a child‘s computer which is comparable to a personal digital
`assistant for grade-school children. The child‘s computer
`might hold assignments. perform calculations. and help kids
`manage their homework. It can interface with other children‘s
`computers to facilitate collaboration. and it can access a
`teacher‘s computer to download assignments or feedback.
`Any wearable or portable device. any office tool or equip-
`ment, home tool or equipment, system for use in vehicles. or
`
`

`

`7
`
`8
`
`US ?,577,910 B]
`
`systems for use in the public (i.e. vending machines, ticketing
`machines. automated teller machines. etc.) may include the
`present invention.
`It is also assumed that a device. as used in connection with
`the present invention. includes a minimum amount of pro-
`cessing power to enable it to participate in the design accord-
`ing to the present invention. These devices are thus also
`referred to as computer devices. Most. ifnot all. ofthe above
`listed devices may be viewed as being devices with limited
`user-interfaces. This may even be the case for a personal
`computer which has a display and a keyboard. There is still
`room for improvement in such a computer’s interface. eg. by
`adding speech input. There are no absolute criteria which can
`be used to decide whether a particular device is a device with
`a limited user-interface or not. There is always room for
`improvement and thus any computer device per se is assumed
`to be a device with a limited user-interface. The present inven-
`tion may be implemented where there is a second computer
`device that has a more powerful user-interface. more
`adequate, more convenient. or superior user-interface capa-
`bilities. Not all aspects of the user—interface have to be supe—
`rior or more powerful. It is sufficient. for example. if there is
`a first device which has no speech input (ie. it has a limited
`user-interface) and a second device which has a speech input.
`The present invention may also be implemented. as selected
`by a user. to control the second computer device that has a
`more powerful user-interlace via the device with the limited
`user—interface. if so desired as a users convenience dictates.
`Some of the above~mentioned devices can be regarded
`hereinafter as the device (controller) whose interface is used
`to interact with the user-interface limited device [controlled
`device).
`A computer device is generally referred to as a computer
`device with a limited user-interface where. for example. one
`or more of the following applies: the user—interface is inad—
`equate for the tasks required: the user-interface is small and
`difl'icult to read. understand. or hear: the userninterface pre«
`sents an inconvenience to the user;
`there is no graphics
`capable display {e.g. a text-only display); there is a restricted
`number ofinput keys. or input keys which are too small: there
`are too malty functions which are mapped to a limited number
`of buttons and thus imposes complicated control structures
`that make it difficult to operate the device without prior exteri—
`sive study of a user’s manual. especially for seldom used or
`advanced functions;
`the user-interface is not powerful
`enough. making its use to slow. or has low resolution. or the
`like.
`
`Devices with a more powerful user-interlace capabilities
`generally fulfill. for example. one or more of the following
`criteria: there is a larger screen; there is a screen with graphics
`capability: there is a full keyboard; there is a pointing device;
`there is a voice-input feature and so forth.
`The user-interface may be any kind of interface used for
`ittleraction between a user and the device. such as a display.
`keyboard. mouse. track poim. audio output. speech recogni-
`tion input. tactile input. etc.
`A typical environment where the present invention is used
`is illustrated in FIG. 1. There is a first computer device 10
`which has a limited user-interface 11 (in the present example
`the user-interface comprises a simple display and a few but-
`tons). When there is a second computer device 12 in vicinity
`of the first device 10 that has superior user-interthce capabili-
`ties (keyboard 13 and display 14) titan that of the first device
`10 and which may be controlled or configured. and assuming
`that the two devices 10 and 12 communicate with each other.
`the superior user-interface capabilities of the second device
`12 may be employed to facilitate and speed-up the use of the
`
`10
`
`3t]
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`first device 10 by transmitting a more powerful userninterface
`(user-interfa