`CLASSIFICATION
`
`6218156| ISSUE " 621618R
`
`PATENT NUMBER
`
`U.S. UTILITY PATENT APPLICATION
`
`eee
`
`
`>
`cG
`Derby:
`.
`
`rege.
`o~[ F
`FILED WiTH: [_] Disk (CRF)
`[_]FICHE
`(Attached In pocket on right side fap)
`
`HO) '
`
`PREPARED AND APPROVEDFORISSUE
`hi
`
`ISSUING CLASSIFICATION
`il
`
`
`
`
`| hesaCROSS REFERENCE(S) ;
`
`
`
`eeeeeaioetteHeepie
`
`of * eseI_——Fe—]
`CI Continuedon issue Slip Inside File Jackel
`
`
`
`
`
`Sheets Drwg. Figs. Drwg.|Print Fig. Total Claims Print Claim for 0.G.
`
`7
`7
`/
`Hb
`/
`
`fT terotpoor
`WWeaquentto (atts)
`been-‘distlaimed.
`
`'
`
`Llib) The term of this patent shall
`
`:
`
`(Asalatant Examiner)
`
`a
`(Dete}
`
`°
`
`p>f
`
`.
`
`
`
`
`
`of extent]beyondthe expirationdate. =—bkbe
`
`
`
`VIET D.VU
`-
`Amount Due
`PRIMARY EXAMINER
`
`
`
`
`(Primary Examiner)
`>
`ix ©\.0
`6.42
`;
`Lo
`ISSUE BATCH NUMBER
`
`U3
`for
`Pend ULAp
`. Bile) The terminal
`months of
`
`
`
`“(LegalExaminer)instruments ~—~=«S=(Date) 7
`
`
`thig patent have been disclaimed.
`
`
`WARNING:
`
`’ Thelintormation disclosed herein may be restricted. Unauthorized disclosure may be prohibited by the United States Code Title 35, Sectlons 122, 181 and 368.
`‘| Poséession outside the U.S. Patent & Trademark Office Is restricted to authorized employees end contractors only.
`
`
`Sli Ni 333 atSsi
`cortiBengal
`_stis}eet_.
`
`
`
`-2)_d0]
`(Date)
`
`|
`iP
`
`—
`
`<
`(FACPS
`
`LGEExhibit-1002/Page 1 of 176
`LGEv. Uniloc
`
`

`

`
`
`
`a
`
`09237609
`
`| 804998"
`
`Date received
`(incl. C. of M.)
`or
`Date Mailed
`
`Date received
`(incl. C. of M.}
`or
`Date Mailed
`
`42.
`
`43.
`
`44.
`
`45.
`
`46.
`
`47.
`
`’
`
`50.
`
`is
`
`51.
`
`52.
`
`53.
`
`54,
`
`55.
`
`56.
`
`57.
`
`58.
`
`59.
`
`60.
`
`61.
`
`62.
`
`oes
`
`
`
`63. :fetf}=
`
`64.
`
`65.
`
`66.
`
`67.
`
`68.
`
`69.
`
`70.
`
`71.
`
`72.
`
`73.
`
`74..
`
`75.
`76.
`
`77.
`
`78.
`
`79.
`
`80.
`
`81.
`
`82.
`
`.
`
`(LEFT OUTSIDE)
`
`LGE Exhibit-1002/Page 2 of 176
`
`“| =BATENT APPLICATIONoe
`INITIALS a
`1 = CA SONS
`1.a 5Nh* papers,
`eelsa 5
`i. fryAtti
`‘ahen/p
`
`DIOfons
`
`48.
`
`49,
`
`ee
`
`

`

`*
`
`ISSUE SLIP STAPLE AREA (for additional cross references)”
`
`
`
`
`dupclasemman=12> I2-4-PF|
`
`
`L.P.E. CLASSIFIER So|
`
`
`a
`|FORMALITYREVIEW|Weenpw
`
`
`INDEX OF CLAIMS
`seitiissinasiiaagaasnisascienszas Rejected
`"WNW
`conesesspeccsiasve
`OS
`}
`sicaseicactied Non-elected
`
`
`; EE.*-,sasniasesena cabinsanteyinsimitys Allowed LR Sracuesasniogians gauaseuennnetse Interference
`
`
`secassogesiait? Appeal
`csasaaantaeasc Objected’
`
`SB
`
`_= (Through numeral)... Canceled
`
`aeitseanascenssemactep oe Aipsigan’ Restricted
`7
`
`Be
`
`Os
`
`sestsscassasesiany
`
`seisasiaisnouasivacs
`
`
`
`TTT WaT
`
`claim|Date
`zg
`
`
`pe WeeTatlectagtetah
`atBaan aEs
`
`eaoece
`
`
`
`
`
`
`
`
`If more than 150 claims or 10 actions
`staple additional sheet here
`
`(LEFT INSIDE)
`
`LGEExhibit-1002/Page 3 of 176
`
`mtTt |TTTT
`
`él Bu
`
`opr
`
`.Pe‘Tee
`ateBePra
`RilesBea
`
`Cee
`
`asaiedecVichaoneee
`re
`cs)aTAeHEHE
`SaCee
`
`eo rs
`
`
`
`etFECETE
`IGELy
`Pfs
`
`
`
`22
`
`
`pet
`eeae
`|frost|TTTT
`|pet|TTa
`|pert|TTa
`
`tt
`Ppea
`
`
`
`|fog|fjt|tTtT
`
`

`

`SEARCH NOTES
`
`(INCLUDINGSEARCHa
`
`
`a
`
`ts ~ = 3
`
`(AIGHT OUTSIDE)
`
`LGE Exhibit-1002/Page 4 of 176
`
`
`
`

`

`US006216158B1
`(10) Patent No.
`a2) United States Patent
`US 6,216,158 Bl
`Luo etal,
`(45) Date of Patent:
`Apr. 10, 2001
`
`
`(54) SYSTEM AND METHOD USING A PALM
`SIZED COMPUTER TO CONTROL
`NETWORK DEVICES
`
`(75)
`
`M
`Inventors: Wenjun Luo, Fremont; Elaine P.
`Lusher, Pleasanton, bot of CA (US)
`
`(73) Assignee: 3COM Corporation, Santa Clara, CA
`(US)
`
`(*) Notice:
`
`Subject to anydisclaimer,the termofthis
`patent
`is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/237,609
`.
`Filed
`
`(22)
`
`7/1999 Jolson él ah. nceJ9STI2
`3,923,885 *
`
`
`ww 709/221
`3.964.836 ~ 10/1999 Rowe et al. ou.
`$.973.696 * 10/1999 Agranat et al. occ. 345/357
`OTHER PUBLICATIONS
`
`M.G. Lamming, Jastinue for Personalized Information
`Environment 1994 “Towards Future Personalised Informa-
`tion Environments” Jechnical Report EPC—i1994—104,
`Dee. 21, 1993.
`McCandless, Michael, “The PalmPiolt and the Handhled
`Revolution”, IEEE Expert, Nov./Dec. 1997, pp. 6-8.
`’
`.
`* cited by examiner
`Primary Examiner—Viet D. Vu
`(74) Attorney, Ageni, or Pirm—Matk A. Haynes; Haynes &
`Beffel LLP
`
`Jan, 25, 1999
`ABSTRACT
`(57)
`.
`
`
`CBS,|Tanta2 visiicicecccusoauaienasisinossicrscqonsiuass GO6F 13/00
`.. 709/217; 709/219; 709/225;
`(52) US. Cl
`Controlling network services using palm sized computersis
`709/313; 709/329
`described. A program on the palm sized computer is used to
`. 709/202, 203,
`access a registry of network services thal may be available.
`(58) Field of Seareh 10.0...
`
`225, 230, 250, 313
`W09/217. 2
`The registry includes descriptions for various services. Each
`319, 328, 329
`description includesat least 4 relerence to program code thal
`can be downloaded to the palm sized computer. Executing
`this program causes the palm sized computer
`to issue
`commands directly to the specific network services needed,
`In some cases, these network services include application
`services for running desktop applicationsthat the palm sized
`computer could not execute
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,504,070 * LO/1T996 Want eb ab, i
`eeseetiectieee 455/507
`
`5,790,977 *
`8/1998. Ezekiel...
`702/122
`3,832,506 * 11/1998 Kuzina ......
`707/200
`5,909,545 *
`6/1999 Frese IL et al.
`..ccseesceneeces, 7U9/208
`
`
`
`20 Claims, 7 Drawing Sheets
`
`Application Service
`
`4, Control
`
`/ovnad
`
`PowerPointContra
`
`
`
`
`
`120
`
`10Network Hg
`
`
`
`Storage
`Service
`
`140 Display Service
`
`LGEExhibit-1002/Page 5 of 176
`
`

`

`U.S. Patent
`
`Apr. 10, 2001
`
`Sheet 1 of 7
`
`US 6,216,158 Bl
`
`4, Control
`
`3. Download
`
`Application Service
`
`
`|
`
`150
`Storage
`Service
`
`
`
`140 Display Service
`
`FIG. 1
`
`LGE Exhibit-1002/Page 6 of 176
`
`

`

`U.S. Patent
`
`Apr. 10, 2001
`
`Sheet 2 of 7
`
`US 6,216,158 B1
`
`
`
`
`
`
`
`wayshsBuneiadgaanageamesTaz
`
`
`
`
`
`Voweariddyjo.ju09sawwes092
`
`
`
`pasegYomjany062
`
`
`
`S01AaSJayndwio)
`
`sas
`
`B0IM3S
`
`syoalgg
`
`soneiNfae
`“~joomio),Aiaoosig
`
`
`DedjJuMogaeMyjos
`
`SAQiMAS40A1O}2auIQ
`
`022
`
`
`
`JoalqQ)aOInagluly222
`
`palgjsawnacuoneiuasalapie222
`
`
`
`JBDRUEYJOD0}OI4aleMalpp\N|G12
`
`
`
`TpalqQeoieseU3zzz
`
`SeMALDPIIN812
`
`
`
`
`
`\Wesfioy4SUONeOIUNWWO’OMAN)2OZ
`
`
`yoalagaawagxe,222
`
`
`
`FADRUEYY[odOqOL4jsNUO')UOWLO\dyBIZ
`
`
`
`
`
`volyeatiddyoquor)O12
`
`
`
`\waysk¢funeisdyaainagjoxu09TOZ
`
`
`
`
`
`os
`
`ooo}uoneysifey§—garnagjouyuor)o0z
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Apr. 10, 2001
`
`Sheet 3 of 7
`
`US 6,216,158 Bl
`
`
`
`
`
`uoneaiiddyjouyu0)aowmasO92
`
`
`
`
`
`Jabeueyy000014oru07yo4yiaMo4PSE
`
`
`
`FaHeUeW\J090]014aleMaDDIWNSOMES992
`
`
`
`
`
`
`
`
`
`
`
`WelBol4SUOREOIUNUWO’BaIAaqaINaSZz
`
`WUOgaNO4BOE
`
`
`
`ypalqgaoineclew722
`
`
`
`BUIUIEYY[ENYABnepTE
`
`
`
`aliasJayndwo4
`
`pasegWOMJaKj092_-pohawe
`
`€3ld _-oni
`
`
`
`noasiqlifseyndwiogpazigweyoot
`
`
`
`
`
`
`
`WayshgBunesedeaoinagaaiesTSz
`
`dootwitSaENDSsy
`“~jog0}014
`
`
`
` S01640AlojoallqO2zwaysisBuesadgwedTOE
`
`
`
`uoneoniddyjonu09O12
`
`a0A9S
`
`s}99Iq0
`
`joalqQeawecuonequasalsepisZz2
`
`
`JaHEUEyOO}OYALMalOpIWy1z
`
`{oalqQaainiagreyZz
`
`
`
`JaBeueyyoonjdl,ox\u0}yuloglamayPTE
`
`
`
`Joalgawagiuiy222
`
`\ng212
`
`
`
`
`
`WelBolySUONBDIUNWWO}WOMAN)202
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Apr. 10, 2001
`
`Sheet 4 of 7
`
`US 6,216,158 BI
`
`210 Control Application
`
`410 CPU Service
`
`Middleware Protocol Layer
`
`
`
`
`
`
`=ControlProtocolManager
`
`
`
`
`
`
`
`430 Projector Service
`420 Storage Service
`
`
`}| Middleware Protocol Layer
`Middleware Protocol Layer i
`eae
`4
`
`
`
`Projector controlfront end
`LDAPfront end
`
`
`
`
`
`
`
`PowerPoint Control
`CPowerPoint
`
`Backend LDAP Database
`CLdapDb
`
`Projector Control
`CProjector
`
`FIG. 4
`
`LGE Exhibit-1002/Page 9 of 176
`
`

`

`U.S. Patent
`
`Apr.10, 2001
`
`Sheet 5 of 7
`
`US 6,216,158 B1
`
`Object Class Hierarchy
`
`
`
` Presentation
`
`
`Seon
`
`-
`
`FIG. 5
`
`LGE Exhibit-1002/Page 10 of 176
`
`

`

`U.S. Patent
`
`Apr. 10, 2001
`
`Sheet 6 of 7
`
`US 6,216,158 B1
`
`Object Class Hierarchy for Network Services
`
`Network Service
`
`Application
`service
`
`Operating System
`Service
`
`Storage
`Service
`
`Projection
`Service
`
`Service Location
`Service
`
`Slide Presentation||Speech Recognition||Groupware||Email
`Service
`Service
`Service
`Service
`
`Object Instance
`
`Object Class
`
`FIG. 6
`
`LGE Exhibit-1002/Page 11 of 176
`
`

`

`U.S. Patent
`
`Apr. 10, 2001
`
`Sheet 7 of 7
`
`US 6,216,158 B1
`
`Object Class Hierarchy for Application Services
`
`Application
`Service
`
`Slide Presentation
`Service
`
`Speech Recognition
`Service
`
`Groupware
`Service
`
`Email
`Service
`
`Naturally AltaVistaffLotus§=fEudora}||Microsoft
`
`
`
`Speaking Forumjf}Notes Mail Mail
`
`
`Netscape
`Mail
`
`Object Instance
`
`Object Class
`
`FIG. 7
`
`LGE Exhibit-1002/Page 12 of 176
`
`

`

`1
`SYSTEM AND METHOD USING A PALM.
`SIZED COMPUTER TO CONTROL.
`NETWORK DEVICES
`
`BACKGROUNDOF THE INVENTION
`1. Field of the Invention
`
`US 6.216.158 Bl
`
`2
`PIG. 7 illustrates an example class hierarchy for applica-
`tion services that can be provided in the system of FIG. L.
`
`a Ly
`
`‘This invention relates tothe field of networked computer.
`In particular, the invention relates to 4 system and method to
`control network devices using a palm sized, or otherwise
`reduced functionality, computer.
`2, Description of the Related Art
`Palm sized computers, also referred to as Personal Digital
`Assistants (PDAs), are portable devices which perform an
`array of personal managementtasks such as calendar man-
`agement and address book storage. The adoption of palm
`sized computers has been rapid. Some palm sized computers
`are able to interface with conventional computing devices,
`such us PCs, on an as-neededbasis. For example, palm sized
`compulers such as 3Com’s Palm Platform™ computers can 2
`upload personal appointments to a PC-based calendar.
`Palm sized computers generally have the following char-
`acteristics. Relative to desktop and Japtop computers, palm
`sized computers have limited processing, display and inpul
`capabilities. As a result of these limilations, palm. sized
`computers do nol run the same applications as desktop or
`laptop computers. Otherlimitations of palm sized computers
`include limited battery life and lower bandwidth communi-
`cations with other devices.
`
`31 Se
`
`DETAILED DESCRIPTION
`s A. System Overview
`A palm sized compuler can serve as a network portal (0
`usher in a new generation of mobile computing. Palm sized
`computers are the ideal next generation computing device in
`that they are inherently mobile and haye the hghiweighi
`form factor necessary for mobile computing. The fundamen-
`tal obstacle to palm sized computers as the next generation
`computing device cin be removed by viewing the network
`as an extension of the palm sized computer's resources,
`Functions can be downloaded into the device as needed, and
`overlaid after they have been used. This allows the palm
`sized computer to adapt fo a changing environment(as (he
`mobile user’s location changes) and lo access exactly the set
`of servicesit needs, These services are stored on the network
`and can be used at will. Many of these services may never
`reside on the device and are more suitable for execution by
`a conventional computer. However, they are accessible and
`can be controlled via a lightweight computing device, such
`as a palm sized computer.
`B. Building Blocks of Neiwork-Enabled Palm Sized Com-
`pulers
`The building blocks for lightweight mobile computing
`include a palm sized computer, a compute/memory/storage-
`intensive device(s), and a network.
`Middleware allows palm sized computers to discover
`network-based computing resources. Once discovered, this
`middleware provides a mechanism for the palm sized com-
`puter to use these resources. This middleware typically
`includes a directory of resources (or services), a protocol for
`storing and retrieving fromthe directory, and mechanismslo
`transfer software from the directory to a palm sized com-
`puter,
`Control applications reside on a lightweight computing
`device, such as a palm sized computer, but manipulate
`computing services on the network. Control applications
`assume the use of middleware, such as Sun Microsystems
`Jim, for access to network-based resources. (Note in some
`embodiments, the control application ts distributed between
`the palm sized computer and a control proxy server.)
`Example Registry and Control Infrastructure: Jini
`Jini™is a technology developed by Sun Microsystems
`which addresses the problem of computing and network
`complexily. It eases the burden of accessing services: by
`providing seamless access and automatic registration of
`neiwork resources, Jini accomplishes this by adopting a
`model in which devices announce their presence and capa-
`bilities to the network, and access the same type of infor-
`mation in order to locate services they wish to use.
`The Jini approach relies on Java and a Jini “registry”(i.e.
`database of seryices) as the underlying infrastructure, Bach
`device is expected to run a Java Virtual Machine (JVM), or
`rely on a Jini proxy which runs a TYM on the device’s
`behalf. Key to Java is the idea that soltware (as well as data)
`can be dynamically dowaloaded to a device, The Java model
`assumes a distributed, network-centric model in which the
`behavior of a device can be dynamically altered to accom-~
`modate changing conditions,
`Jini cases network connectivity problems. Jini acts as
`middleware to access network resources, as it lets devices
`locate services and download software for those services.
`Other middleware could be substituted for Jini if it provides
`discovery and software download for network-based ser-
`Vices.
`
`One big advantage of palm sized computers is their
`portability, Therefore,
`it
`is desirable to be dble to access
`desktop functionality from palm sized computers.
`
`SUMMARY OF THE INVENTION
`
`35
`
`AL
`
`Controlling network services using palm sized computers
`is described. A program on the palm sized computeris used
`lo access a registry of network services that may be avail-
`able. The registry includes descriptions for various services.
`Each description includes at least a reference to program
`code that can be downloaded to the palm sized computer,
`Executing this program causes the palm sized computer to
`issue commands directly to the specific network services
`needed.
`In some cases,
`these network services include
`application services for running desktop applicationsthatthe
`palm sized computer could oot execute.
`ln some embodiments, the device executing the network
`services and the pulm sized computer are executing middle-
`ware applications for communicating with the registry, In
`some embodiments, this middleware includes Jini technol- s
`ogy from Sun Microsystems. Additionally,
`the programs
`downloaded can include Java program code.
`BRIEF DESCRIPTION OF THE FIGURES
`
`sna
`FIG. 1 illustrates 4 system having a palm sized computer °
`controlling operations of various network devices.
`FIG. 2 illustrates a sofiware architecture for use in the
`system of FIG. 1.
`FIG. 3 illusiraies an example architecture for controlling
`a PowerPoint presentation.
`FIG. 4 illustrates a detailed software architecture for the
`example of FIG, 3.
`VIG. 5 illustrates a class hierarchy that can be used in the
`example of FIG. 3.
`MG, 6 illustrates an example class hierarchy for network
`services that can be provided in the system of FIG, 1,
`
`60
`
`aa
`
`LGEExhibit-1002/Page 13 of 176
`
`

`

`US 6.216.158 Bl
`
`3
`
`wn
`
`io
`
`©, Detinitions
`Acontroldevice is a device which runs Jini (or some other
`discovery and software download technology) and is
`capable of accessing a wide range of network-centric
`services, including services which are incapable of running
`on the control device. The control device is the access device
`for a wide class of computing services otherwise inacces-
`sible to devices with restricted computing power. 3Com’s
`palm sized computer, the Palm Computing platform, is an
`example of such a control device. Windows CE compatible
`devices may also be used,
`A control application is as an application that relies on
`resources located off of the control device (e.g. on a
`network), but uses a control deviceto initiate and control the
`application. The resources are accessed and controlled, but
`nol
`resident, on the contro] device, Examples of such
`compute/memory-inlensive services include PowerPoint
`slide presentations and speech recognition systems. These
`services can be both invoked and controlled via a control
`device.
`Network-based services can be used by any control
`device. Services offer a discrete task to the control device,
`This contrasts with a traditional model
`in which devices,
`such as a PC, house the entirety of resources a control device
`might utilize. Services are tine-grained and represent
`4
`distributed set of capabilities residing on a network. Services
`may or may not be co-located with other services on the
`same physical device. Services are offered (via a network
`and a Directory of Services, such as the Jini Lookup) io as
`small a unit as is justifiable given the application tasks users
`will typically Want to accomplish. Fine-grained services can
`be used by a variety of consumers who need precisely that
`function. The model that emerges is a network-based mode}
`that offers a wide range of narrowly-defined computing
`servicesofinterest to a wide range of applications, Services
`will be physically distributed across devices but accessible
`Vii a ventral repository (e.g. database) of services. This
`model ofdistributed computing is inherently scalable in that
`the set of services offered via the nctwork grows seamlessly
`as devices “plug in” and announce their capabilities.
`D. An Example of the Control Application
`To belp ilustrate the use of a control device and a contro)
`application, a PowerPoint slide presentation control
`is
`described herein. (FIG. 1 illustrates a generalized view of
`such an cxample.) In this example, a palm sized computer
`100 accesses network 110 resources (via a Directory of
`Services, such as the lookup service 120) to locate the
`elements il requires to remotely control a presentation
`located on the network. The palm sized computer 100 uses
`three services to function as a control device to display a ,
`PowerPoint presentation:
`an application service 130 (lo nin PowerPoint)
`a persistent storage service 150 (lo store the presentation)
`a display service 140 (to display the presentation images)
`Each of these services will have registered with the 5
`lookup service 100. A service in this example, is tied to
`different hardware devices. However, ihis is nol required,
`Storage, processing, and/or display could be supported by
`one Or more computer systems.
`None of these services are resident on the palm sized
`computer 100, Once the palm sized computer LOO has
`located the necessary services,
`it downloads the code
`required to control
`those services (using the lookup and
`download protocols). Middleware, such as Sun's Java/Jioi
`technology, is used to move the code,
`The palm sized computer 100 is then capable of directly
`controlling the services it requires,
`
`30
`
`40
`
`60
`
`4
`In this example, all the devices can communicate on the
`network 110, however, they need not all be connected to the
`network in the same way.
`Functionally, the devices play the following roles:
`palm sized computer 100: the remote control device
`application services: a set of resources and services, such
`as compule power and applications
`network 110:
`the physical medium which connects
`devices and services
`
`lookup service 120; a database of network services
`b. Architecture
`As noled above, the palm sized computer 100 functions as
`the remote control device for the PowerPoint presentation. I
`is a multi-function control device in that it can control a host
`of resources accessible via the network. The palm sized
`computer LOO accomplishes this via middleware (e.g. Jini)
`uod a generic control protocol capable of issuing control
`commands to an offboard resource. The palm sized coms
`puter 100 implements this control via the software compo-
`nents depicted io FIG. 2. These software components rep-
`resenl a generic architecture for control of any network-
`based resource using a reduced functionality computer such
`as a palm sized computer. The software architecture,
`middleware, and control protocol implement a new model
`for lightweight mobile computing. This model of light-
`weight mobile computing is particularly well-served by
`devices such as a palm sized computer, as they possess the
`requisile size and portability,
`In FIG. 2, the control device 200 ts able to contro] services
`on the network 110, such as the network based service 250,
`The directory of services 220 provides the registry functions
`used to enable the system.
`The control device 200 includes a control device operal-
`ing system 201 which supports a network communications
`program 202 and a control application 210. The network
`communications program 202 allows the control device 2(W)
`to communicate with other devices on the network. The
`control application 210 includes the following clements:
`a GUI 212 todisplaythe available network-based services
`and accept user input
`an application control protocol manager 214 which inter-
`faces between the control device 200 and the network
`based computer service 250 by requesting tasks from
`that service (c.g. slide manipulation), This module is
`responsible for generating the application control pro-
`tocol to command the selected service,
`a middleware protocol manager 216 to transfer data
`between the contro] device 200 and the directory of
`services 220 (¢.2. communication between Palm plat-
`form and the Jini Lookup). This module is responsible
`for generating the syntax necessary lo Communicate
`based on the type of middleware being used.
`The service device operating system 251 includes a
`service device communications program 252 and a service
`control application 260, The service device communications
`program 252 is responsible for communicating with the
`network LLOQ. The service control application 260 includes
`the following components:
`a Service application control protocol manager 262 that
`interfaces between the network based computer service
`250 and the control device 200 and accepts control
`lasks (e.g. slide manipulation) issued from the control
`device 200. This module is capable of sending back a
`response (e.g. status) lo the control device 200,
`a middleware protocol manager 266 to transfer data
`between the network based computer service 250 and
`
`LGEExhibit-1002/Page 14 of 176
`
`

`

`US 6.216.158 BI
`
`6
`&. Via lhe service control GUI 212, accept user input, such
`as the selection ofan application (for example,.a Power-
`Point application) to be controlled.
`9, Optionally register the conirol device 200 with the direc-
`lory service via a registration protocol, such as the Jini
`Discovery Protocol. This step is the sameas the above one
`for the other services. It is executed only if the control
`device 200 has resources tooffer.
`10, Search the directory service 220 snd download the
`desired service descriptor. In the case of Jini, after ihe
`control device 200 receives the response from the direc-
`tory service 220, it uses the included interface to search
`the directory service 220 for an application service using
`the object type representing the service (such as an object
`type of PowerPoint presentation service) and the desired
`service aftributes (such as the name and the physical
`location of the service), Once the directory service 220
`finds such a service entry,
`the control device 200 is
`nolified, which in turn uses the downloading interlace to
`download the application service descriptor. One example
`of these services is the GUI code for controlling a
`PowerPoint presentation.
`11. Send requests from the control device 200 to the network
`based computer service 250 to control the desired appli-
`cation. For example, u “next slide” request could be sent
`from a palm sized computer LOO to an application service
`130 moning PowerPoint. The communication can be
`based on a protocol such as the following onc:
`
`a 5
`
`20
`
`
`
`
`
`5
`the directory service (e.g. communication between the
`application service 130 and the Jini Lookup), This
`module is responsible for generating the syntax used to
`communicate based on the type of middleware being
`used.
`an application 268 to perform functions on the network.
`The application 268 can be a desktop application that
`would not execute on a palm sized computer.
`Returning to the specific example of the PowerPoint
`prescotation,FIG. 3 illustrates the architecture that could be
`used to implement such a system. In FIG, 3, the control
`device 200 has been replaced with a palm sized computer
`100 executing the Palm OS 301. The GUI 212is specifically
`for PowerPoint control (see FIG. 1 for an example of such
`a GUI). The Java Virtual Machine 318 is executing on the
`palm sized computer 100 and replaces the middleware 218.
`The middleware protocol manager 216 supports Jiai
`discovery, lookup and download protocols. PowerPoint con-
`trol commands are issued to the network based computer
`service 250. The PowerPoint control protocol manager 364
`provides the interface for these commands and controls the
`PowerPoint application 368.
`F. Control Flow
`The process for accessing and controlling network-based
`services is descnbed below. The specific implementation
`control of a PowerPoint Service via a 3Com Palm Comput-
`ing platform is used as an example. Refer to FIG. 1,
`i, Configure a computer hosting the directory of services
`220 and connectit to the network 110, Por instance, a Jini
`lookup service is configured to listen at a preset TCP, or
`30
`UDP. port
`for service registration or service lookup
`Control Type File Nsme=Conuol FunctionApplicstion
`requests.
`2, Establish u network connection from the network based
`computer service 250 to the network 110.
`3. Register the computing service with a directory of ser-
`vices 220, For example, in the Jini model, the computing
`service initially sends out a noulticast packet announcing
`ils presence on the network. Once the directory service
`220 receives the packet, it sends a unicast packet back to
`the announcer, The packet includes an interface (e.g. Java
`code) for uploading code back to the directory service 220
`as well as for searching and downloading code from the
`directory service 220,
`4, Upload service description to the directory service 220. If
`Jini is adopted as the middleware, the application service
`130 receives the response from the directory service 220
`and uses the included interface to upload its service
`interface to the directory service 220. When the service
`interface is called, it contacts the directory service 220
`which in turn creates an entry (represented by an object)
`wa>
`for this new service and sets the proper fields such as s
`service name,attributes and optionally the corresponding
`service fatecface. Other middleware may choose fo use
`protocol-based approaches such as FTP or TFTP for the
`uploading process.
`5. Register the storage service 150 and display service 140 s
`via the same process.
`6.co
`Establish a network coonection from the control device
`200 to the network 110. For Palm computers, there are
`multiple options for network connectivity. Possible solu-
`tions include using the infrared (IR) port
`to talk to a
`IR-LAN bridge or router, using the serial pori to talk to a
`serial-to-LAN bridge or router, using either the IR or the
`senal port to talk to a digital cell phone and dial up a
`modem server, and/or using wireless data Communica-
`lions.
`7. Launch the service control graphical user interface (GUI)
`212 on the control device 20).
`
`Where:
`Control Type={Request, Reply}
`Application={PowerPoint, FAX, Print, Email.
`Phonebook, .
`.
`.
`}
`File Name={3ComPalm VISpecitication}
`Control function={File Load, Slide Forward, Slide
`Backward, File Close, .
`.
`.
`}
`Alternatively, techniques such as Java's Remote Method
`Invocation (RMI) can be used to achieve the same goal. In
`this case, the control device makes a local function call such
`as doForwardSlide( ). The RMI mechanism will transfer the
`call lo a remote machine which implements and carries oul
`the function call. The PowerPoint presentation service may
`in turn use other services suchas the storage service 150 and
`the display service 140. The procedure to employ these
`services is similar to steps 10-11 above,
`12. Accept any response to requests sent from the control
`device 200 to the network based computer server 250 and
`process any errors.
`G. Control Device GUL
`An important clement ofthe control application 210 is a
`GUIfront-end which accepts user input for controlling the
`PowerPoint presentation (or other application) and a contro}
`protocol manager backend which takes user input and trans-
`lates it into commands to the CPUservice, An example GUI
`is depicted in FIG. 1. The example GUI allows the user to
`click on “forward”, “backward”, “go-to-firsi-page” or *gu-
`to-last-page” buttons to control the slide show. The user can
`also click a “get-list” button to get a Full list of the slidetitles
`in the current presentation and choose fo go to @ particular
`slide. By clicking the “scribble” button,
`the window
`Switches to graphics mode. In this mode, the user can draw
`at random onthe panel, and the result of the drawing will be
`sent
`to the CPU service and eventually displayed on the
`projection service,
`
`4p
`
`45
`
`60
`
`65
`
`LGE Exhibit-1002/Page 15 of 176
`
`

`

`US 6.216.158 BI
`
`7
`there are
`As explained in the software flow section,
`multiple ways to implementthe application control protocol
`minager 214, the following illusirates one approach, The
`protocol takes the following form:
`
`wn
`
`
`
`
`
`Application File NameContiol Type Control Function
`
`
`
`
`
`
`
`Property/Method Name
`
`‘Type
`
`Deseription
`PowerPoint
`
`Presentations
`
`Collection List of open presentations
`Presentation
`
`Slides
`Lovation
`
`
`
`String
`Methoo
`
`Method
`
`Title for the slide
`Add given comments to slide al
`grven coordinates
`Add given graphics to slide nt
`given coordinates
`Stornge
`
`Title
`AddComments
`
`AddSeribble
`
`Host
`
`FileName
`
`Collection Asrayof slides in presentation
`Storage
`Location where presentation lonctcel
`Frou
`Where:
`View Projectar=Projector where presentation is
`Control Type={Requesi, Reply}
`viewed
`Slide
`Application={PowerPoint, FAX, Print, Email,
`Phonebook, .
`.
`.
`}
`File Name={3ComPalmVIISpecification}
`Control function ={File Load, Slide Forward, Slide
`Backward, File Close, ... }
`When the user clicks any button or draws something oo
`the GUI,
`the application control protocol manager 214
`generates the corresponding ficld in the protocol and sends
`a command to the network based computer service 250 via
`a TCP/IP channel.
`
`String
`
`String
`
`LDAP Server host where the
`Presentation stored
`File same and other details for
`presentation
`Projector
`
`Hl. Service Control Application Design
`FIG. 4 illustrates 4 detailed software architecture for the
`service control application of FIG. 3.
`‘The architecture
`includes three main elements:
`a CPU service 410
`
`(corresponding to the application service 13), a storage
`service 420 (corresponding to the storage service 150) and
`a projector service 430 (corresponding to the display service
`140). Each of the elements include middleware protocol
`layer management modules. Each module has a correspond-
`ing service application control module (e.g., PowerPoint
`Control Protocol Manager). To control specific network
`services, instances of those services are instantiated and are
`used by the corresponding control protocol manager. The
`following describes examples of such classes that can be
`instantiated for specilic network services.
`PowerPoint Control
`
`Once a presentation is open, the PowerPoint contro) can
`have facilities such as Add Comments and Add Scribbled
`Graphics to the current slide, Additionally, it can allow the
`adding of new slides to the presentation.
`FIG.5 illustrates an example object class hierarchy for (he
`PowerPoint control class. The following describes the ele-
`meats of FIG. 5 in more detail.
`
`through OLE automation,
`The PowerPoint control,
`instintiates and controls an instance of PowerPoint applica-
`oea
`tion, The PowerPoit control also communicates with the
`
`
`5 Propetty/Method Name—‘l'ype Description
`storage service 420 to store/retrieve presentations and the
`RilePilter
`String
`Fite filter used to get file List
`projector service 430 to view slides.
`FileList
`Collection
`List offiles found on
`siorage
`The PowerPoint control has the ability to have many
`presentations Open at
`the same time and is capable of
`swilching between presentations. A preseniation has a col-
`lection of slides in it, Once open, the PowerPoint control
`allows easy traversal of the slides either by commands like
`previous, next slide or by direet aceess (e.g. slide number or
`slide tite). When aoy change in slide position occurs, the
`PowerPoint application automatically generates the image
`that needs to be shown by the projector control and invokes
`the method on the projector control to show the changed
`slide.
`
`Projector host where slides shown
`String
`Host
`
`
`ShowSlide Showstide image on projector Method
`
`30
`
`tabr
`
`LDAP Database Control
`
`The LDAP database control provides lile system services
`(a store and retrieve presentations. This control gets com-
`mands from the PowerPoint c