I 1111111111111111 11111 1111111111 111111111111111 IIIII IIIII IIIIII IIII 11111111
`US007043532Bl
`
`c12) United States Patent
`Humpleman et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,043,532 Bl
`*May 9, 2006
`
`(54) METHOD AND APPARATUS FOR
`UNIVERSALLY ACCESSIBLE COMMAND
`AND CONTROL INFORMATION IN A
`NETWORK
`
`5,347,304 A
`5,387,927 A
`5,389,963 A
`
`9/1994 Moura et al.
`2/1995 Look et al.
`2/1995 Lepley et al.
`
`(Continued)
`
`(75)
`
`Inventors: Richard Humpleman, Fremont, CA
`(US); Dongyan Wang, Santa Clara, CA
`(US)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`84110755.0
`
`9/1984
`
`(73) Assignee: Samsung Electronics Co., Ltd., Suwon
`(KR)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by O days.
`
`This patent is subject to a terminal dis(cid:173)
`claimer.
`
`(21) Appl. No.: 09/307,004
`
`(22) Filed:
`
`May 7, 1999
`
`Related U.S. Application Data
`
`(60) Provisional application No. 60/084,578, filed on May
`7, 1998.
`
`(51)
`
`Int. Cl.
`G06F 151173
`(2006.01)
`G06F 15116
`(2006.01)
`(52) U.S. Cl. ....................................... 709/208; 709/223
`(58) Field of Classification Search ................ 709/223,
`709/208, 328,227; 345/329, 733, 734; 707/102;
`715/733, 734; 719/317, 328
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,031,543 A
`4,860,006 A
`5,249,043 A
`5,257,366 A *
`5,293,635 A
`
`6/1977 Holz
`8/ 1989 Barall
`9/1993 Grandmougin
`10/1993 Adair et al.
`................... 707/4
`3/1994 Faulk, Jr. et al.
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`Evans , G. "Solving home automation problems using
`artificial intelligence techniques", IEEE Trans. on Consumer
`Electronics, pp. 395-400, Aug. 1991.*
`
`(Continued)
`
`Primary Examiner-Patrice Winder
`(74) Attorney, Agent, or Firm-Michael Zarrabian, Esq.;
`Kenneth L. Sherman, Esq.; Myers, Dawes, Andras &
`Sherman, LLP
`
`(57)
`
`ABSTRACT
`
`A method and system for performing a service on a home
`network, by: connecting a first and a second home device to
`the home network; providing a database including a plurality
`of application interface description data objects, where each
`application interface description data object includes infor(cid:173)
`mation in a structured format for commanding and control(cid:173)
`ling of a home device by one or more other home devices
`connected to the network; the second home device accessing
`a first application interface description object for the first
`home device in the database; the first home device accessing
`a second application interface description object for the
`second home device in the database; sending control and
`command data from the first home device to the second
`home device utilizing the second application interface
`description object over the network; and sending control and
`command data from the second home device to the first
`home device utilizing the first application interface descrip(cid:173)
`tion object over the network. Whereby, the first and second
`home devices perform said service.
`
`27 Claims, 14 Drawing Sheets
`
`____ Library(eases 3,4)
`
`Case 1. XMLA\:lplleation-C INTERFACE (A)for remotaquery by (E)
`Casa 2. Move XMLApplicalion-C INTERFACE (A) to cofrtroller fer 1oc:a1 quar, by (El
`Case 3. Move XML Applicatlon.C INTERFACE (A) to 3rd party devi=e-a eollectlng
`place (library) for all lnterfacas for remote query by (E). Tiw Ubrary would have to
`lla<Jethtaddreu{URl)oflheauaeiatedappUcallonavalllll:l!efordirec:tcontrol
`ar;tlonandiuponses.
`Case4.l!1Ga!1e3but:he'lndirecl'conlroladlon{andresponse)lsalll.0dlrededat
`thelibraryde\lica.lnthl1Jcall91heibrayis'ouilt\llitha'fo!'warder'.
`
`Ex.1008
`APPLE INC. / Page 1 of 35
`
`

`

`US 7,043,532 Bl
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5,392,033 A
`5,452,291 A
`5,488,412 A
`5,495,561 A *
`5,526,483 A *
`5,546,484 A
`5,561,709 A
`5,570,085 A *
`5,572,643 A
`5,579,308 A
`5,596,702 A *
`5,612,730 A
`5,636,211 A *
`5,657,221 A *
`5,740,362 A *
`5,778,226 A *
`5,790,789 A *
`5,826,000 A
`5,860,010 A *
`5,864,669 A *
`5,886,732 A
`5,909,183 A *
`5,940,072 A
`5,940,387 A
`5,953,526 A *
`5,956,487 A *
`6,005,861 A
`6,020,924 A
`6,032,202 A *
`6,037,933 A
`6,052,750 A
`6,078,783 A
`6,085,236 A
`6,101,499 A
`6,128,619 A *
`6,134,594 A *
`6,151,624 A *
`6,175,362 Bl
`6,181,333 Bl
`6,182,094 Bl
`6,188,397 Bl
`6,189,019 Bl*
`6,191,781 Bl
`6,198,479 Bl*
`6,288,716 Bl *
`6,349,352 Bl *
`
`2/1995 Oman et al.
`9/1995 Eisenhandler et al.
`1/1996 Majeti et al.
`2/1996 Holt .......................... 358/1.15
`6/1996 French et al ................... 714/4
`8/ 1996 Fling et al.
`10/ 1996 Remillard et al.
`10/1996 Bertsch ..................... 340/3.54
`ll/ 1996 Judson
`11/1996 Humpleman
`1/1997 Stucka et al. ............... 345/746
`3/ 1997 Lewis
`6/1997 Newlin et al. .............. 370/465
`8/1997 Warman et al. ............... 700/83
`4/1998 Buickel et al .............. 709/201
`7 / 1998 Adams et al.
`.............. 709/311
`8/ 1998 Suarez ....................... 709/202
`10/ 1998 Hamilton
`1/1999 Attal .......................... 717/137
`1/1999 Osterman et al ............ 709/203
`3/1999 Humpleman
`6/ 1999 Borgstahl et al. .. ... . 340/825 .22
`8/1999 Jahanghir et al.
`8/1999 Humpleman
`9/1999 Day et al .................... 717/108
`9/ 1999 Venkatraman et al. ...... 709/218
`12/1999 Humpleman
`2/2000 Jahanghir
`.. ... .. ... ... ... ... ... 710/8
`2/2000 Lea et al.
`3/2000 Blonstein et al.
`4/2000 Lea et al.
`6/2000 Kawamura et al.
`7/2000 Lea
`8/2000 Ford et al.
`............ 707/102
`10/2000 Fogarasi et al.
`10/2000 Helland et al .............. 709/229
`11/2000 Teare et al .................. 709/217
`1/2001 Harms et al.
`1/2001 Chaney et al.
`1/2001 Humpleman et al.
`2/2001 Humpleman
`2/2001 Blumer et al. .............. 707 /513
`2/2001 Chaney et al.
`3/2001 Humpleman et al ........ 345/733
`9/2001 Humpleman et al ........ 345/733
`2/2002 Lea ... ... ... ... .. ... ... ... ... .. . 710/72
`
`6,466,971 Bl* 10/2002 Humpleman et al ........ 709/220
`6,539,422 Bl *
`3/2003 Hunt et al .................. 709/217
`6,546,419 Bl*
`4/2003 Humpleman et al ........ 709/223
`6,560,639 Bl *
`5/2003 Dan et al .................... 709/218
`6,618,764 Bl*
`9/2003 Shteyn ....................... 709/249
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`EP
`EP
`EP
`EP
`EP
`EP
`EP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`WO
`WO
`WO
`WO
`WO
`
`90305213.2
`91401245.5
`96304706.3
`96306507.3
`96307200.4
`96116873.9
`97100356.1
`97117812.4
`9-261355
`9-282263
`10-145773
`11-88406
`11-194987
`11-317756
`11-355294
`11-355357
`PCT/EP95/00191
`PCT/US95/00354
`PCT/US95/17108
`PCT/US96/18798
`PCT/US97/08490
`
`5/1990
`5/1991
`6/1996
`9/1996
`9/1996
`10/1996
`1/1997
`10/1997
`10/1997
`10/1997
`5/1998
`3/1999
`7 /1999
`11/1999
`12/1999
`12/1999
`1/1995
`11/1995
`12/1995
`11/1996
`5/1997
`
`OTHER PUBLICATIONS
`
`Deng, Shuang "Capture effect in residential Ethernet LAN",
`IEEE GLOBEC, ISBN: 0-7803-2509-5, pp. 1678-1682,
`Nov. 1995.*
`Kokubun, T. et al. "Object-oriented database system with
`GIS for optical cable network operation", IEEE GLOBEC,
`ISBN: 0-7803-3336-5, pp. 1521-1527, Nov. 1996.*
`Corcoran, P.M. et al. "Browser-style interfaces to a home
`automation network", IEEE Trans. on Consumer Electron(cid:173)
`ics, pp. 1063-1069, Jun. 1997.*
`Corcoran, P.M. "Mapping home-network appliances to TCP/
`IP sockets using hree-tiered home architecture", IEEE Trans
`on Consumer Electronics, pp. 729-736, Jun. 1998.*
`
`* cited by examiner
`
`Ex.1008
`APPLE INC. / Page 2 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 1 of 14
`
`US 7,043,532 Bl
`
`, s
`
`,z.
`
`BaildJq Blodcl or Modules
`
`Ac,. t
`
`CLIENT
`
`SERVEl\
`
`I I--------.}-~ --1 Et::
`™<N ~ Lm
`~ E 11--------c ... '_"' ___ , Et 1
`10 -
`
`0"
`
`
`
`"
`
`SERVY.R
`
`I i..f
`
`.____
`SERVD
`
`1.,0
`
`Maldple Client Multiple Server Bame Network Control Model
`CLIENTS
`SERVERS
`
`SERVERS
`
`h6,_ 3
`
`Access Network
`
`Acc:eu Netwark
`
`D-
`VCR
`
`DVD
`p
`
`I &/
`
`I '1
`
`CableM
`AccessD
`
`141
`
`I '1
`1i.f r
`
`,.,
`
`DBS
`AccesaD
`
`Toaas
`
`Smart
`Card
`
`PC
`
`l l-
`
`DTV- -~
`I c./
`
`b/w
`
`1
`
`1'1...
`
`DTV -~
`l
`11./
`
`b./w
`
`-
`
`'z.
`
`PDA
`
`11..
`
`1(p
`
`B me Network
`
`Ex.1008
`APPLE INC. / Page 3 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 2 of 14
`
`US 7,043,532 Bl
`
`457
`
`GUI
`
`HUMAN
`
`Control System
`
`Commanu
`• • Communicadam • •
`
`zo
`
`CLIENT
`
`Service
`Control
`Pntgram
`
`SERVER
`
`1'4
`
`Commaad:I
`- • Commumcatiom - -
`
`Jt)
`
`Modale EJcmeaa GUI Co11trol
`
`2, 0
`
`Service
`Control
`PrOP"Bm
`
`SERVER
`
`20
`
`Sentcc
`Control
`Proiram
`
`SERVER
`
`GUI Actions/ Responses
`
`CLIENT
`
`I 2.
`
`GUI
`R.enduer
`
`L1-
`
`Control
`Procram
`C4atroJ
`State
`
`.,,._ ---
`
`•Graphic
`: Conttol
`I Object
`
`SERVER
`
`..., ' , - GCO Transfer Communication
`
`-----------------
`
`.,.
`,,
`- , .,,. .,
`
`z.o
`
`z.O
`
`Caacrol
`Prop-am
`Coatrol
`Slate
`
`Cock
`
`2,. lci
`
`3o Modale Elements for S«ver/Sener CL
`CL
`Actions / Raponses
`
`Command
`Langaaee
`laterfac:e
`
`Commaud
`Lanpap
`Iaterface
`
`30
`
`Coll1r0•
`Proo-am
`Coatrol
`Stace
`
`Clock
`
`• t.(
`
`SERVER
`
`2.0
`
`2.C.
`
`SERVER
`
`\~
`
`,s
`
`Ex.1008
`APPLE INC. / Page 4 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 3 of 14
`
`US 7,043,532 Bl
`
`lloxa wffla Cliell'tl
`DTV(STB)
`r -
`-
`-,
`I Humaa I CLlENT
`Control
`I GUI
`l
`
`' - - -~ l " l -Fl SINK
`
`~SERVER
`1'1
`
`PC
`r -
`-
`,
`Buman
`I Cantrui I
`I GUI
`I
`L---~,1-
`
`CUENr
`
`~~, -=
`L; 7"&n1a k.o'f
`
`1.. i
`I
`1 . . - - . J
`
`Bo:r.es with Server(a)
`AUDIO
`AMPLD"lf'T!Tr'IE-R
`
`SINK
`SERVER.
`
`DVCR
`
`LJ SOURCE OR SINK
`
`SERVER
`\ '{
`
`,'-{
`
`BOX MANAGEMENT
`SERVER
`
`PDA (RemoteC)
`r -
`-
`.,
`I Haman I CLIENT
`Cantrol
`! GUI
`I
`L---1
`{ "2...
`
`DVDP
`Smart Card
`
`SER.VER
`
`□ SOURCE
`
`) "\
`
`LJ l'-{
`LJ
`
`DBS Access Device
`Cable TV Access Device
`ATSC Access Device
`Intemet Access Device
`
`REMOTE SERVERS
`
`r -
`-
`,
`I Service L
`r--
`,
`I
`I Service L
`r -
`,
`L. 7
`I Service h I ~
`Li
`I
`'- - - ~
`
`Ex.1008
`APPLE INC. / Page 5 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 4 of 14
`
`US 7,043,532 Bl
`
`Buie I.JV, ClleatJSffver/Sernr, Model
`r-- ✓ rz ..
`I Buman I OJENl'
`C ntral
`I
`I GUI
`,r-f .J
`Control
`I Action/RespoDsc
`
`-
`
`-
`
`-
`
`Control
`A c t i o ~ - -
`
`---
`
`-
`
`Data transfen
`
`HUMAN
`Session Manager
`Input ~
`I serv1 I I serv2 I
`: -ssrvS -: : · sarv4 · :
`
`~
`
`/2..
`
`16
`CLIENT (eg 1 /2 DTV)
`
`/
`
`/
`
`fetch control
`object (GUI)
`I
`I
`I
`I
`I
`
`Serv1
`contrl
`object
`
`Serv2
`contrl
`object
`
`.,,. _,,. -
`
`-
`
`-
`
`'
`
`,
`
`fetch control
`' object (GUI)
`'
`\
`
`\
`\
`I
`I
`
`SERVER 2
`SERVER 1
`(data sink
`(data source
`eg 1/ 2 DTV)
`DATA
`eg DVCR)
`~-State-~ ~ cia-c"k- ~ . . - - - - - - - - - - -~ f-sfata-: ~-Clock~
`and Control
`--~~~t;;,_J-
`--~~t;; •. J-· 2
`2 ~
`
`:
`
`:
`
`I
`
`I
`
`I
`
`t
`
`I
`
`I
`
`~
`
`10
`
`I '-t
`
`Ex.1008
`APPLE INC. / Page 6 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 5 of 14
`
`US 7,043,532 Bl
`
`Table 1: Capabilities Table
`
`Table 2: Attributes Table
`
`Ex.1008
`APPLE INC. / Page 7 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 6 of 14
`
`US 7,043,532 Bl
`
`-
`
`Faacdoaal
`Spedfiatioa
`ofSenice
`
`Malage
`Composition
`
`M.eaage eoastnictloa ued by CAL and AV/C
`yt.\
`
`... , 1-
`
`Applayer< 1
`
`Industry
`Standard
`Format
`
`Messap
`Compressio11
`
`Masap
`String
`Construction
`
`IIF
`
`(..\(g
`
`<; i
`
`Message coastnction of new XCF.IXML
`
`{': l,. IJ
`
`~o
`
`~~
`
`Fuactional
`Speciftcadoa
`olServiee
`
`Iadllltry
`Standard
`Format
`(vocabulary)
`
`•. p-
`
`Message
`Composition
`
`App layer<
`
`> HTTP Ll layer
`
`Maap
`String
`Coastnctlon
`
`Message
`CompRSSion
`
`I/F
`
`A
`
`HUMAN
`◄-
`
`' 2..
`
`B
`
`SERVICE
`AP PU CATION
`
`C
`
`'-18
`
`'-f- lP
`
`Client Server Models Usina Web and XML
`
`SERVICE
`
`APPLICATION s
`
`App
`
`'2.0
`
`◄- ____ HTML or XML
`
`Commands/XML
`
`fie,. ,,J
`
`Ex.1008
`APPLE INC. / Page 8 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 7 of 14
`
`US 7,043,532 Bl
`
`t'a.f
`
`XML API black dla&nm
`
`SERVICE (or device) B
`
`CE.APl'sie
`Objcas, Mcdlads. Psamctas.
`C L.aaguqe (XCE ddlaidoa) rep=- __ :su!Met__, ot
`~
`
`C>.U..DTD
`.
`(ME"l'HODREQUEST)
`Doc:umcat Typa Oetiniriaii
`ta mab: Mcthaci ca!b In
`:oe..
`
`l,.o
`
`SERVICE (or device) A
`
`CE.APr,ic
`Objeas, Methods, Psrmm:tms.
`C Llagvap QCCE di:t!aitian) cep(cid:173)
`nmmmloa
`
`INTERFACE.D'TD
`Doaanent 1yi,e Demlition
`to maim XML Receive illo:(cid:173)
`tilcc far Mcdiods Calls
`
`hlsmtcc of scrvic:c or dcviGo
`IN'TERFACl!-A.XML
`(fer fflidlt.y cbci:ting)
`(.ma Cacililililfc:s file)
`
`APP
`B
`
`C Mcauge TO XML
`LOOK-UP TAB~ tiir
`RTJN•TIME
`
`H/W
`
`t,_O
`
`XML oa NETWORK
`
`XMLTOCM-o,
`LOOK-UP TABl.11: for
`IWN-11M£
`
`APP
`A
`
`Transfer and Learn forelp IN'l'ERFACE-INSTANCE.XML
`De,..,,'tJl.A
`
`TRANSFER XML OBJECT
`
`Ouertes at foreign
`- - - App A Interface by App B
`~..;....,;;,..;;;;.;..;;;~,;"-,,.-,..._,
`
`XML QUERY
`
`Ex.1008
`APPLE INC. / Page 9 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 8 of 14
`
`US 7,043,532 Bl
`
`'
`
`Library (cases 3,4)
`!-·-·---· ··-·-····-·-· ···-----·-·-·····--·-·--···-·-······-··· ·-·-·--···:
`,-----
`~~~
`i l·:~lf~~-~~~1~~11 AP~~~~~CE I r~~~~;;:l~~1 l
`l I Description{s) I ! Oescription(s)
`i Oescription(s)
`i I
`. _ ! al Ap~;catlon(C1) j ~. :• ~lcl1kl~ j al Appication(C2) j j
`
`~ Quary
`lntarface
`C--3
`
`API
`Transfer
`Cue-3
`
`•••
`
`•••- •• • •• I
`
`NETWORK
`
`i
`
`CONTROLLER
`
`I•••••••-•---'" -•••-- •-••••
`
`: ..
`
`CONTROLLED
`, ....... ·---·-----------··-···--·--·-
`,!:a1-f
`, - - - - - - - - , '
`'
`I ;
`l
`(A-mcved}
`AP!
`\
`{A)
`I APP. INTERFACE I l
`·. Case-2 ·APP.INTERFACE
`Trans1arcas.2 i
`Ii:
`, Description(s)
`r I of Appucation(C)
`.-·-·-·---·;-----~ Description(s)
`!
`t
`: cf Applicaticn(C) , ;
`I ·-·-·-·-·-·-·-·-· )
`I
`
`zo
`
`'
`
`Ac:ion
`
`~esoonse:
`
`1Ackn~
`
`Ll /:·
`I-,
`
`i
`..... -·. -----·---·····-··. -----------···· ........................... .
`'
`
`'
`
`2.0
`
`HAR~~ARE I !
`I!
`1 ;
`
`OR
`SERVICE
`
`Case 1. XML Application-C INTERFACE (A) for remote query by (E)
`Case 2. Move XML Application-C INTERFACE (A) to controller for local query by (E)
`Case 3. Move XML Application-C INTERFACE (A) to 3rd party device -a collecting
`place (librar1) for all Interfaces for remote query by (E). The library would have to
`have the address (URI) of tt,e associated application available for direct control
`action and responses.
`Case 4. Is case 3 but ~he 'indirect' control action (and response) is also directed at
`the library device. In U,is case the library is built with a 'forwarder'.
`
`Ft(,. 11-
`
`Ex.1008
`APPLE INC. / Page 10 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 9 of 14
`
`US 7,043,532 Bl
`
`XCE/XML Protoeoi Stack
`
`r -App.otigin•llyiiiAWC- , ... ---
`-teonv=tiiF-
`1- -
`- -.- ..... -
`-
`App. onginaily m CAL
`ID XCEIXML
`
`~-------j
`I
`
`Appllcadan In XCE for XML iDterface
`
`-
`
`-
`
`---
`I
`'· ~ XML
`+XML
`,o ~---
`
`1 XCE deflni-
`I
`tion
`I
`___ J
`CALLJ>m
`I
`
`h..C..B
`I XCEdefini-
`- --7
`tion
`I
`INTER-
`I
`FACE.DTD
`I
`
`I
`
`Layer
`JN
`
`Layer
`OUT
`
`BTTPDl.1
`WebServerLayer
`IP Networil Layer
`
`Physiwl..ayer
`
`<,,b? • . .
`
`Commun1canon
`Stack
`
`--
`
`Command Tramlatlaa Process.
`
`/'
`t-------RECEIVER.-------1
`
`Command
`Language
`intcrprctcr
`
`Applic:a(cid:173)
`uon
`
`Convert
`
`Service
`Software
`
`Driven
`
`DfftceBaniware
`
`Service
`or Device
`S/W,H/W
`Stack
`
`SENDER.
`
`, -i. O
`
`Command
`Origimm,r
`
`1"'
`I I
`I I
`f ..J.
`
`Tramlatc
`i.ntcnally.-
`
`IP Home
`NetWOrk
`
`TRANSLATION
`SER.VER.
`-· or fulfill ttllDSJarion over the Inti:met.
`
`Ex.1008
`APPLE INC. / Page 11 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 10 of 14
`
`US 7,043,532 Bl
`
`HN De\ice A: Controller Module
`
`HN De>ice B: Controlled Module
`
`Hardware Service
`
`Handlers
`
`XML RPC Action
`
`XML RPC Response
`
`CALL.OTO describes the message format of XMLRPC output. It can be used by receiver of
`XMLRPC message for validity check.
`
`Io -
`
`Ex.1008
`APPLE INC. / Page 12 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 11 of 14
`
`US 7,043,532 Bl
`
`Relation between Interface Library & XCE Database
`
`Standardiied CE
`Standardized
`Interface Document
`Definition of
`Type Definition
`Objects, M:thods,
`(010)
`Parameter.i
`Standardized XCE Database
`
`................................ i
`tof3 /' =~-.- <>mom.
`
`1.
`
`Manufacturert
`
`Hom,
`Nenwrk
`
`I
`
`__ ..._ __ , - - - - - - -1 , - - - - - - - ,
`I
`11
`Dc'Vicc XML
`Device XML
`I
`I I
`I
`lnterfilce 3
`I I
`lnteriace 2
`____ _,II ____ _. I
`__ T ________ T _ _ _ _
`
`•
`
`•
`
`•
`
`, - - - - - - - ,
`I
`I Device XML
`Interlace N
`I
`I - - - - . . . -
`----T--
`
`I
`1
`, "l. RcgistclfUpdaleln1crtaccf- - -
`
`- -
`
`-
`
`-
`
`- -
`
`-
`
`- -
`
`- -
`
`-
`
`I
`
`7"2.
`
`,----- -----·- - ---
`1/P r-:_ Gener~g ~oftwo.r,z. ot"t~ 1
`r----------------I----------------,
`:
`I
`I
`HN IL (Home Network lnterfa:e Library)
`__,
`................. .:
`'
`r
`,~~ J _________________________________ _
`,~~~m
`
`1
`
`Interface Library
`
`~
`
`!
`
`Relation between Interface Library & XCE Database
`
`Ex.1008
`APPLE INC. / Page 13 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 12 of 14
`
`US 7,043,532 Bl
`
`HN Middleware
`Layer
`
`HNORB&ll...
`
`Connected to Internet
`
`RF
`
`Powerlire
`
`Device 8 Interface
`
`Device
`Interface
`-----------------------·----·---- ll ,e, _________ _
`
`Example of Hierarchical IL & Device Interfaces
`
`Links, e.g.
`XLinkor
`XPointer
`
`HN
`Interface
`
`Ex.1008
`APPLE INC. / Page 14 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 13 of 14
`
`US 7,043,532 Bl
`
`Hierarchical Interface Structure. Four layers.
`
`,-
`
`...I
`/
`
`La)'cr
`I
`2
`
`3
`4
`
`lnlcrfacc·E:rnmpl~ --
`Interface La~ er
`XML interface for each HN, listing cWTent available devices.
`MyHN (VCRs, TVs ... )
`General XML interfaces for each device, listing function
`General VCR Interface .
`categories.
`(Record, play ... )
`Specific XML interface for each function category for a device. VCR.record category.
`Specific XML interface for each function in a function category. VCR.record.tdr function
`
`h ~- 2..'2..
`
`Ex.1008
`APPLE INC. / Page 15 of 35
`
`

`

`U.S. Patent
`
`May 9, 2006
`
`Sheet 14 of 14
`
`US 7,043,532 Bl
`
`Table 3: Example Formats and Types
`
`Type
`
`command
`
`•
`image
`
`language
`
`-
`video 5tleam
`-
`video clib
`
`text
`
`audio
`
`application
`
`Fonnat
`CAL, AV/C, X-10
`
`jpeg, ~ bitmap,
`tiff
`
`EngJish,F~
`etc ..
`
`mpcg2
`
`a.vi, quicktime,
`mpcg
`
`html, plain
`wav, aiff'
`
`msword,pdf,
`postscript, gzip
`
`Ex.1008
`APPLE INC. / Page 16 of 35
`
`

`

`US 7,043,532 Bl
`
`1
`METHOD AND APPARATUS FOR
`UNIVERSALLY ACCESSIBLE COMMAND
`AND CONTROL INFORMATION IN A
`NETWORK
`
`CROSS-REFERENCES TO RELATED
`APPLICATIONS
`
`2
`manufacturers have developed control units for controlling
`and commanding their home devices from a single interface.
`One drawback associated with using the remote control
`unit to command and control home devices is that it provides
`5 static and command logic for controlling and commanding
`each home device. Another drawback associated with using
`remote control units is that known remote control units
`cannot control a plurality of diverse devices, and more
`particularly, cannot control a plurality of devices having
`10 different capabilities to communicate with each other in
`order to accomplish tasks or provide a service.
`In conventional network systems a user provides com(cid:173)
`mands using a remote control unit or device control panel.
`Once the user ceases, there is no controller unit or device in
`the network to provide commands for automatic operation.
`After a user initially controls and commands a first set of
`devices, conventional systems do not provide a mechanism
`for the first set of devices to automatically communicate
`with a second set of devices in the network as necessary in
`order to accomplish tasks without direct user control and
`command of the second set of devices. Further, conventional
`systems do not provide an efficient method for various
`network devices to obtain information about other network
`devices in the network for command and control.
`There is, therefore, a need for a method and a system
`which provides dynamic control and command devices in a
`home network. There is also a need for such a method and
`system to provide the ability to control a plurality of diverse
`devices having different capabilities to communicate with
`each other in order to accomplish tasks or provide a service.
`There is also a need for such a method and system to provide
`the ability for various network devices to automatically
`command and control other various network devices. There
`is also a need for such a method and system to provide
`universally accessible command and control information for
`inter-device communication.
`
`Applicant claims the benefit of U.S. Provisional Applica(cid:173)
`tion No. 60/084,578 entitled "Command and Control Using
`XML", filed on May 7, 1998 which is incorporated herein by
`reference. The present application is related to the following
`copending applications that are commonly assigned and
`which are incorporated herein by reference: U.S. patent
`application Ser. No. 09/104,299, entitled "Browser Based 15
`Command and Control Home Network" now U.S. Pat. No.
`6,288,716; U.S. patent application Ser. No. 09/104,297,
`entitled "Method and Apparatus for A Home Network Auto(cid:173)
`Tree Builder"; U.S. patent application Ser. No. 09/104,298,
`entitled "Improved Home Network, Browser Based, Com- 20
`mandand Control" now U.S. Pat. No. 6,198,479; U.S. patent
`application Ser. No. 9/103,469, entitled "Method and Appa(cid:173)
`ratus for Creating Home Network Macros" now U.S. Pat.
`No. 6,243,707; and U.S. patent application Ser. No. 09/104,
`606, entitled "Progranmiing Tool For Home Networks now 25
`U.S. Pat. No. 6,182,904.
`
`FIELD OF THE INVENTION
`
`The present invention relates to the field of network
`systems, and more particularly, to home network having
`multiple devices connected thereto.
`
`BACKGROUND OF THE INVENTION
`
`30
`
`35
`
`A network generally includes a communication link and
`various devices with communication capability connected to
`the communication link. The devices include computers,
`peripheral devices, routers, storage devices, and appliances
`with processors and communication interfaces. An example 40
`of a network is a home network for a household in which
`various devices are interconnected. A usual household can
`contain several devices including personal computers and
`home devices that are typically found in the home. As such
`the term "device" typically includes logical devices or other
`units having functionality and an ability to exchange data,
`and can include not only all home devices but also general
`purpose computers. Home devices include such electronic
`devices as security systems, theater equipment, TVS, VCRs,
`stereo equipment, and direct broadcast satellite services or
`(DBSS), also known as digital satellite services (DSS),
`sprinkler systems, lighting systems, micro waves, dish
`washer, ovens/stoves, washers/dryers, and a processing sys(cid:173)
`tem in an automobile.
`In general, home devices are used to perform tasks that
`enhance a homeowner's life style and standard ofliving. For
`example, a dishwasher performs the task of washing dirty
`dishes and relieves the homeowner of having to wash the
`dishes by hand. A VCR can record a TV program to allow
`a homeowner to watch a particular program at a later time. 60
`Security systems protect the homeowner' s valuables and can
`reduce the homeowner's fear of unwanted entry.
`Home devices, such as home theater equipment, are often
`controlled using a single common control unit, namely a
`remote control device. This single common control unit 65
`allows a homeowner to control and command several dif(cid:173)
`ferent home devices using a single interface. Thus, may
`
`SUMMARY OF THE INVENTION
`
`The present invention satisfies these needs. In one
`embodiment the present invention provides a method and
`system for performing a service on a home network, by:
`connecting a first and a second home device to the home
`45 network; providing a database including a plurality of appli(cid:173)
`cation interface description data objects, where each appli(cid:173)
`cation interface description data object includes information
`in a structured format for commanding and controlling of a
`home device by one or more other home devices connected
`50 to the network; the second home device accessing a first
`application interface description object for the first home
`device in the database; the first home device accessing a
`second application interface description object for the sec(cid:173)
`ond home device in the database; sending control and
`55 command data from the first home device to the second
`home device utilizing the second application interface
`description object over the network; and sending control and
`command data from the second home device to the first
`home device utilizing the first application interface descrip(cid:173)
`tion object over the network. Whereby, the first and second
`home devices perform said service.
`In one version of the invention, the first home device
`stores first application interface data therein, and the second
`home device stores second application interface data therein.
`The database is formed by querying the first and second
`home devices to transfer said application interface data for
`the first and second home devices to the database device.
`
`Ex.1008
`APPLE INC. / Page 17 of 35
`
`

`

`US 7,043,532 Bl
`
`3
`The database can be stored in a database device and con(cid:173)
`nected to the network for universal access by network
`devices. In that case, the first application interface descrip(cid:173)
`tion object for the first home device can be provided from
`the data base to the second home device over the network. 5
`Further, the second application interface description object
`can be provided from the data base to the first home device
`over the network.
`Further, three or more home devices can be connected to
`the network, wherein at least one home device accesses the 10
`database to query the application interface description
`objects of a plurality of home devices for sending command
`and control data to the plurality of home devices over the
`network. Each application interface description object can
`include data in a structured format. The structured format 15
`can include XML format.
`
`4
`FIG. 19 shows another embodiment of server device to
`server device command and control architecture;
`FIG. 20 shows the relationship between a device interface
`library and consumer electronics definition data base for
`home devices;
`FIG. 21 shows hierarchal form of an embodiment of a
`device interface definition;
`FIG. 22 shows an example of layers in device interface
`definition of FIG. 21;
`FIG. 23 shows a command transmission and interpreta(cid:173)
`tion process between a sender and receiver device; and
`FIG. 24 shows an example table of a partial list of packet
`types and formats for providing translation services accord(cid:173)
`ing to an aspect of the present invention.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`25
`
`45
`
`In one aspect, the present invention provides inter-device
`These and other features, aspects and advantages of the 20 communication in a network such as a home network. As
`home devices become more intelligent and can share infor(cid:173)
`present invention will become better understood with regard
`mation, inter-device communication allows a user to inter(cid:173)
`to the following description, appended claims and accom(cid:173)
`connect devices in a network to take advantage of the
`panying drawings where:
`information sharing capabilities of those devices. As such,
`FIG. 1 shows a block diagram on an embodiment of a
`inter-device communication plays a crucial role in affording
`network according to one aspect of the present invention;
`a user with the ability to fully and flexibly utilize the
`FIG. 2 shows the block diagram of FIG. 1 in an example
`networked devices.
`device control and communication scenario;
`Referring to FIG. 1, in an embodiment of the present
`FIG. 3 shows a block diagram of an example home
`invention, a network 10 includes at least one client device 12
`network system according to the present invention which
`30 and at least one server device 14 interconnected via a
`includes a plurality of client and server devices;
`communication link 16. The communication link 16 can
`FIG. 4 shows a block diagram of example embodiments
`include a 1394 serial bus providing a physical layer (me(cid:173)
`of a client device and a server device of FIG. 3;
`dium) for sending and receiving data between the various
`FIG. 5 shows example embodiments of client devices;
`connected home devices. The 1394 serial bus supports both
`FIG. 6 shows example embodiments of server devices;
`FIG. 7 shows a block diagram of two example networked 35 time-multiplexed audio/video (AN) streams and standard IP
`(Internet Protocol) communications. In certain embodi(cid:173)
`server devices capable of communication with, and control
`ments, a home network uses an IP network layer as the
`of, one another;
`communication layer for the home network. However, other
`FIG. 8 shows a block diagram of an example architecture
`communication protocols could be used to provide commu-
`of an audio/video (AN) model including examples of a
`source server device, a sink server device and a client device 40 nication for the home network.
`Each client device 12 may communicate with one or more
`in a network;
`server devices 14 in the network 10. Further, each server
`FIG. 9 shows another example audio/video (A/V) model;
`device 14 may communicate with one or more other server
`FIG. 10 shows an example capabilities data table for a
`devices 14, and one or more client devices 12, in the network
`network device;
`10. Each client device 12 can include a user communication
`FIG. 11 shows an example attribute data table for a
`interface including input devices such as a mouse and
`network device;
`keyboard for receiving user input, and a display for provid(cid:173)
`FIG. 12 shows an example configuration of building
`ing a control user interface for a user to interact with the
`blocks for generating command messages among networked
`networked devices. The user interface can include a graphi-
`devices;
`50 cal user interface (GUI) display 18 for providing informa(cid:173)
`FIG. 13 shows another example configuration of the
`tion to the user. Referring to FIG. 2, as defined herein, each
`building blocks of FIG. 12 for generating command mes(cid:173)
`server device 14 provides a service for the user, except
`sages;
`control user interface, and each client device 12 provides
`FIG. 14 shows three examples of interaction among
`control user interface for user interaction with the network
`networked client and server devices;
`55 10. As such, only client devices 12 interact directly with
`FIG. 15 shows an example block diagram for definitions
`users, and server devices 14 interact only with client devices
`of API extensions of networked device interfaces;
`12 and other server devices 14. Example services can
`FIG. 16 shows an example architecture for a server device
`include MPEG sourcing/sinking and display services.
`application accessing the interface description document of
`FIG. 3 shows a block diagram of an example home
`another server device;
`60 network 10 that includes a plurality of client devices 12 and
`FIG. 17 shows another example inter-device control
`a plurality of server devices 14. Each server device 14 may
`architecture between a controller server device and a con(cid:173)
`include hardware as a resource in the network for providing
`trolled server device;
`services to the user. Further, each server device 14 may store
`FIG. 18 shows an embodiment of an XML protocol
`a server or service control program 20 for controlling the
`providing a Web standard common middleware layer in a
`65 server hardware, and include a graphical control object
`(GCO) user interface description 22 for user interface with
`communication stack at the API level between networked
`the server control program 20 as shown in FIG. 4.
`devices;
`
`Ex.1008
`APPLE INC. / Page 18 of 35
`
`

`

`US 7,043,532 Bl
`
`5
`For control between a controlling client device 12 and a
`controlled server device 14, the client device 12 accesses the
`GCO 22 of the server device 14 by, for example, transferring
`the GCO 22 from the server device 14 to the client device
`12 over the network. The client device 12 then uses the 5
`transferred GCO 22 to create a control user interface GUI 18
`for the user to communicate with the control program 20 of
`the server device 14 from the client device 12 over the
`network. The user provides command and control to at least
`the control program 20 of the server device 14 from the
`client device 12.
`Storing the GCO 22 of each server device 14 in the server
`device itself may reduce the processing and storage require(cid:173)
`ments of the client devices 12 in net