UT36709
`
`
`US0071
`
`(12) United States Patent
`US 7,136,709 B2
`(10) Patent No.:
`Arling et al,
`(45) Date of Patent:
`Nov. 14, 2006
`
`
`(54)
`
`(75)
`
`HOME APPLIANCE CONTROL SYSTEM
`AND METHODS IN A NETWORKED
`ENVIRONMENT
`
`Inventors: Paul D. Arling, Irvine, CA (US);
`Jeremy Black, Santa Ana, CA (US);
`Christopher Chambers, Stanton, CA
`(US); Mauro Dresti, West Covina, CA
`(US); Patrick H. Hayes, Mission Viejo,
`CA (US); Robert Lilleness, Cypress,
`CA (US); Wayne Scott, Newport
`Beach, CA (US); Allen Yuh, Diamond
`Bar, CA (US)
`
`(73)
`
`Assignee: Universal Electronics Inc., Cypress,
`CA (US)
`
`Notice:
`
`Subject to anydisclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 18 days.
`
`(21)
`
`Appl. No.: 10/978,860
`
`(22)
`
`Filed:
`
`Nov. 1, 2004
`
`(65)
`
`Prior Publication Data
`
`US 2005/0096753 Al
`
`May 5, 2005
`
`Related U.S. Application Data
`
`(60)
`
`Provisional application No. 60/517,283, filed on Nov.
`4, 2003.
`
`(51)
`
`Int. Cl.
`
`GOSB 19/18
`GOSB 15/02
`GO5B 11/01
`GO5B 23/02
`GO8B 1/08
`H040 7/00
`
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`
`(58)
`
`(56)
`
`Field of Classification Search............... 700/9-13,
`700/17, 19, 20, 65; 709/203, 224, 217-219,
`709/249, 250; 710/16, 8-11, 15-19, 105,
`710/305, 306, 311-315; 340/500, 531, 539.1,
`340/539.14, 3.1, 3.5
`See application file for complete search history.
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`4,894,789 A
`
`1/1990 Yee
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`0561435 A2
`
`9/1993
`
`(Continued)
`OTHER PUBLICATIONS
`
`Pronto UserGuide, Universal Electronics Inc., 1999, pp. 1-56.
`Primary Examiner—Crystal J. Barnes
`(74) Attorney, Agent, or Firm—Gary R. Jarosik
`
`(57)
`
`ABSTRACT
`
`A home control and automation system and method. The
`system generally includes one or more wireless remote
`controls,
`a plurality of appliances (including network
`enabled appliances and traditional legacy appliances in the
`exemplary system), associated control centric devices, and
`associated content centric devices. Various methods for
`control and operation of appliances in the networked home
`control and automation environmentare disclosed. Specifi-
`cally, methods are disclosed which enable advanced home
`control features such as location based control setup and
`operation, network enabled legacy appliances and system
`integration, save and recall capabilities for appliance and
`media states, generic command based appliance controls,
`dynamic/complex macro command generation, and Internet
`based control capabilities from remote locations are dis-
`closed in conjunction with the system of the current inven-
`tion.
`
`16 Claims, 6 Drawing Sheets
`
`(52)
`
`USS Cl sccumsscansnave: COOGS: TOO; FOOL:
`700/19: 700/20; 340/3.1; 340/539.14
`
`
`
`Roku EX1016
`U.S. Patent No. 9,716,853
`
`Roku EX1016
`U.S. Patent No. 9,716,853
`
`

`

`US 7,136,709 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`
`
`11/2000 Schein et al.
`6,151,059 A
`1/2001 Etheredge
`6,172,674 B
`1/2001 Alexander et al.
`6,177,931 B
`2/2001 Ketcham
`6,195,589 B
`4/2001 Choi etal.
`6.211.856 Bl
`4/2001 Lazaridis etal.
`6,219,694 B
`7/2001 Allport
`6,256,019 Bl
`8/200L Darbee etal.
`6,278,499 Bl
`9/2001 Kushiro et al.
`6,285,357 Bl
`1/2002 Schein et al.
`6341374 BL
`4/2002 Barnett et al.
`6,369,840 B
`4/2002 Shima etal.
`6,381,507 B
`6/2002 Sato
`6,408,435 B
`8/2002 Huanget al,
`6,437,836 Bl
`........ 704/270
`8/2002 Oberteuffer et al.
`6,438,523 BL*
`9/2002 Sharood et al.
`.............. 62/127
`6,453,687 BL*
`10/2002 Godfrey etal.
`6,463,463 Bl
`3/2003 Proehlet al.
`6,532,589 Bl
`3/2003 Kawajiri
`6,538,556 Bl
`5/2003 Kemink etal.
`6,563,430 Bl
`6/2003 Proehl etal.
`6,577,350 Bl
`6/2004 Howard ........:0: 345/158
`6,747,632 BL*
`6/2004 Davies etal.
`6,753,790 Bl
`9/2004 Krzyzanowski etal.
`6,792,323 Bl
`10/2004 Balasubramaniam et al.
`6,803,728 Bl
`2/2005 Park wc. sessscssseeeene 340/7.1
`6,850,149 Bl*
`» 340/538
`6/2005 Van der Meulen .
`6,906,617 BL*
`
`« 382/100
`....
`8/2005 Roberts et al.
`6,937,742 BL*
`
`.. 709/224
`11/2005 Diong 0.0...
`6,965,935 BL*
`
`5/2006 Dubrovskyetal.
`+ 709/225
`7,051,101 B1*
`6/2003 Ghulam .............c00008 345/173
`2003/0117376 Al*
`11/2003 Wilding et al.
`2003/0212788 Al
`2003/0233432 Al*® 12/2003 Davis et al.
`...........00 709/222
`
`7/2004 Krzyzanowski et al.
`2004/0133704 Al*
`.... 709/250
`2004/0203387 Al* 10/2004 Grannan ..
`455/412
`2005/0035846 Al
`2/2005 Zigmond etal.
`2005/0078088 Al*
`4/2005 Davis etal. oc... 345/163
`
`4/1991
`Skinner
`5,005,084
`3/1992
`5,101,191 /
`MacFadyen etal.
`4/1992
`5,109,222
`Welty
`3/1994
`Hallenbeck
`5,293,357
`4/1994
`Baskin et al.
`5,307,055
`Goldstein
`4/1995
`5,410,326
`9/1996
`Lenchik
`5,552,806
`Selker
`10/1996
`5,565,888
`Hamlin
`11/1996
`5,574,964
`Rothmuller
`6/1997
`5,635,989
`Small etal.
`6/1997
`5,642,303
`Kumar
`F/1997
`5,648,760
`Lazaruset al.
`F1997
`5,652,613
`9/1997
`5,671,267
`August et al.
`Nelson
`1/1998
`5,710,605
`3/1998
`5,724,106
`Autry et al.
`Ivie etal.
`3/1998
`§,727,055
`Forson
`5/1998
`5,751,372
`6/1998
`Wolzien
`5,761,606
`Lee et al.
`6/1998
`5,767,919
`Bedard
`8/1998
`5,793,438
`Schein et al,
`9/1998
`5,801,787
`9/1998
`Ivie etal.
`5,815,086
`LO/1998
`Bruette et al.
`5,828,419
`L1/1998
`Diehlet al.
`5,835,864
`LL/1998
`Montalbano
`5,838,775
`Huemoelleret al.
`12/1998
`5,855,006
`Haitanietal.
`5/1999
`5,900,875
`Nakanoet al.
`5/1999
`5,901,366
`6/1999
`Mankovitz
`5,915,026
`9/1999
`Gouldenet al.
`5,956,025
`Yuen et al.
`10/1999
`5,970,206
`Yuen et al.
`10/1999
`5,974,222
`Schein et al.
`12/1999
`6,002,394
`12/1999
`Darbee etal.
`6,002,450
`1/2000
`6.018.372
`Etheredge
`FOREIGN PATENT DOCUMENTS
`2/2000
`6,020,881
`Naughtonet al.
`Yuen et al.
`2/2000
`6,028,599
`0967797 A2
`12/1999
`EP
`3/2000
`6,040,829
`Croy et al,
`O987888 Al
`3/2000
`EP
`svssseeeesee 109/220
`LOG: sitccdinatiarss
`7/2000
`6,085,236
`
`EP 1204275 A2—$/2002
`8/2000
`6,097,441
`Allport
`GB
`2343073 A
`4/2000
`8/2000
`6,104,334
`JP
`2001209551 A
`$/2001
`10/2000
`6,127,941
`Wo
`WO 00/40016
`7/2000
`6,130,726
`10/2000
`WO
`WO 00/58935
`10/2000
`10/2000
`6,133,909
`>
`Wo
`WO 01/20572 Al
`3/2001
`6,137,549
`10/2000
`11/2000
`6,148,241 A
`
`Van Ryzin
`Darbee etal.
`Schein et al.
`Rassonetal.
`Ludtke etal.
`
`* cited by examiner
`
`PPPrrrrerrrrrrrreeeS
`
`
`
`Perrerrrrerrereereeeeeeerereeeee
`
`

`

`U.S. Patent
`
`Nov.14, 2006
`
`Sheet 1 of 6
`
`US 7,136,709 B2
`
`Translation /
`Command Passing
`Server
`
`Pa
`
`-
`
`----.
`
`~ ~
`
`x.
`
`ROOM 1
`
`—_——.
`
`\
`
`——_—
`
`~-
`
`Server /
`Content
`Source 1
`
`
`
`Networked
`Appliance 2
`Server / Content
`Source 2
`
`
`
`
`SeCERae12
`
`|NetworkedAppliance1|
`/
`‘\
`/
`\
`00.ab)
`/
`
`—
`
`!TM Cncetit
`NetworkedAppliance=
`/
`Contro|POD/
`
`12,16
`
`FIGURE 1
`
`

`

`U.S. Patent
`
`Nov. 14, 2006
`
`Sheet 2 of 6
`
`US 7,136,709 B2
`
`Power source
`
`28
`
`Transceiver
`
`Receiving
`¢
`circuit
`
`Key matrix
`Transmitting
`circuit
`
`33
`
`
`
`Non-
`volatile
`memory
`
`
`34
`
`5
`
`w8
`
`Clock and
`timerlogic
`
`27
`
`30
`
`24
`
`FIGURE 2
`
`38
`
`
`=.
`8
`
`2=
`
`Visual feedback
`device
`
`Audio feedback
`device
`
`36
`
`

`

`U.S. Patent
`
`Nov. 14, 2006
`
`Sheet 3 of 6
`
`US 7,136,709 B2
`
`55
`- Transmitter
`
`58
`
`T Power
`
`T
`
`62
`
`64
`
`_ i”
`
`C__)
`Crystal
`
`54
`
`56
`
`52
`
`51
`
`¥
`
`- Receiver
`a Nonwolatie
`
`memory
`
`ROM
`memory
`
`RAM
`
`memory
`
`5B:a 85=
`
`Clock and
`timer logic
`
`disible
`LED
`
`3
`
`53
`
`50
`
`FIGURE 3
`
`

`

`U.S. Patent
`
`Nov. 14, 2006
`
`Sheet 4 of 6
`
`US 7,136,709 B2
`
`70
`
`
` Virtual Network
`Layer
`Content
`IP Registry
`Delivery
`
`
`Control Manager
`
`Content Gateway
`
`1
`
`
`Content
`Sourcing
`
`FIGURE 4
`
`74
`
`

`

`U.S. Patent
`
`Nov. 14,2006
`
`Sheet 5 of 6
`
`US 7,136,709 B2
`
` Remote
`
`Device
`
`10
`
`
`
`
`
`Internet Based ie
`
`
`Server
`
`re
`
`
`Automation
`
`PC with Home
`
`FIGURE5
`
`

`

`U.S. Patent
`
`Nov. 14,2006
`
`Sheet 6 of6
`
`US 7,136,709 B2
`
`601
`
`Button press
`
`602
`
`Recieve User Input
`
`603
`
`604
`
`Text entry
`
`Gesture
`
`650go
`
`Control Pod
`
`
`
`
`
`
`
`Portable Device 620
`
`
`
`610
`
`Local and/or remote
`database(s)
`*
`Key codes
`*
`Key labels
`*
`Phonics
`Thesaurus/dictionary
`cross-referenced to
`action descriptors
`
`Normalize input
`
`Action descriptar
`
`100
`
`Select device
`(location and
`activity based)
`
`660
`
`Translate action
`and retrieve
`Local and/or remote
`
`command data,
`database
`
`based on action
`*
`Action descriptors
`
`descriptor and
`*
`Device type/model
`device
`cross-referenced to
`command codes
`
`
`Issue
` device
`com)
`662
`mand
`
`
`
`FIGURE 6
`
`

`

`US 7,136,709 B2
`
`1
`HOMEAPPLIANCE CONTROL SYSTEM
`AND METHODS IN A NETWORKED
`ENVIRONMENT
`
`RELATED APPLICATION DATA
`
`This application claims the benefit of U.S. Provisional
`Application Ser. No, 60/517,283 filed on Nov. 4, 2003,
`which application is hereby incorporated by reference in its
`entirety.
`
`BACKGROUND
`
`The following relates generally to home control systems
`and, more particularly, relates to a system and associated
`methods for controlling and operating a plurality of home
`appliances from a variety of different locations in a net-
`worked home control and automation environment.
`Current home appliances and associated remote controls
`(“legacy” appliances) have limited functional abilities and
`minimal
`interoperability features. Emerging advances in
`wireless and distributed computing technologies, coupled
`with network enabled home appliances (such as those that
`may be addressed, accessed, monitored, and managed
`remotely over networks and the Internet) and industry wide
`interoperability standards (e.g., Universal Plug and Play
`(“UPnP”) and Home Audio Video interoperability (““HAV1")
`standards) provide a rough framework for network enabled
`home environments having increasingly integrated home
`control capabilities, functions, and features. A background
`and overview of home interoperability technologies and
`projected features may be found in the Intel Technical
`Journal, Volume 6, Issue 4, published Nov. 15, 2002 entitled
`“Interoperable HomeInfrastructure” whichis fully incorpo-
`rated by reference herein.
`Though promising in their goals and objectives, the UPnP,
`HAVi, Intel Interoperability Infrastructure, etc. do not cur-
`rently provide for many advanced control based features
`relating to networked home control and automation envi-
`ronments. Accordingly, it is desired to provide a system and
`method that functions to enable advanced home control
`features such as location based control setup and operation,
`network enabled legacy appliances and system integration,
`save and recall capabilities for appliance and mediastates,
`generic command based appliance controls, dynamic/com-
`plex macro commandgeneration, and Internet based control
`capabilities from remote locations. Additional
`inventive
`features and functions will also be evident from the home
`appliance control system and methodshereinafter described.
`
`SUMMARY
`
`In accordance with these needs and desires, a control
`systemand associated methods for control and operationare
`described for use in a networked homecontrol and automa-
`tion environment.
`Generally,
`the system includes one or more wireless
`remote controls, a plurality of appliances (including network
`enabled appliances and traditional legacy appliances in the
`exemplary system), associated control centric devices (gen-
`erally devices whose chief function is providing control
`based functions and services, including but not limited to
`control pods, control servers, device state monitors, etc.) and
`associated content centric devices (generally devices whose
`chief function is providing content based functions and
`services,
`including but not
`limited to content managers,
`content servers, content renderers, etc.).
`
`1
`
`wa
`
`20
`
`30
`
`35
`
`45
`
`50
`
`60
`
`2
`Additionally, various methods for control and operation
`of appliances in the networked home control and automation
`environmentare disclosed. Specifically, methods that func-
`tion to enable advanced home control
`features such as
`location based control setup and operation, network enabled
`legacy appliances and system integration, save and recall
`capabilities for appliance and media states, generic com-
`mand based appliance controls, dynamic/complex macro
`command generation, and Internet based control capabilities
`from remote locationsare disclosed in conjunction with the
`systemof the current invention.
`A better understanding of the objects, advantages, fea-
`tures, properties and relationships of the subject system and
`methods will be obtained from the following detailed
`description and accompanying drawings which set
`forth
`illustrative embodiments whichare indicative of the various
`ways in which theprinciples of the system and method may
`be employed.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`For a better understanding of the system and method
`described hereinafter, reference may be had to preferred
`embodiments shown in the following drawings in which:
`FIG. 1 illustrates an exemplary networked home control
`and automation system;
`FIG, 2 illustrates a block diagram of components of an
`exemplary remote control;
`FIG. 3 illustrates a block diagram of components of an
`exemplary control device;
`FIG. 4 illustrates an exemplary network enablement
`device;
`FIG. 5 illustrates a VPN implemented with the networked
`home control environment; and
`FIG. 6 illustrates an exemplary system for implementing
`a generic command structure within a home appliance
`network.
`
`DETAILED DESCRIPTION
`
`With reference to the Figures, an integrated home control
`system and associated control and operation methods are
`described for use in a networked homecontrol and automa-
`tion environment.
`The system envisaged by the current invention provides
`for both the interoperability of various network enabled and
`legacy home appliances, and advanced command and con-
`trol functions for such appliances. To this end, the networked
`home control and automation system (shown generally in
`FIG, 1) includes one or more remote controls 10, a plurality
`of appliances 12 (including network enabled appliances and
`traditional
`legacy appliances in the exemplary system),
`associated control centric devices 14 (generally devices
`whose chief function, is providing control based functions
`and services,
`including, but not
`limited to, control pods,
`control servers, device state monitors, etc.) and associated
`content centric devices 16 (generally devices whose chief
`function is providing content based functions and services,
`including, but not
`limited to, content managers, content
`servers, content renderers, etc.).
`By way of example only, the appliances 12 may include,
`but are not
`limited to,
`televisions, VCRs, DVRs, DVD
`players, cable converter boxes, amplifiers, CD players, game
`consoles, home lighting, drapery, fans, HVAC systems,
`thermostats, personal computers, security systems, network
`enabled automobiles, etc. According to the exemplary sys-
`tem, some or all of the above devices, appliances, and
`
`

`

`US 7,136,709 B2
`
`4
`24 (a so called “microcontroller”) and, as such, they are
`shown separately in FIG. 2 only for the sake ofclarity.
`To identify home appliances by type and make (and
`sometimes model) such that the remote control device 10 is
`adapted to transmit recognizable command codes in the
`format appropriate for such identified appliances 12, data
`maybe entered into the universal remote control device 10.
`Since methods for setting up a remote control to control the
`operation of specific home appliances are well-known, such
`methods need not be described in greater detail herein.
`Nevertheless,
`for additional
`information pertaining to
`remote control setup, the reader may turn to U.S. Pat. Nos.
`4,959,810, 5,614,906, and 6,225,938. It will also be appre-
`ciated that the remote control 10 may beset up to control an
`appliance 12 by being taught the command codes needed to
`control such appliance as described in U.S. Pat. No. 4,623,
`887.
`
`1
`
`wa
`
`3
`components are configured to be network enabled and
`interoperable (e.g., as defined and described in the UPnP
`and/or HAVi specifications which can be found at
`the
`upnp.org Web site and the havi.org Website respectively,
`whichspecifications are incorporated herein by reference in
`their entirety) such that basic device addressing, accessibil-
`ity, monitoring, remote management, and other network
`communication based features are possible in a home con-
`trol and automation environment. The networked home
`control and automation environment 100 itself may utilize
`well knowncentralized networking methods, ad hoc net-
`working methods (including peer-to-peer networking), pow-
`erline or phoneline based networking methods, or a combi-
`nation of the above (whether wired or wireless)
`to
`accomplish the features and goals of the present invention.
`Additionally,
`it will be appreciated that particular appli-
`ances, devices, and components of the exemplary system
`may incorporate someorall ofthe features, functions, and
`capabilities of other individual devices such that a minimal
`numberofseparate physical devices are required in a given
`home environment to accomplish the goals of the current
`invention. For instance, devices such as a Media Center PC
`or network enabled TY(e.g., one compliant with the UpnP
`and/or HAVi specifications) may include,
`in additional to
`standard computing and/or audio/video playback functions,
`system wide control capabilities, content management and
`rendering abilities, and location based services. As suchit
`will be appreciated that the various system elements defined
`and described herein should be considered as functional
`modules capable of implementation individually or collec-
`tively in a single physical device or a sofiware module in a
`software product, or one or more elements may be imple-
`mented in separate physical devices or software modules,
`without departing from the scope and spirit ofthe present
`invention.
`
`Looking now to FIG, 2, for use in transmitting command
`codes, generic commands, macro commands, ete. to one or
`more ofthe appliances (both network enabled and legacy)
`the remote control 10 may include, as neededfor a particular
`application, a processor 24 coupled to a memory device
`(such as ROM memory 26, RAM memory 27, and/or a
`non-volatile memory 34), a key matrix 28 (e.g., physical
`buttons, a touchscreen display, or a combinationthereof), an
`internal clock and timer 30,
`transmission circuit(s) 32,
`receiver circuit(s) 33, and/or transceiver circuit(s) (e.g., IR
`and/or RF), a means 36 to provide feedbackto the user(e.g..
`LED,display, speaker, and/or the like), and a power supply
`38 as generally illustrated in FIG. 2. As will be understood
`by those of skill in the art, the memory device may include
`executable instructions that are intended to be executed by
`the processor 24 to control
`the operation of the remote
`control 10. In this manner, the processor 24 may be pro-
`grammed to control
`the various electronic components
`within the remote control 10, e.g.,
`to monitor the power
`supply 38, to cause the transmission of signals, etc.
`The non-volatile read/write memory 34, for example an
`EEPROM,battery-backed up RAM, Smart Card, memory
`stick, or the like, may be provided to store setup data and
`parameters as necessary. It is to be additionally understood
`that the memory devices may take the form of any type of
`readable media, such as, for example, ROM, RAM, SRAM,
`FLASH, EEPROM, Smart Card, memory stick, a chip, a
`hard disk, a magnetic disk, and/or an optical disk. Still
`further,
`it will be appreciated that some or all of the
`illustrated memory devices 26, 27, and 34 may be physically
`incorporated within the same IC chip as the microprocessor
`
`To cause the remote control 10 to perform an action, the
`remote control 10 is adapted to be responsiveto events, such
`as a senseduserinteraction with the key matrix 28, receipt
`ofa transmission, etc. In response to an event, appropriate
`instructions within the memory 26 may be executed. For
`example, when a command keyts activated on the remote
`control 10, the remote control 10 may retrieve a command
`s code corresponding to the activated command key from
`memory 26 and transmit the command codeto a device in
`a format recognizable by the device. It will be appreciated
`that the instructions within the memory 26 can be used not
`only to cause the transmission of command codes and/or
`) data to the appliances 12 but also to perform local opera-
`tions,e.g., location based features and functions as described
`in greater detail below. While not
`limiting, other local
`operations that may be performed by the remote control 10
`include displaying information/data, favorite channel setup,
`macro button setup, function key relocation, etc. Examples
`oflocal operations can be found in U.S. Pat. Nos. 5,481,256,
`5,959,751, and 6,014,092. Additional examples of remote
`controls 10 may be found in commonly owned, US. Pat. No.
`6,225,938 and U.S. application Ser. Nos. 60/264,767,
`09/905,423, 09/905,432, and 09/905,396.
`Remote control 10 may also include, as needed for a
`particular application, CCD or other imaging sensors, micro-
`phone, and/or touchscreen display for receipt of various
`types of user input for effecting operations of the remote
`5 control. Remote control 10 may also be configured to
`include appliance, control server, and content server func-
`tions, as will be described in greater detail below. For
`instance, remote control 10 may be implemented ona laptop
`computer, smart panel, tablet PC, wireless enabled PDA,
`) mobile phone, ete which mayall include programming to
`cause remote control 10 to playback music, videos, pictures
`and the like, monitor appliance feature and state tables for
`advanced control functions, and include programming for
`location determination methods. Remote control 10 may
`thus be viewed as a wireless appliance having functions
`similar to less portable appliances of the networked control
`environment, and as such the following discussions and
`descriptions of the remote controls, and controlling devices
`in general, should be taken in the broadest sense possible.
`Each network enabled appliance ofthe system(including
`legacy appliances that have been network enabled as
`described in greater detail below) include, or are assigned a
`unique address. According to the exemplary embodiment,
`this address is in the form of a Uniform Resource Identifier
`(URI) which can be any Uniform Resource Locator (URL)
`or Uniform Resource Name (URN), typically assigned via a
`control server of the system, such that all appliances in
`
`>
`
`40
`
`a
`
`60
`
`

`

`US 7,136,709 B2
`
`6
`remote control
`and state data for use with appliances,
`interfaces, and desired user functions. Preset control instruc-
`tions on control devices may dictate how and when com-
`mand and state data is captured and processed, or user
`interaction with an appliance or remote control device may
`dictate the manner of commandandstate data capture and
`processing. Saved and/or processed command and state
`playback may likewise be effectuated by preset instructions,
`automated system processes, or user interaction with appli-
`ances and devices of the system.
`Generally, the functions performed bya particular control
`device will depend on included firmware, software and
`programming, additional components, and level of integra-
`tion with other appliances of the system. As such the general
`control device as shown and described in FIG. 3 may be
`configured and modified as further described herein to
`function as a location signaling device, network enablement
`device, command/state monitor, control pod, network bridge
`(such as for example an IR, RF, X-10, or Simple Control
`Protocol (SCP) bridge), or any combination of the above.
`The control device may also be fully incorporated into any
`ofthe appliances of the home environmentas a functional
`element(i.e., as a service or application software running on
`the appliance).
`For use in accomplishing one or more of the above
`described contro] features in the networked control and
`
`Lt
`
`wa
`
`20
`
`rm ws
`
`a
`communication with the control server may be accessed
`fromat least the control server. It will be appreciated that
`multiple appliances may include control and addressing
`functions in a given system such that each individually
`addressed appliance may be accessed by multiple control
`servers, control pods, or other control devices. Additionally,
`it will be understood that networked devices need not
`include fully functional web server applications or similar
`features, but may be configured to simply broadcast and/or
`listen on specified IP ports to accomplish system connec-
`tivity and communications.
`Network enabled appliances of the system additionally
`include basic feature and state reporting functions for
`enabling other appliances, control server(s), and remote
`control(s) to readily ascertain the available control functions
`ofan appliances, and the current state of the control fune-
`tions. As will be appreciated by those skilled in the art, an
`Extensible Markup Language (XML) or similar format may
`be used to transmit and receive such information between
`appliances, servers, and remote controls. It is also possible
`to modify the URI or other uniqueidentifier assigned each
`appliance suchthat contained in each appliance address is an
`indication of the available functions, and even an indication
`of current appliance state. One method of implementing
`such a system would beto include appropriate device code
`identifiers (e.g., an IR command library, individual IR code,
`and/or graphical user interface element reference number(s))
`in the URIfor each appliance, thus broadcasting capabilities
`(and thus available features) of the device to any control
`server with access to the associated device library. One
`example
`appliance
`address
`(URI) may
`be
`“http://
`123.456.789.0/wxyz,” where “123.456.789.0" is the main
`appliance address, and “wxyz” in the reference number
`corresponding to an IR command codesetfor that particular
`appliance.
`For enabling frequent feature set and state updates of
`appliances present in the networked control environment on
`associated control servers and remote controls, well known
`multicasting methods for appliance advertising and discov-
`ery may be used (e.g., IP Multicasting). In this way appli-
`ances, servers, and remote controls may easily monitor and
`register feature set and state changes on the networked
`control environment.
`As described above and shown in FIG. 1, the control and
`content servers (including control pods, command/state
`monitors, content renderers, etc.) of the present invention
`may be separate devices, may be incorporated into one or
`more of the appliances such that no individual “server
`devices” are present, etc. Functionally, content centric
`devices can serve as a repository for digital media files
`(audio, video, photo, and multimedia content), offer search
`and cataloging based services, and servefiles to one or more
`appliances for playback. Generally computing based devices
`such as Media Center PC’s, “Content Servers,” and “Con-
`tent Renderers,” as defined and described in, for example.
`the UPnP specification, are examples of content centric
`devices in a home control and automation environment.
`Control centric devices generally function to receive,
`save, process, and transmit media and state data for appli-
`ances and devices of the home control and automation
`environment. Various types of command data (IR, RF,
`Powerline commands, Phoneline commands, etc.) may be
`saved by command/state monitors as described in greater
`detail below. Whether implemented on a single device, or
`distributed across multiple devices in the home control and
`automation environment, these control devices are capable
`of performing logic based operations on saved command
`
`>
`
`35
`
`40
`
`45
`
`automation system, a control device 14 is described. The
`control device 14 may be a device separate and apart from
`the other system appliances or may be integrated into one or
`) more ofthe appliancesasis illustrated in FIG. 1. In either
`case, the control device 14 may include, as needed for a
`particular application, a processor 50 coupled to a memory
`device (such as ROM memory 52, RAM memory 51, and/or
`non-volatile read/write memory 56), an internal clock and
`timer 53, receiver circuit(s) 54, transmission circuit(s) 55
`and/or transceiver circuit(s) (e.g., IR and/or RF), a means 58
`to provide feedback to the user (e.g., LED, display, speaker,
`and/or the like), a power supply 62, and input/output means
`64,
`(e.g., serial
`I/O port, wireless transceiver, bar code
`scanner, X-10 bridge, SCP Bridge, Ethernetport, etc.), as is
`generally illustrated in FIG. 3. The memory device may
`include executable instructions that are intended to be
`executed by the processor 50 to control the operation of the
`control device 14. In this manner, the processor 50 may be
`programmed to control the various electronic components
`within the control device 14, e.g.,
`to monitor the power
`supply 62, to cause the transmission ofsignals, to provide
`audio or visual prompts to a user, etc. The non-volatile
`read/write memory 56, for example an EEPROM,battery-
`) backed up RAM, Smart Card, memory stick, or the like, may
`also be provided to store setup data and parameters as
`necessary. While the memory 52 is illustrated and described
`asa ROM memory, memory 52 can also be comprised ofany
`type of readable media, such as ROM, RAM, SRAM,
`FLASH, EEPROM,orthe like. Preferably, the memory 56
`is non-volatile or battery-backed such that data is not
`requiredto be reloaded after battery changes. In addition,the
`memories 51, 52 and 56 may take the form ofa chip, a hard
`disk, a magnetic disk, and/or an optical disk. It will also be
`appreciated that in cases where control device capability is
`integrated into an appliance, someorall of the functional
`elements described abovein conjunction with FIG. 3 may be
`combined with similar elements already present
`in the
`appliance for other purposes.
`It will be appreciated that the abovementioned transmitter
`55, receiver 54, and/or input/output means 64 may be used
`to connect control device 14 to one or more networked
`
`a
`
`6
`
`

`

`US 7,136,709 B2
`
`Lt
`
`wa
`
`8
`ment the device is in, “StateData” includes one or more
`indications ofthe device state, and “ServicesData”includes
`an indication of the services offered by the device. Other
`methods of publishing determined location state data are
`possible given the disclosure and such methods are well
`within the routine skill of a programmerskilled in theart.
`Based on the present location of the remote control in an
`environment, only those appliance within a specified prox-
`imity to the remote control may be caused to report their
`function and state data, or commandstransmitted from the
`remote control may be prioritized for appliances that are
`determinedto be in close proximity to the remote control at
`any given time. Dynamic and/or complex macro commands
`(as described in greater detail below) may be configured to
`cause appliance states to change any time a_ particular
`location is reported by the remote control (e.g.,
`lighting
`fixtures may be configured to receive location data based on
`the remote control location determination and modify their
`power state or brightness level continuously based on
`remote control location).
`The remote control location determination data may also
`be used to provide error correction or checking mechanisms
`for commands issued in a home automation environment.
`For instance, if a remote control location outside the home
`control environment is detected (for example if the remote
`control is connected to the home network from a remote
`location via the Internet) and a command from the home
`automation system activates the oven in the home, a prompt
`may be sent to the remote control inquiring whether such
`“ command to the oven was correct based on the detected
`location of the remote.
`
`7
`appliances, content servers, computers, the Internet, or other
`devices and appliances of a networked home control envi-
`ronment. As such programming and instructions on the
`control device may control the operation of other devices
`and appliances in the system, access content and command
`data stored or maintained either entirely or in part at a
`location physically separate fromthe control device 14 (such
`as for example in a server or personal computer located
`elsewhere in the home or remotely via the Internet), and
`communicate wirelessly with a remote control unit
`to
`accomplish command and control features of the current
`invention.
`In order to enable location based control and automation
`functions, a remote control position determination system
`and methodis described in conjunction with the networked
`control environmentofthe present invention. In general, the
`remote control position determination system includes one
`or more location signaling devices within multiple control
`environments (or control zones), each control environment
`including homeappliances operable by the remote control.
`In one exemplary system of the remote control position
`determination system, a location signaling device (i.e., one
`implementation of control device 14) is placed within each
`control environmentin order to send one or more signals to
`the remote control for location determination operations.
`Various methods for determining the location of the remote
`control relative to the various control environments and
`zones, or similarly determining the proximity of various
`homeappliances