111111
`
`1111111111111111111111111111111111111111111111111111111111111111111111111111
`US 20040260427Al
`
`(19) United States
`(12) Patent Application Publication
`Wimsatt
`
`(10) Pub. No.: US 2004/0260427 Al
`Dec. 23, 2004
`(43) Pub. Date:
`
`(54) HOME AUTOMATION CONTEXTUAL USER
`INTERFACE
`
`(52) U.S. Cl. ............................. 700/275; 700/276; 700/83
`
`(76)
`
`Inventor: William Wimsatt, Littleton, CO (US)
`
`(57)
`
`ABSTRACT
`
`Correspondence Address:
`HOGAN & HARTSON LLP
`ONE TABOR CENTER, SUITE 1500
`1200 SEVENTEENTH ST
`DENVER, CO 80202 (US)
`
`(21) Appl. No.:
`
`10/820,426
`
`(22) Filed:
`
`Apr. 8, 2004
`
`Related U.S. Application Data
`
`( 60) Provisional application No. 60/461,307, filed on Apr.
`8, 2003.
`
`Publication Classification
`
`(51)
`
`Int. Cl? ........................... G05B 13/00; G05B 15/00
`
`A user interface having a plurality of user interface elements
`including: background, passive elements such as frames and
`borders, information display elements that present informa(cid:173)
`tion from application software operating on the control unit,
`and control elements that cause application software oper(cid:173)
`ating on the control unit to initiate programmed behaviors.
`The user interface as a whole is contextually sensitive in that
`the appearance of user interface elements (e.g., color, size,
`font, contrast, order, grouping, arrangement, etc.) and/or the
`behavior of user interface elements are varied in a manner
`that is dependent on the context of the control unit. The
`context of the control unit is represented by state inform a(cid:173)
`tion known to the control unit, which includes context(cid:173)
`specific state information known to a particular control unit
`as well as global context information known to multiple or
`all control units in a system.
`
`601
`
`603
`
`SecureNet Technologies, LLC Exhibit 1004 Page 1
`
`

`
`ENTERTAINMENT
`CONTROL
`SUBSYSTEM
`
`121
`
`115
`
`101
`
`CONTROL 1------_J
`PANEL
`
`101
`
`101
`
`. . .
`
`CONTROL 1------'
`PANEL
`
`107~
`107~d¢ 105
`
`107
`
`HOME CONTROL ~
`SUBSYSTEM
`
`113
`
`ANALOG
`,----------1 SUBSYSTEM 1-----------l
`INTERFACE
`
`ANALOG DEVICE
`(SECURITY
`CAMERA,
`LANDSCAPE,
`TELEPHONY, ETC.)
`
`125
`
`103
`
`117
`
`127
`
`IP
`CAMERA
`
`109
`
`111
`
`129
`
`FIG. 1
`
`""C
`
`~ .... ~ = ....
`~ 't:l -....
`~ ....
`.... 0 =
`~ -a -....
`~ ....
`.... 0 =
`
`(')
`
`(')
`
`~
`~
`!"l
`N
`~~
`
`N c c
`
`~
`
`'JJ. =(cid:173)~
`~ ....
`"""" 0 ......,
`""""
`""""
`
`Cj
`'JJ.
`
`N c c
`~ c
`N
`0'1 c
`~
`N
`-..J
`
`>
`""""
`
`SecureNet Technologies, LLC Exhibit 1004 Page 2
`
`

`
`ROM
`
`I
`
`I
`
`RAM
`
`.------
`r - -
`
`CONTROL SUBSYSTEM
`INTERFACE(S)
`
`1--
`
`RS-232, RS-485 ETC
`
`----r- 205
`
`...___
`
`r - -
`
`-
`-
`
`-
`
`-
`
`NETWORK INTERFACE
`
`1--
`
`CAT-5, IEEE-1394, USB, ETC
`
`GRAPHICS SUBSYSTEM
`
`f---
`
`TOUCHSCREEN DISPLAY
`
`-
`- .
`
`AUDIO SUBSYSTEM
`
`-
`
`-
`
`SENSE 1/0
`
`-
`
`-
`
`SPEAKERS
`
`THERMO, IR, MOTION, ETC.
`
`I
`
`I
`I
`
`I
`I
`
`PROCESSOR
`
`2t
`
`f_J-203
`
`'-
`
`CONTROL PANEL
`
`101
`
`FIG. 2
`
`(')
`
`~ .....
`
`""C
`~ .....
`~ = .....
`~ 't:l -....
`.... 0 =
`~
`0' -....
`.... 0 =
`
`(')
`
`~ .....
`
`~
`~
`!"l
`N
`~~
`
`N c c
`
`~
`
`'JJ. =(cid:173)~
`~ .....
`N
`0 ......,
`'"""'
`'"""'
`
`Cj
`'JJ.
`
`N c c
`~ c
`N
`0'1 c
`~
`N
`-..J
`
`>
`'"""'
`
`SecureNet Technologies, LLC Exhibit 1004 Page 3
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 3 of 11
`
`US 2004/0260427 Al
`
`0:::
`UJ
`
`I b
`
`0:::
`UJ
`
`CJ) s
`0
`0:::
`[()
`
`~
`0:::
`
`0 s UJ
`~ u.
`
`~
`
`0:::
`[()UJ
`UJii:
`Sw
`CJ)
`
`UJO:::
`<.?w
`<(~
`CJ)o
`CJ)o:::
`UJ[(l
`~
`
`I
`CJ)
`
`::s
`u.
`
`a.
`c
`a..
`:::)
`
`CJ)
`
`~
`UJ
`1-
`CJ) >-
`<.? z
`~
`
`UJ
`a..
`0
`
`CJ)
`0:::
`UJ >
`i:t:
`0
`
`CJ)
`0:::
`
`i:t:
`0
`~
`0:::
`0 u.
`
`- w
`UJ > [I]
`~ [!]
`
`a..
`
`-
`
`l)l)l
`
`a..
`a..
`<(
`
`z
`I
`<.?
`:::)
`_J a..
`
`y
`
`0:
`<(
`
`j
`
`SecureNet Technologies, LLC Exhibit 1004 Page 4
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 4 of 11
`
`US 2004/0260427 Al
`
`0000
`
`00
`
`(J
`
`.. 0
`cnw
`::::>:::::2
`J-0::
`<(<(
`I- (f)
`cn-o
`
`z-.::t
`-
`,..._
`
`w
`:::::2
`0
`I
`
`D D
`
`SecureNet Technologies, LLC Exhibit 1004 Page 5
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 5 of 11
`
`US 2004/0260427 Al
`
`0000 00
`
`I-
`z
`0
`I-I-
`~ 1--UJ
`I>- I z
`CJ)
`C)~
`52::5
`z
`-CJ)
`ZUJ Zz
`>-
`0
`<(
`0
`
`~
`~
`
`z
`0
`~
`~
`
`UJCJ)
`z~
`0~
`NCJ)
`
`~(/)
`UJ::J
`~~
`>-I-
`(/)(/)
`
`.. 0
`U)W
`:J~
`1-0:::
`<(<(
`1-Cf)
`en-0
`
`0:::: w
`1-z
`w
`()
`...J
`0
`0::::
`1-
`z
`0
`()
`
`(j)
`1-
`I
`<.9
`::::i
`...J
`...J
`<(
`
`z
`0
`
`LL
`LL
`0
`
`0 0
`
`CJ)
`ti:j
`
`00
`(/) ®
`
`~
`
`SecureNet Technologies, LLC Exhibit 1004 Page 6
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 6 of 11
`
`US 2004/0260427 Al
`
`0000 00
`
`~!JGY
`888®
`8888
`888
`
`~ r---.
`w
`0
`0
`(..)
`~
`w
`I-z
`w
`
`iC
`iC
`iC
`iC
`
`000
`
`~
`
`D D
`
`w
`@
`
`,.------.
`
`.._...
`
`0
`~
`•
`(!)
`ii:
`
`SecureNet Technologies, LLC Exhibit 1004 Page 7
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 7 of 11
`
`US 2004/0260427 Al
`
`0000 00
`
`w
`(9>- B cn<.9
`zz ®
`
`oz
`o-
`0<(
`w~
`(/)~
`
`~<!
`::a:s
`0:::<{
`<t:
`
`(.9
`z
`~
`0::
`<(
`
`D 0
`
`SecureNet Technologies, LLC Exhibit 1004 Page 8
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 8 of 11
`
`US 2004/0260427 Al
`
`000 0
`
`00
`
`0
`
`w
`~
`0
`I
`
`z""<:t
`-I'--
`
`D D
`
`SecureNet Technologies, LLC Exhibit 1004 Page 9
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 9 of 11
`
`US 2004/0260427 Al
`
`0000
`
`00
`
`~ I(cid:173)
`(/)
`0
`~
`0:::: w
`I
`I-
`
`D D
`
`r;l
`~
`
`SecureNet Technologies, LLC Exhibit 1004 Page 10
`
`

`
`CONTROLr-------------------------------------~~
`CENTER
`
`HOME
`r-·----~~-~------1
`
`ENTER
`CODE
`r------~~;------1
`
`CONTROL
`CENTER
`
`SECURITY
`
`l ____ = ____ _j I LIGHTING I
`TOOLS 1 MEDIA
`j_
`
`ARMING
`
`CONTROL
`CENTER
`
`r-----~~;-------~
`
`SECURITY
`
`~----------------·)
`
`TOOLS J
`
`LIGHTING
`
`MEDIA
`
`·----------------~·
`
`TOOLS I
`I
`
`MEDIA
`
`TOOLS
`
`[------~~~------.!
`
`i
`-
`i
`... ________________ )
`
`CALIBRATE
`
`CLEAN
`SCREEN
`
`SECURITY
`CODE
`
`TOOLS 1
`
`HOME
`
`FIG. 5
`
`MONITOR
`
`CONTROL
`CENTER
`HVAC
`r-----~~~-----1 f-------l
`. ________________ ...
`LIGHTING
`TOOLS I
`
`HOME
`
`MONITOR MONITOR
`
`~ r-----~~~------1
`. ________________ . .'
`TOOLS I
`
`HVAC
`
`LIGHTING
`
`HOME
`
`LIGHTING MONITOR
`
`~ r-----~~~-----1~-----~
`, ________________ ...
`
`HVAC
`
`MEDIA
`
`TOOLS I
`
`HOME
`
`MEDIA
`
`MONITOR
`
`HVAC
`r·-----~~~------.1~-----~
`. ________________ ,.
`MUSIC
`TOOLS I
`
`HOME
`
`(')
`
`~ .....
`
`""C
`~ .....
`~ = .....
`~ 't:l -....
`.... 0 =
`~
`0' -....
`.... 0 =
`
`(')
`
`~ .....
`
`~
`~
`!"l
`N
`~~
`
`N c c
`
`~
`
`'JJ. =(cid:173)~
`~ .....
`'"""' c
`0 ......,
`'"""'
`'"""'
`
`Cj
`'JJ.
`
`N c c
`~ c
`N
`0'1 c
`~
`N
`-..J
`
`>
`'"""'
`
`SecureNet Technologies, LLC Exhibit 1004 Page 11
`
`

`
`Patent Application Publication Dec. 23, 2004 Sheet 11 of 11
`
`US 2004/0260427 Al
`
`SecureNet Technologies, LLC Exhibit 1004 Page 12
`
`

`
`US 2004/0260427 Al
`
`Dec. 23,2004
`
`1
`
`HOME AUTOMATION CONTEXTUAL USER
`INTERFACE
`
`RELATED APPLICATIONS
`
`[0001] The present invention claims the benefit of U.S.
`Provisional Patent Application Ser. No. 60/461,307 filed on
`Apr. 8, 2003 entitled Home Automation Control Architec(cid:173)
`ture.
`
`BACKGROUND OF THE INVENTION
`
`[0002] 1. Field of the Invention
`
`[0003] The present invention relates, in general, to auto(cid:173)
`mation systems, and, more particularly, to software, systems
`and methods for implementing a contextual user interface
`for home automation systems.
`
`[0004] 2. Relevant Background
`
`[0005] Home automation systems enable control of light(cid:173)
`ing, heating and air conditioning, window shades or cur(cid:173)
`tains, pool heaters and filtration systems, lawn sprinklers,
`ornamental fountains, audio/visual equipment, and other
`appliances. Home automation systems include relatively
`simple systems that control one or a few functions in a home
`to more elaborate systems that control multiple, disparate
`features. Home automation systems may be integrated with
`a home security system so that when a fire alarm is raised,
`for example, internal and external lights will be turned on.
`Entertainment equipment such as audio, video, and home
`theatre equipment are available with control interfaces that
`enable a remote device to activate the equipment according
`to programmed schedules or remotely input commands.
`
`[0006]
`In general, a home automation or control system
`comprises one or more controlled devices, one or more
`controllers, and a command communication link coupling a
`controller to a controlled device. The controllers may be
`directly programmable in which case they include some
`form of human interface for setting switches, event timing,
`and the like. Alternatively, controllers may be indirectly or
`remotely programmable in which case a separate human
`interface may be implemented by a personal computer or the
`like. Systems may be programmed using either a simple
`command language or using a graphical user interface that
`requires a computer with a monitor. These systems are
`expensive and require substantial investment by the user in
`time and energy to install and modify programming. To enter
`and/or change a program, a user must consult a user's
`manual or call a programming specialist. Hence, these
`systems are difficult to install and adapt to changing needs.
`Moreover, they are difficult to expand by adding new
`controlled devices or new software to add functionality.
`
`[0007] The home automation market has been fractured
`because most of the automation control manufacturers
`address narrow, vertical market segments, and use propri(cid:173)
`etary interfaces to protect their market. For example, some
`leading control manufacturers offer systems that focus on
`heating, ventilation, and air conditioning (HVAC) systems
`control. These manufacturers have little interest in control(cid:173)
`ling lighting, security systems, entertainment systems, and
`the like as these markets are entirely foreign to them. Other
`manufacturers make, for example, home entertainment con(cid:173)
`trollers that integrate various video and audio components,
`but the primary focus has been to offer integrated control
`
`over only their own components. As a result, consumers face
`an array of control systems that do not intemperate, and that
`have proprietary interfaces that are difficult to understand
`and program.
`
`[0008] Some efforts have been made to provide integrated
`interfaces-single devices that "talk" to various control
`systems in a residence. One available system offers a rigid
`architecture that is easy to install because it offers few
`customization options, however, the rigid architecture limits
`its functionality. Other systems offer more flexible inter(cid:173)
`faces, but in each case the implementations include limita(cid:173)
`tions that make the products expensive and/or difficult to
`install.
`
`[0009] Server-based control systems involve a central
`control mechanism or server that issues commands to each
`of the controlled devices either directly, or through subor(cid:173)
`dinate controllers. Server-based systems may be easier to
`program as the operator may need to be come familiar with
`a single program, but are more complex to install as each of
`the controlled devices must be coupled to and in commu(cid:173)
`nication with the central server. Moreover, because the
`server must be programmed to interact with the various
`controlled devices and/or subordinate controls, the operator
`must still become intimately familiar with the protocols and
`vagaries of each controlled device, defeating the advantages
`of a single software interface.
`
`[0010] Another common limitation of control systems
`arises from the control interface of the controlled devices
`themselves. A typical controlled device will implement a
`single control interface for receiving commands from a
`controller. This single interface is usually restricted to a
`single signaling protocol that makes a subset of the con(cid:173)
`trolled devices functions accessible to the controller. Hence,
`the controlled device is designed to interact with a single
`controller and is unable to interact with a plurality of
`controllers. Further, the functionality that can be imple(cid:173)
`mented is restricted by the controller hardware and/or soft(cid:173)
`ware and cannot be readily extended.
`
`[0011] Hence, a need exists for a home automation and
`control architecture that is easy to install, easy to use, and at
`the same time flexible and extensible to accommodate new
`devices and new functionality.
`
`SUMMARY OF THE INVENTION
`
`[0012] A home automation and control architecture having
`a contextually relevant user interface. The user interface is
`generated on one of a plurality of control units located
`throughout a controlled environment such as a home or
`office building. Each user interface comprises mechanisms
`for presenting information (e.g., a visual display, audio
`output systems and the like) as well as mechanisms for
`receiving user input (e.g., touch screen input, mouse or other
`pointing device, microphones, cameras and the like). Auto(cid:173)
`mation and/or control application software executes on the
`control unit to communicate control information such as
`commands, sensor messages, status messages, and the like
`with other control units as well as controlled systems (e.g.,
`security systems, entertainment systems, HVAC systems,
`and the like). The present invention involves two levels of
`contextual relevance: first, the user interface as a whole is
`contextually sensitive and second, interface elements within
`the user interface are contextually sensitive.
`
`SecureNet Technologies, LLC Exhibit 1004 Page 13
`
`

`
`US 2004/0260427 Al
`
`Dec. 23,2004
`
`2
`
`[0013] A user interface comprises a plurality of user
`interface elements including: background, passive elements
`such as frames and borders, information display elements
`that present information from application software operating
`on the control unit, and control elements that cause appli(cid:173)
`cation software operating on the control unit to initiate
`programmed behaviors. The user interface as a whole is
`contextually sensitive in that the appearance of user inter(cid:173)
`face elements (e.g., color, size, font, contrast, order, group(cid:173)
`ing, arrangement, etc.) and/or the behavior of user interface
`elements are varied in a manner that is dependent on the
`context of the control unit. The context of the control unit is
`represented by state information known to the control unit,
`which includes context-specific state information known to
`a particular control unit as well as global context inform a(cid:173)
`tion known to multiple or all control units in a system.
`[0014] Moreover,
`themselves
`individual elements are
`made contextually sensitive. In a particular implementation,
`contextually sensitive interface elements include interactive
`screen elements such that a single screen element can
`simultaneously display information about the context (e.g.,
`current temperature, sound volume, light level, etc.) as well
`implementing behavior to send messages to a controlled
`system that can affect change in the displayed information
`(e.g., a thermostat, an entertainment system, or a lighting
`subsystem). Preferably the context information is displayed
`in a graphical rather than textual form, and the behavior is
`implemented by interacting with the user interface element
`itself.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`[0015] FIG. 1 shows an networked control environment in
`which the present invention is implemented;
`[0016] FIG. 2 illustrates a hardware-oriented view of a
`control panel device in accordance with the present inven(cid:173)
`tion;
`[0017] FIG. 3 illustrates a logical view of processes
`implemented by a control panel device in accordance with
`the present invention;
`[0018] FIG. 4A-4F show exemplary user interface flows
`illustrating operation in accordance with the present inven(cid:173)
`tion;
`[0019] FIG. 5 illustrates an exemplary screen flow using
`a simplified representation of user interface screen;
`[0020] FIG. 6 illustrates in story-board format an exem(cid:173)
`plary screen flow illustrating transitions of a user interface in
`accordance with the present invention;
`[0021] FIG. 7 shows a specific an alternative version of
`the graphical user interface of FIG. 4A in a new context.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`[0022]
`In general, the present invention relates systems
`and methods for implementing a contextual human interface
`to a control system. The particular examples involve a
`control panel that presents an interactive graphical user
`interface that generates commands typically relate to opera(cid:173)
`tional commands to turn on/off a controlled device, adjust
`settings on a controlled device, query the status of a con(cid:173)
`trolled device, and the like. The operational commands are
`
`communicated to a variety of controlled devices (e.g., fans,
`lights, media equipment ant the like) that together make up
`a "controlled system". In accordance with the present inven(cid:173)
`tion, the human interface changes various features depend(cid:173)
`ing on the context in which the controlled system operates.
`These changes may involve altering the number and type of
`controls that are displayed, changing the size, color, or other
`features of a control, changing the focus of the graphical
`display, and the like.
`[0023] The present invention is particularly useful in home
`automation environments because it builds on top of the vast
`array of controlled devices and subsystems that already exist
`for managing lighting, security systems, heating and air
`conditioning, window shades or curtains, pool heaters and
`filtration systems, lawn sprinklers, ornamental fountains,
`audio/visual equipment, and other appliances. Hence, while
`it is contemplated that the present invention may be adapted
`to handle special-purpose and proprietary controlled devices
`and subsystems, a particular advantage is that the present
`invention adapts to existing controlled devices and sub(cid:173)
`systems and leverages their advantages. In essence, the
`present invention provides human interface to a controlled
`system that abstracts the human interface organization from
`the physical interconnection to controlled devices.
`[0024] The present invention is illustrated and described in
`terms of a distributed computing environment having nodes
`distributed throughout a building. However, an important
`feature of the present invention is that it is readily scaled
`upwardly and downwardly to meet the needs of a particular
`application. Accordingly, unless specified to the contrary the
`present invention is applicable to significantly larger, more
`complex network environments such as wide area networks
`(WANs), as well as small network environments such as
`conventional local area networks (LAN) systems or non(cid:173)
`networked environments.
`[0025] Unlike prior user interfaces for controlled systems,
`the present invention implements a contextual interface. A
`contextual interface is a product of a design process in which
`the various interface features are selected to improve the
`user's ability to operate the interface. Examples of interface
`features that can be controlled include the selection of
`controls, size, shape and position of controls, background
`and foreground color schemes, sounds, as well as the
`selection of actions that are initiated by operating a control.
`In most graphical user interfaces these items remain static so
`that irrespective of the context, a control such as a "start"
`button, remains in the same place on a display with the same
`appearance and performs the same function.
`[0026]
`In contrast, the present invention provides an easily
`programmed user interface that can readily take into account
`the user's ongoing experience with the controlled system,
`the nature of a particular user's need to control portions of
`a controlled system, the impact the human interface has on
`the user, the status of the controlled system and components,
`and the like. This means that the human interface will appear
`and behave differently for every user, if desired. Similarly,
`the human interface may appear and behave differently in
`each installation depending on differences in the controlled
`system, and may appear and behave differently over time
`depending on current status of the controlled system.
`[0027] More specifically, a user interface comprises a
`plurality of user interface elements including: background,
`
`SecureNet Technologies, LLC Exhibit 1004 Page 14
`
`

`
`US 2004/0260427 Al
`
`Dec. 23,2004
`
`3
`
`passive elements such as frames and borders, information
`display elements that present information from application
`software operating on the control unit, and control elements
`that cause application software operating on the control unit
`to initiate programmed behaviors. The user interface as a
`whole is contextually sensitive in that the appearance of user
`interface elements (e.g., color, size, font, contrast, order,
`grouping, arrangement, etc.) and/or the behavior of user
`interface elements are varied in a manner that is dependent
`on the context of the control unit. The context of the control
`unit is represented by state information known to the control
`unit, which includes context-specific state information
`known to a particular control unit as well as global context
`information known to multiple or all control units in a
`system.
`
`[0028] Moreover,
`themselves
`individual elements are
`made contextually sensitive. In a particular implementation,
`contextually sensitive interface elements include interactive
`screen elements such that a single screen element can
`simultaneously display information about the context (e.g.,
`current temperature, sound volume, light level, etc.) as well
`implementing behavior to send messages to a controlled
`system that can affect change in the displayed information
`(e.g., a thermostat, an entertainment system, or a lighting
`subsystem). Preferably the context information is displayed
`in a graphical rather than textual form, and the behavior is
`implemented by interacting with the user interface element
`itself.
`
`[0029]
`In this manner, a user interface may present ele(cid:173)
`ments that present information and enable control of light(cid:173)
`ing, security systems, entertainment center and HVAC sys(cid:173)
`tem to adult users. In contrast, a child user is presented only
`entertainment center controls and information. Further, pas(cid:173)
`sive elements such as backgrounds and frames may change
`color and/or shape and/or size based on the specific user or
`a group to which the user belongs. Similarly, the behavior of
`a user interface control may be altered depending on the user
`or a group to which the user belongs. One user may prefer
`a light control to implement a simple on/off whereas another
`user may desire the light control to implement a dimmer.
`
`[0030] The present invention contemplates a user interface
`that is sensitive to a wide variety of contextual information
`beyond user identification. In one implementation, when a
`security system is triggered indicating a fire, intruder, or
`other hazard, passive elements such as backgrounds and
`borders change color or flash to alert the user of the
`condition. Likewise, when a controlled system (or a portion
`of the home automation system itself) fails or otherwise
`becomes unavailable a change in color of passive elements
`may be used to alert the user to the condition. In the
`particular examples the user interface may contain the same
`set of controls during the hazard/failure condition, or the set
`of controls may be changed to become more relevant to the
`current situation (e.g., lighting controls and entertainment
`system controls become unavailable during the condition).
`
`[0031]
`In another example, tripping a zone on a burglar
`alarm, which may indicate a household member entering the
`house, causes the control units to display a security screen
`having controls that allow the alarm to be de-activated by
`entry of a valid code so that the time and stress of navigating
`to the security screen is avoided by automatically adapting
`to the new context. Once a valid security code is entered into
`
`any control panel, each control panel can return to the user
`interface that was presented before the alarm event, thereby
`automatically adjusting to the current context information of
`that control unit.
`[0032] From a programming level, a contextual interface
`works cooperatively with graphical user interface (GUI)
`components. For example, in a typical architecture a data
`processor communicates with a set of GUI processes. The
`GUI processes implement screen buffers and the like for
`translating information from a data processor into one or
`more arrays of pixels that can be displayed to a user.
`Likewise, the GUI processes may receive information form
`a pointer device such as a mouse, for receiving user input.
`[0033] The present invention alters this conventional
`architecture with layer of interpretive processes that run on
`top of the GUI processes, altering the display of information
`and altering how the data processor responds to user input
`based on contextual factors. In this manner, the contextual
`interface of the present invention allows the human interface
`to communicate more effectively with a user by enabling the
`presentation of information in a way that is adapted to the
`particular sensitivities of a user. As a simple example, a child
`may be more responsive to vibrant colors and cartoon-like
`animation whereas an adult may be more responsive to
`soothing tones and subtle graphics. A contextual interface to
`a control system can provide a simple light switch with an
`interface that presents a light on/off control in different ways
`to match the particular sensitivities of a user. These differ(cid:173)
`ences are not only aesthetic as they increase the usability and
`often the functionality of the human interface. This capa(cid:173)
`bility becomes increasingly useful as the number, variety,
`and complexity of the controls increases.
`[0034] FIG. 1 shows an exemplary control environment in
`which the present invention is implemented. A plurality of
`control panels 101 implement a programmable human inter(cid:173)
`face in the particular embodiment. Control panels 101 are
`conveniently implemented using computer industry standard
`components and software to the extent practical, although
`special purpose, non-standard components and software are
`a suitable equivalent in particular applications. In a particu(cid:173)
`lar example, control panels 101 comprise variants of a
`personal computer (PC) architecture to take advantage of
`price and performance features of the personal computer
`market. Control panels 101 are mounted throughout a build(cid:173)
`ing at locations where it is convenient or desired to exercise
`control over controlled systems. For example, a control
`panel101 can be provided in each bedroom of a house, as
`well as a kitchen, office, entertainment areas and the like.
`Alternatively, one or two control panels 101 may be pro(cid:173)
`vided in central locations for shared access by all members
`of a household.
`[0035] As a particular example, the Companion™ 6 and
`Companion™ touch-screen interface units produced by
`CorAccess Systems of Golden, Colo., assignee of the
`present invention, are suitable implementations for control
`panels 101. These devices implement a touch-screen graphi(cid:173)
`cal user interface and are compact fiat screen devices that are
`readily wall mounted. These devices have suitable comput(cid:173)
`ing power and resources to implement a variety of applica(cid:173)
`tions for exercising home automation and control functions
`contemplated by the present invention.
`[0036] A hub 103, such as a conventional internet protocol
`(IP) hub, provides a network interconnection between con-
`
`SecureNet Technologies, LLC Exhibit 1004 Page 15
`
`

`
`US 2004/0260427 Al
`
`Dec. 23,2004
`
`4
`
`trol panels 101 and other devices. Hub 103 may be imple(cid:173)
`mented as a hub, router, switch, access point, or similar
`device that couples network devices. While IP transport is
`used in the particular implementations described herein,
`other standard and/or proprietary transport protocols are
`suitable substitutes. For example, Ethernet®, Fibre Channel,
`and other transport protocols may be appropriate in particu(cid:173)
`lar installations. Hub 103 may implement any number of
`ports to meet the needs of a particular application, and may
`be implemented by a plurality of physical devices to provide
`more ports and/or a more complex network including sub(cid:173)
`networks, zones, and the like.
`
`In addition or alternatively, the present invention
`[0037]
`may be implemented using wireless networking components
`such as a wireless access point/router 105 and wireless
`control panels 107. When used in combination with a wired
`network, access point 105 may be coupled to the network via
`hub 103. Alternatively, access point/router 105 may imple(cid:173)
`ment the hub/router/switch functionality to replace hub 103
`altogether. Wireless control panels 107 implement similar
`functionality to control panels 101 and may be implemented
`by devices such as a Mobile Companion™ or Mobile
`Companion™ X available from CorAccess Systems. Wire(cid:173)
`less control panels 107 may also be implemented by a
`variety of wireless general-purpose computing devices such
`as laptop computers, handheld computers, palm computers
`and the like as well as special purpose devices provided in
`the form of, for example, remote controls, key fobs, smart
`cards, and the like.
`
`[0038] Several basic types of controlled devices are shown
`in FIG. 1. First, controlled devices may be directly con(cid:173)
`nected to the network via, for example, hub 103. For
`example, an IP camera 109 comprises a camera that imple(cid:173)
`ments its own IP interface. A variety of security, telecom(cid:173)
`munications, environmental sensors, and the like are avail(cid:173)
`able with suitable IP interfaces. These controlled devices
`communicate control messages with a network-coupled con(cid:173)
`trol panel 101 or 107.
`
`[0039] A second type of controlled device is coupled to a
`particular control panel101!107 through a subsystem inter(cid:173)
`face. For example, one control panel 101 couples to a
`lighting control subsystem 113 while another control panel
`101 couples to an entertainment control subsystem 115. The
`subsystem interfaces comprise, for example, a control
`device that is provided with a particular third-party sub(cid:173)
`system that may have a special-purpose or proprietary
`signaling protocol. The control panel 101 couples to the
`subsystem interface using the physical, electrical, and sig(cid:173)
`naling protocols adopted by that subsystem. For example, a
`serial connection such as an RS-232 or RS-485 connection
`is used in many cases.
`
`[0040] Alternatively, a subsystem interface may couple
`with hub 103 such as the case with analog subsystem
`interface 117. A variety of controlled devices are available
`such as security cameras, landscape controllers, telephony
`devices, HVAC systems, and the like that do not commu(cid:173)
`nicate using standard computer protocols. An analog sub(cid:173)
`system interface 117 implements control functions to the
`extent possible with such devices and provides a network
`interface for coupling to other systems. An example of such
`a system is a variety of XlO devices and controls marketed
`by XlO Wireless Technology, Inc. of Seattle, Wash.
`
`[0041] The present invention also contemplates imple(cid:173)
`menting shared services and/or resources such as shared
`mass storage 111 and internet access services through an
`internet gateway 127. Mass storage 111 may be coupled via
`hub 103, or may be directly coupled to one or more control
`panels 101!107. Internet gateway 127 may couple to hub 107
`or may be integrated with hub 103 when implemented as a
`router or access point. Internet gateway 127 may implement
`a hardware and/or software firewall or other access control
`mechanisms for increased security.
`
`[0042] Optionally, one or more conventional personal
`computers 129 may be coupled to the network as well via
`hub 103 and/or wireless access point 105. In addition to
`implementing function similar to control panels 101!107, a
`personal computer 129 may implement applications that are
`not installed on or readily executable by control panels
`101!107. Additionally, personal computer 129 may imple(cid:173)
`ment common applications and/or computationally intensive
`applications such as word processin