`
`US009875647B1
`
`( 12 ) United States Patent
`Tannenbaum et al .
`
`( 10 ) Patent No . :
`( 45 ) Date of Patent :
`
`US 9 , 875 , 647 B1
`Jan . 23 , 2018
`
`( * ) Notice :
`
`( 54 ) SYSTEMS AND METHODS FOR
`PRESENTING SECURITY QUESTIONS VIA
`CONNECTED SECURITY SYSTEM
`( 71 ) Applicant : GOOGLE INC . , Mountain View , CA
`( US )
`( 72 ) Inventors : Harry Tannenbaum , San Francisco ,
`CA ( US ) ; Scott Hong , Palo Alto , CA
`( US )
`( 73 ) Assignee : GOOGLE INC . , Mountain View , CA
`( US )
`Subject to any disclaimer , the term of this
`patent is extended or adjusted under 35
`U . S . C . 154 ( b ) by 297 days .
`( 21 ) Appl . No . : 14 / 697 , 505
`Apr . 27 , 2015
`( 22 ) Filed :
`Int . Cl .
`( 2006 . 01 )
`G08B 29 / 18
`( 2006 . 01 )
`G08B 13 / 22
`( 2013 . 01 )
`G06F 21 / 34
`( 2006 . 01 )
`G08B 13 / 00
`( 2006 . 01 )
`H04L 12 / 24
`U . S . CI .
`CPC . . . . . . . . . . . G08B 29 / 185 ( 2013 . 01 ) ; G08B 13 / 22
`( 2013 . 01 ) ; G06F 21 / 34 ( 2013 . 01 ) ; G08B 13 / 00
`( 2013 . 01 ) ; H04L 41 / 22 ( 2013 . 01 )
`( 58 ) Field of Classification Search
`CPC . . . . . . GO8B 29 / 185 ; G08B 13 / 22 ; G08B 13 / 00 ;
`HO4L 41 / 22 ; G06F 21 / 34
`See application file for complete search history .
`
`( 52 )
`
`104
`
`150
`
`166
`
`120V
`
`( 56 )
`
`References Cited
`U . S . PATENT DOCUMENTS
`6 , 796 , 799 B1 *
`9 / 2004 Yoshiike . . . . . . . . . . . . G06F 19 / 3418
`434 / 236
`2007 / 0216764 Al *
`9 / 2007 Kwak . . . . . . . . . . . . . . . . . . . . . H04L 12 / 64
`348 / 14 . 06
`2012 / 0280783 A1 * 11 / 2012 Gerhardt . . . . . . . . . . . . G07C 9 / 00309
`340 / 5 . 6
`2013 / 0057695 Al *
`3 / 2013 Huisking . . . . . . . . . . . . . . . H04N 7 / 186
`348 / 156
`2015 / 0325067 A1 * 11 / 2015 Lee . . . . . . . . . . . . . . G07C 9 / 00079
`340 / 5 . 65
`* cited by examiner
`Primary Examiner — Mohamed Barakat
`( 74 ) Attorney , Agent , or Firm — Morgan , Lewis &
`Bockius LLP
`ABSTRACT
`( 57 )
`A computer system detects a trigger event , which includes
`detecting an unverified user within the premises . A notifi
`cation is provided regarding the detected trigger event to one
`or more authorized users distinct from the unverified user .
`Further , a first user input responsive to the notification and
`corresponding to instructions to execute a first security
`protocol is received from a first authorized user of the one
`or more authorized users . In response to receiving the first
`user input , the first security protocol is executed . Executing
`the first security protocol includes presenting to the unveri
`fied user an authentication request and monitoring the prem
`ises for a response to the authentication request . Further
`more , executing the first security protocol includes
`executing or declining to execute a second security protocol
`based on the response to the authentication request .
`20 Claims , 19 Drawing Sheets
`
`120V
`154
`
`120V .
`
`118
`
`100
`
`140
`
`142
`103
`
`HVAC
`
`102
`
`110
`
`114
`
`POOL
`HEATER
`
`IRRIGATION
`
`116 pomo los
`
`168
`
`198
`
`156
`108
`
`112
`
`wwws
`
`4
`
`4
`
`+
`
`H
`
`4 +
`
`W WiFi 180
`
`JO
`160
`
`NETWORKS )
`
`22 CD - 106 120
`APPLIANCE 1
`
`MISC .
`LEGACY
`+ APPLIANCE
`
`140
`
`«
`
`44
`
`162
`SMART
`HOME
`PROVIDER
`SERVER
`SYSTEM
`
`E
`
`164
`
`iApartments Ex. 1023
`
`Page 1 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 1 of 19
`
`US 9 , 875 , 647 B1
`
`OZN
`
`140
`
`100
`
`152
`
`- - - - - - - - - - - - - - - - - - - - -
`
`- -
`
`ROCK
`
`( 4 )
`154 154
`
`TOZT - - - - - - - - - . . . . . .
`
`)
`
`- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`
`# # # ??? + ????????? # H # # ??????????????????? # # ???????????????? # # ?? " ?????????
`
`- - - - - - - - - - * * * * * * * * * OLL
`
`168
`
`158
`
`140
`
`APPLIANCE
`MISC . LEGACY APPLIANCE
`IT WiFi 180
`
`E
`164
`
`SMART HOME PROVIDER SERVER SYSTEM
`
`162
`
`( NETWORK ( S )
`
`Figure 1
`
`. . . .
`
`. . . . . .
`
`. . . .
`
`. . . . . . .
`
`. . . . . .
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- -
`
`-
`
`-
`
`160
`
`112
`
`1
`
`)
`
`156 | T 168
`108
`108
`
`166
`
`150
`
`VOZL
`
`104
`
`I 110
`
`102 102
`
`POOL HEATER
`
`114
`
`HVAC
`
`IRRIGATION
`
`116
`
`- - - -
`
`9
`
`142 103
`
`iApartments Ex. 1023
`
`Page 2 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 2 of 19
`
`US 9 , 875 , 647 B1
`
`Network Architecture 200 _
`
`Smart Home Provider
`Server System
`164
`
`Network ( s )
`162
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`wwwwww
`
`w wwwww
`
`we
`
`-
`
`www www
`
`mwen
`
`weer Www
`
`wwwwwww
`
`-
`
`-
`
`- -
`
`-
`
`Device
`204 - 3
`
`Device
`204 - 8
`
`Smart Home Network 202
`—
`—
`—
`—
`—
`
`-
`
`-
`
`Smart Home Environment 100
`
`. .
`
`. .
`
`. .
`
`.
`
`_
`_
`_
`_
`_
`_
`Network Interface 160
`T
`-
`-
`-
`-
`-
`T
`
`Device
`204 - 1
`
`-
`
`-
`
`-
`
`Device
`204 - 2
`
`- - - - - - - -
`
`- - - - - - - - red - e
`
`Device
`204 - 5
`
`HUB
`180
`
`Device
`204 - 6
`
`-
`
`-
`
`-
`
`-
`
`Device
`204 - 7
`
`Device
`204 - 4
`
`Device
`204 - n
`
`1
`
`—
`
`-
`
`-
`
`- - - - -
`
`-
`
`-
`
`Figure 2
`
`iApartments Ex. 1023
`
`Page 3 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 3 of 19
`
`US 9 , 875 , 647 B1
`
`
`
`
`
`Application Programming Interfaces
`
`164
`SMART HOME
`PROVIDER
`SERVER SYSTEM
`302
`
`HOME DATA
`
`SERVICES
`
`ENGINES :
`- - STATISTICS
`- - INFERENCES
`- - INDEXING
`
`308
`
`wwwwwwwwwww
`
`DERIVED
`HOME
`DATA
`
`. . * . * . . *
`
`. . .
`
`. . . . . . .
`
`.
`
`310
`
`too
`
`CHARITIES
`
`314
`
`GOVERNMENTS - 316
`_ 318
`- 320
`
`ACADEMIC
`INSTITUTIONS
`
`BUSINESSES
`
`UTILITIES 1324
`
`NETWORK ( S )
`( E . G . , INTERNET M
`
`162
`
`DEVICES
`
`8100
`DEVICES
`HUB 180
`NETWORK !
`INTERFACE
`
`Figure 3
`
`iApartments Ex. 1023
`
`Page 4 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 4 of 19
`
`US 9 , 875 , 647 B1
`
`412
`
`EXTRINSIC
`INFORMATION ( e . g . ,
`FROM INTERNET )
`
`EX :
`- - WEATHER FORECAST
`- - PRICES
`- - NEIGHBORHOOD / HOME INFORMATION
`
`PROCESSING
`PARADIGMS
`
`MANAGED
`SERVICES
`
`ADVERTISING /
`COMMUNICATION
`
`EX :
`- - SECURITY
`- - DEMANDI
`RESPONSE
`410a
`
`410b
`
`410C
`
`SOCIAL
`
`CHALLENGES
`RULES /
`COMPLIANCE
`REWARDS
`
`Search 410d
`
`404
`
`402 SL posted 406
`. .
`
`408
`
`DS SS
`DCSON
`
`306
`
`PROCESSING
`
`pe r
`
`DS SS
`DCSC
`
`DS SS
`DC SC
`
`Doors
`
`DS SS
`DC SC
`
`Bed
`
`DEVICES OF SMART HOME NETWORK 202
`EX :
`DS = DATA SOURCE
`- - LIGHTS , HVAC , WATER CONTROLLERS /
`DC = DATA CONSUMER
`Devices
`SS = SERVICE SOURCE
`SC = SERVICE CONSUMER
`- - HOME APPLIANCES
`- - SMOKE / CO / HAZARD DEVICES / ALARMS
`
`Figure 4
`
`iApartments Ex. 1023
`
`Page 5 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 5 of 19
`
`US 9 , 875 , 647 B1
`
`500 -
`
`Server - Side
`Module 506
`
`Hub Server
`System 508
`
`514 39 Processor ( s ) 512
`
`Video
`Storage
`Database
`
`Device and
`Account
`Databases
`516
`
`1 / 0 Interface to
`Client ( s ) 518
`
`I / O Interface to
`Cameras 520
`
`Network ( s ) 162
`
`und
`
`h504 - 1
`- 502 - 1
`
`Client
`Side
`Module
`
`- -
`
`-
`
`-
`
`1620
`
`162A
`
`LAN
`160 _ ( Q )
`
`1620
`
`
`
`Prome more
`
`-
`
`-
`
`I - - - -
`522 - 7
`
`522 - 1
`- - 1
`zorno @ motoren
`voro
`
`162B LA
`162B
`400 180
`_ -
`
`-
`
`-
`
`- -
`
`-
`
`-
`
`. .
`
`.
`
`162A
`
`504 - m
`
`162A
`Ø e
`
`-
`
`-
`
`502 - m
`
`Client
`Side
`Module
`
`. . .
`. .
`
`. .
`. .
`
`. . .
`.
`. .
`
`. . .
`
`.
`. . .
`. . .
`
`. . .
`
`Figure 5
`
`iApartments Ex. 1023
`
`Page 6 of 41
`
`
`
`atent
`
`Jan . 23 , 2018
`
`Sheet 6 of 19
`
`US 9 , 875 , 647 B1
`
`Hub Device 180
`
`Radios
`640
`
`602
`CPU ( s )
`
`Input
`Device ( s )
`610
`
`Location
`Detection
`Device 614
`
`604 )
`Comm .
`Interface ( s )
`
`Memory 60614 Operating Logic 616
`Hub Communication Module 618
`Radio Communication Module 620
`User Interface Module 622
`Hub Database 624
`Sensor information 6240
`Device settings 6242
`Communication Protocol
`Information 6244
`
`608
`
`Output
`Device ( s )
`612
`
`Trigger detection module 626
`Security protocol module 628
`
`Figure 6
`
`iApartments Ex. 1023
`
`Page 7 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 7 of 19
`
`US 9 , 875 , 647 B1
`
`Operating System 710
`Network Communication Module 712
`Server - Side Module 714
`Data Receiving Module 7140
`Hub and Device Control Module
`7142
`Data Processing Module 7144
`Trigger detection module 7146
`Security protocol module 7148
`
`Hub Server System 508
`
`Memory 706
`
`702
`CPU ( s )
`
`708
`
`Network
`Interface ( s )
`
`_ 704
`
`Server Database 716
`Data Storage Database 7160
`
`. .
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . .
`
`. . . . . .
`
`Account Database 7162
`Device Information Database 7164
`
`Figure 7
`
`iApartments Ex. 1023
`
`Page 8 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 8 of 19
`
`US 9 , 875 , 647 B1
`
`Client Device 504
`Memory 8064Operating System 818
`Network Communication Module 820
`Input Processing Module 822
`Application ( s ) 824
`User Interface Module 826
`Client - Side Module 828
`Hub and Device Control Module
`8280
`Data Review Module 8282
`
`804
`Network
`Interface ( s )
`
`8027
`CPU ( S )
`
`Client Data 830
`
`Account Data 8300
`Local Data Storage Database 8302
`
`Trigger detection module 832
`Security protocol module 834
`
`808
`
`User Interface 810
`Output
`Device ( s ) 812
`
`Input
`Device ( s ) 814
`
`Location Detection
`Device
`816
`
`Built - in Sensors
`890
`
`Figure 8
`
`iApartments Ex. 1023
`
`Page 9 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 9 of 19
`
`US 9 , 875 , 647 B1
`
`Smart Device 204
`Memory 906
`
`Image !
`Video
`Capture
`Device
`918
`
`CPU ( S )
`902
`
`908
`
`Radios
`940
`
`Operating Logic 920
`Device Communication Module 922
`Radio Communication Module 924
`Input Processing Module 926
`User Interface Module 928
`Application Module ( s ) 930
`Device - side Module 932
`- Command Receiving Module
`9320
`Data Processing Module 9322
`
`910 H - 12 Comm .
`
`User Interface
`
`Output
`Device ( s )
`912
`Input
`Device ( s )
`914
`
`Device Data 934
`Account Data 9340
`Local Data Storage Database
`9342
`
`Trigger detection module 936
`Security protocol module 938
`
`Interface ( s )
`904
`
`Location
`Detection
`Device
`916
`
`Built - in
`Sensors
`990
`
`Figure 9
`
`iApartments Ex. 1023
`
`Page 10 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 10 of 19
`
`US 9 , 875 , 647 B1
`
`Smart Home Provider
`Server System 164
`
`Memory 10062 Operating System 1010
`Network Communication Module 1012
`Server - Side Module 1014
`Data Receiving Module 10140
`Device Control Module 10142
`Data Processing Module 10144
`Trigger detection module 10146
`Security protocol module 10148
`
`1008
`
`10027
`
`CPU ( S )
`
`Network
`Interface ( s )
`
`1004 Server Database 1016
`Data Storage Database 10160
`Account Database 10162
`
`Figure 10
`
`iApartments Ex. 1023
`
`Page 11 of 41
`
`
`
`… . … . … . … . …
`EMERGENCY RESPONDERS
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 11 of 19
`
`US 9 , 875 , 647 B1
`
`504 - 2
`
`100 - 3 100 - 3
`
`?
`
`100 - 2
`
`? ? ?
`
`504 - 3
`
`- - - - - - - - - -
`
`1104
`
`-
`
`-
`
`- - -
`
`1100
`
`164
`
`?
`
`HOME PROVIDER SERVER
`SYSTEM
`SMART
`
`- -
`-
`
`???? ?
`| ????
`|
`|
`/
`| |
`|
`11
`| |
`| |
`|
`| | 11i 11 / |
`/
`/
`
`? /
`
`DEVICES
`
`! NETWORK ; INTERFACE ] Figure 11
`
`c100 - 1
`
`NETWORK ( S )
`
`? ( E . G . INTERNET
`
`162
`
`- - - - - - - - - -
`
`- - - - - - - - - - A? ,
`
`-
`
`1102
`
`
`
`SECURITY CALL CENTER
`
`504 - 1
`
`iApartments Ex. 1023
`
`Page 12 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 12 of 19
`
`US 9 , 875 , 647 B1
`
`DO 118 - 1
`
`Drb 106 m 106 120
`
`132
`c
`
`100
`
`1200 - 3
`
`118 - 2
`
`1202
`1182 , 202
`saw )
`
`180
`WiFi
`
`
`
`
`
`
`
`
`
`??? ?? ??? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??? ??? ??? ??? ?? ?? ?? ?? ??? ??? ?? ?? ?? ??
`
`162
`
`( NETWORK ( S )
`
`Figure 12
`
`N
`
`120V
`
`* * * *
`
`160
`
`?? ?? ?? ?? ?? ?? ??? ?? ?? ?? ?? ??? ??? ??? ??? ?? ??
`
`504 - 1
`
`6 . 0 .
`corar
`
`150
`
`104
`
`1200 - 2 1200 - 2
`
`G2 - 108
`
`RC
`
`1200 - 1 1200 - 1
`
`110
`
`102
`
`iApartments Ex. 1023
`
`Page 13 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 13 of 19
`
`US 9 , 875 , 647 B1
`
`1330
`
`1318 - 2
`
`- 1318 - 1
`
`- 1318 - 3
`
`_ 1318 - 4
`
`1318 - 5
`
`
`
`
`
`
`
`What question would you like to present ?
`
`1300
`
`- 1310
`
`would you like to do ? detected in Room 1200 - 3 at 1 : 00 AM . What Suspicious activity has been
`
`
`
`
`
`
`
`
`
`View Live Video Feed
`
`
`
`What is my dog ' s name ?
`
`
`
`What college did I attend ?
`
`What is my wife ' s name ?
`
`
`
`What ' s your name ?
`
`Connect to speaker
`
`
`
`Figure 13B
`
`c
`
`1311 M41312 1314 1316
`
`Ignore
`View Clip / Photo
`
`
`
`
`
`Defer Action
`
`
`
`Sound Alarm
`
`1318
`
`1320 1322
`
`
`
`
`
`Present Security Question
`
`Figure 13A
`
`Alert Security Call Center Alert
`
`
`
`Police
`
`. . .
`
`ww . . . . . . . .
`
`w
`
`w
`
`504
`_
`
`504
`
`
`
`iApartments Ex. 1023
`
`Page 14 of 41
`
`
`
`atent
`
`Jan . 23 , 2018
`
`Sheet 14 of 19
`
`US 9 , 875 , 647 B1
`
`1360
`
`1314 - 1
`
`1314 - 2
`
`L1314 - 3
`
`
`
`like to defer this action ?
`How would you
`
`
`
`
`Contact Neighbor
`
`
`
`Contact Brother
`
`
`
`Contact Wife
`
`
`
`Figure 13D
`
`1350 1350
`
`1352
`
`1312 1314
`- 1316
`
`1320
`
`1322
`
`N
`
`41340
`
`
`
`Response received : " Pepper "
`
`504
`
`504
`
`
`
`
`
`What would you like to do ?
`
`
`
`Listen to Recorded Audio
`
`
`
`
`
`Present Another Question
`
`Ignore
`
`
`
`Defer Action
`
`
`
`Sound Alarm
`
`
`
`Alert Police
`
`
`
`Figure 13C
`
`
`
`Alert Security Call Center
`
`
`
`
`
`iApartments Ex. 1023
`
`Page 15 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 15 of 19
`
`US 9 , 875 , 647 B1
`
`1400
`
`-
`
`-
`
`-
`|
`-
`
`1
`
`-
`
`-
`
`-
`
`-
`
`L
`4
`
`1402
`1404
`
`1406
`
`n
`
`1408
`
`-
`
`-
`
`—
`
`—
`-
`
`—
`-
`
`—
`-
`
`—
`-
`
`—
`-
`
`—
`-
`
`—
`
`—
`
`—
`
`—
`
`U
`
`Trommel
`
`-
`-
`
`-
`
`Detect a trigger event , including detecting an unverified user within the premises
`-
`-
`-
`-
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Detect the opening of a door
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Receive inputs from one or more distinct devices of a computer system
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Obtain an activity pattern of an authorized user , of one or more authorized users i
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`| The activity pattern for the authorized user indicates , for a respective day !
`I of the week and / or a respective range of time : a time spent in an area of
`the premises , and / or the number of instances in which the authorized user
`entered the area of the premises
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`comune
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`—
`-
`-
`- -
`- . - - - -
`-
`-
`-
`,
`1
`The activity pattern is determined over a predefined period of time N
`i !
`TL IT
`ILI I
`- - -
`-
`-
`.
`-
`-
`-
`-
`- -
`—
`L
`- - - - -
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`—
`—
`Identify an activity pattern of the unverified user
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`-
`- - - - - - - -
`-
`Determine a current date and time
`seguimen mene me never seen -
`-
`-
`antes e
`n
`e
`estado e
`
`-
`
`L
`
`-
`
`-
`
`-
`
`e
`
`-
`
`e
`
`-
`
`-
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`!
`
`-
`-
`- -
`
`n
`
`e
`
`se
`
`u even more
`
`M
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`A
`
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Determine that the activity pattern of the unverified user is at least partially
`distinct from the activity pattern of the authorized user
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Il Determining that the activity pattern of the unverified user is at least II
`even
`Ti partially distinct from the activity pattern of the authorized user is with W
`T
`respect to the current date and time
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`I T
`- -
`- - - -
`-
`- - - -
`- - - -
`-
`- - -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`
`- - - - - - - an d
`
`J
`
`II
`
`11 Il IT
`
`Figure 14A
`
`1410
`
`1 412
`
`1416
`
`1418
`
`1420
`
`iApartments Ex. 1023
`
`Page 16 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 16 of 19
`
`US 9 , 875 , 647 B1
`
`Provide a notification regarding the detected trigger event to one or more authorized to 1422
`users distinct from the unverified user
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`i
`The notification includes information identifying the trigger event
`i
`n
`n contra
`ste come across wel - -
`- -
`-
`-
`- -
`-
`-
`-
`- - ve se
`s - - - - men se
`Ben nemen
`average
`made
`| Providing the notification is in accordance with detecting that the one or more
`|
`authorized users are not within the premises
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`
`-
`
`-
`
`-
`
`Ó
`
`Ó
`
`Figure 14B
`
`Ó
`
`-
`
`-
`
`-
`
`1424
`
`1426
`
`iApartments Ex. 1023
`
`Page 17 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 17 of 19
`
`US 9 , 875 , 647 B1
`
`Receive , from an authorized user of the one or more authorized users , a user input
`responsive to the notification and corresponding to instructions to execute a first
`security protocol
`
`41428
`
`In response to receiving the user input , execute the first security protocol
`
`-
`
`-
`
`Present to the unverified user an authentication request
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`The authentication request includes a security question , and the response
`comprises an answer responsive to the question
`- - - - -
`-
`- -
`- - - - - - -
`-
`-
`-
`- - -
`- - - - - - -
`- - - - - - - - - - - -
`Instructions include a selection of the security question , of a plurality L
`of security questions , by the authorized user
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`. . . . . .
`. .
`. . . . .
`.
`.
`Play a pre - recorded audio file
`-
`-
`-
`-
`
`. .
`
`. . .
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`N
`-
`
`-
`
`-
`
`Bowen -
`-
`
`Monitor the premises for a response to the authentication request
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`The response to the authentication request is a recorded audio file
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`—
`—
`—
`—
`—
`—
`—
`—
`—
`—
`—
`—
`—
`—
`—
`The response to the authentication request includes a personal
`identification number
`
`-
`
`-
`
`1430
`Th1432 1432
`
`1434
`
`1436
`
`1438
`
`1440
`
`1442
`
`1444
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`.
`-
`-
`en a
`L
`e
`den werden . .
`- -
`-
`-
`Determine whether the unverified user is an authorized user based on the
`response to the authentication request
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Apply audio processing techniques to process the recorded audio file in 1448
`-
`-
`-
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Present to the authorized user the response to the authentication request
`-
`-
`-
`-
`-
`
`- -
`
`.
`
`1446
`
`-
`-
`-
`- -
`-
`Receive , from the first authorized user , a user input responsive to the
`user input indicating whether the unverified user is authorized il 1452
`presented response to the authentication request , wherein the second
`Il
`- -
`-
`- J
`- J
`T1
`I
`
`- -
`
`-
`-
`
`-
`-
`
`-
`-
`
`-
`-
`
`-
`-
`
`-
`-
`
`-
`-
`
`I
`
`-
`.
`
`-
`
`I
`
`-
`
`-
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`I I I
`
`- -
`-
`
`Figure 140
`
`iApartments Ex. 1023
`
`Page 18 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 18 of 19
`
`US 9 , 875 , 647 B1
`
`In response to receiving the user input , execute the first security protocol
`
`1430
`
`Execute or decline to execute a second security protocol based on the response
`to the authentication request
`-
`-
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`| Executing the second security protocol includes sounding an alarm device
`-
`-
`-
`-
`-
`-
`-
`-
`-
`- - -
`- -
`- - -
`-
`-
`- -
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`- -
`-
`-
`-
`- -
`-
`-
`- - -
`-
`Executing the second security protocol includes providing a notification to
`one or more other authorized users
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`! Executing the second security protocol includes sending instructions to W
`other devices
`o see -
`-
`-
`-
`-
`- - -
`Les seves
`-
`-
`-
`-
`-
`- as e
`s mesures - - -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`If the unverified user is not an authorized user , execute the second
`1
`| security protocol , and if the unverified user is an authorized user , decline N
`execution of the second security protocol
`-
`-
`www
`-
`-
`- -
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`1454
`
`1456
`
`1458
`
`1460
`
`1462
`
`h
`
`T
`
`-
`
`-
`
`-
`
`-
`
`- -
`
`Figure 14D
`
`iApartments Ex. 1023
`
`Page 19 of 41
`
`
`
`U . S . Patent
`
`Jan . 23 , 2018
`
`Sheet 19 of 19
`
`US 9 , 875 , 647 B1
`
`-
`
`|
`
`-
`
`a
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Receive , from an authorized user , a user input responsive to the notification and
`corresponding to instructions to execute the second security protocol
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`In response to receiving the user input , execute the second security protocol
`-
`- - - umelecom -
`are reunescu -
`comment avec une seule
`seconda e
`con
`contre
`
`was
`
`1466
`
`-
`
`-
`
`?
`????
`????
`???
`???
`???
`??
`??
`???
`????
`???
`????
`?????
`? ?
`?
`??
`????
`?????
`???
`???? ??? ????
`????
`??? ???
`??
`???
`???
`| Receive a user input corresponding to instructions to access other devices , distinct
`from one or more devices of the computer system
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`?
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`Menu -
`
`-
`
`- - sarees - -
`
`- -
`
`-
`
`-
`-
`-
`-
`-
`-
`-
`Retrieve data from the other devices
`- - -
`-
`- -
`T
`- -
`- -
`-
`-
`- - - - - -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- es mesures
`
`meet - Menu
`
`1470
`
`-
`
`-
`
`-
`
`-
`
`-
`
`camente desconocerse e
`
`n
`
`- -
`-
`- -
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`Provide the data to the one or more authorized users
`ntre em como
`de
`comedy scenes comments are more common comment commentare
`com a c
`como
`
`-
`
`- I -
`-
`
`- 1
`
`1472
`1472
`
`-
`
`- - -
`
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`i In the absence of receiving a user input responsive to the notification , execute the first 1
`security protocol .
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`-
`
`1474
`
`Figure 14E
`
`iApartments Ex. 1023
`
`Page 20 of 41
`
`
`
`SYSTEMS AND METHODS FOR
`PRESENTING SECURITY QUESTIONS VIA
`CONNECTED SECURITY SYSTEM
`
`US 9 , 875 , 647 B1
`Thus , computer systems are provided with more effective
`and efficient methods for detecting trigger events and
`executing security protocols , thereby increasing the effec
`tiveness and efficiency of such devices and systems .
`TECHNICAL FIELD
`BRIEF DESCRIPTION OF THE DRAWINGS
`This relates generally to security systems , including but
`For a better understanding of the various described imple
`not limited to methods and systems for detecting trigger
`mentations , reference should be made to the Description of
`events and executing security protocols .
`10 Implementations below , in conjunction with the following
`drawings in which like reference numerals refer to corre
`BACKGROUND
`sponding parts throughout the figures .
`FIG . 1 is an example smart home environment , in accor
`Security systems have traditionally lacked granularity
`dance with some implementations .
`with respect to identifying and classifying a detected threat .
`An armed system that detects movement on a premises , for * 15
`FIG . 2 is a block diagram illustrating an example network
`architecture that includes a smart home network , in accor
`example , will trigger an alarm regardless of whether the
`dance with some implementations .
`cause truly warrants an alarm . Consequently , isolating false
`FIG . 3 illustrates a network - level view of an extensible
`alarm scenarios from legitimate threats within an environ
`devices and services platform with which the smart home
`ment has long been a challenge with typical security sys - 20 environment of FIG . 1 is integrated , in accordance with
`some implementations .
`tems .
`FIG . 4 illustrates an abstracted functional view of the
`SUMMARY
`extensible devices and services platform of FIG . 3 , with
`reference to a processing engine as well as devices of the
`Accordingly , there is a need for methods , systems , and 25 smart home environment , in accordance with some imple
`mentations .
`interfaces for detecting trigger events and executing security
`protocols . By utilizing information and data gathered by one
`FIG . 5 is a representative operating environment in which
`or more devices or systems in a connected network , precise
`a hub device server system interacts with client devices and
`and accurate context is provided with respect to events that
`hub devices communicatively coupled to
`local smart
`trigger an alarm in an environment . Furthermore , by execut - 30 devices , in accordance with some implementations .
`ing predefined security protocols in response , such as pre -
`FIG . 6 is a block diagram illustrating a representative hub
`senting unverified users with security questions that they
`device , in accordance with some implementations .
`must correctly answer , false alarm scenarios can be better
`FIG . 7 is a block diagram illustrating a representative hub
`distinguished from legitimate threats and proper actions can
`server system , in accordance with some implementations .
`be taken in response .
`FIG . 8 is a block diagram illustrating a representative
`In accordance with some implementations , a method is
`client device associated with a user account , in accordance
`performed at a computer system ( e . g . , one or more smart
`with some implementations .
`devices in a smart home network ) with one or more pro -
`FIG . 9 is a block diagram illustrating a representative
`cessors and memory storing instructions for execution by the
`smart device , in accordance with some implementations .
`one or more processors . The method includes detecting a 40
`FIG . 10 is a block diagram illustrating a representative
`trigger event , including detecting an unverified user within
`smart home provider server system , in accordance with
`the premises . A notification is provided regarding the
`some implementations .
`detected trigger event to one or more authorized users
`FIG . 11 is an example smart security network , in accor
`distinct from the unverified user . The method further
`dance with some implementations .
`includes receiving , from a first authorized user of the one or 45
`FIG . 12 is an example smart home environment in a smart
`more authorized users , a first user input responsive to the
`security network , in accordance with some implementations .
`notification and corresponding to instructions to execute a
`FIGS . 13A - 13D illustrate examples of graphical user
`first security protocol . In response to receiving the first user
`interfaces for displaying notifications and executing opera
`input , the first security protocol is executed . Executing the
`tions responsive to notifications , in accordance with some
`first security protocol includes presenting to the unverified 50 implementations .
`user an authentication request and monitoring the premises
`FIGS . 14A - 14E are flow diagrams illustrating a method of
`for a response to the authentication request . Furthermore ,
`detecting trigger events and executing security protocols , in
`executing the first security protocol includes executing or
`accordance with some implementations .
`declining to execute a second security protocol based on the
`Like reference numerals refer to corresponding parts
`55 throughout the several views of the drawings .
`response to the authentication request .
`In accordance with some implementations , a computer
`system ( e . g . , one or more smart devices in a smart home
`DESCRIPTION OF IMPLEMENTATIONS
`network ) includes one or more processors , memory , and one
`Reference will now be made in detail to implementations ,
`or more programs ; the one or more programs are stored in
`the memory and configured to be executed by the one or 60 examples of which are illustrated in the accompanying
`more processors . The one or more programs include instruc -
`drawings . In the following detailed description , numerous
`tions for performing the operations of the method described
`specific details are set forth in order to provide a thorough
`above . In accordance with some implementations , a non -
`understanding of the various described implementations .
`transitory computer - readable storage medium has stored
`However , it will be apparent to one of ordinary skill in the
`therein instructions that , when executed by the computer 65 art that the various described implementations may be
`system , cause the computer system to perform the opera
`practiced without these specific details . In other instances ,
`well - known methods , procedures , components , circuits , and
`tions of the method described above .
`
`35
`
`iApartments Ex. 1023
`
`Page 21 of 41
`
`
`
`US 9 , 875 , 647 B1
`
`may again be the landlord with respect to remote control
`networks have not been described in detail so as not to
`functionality . Importantly , while the identity of the person
`unnecessarily obscure aspects of the implementations .
`performing the action may be germane to a particular
`It will also be understood that , although the terms first ,
`second , etc . are , in some instances , used herein to describe
`advantage provided by one or more of the implementations ,
`various elements , these elements should not be limited by 5 such identity should not be construed in the descriptions that
`these terms . These terms are only used to distinguish one
`follow as necessarily limiting the scope of the present
`element from another . For example , a first type of security
`teachings to those particular individuals having those par
`protocol could be termed a second type of security protocol ,
`ticular identities .
`and , similarly , a second type of security protocol could be
`FIG . 1 is an example smart home environment 100 in
`termed a first type of security protocol , without departing 10 , accordance with some implementations . The Smart home
`from the scope of the various described implementations .
`environment 100 includes a structure 150 ( e . g . , a house ,
`The first type of security protocol and the second type of
`office building , garage , or mobile home ) with various inte
`security protocol are both types of security protocols , but
`grated devices . It will be appreciated that devices may also
`they are not the same type of security protocol .
`The terminology used in the description of the various 15 be integrated into a smart home environment 100 that does
`not include an entire structure 150 , such as an apartment ,
`described implementations herein is for the purpose of
`describing particular implementations only and is not
`condominium , or office space . Further , the smart home
`intended to be limiting . As used in the description of the
`environment 100 may control and / or be coupled to devices
`various described implementations and the appended claims ,
`outside of the actual structure 150 . Indeed , several devices
`the singular forms “ a ” , “ an ” and “ the ” are intended to 20 in the smart home environment 100 need not be physically
`include the plural forms as well , unless the context clearly
`within the structure 150 . For example , a device controlling
`indicates otherwise . It will also be understood that the term
`a pool heater 114 or irrigation system 116 may be located
`" and / or ” as used herein refers to and encompasses any and
`outside of the structure 150 .
`all possible combinations of one or more of the associated
`The depicted structure 150 includes a plurality of rooms
`listed items . It will be further understood that the terms 25 152 , separated at least partly from each other via walls 154 .
`“ includes , ” “ including , ” “ comprises , " and / or “ comprising , ”
`The walls 154 may include interior walls or exterior walls .
`when used in this specification , specify the presence of
`Each room may further include a floor 156 and a ceiling 158 .
`stated features , integers , steps , operations , elements , and / or
`Devices may be mounted on , integrated with and / or sup
`components , but do not preclude the presence or addition of
`ported by a wall 154 , floor 156 or ceiling 158 .
`one or more other features , integers , steps , operations , 30
`In some implementations , the integrated devices of the
`elements , components , and / or groups thereof .
`smart home environment 100 include intelligent , multi
`As used herein , the term " if " is , optionally , construed to
`sen
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ ChargeStill Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site