Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 1 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 1 of 22
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT 8
`EXHIBIT 8
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 2 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 2 of 22
`
`
`
`
`
`
`
`
`
`
`a2) United States Patent
`US 9,490,998 B1
`(0) Patent No.:
`
`
`
`
`
`
`
`Danciu et al.
`Nov.8, 2016
`(45) Date of Patent:
`
`US009490998B1
`
`
`
`(75)
`
`
`
`
`
`
`
`
`
`(54) NETWORK-BASED REMOTE CONTROL
`
`
`
`
`
`
`Inventors: Daniel Danciu, Zurich (CH); Yaniv
`
`
`
`
`Bernstein, Zurich (CH): Ramona
`
`
`
`Bobohalma, Adliswil (CH); Oliver
`Heckmann, Bach (CH); Jasmine
`
`
`
`
`
`
`
`
`Langridge, Zurich (CH); Alin
`
`
`
`
`
`
`
`Sinpalean, Zurich (CH)
`
`
`
`
`
`
`
`73) Assignee: Google Inc., Mountain View, CA (US)
`
`
`
`
`
`
`
`
`
`Subject to any disclaimer, the term of this
`*) Notice:
`
`
`
`
`patent is extended or adjusted under 35
`
`
`
`
`U.S.C. 154(b) by 1041 days.
`
`
`
`
`21) Appl. No.: 13/041,964
`
`
`
`
`22) Filed:
`
`
`
`
`Mar. 7, 2011
`
`
`
`2002/0143805 Al
`
`
`2005/0022134 Al*
`2005/0096753 Al
`
`
`
`
`
`
`
`
`
`
`10/2002 Hayeset al.
`1/2005 Tokashiki 0... 715/764
`$/2005 Arling et al.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`
`
`Schwartz. “Xfinity Remote Prototype: iPad Demo at NCTA Show,”
`
`
`
`
`
`
`
`May 12, 2010 [online]. Retrieved from the Internet: <http://blog.
`
`comcast.com/2010/05/xfinity-remote-prototype-ipad-demo-at-
`
`
`neta-show.html> (5 pgs.).
`
`
`
`
`
`
`
`
`“The Xfinity TV App,” [online]. Comcast Interactive Media, LLC,
`2011. First accessed on Feb. 24, 2011. Retrieved trom the Internet:
`
`
`
`
`
`
`
`
`
`
`
`
`<http://www.xfinity.com/help/internet/mobile-tv-app/> (1 pg.).
`
`
`
`
`
`
`
`
`
`
`“iTunes the all-new Remote. Pul your iPhone, iPad, or iPod touch
`
`
`
`
`
`
`
`
`
`in charge of the show.” [online] Apple Inc., 2011. First accessed on
`
`
`
`
`
`
`
`Sep. 14, 2010. Retrieved from the Internet: <http://www.apple.com/
`itunes/remote/> (4 pgs.).
`
`
`
`(Continued)
`
`
`
`
`
`
`
`
`
`
`
`Primary Examiner — Jennifer Mehmood
`
`
`
`
`Assistant Examiner — Yong Hang Jiang
`
`
`
`
`
`(74) Attorney, Agent, or Firm — Shumaker & Sieffert,
`PA.
`
`
`
`
`
`ABSTRACT
`(57)
`
`
`
`
`
`
`
`
`
`The subject matter of the present disclosure can be imple-
`
`
`
`
`
`
`mented in, among other things a computer-readable storage
`
`
`
`
`
`
`
`medium encoded with instructions for causing a program-
`
`
`
`
`
`
`
`mable processorto receive, by a server, a first message from
`a remote control
`that is distinct from and external to the
`
`
`
`
`
`
`
`
`
`server, wherein the first message includes a remote control
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`identifier and control
`information for controlling one or
`more functions of at least one device other than the remote
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`control. The instructions also cause the programmable pra-
`
`
`
`
`
`
`cessorto retrieve, bythe server, a controlled device identifier
`
`
`
`
`
`
`
`that uniquely identifies a controlled device that is distinct
`from and external to the server. The instructions also cause
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the programmable processor to send a second message from
`
`
`
`
`
`
`
`
`
`the server to the controlled device identified by the con-
`
`
`
`
`
`
`
`
`
`trolled device identifier to control an operation of the
`controlled device.
`
`
`
`
`
`
`18 Claims, 7 Drawing Sheets
`
`
`
` 56)
`
`
`
`
`
`Related U.S. Application Data
`
`
`
`
`
`
`60) Provisional application No. 61/411,386, filed on Nov.
`8, 2010.
`
`
`
`
`Int.Cl
`
`(2006.01)
`GOG6F 15/177
`
`
`(2006.01)
`HOAL 12/28
`
`
`(2006.01)
`HOAL 29/12
`52) U.S. Cl.
`
`
`
`
`
`
`CPC HO4L 12/2818 (2013.01); HO4L 29/1233
`
`
`
`
`(2013.01)
`
`51)
`
`
`
`
`
`
`
`
`
`
`
`
`
`58) Field of Classification Search
`
`
`
`USPC cece ecceeeensesesceeeeee 340/12.22, 12.23
`
`
`
`
`
`
`
`See application file for complete search history.
`
`
`
`
`
`
`
`
`
`
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`
`
`
`
`
`
`
`
`6,587,125 BL*
`7,240,836 B2*
`
`
`2002/0073183 A1*
`
`
`
`
`
`
`
`7/2003 ParoZ wcrc 715/740
`
`7/2007 Vrotsos et al.
`wc 235/439
`
`
`
`
`
`6/2002 Yoon ou. GO05B 19/0421
`
`
`
`
`
`
`709/220
`
`
`
`
`
`
`
`
`
`
`REMOTE
`
`CONTROL
`
`414
`
`
`NETWORK
`22
`
`
`
`
`
`SERVERS
`24A-24N
`
`
`
`
`
`
`Breaking News
`ry
`~
`“Oo
`
`eer
`
`~AL
`CONTROLLED
`
`DEVICE
`
`48
`
`
` =
`
`GOOG-PA-00000817
`
`GOOG-PA-00000817
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 3 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 3 of 22
`
`
`
`US 9,490,998 B1
`
`Page 2
`
`(56)
`
`
`
`
`References Cited
`
`
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`
`
`
`
`
`
`“TubeMote the remote control for the web.” [online] TubeMote,
`
`
`
`
`
`
`
`
`
`
`2011. First accessed on Sep. 14, 2010. Retrieved. trom the Internet:
`
`
`
`<http://www.tubemote.com> (3 pgs.).
`
`
`
`
`
`
`
`
`
`Office Action from U.S. Appl. No. 13/249,982, dated May 24, 2012,
`
`
`12 pgs.
`
`
`
`
`
`
`
`
`
`
`Office Action from U.S. Appl. No. 13/249,982, dated Jan. 25, 2012,
`
`
`11 pgs.
`
`
`
`
`
`
`
`
`
`
`Response to Office Action dated Jan. 25, 2012 from U.S. Appl. No.
`
`
`
`
`
`
`13/249,982, filed Apr. 18, 2012, 13 pgs.
`
`
`
`
`
`
`
`
`Advisory Action before the Filing of an Appeal Brief from US.
`
`
`
`
`
`
`
`
`Appl. No. 13/249,982, mailed Jun. 21, 2016 3 pgs.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Final Office Action from U.S. Appl. No. 13/249,982, mailed Mar.
`
`
`
`23, 2016 17 pgs.
`
`
`
`
`
`
`
`
`
`Response to Final Office Action mailed Mar. 23, 2016 from U.S.
`
`
`
`
`
`
`
`Appl. No. 13/249,982,filed Jun. 2, 2016 11 pgs.
`
`
`
`
`
`
`
`
`
`Office Action from U.S, Appl. No. 13/249,982, dated Oct. 1, 2015,
`
`
`12 pp
`
`
`
`
`
`
`
`
`Response to Office Action from U.S. Appl. No. 13/249,982, filed
`
`
`
`Dec. 3, 2015 9 pp.
`
`
`
`
`
`
`
`
`
`Decision on Appeal from U.S. Appl. No. 13/249,982, dated Aug. 4,
`
`
`2015, 7 pp.
`
`
`
`
`
`
`
`
`
`Response to the Final Office Action dated May 24, 2012, and the
`
`
`
`
`
`
`
`
`
`Decision on Appeal dated Aug. 4, 2015, from U.S. Appl. No.
`
`
`
`
`
`13/249.982, filed Sep. 2, 2015, 9 pp.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`* cited by examiner
`
`
`
`GOOG-PA-00000818
`
`GOOG-PA-00000818
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 4 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 4 of 22
`
`
`U.S. Patent
`
`
`
`Nov.8, 2016
`
`
`
`
`Sheet 1 of 7
`
`
`
`
`
`US 9,490,998 B1
`
`
`
`[T|LS)[enLy
` NETWORK 22
`
`FIG.1
`
`GOOG-PA-00000819
`
`CONTROLLED
`
`DEVICE
`
`
`
`
`18
`
`SERVERS
`
`24A-24N
`
`a
`Ww
`Be
`Ory= 277
`Wo
`ro
`
`
`
`
`
`
`
`
`
`WY
`=o
`
`=>
`
`‘Ssx
`
`% &a
`
`
`
`
`
`GOOG-PA-00000819
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 5 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 5 of 22
`
`
`U.S. Patent
`
`
`
`
`Nov.8, 2016
`
`
`
`
`Sheet 2 of 7
`
`
`
`US 9,490,998 B1
`
`
`REMOTE
`
`CONTROL
`
`62A
`
`
`
`
`REMOTE
`
`CONTROL
`
`62N
`
`
`
`60
`
`
`
`
`
`CONTROLLED
`
`DEVICE
`
`
`
`
`CONTROLLED
`
`DEVICE
`
`S4N
`
`NETWORK
`
`66
`
`
`
`
`SERVERS
`
`68A-68N
`
`
`FIG. 2
`
`64A
`
`80
`
`NETWORK
`
`
`MODULE
`
`100
`
`
`
`
`STORAGE DEVICE
`92
`
`APPLICATIONS
`
`94
`
`
`
`PROCESSOR(S)
`
`96
`
`
`
`
`USER
`INTERFACE
`
`84
`
`
`
`
`
`DISPLAY
`
`88
`
`BATTERY
`
`
`
`
`
`FIG. 3
`
`GOOG-PA-00000820
`
`GOOG-PA-00000820
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 6 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 6 of 22
`
`
`U.S. Patent
`
`
`
`Nov.8, 2016
`
`
`
`
`Sheet 3 of 7
`
`
`
`
`US 9,490,998 B1
`
`118
`
`
`
`
`
`
`STORAGE DEVICE
`
`128
`APPLICATIONS
`
`130
`
`
`
`PROCESSOR(S)
`
`132
`
`
`
`
`USER
`INTERFACE
`120
`
`
`
`
`
`
`DISPLAY
`
`124
`
`
`NETWORK
`MODULE
`
`
`
`136
`164
`
`
`
`
`FIG. 4
`
`
`
`SERVER
`
`
`160
`
`
`DATA
`REPOSITORY
`
`172
`
`
`
`
`DEVICE
`MANAGEMENT
`
`MODULE
`
`168
`
`
`
`
`INTERFACE MODULE
`
`
`
`
`
`FIG. 5
`
`GOOG-PA-00000821
`
`GOOG-PA-00000821
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 7 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 7 of 22
`
`U.S. Patent
`
`Nov.8, 2016
`
`Sheet 4 of 7
`
`US 9,490,998 B1
`
`G4TIOULNOD
`
`ADSIAIAG
`
`
`
`ASIAAdGATIONLNOD
`
`
`
`NOILOVSALNDAX4A
`
`ASVSSAWTOHLNOD
`
`OLONIGHOISV
`
`(S)ZDIAAGGATIOULNOD
`
`
`
`JOULNODLINSNVYLL
`
`
`
`OlJ9VSSSW
`
`AAILON JOULNOD
`
`YaAudss
`
`ALOWSAY
`
`ONVY3SANgS
`
`dis3AI9935u
`
`($)a9IAaa
`
`GATIOULNODASILN3OGI
`
`
`O1ADVSSAWTONLNOD
`
`ONVCILINSNVYL
`
`YaAdsas
`
`GOOG-PA-00000822
`
`GOOG-PA-00000822
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 8 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 8 of 22
`
`
`U.S. Patent
`
`
`
`Nov.8, 2016
`
`
`
`
`Sheet 5 of 7
`
`
`
`
`US 9,490,998 B1
`
`220
`
`
`
`224
`
`
`
`
`
`
`
`RECEIVE MESSAGE
`
`
`HAVING REMOTE
`
`
`CONTROL IDENTIFIER/
`
`
`CONTROL MESSAGE
`
`
`RETRIEVE CONTROLLED
`
`
`DEVICE IDENTIFIER
`
`
`BASED ON REMOTE
`
`
`CONTROL IDENTIFIER
`
`
`
`MESSAGE
`
`228
`
`
`
`TRANSMIT MESSAGE TO
`
`
`CONTROLLED DEVICE
`
`
`BASED ON CONTROL
`
`
`
`FIG. 7
`
`
`
`GOOG-PA-00000823
`
`GOOG-PA-00000823
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 9 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 9 of 22
`
`U.S. Patent
`
`Nov.8, 2016
`
`Sheet 6 of 7
`
`US 9,490,998 B1
`
`ALOWSY
`
`TONLNODS
`
`YaAYSS
`
`Q3aTIONLNOS
`
`NV39VSSAWSAIS934
`
`ONVCILINSNVYL
`
`
`JLOWS3YASLLNAGI
`
`O1ADVSSAWLNALNOD
`
`(SMOYLNOD
`
`y3Au4aS
`
`ONVYSANgS
`
`
`
`daisSAIB93u
`
`ASILON ASIAAG
`
`
`
`NOILOVALNDAXA
`
`
`
`OlSNIGHODDV
`
`
`
`AJSVSSAWLNALNOD
`
`
`
`LNSLNOODLINSNVUL
`
`JLOWSYOLADVSSAN
`
`(SNOYLNOD
`
`GOOG-PA-00000824
`
`GOOG-PA-00000824
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 10 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 10 of 22
`
`
`U.S. Patent
`
`
`
`Nov.8, 2016
`
`
`
`
`Sheet 7 of 7
`
`
`
`
`US 9,490,998 B1
`
`250
`
`
`
`254
`
`
`
`
`
`
`
`RECEIVE MESSAGE
`
`HAVING CONTROLLED
`
`
`DEVICE IDENTIFIER/
`
`
`CONTENT MESSAGE
`
`
`
`
`
`RETRIEVE REMOTE
`
`
`CONTROL IDENTIFIER
`
`
`BASED ON REMOTE
`
`
`CONTENT IDENTIFIER
`
`MESSAGE
`
`258
`
`
`
`TRANSMIT MESSAGE TO
`
`
`REMOTE CONTROL
`
`
`BASED ON CONTROL
`
`
`
`FIG. 9
`
`
`
`GOOG-PA-00000825
`
`GOOG-PA-00000825
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 11 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 11 of 22
`
`
`1
`NETWORK-BASED REMOTE CONTROL
`
`
`
`
`
`
`
`US 9,490,998 B1
`
`
`2
`
`
`
`
`
`
`
`
`
`ing device to send a second message from the server to the
`
`
`
`
`
`
`
`
`
`at least one controlled device identified by the at least one
`
`
`
`
`
`
`
`controlled device identifier, wherein the second message
`
`
`
`
`
`
`includes the control information to control an operation of
`the at least one controlled device
`
`
`
`
`
`
`
`
`
`
`
`
`Tn another example, the disclosure is directed to a method
`
`
`
`
`
`
`
`
`that includesreceiving a first message from a remote control,
`
`
`
`
`
`
`
`
`wherein the first message includes a remote controlidentifier
`
`
`
`
`
`
`
`
`
`that uniquelyidentifies the remote control, and wherein the
`
`
`
`
`
`
`
`first message further includes control
`information. The
`
`
`
`
`
`
`
`
`method also includes retrieving at
`least one controlled
`
`
`
`
`
`
`
`
`device identifier from a data repository based on the remote
`
`
`
`
`
`
`
`
`control identifier, wherein the at least one controlled device
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`identifier uniquely identifies at least one controlled device.
`Various computing devices maybe used to control a home
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The method also includes sending a second message to the
`electronic device such as a television, personal computer,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`at least one controlled device identified by the at least one
`tablet computer, stereo, or other computing device capable
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`controlled device identifier, wherein the second message
`of outputting audio and/or video content. Remote controls
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`includes the control information to control an operation of
`typically communicate directly with the device being con-
`the at least one controlled device.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`trolled. For example, a user may actuate buttons on the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`remote control, which causes the remote control to transmit
`In another example, a computing device includes one or
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`more processors, a data repository configured to store data,
`a signal directly to the device being controlled. The device
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`interprets the received signal and performs a corresponding a meansfor receivingafirst message from a remote control,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`action (e.y., altering the content being played on the device).
`and a device management module. The first message
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`For instance, a user may use a remote control to change the
`received by the means includes a remote control identifier
`channel of a television.
`
`
`
`
`
`
`
`
`
`
`
`that uniquely identifies the remote control, and the first
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`In one conventional solution, a user may control the
`message further includes control information. The device
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`playback of video on a television via a web application
`management moduleretrieves at least one controlled device
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`displayed on the television using a keyboard and mouse. In
`identifier from the data repository based on the remote
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`control identifier, wherein the at least one controlled device
`another conventional solution, the user may pair a device
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`that acts as a remote control directly with the device out-
`identifier uniquely identifies at least one controlled device,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`putting the audio and video (e.g., a television or stereo), such
`and sends a second message to the at least one controlled
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`that the device outputting the audio and video acts as a
`device identified by the at
`least one controlled device
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`server to the remote control by, for example, communicating
`identifier, wherein the second message includes the control
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`directly with the remote control and accepting incoming
`information ta control an operation of the at
`least one
`connections from the remote control.
`controlled device.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The details of one or more examplesof the disclosure are
`
`
`
`
`
`
`
`
`set forth in the accompanying drawings andthe description
`
`
`
`
`
`
`
`
`below. Other features, objects, and advantages of the dis-
`
`
`
`
`
`
`
`
`closure will be apparent from the description and drawings,
`and from the claims.
`
`
`
`
`
`
`
`
`
`
`
`
`This application claims the benefit of U.S. Provisional
`
`
`
`
`
`
`
`Application No. 61/411,386, filed Nov. 8, 2010, the entire
`
`
`
`
`
`content of which is incorporated by reference herein.
`TECHNICAL FIELD
`
`
`
`
`
`
`
`
`
`
`
`
`
`The disclosure relates to techniques for controlling net-
`worked devices.
`
`
`
`
`
`10
`
`
`
`BACKGROUND
`
`
`
`hoa
`
`
`
`
`
`& a
`3
`
`
`
`ws A
`
`
`
`40
`
`
`
`BRIEF DESCRIPTION OF DRAWINGS
`
`
`
`
`
`45
`
`
`
`wn c
`
`
`
`
`
`aa
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 1 is a block diagram illustrating an example net-
`worked environment with a remote control and controlled
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`device, in accordance with one aspect ofthe present disclo-
`sure.
`
`
`
`
`
`
`
`
`FIG. 2 is a block diagram illustrating an example net-
`
`
`
`
`
`
`
`worked environment having remote controls and controlled
`
`
`
`
`
`
`
`
`devices, in accordance with one aspect of the present dis-
`closure.
`
`
`
`
`
`
`
`FIG.3 is a block diagram illustrating an example remote
`
`
`
`
`
`
`
`control, which may be an example of the remote control
`shownin FIG,1 or one of the remote controls shownin FIG.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`2, in accordance with one aspect of the present disclosure.
`
`
`
`
`
`
`
`FIG. 4 is a block diagram illustrating an example con-
`
`
`
`
`
`
`trolled device, which may be an example of the controlled
`device shown in FIG. 1 or one of the controlled devices
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`shown in FIG. 2,
`in accordance with one aspect of the
`
`
`present disclosure.
`
`
`
`
`
`
`FIG.5 is a block diagram illustrating an example server,
`
`
`
`
`
`
`which maybe an example of the server shownin FIG.1, or
`
`
`
`
`
`
`
`included within the network shown in FIG. 2, in accordance
`
`
`
`
`
`
`with one aspect of the present disclosure.
`
`
`
`
`
`FIG.6 is a flowchart illustrating an example operation of
`
`
`
`
`
`
`
`a remote control communicating with a network server, in
`
`
`
`
`
`
`
`accordance with one aspect of the present disclosure.
`
`GOOG-PA-00000826
`
`SUMMARY
`
`
`
`
`
`
`
`
`
`
`In general, this disclosure is directed to techniques for
`
`
`
`
`
`
`exchanging information between a networked device, such
`
`
`
`
`
`
`as a network-enabled television, and web-enabled device,
`
`
`
`
`
`
`
`such as a remote control, via a network service (e¢.g., a
`
`
`
`
`
`
`
`“cloud service”). In an example, the web-cnabled device can
`transmit control information via the network service to the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`networked device to control playback of media content (e.g.,
`
`
`
`
`
`
`
`
`
`audio and/or video content) on the networked device. In
`
`
`
`
`
`
`
`
`another example, the networked device can transmit content
`information via the network service to the web-enabled
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`device, such as status information concerning the networked
`device.
`
`
`
`
`
`
`
`
`In one example, the disclosure is directed to anarticle of
`
`
`
`
`manufacture
`comprising
`computer-readable
`storage
`a
`
`
`
`
`
`
`
`
`medium encoded with instructions for causing one or more
`
`
`
`
`
`programmable processors of a computing device to receive,
`
`
`
`
`
`
`
`
`bya server, a first message from a remote control, wherein
`
`
`
`
`
`
`
`
`the first message includes a remote control
`identifier that
`
`
`
`
`
`
`
`
`
`uniquely identifies the remote control, and wherein thefirst
`
`
`
`
`
`
`
`messagefurther includes control information for controlling
`one or more functions of at least one device other than the
`
`
`
`
`
`
`
`
`
`remote control. The instructions also cause on or more
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`programmable processors of the computing device to
`
`
`
`
`
`
`
`
`retrieve, by the server, at least one controlled device iden-
`
`
`
`
`
`
`
`
`tifier from a data repository based on the remote control
`
`
`
`
`
`
`
`
`identifier, wherein the at least one controlled device identi-
`
`
`
`
`
`
`
`
`fier uniquely identifies at least one controlled device that is
`distinct from and external to the server. The instructions also
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`cause on or more programmable processors of the comput-
`
`GOOG-PA-00000826
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 12 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 12 of 22
`
`
`
`US 9,490,998 B1
`
`
`3
`
`
`
`
`
`FIG. 7 is a flowchart illustrating an example operation of
`
`
`
`
`
`
`
`a network server communicating with a remote control, in
`
`
`
`
`
`
`
`accordance with one aspect of
`the present disclosure.
`
`
`
`
`
`FIG.8 is a flowchart illustrating an example operation of
`
`
`
`
`
`
`
`a controlled device communicating with a networkserver, in
`
`
`
`
`
`
`
`accordance with one aspect of
`the present disclosure.
`
`
`
`
`
`FIG. 9 is a flowchart illustrating an example operation of
`
`
`
`
`
`
`
`a network server communicating with a controlled device, in
`
`
`
`
`
`
`
`accordance with one aspect of
`the present disclosure.
`DETAILED DESCRIPTION
`
`
`
`
`5
`
`
`
`10
`
`
`
`
`
`
`
`
`
`
`
`
`
`Techniques of this disclosure relate to a network service
`
`
`
`
`
`
`
`
`or “cloud service” that acts as an intermediary between a
`
`
`
`
`
`
`
`
`remote control device and a controlled device. For example,
`
`
`
`
`
`
`
`
`the network service may receive commands from a remote
`control and transmit the commandsto a controlled device.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The network service may also receive commands or other
`2 a
`
`
`
`
`
`
`
`
`information from the controlled device and transmit those ,
`commandsor other information to the remote control. The
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`remote control may include a remote control application
`
`
`
`
`
`
`executing on a mobile device, such as a cellular telephone or
`
`
`
`
`
`
`
`
`
`a tablet computer. The controlled device may include any
`
`
`
`
`
`Internet-connected device capable of receiving commands,
`
`
`
`
`
`
`such as an Internet-connected television, a set top box, a
`
`
`
`
`
`
`
`
`personal video recorder, a gaming console, or other net-
`
`
`
`
`
`
`
`
`
`
`worked device. In one aspect, the remote control and the
`
`
`
`
`
`
`
`controlled device may operate as simple Hypertext Transfer
`& a
`
`
`
`
`
`
`
`
`Protocol HTTPclients of the network service. That is, the >
`
`
`
`
`
`
`
`
`controlled device does not operate as a server to the remote
`
`
`
`
`
`
`
`control. Thus, any HTTP-enabled device may operate as a
`remote control or as a controlled device.
`
`
`
`
`
`
`
`
`
`
`
`
`
`In general, the remote control and the controlled device
`
`
`
`
`
`
`
`
`
`are configured to both listen for messages from the network
`
`
`
`
`
`
`
`
`service and send messages to the network service. In some
`
`
`
`
`
`
`
`
`examples, the network service controls pairing one or more
`
`
`
`
`
`
`
`
`remote controls and one or more controlled devices, receives
`information or commands from remote controls and con-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`trolled devices, and sends information or commands to
`remote controls and controlled devices. The network service
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`may direct received information and commands to the
`
`
`
`
`
`
`
`appropriate devices based on pairing information main-
`
`
`
`
`
`
`
`
`tained by the network service. A remote control may be
`
`
`
`
`
`
`configured to send a message to a controlled device to
`
`
`
`
`
`
`
`perform a task, such as stopping playback of media content
`
`
`
`
`
`
`
`
`playing on the controlled devices or changing the media
`
`
`
`
`
`
`
`content playing on the controlled devices. To accomplish the
`wv c
`
`
`
`
`
`
`
`
`
`task, the remote control first sends a message to the network 5
`service. The network service then determines the controlled
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`device that is paired with the remote control and forwards
`
`
`
`
`
`
`
`
`the message to the appropriate controlled device. The con-
`
`
`
`
`
`
`
`
`
`trolled device receives the message from the network service
`
`
`
`
`
`
`
`
`and performs the task in response to receiving the message.
`
`
`
`
`
`
`
`Tn addition, the controlled device may be configured to
`
`
`
`
`
`
`
`
`send a message to the remote control to notify the remote
`
`
`
`
`
`
`
`
`control of an event. For example, the controlled device may
`
`
`
`
`
`
`
`
`send a message to the remote control to notify the remote
`aa
`
`
`
`
`
`
`
`
`
`control device that playback of media content has com-
`
`
`
`
`
`
`
`
`pleted. In this example, the controlled device sends a mes-
`
`
`
`
`
`
`
`
`sage to the network service. The network service determines
`
`
`
`
`
`
`
`
`
`the remote control that is paired with the controlled device
`
`
`
`
`
`
`
`
`
`and forwards the message to the remote control paired with
`
`
`
`
`
`
`
`
`the sending controlled device. The remote control receives
`
`
`
`
`
`
`
`
`the message and performs various actions in response to
`
`
`
`
`
`
`
`
`receiving the message. The remote control may,
`for
`
`ws A
`
`40
`
`45
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`4
`
`
`
`
`
`
`
`
`example, update a user interface of the remote control or
`
`
`
`
`
`
`
`send a messageto the controlled device to begin playback of
`another video.
`
`
`
`
`
`
`
`
`
`According to some examples, the network service may
`
`
`
`
`
`
`
`
`assign each remote control and each controlled device a
`
`
`
`
`
`
`
`
`unique identifier. When pairing devices, the network service
`
`
`
`
`
`
`
`
`
`may utilize the uniqueidentifier associated with each device
`
`
`
`
`
`
`
`
`to roule communication signals properly. For example, the
`
`
`
`
`
`
`
`
`network service may initiate a scssion that includes cach
`
`
`
`
`
`
`
`unique identifier of remote controls and controlled devices
`that are authorized to communicate with each other. The
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`network service can then route messages to members ofthe
`
`
`
`
`
`
`
`
`session. Any number of remote controls may be paired with
`
`
`
`
`
`
`
`
`
`a single controlled device and one remote control may be
`
`
`
`
`
`
`
`paired to any numberofcontrolled devices. When pairing
`
`
`
`
`
`
`
`multiple remote controls and multiple controlled devices
`
`
`
`
`
`
`
`
`
`associated with a single user, the user may identify a subset
`
`
`
`
`
`
`of the remote controls as paired to a subset ofthe controlled
`
`
`
`
`
`
`
`
`devices, and manage which remote controls control which
`controlled devices.
`
`
`
`
`
`
`
`
`
`Remote controls and controlled devices may be paired
`
`
`
`
`
`
`
`
`
`using any one of several different
`techniques. As one
`
`
`
`
`
`
`
`
`example, a user may maintain a user account using the
`
`
`
`
`
`
`
`
`network service, and the remote controls and controlled
`
`
`
`
`
`
`
`
`devices may be associated with the user account. For
`
`
`
`
`
`
`
`example, upon connecting to a network service, the remote
`
`
`
`
`
`
`
`
`controls and controlled devices may notify the network
`service that the remote controls and controlled devices are
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`connected to the network. The network service may, in some
`
`
`
`
`
`
`
`
`examples, determine whether the remote controls and con-
`trolled devices are authorized to be associated with the user
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`account. If authorized, the networkservice initiates a session
`
`
`
`
`
`
`
`
`and assigns the remote controls and controlled devices
`
`
`
`
`
`
`
`
`unique identification numbers. The network service uses the
`
`
`
`
`
`
`
`unique identification numbers for pairing during a session.
`
`
`
`
`
`
`
`
`In another example, a user may be presented with a quick
`
`
`
`
`
`
`
`
`
`
`response (“QR”) code via the controlled device that the user
`
`
`
`
`
`
`
`
`
`scans with the remote control (e.g., using a camera of the
`
`
`
`
`
`
`
`remote control). The QR code identifies a user account or
`
`
`
`
`
`
`
`previously initiated session maintained by the network ser-
`
`
`
`
`
`
`
`
`
`vice. Upon scanning the QR code, the remote control may
`
`
`
`
`
`
`
`
`send a message to the network service indicating, that the
`
`
`
`
`
`
`
`network service should assign a unique identification num-
`
`
`
`
`
`
`
`
`
`
`
`ber to the remote control and pair the remote control with the
`
`
`
`
`
`
`user account or session identified by the QR code.
`
`
`
`
`
`
`
`
`
`In this manner, one or more remote controls may control
`one or more controlled devices via the network service.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Using the network service to transmit and receive messages
`
`
`
`
`
`
`
`between a remote control and a controlled device may
`
`
`
`
`
`
`
`
`enable non-traditional devices having rich input and display
`
`
`
`
`
`
`capabilities to act as a remote control. In addition, by using
`
`
`
`
`
`
`
`the network service as an intermediary, the remote control
`
`
`
`
`
`
`
`
`and the controlled device, in various instances, may not need
`
`
`
`
`
`
`
`
`
`to be connected to the same local area network, nor in
`
`
`
`
`
`
`
`
`physical proximity to each other. The network service may
`
`
`
`
`
`
`
`also enable pairing of a nearly limitless number of remote
`controls and controlled devices.
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 1 is a block diagram illustrating an example net-
`worked environment 10 with a remote control 14 and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`controlled device 18, in accordance with one aspect of the
`
`
`
`
`
`present disclosure. According to an aspectofthe disclosure,
`remote control 14 communicates with controlled device 18
`
`
`
`
`
`
`
`
`
`
`
`
`
`via network 22 and servers 24A-24N (collectively “servers
`
`
`
`
`
`
`
`24”) in network 22. As shownin FIG. 1, according to some
`
`
`
`
`
`
`
`
`examples, remote control 14, controlled device 18, and
`
`
`
`
`
`
`
`servers 24 may be distinct components (e.g., physically
`
`distinct).
`
`GOOG-PA-00000827
`
`GOOG-PA-00000827
`
`

`

`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 13 of 22
`Case 6:21-cv-00569-ADA Document 153-4 Filed 01/12/23 Page 13 of 22
`
`
`
`US 9,490,998 B1
`
`
`6
`
`
`
`
`
`
`
`
`
`
`
`is rendered on display 38 of
`to alter the content
`that
`
`
`
`
`
`
`
`controlled device 18. In another example, controlled device
`
`
`
`
`
`
`
`18 transmits a message to remote control 14 via network 22
`
`
`
`
`
`
`
`
`and servers 24 to notify the user of remote contro! 14 that
`
`
`
`
`
`
`
`playback of content has been completed. Other messages
`can also be transmitted between controlled device 18 and
`
`
`
`
`
`
`
`remote control 14 via network 22 and servers 24. For
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`example, controlled device 18 may transmit content infor-
`
`
`
`
`
`
`
`
`
`
`mation such as a preview of the content
`is being
`that
`
`
`
`
`
`
`
`
`displayed on controlled device 18, a playlist of future
`
`
`
`
`
`
`
`
`to be displayed on controlled device 18, rating
`content
`
`
`
`
`
`
`
`information associated with the content being displayed on
`
`
`
`
`
`controlled device 18, or the like.
`
`
`
`
`
`
`
`
`As shown in FIG. 1, network 22 and servers 24 are
`connected to remote control 14 and controlled device 18.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Network 22 may be connected to remote control 14 and
`controlled device 18 via wired and/or wireless links. For
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`example, remote control 14 and controlled device 18 may be
`
`
`
`
`
`
`coupled to network 22 via a combination of any wireless or
`
`
`
`
`
`
`wired communication medium, such as a radio frequency
`
`
`
`
`
`
`
`(RF) spectrum or one or more physical transmission lines, or
`
`
`
`
`
`
`
`
`
`any combination of wireless and wired media. The links may
`
`
`
`
`
`
`
`form part of a packet-based network, such as a local area
`
`
`
`
`
`
`network, a wide-area network, or a global network such as
`
`
`
`
`
`
`
`
`the Internet. The links generally represent any suitable
`
`
`
`
`
`communication medium, or collection of different commu-
`
`
`
`
`
`
`
`
`nication media, for transmitting signals from remote control
`14 to network 22 and from controlled device 18 to network
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`22, including any suitable combination ofwired or wireless
`
`
`
`
`
`
`
`
`
`media. The links may include routers, switches, base sta-
`
`
`
`
`
`
`
`
`tions, or any other equipment that maybe usefulto facilitate
`communication from remote control 14 and controlled
`
`
`
`
`
`
`
`device 18 to network 22.
`
`
`
`
`
`
`
`
`
`
`Network 22 may include a wide-area network suchas the
`
`
`
`
`
`
`Internet, a local-area network (LAN), an enterprise network,
`
`
`
`
`
`
`
`
`or one or more other types of networks. Servers 24 may be
`
`
`
`
`
`
`
`
`
`any of several different
`types of network devices. ['or
`
`
`
`
`
`
`
`instance, servers 24 may be conventional web servers,
`
`
`
`
`
`
`specialized media servers, personal computers operating in
`
`
`
`
`
`
`
`a peer-to-peer fashion, or other types of network devices. As
`
`
`
`
`
`
`
`
`described in greater detail with respect to FIG. 5, in some
`
`
`
`
`
`
`
`examples, servers 24 include one or more data repositories
`
`
`
`
`
`
`
`or databases for storing information, such as identification
`
`
`
`
`
`
`
`numbers, session identifiers, and any other information
`
`
`
`
`
`
`
`
`required to carry out data transfer between remote control 14
`
`
`
`
`
`
`
`
`
`and controlled device 18.
`In some examples,
`the data
`
`
`
`
`
`
`repositories of servers 24 may bestructured as one or more
`
`
`
`
`
`
`
`
`
`tables, database systems, linked lists, radix trees, or other
`suitable data structure.
`
`
`
`
`
`
`
`
`According to an aspect of the disclosure, network 22 and
`
`
`
`
`
`
`
`
`servers 24 facilitate an exchange of data between remote
`
`
`
`
`
`
`
`
`control 14 and controlled device 18. For example, servers 24
`
`
`
`
`
`
`associated with network 22 may operate as an intermediary
`
`
`
`
`
`
`
`between remote control 14 and controlled device 18 to relay
`data between remote control 14 and controlled device 18. In
`
`