(12) United States Patent
`Strober
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,782,528 B2
`Jul. 15, 2014
`
`US008 782528B2
`
`(54)
`
`(71)
`
`(72)
`(73)
`
`(*)
`
`(21)
`(22)
`(65)
`
`(63)
`
`(60)
`
`(51)
`
`(52)
`
`(58)
`
`PLAY CONTROL OF CONTENT ON A
`DISPLAY DEVICE
`
`Applicant: Touchstream Technologies, Inc.,
`Valhalla, NY (US)
`Inventor: David Strober, Rye, NY (US)
`Assignee: Touchstream Technologies, Inc., New
`York, NY (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`Notice:
`
`Appl. No.: 13/736,590
`
`Filed:
`
`Jan. 8, 2013
`
`Prior Publication Data
`US 2013/O124759 A1
`May 16, 2013
`
`Related U.S. Application Data
`Continuation of application No. 13/245,001, filed on
`Sep. 26, 2011, now Pat. No. 8.356,251, which is a
`continuation of application No. 13/157,821, filed on
`Jun. 10, 2011.
`Provisional application No. 61/477,998, filed on Apr.
`21, 2011.
`
`Int. C.
`G06F 3/00
`U.S. C.
`USPC ........... 715/740; 715/734; 715/738; 715/751:
`715/764; 715/835
`
`(2006.01)
`
`Field of Classification Search
`USPC .................. 715/740, 738, 734, 751, 764, 835
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,855,842 A * 8/1989 Hayes et al. .................. 386,206
`7,058,356 B2
`6/2006 Slotznick
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`CN
`CN
`CN
`CN
`
`9, 2009
`1.01534449
`11, 2009
`1 O157765O
`T 2010
`101778198
`8, 2010
`101815073
`OTHER PUBLICATIONS
`anyremote.sourceforge 2013.*
`Bing search q=control+television+phone&go=Submit Feb. 6, 2014.*
`(Continued)
`Primary Examiner — Boris Pesin
`Assistant Examiner — John Heffington
`(74) Attorney, Agent, or Firm — Fish & Richardson P.C.
`(57)
`ABSTRACT
`A system for presenting and controlling content on a display
`device includes a network, a server system coupled to the
`network and comprising one or more servers, a display device
`coupled to the network and having a display, and a personal
`computing device operable to transmit a first message accord
`ing to a specified format over the network to the server sys
`tem. The server system stores an association between the
`personal computing device and the display device. The first
`message identifies user-selected content and a media player to
`play the content. The server system is operable, in response to
`receiving the first message from the personal computing
`device, to provide to the display device a second message
`identifying the user-selected content and the media player to
`play the content. In response to receiving the second message,
`the display device is operable to obtain a first media player
`needed to play the content, to load the media player and to
`present the content on the display.
`30 Claims, 9 Drawing Sheets
`
`Server System
`
`Switchboard
`
`10
`24
`
`28
`
`34
`
`26
`APAdapter
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 1 of 19
`
`
`
`Database
`
`e-e
`
`36
`look-up
`
`32
`
`able - Databasel
`
`
`
`Contert Providers
`
`

`

`US 8,782.528 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2009/0254827 A1* 10, 2009 Gonze et al. .................. 715,716
`2009,0259944 A1* 10, 2009
`715.738
`2009/0282470 A1* 1 1/2009 Yang et al. ...................... T26, 12
`2010, OO27974 A1
`2/2010 Ansari
`2010/0094728 A1* 4/2010 Denning et al. ................ 705/27
`2010.0137028 A1
`6, 2010 Farris et al.
`2010, 013874.6 A1* 6, 2010 Zarom .......................... 71.5/720
`2010/0174993 A1* 7/2010 Pennington et al.
`... 715.738
`2010, 019886.0 A1* 8, 2010 Burnett et al. ................ 707/769
`2010/0205628 A1
`8/2010 Davis et al.
`2010/0208136 A1* 8, 2010 Castano ........................ 348,553
`2010, O257569 A1* 10, 2010 O’Hanlon ....
`T25,110
`2010/0265939 A1 * 10/2010 Parlamas et al. .............. 370,352
`2010/0281042 A1* 11/2010 Windes et al. ................ 707/756
`2010/0283586 A1* 11/2010 Ikeda et al. .....
`340, 10.42
`2010/0313135 A1 12/2010 Johnson et al. .
`... 715.738
`2010/0325552 A1* 12/2010 Slooet al. ....
`715.738
`2011/0007901 A1
`1/2011 Ikeda et al. .....
`380,270
`2011/00 14972 A1
`1/2011 Herrmann et al. .............. 463,25
`2011/003.0020 A1
`2/2011 Halttunen
`2011/0032870 A1
`2/2011 Kumar .......................... 370,328
`2011/0035692 A1
`2/2011 Sandone et al.
`... 715,769
`2011 OO60998 A1* 3, 2011 Schwartz et al.
`... 715.738
`2011/0090898 A1* 4, 2011 Patel et al. ......
`370,352
`2011/0107227 A1* 5/2011 Rempell et al.
`... 715.738
`2011/O125594 A1* 5, 2011 Brown et al. ...
`705/14.73
`2011 0137729 A1* 6, 2011 Weisman et al. .......... 705, 14.58
`2011/O138354 A1* 6, 2011 Hertenstein et al.
`717,115
`2011/O156879 A1* 6, 2011 Matsushita et al. .......... 34.0/10.1
`2011/O161396 A1* 6, 2011 Fibrich et al. ..
`TO9,203
`2011/0202466 Al
`8, 2011 Carter
`..........
`705/67
`2011/0214148 A1* 9, 2011 Gossweiler et al.
`725/46
`2011/0228768 A1* 9, 2011 Gelter et al. ....
`370,389
`2011/0231265 A1* 9, 2011 Brown et al.
`705/14.73
`2011/0231565 A1
`9, 2011 Gelter et al. .................. TO9,231
`2011/0231566 A1* 9, 2011 Gelter et al. .................. TO9,231
`2011/0267981 A1* 11/2011 Davies .....
`370,255
`2011/02894.19 A1 1 1/2011 Yu et al. .
`715.738
`2011/0296454 A1* 12/2011 Xiong et al.
`.................. 725/30
`2011/0296465 A1 12/2011 Krishnan et al.
`2012/0072846 Al
`3/2012 Curtis............................ 715.738
`2012/0110074 A1* 5, 2012 Getchius ..
`TO9.204
`2012/01 10464 A1* 5, 2012 Chen et al. ...
`... 715.738
`2012/0166560 A1* 6, 2012 Nobori et al. ................. TO9,206
`2012/0182994 A1* 7, 2012 Dec et al. ...................... 370,392
`2012fO185887 A1* 7, 2012 Newell .....
`725/12
`2012/0239218 A1* 9/2012 Forbes, Jr. ...
`7OO 295
`2012fO254931 A1* 10/2012 Oztaskent et al.
`725, 112
`2013/0014142 A1
`1/2013 Newell ............................ 725/12
`
`
`
`OTHER PUBLICATIONS
`
`7,088,823 B2 * 8/2006 Fetkovich ..................... 380,255
`7,330,875 B1* 2/2008 Parasnis etal
`709/204
`7,424,718 B2 * 9/2008 Dutton .......................... T19,318
`7,433,922 B2 * 10/2008 Engstrom ..................... 709/205
`7.440,972 B2 * 10/2008 Oetzel .........
`386.252
`7,453.454 B2 * 1 1/2008 Allen et al. ........
`345,418
`7,466,823 B2 * 12/2008 Vestergaard et al. ......... 380/227
`7,769,827 B2* 8/2010 Girouard et al.
`709,219
`7,774,708 B2* 8/2010 Bell et al. .......
`715.738
`7.814,144 B2 * 10/2010 Koyama et al. ............... TO9,203
`7,835,505 B2 11/2010 Toyama et al.
`7,849,485 B2 12/2010 Paik et al.
`8,086,679 B2 * 12/2011 Nobori et al. ................. TO9,206
`8,171,507 B2 *
`5/2012 Hironaka et al. ..
`725/12
`8.433,812 B2 * 4/2013 Mahajan et al. ...
`709,231
`2002fOO75332 A1* 6, 2002 GeilfuSS et al. ....
`345,859
`2002/O120666 A1* 8, 2002 Landsman et al. .
`709/200
`2002/01291.02 A1* 9, 2002 Landsman et al. .
`709f2O3
`2002/0133518 A1* 9, 2002 Landsman et al. ....
`707/513
`2002/0146122 A1* 10/2002 Vestergaard et al. ......... 380/231
`2002fO198778 A1* 12/2002 Landsman et al. .............. 705/14
`2003/0004804 A1
`1/2003 Landsman et al. .
`TO5/14
`2003/0005000 A1
`1/2003 Landsman et al. ............ 707/513
`2003/0018885 A1
`1/2003 Landsman et al. ................ 713/2
`2003/0023488 A1
`1/2003 Landsman et al. .............. TO5/14
`2003/0028565 A1
`2/2003 Landsman et al. ............ 707/513
`2003. O112258 A1* 6, 2003 Dietz et al. .....
`345/700
`2003. O131251 A1* 7, 2003 Fetkovich
`713, 193
`2003. O1421.27 A1* 7, 2003 Marke1 .......
`345.738
`2003/0182663 A1* 9, 2003 Gudorf et al.
`725,110
`2003.0193520 A1* 10, 2003 Oetzel ......
`345/723
`2004/OOO8972 A1* 1 2004 Haken ......
`... 386/83
`2004/0088728 A1* 5, 2004 Shimizu ...
`725/89
`2004/0268224 A1* 12/2004 Balkus et al. .............. T15,500.1
`2004/02684.51 A1* 12/2004 Robbin et al. ......... 999,999.999
`2005.0034151 A1
`2/2005 Abramson
`2005/0144305 A1* 6/2005 Fegan et al. ................... TO9,231
`2006/0062544 A1
`3f2006 Southwood et al. ............ 386/46
`2006, 0083194 A1
`4/2006 Dhrimajet al.
`2006/0098.624 A1* 5/2006 Morgan et al. ................ 370,352
`2006/0101098 A1* 5/2006 Morgan et al.
`707,204
`2006/0200832 A1* 9, 2006 Dutton .......................... T19,318
`2006/02O3758 A1
`9, 2006 Tee et al.
`2006/0263038 A1* 1 1/2006 Gilley ............................. 386,52
`2006/0265657 A1* 1 1/2006 Gilley ........................... 715,730
`2007/005.0054 A1* 3/2007 Sambandam Guruparan
`et al. ............................... TOOf 65
`2007/0055986 A1* 3/2007 Gilley et al.
`T25/34
`Bing search q=control+television+server+phone&qs Feb. 6, 2014.*
`2007, 0083540 A1* 4, 2007 Gundla et al. .
`707/101
`Boxee-smartphone-remote-control Dec. 11, 2011.*
`2007/0094408 A1* 4, 2007 Gundla et al. .
`792
`ck
`F1
`bill
`h
`2007/01 12785 A1* 5/2007 Murphy et al. ................. 707/10
`ile guru mobile phone television remote control 2013.
`ck
`2007/0136778 A1* 6/2007 Birger et al. .................. 725, 117
`Mashable Smartphone tv remote control apps Nov. 28, 2012.
`2007/0150963 A1* 6/2007 Lee et al. ........................ 726/27
`Hachman, M., "Snapstick's Media Streaming AppBox: Hands On.”
`2007/0202923 A1
`8/2007 Jung et al.
`www.pcmag.com/article/2/0,2817.2375455,00.asp, 2 pages, (Jan. 8,
`2007/0288715 A1 12, 2007 Boswell et al.
`2011).
`2007/0294621 A1* 12/2007 Hansen et al. ................ 715,716
`Dolcourt, J., CES: Snapstick takes on Apple TV. Google TV. http://
`2008, OOO8439 A1
`1/2008 Liu et al. .....
`... 386/46
`news.cnet.com/8301-17938 105-20025100-1.html, 3 pages, (Dec.
`2008/0028037 A1* 1/2008 Moyer et al.
`702
`9, 2010).
`2008.0034394 A1
`2/2008 Jacobs et al. .................... 725/98
`Shaivitz, M.. “The Webto Your TV. With a Flick of a Wrist'? Slapstick
`38885 A. 39.
`Sam
`7002
`Says Yes.” http://techcocktail.com/the-web-to-our-tv-with-a-flick
`2008/O126943 A1* 5, 2008 Parasnis etal
`715,730
`of-a-Wrist-slapstick-says-yes-2010-12, 2 pages, (Dec. 10, 2010).
`2008/O140849 A1* 6/2008 Collazo ......................... 709,229
`SnapStick—Home, "SnapStick.
`http://www.SnapStick.com/, 2
`2008. O155600 A1
`6/2008 Klappert et al.
`pages, printed on Mar. 2, 2011.
`2008/0162670 A1* 7/2008 Chapweske et al. .......... TO9,219
`3886- A. 2.
`ity
`"'s Paul, I., Hands On: YouTube Leanback, PCWORLD, http://www.
`2008/0267369 A1* 10, 2008 Parlamas et al.
`379,93.01
`Rosslysis 200769 hands on youtube lenback.html. 3
`2008/0270881 A1* 10/2008 Meyer et al. .................. 713,203
`pages, (Jul 9,
`).
`2008/0301737 A1* 12/2008 Hjelmeland Almas
`Using AirPlay, Article HT4437, http://support.apple.com/kb/
`et al. ............................... 725/61
`HT4437, 3 pages, (Apr. 18, 2011).
`2009.0049373 A1* 2, 2009 Sharma et al.
`715,234
`Cheng, J., “Stream AirPlay video to regular TV'? Apple might make
`2009,009.4331 A1* 4, 2009 Nobori et al. ..
`709/205
`it happen.” http://arstechnica.com/apple/news/2011/03/stream-air
`2009/0164641 A1* 6/2009 Rogers et al. ................. 709,227
`play-video-to-a-regular-tv-apple-migh. . . , 1 page, printed on Jun. 7.
`2009/0228919 A1
`9, 2009 Zott et al.
`2011.
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 2 of 19
`
`OW ca. ...
`
`

`

`US 8,782.528 B2
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`“Using the Play To feature to stream media.” http://windows.
`microsoft.com/en-US/windows7/using-the-play-to-feature-to
`stream-media, 3 pages, printed on Jun. 7, 2011.
`“YouTube—Leanback.” http://www.youtube.com/t/leanback,
`page, printed on Jun. 7, 2011.
`“Yahoo!7 TV Guide for iPhone, iPod touch and iPad on the iTunes
`App Store.”
`http://itunes.apple.com/au/app/yahoo-7-tv-guide/
`id4247 1992?mt=8, 2 pages, printed on Jun. 7, 2011.
`Hu, C., et al., “MobileMedia Content Sharing in UPnP-Based Home
`Network Environment.” Journal of Information Science and Engi
`neering 24, 1753-1769. (2008).
`Fallahkhair, S., et al., “Dual Device User interface Design for Ubiq
`uitous Language Learning: Mobile Phone and Interactive Television
`
`1
`
`(iTV).” Proceedings of the 2005 IEEE Int’l Workshop on Wireless an
`Mobile Technologies in Education, 8 pages, 2005.
`Ask Search Internet Search, Session identifier random, printed on
`Nov. 19, 2011.
`Webopedia computer dictionary, Session cookie, printed on Nov. 19.
`2011.
`Webopedia computer dictionary, web identifier, printed on Nov. 19.
`2011.
`Webopedia computer dictionary, user session, printed on Nov. 19.
`2011.
`www.vbulletin.com, Best way to generate Random, Unique ID's,
`printed on Nov. 19, 2011.
`www.vbulletin.com, Best way to generate Random, Unique ID's,
`Internet Archive Wayback Machine, Jan. 16, 2009.
`* cited by examiner
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 3 of 19
`
`

`

`U.S. Patent
`
`Jul. 15, 2014
`
`Sheet 1 of 9
`
`US 8,782,528 B2
`
`fa- ---------ca-car
`-
`SWitCOard
`
`-a-
`
`26
`APAdapter
`- -
`
`
`
`
`
`
`
`
`
`Ratabase
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 4 of 19
`
`internet
`
`FG.
`
`

`

`U.S. Patent
`
`|-?s???,TELGETEIST?T
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 5 of 19
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Jul. 15, 2014
`
`Sheet 2 of 9
`
`US 8,782,528 B2
`
`| ---------------------------~~~~
`oºppaeae peori|
`
`
`
`—e;
`
`

`

`U.S. Patent
`
`Jul. 15, 2014
`
`Sheet 3 of 9
`
`US 8,782,528 B2
`
`ransmission Code
`UserID targetid MediaPlayerID
`data
`Connan
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ise -
`Smartphone
`
`
`
`26
`
`Universa AP Adapter
`Universal Command MediaPlayer
`Specific Player Command
`New Video
`vote ---
`y loadVideo
`Ted.com
`getVideo
`Vimeo
`oadMewVideo
`YouTube
`yt pauseVideo
`Ted. CO
`pauseVideo
`Vired
`pause
`F.G. 5
`
`
`
`
`
`
`
`--
`
`
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 6 of 19
`
`Pase
`
`--
`
`---
`
`-
`
`- ---
`
`-
`
`

`

`U.S. Patent
`
`Jul. 15, 2014
`
`Sheet 4 of 9
`
`US 8,782,528 B2
`
`-1 the Media Player that N
`Y
`re-is Needed to Play the Yeouid
`S Already loaded in the
`NDisplay Deviceu1
`N2-1
`No
`
`Yes
`
`Request and Obtain a
`Copy of the Appropriate
`l
`Media Player and a Copy
`-1 N of the Video Fe fron
`Content Provider
`:
`is
`a Copy of the
`Sirr
`Video Aiready loaded in)
`the Display device
`
`Media Player
`load the
`
`
`
`
`
`NO
`
`Obtain a Copy
`of the Video
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 7 of 19
`
`FG. 6
`
`

`

`U.S. Patent
`
`Jul. 15, 2014
`
`Sheet 5 of 9
`
`US 8,782,528 B2
`
`22s
`
`Display Device
`
`Splash Page
`48
`
`435-05-342
`
`-- Sync-Code
`
`
`
`
`
`Sync-code Look-up Table
`Cookie
`IPAddress
`erg988dhuj
`169.343,231.234
`FG. 73
`
`Sync-code
`435-05-342
`
`ar Display Device
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 8 of 19
`
`
`
`Smart Pho
`20
`
`e
`
`SmartPhone
`
`
`
`isplay Device
`
`spie
`
`20 na
`
`F.G. 8
`
`(A
`(A)
`
`B)
`F.G. 9 (B)
`
`

`

`U.S. Patent
`
`Jul. 15, 2014
`
`Sheet 6 of 9
`
`US 8,782,528 B2
`
`Display Device
`
`
`
`smartPhone r Phone smartPhone
`|
`20
`20
`20
`Moderator
`Contro of
`Display
`
`R
`-a-ra
`
`F.G. O.
`
`------------------
`
`Display Device
`
`
`
`Display Device
`
`
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 9 of 19
`
`

`

`U.S. Patent
`
`Jul. 15, 2014
`
`Sheet 7 of 9
`
`US 8,782,528 B2
`
`Display Device
`
`Display Device
`
`
`
`st u
`
`Smart Phone Smart Phone Smart phone
`20
`f"
`Contro of
`Display
`
`Moderator
`
`F.G. 12
`
`Display Device
`
`Display device
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 10 of 19
`
`|Smar Oe
`
`re. -
`
`| Third Party
`
`Smart Phone
`
`Piayist and Video links
`FG. 3
`
`

`

`U.S. Patent
`
`
`
`US 8,782,528 B2
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 11 of 19
`
`

`

`U.S. Patent
`
`Jul. 15, 2014
`
`Sheet 9 of 9
`
`US 8,782,528 B2
`
`T? T? T?
`
`0:0. V | X | 4 |||||—
`
`±¿?|
`
`?), e )
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 12 of 19
`
`

`

`US 8,782,528 B2
`
`1.
`PLAY CONTROL OF CONTENT ON A
`DISPLAY DEVICE
`
`BACKGROUND
`
`This disclosure relates to play control of content on a
`display device. Such display devices include, for example,
`television displays used by consumers in their home for view
`ing videos and other media that are either provided from the
`Web or previously stored. In particular, the disclosure relates
`to the creation, storage, manipulation and access of media
`playlists used in conjunction with display devices and control
`of the display devices.
`Web media often is played on computers rather than tele
`vision displays. Although it is knownto connecta computer to
`a television set in order to watch Web media, it is difficult to
`control Such a system within the typical scenario for televi
`sion watching where the viewer is positioned some distance
`from the television. Furthermore, although a wireless device
`can enable the user to control the television from a distance, it
`can be difficult to view a web browser display on the televi
`sion set and may interfere with normal television program
`viewing by other persons.
`Given the desire to watch various World WideWeb media
`on a family's primary television set, and to control this opera
`tion from the comfort of one’s couch, there is a need to
`operate a television set or other display remotely from a
`personal computing device. Such as a mobile phone. It also is
`desirable to allow a user to perform a general Web search to
`locate and capture Web media, and to control a television or
`other display remotely using the personal computing device.
`
`10
`
`15
`
`25
`
`30
`
`2
`that includes the corresponding command, and the display
`device is operable, in response to receiving the message from
`the server System, to execute the command.
`In some implementations, the personal computing device
`is, for example, a mobile phone, and the display device is a
`television set. Other personal computing devices or display
`devices can be used in other implementations. The network
`can include, for example, the Internet.
`In some implementations, the server system stores a look
`up table that includes a synchronization code uniquely asso
`ciated with the display device. A message from the personal
`computing device can include the synchronization code, and
`in response to receiving the message from personal comput
`ing device, the server system can use the synchronization
`code and the look-up table to identify the display device on
`which the content is to be played. The synchronization code
`can be different from an IP address associated with the dis
`play device and/or a media access control address associated
`with the display device.
`In various implementations, the system can facilitate
`allowing a personal computing device to be used to select
`different content to be played on a remote display even if
`different media players are required to present the different
`content. The system also can allow the user to control how the
`content is displayed on the display device using the personal
`computing device. For example, user-initiated play com
`mands can be passed from the user's personal computing
`device, through the server system, to the display device.
`Other aspects, features and advantages will be apparent
`from the following detailed description, the accompanying
`drawings, and the claims.
`
`SUMMARY
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Various aspects of the invention are set forth in the claims.
`For example, according to one aspect, a system for present
`ing and controlling content on a display device includes a
`network, a server system coupled to the network and com
`prising one or more servers, a display device coupled to the
`network and having a display, and a personal computing
`device operable to transmit a first message according to a
`specified format over the network to the server system. The
`server system stores an association between the personal
`computing device and the display device. The first message
`identifies user-selected content and a media player to play the
`content. The server system is operable, in response to receiv
`ing the first message from the personal computing device, to
`provide to the display device a second message identifying
`the user-selected content and the media player to play the
`content. In response to receiving the second message, the
`display device is operable to obtain a first media player
`needed to play the content, to load the media player and to
`present the content on the display.
`In some implementations, the display device is operable, in
`response to receiving the second message, to obtain the first
`media player from the content provider only if the first media
`player is not already loaded in the display device.
`In some implementations, the personal computing device
`is operable to transmit a message according to a specified
`format over the network to the server system. The message
`can include a command for controlling playing of the content
`on the display device. The server system is operable, in
`response to receiving the message, to convert the command
`into a corresponding command recognizable by the media
`player if the command received from the personal computing
`device is not recognizable by the media player. The server
`system is operable to provide to the display device a message
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`FIG. 1 is a block diagram illustrating an example of a
`system according to the invention.
`FIG. 2 illustrates various details of the flow of information
`and signals according to some implementations.
`FIG.3 illustrates an example of a transmission code incor
`porated into a message from a personal computing device.
`FIG. 4 illustrates an example of a look-up table that forms
`part of a server system.
`FIG. 5 illustrates an example of entries in a universal API
`adapter.
`FIG. 6 is a flow chart showing steps for display device to
`load a video player and video.
`FIG. 7A illustrates an example of a display device includ
`ing a synchronization code.
`FIG. 7B illustrates an example of a synchronization code
`look-up table.
`FIGS. 8-13 illustrate examples of various scenarios in
`which the invention can be used.
`FIGS. 14A through 14E illustrate examples of display
`screens that may appear on a user's personal computing
`device in accordance with the invention.
`FIG. 15 illustrates further information that can be stored in
`the look-up table in the server system.
`
`DETAILED DESCRIPTION
`
`As shown in FIG. 1, a system 10 facilitates synchronizing
`a connection between two or more devices 20, 22 connected
`to the Internet 21 or other computer network. The connection
`is designed to be made by a first device (e.g., a personal
`computing device) 20 that acts as a controller and a second
`device (e.g., a television set 22 with a display 23) that acts as
`a receiver to play content selected by a user of the first device
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 13 of 19
`
`

`

`4
`content to a playlist, (ii) play a listed item on a secondary
`device, (iii) play a listed item on the mobile phone 20. If the
`user selects to add new content to the playlist, the user is
`presented with a screen that allows him to enter user-defined
`search parameters oro select predefined search parameters to
`request video data. The search parameters are sent from the
`mobile phone 20 as part of a request for video data that satisfy
`the search parameters (block 102). The request is transmitted
`via the Internet 21 and through the server system 24 to the
`appropriate content provider web site. In response, the con
`tent provider 30 provides metadata (e.g., titles, links to the
`videos) for one or more video files that satisfy the search
`parameters (block 104). The metadata can be provided to the
`mobile phone 20, for example, in the form of an XML data
`file. Upon receiving the data file, the mobile phone 20 dis
`plays a list of one or more videos based on the information
`received from the content provider 30 (block 106).
`If desired, the user can take one of several actions, includ
`ing selecting one of the videos from the displayed list to be
`played on the television display 23 or initiating a command
`with respect to a video that already has been loaded to the
`television set 22 (block 108). The mobile phone 20 then
`formats and transmits a message to the server system 24
`(block 110). The message from the mobile phone 20 contains
`a transmission code that includes data regarding the user
`information (e.g., user identification or account number), the
`secondary display it wants to connect to (e.g., television set 22
`with display 23), the location and name of the media player
`for the selected video, the command (e.g., play, pause,
`rewind, etc.), and the video file to be acted upon. An example
`of the format of a transmission code from the mobile phone 20
`to the server system 24 is illustrated in FIG. 3. Different
`formats and/or different information may be appropriate for
`other implementations.
`The message from the mobile phone 20 is transmitted over
`the Internet 21 and is received by the server system 24 (block
`112). Based on information in the message from the mobile
`phone 20, the server system 24 verifies that the user has an
`account (block 114), and the contents of the message, as well
`as the date and time of receipt of the message, are added to a
`personal computing device database 32 (block 116) which
`forms part of a Switchboard 28. In general, all messages from
`a particular user's personal computing device 20 are stored in
`the database32 corresponding to an account for the particular
`user. Thus, the database 32 stores a record of all messages
`received from a user's personal computing device 20, as well
`as the user's identification, an indication of the target device
`22, an identification of the media player that is required for the
`selected video, and an identification of the selected video.
`The switchboard 28 also includes a look-up table 34 that
`stores a correspondence between a particular personal com
`puting device (such as mobile phone 20) and target devices
`(e.g., the television set 22) to which the user command is
`directed. An example of the look-up table 28 is illustrated in
`FIG. 4. In this example, it is assumed that, at most, a single
`connection is established at any given time between a particu
`lar mobile phone and a display device. However, as explained
`below, other scenarios are also possible to establish group
`connections (e.g., multiple mobile phones connected to the
`same display device). The server system 24 performs a target
`verification (block 118), which includes checking whether a
`connection to a particular display device already is estab
`lished for the mobile phone 20 and, if so, checking the iden
`tification of the display device. During the target verification,
`if the look-up table indicates that there is no connection
`established between the mobile phone 20 and a particular
`display device, then the server system 24 sends a message to
`
`Case 6:21-cv-00569-ADA Document 1-2 Filed 06/04/21 Page 14 of 19
`
`45
`
`55
`
`60
`
`65
`
`US 8,782,528 B2
`
`15
`
`25
`
`35
`
`3
`and to respond to commands that originate at the personal
`computing device. The personal computing device 20 is oper
`able to display an application or web site that contains infor
`mation and links to content providers 30 on the Internet 21.
`The television set 22 is operable to link back to a server
`system 24 from which the television set receives commands.
`When a user makes a selection using the personal computing
`device 20 for particular content to be displayed on the televi
`sion display 23, a signal is sent through the Internet (or other
`network) 21 to the server system 24. A corresponding com
`10
`mand signal then is passed along to the connected television
`set 22, which acts on a transmission code contained within the
`signal and performs specified commands. For example, in
`Some scenarios, the commandinstructs the television set 22 to
`access a content provider 30 through the Internet 21, load a
`specific media player, load the media player-specific content
`(e.g., a video) and play the content on the television display
`23. The user can use the personal computing device 20 to
`control how the content is played on the television display 23.
`The user may subsequently visit the same or another Web site
`using the personal computing device 20 to select different
`content (e.g., a second video) to he played on the television
`display 23. In that case, another signal would be sent through
`the server system 24 to the television set 22. A transmission
`code associated with this command signal instructs the tele
`vision set 22 to load a new media player (if needed) over the
`Internet and to load the specified video file to be played on the
`display 23. Thus, the system 10 allows a personal computing
`device 20 to be used to select different content to be played on
`a remote display 23 even if different media players are
`required for the different content. The user also can control
`how the content is displayed (e.g., play, pause, stop, rewind,
`fast forward, etc.) on the display 23 using the personal com
`puting device 20. The user-initiated play commands are
`passed from the user's personal computing device 20,
`through the server system 24, to the television set 22.
`Although the following detailed discussion describes vid
`eos as an example of the type of content to be played on the
`display 23, the system 10 can be used for other types of
`content as well. Thus, depending on the implementation, the
`40
`content may include one or more of the following: video,
`audio, interactive video game, streaming media, multimedia,
`images, slides (e.g., a PowerPoint presentation) or other types
`of dynamic content. Furthermore, in the following discus
`Sion, it is assumed that the personal computing device 20 is a
`mobile phone that includes a display, an internal micropro
`cessor or other processing circuitry, a keypad, keyboard,
`touchscreen, mouse, trackball, or other device to receive user
`selections and other input, and a transceiver to establish com
`munications to the Internet 21 or other communications net
`50
`works. More generally, however, the personal computing
`device 20 can be any type of handheld or other Internet
`enabled personal computing device, including personal com
`puters, e-books, kiosks, tablets, Smartphones, media players,
`and motion and touch sensory interfaces. In some cases, input
`from the user can be received in forms other than tactile input
`(e.g., acoustic or speech).
`FIG. 2 illustrates further details of the flow of information
`and signals according to some implementations. The personal
`computing device (e.g., mobile phone) 20 is operable to dis
`play an application or web site that contains information and
`links to content providers 30 on the Internet 21. The user
`operates the mobile phone 20 so as to start the application or
`access the web site (block 100). In some implementations, a
`logo appears on the mobile phone's display. By selecting the
`logo, the user causes a menu to expand and present various
`options. The options can include, for example: (i) add new
`
`30
`
`

`

`US 8,782,528 B2
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`5
`the mobile phone 20 to prompt the user to identify the device
`on which the video is to be displayed.
`A user can identify the device on which the video is to be
`displayed in one of several ways, depending on the imple
`mentation. In some implementations, the user can select the
`display device from a list of devices displayed on the mobile
`phone 20. The list can include a field populated with names or
`identifications of display devices that previously have been
`initialized for connection. Alternatively, the user can select
`the display device by entering a synchronization code
`uniquely associated with the particular display device. As
`illustrated in FIG. 7A, the synchronization code 48 can be
`displayed, for example, on a splash page of the display device
`as text on the screen or as an image Such as a QR code and can
`be entered into the mobile phone 20, for example, manually
`by the user or by scanning the code into the mobile phone. The
`code can be scanned, for example, using optical scanning or
`RFID techniques. Preferably, the synchronization code is
`different from the IP address associated with the device 22.
`The IP address also can be different from the media access
`control (MAC) address associated with the device 22. For
`example, in some implementations, the synchronization code
`is generated randomly and assigned to the display device 22
`each time it connects to the server system 24. Thus, a particu
`lar display device 22 may have an IP address, a MAC address,
`a web or browser cookie, and a synchronization code (“sync
`code') assigned to it at any given time. This i