Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 1 of 20 PageID #: 1221
`
`
`
`Exhibit 2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 2 of 20 PageID #: 1222
`
`
`
`
`
`11111111111111111111111111111)T11)1511111111111111111111111111
`
`(12) United States Patent
`Strober
`
`(10) Patent No.: US 11,048,751 B2
`(45) Date of Patent:
`*Jun. 29, 2021
`
`(54) PLAY CONTROL OF CONTENT ON A
`DISPLAY DEVICE
`
`(71) Applicant: Touchstream Technologies, Inc.,
`Valhalla, NY (US)
`
`(72)
`
`Inventor: David Strober, Rye, NY (US)
`
`(73) Assignee: Touchstream Technologies, Inc.,
`Valhalla, NY (US)
`
`( 5 ) Notice:
`
`Subject to any disclaimer. the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis-
`claimer.
`
`(21) Appl. No.: 15/687,249
`
`(22) Filed:
`
`Aug. 25, 2017
`
`(65)
`
`Prior Publication Data
`
`US 2017/0351757 Al
`
`Dec. 7, 2017
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 13/532,546, filed on
`Jun. 25, 2012, now Pat. No. 9,767,195, which is a
`(Continued)
`
`(51) Int. Cl.
`GO6F 17/30
`GO6F 9/44
`
`(2006.01)
`(2018.01)
`(Continued)
`
`(52) U.S. Cl.
`CPC
`
` GO6F 16/74 (2019.01); GO6F 9/452
`(2018.02); GO6F 16/951 (2019.01)
`(58) Field of Classification Search
` 715/716
`USPC
`Sec application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,613,137 A
`5,878.311 A
`
`3/1997 Bertram et al.
`3/1999 Ichinokawa
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`CN
`CN
`
`9/2009
`101534119 A
`11/2009
`101577650 A
`(Continued)
`
`OTHER PUBLICATIONS
`
`Ask Search Internet Search, session identifier random, printed on
`Nov. 19, 2011.
`
`(Continued)
`
`Primary Examiner — Tadesse Hailu
`Assistant Examiner — Darrin Hope
`(74) Attorney, Agent, or Firm — Shook, Hardy & Bacon
`LLP
`
`ABSTRACT
`(57)
`A system for presenting and controlling content on a display
`device includes a network, a server system coupled to the
`network and comprising one or more servers, a display
`device coupled to the network and having a display, and a
`personal computing device operable to transmit a first mes-
`sage according to a specified format over the network to the
`server system. The server system stores an association
`between the personal computing device and the display
`device. The first message identifies user-selected content and
`a media player to play the content. The server system is
`operable, in response to receiving the first message from the
`personal computing device, to provide to the display device
`a second message identifying the user-selected content and
`the media player to play the content. In response to receiving
`the second message, the display device is operable to obtain
`a first media player needed to play the content, to load the
`media player and to present the content on the display.
`
`20 Claims, 9 Drawing Sheets
`
`Smart Phase
`
`Bacinnd Servers
`
`241
`
`100
`
`Tarps
`
`wafbn
`
`tie
`
`smar.Pi
`
`'•
`
`to
`r0lebase
`
`AccilLrt Ver :Imam
`112
`
`'ved
`
`120-,
`LUniversal API Adapter
`
`122
`Massage Copred to
`Display Doke Database
`
`'ere
`
`File
`
`Start Apo
`
`102
`Request Video Data
`
`42Vdeo Files
`
`so
`
`SpIiFjitJVkie
`108a
`
`Vid
`_1
`
`22
`
`110
`
`Action Selected
`
`oaY is aassooso
`
`DoP
`
`LSwitch Vidlo Mayer I
`
`Load Neci
`
`130
`resentalictiTiVideel
`
`124
`Display Device Checks
`Database far New Messages
`120-,
`Nes Message Received
`by Display Device
`128-,
`Display L8ViG0-ixeCules
`New Meseaae
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 3 of 20 PageID #: 1223
`
`US 11,048,751 B2
`Page 2
`
`Related U.S. Application Data
`
`continuation-in-part of application No. 13/157,821,
`filed on Jun. 10, 2011, now Pat. No. 8,904,289.
`
`(60) Provisional application No. 61/477,998. filed on Apr.
`21, 2011.
`
`(51) hit. Cl.
`GO6F 16/74
`GO6F 16/951
`GO6F 9/451
`
`(2019.01)
`(2019.01)
`(2018.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6,181,713 B1
`6,252,889 B1
`6,756,965 B2
`6,978,424 B2
`7,058,356 B2
`7,114,173 B2
`7,330,875 B1
`7,424,718 B2
`7,433,922 B2
`7,440,972 B2
`7,453,454 B2
`7,509,588 B2
`7,614,070 B2
`7,689,931 B2
`7,769,827 B2
`7,774,708 B2
`7,814,144 B2
`7,835,505 B2
`7,849,485 B2
`7,878,904 B2
`7,949,606 B1
`7,956,846 B2
`8,060,631 B2
`8,086,679 B2
`8,171,507 B2
`8,230,360 B2
`8,238,887 B2
`8,250,608 B2
`8,255,968 B2
`8,316,308 B2
`8,344,870 B2
`8,356,251 B2
`8,373,660 B2
`8,402,494 B1
`8,418,084 B1
`8,572,488 B2
`8,614,625 B2
`8,620,284 B2
`8,659,553 B1
`8,671,440 B2
`8,738,536 B2
`8,738,737 B2
`8,739,074 B2
`8,743,284 B2
`8,744,434 B2
`8,745,228 B2
`8,745,388 B2
`8,751,159 B2
`8,751,520 B1
`8,751,793 B2
`8,752,016 B2
`8,755,919 B2
`8,756,333 B2
`8,756,505 B2
`8,761,351 B1
`8,761,792 B2
`8,762,240 B2
`8,762,548 B1
`8,763,081 B2
`8,782,262 B2
`8,782,528 B2
`
`1/2001 Path et al.
`6/2001 Path et al.
`6/2004 Combs et al.
`12/2005 Safadi
`6/2006 Slotznick
`9/2006 Urdang et al.
`2/2008 Parasnis et al.
`9/2008 Dutton
`10/2008 Engstrom
`10/2008 Oetzel
`11/2008 Allen et al.
`3/2009 Van Os et al.
`11/2009 Urdang et al.
`3/2010 Koga et al.
`8/2010 Girouard et al.
`8/2010 Bell et al.
`10/2010 Koyama et al.
`11/2010 Toyama et al.
`12/2010 Paik et al.
`2/2011 Maehiro
`5/2011 Sweet
`6/2011 Ording et al.
`11/2011 Collart et al.
`12/2011 Nobori et al.
`5/2012 Hironaka ct al.
`7/2012 Ma et al.
`8/2012 F'ilipov
`8/2012 Hayes et al.
`8/2012 Louie el al.
`11/2012 Sherman et al.
`1/2013 Evans et al.
`1/2013 Strober
`2/2013 Pallakoff
`3/2013 Hu et al.
`4/2013 Tischer
`10/2013 Phillips et al.
`12/2013 Alsina et al.
`12/2013 Filipov
`2/2014 Chan et al.
`3/2014 Damola et al.
`5/2014 Strom et al.
`5/2014 Baldini et al.
`5/2014 Kinoshita
`6/2014 Russell et al.
`6/2014 Funderburk et al.
`6/2014 Beckert et al.
`6/2014 Kanungo
`6/2014 Hall
`6/2014 Bhattacharjee et al.
`6/2014 Ginter et al.
`6/2014 Hernandez Porras et al.
`6/2014
`Pyle
`6/2014
`Jannink et al.
`6/2014 Gonze et al.
`6/2014 Daly et al.
`6/2014 Sennett et al.
`6/2014 Sogo et al.
`6/2014 Kessel et al.
`6/2014 Bogdanovic et al.
`7/2014 Collart et al.
`7/2014 Strober
`
`..... . ..... HOlL 63/08
`
`Demchenko et al.
`10/2014
`8,875,180 B2
`Morris
`11/2014
`8,880,491 B2
`Alsina et al.
`6/2015
`9,071,792 B2
`Filipov
`9/2015
`9,148,756 B2
`9,185,171 B2 * 11/2015 Pahlavan
`9,420,025 B2
`8/2016 Park
`9,720,887 B2
`8/2017 Pappu et al.
`2002/0021289 Al
`2/2002 Combs et al.
`2002/0034193 Al
`3/2002
`Patki et al.
`2002/0075332 Al
`6/2002
`Geilfiiss, Jr. et al.
`2002/0083147 Al
`6/2002 Rippergcr
`2002/0120666 Al
`8/2002 Landsman et al.
`2002/0129102 Al
`9/2002 Landsman et al.
`2002/0133518 Al
`9/2002 Landsman et al.
`2002/0146122 Al
`10/2002 Vestergaard et al.
`2002/0198778 Al
`12/2002 Landsman et al.
`2003/0004804 Al
`1/2003 Landsman et al.
`2003/0005000 Al
`1/2003 Landsman et al.
`2003/0018885 Al
`1/2003 Landsman et al.
`2003/0023488 Al
`1/2003 Landsman eL al.
`2003/0028565 Al
`2/2003 Landsman et al.
`2003/0071792 Al
`4/2003 Safadi
`2003/0112268 Al
`6/2003 Wettach
`2003/0131251 Al
`7/2003 Fetkovich
`2003/0142127 Al
`7/2003 Markel
`2003/0182663 Al
`9/2003 Gudorf et al.
`2003/0193520 Al
`10/2003 Oetzel
`2003/0208765 Al
`11/2003 Urdang et al.
`2004/0008972 A 1
`1/2004 Haken
`2004/0049743 Al
`3/2004 Bogward
`2004/0056837 Al
`3/2004 Koga et al.
`2004/0088728 Al
`5/2004 Shimizu
`2004/0268224 Al
`12/2004 Balkus et al.
`2004/0268451 Al
`12/2004 Robbin et al.
`2005/0012723 Al
`1/2005 Pallakoff
`2005/0034151 Al
`2/2005 Abramson
`2005/0055716 Al
`3/2005 Louie et al.
`2005/0144305 Al
`6/2005 Fegan et al.
`2005/0149970 Al
`7/2005 Fairhurst et al.
`2005/0192096 Al
`9/2005 Maehiro
`2006/0062544 Al
`3/2006 Southwood et al.
`2006/0083194 Al
`4/2006 Dhrimaj et al.
`2006/0098624 Al
`5/2006 Morgan et al.
`2006/0101098 Al
`5/2006 Morgan et al.
`2006/0200832 Al
`9/2006 Dutton
`2006/0203758 Al
`9/2006 Tee et al.
`2006/0263038 Al
`11/2006 Gilley
`2006/0265657 Al
`11/2006 Gilley
`2007/0050054 Al
`3/2007 Sambandam Guruparan et al.
`2007/0052868 A 1
`3/2007 Chou et al.
`2007/0055986 Al
`3/2007 Gilley et al.
`2007/0083540 Al
`4/2007 Gundla et al.
`2007/0089147 Al
`4/2007 Urdang et al.
`2007/0094408 Al
`4/2007 Gundla et al.
`2007/0112785 Al
`5/2007 Murphy et al.
`2007/0136778 Al
`6/2007 Birger et al.
`2007/0150963 Al
`6/2007 Lee et al.
`2007/0152978 Al
`7/2007 Kocienda et al.
`2007/0152980 Al
`7/2007 Kocienda et al.
`2007/0155506 Al
`7/2007 Malik
`2007/0156855 Al
`7/2007 Johnson
`2007/0157089 Al
`7/2007 Van Os el al.
`2007/0202923 Al
`8/2007 Jung et al.
`2007/0288715 Al
`12/2007 Boswell et al.
`2008/0008439 Al
`1/2008 Liu et al.
`2008/0028037 Al
`1/2008 Moyer et al.
`2008/0034394 Al
`2/2008 Jacobs et. al.
`2008/0040758 Al
`2/2008 Beetcher et al.
`2008/0077526 Al
`3/2008 Arumugam
`2008/0104267 Al
`5/2008 Dawson
`2008/0126943 Al
`5/2008 Parasnis et al.
`2008/0140849 Al
`6/2008 Collazo
`2008/0155600 A 1
`6/2008 Klappert et al.
`2008/0178198 A 1
`7/2008 Gauba
`2008/0187279 Al
`8/2008 Gilley et al.
`2008/0189617 Al
`8/2008 Covell et al.
`2008/0216001 Al
`9/2008 Ording et al.
`2008/0250190 Al
`10/2008 Johnson
`2008/0267369 Al
`10/2008 Parlamas et al.
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 4 of 20 PageID #: 1224
`
`US 11,048,751 B2
`Page 3
`
`(56)
`
`References Cited
`
`U.S. PAI ENI DOCUMENTS
`
`10/2008
`2008/0270881 Al
`12/2008
`2008/0301737 Al
`2/2009
`2009/0049373 Al
`3/2009
`2009/0077467 Al
`4/2009
`2009/0094331 Al
`4/2009
`200910100477 Al
`4/2009
`2009/0108057 Al
`6/2009
`2009/0150553 Al
`6/2009
`2009/0164641 Al
`7/2009
`2009/0172780 Al
`7/2009
`2009;0177989 Al
`7/2009
`2009/0193466 Al
`9/2009
`2009/0228919 Al
`2009/0248802 Al * 10/2009
`
`Meyer et al.
`Hjelmeland Almas et al.
`Sharma et al.
`Adappa et al.
`Nobori et al.
`Jeffs
`Mu et al.
`Collaft el al.
`Rogers et al.
`Sukeda et al.
`Ma et al.
`Ehreth et al.
`Loft et al.
`Maim:Ian
`
`
`
`GO6F 9/541
`709/204
`
`2009/0254827 Al
`2009/0259944 Al
`2009;0259969 Al
`2009/0282470 Al
`2010/0027974 Al
`2010/0081375 Al
`2010/0094728 Al
`2010;0094900 Al
`2010/0127847 Al
`2010/0137028 Al
`2010/0138746 Al
`2010/0138780 Al
`2010;0174993 Al
`2010/0180307 Al
`2010/0198860 Al
`2010/0205628 Al
`2010/0208136 Al
`2010;0241699 Al
`2010/0265939 Al
`2010/0281042 Al
`2010/0283586 Al
`2010/0313135 Al
`201010325552 Al
`201 1;0007901
` Al
`2011/0014972 Al
`2011/0030020 Al
`2011/0032870 Al
`2011/0035692 Al
`2011;0060998 Al
`2011/0084900 Al
`2011/0090898 Al
`2011/0107227 Al
`2011/0125594 Al
`2011;0131607 Al
`2011/0137729 Al
`2011/0138354 Al
`2011/0156879 Al
`2011/0161396 Al
`2011;0202466 Al
`2011/0214148 Al
`2011/0228768 Al
`2011/0231265 Al
`2011/0231565 Al
`2011/0231566 Al
`201 1/02391 19
` Al
`2011/0267981 Al
`2011/0289419 Al
`2011/0296454 Al
`2011/0296465 Al
`2011;0314386 Al
`2012/0054616 Al
`2012/0059876 Al
`2012/0072846 Al
`2012/0102209 Al
`2012/0110074 Al
`2012/0110464 Al
`20 12/0 1143 13 Al
`2012/0130971 Al*
`
`10/2009 Gonze et al.
`10/2009 Wu
`10/2009 Pallakoff
`11/2009 Yang et al.
`2/2010 Ansari
`4/2010 Rosenblatt et al.
`4/2010 Denning et al.
`4/2010 Hughes, Jr.
`5/2010 Evans et al.
`6/2010 Farris et al.
`6/2010 Zarom
`6/2010 Marano et al.
`7/2010 Pennington et al.
`7/2010 Hayes et al.
`8/2010 Burnett et al.
`8/2010 Davis et al.
`8/2010 Castano
`9/2010 Muthukumarasamy et al.
`10/2010 Parlamas et al.
`11/2010 Windes et al.
`11/2010 Ikeda et al.
`12/2010 Johnson et al.
`12/2010 Sloo et al.
`1/201 1
`Ikeda et al.
`1/2011 Herrmann et al.
`2/2011 lIalttunen
`2/2011 Kumar
`2/2011 Sandone et al.
`3/2011 Schwartz et al.
`4/2011 Jacobsen et al.
`4/2011 Patel et al.
`5/2011 Rcmpc11 ct al.
`5/2011 Brown et al.
`6;2011 Thomas et al.
`6/2011 Weisman et al.
`6/2011 Hertenstein et al.
`6/2011 Matsushita et al.
`6/2011 Filbrich et al.
`8;2011 Carter
`9/2011 Gossweiler. III et al.
`9/2011 Getter et al.
`9/2011 Brown et al.
`9/2011 Getter et al.
`9/2011 Getter et al.
`9/201 1 Phillips et al.
`11/2011 Davies
`11/2011 Yu et al.
`12/2011 Xiong et al.
`12/2011 Krishnan et al.
`12;2011 Jeong et al.
`3/2012 Mittal
`3/2012 Chinta et al.
`3/2012 Curtis
`4/2012 Fok Ah Chuen et al.
`5/2012 Getchius
`5/2012 Chen et al.
`5/2012 Phillips et al.
`5/2012 Morris
`
` HO4N 21/234309
`707/706
`
`2012/0166560
`2012/0182994
`2012/0185887
`2012/0192225
`2012/0239218
`2012/0254931
`2012/0272148
`2013/0014142
`2013/0124759
`2013/0250181
`2014/0033198
`2016/0241912
`
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`Al
`
`6/2012
`7/2012
`7/2012
`7/2012
`9/2012
`10/2012
`10/2012
`1/2013
`5/2013
`9/2013
`1/2014
`8/2016
`
`Nobori et al.
`Dec et al.
`Newell
`Harwell et al.
`Forbes, Jr.
`Oztaskent et al.
`Strober
`Newell
`Strober
`Zhang
`Umapathy ct al.
`McCarthy et al.
`
`FOREIGN PATENT DOCUMENTS
`
`CN
`CN
`EP
`JP
`WO
`WO
`WO
`WO
`
`101778198 A
`101815073 A
`2175607 Al
`2004356695 A
`2004100500 A2
`2007078745 Al
`2008070050 A2
`2008108718 Al
`
`7/2010
`812010
`4/2010
`12/2004
`11/2004
`7/2007
`6/2008
`9/2008
`
`OTHER PUBLICATIONS
`
`Webopedia computer dictionary, session cookie. printed on Nov. 19,
`2011.
`Webopedia computer dictionary, user session, printed on Nov. 19,
`2011.
`www.vbulletinscom, Best way to generate Random, Unique ID's,
`printed on Nov. 19, 2011.
`www.vbulletin.com, Best way to generate Random, Unique ID's,
`Internet Archive Wayback Machine, Jan. 16, 2009.
`Hachman, M. "Snapstick's Media Streaming App/Box: Hands on."
`www.pcmag.corMarticic2/0,2817,2375455,00.asp, 2 pages (Jan. 8,
`2011).
`Dolcourt, J., CES: Snapstick takes on Apple TV, Google TV,
`http://news.cnet.com/8301-17938105-20025100-1.html, 3 pages. (Dec.
`9, 2010).
`Snapstick-Home, "Snapstick," http://www.snapstick.com, 2 pages,
`printed on Mar. 12, 2011.
`Paul, I., Hands On: YouTube Leanback, PCWORLD http://www.
`pcworld.com/articlei200769/hands on_youtube leanback.html, 3 pages.
`(Jul. 9, 2010).
`Using AirPlay, Article HT4437, httQ://support.apple.com/kb/
`HT4437, 3 pages, (Apr. 18, 2011).
`Cheng, J., "Stream AirPlay video to regular TV? Apple might make
`it happen," http://arstechnica.com/appleinewsi201F03/sfteam-airplay-
`video -to -a-regular-tv-apple-might, 5 pages, Mar. 23, 2011.
`"Using the Play to feature to stream media", http://windows.microsoffi
`com/en-LTS/wind ows7/11 sing-the-play-to -feature-to -stream-media, 3
`pages, printed on Jun. 7, 201 1.
`"YouTube—T,eanback", http:www.youtube.com/t/leanback, I page,
`printed on Jun. 7, 2011.
`Hu. C., et al., "Mobile Media Content Sharing in UPnP-Based
`Home Network Environment," Journal of Information Science and
`Engineering 24, 1753-1769. (2008).
`Fallahkhair, S., et al., "Dual Device User interface Design for
`Ubiquitous Language Learning: Mobile Phone and Interactive Tele-
`vision (iTV)," Proceedings of the 2005 IEEE Intl Workshop on
`Wireless and Mobile Technologies in Education, 8 pages, 2005.
`Bing search q=mobile+server +television+control&q Jun. 26, 2014.
`Bing search q=phone+server+television+control&qs Jun. 26, 2014.
`Bing search q—phone+server+television+control+me Jun. 26, 2014.
`Bing search q—mobile+server +television+control+m Jun. 26, 2014.
`Webopedia computer dictionary, web identifier, printed on Nov. 19,
`2011.
`Preinterview First Office Action dated Sep. 3, 2020, in U.S. Appl.
`No. 16/917,095, 24 pages.
`First Action Interview Office Action dated Dec. 10, 2020 in U.S.
`Appl. No. 16/917,095, 25 pages.
`
`* cited by examiner
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 5 of 20 PageID #: 1225
`
`U.S. Patent
`
`Jun. 29, 2021
`
`Sheet 1 of 9
`
`US 11,048,751 B2
`
`10-
`
`24--\
`
`26-\
`
`API Adapter
`
`Server System
`
`28-\
`
`34-•
`
`Switchboard
`
`36 -N
`
`32-‘
`
`Database
`
`..........
`
`Look-up
`Table
`
`...........
`
`Database
`
`23-\
`
`21
`
`22
`
`Internet
`
`q 0
`
`0
`
`0
`
`0
`
`0
`
`o Et
`
`0
`
`0
`
` •••••••=...14 1
`
`30-\
`
`Content Providers
`
`FIG. 1
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 6 of 20 PageID #: 1226
`
`lualud 'S11
`
`'oof
`
`IZOZ
`
`6 Jo Z laagS
`
`Isestobt t
`
`Display Device Database
`
`Message Copied to
`
`x122
`
`niversal API Adapter
`
`120-Th
`
`Li
`
`from Smart Phone
`Message Received
`
`112-Th
`
`Account Verification
`
`114-Th
`[Smart Phone Database
`
`f
`
`Message Added to
`
`116
`
`/Switchboard
`
`Target Verification
`
`Back-end Servers
`
`118
`
`24-Th
`
`FIG. 2
`
`New Message
`
`by Display Device
`
`New Message Received
`
`126-
`Database for New Messages
`
`Display Device Checks
`
`Display Device Executes
`
`128-Th T
`
`Play/Pauseiffwd/etc.,.
`
`Presentation of Video
`130-Th
`L._
`
`I Load New Video
`
`Switch Video Player
`
`Display Device
`
`Selection Sent to Database
`110-,,
`
`r
`
`Action Selected
`
`108-Th
`
`Display List of Videos
`
`Nah
`
`loa--,,
`
`Request Video Data
`
`--
`
`30
`
`1
`
`Start App
`
`Smart Phone
`
`102-Th
`
`100-\
`
`20
`
`22-Th
`
`Video Player
`40-Thic
`
`! Video Files
`42-Th
`XML Data File
`104-\
`Content Providers
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 7 of 20 PageID #: 1227
`
`U.S. Patent
`
`Jun. 29, 2021
`
`Sheet 3 of 9
`
`US 11,048,751 B2
`
`,
`
`Transmission Code
`UserlD TargetiD MediaPlayerlD Command Data
`
`Single Connection Look-up Table
`
`Display Device
`
`User -
`Smartphone
`
`2
`1
`3
`4
`
`A
`C
`0
`
`26
`
`FIG. 3
`
`FIG. 4
`
`Universal API Adapter
`Universal Command MediaPlayerlD Specific Player Command
`yt loadVideo
`New Video
`YouTube
`getVideo
`Ted.com
`Vimeo
`YouTube
`Ted.com
`Vimeo
`FIG. 5
`
`loadNewVideo
`yt.pauseVideo
`pauseVideo
`pause
`
`Pause
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 8 of 20 PageID #: 1228
`
`U.S. Patent
`
`Jun. 29, 2021
`
`Sheet 4 of 9
`
`US 11,048,751 B2
`
`Yes
`
`Is
`the Media Player that
`is Needed to Play the Video
`Already Loaded in the
`Display Device
`
`No
`
`Request and Obtain a
`Copy of the Appropriate
`Media Player and a Copy
`of the Video File from
`Content Provider
`
`Load the Media Player
`
`Yes
`
`Is
`a Copy of the
`Video Already Loaded in
`the Display Device
`
`No
`
`Obtain a Copy
`of the Video
`
`Play the Video
`
`FIG. 6
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 9 of 20 PageID #: 1229
`
`U.S. Patent
`
`Jun. 29, 2021
`
`Sheet 5 of 9
`
`US 11,048,751 B2
`
`22 -\
`
`C!
`er.
`
`Display Device
`
`Splash Page
`
`48--\
`435-05-342
`FIG. 7A
`
`Sync-code
`
`IP Address
`169.343.231.234
`
`Sync-code Look-up Table
`Cookie
`erig988dhuj
`FIG. 7B
`
`Sync-code
`435-05-342
`
`Display Device
`
`22
`
`Display Device
`
`Smart Phone
`
`Smart Phone
`
`Smart Phone
`
`20
`
`FIG. 8
`
`20
`
`(A)
`
`20
`(B)
`
`FIG. 9
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 10 of 20 PageID #: 1230
`
`U.S. Patent
`
`Jun. 29, 2021
`
`Sheet 6 of 9
`
`US 11,048,751 B2
`
`Display Device
`
`22
`
`zif
`
`Smart Phone
`
`Smart Phone
`
`Smart Phone
`
`20
`
`20
`
`20
`
`Moderator
`
`Control of
`Display
`
`FIG. 10
`
`Display Device
`
`Display Device
`
`22
`
`22
`
`Smart Phone
`
`20
`
`FIG. 11
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 11 of 20 PageID #: 1231
`
`U.S. Patent
`
`Jun.29,2021
`
`Sheet 7 of 9
`
`US 11,048,751 B2
`
`Display Device
`
`Display Device
`
`Display Device
`
`22
`
`22
`
`22
`
`Smart Phone
`
`Smart Phone
`
`Smart Phone
`
`20
`
`20
`
`Moderator
`
`Control of
`Display
`
`FIG. 12
`
`Display Device
`
`22
`
`Display Device
`
`22
`
`Smart Phone
`
`Third Party
`
`Smart Phone
`
`20
`
`20
`
`Playlist and Video Links
`
`FIG. 13
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 12 of 20 PageID #: 1232
`
`lualud 'S'il
`
`'oof
`
`IZOZ
`
`6 Jo 8 loollS
`
`Isestobit sn
`
`AD SPACE
`
`MENUICHANNELSIPLAYLISTISEARCH
`
`HEADER
`
`friend's laptop 0
`living-roorn PC 0
`0]
`DISPLAY.] I DISPLAY 2IDISPLU3IDISPlAY 41
`GUEST
`
`r
`
`my laptop
`
`CONTROL PANEL
`
`FIG. 14C
`
`FIG. 14B
`
`FIG. 14A
`
`tif)
`
`FOOTER
`
`,[SEND]
`
`1
`
`Enter the Sync-code below or
`
`devices listed above
`choose from one of your
`
`FOOTER
`
`SPLASH PAGE
`
`MENUICHANNELSIPLAYLISTISEARCH
`
`HEADER
`
`0
`
`4
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 13 of 20 PageID #: 1233
`
`lualud 'S11
`
`'orlf
`
`IZOZ
`
`6 Jo 6 1"11S
`
`Isestobt t
`
`FIG. 14E
`
`FIG. 14D
`
`FIG. 15
`
`SEND
`
`* = Control of Display Device
`" = Group Moderator,
`
`B"*,E
`A",C*,D
`
`Y
`X
`
`1,3
`2
`
`Srnartphone
`
`User -
`
`Group
`
`Device
`Displa
`
`Group Connection
`
`Look-up Table
`
`0
`
`r-
`
`MEI GUEST7111GUEST 2IGUEST 3IGUEST .. . .
`(DISMAYS'
`
`CONTROL PANEL
`
`ir
`
`FOOTER
`
`MORE INFO...
`OPTION I
`
`AD SPACE
`
`NOW RAYING
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 14 of 20 PageID #: 1234
`
`US 11,048,751 B2
`
`1
`PLAY CONTROL OF CONTENT ON A
`DISPLAY DEVICE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`The present application is a continuation of U.S. appli-
`cation Ser. No. 13/532,546, filed on Jun. 25, 2012, which is
`a continuation-in-part of U.S. application Ser. No. 13/157,
`821, filed on Jun. 10, 2011, which issued on Dec. 2, 2014 as
`U.S. Pat. No. 8,904,289, which claims the benefit of priority
`to U.S. Provisional Patent Application No. 61/477,998, filed
`on Apr. 21, 2011. The contents of the earlier applications are
`incorporated herein by reference.
`
`BACKGROUND
`
`This disclosure relates to play control of content on a
`display device. Such display devices include, for example,
`television displays used by consumers in their home for
`viewing videos and other media that are either provided
`from the Web or previously stored. In particular, the disclo-
`sure relates to the creation, storage, manipulation and access
`of media playlists used in conjunction with display devices
`and control of the display devices.
`Web media often is played on computers rather than
`television displays. Although it is known to connect a
`computer to a television set in order to watch Web media, it
`is difficult to control such a system within the typical
`scenario for television watching where the viewer is posi-
`tioned some distance from the television. Furthermore,
`although a wireless device can enable the user to control the
`television from a distance, it can be difficult to view a web
`browser display on the television set and may interfere with
`normal television program viewing by other persons.
`Given the desire to watch various World Wide Web media
`on a family's primary television set, and to control this
`operation from the comfort of one's couch. there is a need
`to operate a television set or other display remotely from a
`personal computing device, such as a mobile phone. It also
`is desirable to allow a user to perform a general Web search
`to locate and capture Web media, and to control a television
`or other display remotely using the personal computing
`device.
`
`SUMMARY
`
`Various aspects of the invention are set forth in the claims.
`For example, according to one aspect, a system for
`presenting and controlling content on a display device
`includes a network, a server system coupled to the network
`and comprising one or more servers, a display device
`coupled to the network and having a display, and a personal
`computing device operable to transmit a first message
`according to a specified format over the network to the
`server system. The server system stores an association
`between the personal computing device and the display
`device. The first message identifies user-selected content and
`a media player to play the content. The server system is
`operable, in response to receiving the first message from the
`personal computing device, to provide to the display device
`a second message identifying the user-selected content and
`the media player to play the content. In response to receiving
`the second message, the display device is operable to obtain
`a first media player needed to play the content, to load the
`media player and to present the content on the display.
`
`5
`
`2
`In some implementations, the display device is operable,
`in response to receiving the second message, to obtain the
`first media player from the content provider only if the first
`media player is not already loaded in the display device.
`In some implementations, the personal computing device
`is operable to transmit a message according to a specified
`format over the network to the server system. The message
`can include a command for controlling playing of the
`content on the display device. The server system is operable,
`10 in response to receiving the message, to convert the com-
`mand into a corresponding command recognizable by the
`media player if the command received from the personal
`computing device is not recognizable by the media player.
`The server system is operable to provide to the display
`15 device a message that includes the corresponding command,
`and the display device is operable, in response to receiving
`the message from the server system, to execute the com-
`mand.
`In some implementations, the personal computing device
`20 is, for example, a mobile phone, and the display device is a
`television set. Other personal computing devices or display
`devices can be used in other implementations. The network
`can include, for example, the Internet.
`In some implementations, the server system stores a
`25 look-up table that includes a synchronization code uniquely
`associated with the display device. A message from the
`personal computing device can include the synchronization
`code, and in response to receiving the message from per-
`sonal computing device, the server system can use the
`30 synchronization code and the look-up table to identify the
`display device on which the content is to be played. The
`synchronization code can be different from an IP address
`associated with the display device and/or a media access
`control address associated with the display device.
`In various implementations, the system can facilitate
`allowing a personal computing device to be used to select
`different content to be played on a remote display even if
`different media players are required to present the different
`content. The system also can allow the user to control how
`40 the content is displayed on the display device using the
`personal computing device. For example, user-initiated play
`commands can be passed from the user's personal comput-
`ing device, through the server system. to the display devices.
`Other aspects, features and advantages will be apparent
`45 from the following detailed description, the accompanying
`drawings, and the claims.
`
`35
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`50
`
`55
`
`60
`
`The present technology is illustrated by way of example,
`and not by way of limitation, in the figures of the accom-
`panying drawings.
`FIG. 1 is a block diagram illustrating an example of a
`system according to the invention.
`FIG. 2 illustrates various details of the flow of information
`and signals according to some implementations.
`FIG. 3 illustrates an example of a transmission code
`incorporated into a message from a personal computing
`device.
`FIG. 4 illustrates an example of a look-up table that forms
`part of a server system.
`FIG. 5 illustrates an example of entries in a universal API
`adapter.
`FIG. 6 is a flow chart showing steps for display device to
`65 load a video player and video.
`FIG. 7A illustrates an example of a display device includ-
`ing a synchronization code.
`
`

`

`Case 2:23-cv-00059-JRG Document 42-2 Filed 04/25/24 Page 15 of 20 PageID #: 1235
`
`US 11,048,751 B2
`
`3
`FIG. 7B illustrates an example of a synchronization code
`look-up table.
`FIGS. 8-13 illustrate examples of various scenarios in
`which the invention can be used.
`FIGS. 14A through 14E illustrate examples of display
`screens that may appear on a user's personal computing
`device in accordance with the invention.
`FIG. 15 illustrates further information that can be stored
`in the look-up table in the server system.
`
`DETAILED DESCRIPTION
`
`As shown in FIG. 1, a system 10 facilitates synchronizing
`a connection between two or more devices 20, 22 connected
`to the Internet 21 or other computer network. The connec-
`tion is designed to be made by a first device (e.g.. a personal
`computing device) 20 that acts as a controller and a second
`device (e.g., a television set 22 with a display 23) that acts
`as a receiver to play content selected by a user of the first
`device and to respond to commands that originate at the
`personal computing device. The personal computing device
`20 is operable to display an application or web site that
`contains information and links to content providers 30 on the
`Internet 21. The television set 22 is operable to link back to
`a server system 24 from which the television set receives
`commands. When a user makes a selection using the per-
`sonal computing device 20 for particular content to be
`displayed on the television display 23, a signal is sent
`through the Internet (or other network) 21 to the server
`system 24. A corresponding command signal then is passed
`along to the connected television set 22, which acts on a
`transmission code contained within the signal and performs
`specified commands. For example, in some scenarios, the
`command instructs the television set 22 to access a content
`provider 30 through the Internet 21, load a specific media
`player, load the media player-specific content (e.g., a video)
`and play the content on the television display 23. The user
`can use the personal computing device 20 to control how the
`content is played on the television display 23. The user may
`subsequently visit the same or another Web site using the
`personal computing device 20 to select different content
`(e.g., a second video) to be played on the television display
`23. In that case, another signal would be sent through the
`server system 24 to the television set 22. A transmission
`code associated with this command signal instructs the
`television set 22 to load a new media player (if needed) over
`the Internet and to load the specified video file to be played
`on the display 23. Thus, the system 10 allows a personal
`computing device 20 to be used to select different content to
`be played on a remote display 23 even if different media
`players are required for the different content. The user also
`can control how the content is displayed (e.g., play, pause,
`stop, rewind, fast forward, etc.) on the display 23 using the
`personal computing device 20. The user-initiated play com-
`mands are passed from the user's personal computing device
`20, through the server system 24, to the television set 22.
`Although the following detailed discussion describes vid-
`eos as an example of the type of content to be played on the
`display 23, the system 10 can be used for other types of
`content as well. Thus, depending on the implementation, the
`content may include one or more of the following: video,
`audio, interactive video game, streaming media, multimedia,
`images, slides (e.g., a PowerPoint presentation) or other
`types of dynamic content. Furthermore, in the following
`discussion, it is assumed that the personal computing device
`20 is a mobile phone that includes a display, an internal
`microprocessor or other processing circuitry, a keypad,
`
`4
`keyboard, touchscreen, mouse. trackball. or other device to
`receive user selections and other input, and a transceiver to
`establish communications to the Internet 21 or other com-
`munications networks. More generally, however, the per-
`5 sonal computing device 20 can be any type of handheld or
`other Internet-enabled personal computing device, including
`personal computers, e-books, kiosks, tablets, smart phones,
`media players. and motion and touch sensory interfaces. In
`some cases, input from the user can be received in forms
`o other than tactile input (e.g., acoustic or speech).
`FIG. 2 illustrates further details of the flow of information
`and signals according to some implementations. The per-
`sonal computing device (e.g., mobile phone) 20 is operable
`to display an application or web site that contains informa-
`15 tion and links to content providers 30 on the Internet 21. The
`user operates the mobile phone 20 so as to start the appli-
`cation or access the web site (block 100). In some imple-
`mentations, a logo appears on the mobile phone's display.
`By selecting the logo, the user causes a menu to expand and
`20 present various options. The options can include, for
`example: (i) add new content to a playlist, (ii) play a listed
`item on a secondary device, (iii) play a listed item on the
`mobile phone 20. If the user selects to add new content to the
`playlist, the user is presented with a screen that allows him
`25 to enter user-defined search parameters or o select pre-
`defined search parameters to request video data. The search
`parameters are sent from the mobile phone 20 as part of a
`request for video data that satisfy the search parameters
`(block 102). The request is transmitted via the Internet 21
`30 and through the server system 24 to the appropriate content
`provider web site. In response, the content provider 30
`provides metadata (e.g., titles, links to the videos) for one or
`more video files that satisfy the search parameters (block
`104). The metadata can be provided to