`

`US 11,039,218 Bl
`Page 2
`
`Related U.S. Application Data
`
`No. 16/580,552, filed on Sep. 24, 2019, now Pat. No.
`10,805,687, which is a continuation of application
`No. 16/267,887, filed on Feb. 5, 2019, now Pat. No.
`10,425,697, which is a continuation of application
`No. PCT/US2017/045801, filed on Aug. 7, 2017.
`
`2016/0037215 Al
`2016/0249108 Al
`2016/0360261 Al
`2017/0034237 Al
`2017/0099516 Al
`2017/0188054 Al
`2020/0162796 Al
`
`2/2016 Cardona
`8/2016 Sexton
`12/2016 Makhlouf
`2/2017 Silver
`4/2017 Barbulescu et al.
`6/2017 Ma et al.
`5/2020 Azuolas et al.
`
`(60) Provisional application No. 62/518,506, filed on Jun.
`12, 2017, provisional application No. 62/485,878,
`filed on Apr. 14, 2017, provisional application No.
`62/435,361, filed on Dec. 16, 2016, provisional
`application No. 62/371,558, filed on Aug. 5, 2016.
`
`(51)
`
`(52)
`
`(2011.01)
`(2006.01)
`(2011.01)
`(2011.01)
`(2011.01)
`(2011.01)
`(2011.01)
`
`Int. Cl.
`H04N 2112187
`H04L 29106
`H04N 2114788
`H04N 2112343
`H04N 211472
`H04N 211431
`H04N 211845
`U.S. Cl.
`CPC ............ H04L 65/608 (2013.01); H04L 65/80
`(2013.01); H04N 2112187 (2013.01); H04N
`2112343 (2013.01); H04N 2114316 (2013.01);
`H04N 2114788 (2013.01); H04N 21/47217
`(2013.01); H04N 2118456 (2013.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`8,341,662 Bl
`8,549,574 B2
`8,595,186 Bl
`8,874,778 B2
`9,055,271 B2
`9,106,934 B2
`9,149,682 B2
`9,251,852 B2
`9,288,278 B2
`9,509,793 B2
`9,516,390 B2
`9,584,858 B2
`9,591,054 B2
`9,641,566 Bl
`9,654,844 B2
`9,692,800 B2
`9,706,443 B2
`10,425,697 B2
`10,484,743 B2
`10,740,305 B2
`10,805,687 B2
`10,848,792 B2
`2001/0001160 Al*
`
`12/2012 Bassett et al.
`10/2013 Perlman et al.
`11/2013 Mandyam et al.
`10/2014 Xu et al.
`6/2015 Verna et al.
`8/2015 Horen et al.
`10/2015 Dornbush et al.
`2/2016 Burns
`3/2016 Panje et al.
`11/2016 Brown et al.
`12/2016 Lau et al.
`2/2017 Vinson et al.
`3/2017 Thornburgh et al.
`5/2017 Hiremath
`5/2017 Kim et al.
`6/2017 Gaunt et al.
`7/2017 Oyman et al.
`9/2019 Schupak et al.
`11/2019 Cox
`8/2020 Barthel et al.
`10/2020 April et al.
`11/2020 Evans
`5/2001 Shoff
`
`H04N 5/44543
`725/51
`2001/0039209 Al* 11/2001 DeWeese ............ G07F 17 /3288
`463/40
`
`2004/0003101 Al
`2008/0022347 Al
`2008/0062318 Al
`2008/0168493 Al
`2009/0222754 Al
`2010/0299703 Al
`2011/0083144 Al
`2011/0086144 Al*
`
`1/2004 Roth et al.
`1/2008 Cohen
`3/2008 Ellis et al.
`7/2008 Allen et al.
`9/2009 Phillips et al.
`11/2010 Altman
`4/2011 Bocharov et al.
`4/2011 Arampongpun .
`
`2011/0090960 Al
`2011/0280540 Al
`2013/0222597 Al
`2013/0227596 Al
`2014/0129680 Al
`2014/0359075 Al
`
`4/2011 Leontaris et al.
`11/2011 Woodman
`8/2013 Brink et al.
`8/2013 Pettis et al.
`5/2014 Mukherjee
`12/2014 Amidei et al.
`
`A23G9/26
`426/250
`
`OTHER PUBLICATIONS
`
`International Search Report and Written Opinion in International
`Patent Application No. PCT/US2017/045801 dated Oct. 30, 2019,
`18 pages.
`International Search Report and Written Opinion in International
`Patent Application No. PCT/US2018/033016 datedAug. 6, 2018, 29
`pages.
`Analyze & Optimize. Maestro 2020. Accessed at https:/ /info.maestro.
`io/analyze-and-optimize on Apr. 21, 2021. 3 pages.
`Around-the-clock action for you sportsbook. BetGenius Brochure.
`Accessed at https:/ / geniussports.com/ sportsbook/content/
`streaming/ on Mar. 19, 2021. 1 page.
`Betradar Live Streaming Brochure 2021. Accessed at https://www.
`betradar.com/wp-content/uploads/sites/4/2014/ 11/Betradar-Live(cid:173)
`Streaming-Brochure.pdf on Apr. 29, 2021. 7 pages.
`Betradar Product Brochure 2021. Accessed at https://www.betradar.
`com/wp-content/uploads/ sites/4/2021/02/Betradar-Product -Brochure-
`2021. pdf on Apr. 29, 2021. 24 pages.
`Betradar. Sportradar. Accessed at https://www.betradar.com/ on Apr.
`21, 2021. 2 pages.
`Customize & Control. Maestro 2020. Accessed at https://info.
`maestro.io/customize-and-control on Apr. 21, 2021. 2 pages.
`Empson, Baseball's Digital Trifecta and How America's Pastime is
`Setting the Pace for Sports Online. TechCrunch Nov. 4, 2013.
`Accessed at https:/ /techcrunch.com/2013/ 11/04/baseballs-digital(cid:173)
`trifecta-and-how-americas-pastime-is-setting-the-pace-for-sports(cid:173)
`online/. 10 pages.
`Engage & Monetize. Maestro 2020. Accessed at https:/ /info.maestro.
`io/engage-and-monetize on Apr. 21, 2021. 3 pages.
`Esports-Take your fan experience to the next level and build
`community. Maestro 2020. Accessed at https://info.maestro.io/
`esports on Apr. 21, 2021. 4 pages.
`Genius Sports Group Analyst Day Presentation Jan. 2021. Accessed
`at https://s27 .q4cdn.com/5 52951210/files/doc_downloads/GSG_
`Analyst_Day_Presentation_January_202l_vFF _2.pdf on Apr. 29,
`2021. 52 pages.
`Genius Sports Group Investor Presentation Oct. 2020. Accessed at
`https://news .geniussports .com/wp-content/uploads/2020/ 10/GSG(cid:173)
`Investor-Presentation.pdf on Apr. 29, 2021. 29 pages.
`Ha, Major League Baseball's "At Bat" App Gets updated to
`Supported Expanded Instant Replay. TechCrunch Mar. 31, 2014.
`Accessed https:/ /techcrunch.com/2014/03/31/mlb-at-bat-expanded(cid:173)
`instant-replay/. 4 pages.
`LA Techwatch, LA Startup Maestro Just Raised $3M to Broadcast
`eSports at Scale. LA Tech Watch Sep. 19, 2017. Accessed at https://
`www.latechwatch.com/2017 /09/la-startup-maestro-just-raised-3m(cid:173)
`broadcast-esports-scale/, 13 pages.
`MLB App. MLB 2021. Accessed at https://www.mlb.com/apps/mlb(cid:173)
`app on Apr. 21, 2021. 1 page.
`Perez, Live Video Viewing up 86% over last year in MLB's at Bat
`app, thanks for Addition of multitasking. TechCrunchApr. 15, 2016.
`Accessed at https:/ /techcrunch.com/2016/04/ l 5/live-video-viewing(cid:173)
`up-86-over-last-year-in-mlbs-at-bat-app-thanks-to-addition-of(cid:173)
`multitasking/. 4 pages.
`Perez, MLB.com At Bat and NHL are first to Launch Personalized
`App Icons on iOS 10.3. TechCrunch Mar. 28, 2017. Accessed at
`https:/ /techcrunch.com/2017 /03/28/mlb-com-at-bat-and-nhl-are-first(cid:173)
`to-launch-personalized-app-icons-on-ios- l 0-3/. 5 pages.
`Sports-Personalized fan-first experiences that go beyond the field.
`Maestro 2020. Accessed at https://Info.maestro.Io/sports on Apr. 21,
`2021. 3 pages.
`Sports Betting. Sportradar. Accessed at https://sportradar. us/betting(cid:173)
`services/ on Apr. 21, 2021. 4 pages.
`
`Sportradar 1009
`Page 2
`
`

`

`US 11,039,218 Bl
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Washington, Betgenius launches live streaming service for sportsbooks.
`Genius Sports Oct. 8, 2019. Accessed at https://news.geniussports.
`corn/betgenius-launches-live-strearning-service-for-sportsbooks/ on
`Apr. 30, 2021. 5 pages.
`
`* cited by examiner
`
`Sportradar 1009
`Page 3
`
`

`

`

`

`

`

`~ = ~
`
`~
`~
`~
`
`00 .
`e .
`
`"'""' 00 = "'""'
`"'""' = w
`"'""'
`d r.,;_
`
`'N
`\0
`
`~
`
`('D
`('D
`
`0 ....
`.....
`rJJ =(cid:173)
`
`O'I
`Ul
`
`? ....
`2'
`
`N ....
`
`0
`N
`~Ul
`
`504A, 5048 "ft>~
`
`News Feeds
`
`(RSS)
`
`Provider
`Event Info
`
`&i 502A,502B _!_]
`
`!500
`
`1000•
`
`'..5~
`
`FIG. 2
`
`~2oi3c
`
`Viewer Client Device #3
`
`Viewer Client Device #1
`
`' \2;: ~cl
`
`~,
`
`:21oa•
`
`,5o4B,
`
`~ \
`
`i_~QQQ~
`
`triOiJI
`
`~/
`
`!_200C\
`
`[!{cf66! ~ Client App I I
`
`{
`
`{
`
`l.2QQAI~ {
`
`.......... --.-......... -
`
`[266A J9Q4-AJ ::~ ..... --r-"--1
`
`•·e,q,?,-1•: .... i
`i 604A •
`
`0=---:4c__=:__C;
`
`L_ ________ ....:.i
`
`72
`
`202A~ t_
`·
`
`'---J
`
`looJ
`
`~---___._-----,
`
`i_J_llQ!
`
`(j~Q;
`
`i 34i~4-A •
`
`_,.. .---__._----. i1:1:~rg(j·
`
`.....____
`
`Server
`Control
`
`Architecture
`HLS Server
`
`Server(s)
`
`e ia
`
`WebRTC
`
`M d.
`
`CON
`RTMP
`
`Server(s)
`
`Media
`.
`RTMP
`
`~__.__,
`
`,-------~
`
`i 300
`
`Broadcast/Viewir:ig__~_~rvers and Memory Storage Device
`
`Media Sources
`
`Data
`
`Database
`
`I
`
`Memory System
`
`_4QQJ
`
`rloiQ1
`
`f166ii
`
`Device #2
`
`Broadcaster Client
`
`rJQ.?AJ
`Device~--
`
`Broadcaster .. ~lient WidOA]
`
`Sportradar 1009
`Page 6
`
`

`

`U.S. Patent
`
`Jun.15,2021
`
`Sheet 4 of 56
`
`US 11,039,218 Bl
`
`!420!
`/'············'
`
`I
`
`1000A
`
`/
`
`AWS Hosted Systems
`
`'10008
`
`. /
`
`/-[too:
`
`r
`.......
`Database
`
`~
`
`~
`
`!600:
`
`II
`
`Socket
`Servers
`
`-
`
`\~
`
`Web Server
`~
`Pool (API
`I
`and
`' \ Website)
`t ~
`'
`t
`I
`-
`....
`.,,
`Database
`Shard
`-
`i
`
`----
`
`t
`
`.,,
`
`......
`-
`.,,
`Database
`Shard
`
`.,,,-
`
`.....
`
`;460
`
`............ ,...________
`
`I '
`
`.-#
`
`~
`
`.,,
`
`Control
`--':,;, Server
`csoo:J t
`
`Redis'
`Based
`Memory
`\) Cache)
`
`!320
`·············• ~
`RTMP/App
`Media
`
`\
`
`-
`
`~ ~
`
`!
`''ye~
`
`1
`
`I
`
`\
`
`RTMP CON
`Server 1-E--
`Pool
`
`\ <: ........
`
`:340
`
`~
`
`ssoV
`~440
`~ 360
`-
`1~
`S3 Data
`_Storage~
`Media -
`'
`
`Async
`Queue
`
`J
`
`- WebRTC
`
`\
`
`Servers
`/,,,,,-
`
`HLS Server
`Architecture
`
`-
`
`Gateway
`NAS
`Server
`
`1.3;>
`
`\380!
`
`J
`
`'
`
`I
`!S10:/
`
`I
`
`Transcoding
`Server
`Pool
`
`\
`
`\
`
`I\
`
`&2Q
`
`FIG. 3
`
`Sportradar 1009
`Page 7
`
`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`"'""' 00 = "'""'
`"'""' = w
`"'""'
`d r.,;_
`
`'N
`\0
`
`10
`
`FIG.
`
`(RSS Feeds)
`News Importer
`
`II
`
`User Logging
`Anonymous
`
`Monitoring
`Async Queue
`
`0 ....
`-....J
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`O"I
`Ul
`
`(FIG. 18)
`Thumbnail
`Take Stream
`SERVICE-
`
`v
`
`(Fig. 16)
`
`Data Monitor
`Live Event
`SERVICE -
`
`(Fig. 15)
`
`Data Ingress
`
`Event
`
`SERVICE-
`
`J
`
`Ii
`
`(Fig. 13-14)
`Watchdog
`
`Server
`
`Stream and
`SERVICE-
`
`'I
`
`? ....
`2'
`
`N ....
`
`0
`N
`~Ul
`
`~ = ~
`
`~
`~
`~
`
`00 .
`e .
`
`✓_g_QQ_j
`
`(Fig. 17)
`
`Task Processor
`Asnychronous
`
`SERVICE-
`
`( Control Server )
`
`(Fig. 12)
`System
`
`Server Scaling
`
`RTMP CON
`SERVICE-
`
`I
`
`(Fig. 11)
`System
`
`Server Scaling
`RTMP Media
`SERVICE-
`
`Sportradar 1009
`Page 20
`
`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`

`US 11,039,218 Bl
`
`1
`SYSTEMS, APPARATUS AND METHODS
`FOR RENDERING DIGITAL CONTENT
`RELATING TO A SPORTING EVENT WITH
`ONLINE GAMING INFORMATION
`
`CROSS-REFERENCES TO RELATED
`APPLICATIONS
`
`5
`
`2
`recorded video content to a user's home. Also, additional
`increase of Internet access was sparked by the rise of the
`smartphone and the ability for smartphones to access the
`Internet on-the-go. In 2005, YouTube™ began to offer
`Internet users the ability to upload, edit, view, rate, share and
`comment on a wide variety of user-generated and corporate
`media video content. Examples of such content include
`video clips, television show clips, music videos, audio
`recordings, movie trailers, video biogs, short original videos,
`10 and educational videos. Within one year of its inception,
`YouTube™ reached 65 thousand daily video uploads, with
`100 million daily views.
`The first Internet live video streaming platform in the
`United States, aptly called Livestream, was launched in
`15 2007. Livestream, as well as other nascent live video stream(cid:173)
`ing platforms, were content agnostic, and over time Internet
`viewers desired more specialized, content-specific, and
`niche live video streaming platforms. To accommodate
`evolving viewer demand, various live video streaming plat-
`20 forms have launched since Livestream; examples of such
`more specialized platforms include Twitch.tv (a live video
`streaming platform for creative arts and gaming content,
`which launched in 2011), and musical.ly (a live video
`streaming platform designed for music video content, which
`25 launched in 2014).
`Live video streaming platforms have also aimed to attract
`social network users, and social networking platforms have
`evolved to include live video streaming capabilities. For
`example, Twitch.tv includes social networking components,
`30 and in 2015 the social media platform Twitter acquired the
`live video streaming platform Periscope. Other social media
`platforms have followed suit, with both Facebook and
`Instagram implementing live video streaming capabilities
`into their mobile applications in 2016. The widespread use
`35 of smartphones enables users of these social media plat(cid:173)
`forms to share live videos with their social network.
`
`The present application is a Continuation Application of
`U.S. application Ser. No. 17/027,219, filed Sep. 21, 2020,
`and entitled SYSTEMS, APPARATUS, AND METHODS
`FOR SCALABLE LOW-LATENCY VIEWING OF
`BROADCAST DIGITAL CONTENT STREAMS OF LIVE
`EVENTS, AND SYNCHRONIZATION OF EVENT
`INFORMATION WITH VIEWED STREAMS, VIA MUL(cid:173)
`TIPLE INTERNET CHANNELS, which is a Continuation
`Application of U.S. application Ser. No. 16/580,552, filed
`Sep. 24, 2019, and entitled "SYSTEMS, APPARATUS,
`AND METHODS FOR SCALABLE LOW-LATENCY
`VIEWING OF BROADCAST DIGITAL CONTENT
`STREAMS OF LIVE EVENTS, AND SYNCHRONIZA(cid:173)
`TION OF EVENT INFORMATION WITH VIEWED
`STREAMS, VIA MULTIPLE INTERNET CHANNELS,"
`which is a continuation of U.S. application Ser. No. 16/267,
`887, filed Feb. 5, 2019, entitled "SYSTEMS,APPARATUS,
`AND METHODS FOR SCALABLE LOW-LATENCY
`VIEWING OF BROADCAST DIGITAL CONTENT
`STREAMS OF LIVE EVENTS, AND SYNCHRONIZA(cid:173)
`TION OF EVENT INFORMATION WITH VIEWED
`STREAMS, VIA MULTIPLE INTERNET CHANNELS,"
`which is a Bypass Continuation Application of International
`Patent Application No. PCT/US2017/045801, filed Aug. 7,
`2017, entitled "SYSTEMS, APPARATUS, AND METH(cid:173)
`ODS FOR SCALABLE LOW-LATENCY VIEWING OF
`BROADCAST DIGITAL CONTENT STREAMS OF LIVE
`EVENTS, AND SYNCHRONIZATION OF EVENT
`INFORMATION WITH VIEWED STREAMS, VIA MUL(cid:173)
`TIPLE INTERNET CHANNELS," which claims a priority
`benefit to the following U.S. provisional patent applications:
`U.S. Provisional Patent Application Ser. No. 62/371,558, 40
`filed Aug. 5, 2016, entitled "SYSTEMS, APPARATUS,
`AND METHODS FOR LIVE COMMENTARY;" U.S. Pro(cid:173)
`visional Patent Application Ser. No. 62/435,361, filed Dec.
`16, 2016, entitled "SYSTEMS,APPARATUS,AND METH(cid:173)
`ODS FOR LIVE COMMENTARY;" U.S. Provisional Patent 45
`Application Ser. No. 62/485,878, filed Apr. 14, 2017,
`entitled "SYSTEMS, APPARATUS, AND METHODS FOR
`LIVE STREAMING, RECORDING, AND REPLAY OF
`BROADCAST VIDEO AND/OR AUDIO VIA THE
`INTERNET;" and U.S. Provisional Patent Application Ser.
`No. 62/518,506, filed Jun. 12, 2017, entitled "SYSTEMS,
`APPARATUS, AND METHODS FOR SCALABLE LOW(cid:173)
`LATENCY VIEWING OF BROADCAST DIGITAL CON(cid:173)
`TENT STREAMS OF LIVE EVENTS, AND SYNCHRO(cid:173)
`NIZATION OF EVENT INFORMATION WITH VIEWED
`STREAMS VIA MULTIPLE INTERNET CHANNELS
`AND AN HTTP LIVE STREAMING (HLS) CACHING
`SERVER ARCHITECTURE." Each of the above-identified
`applications is hereby incorporated herein by reference in its
`entirety.
`
`SUMMARY
`
`In spite of the proliferation of live video streaming over
`the Internet, the Inventors have recognized and appreciated
`various technological problems in conventional techniques
`for transmission of digital content via the Internet that
`adversely impact the live video streaming viewer experi(cid:173)
`ence. One such technological problem relates to viewer
`"latency," i.e., the delay between a first user generating a live
`video stream for transmission via the Internet and a second
`user receiving a copy of the live video stream via the Internet
`for viewing. For example, the live video streaming platform
`50 Twitch.tv has a latency of approximately 15 seconds, and
`Facebook's live streaming platform has an average latency
`of approximately 10 seconds. Other technological chal(cid:173)
`lenges for live video streaming arising from shortcomings in
`conventional techniques, as recognized and appreciated by
`55 the Inventors, include, for example, the difficulties in pro(cid:173)
`viding relatively low latency copies of live video streams
`with relatively high quality (e.g., high definition HD and
`high bit rate, such as 2 to 5 megabits per second), synchro(cid:173)
`nizing such low latency and high quality copies of a live
`60 video stream amongst multiple viewers (particularly as the
`number of viewers of a given live video stream significantly
`increases), and allowing for different classes or types of
`viewers to flexibly access copies of a live video stream via
`different live streaming formats (which may lead to different
`65 qualities of service).
`With these various technological challenges in mind, the
`present disclosure relates generally to inventive systems,
`
`BACKGROUND
`
`The public's access to the Internet, as well as the amount
`of bandwidth for Internet users, increased greatly in the
`mid-1990s to the early 2000s. The available Internet band(cid:173)
`width allowed for the streaming of both recorded audio and
`
`Sportradar 1009
`Page 60
`
`

`

`US 11,039,218 Bl
`
`3
`apparatus, and methods for facilitating one or more broad(cid:173)
`casters to create/provide video and/or audio (also referred to
`herein as a "broadcast") and allow one or more viewers to
`consume the video and/or audio, either by receiving a copy
`of a live stream representing the video and/or audio essen(cid:173)
`tially in real-time as created/provided by a given broad(cid:173)
`caster, or by retrieving and playing a recording of the live
`stream at a later time. In the context of essentially real-time
`viewing of live streams, in various implementations the
`inventive systems, apparatus and methods discussed in 10
`detail herein address one or more technological problems
`relating to viewer latency, synchronization amongst different
`numbers of viewers, and providing scalable and flexible
`access to live streams to different classes/types of viewers
`and/or with different qualities of service.
`With reference to FIG. lA, in various implementations a
`given broadcaster uses a network-connected client device
`(e.g., a first smart phone or other personal computing device
`communicatively coupled to the Internet) to generate a live
`stream of digital content corresponding to the video and/or 20
`audio created/provided by the broadcaster, wherein the live
`stream is transmitted to an inventive server and memory
`storage architecture (additional details of the server and
`memory storage architecture are shown, for example, in
`FIGS. 2 and 3). The inventive server and memory storage 25
`architecture processes the broadcaster's live stream to gen(cid:173)
`erate multiple copies of the live stream which are provided
`to respective viewers, and also records the live stream and
`stores the recording for later replay. One or more viewers
`using different network-connected client devices (e.g., a 30
`second and third smart phone or other personal computing
`device communicatively coupled to the Internet) may com(cid:173)
`municatively couple to the server and memory storage
`architecture to receive a copy of the live stream of the digital
`content as a real-time or essentially real-time consumer of 35
`the broadcast created/provided by the broadcaster, or
`retrieve and play a recording of the live stream generated by
`the broadcaster client device ( and recorded by the server and
`memory storage architecture). In some implementations, a
`given broadcaster themselves may view their own broadcast 40
`as a viewer on the same client device ( e.g., by opening a
`separate tab on their client device and connecting to the
`server and memory storage architecture as both a broad(cid:173)
`caster and a viewer). In one aspect, the inventive server and
`memory storage architecture allows any number of broad- 45
`casters to generate respective live streams of digital content,
`and any number of viewers to receive respective copies of a
`given broadcaster's live stream of digital content. In another
`aspect, the inventive server and memory storage architecture
`allows a given viewer of a first broadcaster's live stream to 50
`effectively switch in essentially real-time to view one or
`more other broadcasters' live streams.
`In some implementations, the inventive systems, appara(cid:173)
`tus, and methods further facilitate a social platform in
`tandem with broadcasting video and/or audio to one or more
`viewers, in which a given broadcaster and their viewer(s)
`may communicate with one another ( e.g., via chat messages)
`in essentially real-time during a broadcast. In one aspect,
`one or more chat messages, as well as various viewer-related
`information (e.g., name, surname, profile photo), may be
`displayed on respective client devices used by the broad(cid:173)
`caster and the one or more viewers as the video and/or audio
`is rendered on the respective client devices.
`In some implementations, the video and/or audio created/
`provided by one or more broadcasters is video-based com(cid:173)
`mentary about a live event being observed by a given
`broadcaster. In various aspects, the broadcaster may take the
`
`4
`role of a main commentator for the event, or a "color
`commentator" (e.g., analyst or s=arizer) for the event; in
`a given role, the broadcaster may describe the event as it
`occurs (e.g., provide chronological or "play-by-play"
`updates to "call" the event), and/or may provide expert
`analysis, background or anecdotal information about the
`event and/or participants in the event, and/or relevant sta(cid:173)
`tistics regarding the event, participants in the event, or
`related events. FIG. 1B illustrates a display of an example
`client device, in which a broadcaster is providing video(cid:173)
`based commentary about a sports game, and in which chat
`messages from one or more viewers, event information
`about the sports game, and other graphics and/or animations
`are displayed together with the broadcaster's video-based
`15 commentary.
`Examples of various activities constituting events accord(cid:173)
`ing to the present disclosure include, but are not limited to,
`a variety of sports games (e.g., professional, semi-profes(cid:173)
`sional, intramural, community-oriented) or sporting activi(cid:173)
`ties (e.g., exercise-related activities, physical training activi(cid:173)
`ties, fishing, hunting), musical performances, theatrical
`performances, other artistic or entertainment-oriented activi(cid:173)
`ties, speeches or presentations, political activities (e.g.,
`election-related activities, campaign-related activities, pub(cid:173)
`lic or "town hall" meetings, public marches or demonstra(cid:173)
`tions), military activities, professional activities (e.g., meet-
`ings or conferences), academic or educational activities
`( e.g., academic presentations or lectures, research activities,
`medical or scientific procedures, ceremonies), cooking or
`baking activities, competitive activities (e.g., racing activi(cid:173)
`ties), game-related activities (e.g., online gaming, board
`games, chess matches, role-playing games), social activities,
`and news-related activities. In one aspect, the video-based
`commentary provided by one or more broadcasters about the
`event and consumed by one or more viewers may provide a
`"second screen" experience for the viewers; in particular, in
`some implementations, the viewers may consume the video(cid:173)
`based commentary about the event on their respective client
`devices as they are watching the event itself on another
`device (e.g., a television), or watching and/or participating
`in the event at the event's venue.
`As discussed in greater detail below, in various aspects the
`inventive systems, apparatus and methods described herein
`provide an improvement in computer network functionality
`by facilitating scalable and appreciably low-latency viewing
`of copies of multiple broadcasters' live streams of video-
`based commentary about an event by significant numbers of
`viewers as the event unfolds. Particularly in the context of
`a "second screen" experience relating to a live event, dis(cid:173)
`cernible delay between the observation of the event itself
`and a given broadcaster's video-based commentary would
`significantly undermine viewer experience. Accordingly, the
`inventive systems, apparatus and methods described herein
`specifically address multiple technological problems in con-
`55 ventional techniques relating to transport of digital content
`via the Internet by providing multiple technological solu(cid:173)
`tions to ensure a low-latency viewing experience for sig(cid:173)
`nificant numbers of viewers.
`In such implementations relating to video-based commen-
`60 tary about events, the inventive systems, apparatus and
`methods disclosed herein further may facilitate display, on
`respective client devices used by the broadcaster and the one
`or more viewers, of various event information in tandem
`with the video-based commentary rendered on the respective
`65 client devices. For example, in connection with sports
`games, displayed event information may include, but is not
`limited to, one or more of team information ( e.g., team
`
`Sportradar 1009
`Page 61
`
`

`

`US 11,039,218 Bl
`
`5
`names, abbreviations and/or logos), score information (e.g.,
`with essentially real-time score updates synchronized with
`the video-based commentary), player information, venue
`information, game status information (e.g., on-base, at-bat,
`timeouts, fouls, pole position, yards-to-go, yards-to-goal,
`down), team statistics, player statistics and the like. In some
`implementations, such event information about a sports
`game may be rendered in the display of a client device as a
`"scorebug" that may include for example, team abbrevia(cid:173)
`tions or logos, team scores, game status ( e.g., period, quarter,
`inning), and actual or elapsed time for the game, with
`updates to one or more information elements in essentially
`real-time as the game progresses. It should be readily
`appreciated that for other types of events, a wide variety of
`information germane to the event may be displayed as event
`information (e.g., with essentially real-time updates of
`evolving information as the event progresses) together with
`the rendering of the video-based commentary on respective
`client devices. In another aspect, various event information
`or other information germane to a given broadcaster's
`video-based commentary may be rendered on a viewer client
`device in a "lower third" (also sometimes referred to as a
`"bottom third") of a display ( e.g., as an overlay to the
`video-based commentary).
`In view of the foregoing, in other aspects the inventive
`systems, apparatus and methods described herein provide
`additional improvements in computer network functionality
`by facilitating scalable and appreciably low-latency syn(cid:173)
`chronization of displayed event information with multiple
`broadcasters' live streams of video-based commentary about
`an event as viewed by significant numbers of viewers as the
`event unfolds. Particularly in the context of a "second
`screen" experience relating to a live event, discernible delay
`between the observation of the event itself and the status of
`event information displayed in tandem with a given broad(cid:173)
`caster's video-based commentary would significantly under(cid:173)
`mine viewer experience. Accordingly, the inventive systems,
`apparatus and methods described herein specifically address
`multiple technological problems in conventional techniques
`relating to transport of digital content via the Internet by
`providing multiple technological solutions to ensure not
`only a low-latency viewing experience for significant num(cid:173)
`bers of viewers ( e.g., relative to the event about which a
`broadcaster is providing video-based commentary), but
`appropriate synchronization of event information across all
`broadcasters of an event and their associated viewers.
`In some implementations as discussed herein, these tech(cid:173)
`nological solutions contemplate multiple Internet commu(cid:173)
`nication channels to provide relevant and synchronized
`information to client devices (as would be readily appreci(cid:173)
`ated in the relevant arts, a "communication channel" refers
`to a physical or logical connection over a transmission
`medium to convey information signals from one or more
`senders to one or more receivers). For example, in one
`implementation, a first "video" Internet communication
`channel ( e.g., between a media server or other media source
`and a client device) conveys the digital content correspond(cid:173)
`ing to the video-based commentary provided by the broad(cid:173)
`caster and consumed by one or more viewers, and a second
`"event information" Internet communication channel ( e.g.,
`between a particular socket of a socket server and the client
`device) conveys the event information. In other implemen(cid:173)
`tations, an additional "chat/system event" Internet commu(cid:173)
`nication channel (e.g., between another socket of a socket
`server and the client device) is employed to convey chat
`information and other information relating to system events
`regarding a particular broadcaster's live stream. In one
`
`6
`aspect, connections between client devices and a particular
`socket of a socket server are persistent authenticated con(cid:173)
`nections, so that the number of users (broadcasters and
`viewers) connected to a particular socket (e.g., and currently
`watching a particular live stream and/or particular event)
`may be tracked.
`In some implementations, the inventive systems, appara(cid:173)
`tus, and methods further facilitate a variety of screen ani(cid:173)
`mations ( e.g., motion graphics), customized displays or
`10 screen backgrounds, and other special effects graphics (e.g.,
`augmented reality) that are generally associated with the
`video and/or audio created by a broadcaster, and rendered by
`client devices in tandem with a given broadcaster's video
`15 and/or audio.
`In some implementations in which the broadcaster may be
`creating/providing video and/or audio content about an
`event, one or more such screen animations, customized
`displays or screen backgrounds, and other special effects
`20 graphics may be related to some aspect of the event. For
`example, as noted above, a "scorebug" providing informa(cid:173)
`tion about a sporting event may be presented on a viewer's
`display with a variety of motion or other special effects
`graphics. In one aspect, the information provided by such a
`25 scorebug is derived from the synchronized event informa(cid:173)
`tion that is received by the client device on an event
`information Internet communication channel that is separate
`from the video content representing the video-based com(cid:173)
`mentary. In another aspect, the scorebug and/or other special
`30 effects graphics or animations may be rendered in the
`display of the client device pursuant to execution of a client
`app or program on the client device (rather than having these
`display elements integrated into the broadcaster's live
`stream itself). In other aspects, one or more such screen
`35 animations and other special effects may be provided in a
`lower/bottom third of a client device's display, and may be
`interactive ( e.g., in that a user of a client device may select
`or thumb-over an animation, special effect and/or scorebug
`to retrieve additional information and/or initiate other ani-
`40 mations/graphics) and/or user-customizable (e.g., a broad(cid:173)
`caster may select from one of multiple background displays
`so that they may appear to be in any of a variety of locations
`or venues, customize their video-based commentary with a
`broadcaster-generated lower third, and/or add create/provide
`45 introduction videos to be shown before their live stream
`begins).
`In yet other aspects, screen animations, customized dis(cid:173)
`plays or screen backgrounds, and/or other special effects
`graphics may be related to one or more of advertising or
`50 branding ( e.g., by sponsors of a broadcaster, sponsors of an
`event about which a broadcaster is providing commentary,
`participants in the event itself, etc.), premium features and
`digital gifts ( e.g., provided by one or more viewers to one or
`more broadcasters or other viewers). For example, in some
`55 implementations, via a client app or software program
`executing on a client device, users will be able to purchase
`digital gifts for other users ( e.g., a viewer following a
`particular broadcaster may purchase digital beers, various
`digital concession stand items, ticker tapes, penalty flags,
`60 etc.), and the digital gifts will appear in the broadcaster's
`profile and/or be represented on a display ( e.g., as an icon or
`sprite) together with the broadcaster's video-based commen(cid:173)
`tary. In one implementation, information regarding digital
`gifts may be communicated between client devices and the
`65 server and memory storage architecture via a chat/system
`event Internet communication channel associated with a
`given broadcaster's live stream of digital content.
`
`Sportradar 1009
`Page 62
`
`

`

`US 11,039,218 Bl
`
`7
`In some implementations, the video and/or audio created/
`provided by one or more broadcasters may relate