Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 1 of 24
`
`
`Exhibit 3
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 2 of 24
`Case 5‘16'°“'°°349'”° ”°°””"eFI|l|l|||||||f|||1|I|Il”l||l|“|||lI‘|lI||llllflfllfllllllllllfilllllllllll
`
`US007028327B1
`
`(12)
`
`United States Patent
`
`Dougherty et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,028,327 B1
`Apr. 11, 2006
`
`(54) USING THE ELECTRONIC PROGRAM
`GUIDE TO SYNCHRONIZE INTERACTIVITY
`WITH BROADCAST PROGRAMS
`
`(75) mvemors; Brian 1)_ Doughertya Lafayette, CA
`(US); C_ Leo Meier, Berkdey, CA
`(US)
`
`(73) Assignee: Wink Communication, Alameda, CA
`US
`(
`)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`USC. 154(1)) by 0 days.
`
`( * ) Notice:
`
`(21) APP1~N°~ 09/538502
`(22)
`Filed:
`Mar. 29, 2000
`
`12/1996 Wanderscheid et al.
`5,583,563 A
`5,585,858 A * 12/1996 Harper et al.
`............. .. 348/485
`5,586,264 A
`12/1996 Belknap et al.
`5,652,615 A
`7/1997 Bryant et a1.
`5,657,072 A
`8/1997 Aristides et al.
`5,675,373 A
`10/1997 Joiner et al.
`5,689,799 A
`11/1997 Dougherty et al.
`
`............ .. 348/13
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`0942595 A2
`3/1999
`
`Ep
`
`OTHER PUBLICATIONS
`Advanced Television Enhancement Forum Specification
`(ATVEF), Version 1.1r26, Feb. 2, 1999, pp. 1-37.
`.
`(Continued)
`
`Related US‘ Application Data
`(60) Provisional application No. 60/179,825, filed on Feb.
`2, 2000,
`
`Primary Examiner—Kieu-Oanh Bui
`(74) 141107716)/, Agent,
`07’ Firm—SChWegInan, Lundberg,
`Woessner & Kluth P.A.
`
`(51)
`
`Int, Cl,
`(2006.01)
`H04N 7/173
`(52) U.S. Cl.
`.......................... .. 725/93; 725/40; 725/51;
`725/109; 725/112
`(58) Field of Classification Search .......... .. 725/39—4o,
`725/51, 109, 112,113
`See application 1113 for C0mp1ete Search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,751,578 A
`5,307,173 A
`5,334,975 A
`5,343,239 A
`5,400,402 A
`5,448,568 A
`5,504,896 A
`5,539,920 A
`5,557,317 A
`5,559,550 A
`5,563,648 A
`
`6/1988 Reiter et 211.
`4/1994 Yuen et 211.
`8/1994 Wachob et al.
`8/1994 Lappington et al.
`3/1995 Garfinkle
`9/1995 Delpuch et al.
`4/1996 Schell et al.
`7/1996 Menand et al.
`9/1996 Nishio et al.
`9/1996 Mankovitz ................... .. 348/6
`10/1996 Menand et al.
`
`(57)
`
`ABSTRACT
`
`In a broadcasting system, an electronic program guide
`(EPG)1s used to determlne What broadcast program Is on 21
`given channel at a given time in a given location. With the
`particular determined broadcast program as an input, timing
`offset objects defining interactive content related to the
`determined broadcast program can be retrieved from a
`timing offsets database. Each timing offset object specifies a
`time from the beginning of the program, an act to perform
`at the time, and a reference to interactive content on which
`to perform the act at the time. With the reference to the
`interactive content as an input, an interactive application or
`other interactive content can be retrieved from an interactive
`
`content database. The EPG, the timing offsets database, and
`the interactive content database may be located logically or
`physically together or separately, and they may be cached
`locally or referenced from an external source.
`
`36 Claims, 8 Drawing Sheets
`
`Consumer Premise Equipmenlfiso
`
`
`
`
`
`
`Graphics Overlay
`Generator
`11.5
`
`
`
`
`Storage Device
`215
`Operating
`
`System
`33
`
`
`User Input
`User lnpul
`Receiver
`Decoder
`
`E
`Q
`
`IR Commend
` Miawmoessor
`Encoder
`
`210
`E
`
`Communication
`Pan
`2:2
`
`
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 3 of 24
`Case 5:16-CV-00349-NC Document 19-6 Filed 04/11/16 Page 3 of 24
`
`US 7,028,327 B1
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`OTHER PUBLICATIONS
`
`5,719,618
`5,729,279
`5,768,539
`5,774,664
`5,781,228
`5,790,198
`5,818,440
`5,819,034
`5,826,166
`5,835,717
`5,848,352
`5,850,447
`5,861,881
`5,872,589
`5,880,720
`5,931,908
`5,937,331
`5,970,206
`5,974,222
`6,006,256
`6,008,802
`6,018,768
`6,021,433
`6,061,719
`6,064,438
`6,181,335
`6,256,785
`6,295,647
`6,415,438
`
`*
`
`*
`
`D>>>>>>>>D>>>>>>D>D>D>>D>D>>D>D>D>>
`
`B1
`
`B1
`
`2/1998
`3/1998
`6/1998
`6/1998
`7/1998
`8/1998
`10/1998
`10/1998
`10/1998
`11/1998
`12/1998
`12/1998
`1/1999
`2/1999
`3/1999
`8/1999
`8/1999
`10/1999
`10/1999
`12/1999
`12/1999
`1/2000
`2/2000
`5/2000
`5/2000
`1/2001
`7/2001
`9/2001
`7/2002
`
`.......... .. 395/200.79
`............. .. 725/110
`
`........ .. 709/218
`
`Park
`Fuller
`Metz et al.
`Hidary et al.
`Sposato
`............... .. 348/460
`Roop et al.
`Allibhoy et al.
`Joseph et al.
`Brooks et al.
`Karlton et al.
`Dougherty et al.
`Peyret
`Freeman et al.
`Morales ..................... .. 348/13
`Iwafune et al.
`Gerba et al.
`.............. .. 709/219
`Kalluri et al.
`Yuen et al.
`................. .. 386/83
`Yuen et al.
`................. .. 386/83
`Zdepski et al.
`Iki et al. ................... .. 345/327
`Ullman et al.
`............ .. 709/218
`Payne et al.
`Bendinelli et al.
`Miller
`Hendricks et al.
`Klappert et al.
`Ramaswamy
`Blackketter et al.
`
`.......... .. 725/136
`
`...... .. 725/136
`
`Evain, J .-P., The Multimedia Home Platform—an overview,
`EBU Technical Department Review—Spring 1998.
`Pending United States patent application, “Automated
`Control of Interactive Application Execution Using Defined
`Time Periods,” U.S. Appl. No. 09/431,001, filed Nov. 7,
`1999.
`
`Pending United States patent application, Automatic Control
`of Broadcast and Execution Of Interactive Applications To
`Maintain Synchronous Operation With Broadcast Programs:
`Ser. No. 09/333,724, filing date Jun. 15, 1999.
`Pending United States patent application, Automated Retire-
`ment of Interactive Applications Using Retirement instruc-
`tions for Events and Program States, U.S. Appl. No. 09/334,
`131, filed Jun. 15, 1999.
`Statement
`from first named inventor concerning Cited
`Reference entitled Pending United States patent application,
`Automated Retirement of Interactive Applications Using
`Retirement
`instructions for Events and Program States,
`“U.S. Appl. No. 09/334,131, filed Jun. 15, 1999.”
`Wink Communications, Technical Specification, mSubject:
`5.2.17 Serverinfo Define, Dec. 31, 1995.
`ntl Group Ltd. 2000, Interactive Digital Television [online],
`[retrived on Nov. 21, 2000]. Retrived from the Internet
`<URL:
`http ://www.ntl .com/guides/digitaltv/interactive.
`asp>, 2 pages.
`
`* cited by examiner
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 4 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 4 of 24
`
`U.S. Patent
`
`0
`
`8f01Wh
`
`US 7,028,327 B1
`
`298
`
`ucmémm:
`
`8.
`
`2:
`
`6mmoumem
`
`0l2,2:E:I:H2:
`:o_:wmc_8..
`ADn..2:omu_>
`r.Smowmosom
`
`SI
`
`._o>$m
`
`2...
`
`
`
`
`
`
`
`:o_uwo__na<m>_uow.__8:_m:_>tau_m:m_mno30.._o:o_E:._._ou_~_.:92_nom:o_umoo._5._<._o_._n=_.0.59".
`
`
`
`
`
`
`
`
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 5 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 5 of 24
`
`U.S. Patent
`
`l'
`
`I I I I I
`
`
`
`Ownu:mEQ_:Ummm_E2n__oE=wcoo
`
`11'
`
`
`
`
`
`mI_0/«RIuEn.Sm_Wco_fio_::EEoo_m>_._o95_A_
`
`pl0
`
`00
`
`US 7,028,327 B1
`
`38_>mommflem
`
`0
`
`6_
`
`M.Qm.o§m$oES>m:w>omozaeo.
`
`
`§om=xm_2.8__
`
`gem.m.
`
`am
`
`UCNEEOON:
`
`Buoucm
`
`Q
`
`3_omm8oa9o_s_
`
`
`
`:35am:3%..8:
`
`
`
`Euoomo._m>_moom
`
`flE.
`
`3»LL
`
`___
`
`
`
`
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 6 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 6 of 24
`
`U.S. Patent
`
`§wmmnmfio
`
`
`
` QE2:oommmtomc_E_._.
`
`n232".
`
`1.,H1.m.~23.........A..
`
`US 7,028,327 B1
`
`002
`
`:1034..
`
`00
`
`6_mE:mcoo
`
`om_E2n_
`
`mflSEmEa_:cm
`
`mumm::mE<
`
`EmEQ_:UmE5
`
`Beawcoo98
`
`mm_Emcn_SE85
`
`.c:...um.mmcon
`
`Sanfimoumem
`
`as
`
`§mmfiem
`
`ficozmz
`
`Emmoumem
`
`am
`
`
`
`
`
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 7 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 7 of 24
`
`U.S. Patent
`
`002
`
`pl04
`
`13.1_r..D1.A_
`
`6Bcsmcoo
`
`om_E£n_
`
`
`.1...II«Man8»
`
`
`.§_a_am_ucmémoz
`
`E3200
`
`
`
`882893:0Es:
`
`v0.59".
`
`US 7,028,327 B1
`
`._mE:w:oo
`
`mm_Em:n_
`
`.$=a_am_
`
`mfimm:cmE<
`
`ma8..
`
`8_mco_.mz
`
`
`
`Emafimoumem
`
`3»
`
`3Emmoumem
`
`
`
`
`
`
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 8 of 24
`W
`
`_t.+Woe:
`
`
`
`
`
`
`65,..ms5528NB..8308__8__&<
`
`FM
`
`w7
`
`1B7MMJ+5.25as__.§=8
`
`
`
`
`
`
`.u<2mm0:Sm.EmEoo..m00...w>=oSoE_cog:m>_.oSmE_:35
`
`
`
`.o<%SmEflcoom>_.om._mE_tmms
`
`1II.1w>_.oSoE_con:
`
`
`
`
`
`co=8__&<PWinSosaco_.8__&<.3umc_Emu_Emzoom>__om.mE_%J+3&5.“as3528NE..8_8
`
`
`
`
`
`
`
`Au.M8EnSmso:o__mo__&<Eumc_Emu_EmE8m>_.om.mE_
`
`.1_n
`
`6t«S9sure.
`
`._mE=mcoomwmnsmoE980mc_E_._.Boo._:om_%mu...Vo_.moo_\_mcc22mE_.macan.8e%$anEnmm:98:08”3»52%CPmmemi
`
`
`
`
`ommco=mo__&<&m2:9".
`
`mumomD_Emre.
`
`.o<m«MSmEozoomum.%m>__om._oE_con:o>_8EmE_
`
`...omc_e.WinSofia:o_fiu__&<Euo_..__Emu_EmEoo
`
`
`.:_E_:3m_._.WA....
`m>__oEmE_m6AemaoaDM«E328_§__&<$83.5.0E5
`
`nNVK,fume:HWmomflowaofimto
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 9 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 9 of 24
`
`U.S. Patent
`
`r._
`
`2
`
`hS
`
`fl.06
`
`00
`
`US 7,028,327 B1
`
`
`
`
`
`uas2830ammo9__s_:oan
`
`M.So9Emaoa
`
`New:o_.moo_\_m::mco\oE=
`
`gm9Ewaoa
`
`6fimovmosBuniMfume:
`
`Asmaoa
`
`wJ+fume:
`
`
`
`«B.s%o._o_.8__&<
`
`J+fume:
`
`
`
`
`
`
`
`<5Smfioco=mo__&<.3uo_.Ecmu_EmEoom>=oEwE_
`
`
`
`
`
`www._EoEoum>_.om_oE_cog:.o<u+ouumE_«
`
`_._
`
`
`
`8m$258zeso
`
`
`
`Emm~m_._23:0
`
`
`
`
`
`mc_E:.homosomcomommcmmEmEa_=ummm_E9n_$E=m:oo
`
`wo._:m_n_
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 10 of 24
`S
`0
`UM40Au
`
`1B7MnMco:mm.,
`
`Wu7aS
`
`
`
`
`
`,RIu_co=o<co=o<co_.o<
`
`pU3|.E:4
`
`“WP.SU
`
`eMx.9:9".
`_..o.efi8umo._m
`.nI_u_MD.Emaoi
`
`mulw::
`
`DmMn.,
`9mMA
`
`._H.m76t
`
`by00
`
`9%lmamat
`e..98.30m.38Se:mM3as2Egon.
`
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 11 of 24
`U
`e
`0
`
`
`
`
`
`
`
`8..HUH.HU...6tHH...H..HH.M.wovumoflmooufionomoonomonfiomno_oo”S8KSum8;O.oomu:_E
`
`WWEM“WP.
`
`S.U
`
`mummA
`
`w.8»
`
`wo._:m_n_
`
`PNVum»«umman
`
`mI
`
`2
`
`
`
`MW«ESb3»8»Ba8»m8_.j._..88»Now8»So
`
`1BmM08u.Nmm.MNNEEEu.8Eonm”.Emu.EE__e
`PU...........
`
`dIIIIIIIIIm8..HHH.HH...
`._Hmmmmmm...E..mmmm_.mEm>m
`
`6UM£<H.nonuuunun%mm<mm.<nm<mm<nm<EmEmm_tm>um
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 12 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 12 of 24
`
`US 7,028,327 B1
`
`1
`USING THE ELECTRONIC PROGRAM
`GUIDE TO SYNCHRONIZE INTERACTIVITY
`WITH BROADCAST PROGRAMS
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`The present application claims priority from U.S. Provi-
`sional Patent Application Ser. No. 60/179,825, entitled
`“Synchronizing the Delivery of Digital Information—Inter-
`active Applications, Data, and/or Content—With Broadcast
`Television Programs and Advertisements”, filed on Feb. 2,
`2000.
`
`FIELD OF THE INVENTION
`
`The present invention relates generally to the field of
`interactive television, and more particularly to the auto-
`mated control of interactivity in synchrony with television
`broadcast programs.
`
`BACKGROUND OF THE INVENTION
`
`Interactive television attempts to augment broadcast pro-
`grams with viewer selectable interactivity. This interactivity
`is typically delivered in the form of interactive content, such
`as an interactive application, associated with the broadcast
`program. An interactive application allows the viewer to
`participate, often in conjunction with events or situations
`arising in the broadcast program. The interactive content that
`accompanies a particular broadcast program may be dis-
`placed on the television screen simultaneously with the
`broadcast of the broadcast program, and if so, likely covers
`a portion of the viewing screen of the broadcast program.
`For example, a broadcast game show may be accompanied
`by interactivity that allows the viewer to play along and
`input answers to questions being posed during the show.
`This interactivity is displayed on only a portion of the
`television screen, so as to allow the viewer to continue to
`watch the broadcast program.
`For interactivity that accompanies a broadcast program, it
`is desirable to maintain the interactivity in synchrony with
`the broadcast program. This means that the interactivity
`should be available to the viewer whenever the broadcast
`
`program that it accompanies is being broadcast, and that any
`state information that the viewer has created, e.g. an accu-
`mulated point total in a game, should be maintained, even if
`the broadcast program is temporarily suspended and then
`resumed, for example, during a commercial break.
`If further means that
`the interactivity should not be
`displayed when the accompanying broadcast program is not
`being broadcast or is otherwise interrupted. In particular,
`proper synchrony requires that the interactivity for a televi-
`sion show not be displayed when a commercial is broadcast
`during the show. This is for two reasons. First, the commer-
`cial may have its own interactivity accompanying it, such as
`a form for the user to request product information about a
`product advertised during the commercial. Thus, the com-
`mercial’ s interactivity must be displayed, and yet the show’ s
`interactivity must be re-displayed when the commercial (or
`series of commercials) is over and the television show is
`resumed. Redisplay of the show’s interactivity should be
`prompt and as noted, should preserver the state information
`that was available when the commercial began. For
`example,
`if the viewer was accumulating points in an
`interactive game, then the accumulated points should be
`available when the show’s interactivity is resumed. This first
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`concept also holds true for interactivity related to just a
`portion of a show. For example if interactivity is displayed
`allowing the viewer to order trousers like whose worn by the
`show’s host, the viewer’s accumulated points should gain be
`available when the interactive trousers promotion disap-
`pears.
`Second, even if the commercial(s) does not have its own
`interactivity, it is inappropriate to display the interactivity of
`the show during the commercial. This is because the inter-
`activity will override a portion of the television screen,
`thereby covering up part of the commercial, which is unac-
`ceptable to the advertiser that paid for the commercial to be
`broadcast. Also, the interactivity for the show is not asso-
`ciated with the commercial, and its appearance may confuse
`the viewer.
`
`It is also desirable to maintain the interactivity in syn-
`chrony with particular events on the broadcast program. This
`means that a particular interactive element should be avail-
`able to the viewer only when a corresponding broadcast
`event
`takes place on the broadcast program. Take the
`example of the broadcast game show. At the moment that the
`broadcast program displays the game show host offering a
`question to an in-studio contestant, it would be desirable for
`the associated interactive application to display a prompt
`inviting the viewer to answer the same question. Likewise,
`as the contestant begins to give has answer aloud,
`the
`interactive application would inform the viewer that “time is
`up” and cease to accept viewer input.
`Accordingly,
`it
`is desirable to provide an interactive
`television system, method, and software products that cor-
`rectly synchronize the interactivity of various broadcast
`programs as well as events during those programs, and
`accommodate the interruption of broadcast programs by
`other broadcast programs.
`One approach in the art involves sending the interactive
`application along with the broadcast program from the
`national network to the local affiliates. One method specifi-
`cally is to place a server with the ability to insert an
`interactive application into the vertical blanking interval
`(VBI) of a television broadcast at the national network
`uplink facility. The server integrates with the network play-
`list system or other program-identifying techniques to insert
`interactive applications appropriate for the current television
`program. The program-identifying techniques provide
`needed timing information for events such as broadcast
`program starts and stops, commercial break starts and stops,
`and interruption of either by the start of a second broadcast
`program. An overview of this early approach is illustrated in
`FIG. 1.
`
`In this early approach, a server module (software appli-
`cation) running on a broadcast server (BS) 110 receives
`real-time information on the current video program being
`broadcast from the playlist control system 120. The playlist
`control system 120 controls the sequencing of pre-recorded
`video content (shows and advertisements) shown here as
`video sources 130. This information is used to insert an
`
`interactive application, content, or data into the video stream
`via the data insertion unit (DIU) 140.
`The combined video and interactive application is then
`delivered (typically via satellite 170 as shown here) to a
`local network afiiliate 160 where the application is either
`delivered over the air 162 to an interactive enabled settop
`150 or over a cable system that re-retransmits the local
`signal 164 to an interactive enabled settop 150.
`In reality, this configuration sometimes fails to deliver
`interactive television. One problem is that the interactive
`application can be lost, destroyed, or knocked out of synch
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 13 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 13 of 24
`
`US 7,028,327 B1
`
`3
`with the television program between insertion of the appli-
`cation into the video stream by the data insertion unit 140,
`and reception by an interactive enabled settop 150. There are
`technical barriers to delivering interactive content synchro-
`nized with television program all along the path from the
`national network to the local affiliate. Several locations of
`
`potential difficulties 199 are illustrated in FIG. 1, and some
`of the technical barriers are described below.
`
`Multiple Network Feeds—broadcast networks have mul-
`tiple feeds to provide time shifted delivery of content for
`each time zone. The equipment used to record and replay
`video content for different time zones may inadvertently
`destroy the inserted interactive application by not properly
`recording the VBI lines that carry it.
`Digital Uplinks—some networks digitally comprises their
`signal or convert the signal in some other manner that may
`not preserver the VBI in which the inserted interactive
`application is transmitted. Since the digital compression
`facility is not expecting useful data in the VBI, the interac-
`tive application is not compressed and carried over into the
`digitized system along with the broadcast program, and thus
`is not available for the user.
`
`Local Affiliate Issues—not only may data placed in the
`video signal be inadvertently destroyed by equipment at the
`local affiliate, but well-intended local affiliates can corrupt or
`destroy the interactive application while time shifting the
`network feed. Time shifting is achieved by recording the
`video content from a network feed, but playing it back to the
`viewing audience at a later time. The video recording
`equipment can either fail to reload the VBI or improperly
`record it, thus eliminating the interactive application.
`What is needed is a way to deliver the synchronized
`interactivity along with the broadcast program it was
`intended to accompany without losing the interactivity por-
`tion before the broadcast reaches the user.
`
`A further difficulty is that the resource and bandwidth
`capabilities of the subnetworks that receive the network
`broadcast vary significantly. It would be desirable to send
`customized variations of interactive applications to each
`subnet. For example, local-into-local service may take the
`top few broadcast network channels in the country’s top 25
`markets and rebroadcast those channels as digitally encoded
`signals so that
`they can be seen outside of their local
`markets. Both the receiving hardware and broadcast band-
`width allow a richer application (e.g. with more or better
`features, options, functions, graphics, or sound) to be deliv-
`ered to the consumer using this technique. However, existing
`approaches at best merely pass along an existing interactive
`application as it was originally broadcast, without taking
`advantage of the additional subnet bandwidth.
`Accordingly, it is desirable to delivery various types and
`qualities of synchronized interactivity with the various cable
`(or other broadcasting) systems’ broadcasts of the same
`program based on differing hardware and bandwidth capa-
`bilities.
`
`SUMMARY OF THE INVENTION
`
`The present invention provides for automated control of
`interactivity to maintain synchronous display of the inter-
`activity and its respective broadcast program regardless of
`signal path corruption and other technical barriers along the
`broadcast path that may interfere with the conveyance of the
`interactivity in synchrony with the broadcast program.
`In accordance with the present invention, the interactive
`content need not accompany the broadcast program all the
`way along the unreliable path that begins at the national
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`networks. Instead, the broadcast program is augmented with
`the interactive content at a point further along the path
`toward the viewer. Either a server at the local network or
`
`cable head end, or the viewer’s own consumer premise
`equipment (CPE) can control the identification of the broad-
`cast program, and the coordination of the broadcast program
`with associated interactive content and other data.
`
`The first step in augmenting a broadcast program with its
`proper interactive content is to identify the broadcast pro-
`gram. To do so, a device, such as the CPE, references an
`Electronic Program Guide. Electronic program guides for
`television systems are known in the art, particularly with
`regard to cable television systems. They are also known as
`electronic television program guide schedule systems. The
`Electronic Program Guide may be broadcast to and stored on
`a CPE, or be resident on a server on a network such as the
`Internet and be accessed through a CPE. Conventionally, the
`EPG is used by viewers to visually determine the correct
`timing or channel for a particular television show they
`would like to watch, or to find out generally “what’s on TV.”
`The Electronic Program Guide is a database (also called “the
`EPG database”) that provides schedule information for
`broadcast (including cablecast) programs that may be avail-
`able for viewing on, for example, a television. The database
`coordinates geographic location,
`local time, and channel
`information with a corresponding broadcast program. A
`viewer’s CPE thus supplies the EPG database with infor-
`mation regarding what channel is being watched, at what
`time, and in what geographic area (or on what cable or other
`system), and the EPG database returns data regarding which
`broadcast program is being watched, such as a program
`identifier.
`
`In accordance with the invention, the program identifier
`from the EPG is then coordinated with a list of timing offset
`objects specific to the broadcast program. In one embodi-
`ment, the CPE accomplishes this coordination as follows.
`Supplying the program identifier retrieved from the EPG as
`an input, the CPE references a timing offsets database or an
`interactive content database that correlates the broadcast
`
`program identifier with the appropriate timing offsets or the
`appropriate interactive content. The referenced timing olf-
`sets database may be self contained, or it may be a part of
`the EPG database or a content database. The referenced
`
`timing offsets database may be located on the CPE, on a
`server at the local affiliate, or on a server on another network,
`such as the Internet. In one embodiment, the timing offsets
`database returns the timing offsets related to the broadcast
`program and the CPE later uses them to reference to the
`interactive content database and retrieve interactive content.
`
`In another embodiment, the content database may return an
`interactive application, an interactive application identifier,
`or an Internet address (URL) that leads to or controls an
`interactive application or list of timing offset objects as well
`as other data. In one embodiment the content database
`
`containing the interactive applications and interactive con-
`tent is located separately from the timing ofsets database.
`Interactive content
`includes interactive applications, ele-
`ments of interactive applications, and other interactive con-
`tent, but for ease of reference, the terms “interactive con-
`tent”
`and
`“interactive
`application”
`shall
`be
`used
`interchangeably to cover all forms of interactivity for broad-
`cast programs that are described herein or equivalent to
`those disclosed.
`
`Timing offsets generally are synchrony-oriented data such
`as broadcast program start and stop times or other timing
`data useful to the interactive applications. Start and end
`times for broadcast programs listed on the EPG are an
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 14 of 24
`Case 5:16—cv—OO349—NC Document 19-6 Filed 04/11/16 Page 14 of 24
`
`US 7,028,327 B1
`
`5
`example of an absolute timing offset. These times help an
`interactive application server known when to execute and
`terminate batch interactive applications, or when to execute
`and terminate individual interactive applications that com-
`mence execution at the start of a broadcast program, so that
`each element of the interactive application associated with
`the broadcast program will temporally coincide with appro-
`priate triggering events in the broadcast program.
`In one embodiment, synchrony between multiple indi-
`vidual interactive applications or events associated with a
`broadcast program and the broadcast program is maintained
`using relative timing offsets. In one embodiment, a list of
`timing offset objects is associated with a broadcast program,
`each timing offset object containing: a timing offset relative
`to the start of the program, an application object (such as an
`interactive application), and an action to take on the appli-
`cation object (such as execute, execute for a duration,
`suspend, resume, or terminate). Timing olfset objects trigger
`the processor that executes interactive content to initiate
`time-sensitive elements of the interactive content so that the
`
`elements will coincide with events taking place in the
`broadcast program. For example, the timing offset objects
`may contain time olfsets for the start and end times of each
`commercial; likewise, it is desired to synchronize elements
`in the interactive application (e.g. dialog boxes or voting
`buttons) with events in the broadcast program (e.g. game
`show questions), the timing offsets for such events may also
`be encoded. For instance, when a game show host is about
`to reveal the answer, a timing offset triggers termination of
`the “viewer answer window” interactive application, and the
`interactive window in which the viewer could have
`
`responded is removed from display. As another example,
`when a commercial break starts in the middle of a broadcast
`
`the interactive application associated with the
`program,
`broadcast program is triggered by a timing offset to suspend
`display, yet retain information the viewer will need when the
`program and the program’s interactive application resume.
`Timing olfset objects may be retrieved on demand or
`downloaded ahead of time. Timing olfset objects may con-
`tain simply a list of trigger times, relative to the start of a
`broadcast program, or as described above, they may contain
`trigger times in addition to indicators of what elements of the
`interactive application are to be executed, terminated, run
`for a duration, suspended, resumed, or triggered.
`The primary functions described above as being per-
`formed by a viewer’s CPE may also be performed by other
`devices at different locations between the national broad-
`
`caster and the CPE. For example, in one embodiment an
`application server at the local network facility or cable head
`end uses the EPG to coordinate time, channel and location
`information with a program identifier. The application server
`then uses the timing offset database to associate the program
`identifier with a timing offset object or list of timing offset
`objects which contains actions to take on interactive content
`at appropriate times during the broadcast program. The
`application server may then retrieve the interactive content
`from an interactive content database and insert it into the
`
`broadcast for reception by the viewer’s reception equipment
`which will
`then display the interactive content. In one
`embodiment, the content database itself is located at the
`server at the local network.
`
`These functions may also be divided between the appli-
`cation server and the CPE. For instance, in one embodiment,
`the application server uses the EPG to coordinate time,
`channel and location information with a program identifier,
`and then uses the timing offset database to associate the
`program identifier with a timing offset object. The applica-
`
`6
`tion server than inserts the timing offset object, or part
`thereof, into the broadcast stream for reception by the CPE.
`The CPE uses data from the received timing offset
`to
`reference the interactive content database and retrieve inter-
`active content.
`
`Finally, in any of the above embodiments, the interactive
`content is displayed (or otherwise provided) in synchrony
`with the broadcast program. Such display may be accom-
`plished by the CPE. In one embodiment the CPE displays
`the interactive content after the CPE has itself retrieved the
`
`interactive content through the EPG and timing offsets and
`acted upon it. In another embodiment the CPE displays the
`interactive content after having received it in the broadcast
`stream from the local network facility. In another embodi-
`ment the CPE displays the interactive content after having
`received the timing offsets in the broadcast stream, having
`used them to retrieve the interactive content, and then having
`acted upon the interactive content.
`In one embodiment,
`synchrony is controlled with reference to the local system
`clock in the CPE, as further described below. In one embodi-
`ment, this display of the interactive content overlaps the
`broadcast program visually; in another the display of the
`interactive content is on a separate charmel or CPE.
`The features and advantages described in this summary
`and the following detailed description are not all-inclusive,
`and particularly, many additional features and advantages
`will be apparent to one of ordinary skill in the art in view of
`the drawings, specification, and claims hereof. Moreover, it
`should be noted that the language used in the specification
`has been principally selected for readability and instruc-
`tional purposes, and may not have been selected to delineate
`or circumscribe the inventive subject matter, resort to the
`claims being necessary to determine such inventive subject
`matter.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`FIG. 1 is an illustration of the prior art with locations of
`potential corruption to the signal highlighted.
`FIG. 2 is a detailed illustration of the CPE.
`
`45
`
`50
`
`55
`
`60
`
`65
`
`FIG. 3 is an illustration of the overall system wherein the
`primary processing is done of the local network afiiliate.
`FIG. 4 is an illustration of the overall system wherein the
`primary processing is done at the CPE.
`FIG. 5 is an event trace of the embodiment wherein the
`
`primary processing is done at the local network afi‘iliate.
`FIG. 6 is an event trace of the embodiment wherein the
`
`primary processing is done at the CPE.
`FIG. 7 is a diagram of a timing offset object list.
`FIG. 8 is an example timeline with timing offsets in a
`broadcast program.
`The figures depict a preferred embodiment of the present
`invention for purposes of illustration only. One skilled in the
`art will readily recognize from the following discussion that
`alternative embodiments of the structures and methods illus-
`
`trated herein may be employed without departing from the
`principles of the invention described herein.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`I System Architecture
`Referring now to FIG. 3, there is shown an illustration of
`one embodiment of a system in accordance with the present
`invention. It will be appreciated that the system illustrated in
`
`

`
`Case 5:16-cv-00349-NC Document 19-6 Filed 04/11/16 Page 15 of 24
`Case 5:16-cv-OO349-NC Document 19-6 Filed 04/11/16 Page 15 of 24
`
`US 7,028,327 B1
`
`7
`FIG. 3 may be incorporated in to larger, more complex
`systems while still providing the features and benefits of the
`invention.
`
`Generally, system 300 includes one or more national
`broadcasters 310, means of transmission from national
`broadcasters to local broadcasters such as satellite 320, a
`plurality of local broadcasters typically including local net-
`work affiliates 330, and a transmitter 340/345.
`In one
`embodiment, this transmitter is a conventional cable head
`end 340, and in other embodiments the transmitter is a
`conventional television broadcast transmitter 345 or a high-
`definition television digital transmitter. Consume