Ulllted States Patent [19]
`Freeman et al.
`
`USOO5861881A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,861,881
`*J an. 19, 1999
`
`[54] INTERACTIVE COMPUTER SYSTEM FOR
`PROVIDING AN INTERACTIVE
`PRESENTATION WITH PERSONALIZED
`vIDEO, AUDIO AND GRAPHICS
`RESPONSES FOR MULTIPLE VIEWERS
`
`[75] Inventors; Michael J, Freeman; Gregory W,
`Harper, both of New York, NY
`
`[73] Assignee: ACTV, Inc., NeW York, NY.
`
`[ * ]
`
`Notice:
`
`This patent issued on a continued pros-
`ecution application ?led under 37 CFR
`1-53(d), and 15 Subject I0 the twenty year
`patent term PIOViSiOIlS 0f 35 USC-
`l54(a)(2)-
`
`[21] Appl. No.: 598,382
`_
`[22] Flled:
`
`Feb‘ 8’ 1996
`Related US. Application Data
`
`[63]
`
`Continuation-in-part of Ser. No. 443,607, May 18, 1995,
`IPlat.
`26782415091, ghiiilglgi; a tc)onzlinuactlion-?n-part of Ser.
`tinuatioIi of Ser. No. 797,298’, Nov. 25, 17991, abandoned.
`
`CC.
`
`21 an one W 1C 15 a COl'l-
`
`0.
`
`4207885 7/1992 Japan .
`6165170 6/199“ Japan ~
`8102961 10/1981 WIPO
`9403851 2/1994 WIPO
`OTHER PUBLICATIONS
`
`Harless et al., Interactive Video Disc Case Studies for
`Medical Education, Proceedings, 10th Annual Symposium
`on Computer Applications in Medical Care, Oct. 25—26,
`1986, pp. 183—187.
`Bock, Videodisk Standards: A Software VieW of the Tech
`nology, 8012 SMPTE Journal, VOL 92, NO_ 5, May 1983, pp
`571_576_
`Campbell, Optimal Decision Making in a Business Simula
`tion, System Sciences, 1989 Annual Hawaii International
`Conference, vol. 111, pp. 822—831.
`DaWson, Compression on the Fast Track, Cablevision, Apr.
`22, 1991, pp- 22—30
`Martial et al., An Interactive Planner for Open Systems,
`Proceedings of the 4th Conference on Arti?cial Intelligence,
`Mar' 1988’ pp' 293_298'
`(List continued on neXt page.)
`Primary Examiner—stephen S, Hong
`Attorney’ Agent’ Or Flrm—DOrSey & Whnney LLP
`
`'
`
`'
`
`[51] Int. Cl? ................................................... .. H04N 7/173
`[52] US. Cl. ............................................. .. 345/302; 348/10
`[58] Field Of Search ................................... .. 395/762, 806,
`395/807, 173, 174, 327, 328; 434/307 R,
`308; 364/514 A; 348/7, 8, 10; 345/302,
`473, 474, 327, 328; 707/501
`
`[561
`
`References Cited
`Us‘ PATENT DOCUMENTS
`
`2,612,553
`2,777,901
`
`9/1952 Homrighous .
`1/1957 Dostert ............................... .. 179/1002
`
`(List continued on neXt page.)
`
`FOREIGN PATENT DOCUMENTS
`
`16314 10/1980
`European Pat. Off. .
`128481 12/1984
`European Pat. Off. .
`314572 5/1989 European Pat. Off. .
`994233 6/1965 Germany .
`
`ABSTRACT
`[57]
`The present invention is an interactive Computer System
`which may operate or] a computer network' Subscribers
`interact With a fully interactive programthrough the use of
`input devices and a personal computer or a television. The
`multiple video/audio datastreams may be received from a
`broadcast transmission source or may be resident in local or
`external storage. In response to user inputs, a personalized
`graphics, video and/or audio presentation is provided to the
`user either Immediately or at a later time. If not presented
`immediately, the interactive computer system utiliZes “trig
`ger points” to determine When to enable multiple multimedia
`segments during the shoW. The CPU uses embedded or
`stored authoring commands for integrating the various mul
`timedia elements. The interactive multimedia computer
`enables seamless ?icker-free sWitching from one signal to
`another on the same or different channels.
`
`19 Claims, 13 Drawing Sheets
`
`Program
`"/SIBI'I
`
`900
`
`\l'rigger Point
`
`Trigger Points
`0
`900
`
`Standard Audlo
`
`l
`Select Audio
`
`+
`
`@
`
`I—|
`Graphlcs O
`
`PETITIONER EX. 1007 Page 1
`
`

`
`5,861,881
`Page 2
`
`US. PATENT DOCUMENTS
`
`3/1958 Hamilton .................................... .. 35/9
`2,826,828
`2,908,767 10/1959 Fritlinger
`179/100-2
`2,921,385
`1/1960 Hamilton
`...... .. 35/9
`3,008,000 11/1961 Morchand ..
`.. 348/485
`3,020,360
`2/1962 Gratian et al. .
`.. 179/100.2
`3,194,895
`7/1965 TreadWell .... ..
`.. 179/100.2
`3,221,098 11/1965 Feldman et al.
`.. 178/5.8
`3,245,157
`4/1966 Laviana .... ..
`35/9
`3,255,536
`6/1966 Livingston -
`~35/9
`3,273,260
`9/1966 Walker
`~35/9
`3,284,923 11/1966 Leslie
`- 35/8
`3,343,280
`9/1967 Tolnai
`35/35
`3,366,731
`1/1968 Wallerstein
`178/6
`3,387,084
`6/1968 Hine et al. ..
`. 178/6.8
`3,440,342
`4/1969 Beltrami ..
`178/6.8
`3,477,144 11/1969 Stillit . . . . . . . . . .
`. . . .. 35/9
`3,484,950 12/1969 Serrell et al. ..
`35/9
`3,485,946 12/1969 Jackson et al.
`178/6
`
`3/1991 Johnson et al. ......................... .. 348/10
`5,001,554
`4/1991 Makkunietal. .
`.. 348/358
`5,010,500
`5/1991 pocock et a1_ ____________________________ __ 348”
`570147125
`7/1991 Kamimura ............................ .. 434/322
`5,033,969
`7/1991 Von Kohom
`348/13
`570347807
`9/1991 Rhoades
`_ 463/25
`570517822
`3/1992 Hoarty et a1_
`__ 348”
`570937718
`4/1992 Harvey et a1
`_ 3530/9
`571097414
`4/1992 Bohrman
`395/328
`571097482
`571577491 1O/1992 Kassatly
`348/15
`5,174,759 12/1992 Preston etal
`434/317
`5,176,520
`1/1993 Hamilton
`434/350
`5,177,604
`1/1993 Martinez .
`348/13
`571817107
`1/1993 Rhoades
`348/13
`572207420
`6/1993 Hoarty et al_
`348/12
`572277874
`7/1993 Von Kohom "
`705/10
`572367199
`8/1993 Thompson’ 1L
`463/41
`572477347
`9/1993 Litteral et aL _
`348/7
`572617820 11/1993 slye et a1_ __
`463/1
`573187450
`6/1994 Carver
`434/336
`
`3,538,621 11/1970 Mayeda . . . . . . . . . . . . .
`
`2/1976 Yanagimachi 9t 91-
`3,936,595
`4/1976 Freeman ............. ..
`3,947,972
`3,988,528 10/1976 Yanagimachi 9t 91-
`
`. . . .. 35/9
`
`348/24
`434/321
`348/24
`
`573887197
`
`2/1995 Rayner
`
`5,454,722 10/1995 Holland eta.
`574557910 1O/1995 Johnson et aL
`5,467,288 11/1995 Fasciano et al
`
`395/328
`
`.. 434/322
`395/806
`. 364/514 R
`
`4,034,990
`
`7/1977 Baer
`
`463/31
`
`5,541,662
`
`7/1996 Adams . . . . . . . . . .
`
`. . . . .. 348/460
`
`348/11
`4/1981 Freeman
`4,264,924
`348/11
`4264925 4/1981 Freeman 91 a1-
`455/3-1
`4,290,142
`9/1981 Schnee 9t a1~ -
`345/327
`4,305,131 12/1981 Best --------- --
`~- 345/327
`4,333,152
`6/1982 Best ------ ~-
`. 348/14
`4,361,730 11/1982 Barber et al. ..
`385/83
`4,422,105 12/1983 Rodesch et al.
`.348/10
`4,507,680
`3/1985 Freeman ....... ..
`348/15
`4,516,156
`5/1985 Fabris et a1_
`. 380/23
`4,530,008
`7/1985 McVoy ......... ..
`4,536,791
`8/1985 Campbell et al. ...................... .. 380/10
`4,573,072
`2/1986 Freeman .................................. .. 348/10
`476027279
`7/1986 Freeman
`348/10
`4,635,132
`1/1987 Nakamura ............................. .. 348/296
`4,644,515
`2/1987 Allebest et al. ........................ .. 369/32
`476947490
`9/1987 Hawey et aL
`38000
`4,701,896 10/1987 Allebest et al. ........................ .. 369/32
`4,704,725 11/1987 Harvey et al. .......................... .. 380/19
`4,750,036
`6/1988 Martinez ------ --
`-- 348/358
`
`Lehman 618.1. ....................... .. 8/1988 Taub et al. ................................ .. 348/3
`4,768,087
`
`4,786,967 11/1988 Smith, IIIetal. .
`. 348/485
`
`4,807,031
`4,839,743
`4,846,693
`4,847,690
`4,847,700
`4,855,827
`4,862,268
`
`2/1989 Broughton et al. ................... .. 348/460
`6/1989 Best et al. ............................. .. 386/318
`7/1989 Baer ...... ..
`..
`7/1989 Perkins .................................. .. 348/483
`7/1989 Freeman .................................. .. 386/99
`8/ 1989 Best
`~~
`8/1989 Campbell et al. .................... .. 348/463
`giirldegnatt
`"""""""""""" "
`
`.. 396/327
`9/1996 Sampat et al
`5,557,724
`.. 348/485
`5,585,858 12/1996 Harper et al.
`348/10
`5,594,492
`1/1997 O’Callaghan etal.
`.. 455/2
`5,594,935
`1/1997 Reber et al.
`5,724,091
`3/1998 Freeman et al. ........................ .. 348/13
`
`OTHER PUBLICATIONS
`
`Gault, Turning Cold Box Into Interactive TV, Crain’s NeW
`York Business, Jul. 15, 1991, p. 17.
`Tsuruta et aL, A Knowledge Based Interactive Train Sched
`uling System Aiming at Large Scale Complex Planning
`Expert Systems, International Workshop on Arti?cial Intel
`ligence for Industrial Applications, 1988, pp‘ 490495‘
`.
`Gllder, 'Now or Never, Forbes, Oct. 14, 1991, pp. 188—198.
`KIIII, F1rStA11—D1g1ta1 HDTV Developed by W160 clphera
`Multichannel News, Jun. 11, 1990, p. 35.
`Kim, ATC: Technical Tidal Wave to Engulf Cable Industry,
`
`
`
`
`
`13, Multichannel News, Kim, Jerrold Offers Cable Digital Compression, Multichan_
`
`nel News, Mar. 11, 1991, p. 6.
`.
`.
`.
`.
`.
`.
`K1m>TestEqulpmentforDlgltalSnnaPuZZ1e>Mulnchannel
`News, May 27, 1991, P- 24
`Powell, Digitizing TV Into Obsolescence, New York Times,
`Oct. 20, 1991.
`Waterset al., Talking Back to the Tube, NeWsWeek, Dec. 3,
`1990, pp‘ 56_5'7~
`Will it Revolutionize HDTV‘), Broadcasting, Jun. 4, 1990,
`
`4,926,255
`5/1990 von Kohorn ........................... .. 348/13
`4,965,825 10/1990 Harvey et al. ............................ .. 380/9
`49757771 12/1990 Kassatly _______ __
`__ 348/469
`4,987,486
`1/1991 Johnson et al. ......................... .. 348/10
`
`PP- 33_34~
`_
`_
`_
`_
`Vldeo Storage In Dlgltal Transmlsslona Specs Technology,
`Cable Television Laboratories, vol. 3, No. 7, Sep. 1991, pp.
`1—6.
`
`PETITIONER EX. 1007 Page 2
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 1 0f 13
`
`5,861,881
`
`PETITIONER EX. 1007 Page 3
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 2 0f 13
`
`5,861,881
`
`PETITIONER EX. 1007 Page 4
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 3 0f 13
`
`5,861,881
`
`mm
`
`wmmvzmmw
`
`C/
`
`ow
`
`m wSmE
`
`mOIDOm
`
`OM05
`
`wOIDOw
`
`g.
`
`PETITIONER EX. 1007 Page 5
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 4 of 13
`
`5,861,881
`
`mmm_v_<mEw
`
`fl
`
`3mo:zo_>_
`
`mO._._ZO_>_
`
`WW
`
`mmm_v_<mn_m
`
`fl
`
`3m_o:zo_>_
`
`3mmm_v_<mn_m
`
`._.Zm:._O
`
`zoE¢m
`
`fl
`
`»zm__._o
`
`ZO_._.<._.w
`
`mm
`
`»zm_._o
`
`3ZO_._.<._.m
`
`v935
`
`omo_>
`
`%momsom
`
`omn__>
`
`%momaom
`
`om_o_>
`
`Qmm>mmm
`
`om_o_>
`
`PETITIONER EX. 1007 Page 6
`
`PETITIONER EX. 1007 Page 6
`
`
`
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 5 0f 13
`
`5,861,881
`
`wm
`
`m 239m
`
`G
`
`. 0 O I I O 0
`
`Q9
`
`\J
`
`
`
`C mEmO xmoo
`
`PETITIONER EX. 1007 Page 7
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 6 of 13
`
`5,861,881
`
`our
`
`OMQS
`
`m2<m¢
`
`mmumsm
`
`m_o+<._:oos_
`
`E;
`
`mo<mmm»Z
`
`on_
`
`om?
`
`o9:9“.
`
`oz>m
`
`mo»<mmzmo
`
`\mommmmm2oomo
`
`mmooomo
`
`o:
`
`mo_
`
`wow
`
`mo_
`
`PETITIONER EX. 1007 Page 8
`
`PETITIONER EX. 1007 Page 8
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 7 0f 13
`
`5,861,881
`
`om?
`
`
`
`5000mm 1 89>
`
`5&3
`
`
`
`g: x:
`
`1 \
`
`
`
`\/wmm¢%oomn_ \ 4565
`
`A
`
`om;
`
`\
`
`NEE own A \ my 0 \ i506
`H
`mo: \
`1 q 5000mm
`\ mmmmpm
`
`CO
`
`v
`
`m9
`
` r / I / / / <2: 59 $2
`
`
`x250 A 0% A mosmo
`#Eo? “E
`
`N 239m
`
`aEoa OE
`X250 81
`
`/ /
`
`$9 mm?
`, mug/F \
`
`g @950
`
`
`
`A \ _>_oE_>_<m
`
`//
`8?
`
`PETITIONER EX. 1007 Page 9
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 8 0f 13
`
`5,861,881
`
`PETITIONER EX. 1007 Page 10
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 9 0f 13
`
`5,861,881
`
`O O I
`
`Trigger Points
`
`700
`’I\
`I
`
`/900
`rL
`I
`
`t
`
`ts
`I
`I
`ts
`|
`
`I
`
`ts
`
`I
`I
`
`I > I
`I
`I
`I
`I
`I
`|
`I
`Y
`——I I
`I—I I
`I
`0
`I
`I
`I
`g
`I
`|
`II"—"—II
`|
`I
`I
`l
`I
`|
`I
`I
`l
`I
`I
`I
`|
`I
`I
`|
`
`I
`|
`
`I
`|
`
`I
`I
`
`Program
`
`Stan
`
`900
`
`Trigger Point
`"L
`I
`
`_
`Standard AUdlO
`
`|
`Select Audio
`+
`
`|
`I
`
`|
`|
`I
`II
`I I
`I
`.
`I I
`I
`z
`|
`|I——.—-II|"———JI
`|
`‘
`|
`II
`II
`I
`|
`I
`I
`I
`l
`l
`I
`
`I8 I
`Graphics 0
`I
`
`I
`I
`
`Figure 9
`
`PETITIONER EX. 1007 Page 11
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 10 of 13
`
`5,861,881
`
`.9m_:uo_>_
`
`coesm
`
`BBSEU
`
`.8Eo:oO
`
`Mme
`
`u_ooA-:oO
`
`H
`
`.9om:xm_
`
`
`
`own_>ucmnwmmm
`
`
`
`o¢u_>ccmnmmmm
`
`
`
`umfioumSNQ
`
`
`QasoumovEfioim
`amm-
`
`mm
`
`I
`
`_2oEmm
`
`or9:9“.
`
`fl
`
`>._oEw_2
`
`
`
`_._wm_u_s_<m
`
`fluNlwm
`
`PETITIONER EX. 1007 Page 12
`
`PETITIONER EX. 1007 Page 12
`
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Jan. 19, 1999
`
`Sheet 11 0f 13
`
`5,861,881
`
`NI
`
`(D
`
`LL
`
`PETITIONER EX. 1007 Page 13
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 12 of 13
`
`5,861,881
`
`mwm
`
`
`
`o_o=<_9_m_o
`
`>.oEm_>_
`
`mt
`
`._O._._mw§8_mm&__£_m_o
`.2ow._._o0
`
`
`
`o_U:<uaucfim
`
`fl
`
`.2om._mco
`
`
`
`>m_.w>O._o..m._w:m0
`
`
`
`xoo._.50Q
`
`::o.__O
`
`mt
`
`San.
`
`_2om.:xm_
`
`._w__o::oo
`
`EN
`
`aoigwz
`
`.._mm_u_
`
`afi
`
`_>_<m
`
`mm
`
`
`
` 22239_nEBxQ
`
`m
`
`
`
`ommhawSan.
`
`we
`
`
`
`W:2oEmm
`
`N
`
`VoSm_n_
`
`PETITIONER EX. 1007 Page 14
`
`PETITIONER EX. 1007 Page 14
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Jan. 19,1999
`
`Sheet 13 of 13
`
`5,861,881
`
`
`External(Opt.)Data
`
`Storage
`
`302
`
`
`
`%Character
`
`<E
`
`on
`
`Q
`
`Q
`
`LIJ
`
`
`
`
`623
`
`Gen.Lock
`
`175 Data
`
`
`
`StandardAudio
`
`NB
`Circuit
`
`
`
`Extractor
`
`(Optional)
`Printer
`
`
`Generator
`Overlay
`(Optional)
`E Modem
`
`Controller
`
`
`
`
`
`fifl
`
`< <0‘< wig‘
`
`§_><
`
`
`
`PETITIONER EX. 1007 Page 15
`
`L\l< <l'
`
`a|~5
` Figure13
`
`PETITIONER EX. 1007 Page 15
`
`

`
`1
`INTERACTIVE COMPUTER SYSTEM FOR
`PROVIDING AN INTERACTIVE
`PRESENTATION WITH PERSONALIZED
`VIDEO, AUDIO AND GRAPHICS
`RESPONSES FOR MULTIPLE VIEWERS
`
`RELATED APPLICATIONS
`
`This is a continuation-in-part of application Ser. No.
`08/443,607, ?led May 18, 1995 US. Pat. No. 5,724,091,
`Which is continuation-in-part of application Ser. No. 08/166,
`608, ?led Dec. 13, 1993, noW abandoned Which in turn is a
`continuation of application Ser. No. 07/797,298, ?led Nov.
`25, 1991, noW abandoned. The interactive television pro
`gram is of a type disclosed in US. Pat. Nos. 5,585,858,
`4,847,700, 4,507,680, 4,573,072, 3,947,792, 4,602,279,
`4,264,925, and 4,264,924.
`
`BACKGROUND OF THE INVENTION
`
`Interactive video and audio presentation systems are
`currently being introduced into the entertainment and edu
`cational industries. A prominent interactive technology that
`has been applied successfully in these industries is based on
`providing interactivity in a one-Way system through the
`provision of multiple parallel channels of information. For
`example, commonly oWned Freeman et al. patents, US. Pat.
`Nos. 4,264,925 and 4,264,924, Which provide both audio
`and video interactivity, disclose interactive television sys
`tems Where sWitching among multiple broadcast or cable
`channels based on vieWer selections provides an interactive
`capability.
`These systems have been enhanced to include memory
`functions using computer logic and memory, Where selec
`tion of system responses played to the vieWer are based on
`the processing and storage of subscriber responses, as dis
`closed in Freeman patent, US. Pat. No. 4,507,680.
`The bene?ts of providing interactivity through the use of
`different audio responses is disclosed in Freeman, US. Pat.
`Nos. 4,847,698, 4,847,699 and 4,847,700. These television
`systems provide a common video signal accompanied by
`several synchroniZed audio channels to provide content
`related user selectable responses. The audio signals produce
`different audio responses, and in some cases, these are
`syllable synched to a ?rst audio script and to the video signal
`(such as to a person or character on a display), providing the
`perception that the person’s or character’s mouth move
`ments match the spoken Words.
`Interactivity is brought to the classroom in the Freeman
`US. patent application Ser. No. 08/228,355. The distance
`learning system claimed in this application enhances the
`classroom educational experience through an innovative use
`of interactive technology over transmission independent
`media. When an instructor, either broadcast live on video or
`displayed from videotape, asks a question, each and every
`student responds, preferably by entering a response on a
`remote handset, and each student immediately receives a
`distinct and substantive audio response to his or her unique
`selection. The individualiZation of audio response from the
`interactive program is a major aspect of the invention.
`IndividualiZation of audio is brought to the home based on
`the technology disclosed in Freeman US. patent application
`Ser. No. 08/228,355. This system provides a program that
`can be Watched on any conventional television set or mul
`timedia computer as a normal program. But if the vieWer has
`a special interactive program box connected to the
`television, he or she can experience a fully functional
`interactive program. Each interactive vieWer enjoys person
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,861,881
`
`2
`aliZed audio responses and video graphics overlayed on the
`screen. The interactive program can be provided to televi
`sion sets or to computers by cable, direct broadcast satellite,
`television broadcast or other transmission means, and can be
`analog or digital. Unlike previous interactive systems, this
`application covers a system that subtly introduces the inter
`active responses to the vieWer throughout the program. This
`enhanced interactivity is provided through the use of “trig
`ger points” spread throughout the program. Trigger points
`occur at designated times and result in the program content
`being altered to present individual attention to the particular
`vieWer.
`HoWever, What is needed is an interactive personaliZation
`provided via an interactive multimedia computer.
`Furthermore, a system is needed that provides not only the
`ability to branch amongst parallel transmitted datastreams,
`but also, the capability to seamlessly integrate input from
`other media, such as CD-ROMs and laser disks, into the
`presentation. What is needed is a computer-based system for
`branching betWeen a variety of inputs during the same
`interactive session including full-motion video, computer
`graphics, digital video overlays and audio.
`
`SUMMARY OF THE INVENTION
`
`The ACTV system is based upon branches Which occur in
`the course of the full-motion video. Branches may be to
`other full-motion video segments, to graphics Which are
`integrated into the video, and/or to audio segments Which are
`integrated into the shoW.
`Sometimes, the ACTV system Will act upon the user’s
`response immediately; other times, it Will utiliZe ACTV’s
`unique “trigger point” concept to act upon the response later.
`ACTV’s technology enables the computer to “remember”
`the user’s responses and integrate them into the video and
`audio at a later point. Regardless of Whether the action is
`taken as a result of the user’s response immediately or later,
`it is done seamlessly.
`ACTV’s television technology provides the capability to
`seamlessly branch among multiple video and audio sources.
`ACTV’s computer technology provides the ability to seam
`lessly branch not only among the multiple video and audio
`channels, but also to seamlessly integrate input from other
`media, such as CD-ROM’s, laser disks, hard disks, and
`remote servers, connected via the Internet or another
`netWork, into the shoW.
`During a television-based ACTV interactive session, the
`system Will branch among either multiple television chan
`nels or multiple audio sources, depending upon the type of
`implementation. By contrast, during a computer-based inter
`active session, branches may be among a variety of inputs
`from a variety of different sources during the same interac
`tive session: full-motion video, computer graphics and
`audio. Since the computer provides the capability to process
`data from various multimedia inputs simultaneously, ACTV
`technology can integrate seamless sWitching of full-motion
`video, graphics and audio from various sources simulta
`neously during the shoW. The computer-based ACTV imple
`mentation is therefore much more ?exible than the
`television-based ACTV implementation.
`It also provides the user With the capability to interact
`With the shoW utiliZing a variety of input devices. Not only
`can the user interact With the shoW by pressing a multiple
`choice button, but the interaction can also take the form of
`entry via the range of multi-sensory devices available on the
`computer, including mouse entry, full-motion pen entry and
`touch screens. This integration of various input and storage
`
`PETITIONER EX. 1007 Page 16
`
`

`
`3
`devices is particularly valuable in an educational
`environment, since it provides students With the ability to
`participate in their lessons in a variety of Ways. The com
`puter can both store the interactions for future reference and
`also transmit them to the teacher, via either a computer
`netWork or, in a distance learning setting, via a telephone
`netWork.
`An ACTV interactive session can integrate full-motion
`video With user input at the same time. For eXample, the
`full-motion video may be playing on the screen, While the
`user is draWing a diagram in a corner of the screen. Thus, the
`video and audio may provide real-time input Which the user
`is applying during the session on the same computer moni
`tor.
`
`DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a diagram of an interactive computer
`Workstation, receiving inputs from television broadcasts
`and/or local storage devices.
`FIG. 2 is a diagram of an interactive computer Worksta
`tion Which receives its input primarily from television
`broadcasts.
`FIG. 3 is a diagram of an interactive computer Worksta
`tion Which receives its interactive programs entirely from
`local storage, rather than television broadcasts.
`FIG. 4 is a diagram of an interactive netWork for inter
`active processing.
`FIG. 5 is a diagram of an interactive computer system,
`receiving inputs from a multichannel cable transmission and
`shoWing outputs via a conventional television monitor.
`FIG. 6 is a block diagram of one interactive computer
`Workstation embodiment to achieve seamless sWitching
`betWeen video signals.
`FIG. 7 is a block diagram shoWing an alternative inter
`active computer Work station embodiment to achieve seam
`less sWitching betWeen video signals.
`FIG. 8 is a block diagram shoWing another alternative to
`achieve seamless sWitching betWeen video signals.
`FIG. 9 is a time diagram shoWing a representation of
`trigger points and corresponding alternative audio, video or
`graphics segments, one of Which is selected for presentation
`to the subscriber immediately after the eXecution of a trigger
`point function.
`FIG. 10 is a diagram of an interactive computer Work
`station embodiment for branching amongst multiple audio
`segments in a single video channel embodiment, Where the
`interactive audio and data elements are embedded in the
`video channel.
`FIG. 11 is a diagram of a second interactive computer
`Work station embodiment for branching amongst multiple
`audio segments in a single video channel embodiment,
`Where the interactive audio segments are sent in the SAP
`audio channel.
`FIG. 12 is a diagram of a third interactive computer Work
`station embodiment for branching amongst multiple audio
`segments in a single video channel embodiment, Where tWo
`tuners are employed; the ?rst tuner for tuning to and
`demodulating the standard video and audio signal and the
`second of Which is for demodulating a secondary analog
`carrier comprising modulated serial digital audio segments.
`FIG. 13 is a diagram of a fourth interactive computer
`Work station embodiment for branching amongst multiple
`audio segments in a single video channel embodiment, also
`comprising tWo tuners, but With a digital demultipleXer
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`5,861,881
`
`4
`con?guration for demultipleXing the digital audio stream
`into n parallel digital audio channels, Wherein the n parallel
`digital audio channels are time division multiplexed at the
`head-end and transmitted as a separate digital audio stream.
`
`PREFERRED EMBODIMENT
`
`As shoWn in FIG. 1, the present invention is a computer
`based system for receiving a fully interactive program,
`alloWing subscribers to interact With the program through
`the use of a keypad and personal computer. Alternatively, the
`multiple video/audio datastreams may be received from a
`broadcast transmission source or may be resident in local or
`external storage including CD ROM, video datatape, etc., as
`discussed beloW.
`The interactive computer, 6 uses an interactive program
`delivery system With any transmission means including
`satellite, cable, Wire or television broadcast to deliver the
`interactive program (hereinafter “composite interactive
`program”) from a centraliZed location, or operations center,
`for distribution to subscribers in their homes. The program
`may be broadcast live from the operations center. For
`eXample, live sporting events With added interactive ele
`ments can be broadcast from the operations center. Such live
`interactive elements could be different camera angles, sloW
`motion video, etc. Alternatively, the program can be pro
`duced off-line and stored in a program storage means at the
`operations center. Furthermore, the program can be pro
`duced and stored locally at the remote site on CD ROM or
`some other transferrable storage device such as digital or
`audio videotape, or laser disk.
`An interactive presentation can comprise branching
`amongst full motion video, computer graphics and audio,
`With the interactive elements either received over a trans
`mission media or stored locally, or both, all Within the same
`shoW. As shoWn in FIG. 1, the Workstation can branch
`among video segments from television broadcasts, local
`video servers 38, 42 (such as CD-ROMs, laser disks and tape
`players), still images and audio segments from the preceding
`media, as Well as those stored digitally on hard disks 34, and
`segments obtained over netWorks such as the Internet.
`The present invention, as shoWn in FIG. 1, is a system for
`processing on a computer a fully interactive program alloW
`ing users to interact With the program through a computer
`input device 22 connected to a standard computer system 6,
`comprising a CPU 108, hard disk 34, audio card 30 and
`monitor 18. The interactive multimedia computer 6 resides
`in the home of the subscriber or elseWhere, such as at a cable
`headend, as described beloW. If at the home, the interactive
`computer, 6 is usually located near the subscribers’
`television, if connected to the television set. Preferably, any
`of the multimedia computer embodiments, discussed beloW,
`comprise a video demodulator board, a keypad for entering
`subscriber selections, a data eXtractor board 46 (for extract
`ing data from the vertical blanking interval from the video
`signal(s), temporary and permanent data storage, a modem
`14, and a processor 108.
`Broadcast television is received by the video selector 10,
`Which selects among various television channels to capture
`a video signal to be displayed on the computer monitor 18.
`Multiple television channels may be received. FIG. 2 shoWs
`an interactive computer Workstation con?guration Which
`receives its input primarily from television broadcasts. With
`the technology of the present invention, seamless branching
`is provided among these television channels.
`In the preferred embodiment, interactive programming is
`also stored on Video Source devices 38, 42, as shoWn in
`
`PETITIONER EX. 1007 Page 17
`
`

`
`5,861,881
`
`5
`FIGS. 1 and 3. The Video Sources 38, 42 may be any local
`storage device Which is accessible by the computer, includ
`ing CD-ROMs, laser disks, VCR’s and tape players. While
`FIGS. 1 and 3 shoW only tWo video sources, there may be
`any number of such devices.
`When CD-ROM 54 is employed in the present invention,
`it is a component of a unique interactive experience. The
`present invention utiliZes CD-ROM 54 as one of the mul
`tiple input devices. Since branching is alWays seamless in
`the preferred embodiment, the computer 6 may receive input
`from at least tWo devices, regardless of Whether these
`sources are random access. This is necessary to avoid delays
`during search periods. While one device is playing the video,
`the other searches for a neW branch. When the second device
`?nds the segment for output display, the other input device
`searches for a neW branch. When the second device ?nds the
`segment to be shoWn, the branch occurs seamlessly. The
`apparatus and method for seamless branching among vari
`ous video signals is described in the paragraphs beloW.
`Segments of the interactive program may also be stored
`on the computer’s hard disk 34. The segments stored on the
`hard disk, 34 are usually computer graphics, still images or
`audio segments, Which are integrated into the presentation.
`The format for storage on the hard disk 34 is digital. Any
`storage device may, hoWever, store any combination of
`full-motion video, still images, graphics and audio seg
`ments.
`As shoWn in FIGS. 1—3, the interactive commands are
`extracted from the program by the Command Extractor 46.
`Alternatively, these commands may be stored on an auxil
`iary storage device such as the hard disk 34.
`The commands are processed by the computer’s Central
`Processing Unit (CPU) 108, shoWn in FIGS. 1—3. The
`computer may be an IBM Personal Computer (PC)—
`Compatible, an Apple computer or any other type of stan
`dard computer Workstation.
`The CPU 108 determines What video to display and audio
`to play based upon the interactive commands Which it
`receives. Based upon the commands, it plays the appropriate
`input from its input devices, Which are the Video Selector
`10, Video Sources 38, 42 and Hard Disk 34. Audio is
`received and processed by the Audio Card 30 Which sends
`audio to Speakers 26 and/or headphones 50 as shoWn in
`FIGS. 1—3.
`The user interacts With the program through the Input
`Device 22. This device may be a customiZed keypad, a
`standard computer keyboard, a mouse to “point and click” at
`selections and also to draW pictures, a touch screen (enabling
`the user to make a selection by pointing at the screen), a
`pen-based input device (for selecting options or draW
`pictures), a voice recognition device or any other computer
`input device Well knoWn in the art. Furthermore, multiple
`input devices may be accommodated by the system.
`Regardless of the type of input device 22, user inputs can
`be utiliZed by the present invention immediately, or at a later
`time, to result in personaliZed graphics, video and/or audio
`presentation. For example, the present invention utiliZes
`“trigger points,” as described beloW, to enable subsequent
`branches among multimedia segments during the shoW.
`Additionally, more substantive user input, such as pictures
`and text, may be integrated into the interactive presentation.
`These types of user input are particularly useful in computer
`aided learning applications, since they enable students to
`participate in lessons utiliZing various media. The interac
`tive computer 6 provides the frameWork to easily integrate
`the student’s multimedia input into the session and to
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`transmit the multimedia input to other students and teachers,
`via computer netWork and/or television broadcast.
`As shoWn in FIG. 4, the interactive system of the present
`invention may operate on a computer netWork. In this
`con?guration, the program is processed by the Video Server
`70. The programs are sent over the netWork to the Client
`Stations 58, 62, 66. Any number of client stations may be
`supported. The con?guration of each client station is pref
`erably the interactive Workstation as shoWn in FIG. 3.
`The control for integrating the various multimedia ele
`ments is provided by the ACTV authoring language, a
`unique set of interactive commands to facilitate the interac
`tive process. These commands may either be embedded into
`data portions of full-motion video segments or may reside
`separately on a storage medium such as a Winchester disk.
`When the commands are embedded Within the full-motion
`video (for example, Within the vertical blanking interval),
`the interactions occur as soon as the computer completes the
`recognition of a command group. When the commands are
`stored separately from the video segments in a digital
`segment, the timing of their execution is based upon “trigger
`points.” These trigger points are time points at Which the
`interactions are to occur, as explained in more detail beloW.
`The user can vieW the interactive program either directly
`using the television set 90 or via the computer 94 screen as
`shoWn in FIG. 5. FIG. 5 is a diagram of an interactive
`subscriber station, receiving inputs from a multichannel
`cable transmission and shoWing outputs via either the com
`puter 94 screen or a conventional television 90 monitor.
`Cable channels can be shoWn in a WindoW on the PC screen
`using conventional demodulator cards. In this embodiment,
`a cable set top box receives the plurality of analog or digital
`video/audio signals from the multichannel cable. The inter
`active multimedia computer 94 also receives the video/audio
`signals from the multichannel cable and extracts the data
`codes, preferably embedded in the vertical blanking interval
`of the video signal(s). The interactive computer 94 detector
`detects and extracts data codes embedded in the data stream.
`These codes are preferably sent to RAM memory and
`interpreted by the main processor. PersonaliZed audio and/or
`video se