United States Patent [19]
`Throckmorton et al.
`
`USOO5818441A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,818,441
`Oct. 6, 1998
`
`[54] SYSTEM AND METHOD FOR SIMULATING
`TWO-WAY CONNECTIVITY FOR ONE WAY
`DATA STREAMS
`
`[75] Inventors: John A. Throckmorton, Lake OsWego;
`Edward R. Harrison; Burt Perry,
`both of Beaverton, all of Oreg.
`
`[73] Assignee: Intel Corporation, Santa Clara, Calif.
`
`Appl. No.: 490,822
`
`Jun- 15! 1995
`
`l
`[
`[22] Filed?
`[51] Int Cl 6
`H0 4N 7 /10
`.
`.
`............................................... .... ..
`~
`~
`[52] US. Cl. ............................... .. 345/328, 348/13,;25/3
`58
`F, M f S
`h
`345 327 328_
`[
`]
`1e
`0
`earc """"""""""
`13/ 473’ 714;
`’
`’
`
`/ ’
`
`’
`
`[56]
`
`.
`References Clted
`U_S_ PATENT DOCUMENTS
`5 347 304 9/1994 M
`t
`1
`’
`’
`01,1“ e a ' """""""""""""" "
`£25715: 2: :11:
`348/7
`5,583,864 12/1996 Lightfoot et al.
`5,594,491
`1/1997 Hodge ....................................... .. 348/7
`
`348/12
`
`[- 10
`Primary Data
`Stream
`Generation
`
`12
`
`'
`
`5,613,192
`
`3/1997 Ikamietal. .............................. .. 348/9
`
`Primary Examiner—Victor R. Kostak
`Attorney, Agent, or Firm—Blakely, Sokoloff, Taylor &
`Zafman
`
`[57]
`
`ABSTRACT
`
`A system supplying information associated With a broadcast
`television program to a consumer such that said consumer
`perceives the associated data as the result of tWo Way
`interactivity With external sources of data such as online
`services or the Internet. The system includes equipment for
`inserting the associated data into the vertical blanking inter
`val of the television signal on the supplier side of the system.
`On the receiving Side, the System includes a personal Com_
`puter Capable of receiving the television program and Storing
`the associated data locally. The consumer may then interact
`With the stored associated data in an apparently tWo Way
`interactive manner. Additional interactivity may be achieved
`by adding an actual tWo Way communication channel to the
`personal computer so that online services or the Internet may
`be accessed. This tWo Way communication channel is made
`particularly effective if the associated data contains pointers
`to locations in the online services or the Internet Which are
`particularly relevant to the television program.
`
`42 Claims, 5 Drawing Sheets
`
`v F20
`Synchromzer
`A
`f” 22
`
`Data .
`
`V Sequencer
`
`v F14
`S
`24
`
`I
`
`F26
`
`r_30
`
`V Encoder
`
`> Transmitter
`
`S
`28
`
`Associated
`
`,_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
`
`Data
`Generation
`
`\
`
`18
`
`B l 6
`
`|
`|
`
`'
`|
`I
`|
`l
`I
`
`I
`|
`I
`|
`34 f|
`|
`|
`
`’
`
`Human
`Interface
`
`36
`
`\
`
`_
`Receiver
`
`r— 38
`
`Processor
`
`l
`l
`
`|
`|
`'
`|
`|
`|
`
`Printer
`
`L 44
`
`> Memory
`
`I
`|
`I
`I
`I
`L :
`
`APPLE 1003 - Page 1
`
`

`
`U.S. Patent
`
`Oct.6,1998
`
`Sheet 1 of5
`
`5,818,441
`
`$32G
`
`
` >\ a @0522 I Bwmuoo? A| HQMNWWMMMW
`
`w .9;
`
`$280M Ewobm QED Enobm QED
`
`
`
`
`
`5026C :EEQU
`
`APPLE 1003 - Page 2
`
`

`
`U.S. Patent
`
`Oct.6,1998
`
`Sheet 2 of5
`
`5,818,441
`
`SE62“; A|$V
`695cm
`
`3k
`
`8k
`
`EL
`
`5388a
`
`SE52
`
`836::
`
`mm
`
`60:25am AIM,
`2|)
`ENEOEQEAm
`coa?ocow
`i
`8mm
`“2ND
`
`SI/
`
`8k 2
`
`
`
`NED \CQEWE
`258%
`
`892260
`
`APPLE 1003 - Page 3
`
`

`
`U.S. Patent
`
`aO
`
`mm
`
`m.UFm
`
`BEBE
`
`oomucflflmm
`
`m.53528mmmm:o:mo:5EEoUhuuSmm
`
`6.,SadEooqSamRoodo::9.18%vm@mECn_
`
`
`
`
`
`
` mm%owomfiogmBwmcflz5&5wE..%m_unH_$m
`
`5
`
`1
`
`qoBwmsflz
`
`Mcm8.,mm
`
`
`
`
`
`5,fiooouounmHOGOODQ.~®>M®o®N~
`
`E
`
`«EDwBBoomm<
`
`moomom
`
`APPLE 1003 - Page 4
`
`APPLE 1003 - Page 4
`
`
`
`
`

`
`_
`
`_H.__0__M__
`_E252Emmi_
`
`9-955_35H__S.3_
`
`U
`
`P
`
`009
`
`Sa05
`
`
`BwoocmBososwomSan.n,mm36.BNEOEQSW%.§:Em:€r_L
`
`
`
`
`S$_m_=o:§o:oO_3mmLEND
`
`IIIIIII:I1IIIIIIIIIIIIIIIIc8Eoomm<
`
`mmomcm.2
`
`EwobmamNHMED.\CwECLmS
`t.
`
`sosfiocomv
`
`H.
`
`Mo,"3,W_E
`
`APPLE 1003 - Page 5
`
`
`

`
`U.S. Patent
`
`Oct. 6, 1998
`
`Sheet 5 0f5
`
`5,818,441
`
`
`
`
`
`fiozzozSan.wBm_oomm<
`
`mm
`
`
`
`85:85SE32
`
`m.UHm
`
`E
`
`mm
`
`8
`
`«EDBofiomSufiEooqSan.Roodofiihfiomvm?mE:m
`
`
`
`
`
`
`mSwmcflzowmgowm$92525&5:_%n.mwn~HHoM
`
`
`
`.$mw:£Zmcouwowcsfifiou
`
`8mm
`
`omcm
`
`
`
`weBoouoi_oo9o5BwoooflSfiooom
`
`
`
`
`
`653:0E:o:moE:EEoU
`
`b§>-o>>H
`
`
`
`Emmaxz.5w.mEw—2
`
`wmcm
`
`APPLE 1003 - Page 6
`
`APPLE 1003 - Page 6
`
`
`
`
`

`
`1
`SYSTEM AND METHOD FOR SIMULATING
`TWO-WAY CONNECTIVITY FOR ONE WAY
`DATA STREAMS
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`This invention relates to enhancing one-Way broadcast
`data transmissions. More particularly, it relates to enhancing
`the quality and content of a primary information stream by
`creating and transmitting associated data Which provides the
`appearance of an interactive connection to secondary
`sources of information.
`2. Background
`One of the limitations of nearly all forms of electronic
`mass media such as radio, television, audio CD’s and video
`cassettes is that the communication of information or data is
`one Way from the provider to the consumer. The character
`istics of one Way sources of data are that the data is sent
`sequentially and is ?eeting or at best can be recorded for
`later playback. And a consumer cannot interact With the data
`provided to seek additional information or services.
`There is a desire by electronic mass media consumers for
`additional services, and providers of television and radio
`broadcast services are looking for neW sources of revenue.
`In particular, consumers are looking for information or data
`that is related to What they are vieWing in the media. The
`additional information may be in the form of details on the
`content of the current program such as the recipe of a meal
`being demonstrated on a cooking shoW or biographies of
`actors in a drama or historical background information on
`events depicted in a program. Or, it could be program
`highlights such as key plays from the baseball game. In
`addition, consumers Would like access to real time data such
`as stock price, updated baseball scores as they occur, traf?c
`and Weather conditions. Also, consumers Would like to have
`access to special services associated With product advertis
`ing such as information on Where to buy nationally adver
`tised products—possibly With a map to the nearest location,
`or the ability to instantly receive coupons.
`Recently, online services such as provided by the World
`Wide Web of the Internet have become available. Such
`services provide access to immense amounts of data on an
`interactive basis by linking digital computers together over
`sophisticated communications netWorks. At the same time,
`the cost of digital computing poWer is falling rapidly. For
`example, many homes noW have one or more computers.
`And home computers have displays capable of shoWing
`television as Well as audio capability. They also have
`memory and computing poWer.
`But While such services are generally available and com
`puters are becoming more Widespread, ?nding relevant
`information is an arduous task that may take hours of
`searching even for an experienced user.
`In addition, these services require a 2-Way connection
`from a consumer’s location to the online netWork. In the
`consumer market, prolonged utiliZation of a single phone
`line for 2-Way digital communications is expensive.
`Up until noW, there has been no Way for producers of mass
`market broadcast programming to deliver data associated by
`its relevancy to its subject matter that could be interactively
`displayed and manipulated by consumers on a real time
`basis. What is meant by real time is that the consumer
`receives and has access to the relevant data during the
`process of program reception. Therefore the data becomes
`an integral part of the experience desired by the program
`producers.
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`5,818,441
`
`2
`Although received during a program, the information is
`retained and may be traversed by the consumer at a later
`time, as if they Were connected in a 2-Way fashion to an
`online service.
`3. Prior Art
`Perhaps the earliest effort to address the foregoing needs
`is for program content providers to include a telephone
`number in either the radio or television broadcast. The
`consumer is invited to dial a number for additional infor
`mation. HoWever, this telephone number is part of the
`primary data stream Which is transitory.
`A second attempt is the provision of closed caption
`programming for the hearing impaired on television. This
`takes the form of Written text appearing someWhere on the
`television screen typically at the bottom and requires a
`special decoder or mode set on the television. The text is a
`Written rendition of the audio portion of the television
`program. That is, it is a recapitulation of the information
`supplied by the primary data provider and is not stored or
`saved for later access by the consumer.
`Television netWorks also use a part of the unused band
`Width in the video signal to send the schedule of programs
`coming up on that netWork. One netWork is sending a
`comprehensive program listing in digital format over an
`unused section of the NTSC bandWidth. This is called an
`electronic program guide (“EPG”). A second netWork uses a
`portion of the unused NTSC bandWidth to send digitally
`encoded stock quotes and the subject matter of the topics
`covered in the neWs broadcast, and extended data services
`(“XDS”) sends the date, time, name of a scheduled program,
`type of program and hoW much of the program remains.
`Cable services provide additional detail about songs being
`played on subscriber digital audio services such as digital
`music express (“DMX(tm)”); The detail typically consists of
`the name of the artist, the name of the song and the album.
`A system called Gemstar(tm), provides information in digi
`tal format that enables consumers to record programs by
`referencing a number in a program guide.
`Many of the foregoing services are provided on broadcast
`television only over What is called the vertical blanking
`interval (the “VBI”). The VBI is a portion of the bandWidth
`de?ned for broadcast television and in the case of NTSC, for
`example, consists of the ?rst 21 of the 525 raster lines that
`de?ne a video frame. SECAM and PAL have a similar
`arrangement. Of the 21 lines, line 21 is divided into tWo
`frames of 60 b2 bandWidth each. The closed caption is
`provided in the ?rst frame of line 21.
`None of the foregoing alloW a consumer to experience
`apparent interactivity With external data sources. That is,
`none provides data that persists for more than a feW seconds
`and none alloWs the consumer to manipulate directly that
`data.
`
`SUMMARY OF THE INVENTION
`The foregoing problems may be resolved by a system for
`supplying a primary data stream and associated data to a
`consumer such that the consumer perceives the associated
`data as the result of interactivity With external sources of
`information. This system includes a creation unit for creat
`ing the primary data stream and the associated data. The
`primary data stream and associated data are supplied to a one
`Way delivery unit for delivering the primary data stream and
`the associated data to the consumer. At the consumer
`location, the system includes a receiver for receiving the
`primary data stream and a receiver for receiving the asso
`ciated data. These may be the same unit. The system further
`
`APPLE 1003 - Page 7
`
`

`
`5,818,441
`
`3
`includes computer memory storing the received associated
`data. A processor communicates With the memory for
`accessing the associated data. Finally, the system includes a
`user interface connected to the processor through Which the
`consumer may interact With the system.
`The method of the present invention includes the steps of
`?rst generating a stream of primary data and of associated
`data. Next, both the primary data stream and associated data
`are delivered to the consumer. Next, both the primary data
`stream and the associated data are received at the location of
`the consumer. The primary data stream is rendered to the
`consumer as it arrives. The associated data is stored in local
`storage. Next the associated data is accessed by retrieving it
`from local memory.
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`The preferred embodiments of the invention Will noW be
`described in connection With the draWing Wherein:
`FIG. 1 is a block diagram of the invention at the highest
`level of abstraction.
`FIG. 2 is a block diagram shoWing further detail of the
`entire system according to the present invention.
`FIG. 3 is a block diagram shoWing additional detail of the
`system located With the consumer.
`FIG. 4 is a block diagram of a second preferred embodi
`ment of the invention that includes interactive communica
`tions.
`FIG. 5 is a diagram shoWing additional detail of the
`system located With the consumer When interactive commu
`nications are included.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`
`An electronic mass media provider may be said to deliver
`a one-Way stream of electronic data. This stream of data is
`herein referred to as a primary stream of data and includes
`live or pre-recorded information that is created by the
`broadcasting and entertainment industries for the purpose of
`delivery to consumers over delivery mediums such as broad
`cast television, video cassettes, radio and audio CD’s. The
`content of the primary data stream is typically created by
`production studios for mass distribution to the consumer
`market. Often a distributor produces all or part of the
`programming content. For example, a local television neWs
`station obtains pre-produced video neWs and advertising
`content from national providers and inserts locally produced
`content and advertising for broadcast. Equipment used in the
`television industry are video cameras and video recorders. In
`radio, it is microphones and audio recorders. Typically, the
`primary data is organiZed into programs. A primary stream
`of data can be rendered intelligent to a consumer as either
`audio or video or a combination of the tWo.
`The term associated data as used herein refers to a stream
`of data generated separately from the primary data but
`having content that is relevant to the primary data in general
`and usually relevant to a particular program of primary data
`and is in this sense associated. Associated data is intended to
`enhance the utility of the primary data stream, but if it is not
`available, the primary data stream can stand on its oWn
`merits. The creation process typically uses commercially
`available softWare and hardWare that output industry stan
`dard ?le formats such as Hypertext Markup Language
`(HTML) for text and graphic layout, Graphics Interchange
`Format (GIF) Joint Photographic Experts Group (JPEG)
`formats for still images, and so forth.
`
`4
`FIG. 1 is a block diagram of the invention at the highest
`level of abstraction. Referring noW to FIG. 1, data stream
`creation unit 1 performs the function of generating both the
`primary data stream and the associated data stream. The tWo
`data streams may or may not be merged to create a combined
`stream of primary and associated data. The tWo data streams
`are supplied to non-interactive delivery unit 2 Which per
`forms the function of delivering the tWo data streams to the
`consumer. The non-interactive delivery unit includes deliv
`ery by broadcast, cable or a packaged media such as
`cassettes and audio CD’s. Indeed, non-interactive delivery is
`intended to include all one-Way electronic data delivery
`systems. The primary data stream and secondary data stream
`are typically transmitted to a consumer over the same
`delivery medium; hoWever, the invention contemplates situ
`ations Where this is not the case. For example, the primary
`data stream could be delivered by broadcast television and
`the associated data could be delivered over a high speed
`digital netWork, a FM sideband, a direct satellite broadcast,
`a cable netWork, a telephone, etc.
`When the delivered data reaches receiver 3, it is decoded
`from its delivery medium. That is, if it is sent by broadcast
`television, the receiver decodes the signal, separates the
`primary data from the associated data and passes the asso
`ciated data on to processor 4. Processor 4 may further
`decode the associated data and in any event supplies the
`associated data to memory 5 Where it is stored for future use.
`The consumer of the primary data stream may interact With
`the associated data via user interface 6 such as a keyboard,
`mouse or voice activation electronics through Which the
`consumer gives commands to processor 4 and Which in turn
`causes the associated data to be accessed and processed. The
`resulting data is then supplied to display 7.
`FIG. 2 provides additional system detail. Referring noW to
`FIG. 2, reference numeral 10 refers to the primary data
`stream generating sub-system. The output of primary data
`stream generation sub-system 10 is typically an analog
`signal. HoWever, it may be a digital signal as in the case of
`digital broadcast television. Data channel 12 connects pri
`mary data stream generation sub-system 10 and sequencer
`14 and data channel 19 connects it to data synchroniZer 20.
`Data channels 12 and 19 may be initiated by a live video
`feed, or analog or digital tape. Reference numeral 16 refers
`to an associated data generation sub-system. The output of
`associated data generation sub-module 16 is a digital signal
`that is supplied over digital data link 18 to sequencer 14 and
`over digital data link 17 to synchroniZer 20. The digital data
`links of the system may be a conventional digital connection
`such as a serial or parallel or it may be a netWork link.
`Typical connecting media Would be tWisted pair, co-axial
`cable, ?ber optic cable or a Wireless media.
`Reference numeral 20 refers to a data synchroniZing
`sub-system Whose function is to synchroniZe the primary
`data stream generated by sub-system 10 With speci?c asso
`ciated data. The input to data synchroniZing sub-system 20
`is scene information from the primary data stream in the
`form of timecodes and time durations, and data from asso
`ciated data generator sub-system 16. It creates a so called
`script for the delivery and display of associated data at
`speci?c points in time. For example, data synchroniZer 20
`creates a script that speci?es that a detailed data sheet Will
`be delivered to the consumer prior to a speci?c television
`product advertisement, and that the data sheet Will be
`displayed on the consumer’s personal computer display
`When a certain television advertisement starts.
`Data synchroniZer 20 typically includes softWare Which is
`patterned after common digital video editors such as Ado
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`APPLE 1003 - Page 8
`
`

`
`5,818,441
`
`5
`be’s Premiere, Which places parallel video and audio tracks
`on a standard time line such as SMPTE (Society of Motion
`Picture and Television Engineers) time code. This alloWs
`segments of each track to be manipulated independently but
`With reference to a common time line so that segments may
`later be reconstituted. The softWare of data synchroniZer 20
`alloWs associated data to be laid out and manipulated on an
`additional data track. The siZe of particular associated data
`components is coordinated With the knoWn bandWidth of the
`delivery medium. The user of data synchroniZer 20 typically
`speci?es that a unit of information should be displayed at a
`speci?c point in time. Data synchroniZer 20 then calculates
`the time required to transfer the data and inserts a transfer
`speci?cation into the script at the appropriate point prior to
`the display instruction. The output of data synchroniZer
`sub-system 20 is a digital signal representing a script that
`synchroniZes the output of primary data stream sub-system
`10 and associated data stream generator 16.
`The output of data synchroniZer 20 is supplied over digital
`data link 22 to sequencer sub-system 14. Digital data link 22
`is a conventional digital link. Sequencer 14 combines the
`primary data stream With the associated data as speci?ed by
`the script supplied by synchroniZer sub-system 20.
`Sequencer 14 monitors the output from primary data stream
`sub-system 10 to obtain the information necessary to
`sequence transmission of associated data. For example,
`during the broadcast of a television program, data sequencer
`14 obtains a time code information such as SMPTE time
`code Which is generated by the primary data stream. The
`information is used to coordinate mixing of associated data
`With primary data streams. In addition, sequencer 14 mixes
`other data feeds that are being transmitted over the same
`delivery medium. These may include external data services
`such as ?nancial data, emergency broadcast information or
`Weather information. Mixing of existing data streams is
`necessary only for delivery mediums that don’t support
`separate, independent transmission of digital information.
`This function has the capacity to be used for the delivery of
`either live or pre-recorded programming, or a combination
`of both. For live programming, such as a local evening neWs
`program, the data stream sequencer takes input directly from
`the outputs of primary data stream generator 10, data syn
`chroniZer 20, associated data generator 16, and time code
`from the primary data stream. For pre-recorded
`programming, this function processes the output of the data
`synchroniZer 20 and sequences the broadcast of associated
`data With the primary data.
`The output of sequencer 14 is supplied over conventional
`digital data link 24 to encoder 26. Encoder 26 is a hardWare
`component for insertion of digital information into the
`particular delivery medium that Will be received by the
`consumer. If the delivery medium is broadcast television,
`encoder 26 inserts this information directly into the NTSC
`television signal, and thus delivers both the primary and
`associated data over a single delivery medium, broadcast
`television radio Waves. Encoder 26 accepts input streams of
`both video and serialiZed digital information. It breaks the
`serialiZed digital information up into a series of packets, and
`modulates the packets into the (“VBI”) of the video signal.
`Encoder 26 is commercially available hardWare and soft
`Ware. A number of vendors market hardWare for insertion
`and extraction of data into and from standard (NTSC/PAL)
`television signals. Norpak Corporation, EEG, Inc, and
`WavePhore, Inc. are examples.
`The output of encoder 26 is supplied via digital data link
`28 to transmitter sub-system 30 Which performs the function
`of physically transmitting radio frequency Waves into the
`
`10
`
`15
`
`25
`
`35
`
`45
`
`55
`
`65
`
`6
`atmosphere. Such transmitters may be television or radio
`broadcast transmitters or a satellite broadcast system. Also,
`the data may be stored on a tape for later transmission.
`Reference numeral 34 generally indicates the equipment
`that must be present at the consumer’s location. In general,
`system 34 is a computer. That is, it has digital computing
`capability as Well as equipment to receive the primary and
`associated data streams. Receiver 36 could be a personal
`computer add-in adapter board, a television or radio broad
`cast receiver, a cable television converter box or it could be
`a satellite receiver for digital broadcast. In the preferred
`embodiment, it is a broadcast television receiver or tuner
`inside of a personal computer. Connected to receiver 36 is a
`microprocessor 38. Connected to microprocessor 38 is
`memory 40 Which Would typically be semiconductor RAM.
`Additional magnetic media such as a hard disk Would be
`desirable. Microprocessor 38 is connected to human inter
`face 42, Which is typically a CRT monitor, and to printer 44.
`Human interface 42 and receiver 36 could be part of a
`standard television.
`Reference numeral 32 refers to the delivery media Which
`in this case are radio frequency electromagnetic Waves
`passing through the atmosphere. If the delivery media is a
`video tape, laser disk or audio CD, then, transmitter 30
`becomes a video tape recorder, video or audio CD recorder,
`and receiver 36 becomes a VCR, laser disk player or an
`audio CD player respectively.
`FIG. 3 is a high level block diagram shoWing further
`details of system 34 of FIG. 2. Referring noW to FIG. 3,
`receiver 36 receives radio frequency Waves from input 50.
`Receiver 36 demodulates the input signal and supplies the
`primary data stream signal over data path 52 to primary data
`rendering sub-system 54 and the associated data signal over
`data path 56 to associated data decoder 58.
`Primary data rendering sub-system 54 performs the func
`tion of presenting the primary data stream to the consumer
`in the manner in Which a typical consumer Would expect to
`see the data presented. For example, in the case of television,
`the primary data rendering takes the form of a video image
`typically supplied by a cathode ray tube screen, or possibly
`a liquid crystal display screen and audio provided by an
`audio ampli?er and speakers. In the preferred embodiment
`these components are in a personal computer. A second
`example is a broadcast radio demodulator, ampli?er and
`speakers that renders radio Waves received via radio broad
`casting audible to a consumer. Yet a third example is a stereo
`system that renders data encoded on compact disk or tape
`audible to the consumer.
`Decoder 58 performs the function of decoding the asso
`ciated data from the delivery media. In this case decoder 58
`consists of an analog to digital converter that converts
`analog encoded digital data back to digital format.
`Associated data protocol manager 60 is connected by
`one-Way data path 62 to decoder 58 and by one Way data
`path 64 to communications manager 66. Associated data
`protocol manager 60 performs the function of extracting the
`different forms of associated data from the incoming digital
`data steam and converting them to a form that can be used
`by communications manager 66. The types of associated
`data protocols include World Wide Web pages, closed
`captioning, stock quotes, sports scores, control commands
`for microprocessor 38 (of FIG. 2) to execute.
`Communications manager 66 performs the function of a
`common netWork interface by receiving data from several
`different types of communication devices using different
`data transmission protocols. Such devices include telephone
`
`APPLE 1003 - Page 9
`
`

`
`10
`
`15
`
`25
`
`35
`
`7
`modems, ISDN modems, cable modems, Wireless modems,
`satellite modems, broadcast TV, radio and the like. Com
`munication manager 66 converts all data received, regard
`less of the source and protocol, into a standard format that
`can then be utiliZed by the rest of the system. This function
`may be implemented in one of tWo Ways. Proprietary
`interfaces betWeen the communication components may be
`designed, or standard industry interfaces such as Microsoft
`WindoWs’ (a trademark of Microsoft Corporation) sockets
`may be used. The preferred embodiment uses WindoWs
`sockets. A WindoWs socket is a standard application inter
`face to access netWork data.
`Real time trigger 76 is connected to communications
`manager 66 by one-Way data path 78 and to human interface
`88 by data path 87. Human interface 88 provides the
`consumer With input and output to the system. In the
`preferred embodiment, human interface 88 uses the key
`board and alternate input devices such as a mouse of a
`personal computer as input for requests and the display of
`the personal computer for displaying the data.
`Real time trigger 76 accepts commands sent as part of the
`associated data to display a page of information Without the
`user asking for it. The output of realtime trigger is a
`command sent over data path 87 to human interface 88
`Which causes the page of information to be displayed. For
`example, a broadcaster may Want vieWers to see a certain
`page of information as part of a program that is being
`vieWed. Real time trigger alloWs data to be displayed using
`the same data protocols as consumer requested information.
`Local data storage 80 is connected to communications
`manager 66 by one-Way data path 82 and by data path 83 to
`local data manager 84. The hardWare implementation for
`local data storage 80 may be one or more of the folloWing;
`RAM, disk, tape, recordable CD-ROM.
`Local data manager 84 is connected to communications
`manager 66 by data path 86 and to human interface 88 by
`tWo-Way data path 90. Local data manager 84 receives
`commands from human interface 88 to retrieve associated
`data from local data storage 80 and sends that data to human
`interface 88 for presentation to the consumer. For example,
`a “Web BroWser” may be used to display data pages from the
`World Wide Web (the “WWW”). Providers of WW
`broWsers include Netscape Communications Corp., America
`Online, Spyglass and others. Local data storage 80 Will
`typically be of limited capacity. Thus, local data manager 84
`purges older and less used information. This is accomplished
`by assigning an expiration date and/or a priority to associ
`ated data ?les. The criteria for determining Which data to
`purge is settable by the broadcaster and/or the consumer.
`Criteria include total amount of storage available, siZe of
`associated data ?les, expiration date and priority.
`The preferred embodiment illustrated in FIGS. 2 and 3
`operates in the folloWing manner. A primary data stream is
`generated. Associated data is separately generated. A script
`is generated that synchroniZes hoW the primary and associ
`ated data are linked together. The primary data, the associ
`ated data and the script are sent to sequencer 14 Where they
`are combined. The primary data is transmitted in the con
`ventional fashion of its delivery medium. In the television
`model, this is broadcast over the air or on cable. The
`associated data is generally (but not alWays) sent by the
`same delivery medium as the primary data. In the case of
`television, the associated data is encoded in the VBI of the
`television signal. The signal is received by the consumer’s
`equipment. The primary data stream is immediately ren
`dered and the associated data is stored in local data storage
`
`45
`
`55
`
`65
`
`5,818,441
`
`8
`80. At any time, the consumer may broWse the data stored
`in local data storage. This data Will have been professionally
`selected to provide an enhanced vieWing experience. For
`example, difficult to ?nd data sources providing historical
`background to a program Will have been previously
`researched by the associated data provider and sent as
`associated data. The human interface is user friendly and
`Will alloW the consumer to broWse through the associated
`data. This might start With a menu of available information
`from Which the consumer could select just as if the data Were
`coming from an online service. But the data is actually
`stored locally. And it arrived in local storage by being
`transmitted With the primary data stream. The consumer may
`process the associated data in a variety of Ways including
`sorting and indexing relevant information.
`FIG. 4 is a block diagram of a second preferred embodi
`ment of the invention that includes interactive communica
`tions. Reference numerals common to FIGS. 2 and 4 refer
`ence the same matter. Referring noW to FIG. 4, a tWo-Way
`communication channel 46 is connected to microprocessor
`38 and provides interactive access to remote computers over
`such media as the analog telephone netWork, the ISDN
`digital netWork, a Wide area packet sWitched netWork such
`as X25, frame relay or asynchronous transfer mode.
`FIG. 5 is a high level block diagram shoWing further
`details of the system of FIG. 4. Reference numerals common
`to FIGS. 2 and 4 reference the same matter. Referring noW
`to FIG. 5, remote data manager 92 is connected to human
`interface 88 by data path 94 and to communications manager
`by tWo-Way data path 96. Remote data manager 92 receives
`commands from human interface 88 to retrieve data from
`remote computers through tWo-Way communications chan
`nel 74 and to send that data to human interface 88 for
`presentation to the consumer.
`NetWork protocol manager 68 is connected by tWo-Way
`data path 70 to communications manager 66 and by tWo-Way
`data path 72 to tWo-Way communication channel 74. Net
`Work protocol manager 68 performs the function of format
`ting data received and transmitted over tWo-Way communi
`cation channel 74. It contains the various protocols required
`to communicate With remote computers. Again, operating
`systems such as Microsoft’s WindoWs typically provide
`netWork protocol managers With plug in modules called
`drivers to support different protocols and different commu
`nication hardWare. NetWork protocol manager 68 extracts
`incoming data from tWo-Way communication channel 74
`using a netWor