`[19]
`[11] Patent Number:
`5,818,441
`
`Throckmorton ct al.
`[45] Date of Patent:
`Oct. 6, 1998
`
`USOOSSIS441A
`
`[54] SYSTEM AND METHOD FOR SIMULATING
`TWO-WAY CONNECTIVITY FOR ONE WAY
`DATA STREAMS
`
`[75]
`
`Inventors: John A. Throckmorton, Lake Oswego;
`Edward R. Harrison; Burt Perry,
`both of Beaverton, all of Greg.
`
`[73] Assignee:
`
`Intel Corporation, Santa Clara, Calil‘.
`
`[31] Appl- No.: 490,322
`
`Jun. 15, 1995
`
`Filed:
`[22]
`Int. Cl?
`[51]
`.
`v
`152] U'b' L"
`[58] Field of Search
`
`[56]
`
`HIMN W10
`_
`,
`,
`345528’ 3435;13‘33317;
`3453337 3.28“-
`34337, 12’ 13’ 473’ 714
`References Cited
`U.S. PATENT DOCUMENTS
`911994 Moura eta].
`5341304
`£1990 Schiller eta].
`5,499,040
`4t1996 Miyake et a].
`5,513,130
`5533334 1211996 Lightfoot et a1..
`5,594,491
`1,1199? Hodge
`
`348I’12
`348I’12
`.. 343-“?
`
`.. 343-“?
`348:“?
`
`5,613,192
`
`3.31997
`
`[kanii et a].
`
`348i?)
`
`Primary Examiner—Victor R. Kostak
`Attorney; Agent, or Firm—Blakely, Sokoloff, Taylor &
`Zafman
`[57]
`
`ABSTRACT
`
`Asystem supplying information associated with a broadcast
`television program to a consumer such that said consumer
`perceives the associated data as the result of two way
`interactivity with external sources of data such as online
`services or the Internet. The system includes equipment for
`inserting the associated data into the vertical blanking inter-
`gal o; the televisiondsigngl on the supplliecf side of the system.
`n t e receiving s1 e, t e system me u es a persona com-
`puter capable of receiving the television program and storing
`the associated data locally. The consumer may then interact
`with the stored associated data in an apparently two way
`interactive manner. Additional interactivity may be achieved
`by addlng an actual two way communlcatlon channel to the
`personal computer so that online serv1ces or the Internet may
`be accessed. This two way communication channel is made
`particularly efl‘ective if the associated data contains pointers
`to locations in the online services or the [ntemet which are
`particularly relevant to the television program.
`
`42 Claims, 5 Drawing Sheets
`
`Primary Data
`Stream
`Generation
`
`
`
`Data .
`Synchronize-r
`
`'2
`
`14
`
`26
`
`30
`
`32
`
`Sequencer
`
`Encoder
`
`Transmitter
`
`24
`
`28
`
`Associated
`Data.
`Generation
`
`
`
`
`
`
`
`18
`
`16
`
`
`
`l
`
`I
`
`I| I I
`
`34/?
`I
`
`l_—___—_____—____——————————_
`I
`36
`|
`i
`
`Receiver
`
`38
`
`Processor
`
`
`
`Printer
`
`44
`
`Memory
`
`4a
`
`NTFX—1007 I Page 1 of 12
`
`
`
`U
`
`tnetaP
`
`8w
`
`m
`
`5
`
`5,818,441
`
`QMH.65
`
`mN_
`
`
`
`mSimoumbegun22320m.EwobmNEDEnobmfian—
`
`
`
`
`
`
`
`.m.Dog—BEm5&8on.583qu
`
`NTFX—1007 I Page 2 of 12
`
`
`
`
`US. Patent
`
`5,818,441
`
`In”,.“2:52.“;533mBocoscomHofifimfigmmom.
`mmon3
`
`
`
`
`
`
`
`
`
`
`
`IIIIIIIIIIIIIIIIIIIIIIIIIIL$23834mmmE
`
`SET—n—________Smxuooi"mn_o_mm_m_5380m—oofiEBE_0#w_Swazi_%U_om_M:5:90:00__San—
`1_n9“in___l\..Vm_@0502
`
`NTFX—1007 I Page 3 of 12
`
`N.0;
`
`NHNEDinfirm
`
`5:80:00
`
`:3wa
`
`
`
`
`US. Patent
`
`a0
`
`8
`
`mm
`
`m.6:
`
`Swazi
`
`oomfiflfla
`
`6”,9.3:80;Sun.:80]—oECHEumwmEarnin—
`
`
`
`
`
`
`
`mOwomESmSwag—2Bmmtkwzwmmsm
`
`H
`
`w.Swag:8mmmmGOH—MQMESEEOUh..Smm
`
`m»
`
`3
`
`Sun—3,3082%
`
`moowOm
`
`010Emacs—2
`
`l
`
`Mon89mm
`
`
`
`
`
`Rm,385E.368ngSimona
`
`NTFX—1007 I Page 4 of 12
`
`
`
`
`
`US. Patent
`
`m
`
`l
`
`_
`
`
`
`EEOUMIIIIIIIIIIIIIIIIIIIIIIIIII._9_8_Eu_1_on,o"3:530
`
`EEEn—_5ask/63%._Wm"Ix
`_cwNw_
`
`5__f_o_4__a_
`
`Ommmem3
`
`1,mmEa”%
`
`v.UHm
`
`Ewmbm9«EDinc—cm
`
`5:80:00
`
`
`
`
`
`_h25:90:00_ommmL3NDIIIIliliallllliliilillilll@3208?“
`
`
`
`._
`
`_
`
`NTFX—1007 I Page 5 of 12
`
`
`
`
`
`US. Patent
`
`Oct. 6, 1998
`
`Sheet 5 of 5
`
`5,818,441
`
`we
`
`E
`
`mm
`
`
`
`09%ch53.5:
`
`m.0;
`
`3mm—BoEom
`
`Swag—2
`
`3mm:33Sam:33DEE—momwnmeza
`
`
`
`
`
`w«@8onSwag:Swath:wwmflom
`
`
`
` Homngmaoumofisafiou8mm
`
`@3639
`
`:czmoEsEEcU
`
`355:0
`
`
`
`Human—«EEmacs—2
`
`
`
`6030.5“3090.5Hug—”.0000Loioovm
`€95qufian—voufloomwxwwonon
`
`
`mmom
`
`NTFX—1007 I Page 6 of 12
`
`
`
`
`
`
`
`5,818,441
`
`1
`SYSTEM AND METHOD FOR SIMULATING
`TWO-WAY CONNECTIVITY FOR ONE WAY
`DATA STREAMS
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`This invention relates to enhancing one—way broadcast
`data transmissions. More particularly, it relates to enhancing
`the quality and content of a primary information stream by
`creating and transmitting associated data which provides the
`appearance of an interactive connection to secondary
`sources of information.
`
`2. Background
`One of the limitations of nearly all forms of electronic
`mass media such as radio, television, audio CD’s and video
`cassettes is that the communication of information or data is
`one way from the provider to the consumer. The character-
`istics of one way sources of data are that the data is sent
`sequentially and is fleeting or at best can be recorded for
`later playback. And a consumer cannot interact with the data
`provided to seek additional information or services.
`There is a desire by electronic mass media consumers for
`additional services, and providers of television and radio
`broadcast services are looking for new sources of revenue.
`In particular, consumers are looking for information or data
`that is related to what they are viewing in the media. The
`additional information may be in the form of details on the
`content of the current program such as the recipe of a men]
`being demonstrated on a cooking show or biographies of
`actors in a drama or historical background information on
`events depicted in a program. Or,
`it could be program
`highlights such as key plays from the baseball game. In
`addition, consumers would like access to real time data such
`as stock price, updated baseball scores as they occur, traflic
`and weather conditions. Also, consumers would like to have
`access to special services associated with product advertis—
`ing such as information on where to buy nationally adver-
`tised products—possibly with a map to the nearest location,
`or the ability to instantly receive coupons.
`Recently, online services such as provided by the World
`Wide Web of the Internet have become available. Such
`services provide access to immense amounts of data on an
`interactive basis by linking digital computers together over
`sophisticated communications networks. At the same time,
`the cost of digital computing power is falling rapidly. For
`example, many homes now have one or more computers.
`And home computers have displays capable of showing
`television as well as audio capability. They also have
`memory and computing power.
`But while such services are generally available and com-
`puters are becoming more widespread, finding relevant
`information is an arduous task that may take hours of
`searching even for an experienced user.
`In addition, these services require a 2—way connection
`from a consumer’s location to the online network. In the
`consumer market, prolonged utilization of a single phone
`line for 2-way digital communications is expensive.
`Up until now, there has been no way for producers of mass
`market broadcast programming to deliver data associated by
`its relevancy to its subject matter that could be interactively
`displayed and manipulated by consumers on a real
`time
`basis. What is meant by real
`time is that
`the consumer
`receives and has access to the relevant data during the
`process of program reception. Therefore the data becomes
`an integral part of the experience desired by the program
`producers.
`
`10
`
`15
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`Although received during a program, the information is
`retained and may be traversed by the consumer at a later
`time, as if they were connected in a 2-way fashion to an
`online service
`3. Prior Art
`
`
`
`Perhaps the earliest efl‘ort to address the foregoing needs
`is for program content providers to include a telephone
`number in either the radio or television broadcast. The
`consumer is invited to dial a number for additional infor-
`mation. However,
`this telephone number is part of the
`primary data stream which is transitory.
`A second attempt
`is the provision of closed caption
`programming for the hearing impaired on television. This
`takes the form of written text appearing somewhere on the
`television screen typically at
`the bottom and requires a
`special decoder or mode set on the television. The text is a
`written rendition of the audio portion of the television
`program. That is,
`it
`is a recapitulation of the information
`supplied by the primary data provider and is not stored or
`saved for later access by he consumer.
`Television networks also use a part of the unused band-
`width in the video signal .0 send the schedule of programs
`coming up on that network. One network is sending a
`comprehensive program isting in digital format over an
`unused section of the NTSC bandwidth. This is called an
`electronic program guide ("EPG"). Asecond network uses a
`aortion of the unused N'l‘SC bandwidth to send digitally
`encoded stock quotes ant the subject matter of the topics
`covered in the news broadcast, and extended data services
`(“XDS”) sends the date, time, name of a scheduled program,
`ype of program and how much of the program remains.
`Cable services provide at: ditional detail about songs being
`Jlayed on subscriber digital audio services such as digital
`nusic express ("DMX{tm)"); The detail typically consists of
`the name of the artist, the name of the song and the album.
`A system called Gemstar(tm), provides information in digi—
`.al format that enables consumers to record programs by
`referencing a number in a program guide.
`Many of the foregoing services are provided on broadcast
`elevision only over what is called the vertical blanking
`interval (the "VBI"). The VBI is a portion of the bandwidth
`defined for broadcast television and in the case of NTSC, for
`example, consists of the first 21 of the 525 raster lines that
`define a video frame. SECAM and PAL have a similar
`arrangement. Of the 21 lines, line 21 is divided into two
`frames of 60 hz bandwidth each. The closed caption is
`provided in the first frame of line 21.
`None of the foregoing allow a consumer to experience
`apparent interactivity with external data sources. That is,
`none provides data that persists for more than a few seconds
`and none allows the consumer to manipulate directly that
`data.
`
`
`
`SUMMARY OF THE INVENTION
`
`The foregoing problems may be resolved by a system for
`supplying a primary data stream and associated data to a
`consumer such that the consumer perceives the associated
`data as the result of interactivity with external sources of
`information. This system includes a creation unit for creat-
`ing the primary data stream and the associated data. The
`primary data stream and associated data are supplied to a one
`way delivery unit for delivering the primary data stream and
`the associated data to the consumer. At
`the consumer
`location, the system includes a receiver for receiving the
`primary data stream and a receiver for receiving the asso-
`ciated data. These may be the same unit. The system further
`
`NTFX—1007 I Page 7 of 12
`
`
`
`5,818,441
`
`3
`includes computer memory storing the received associated
`data. A processor communicates with the memory for
`accessing the associated data. Finally, the system includes a
`user interface connected to the processor through which the
`consumer may interact with the system.
`The method of the present invention includes the steps of
`first generating a stream of primary data and of associated
`data. Next, both the primary data stream and associated data
`are delivered to the consumer. Next, both the primary data
`stream and the associated data are received at the location of
`the consumer. The primary data stream is rendered to the
`consumer as it arrives. The associated data is stored in local
`storage. Next the associated data is accessed by retrieving it
`from local memory.
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`The preferred embodiments of the invention will now be
`described in connection with the drawing wherein:
`FIG. 1 is a block diagram of the invention at the highest
`level of abstraction.
`
`FIG. 2 is a block diagram showing further detail of the
`entire system according to the present invention.
`FIG. 3 is a block diagram showing additional detail of the
`system located with the consumer.
`FIG. 4 is a block diagram of a second preferred embodi-
`ment of the invention that includes interactive communica-
`tions.
`
`FIG. 5 is a diagram showing additional detail of the
`system located with the consumer when interactive commu-
`nications are included.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`
`An electronic mass media provider may be said to deliver
`a one-way stream of electronic data. This stream of data is
`herein referred to as a primary stream of data and includes
`live or pre-recorded information that
`is created by the
`broadcasting and entertainment industries for the purpose of
`delivery to consumers over delivery mediums such as broad-
`cast television, video cassettes, radio and audio CD’S. The
`content of the primary data stream is typically created by
`production studios for mass distribution to the consumer
`market. Often a distributor produces all or part of the
`programming content. For example, a local television news
`station obtains pre-produced video news and advertising
`content from national providers and inserts locally produced
`content and advertising for broadcast. Equipment used in the
`television industry are video cameras and video recorders. In
`radio, it is microphones and audio recorders. Typically, the
`primary data is organized into programs. A primary stream
`of data can be rendered intelligent to a consumer as either
`audio or video or a combination of the two.
`The term associated data as used herein refers to a stream
`of data generated separately from the primary data but
`having content that is relevant to the primary data in general
`and usually relevant to a particular program of primary data
`and is in this sense associated. Associated data is intended to
`enhance the utility of the primary data stream, but if it is not
`available,
`the primary data stream can stand on its own
`merits. The creation process typically uses commercially
`available software and hardware that output industry stan-
`dard file formats such as Hypertext Markilp Language
`(HTML) for text and graphic layout, Graphics Interchange
`Format [GIF) Joint Photographic Experts Group (JPEG)
`formats for still images, and so forth.
`
`10
`
`15
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`FIG. 1 is a block diagram of the invention at the highest
`level of abstraction. Referring now to FIG. 1, data stream
`creation unit 1 performs the function of generating both the
`primary data stream and the associated data stream. The two
`data streams may or may not be merged to create a combined
`stream of primary and associated data. The two data streams
`are supplied to non-interactive delivery unit 2 which per-
`forms the function of delivering the two data streams to the
`consumer. The non-interactive delivery unit includes deliv-
`ery by broadcast, cable or a packaged media such as
`cassettes and audio CD’s. Indeed, non-interactive delivery is
`intended to include all one-way electronic data delivery
`systems. The primary data stream and secondary data stream
`are typically transmitted to a consumer over
`the same
`delivery medium; however, the invention contemplates situ-
`ations where this is not the case. For example, the primary
`data stream could be delivered by broadcast television and
`the associated data could be delivered over a high speed
`digital network, a FM sideband, a direct satellite broadcast,
`a cable network, a telephone, etc.
`When the delivered data reaches receiver 3, it is decoded
`from its delivery medium. That is, ifit is sent by broadcast
`television,
`the receiver decodes the signal, separates the
`primary data from the associated data and passes the asso-
`ciated data on to processor 4. Processor 4 may further
`decode the associated data and in any event supplies the
`associated data to memory 5 where it is stored for future use.
`The consumer of the primary data stream may interact with
`the associated data via user interface 6 such as a keyboard,
`mouse or voice activation electronics through which the
`consumer gives commands to processor 4 and which in turn
`causes the associated data to be accessed and processed. The
`resulting data is then supplied to display 7.
`FIG. 2 provides additional system detail. Referring now to
`FIG. 2, reference numeral 10 refers to the primary data
`stream generating sub—system. The output of primary data
`stream generation sub-system 10 is typically an analog
`signal. However, it may be a digital signal as in the case of
`digital broadcast television. Data channel 12 connects pri-
`mary data stream generation sub-system 10 and sequencer
`l4 and data channel 19 connects it to data synchronizer 20.
`Data channels 12 and 19 may be initiated by a live video
`feed, or analog or digital tape. Reference numeral 16 refers
`to an associated data generation sub-system. The output of
`associated data generation sub-module 16 is a digital signal
`that is supplied over digital data link 18 to sequencer 14 and
`over digital data link 17 to synchronizer 20. The digital data
`links of the system may be a conventional digital connection
`such as a serial or parallel or it may be a network link.
`Typical connecting media would be twisted pair, eo-axial
`cable, fiber optic cable or a wireless media.
`Reference numeral 2!) refers to a data synchronizing
`sub-system whose function is to synchronize the primary
`data stream generated by sub-system II] with specific asso-
`ciated data. The input to data synchronizing sub—system 20
`is scene information from the primary data stream in the
`form of timecodes and time durations, and data from asso-
`ciated data generator sub-system 16. It creates a so called
`script for the delivery and display of associated data at
`specific points in time. For example, data synchronizer 20
`creates a script that specifies that a detailed data sheet will
`be delivered to the consumer prior to a specific television
`product advertisement, and that
`the data sheet will be
`displayed on the consumer’s personal computer display
`when a certain television advertisement starts.
`
`Data synchronizer 20 typically includes software which is
`patterned after common digital video editors such as Ado-
`
`NTFX—1007 I Page 8 of 12
`
`
`
`5,818,441
`
`5
`be’s Premiere, which places parallel video and audio tracks
`on a standard time line such as SMPTE (Society of Motion
`Picture and Television Engineers) time code. This allows
`segments of each track to be manipulated independently but
`with reference to a common time line so that segments may
`later be reconstituted. The software of data synchronizer 20
`allows associated data to be laid out and manipulated on an
`additional data track. The size of particular associated data
`components is coordinated with the known bandwidth of the
`delivery medium. The user of data synchronizer 20 typically
`specifies that a unit of information should be displayed at a
`specific point in time. Data synchronizer 20 then calculates
`the time required to transfer the data and inserts a transfer
`specification into the script at the appropriate point prior to
`the display instruction. The output of data synchronizer
`sub—system 20 is a digital signal representing a script that
`synchronizes the output of primary data stream sub-system
`10 and associated data stream generator 16.
`The output of data synchronizer 20 is supplied over digital
`data link 22 to sequencer sub-system 14. Digital data link 22
`is a conventional digital link. Sequencer 14 combines the
`primary data stream with the associated data as specified by
`the script supplied by synchronizer sub-system 20.
`Sequencer 14 monitors the output from primary data stream
`sub-system 10 to obtain the information necessary to -
`sequence transmission of associated data. For example,
`during the broadcast of a television program, data sequencer
`14 obtains a time code information such as SMPTE time
`
`10
`
`15
`
`code which is generated by the primary data stream. The
`information is used to coordinate tnixing of associated data
`with primary data streams. In addition, sequencer l4 mixes
`other data feeds that are being transmitted over the same
`delivery medium. These may include external data services
`such as financial data, emergency broadcast information or
`weather information. Mixing of existing data streams is
`necessary only for delivery mediums that don’t support
`separate, independent transmission of digital information.
`This function has the capacity to be used for the delivery of
`either live or pre-recorded programming, or a combination
`of both. For live programming, such as a local evening news
`program, the data stream sequencer takes input directly from
`the outputs of primary data stream generator 10, data syn-
`chronizer 20, associated data generator 16, and time code
`from the primary data stream. For pre-recorded
`programming, this function processes the output of the data
`synchronizer 20 and sequences the broadcast of associated
`data with the primary data.
`The output of sequencer 14 is supplied over conventional
`digital data link 24 to encoder 26. Encoder 26 is a hardware
`component
`for insertion of digital
`information into the
`particular delivery medium that will be received by the
`consumer. If the delivery medium is broadcast television,
`encoder 26 inserts this information directly into the NTSC
`television signal, and thus delivers both the primary and
`associated data over a single delivery medium, broadcast
`television radio waves. Encoder 26 accepts input streams of
`both video and serialized digital information. It breaks the
`serialized digital information up into a series of packets, and
`modulates the packets into the (“VBI”) of the video signal.
`Encoder 26 is commercially available hardware and soft-
`ware. A number of vendors market hardware for insertion
`
`and extraction of data into and from standard (NTSCEPAL)
`television signals. Norpak Corporation, EEG,
`Inc, and
`WavePhore, Inc. are examples.
`The output of encoder 26 is supplied via digital data link
`28 to transmitter sub-system 30 which performs the function
`of physically transmitting radio frequency waves into the
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`atmosphere. Such transmitters may be television or radio
`broadcast transmitters or a satellite broadcast system. Also,
`the data may be stored on a tape for later transmission.
`Reference numeral 34 generally indicates the equipment
`that must be present at the consumer’s location. In general,
`system 34 is a computer. That is, it has digital computing
`capability as well as equipment to receive the primary and
`associated data streams. Receiver 36 could be a personal
`computer add-in adapter board, a television or radio broad-
`cast receiver, a cable television converter box or it could be
`a satellite receiver for digital broadcast. In the preferred
`embodiment,
`it is a broadcast television receiver or tuner
`inside of a personal computer. Connected to receiver 36 is a
`microprocessor 38. Connected to microprocessor 38 is
`memory 40 which would typically be semiconductor RAM.
`Additional magnetic media such as a hard disk would be
`desirable. Microprocessor 38 is connected to human inter-
`face 42, which is typically a CRT monitor, and to printer 4-4.
`Human interface 42 and receiver 36 could be part of a
`standard television.
`
`Reference numeral 32 refers to the delivery media which
`in this case are radio frequency electromagnetic waves
`passing through the atmosphere. If the delivery media is a
`video tape,
`laser disk or audio CD,
`then, transmitter 30
`becomes a video tape recorder, video or audio CD recorder,
`and receiver 36 becomes a VCR, laser disk player or an
`audio CD player respectively.
`FIG. 3 is a high level block diagram showing further
`details of system 34 of FIG. 2. Referring now to FIG. 3,
`receiver 36 receives radio frequency waves from input 50.
`Receiver 36 demodulates the input signal and supplies the
`primary data stream signal over data path 52 to primary data
`rendering sub-system 54 and the associated data signal over
`data path 56 to associated data decoder 58.
`Primary data rendering sub-system 54 performs the func-
`tion of presenting the primary data stream to the consumer
`in the manner in which a typical consumer would expect to
`see the data presented. For example, in the case of television,
`the primary data rendering takes the fortn of a video image
`typically supplied by a cathode ray tube screen, or possibly
`a liquid crystal display screen and audio provided by an
`audio amplifier and speakers. In the preferred embodiment
`these components are in a personal computer. A second
`example is a broadcast radio demodulator, amplifier and
`speakers that renders radio waves received via radio broad-
`casting audible to a consumer. Yet a third example is a stereo
`system that renders data encoded on compact disk or tape
`audible to the consumer.
`
`Decoder 58 performs the function of decoding the asso-
`ciated data from the delivery media. In this case decoder 58
`consists of an analog to digital converter that converts
`analog encoded digital data back to digital format.
`Associated data protocol manager 60 is connected by
`one—way data path 62 to decoder 58 and by one way data
`path 64 to communications manager 66. Associated data
`protocol manager 60 performs the function of extracting the
`different forms of associated data from the incoming digital
`data steam and converting them to a form that can be used
`by communications manager 66. The types of associated
`data protocols include World Wide Web pages, closed
`captioning, stock quotes, sports scores, control commands
`for microprocessor 38 (of FIG. 2) to execute.
`Communications manager 66 performs the function of a
`common network interface by receiving data from several
`different types of communication devices using different
`data transmission protocols. Such devices include telephone
`
`NTFX—1007 I Page 9 of 12
`
`
`
`5,818,441
`
`7
`modems, ISDN modems, cable modems, wireless modems,
`satellite modems, broadcast TV, radio and the like. Com-
`munication manager 66 converts all data received, regard-
`less of the source and protocol, into a standard format that
`can then be utilized by the rest of the system. This function
`may be implemented in one of two ways. Proprietary
`interfaces between the communication components may be
`designed, or standard industry interfaces such as Microsoft
`Windows’ (a trademark of Microsoft Corporation) sockets
`may be used. The preferred embodiment uses Windows
`sockets. A Windows socket is a standard application inter-
`face to access network data.
`
`10
`
`8
`80. At any time, the consumer may browse the data stored
`in local data storage. This data will have been professionally
`selected to provide an enhanced viewing experience. For
`example, diflicult to find data sources providing historical
`background to a program will have been previously
`researched by the associated data provider and sent as
`associated data. The human interface is user friendly and
`will allow the consumer to browse through the associated
`data. This might start with a menu of available information
`from which the consumer could select just as if the data were
`coming from an online service. But
`the data is actually
`stored locally. And it arrived in local storage by being
`transmitted with the primary data stream. The consumer may
`process the associated data in a variety of ways including
`sorting and indexing relevant information.
`FIG. 4 is a block diagram of a second preferred embodi—
`ment of the invention that includes interactive communica-
`tions. Reference numerals common to FIGS. 2 and 4 refer-
`ence the same matter. Referring now to FIG. 4, a two-way
`communication channel 46 is connectcd to microprocessor
`38 and provides interactive access to remote computers over
`such media as the analog telephone network.
`the ISDN
`digital network, a wide area packet switched network such
`as X25, frame relay or asynchronous transfer mode.
`FIG. 5 is a high level block diagram showing further
`details of the system of FIG. 4. Reference numerals common
`to FIGS. 2 and 4 reference the same matter. Referring now
`to FIG. 5, remote data manager 92 is connected to human
`interface 88 by data path 94 and to communications manager
`by two-way data path 96. Remote data manager 92 receives
`commands from human interface 88 to retrieve data from
`remote computers through two-way communications chan-
`nel 74 and to send that data to human interface 88 for
`presentation to the consumer.
`Network protocol manager 68 is connected by two-way
`data path 70 to communications manager 66 and by two-way
`data path 72 to two-way communication channel 74. Net-
`work protocol manager 68 performs the function of format-
`ting data received and transmitted over two-way communi-
`cation channel 74. It contains the various protocols required
`to communicate with remote computers. Again, operating
`systems such as Microsoft’s Windows typically provide
`network protocol managers with plug in modules called
`drivers to support diiferent protocols and different commu-
`nication hardware. Network protocol manager 68 extracts
`incoming data from two-way communication channel 74
`using a network protocol appropriate for the medium sup-
`ported by two-way communication channel 74. Network
`protocol manager 68 also encodes output data using the
`appropriate protocol and passes the data onto two-way
`communication channel 74 for transmission to third party
`computers.
`Two-way communications channel sub-system 74 pro-
`vides the function of connecting the client system interac-
`tively to remote computers that may be server machines,
`machines at online service providers, the Internet or inde-
`pendent Bulletin Board Systems (“BBS”). The network to
`which two-way communication channel 74 is connected
`may be a conventional switched analog telephone system
`interfaced to a modem, a digital switched system such as
`ISDN interfaced to an appropriate adapter card, a wide area
`network connected through an access device, satellite
`technologies, and the like. A two-way communications line
`is used to retrieve and access information that has only thus
`far been received by the client system in the form of
`references, which are pointers to where the information
`actually resides.
`
`NTFX—1007 f Page 10 of 12
`
`15
`
`35
`
`40
`
`45
`
`time trigger 76 is connected to communications
`Real
`manager 66 by one—way data path 78 and to human interface
`88 by data path 87. Human interface 88 provides the
`consumer with input and output
`to the system.
`In the
`preferred embodiment, human interface 88 uses the key-
`board and alternate input devices such as a mouse of a
`personal computer as input for requests and the display of
`the personal computer for displaying the data.
`Real time trigger 76 accepts commands sent as part of the
`associated data to display a page of information without the
`user asking for it. The output of realtime trigger is a
`command sent over data path 87 to human interface 88
`which causes the page of information to be displayed. For “
`example, a broadcaster may want viewers to see a certain
`page of information as part of a program that
`is being
`viewed. Real time trigger allows data to be displayed using
`the same data protocols as consumer requested information.
`Local data storage 80 is connected to communications
`manager 66 by one-way data path 82 and by data path 83 to
`local data manager 84. The hardware implementation for
`local data storage 80 may be one or more of the following;
`RAM, disk, tape, recordable (I'D-ROM.
`Local data manager 84 is connected to communications
`manager 66 by data path 86 and to human interface 88 by
`two-way data path 90. Local data manager 84 receives
`commands from human interface 88 to retrieve associated
`data from local data storage 80 and sends that data to human
`interface 88 for presentation to the consumer. For example,
`a "Web Browser” may be used to display data pages from the
`World Wide Web (the “WWW“). Providers of WW
`browsers include Netscape Communications Corp, America
`Online, Spyglass and others. Local data storage 80 will
`typically be of limited capacity. Thus, local data manager 84
`purges older and less used information. This is accomplished
`by assigning an expiration date andi’or a priority to associ-
`ated data files. The criteria for determining which data to
`purge is settable by the broadcaster auditor the consumer.
`('.'riteria include total amount of storage available, size of
`associated data files, expiration date and priority.
`The preferred embodiment illustrated in FIGS. 2 and 3
`operates in the following manner. A primary data stream is
`generated. Associated data is separately generated. A script
`is generated that synchronizes how the primary and associ-
`ated data are linked together. The primary data, the associ-
`ated data and the script are sent to sequencer 14 where they
`are combined. The primary data is transmitted in the con-
`ventional fashion of its delivery medium. In the television
`model,
`this is broadcast over the air or on cable. The
`associated data is generally (but not always) sent by the
`same delivery m