United States Patent [191
`Batchelor
`
`[54] CD ROM INFORMATION REFERENCES
`DELIVERED TO A PERSONAL COMPUTER
`USING THE VERTICAL BLANKING
`INTERVALS ASSOCIATED DATA
`TECHNOLOGY FROM A NABTS
`COMPLIANT TELEVISION BROADCAST
`PROGRAM
`Inventor: Steve Batchelor' H?lsbom‘ Greg’
`A ~
`: Intel C
`t-
`_ S ta C1 _ Calif‘
`sslgnce
`orpora Ion an
`am
`_
`Appl' No" 558’030
`Filed:
`Nov. 13, 1995
`6
`Int. Cl. ..................................................... .. HMN 5/44
`U.S. Cl. ......................... .. 348/553; 348/460; 348/564
`Field of Search ................................... .. 348/552. 553.
`348/563. 564~ 468. 465. 473, 476. 478.
`460
`_
`Refemnces Cited
`U_S_ PATENT DOCUMENTS
`1/1990 Yee ....................................... .. 3481468
`
`E75]
`[73]
`[21]
`[22]
`[5 1]
`[5 3]
`
`[56]
`
`4,894,789
`
`03
`US0057241 A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,724,103
`Mar. 3, 1998
`
`5,262,860 11/1993 Fitzpatrick et a1. .................. .. 348/468
`5,537,151
`7/1996 Orr et a1. ....... ..
`348/564
`5,561,457 10/1996 Ctagun et a]. ........................ .. 348/553
`Primary Examiner-—Victor R. Kostak
`Attome); Agent, or Firm-—Blakely. Sokoloff. Taylor 8:
`Zafman
`ABSTRACT
`[57]
`A system that displays text and graphic information with
`vertical blanking interval of a publicly broadcasted video
`signal. The system also includes a personal computer that
`has a VBI decoder which can separate the data from the
`video signal. The separated data contains command and
`address information, which instruct the personal computer to
`retrieve text/graphic information from a storage device, and
`display the re‘??ved text/graphic infomm?on on a comma
`monitor, The pcrsonal colnputer also contains 3 NTSC
`decoder which can decode the video signal and display a
`television image on the computer monitor. The broadcaster
`inserts data that retrieves text and graphic information which
`correspond to the television image displayed by the com
`puter monitor. The retrieved text and graphic information
`may be stored for later viewing by the user.
`20 Claims, 3 Drawing Sheets
`
`6 . """""""""""""""""""""""""""""""""""" "l
`
`,% a
`2
`E
`2U\I mm sou/ear I
`i
`I
`)6‘
`m
`i
`_,g
`E
`/
`/
`E
`I VIDEO 50URCE|——I v51 /~s£mzrI—-I r/w/su/rm? III? 5 \
`,
`L ___________________________________________________________ “1'
`r --------- "it; ------------------------------------------------------ -‘ 24
`15
`34
`/
`I MRO DISK o/v/vr I
`1
`
`I
`
`OPT/04L
`015K
`
`12
`
`I
`
`J2
`/
`CPU
`
`IMNSPORI' MTER — APPLICATION PROGRJMMING [MERE/ICE, STA/US MNAG‘HJDJT, TIME SYNC
`NETWORK LAYER — CHANNEL MAME/EVEN): SCHEULING, PACKEI’IL’AI'ION
`?/VALINK MYER — PAC/(U SERIAL/HUM VBI DEV/Cf INTEQFACE
`SERIAL DAM
`L5,! INSEWER l
`
`VIDEO SOURCE
`
`V1050 SIG/1M1
`
`VIDEO SIGNALS AND JNSEWQ) DATA
`
`/-50
`.52
`/
`54
`/
`
`I am SOURCE If”
`FROG/34M DNA
`
`15
`
`PETITIONER EX. 1004 Page 1
`
`

`
`U.S. Patent
`
`Mar. 3, 1998
`
`Sheet 1 of 3
`
`5,724,103
`
`
`
`I!“ §m§ fa _1 .................................................................. .L_ n .1 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
`
`
`
`N at m R6 m m
`
`(NJ/1p) w \.\k $58 $5 /9w w
`
`r ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| -L.
`
`EQSE § m
`
`@N ”
`\ m
`
`Egg 5? m
`
`mw “
`\ m
`
`
`
`.I I I I I I I l l I l I I I l l I l I I l | | l l l l I I l l l I l I I I l l l l l I I I I I I I I l I l I I l I I I l l l I l I l I I I l l l l I l l l l l | l | l l l l l l l I l I l l III“
`
`
`
`4/3 m &\ $E§§ @E? E 1 $3“ 8% m
`
`_ _
`
`m 2 3 m
`m \ _ \ m
`
`g :g 5% kg 9% 3a “
`
`
`
`NW “A, .0“ Rm. m
`
`s, M
`B U m
`
`m
`
`0
`
`J
`
`PETITIONER EX. 1004 Page 2
`
`

`
`US. Patent
`
`Mar. 3, 1998
`
`Sheet 2 of 3
`
`5,724,103
`
`%\
`
`[
`
`
`
`GER $46
`
`PETITIONER EX. 1004 Page 3
`
`

`
`U.S. Patent
`
`3M
`
`8
`
`3m
`
`8
`
`E5=§8E
`
`M,at
`
`
`
`
`
`
`
`92%M35§mE§<§MEEMmG§<.E\<\..w\<§$.v..<u©.<u.2§§:&<I.<.m:3.$bQm.\<wE
`
`
`
`
`
`
`
`
`
`
`
`mm,Est§§cE.%§E§,$<m\E.§<§.§<.<§GI325§§Ez
`
`
`
`
`
`maxeBE»:9%32%.8%.ES?§538%8§%8%.w,EE5%
`
`§$.mSE35ax.E:§::$m52¢..$3¥<.3§\Q
`
`.2E
`
`
`
`
`
` wmSEEQEma%mS52%mEQ35%am.SuS$¢.cma8§.¢muvwmmzubc_GwoEEVGE..+\_
`
`
`
`79we§Em§a.mum35:5Q8magmas:mm§Em.$Em
`
`
`.,.H.MEE3Q0?.mm35%DC
`wmmbxmmbmc
`5.959.5ms%Sm§\%1+
`
`mm%\_Em.§mm©&|u\Ekuvm\_
`wwsmvsxqcmméfi
`
`PETITIONER EX. 1004 Page 4
`
`PETITIONER EX. 1004 Page 4
`
`
`
`
`
`

`
`1
`CD ROM INFORMATION REFERENCES
`DELIVERED TO A PERSONAL COMPUTER
`USING THE VERTICAL BLANKING
`INTERVALS ASSOCIATED DATA
`TECHNOLOGY FROM A NABTS
`COMPLIANT TELEVISION BROADCAST
`PROGRAM
`BACKGROUND OF THE INVENTION
`1. FIELD OF THE INVENTION
`The present invention relates to a method and apparatus
`for inserting database address information into the vertical
`blanking interval of a video signal that is received by a
`personal computer
`2. DESCRIPTION OF RELATED ART
`Some computm systems contain NTSC decoders which
`allow publicly broadcasted video to be displayed on the
`monitor of the computer. The video may be displayed in a
`separate window that allows the user to view the video
`images while performing other operations on the system.
`Personal computers can also be provided with text ori
`ented databases that can be retrieved and viewed by the user.
`By way of example. there is a program marketed under the
`trademark INTERACTIVE ENCYCLOPEDIA by COMP
`TON of San Diego. Calif. which provides an interactive
`encyclopedia. The COMPTON product allows the user to
`select and review text and graphics which correspond to
`various historical events. geographical locations. etc.
`It would be desirable to provide a system that would
`retrieve and display text/graphic data which corresponds to
`a television image displayed on a computer monitor. For
`example. if a user is watching an informational program on
`elephants. it would be desirable if various facts and graphics
`on elephants were presented on the screen along with the
`television image. It would also be desirable if the relevant
`facts/graphics were stored in memory for later viewing by
`the user.
`
`SUNIMARY OF THE INVENTION
`The present invention is a system that displays text and
`graphic information with broadcasted television video. The
`broadcast system includes a vertical blanking interval (VBI)
`inserter which inserts data into the vertical blanking interval
`of a broadcasted video signal. The receiving system includes
`a personal computer that contains a VBI decoder which can
`separate the data from the video signal. The separated data
`contains command and address information. which instruct
`the personal computer to retrieve text/graphic information
`from a storage device. and display the retrieved text/graphic
`information on a monitor. The personal computer also
`contains a NTSC decoder which can decode the video signal
`and display a television image on the monitor. The broad
`caster inserts data that retrieves text and graphic information
`which correspond to the television image displayed by the
`monitor. The retrieved text and graphic information may be
`stored for later viewing by the user.
`BRIEF DESCRIPTION OF THE DRAWINGS
`The objects and advantages of the present invention will
`become more readily apparent to those ordinarily skilled in
`the art after reviewing the following detailed description and
`accompanying drawings. wherein:
`FIG. 1 is a schematic of the system of the present
`invention;
`FIG. 2 is a screen of a video monitor;
`
`5,724,103
`2
`FIG. 3 is a schematic showing the layers of a video
`encoder;
`FIG. 4 is a schematic showing the fonnatting of a video
`frame that is inserted into the video blanking interval of a
`video signal;
`FIG. 5 is a schematic showing the formatting of a video
`packet;
`FIG. 6 is a schematic showing the formatting of a message
`provided to the encoder.
`DETAILED DESCRIPTION OF THE
`INVENTION
`Referring to the drawings more particularly by reference
`numbers. FIG. 1 shows a system of the present invention.
`The system includes a personal computer 10 that receives
`video signals and other information from a broadcaster 12.
`The broadcaster 12 may be any broadcasting system includ
`ing satellite and cable networks. The broadcaster 12 includes
`a vertical blanking interval (VBI) inserter 16 that has one
`input connected to a video source 18 and another input
`connected to a data source 20. The video source 18 generates
`video signals that have a vertical blanldng interval. The VBI
`inserter 16 inserts data generated by source 20 into the
`vertical blanking interval of the video signal. The VBI
`inserter 16 can be a unit sold by Norpak Corp. of Ottawa.
`Ontario, Canada under model number TDS-3. The video
`signal and accompanying inserted data are transmitted by a
`transmitter 22.
`The video signal and inserted data are received by a
`receiver 24. The receiver 24 provides the signal and data to
`a vertical blanking interval (VBI) decoder 26. The VBI
`decoder 26 which separates the data from the video signal.
`The video signal is provided to a NTSC decoder 28. The
`decoder 28 converts the video signal into television signals
`that can be displayed on a computer monitor 30.
`The computer It) typically contains a central processing
`unit (CPU) 32. a dynamic random access memory (DRAM)
`device 34. a massive storage device such as a hard disk drive
`36. and an optical disk drive 38. The optical disk drive 38
`may contain an optical disk which contains a text and
`graphic based database. By way of example. the optical disk
`may be a “CD-ROM” product sold by COMPTON software
`under the trademark INTERACTIVE ENCYCLOPEDIA.
`The COMPTON ENCYCLOPEDIA contains text and
`graphic information which relate to historical events. geo
`graphical locations. etc. typically found in an encyclopedia.
`The text and graphic information is located within addres
`sable memory locations of the optical disk.
`The CPU 32 receives the data separated from the video
`signal by the VBI decoder 26. The data typically contains
`command and address information that instruct the CPU 32
`to retrieve speci?c text and graphic information from the
`optical disk 38 and display the information on the computer
`monitor 30.
`FIG. 2 shows a screen of the computer monitor 30. The
`screen may include a window 40 that displays the television
`image and a separate window 42 that shows the text and
`graphic information retrieved from the optical disk 38. The
`text/graphic information typically relates to the video image
`shown in window 40. For example. the video window 40
`may display an elephant. The text/graphic window 42 may
`display particular facts regarding elephants. The monitor 30
`may have a graphical user interface for input commands 44
`that allows the user to change television channels. The new
`channel may also contain VBI inserted data which retrieves
`and displays text/graphic information that relates to the
`television images shown on the new channel.
`
`65
`
`10
`
`25
`
`35
`
`55
`
`PETITIONER EX. 1004 Page 5
`
`

`
`10
`
`2.5
`
`30
`
`35
`
`20
`
`5,724,103
`4
`3
`the optical disk. Alternatively. a common address may be
`The personal computer may have a resident program
`employed with a speci?c optical disk identi?cation (ID). The
`which stores the text/graphic information retrieved from the
`broadcaster may transmit the speci?c optical disklD and the
`optical disk 38 onto the magnetic disk drive 36.
`Additionally. the program may have a con?guration window
`address data. The computer may have a look-up table which
`which allows the user to select Whether the text/graphic
`allows the CPU to map the requested address information to
`information is to be saved. The text/graphic information may
`the physical addresses of the requested optical disk.
`be saved in a ?le that can be recalled and viewed by the user.
`In operation. the data source 20 provides program data
`The video signal may be converted to a binary format for
`that is inserted into the vertical blanking interval of a video
`storage with the text/graphic information so that the user can
`signal by the V131 inserter 16. The video signal and data are
`view the text and video in combination. Alternatively. the
`transmitted to the V131 decoder 26 which separates the data
`video signal may be stored on a video cassette recorder (not
`from the video signal. The video signal is converted to a
`shown).
`television image that is displayed by a computer monitor 30.
`As shown in FIG. 3. the format of the inserter 16 may
`The separated data is provided to the CPU 32 which gen
`have dilferent application layers. including a transport layer
`erates commands that retrieve data from the optical disk 38
`50. a network layer 52 and a datalink layer 54 which convert
`in accordance with address information provided with the
`the program data from the data source into a serial data
`data. The retrieved data is displayed with the television
`stream that can be inserted into the vertical blanln'ng interval
`image on the monitor 30. The CPU 32 can also command a
`of the video signal by the VBI inserter 16. The transport
`storage of the retrieved data from the optical disk 38 to the
`layer 50 provides an interface with the data source. The data
`magnetic disk 36 for later retrieval.
`source 20 may be a personal computer which is programmed
`While certain exemplary embodiments have been
`to send commands. addresses etc. at predetermined time
`described and shown in the accompanying drawings. it is to
`intervals. For example, the video signal may be transmitting
`be undo-stood that such embodiments are merely illustrative
`images of an elephant between 0-5 minutes and a tiger
`of and not restrictive on the broad invention. and that this
`between 5-10 minutes. The data source 20 can generate
`invention not be limited to the speci?c constructions and
`command/address data to retrieve “elephant" text/graphic
`arrangements shown and desu'ibed. since various other
`data at 0 minutes and then generate command/address data
`modi?cations may occur to those ordinarily skilled in the art.
`to retrieve “tiger" text/ graphic data at 5 minutes. and so forth
`What is claimed is:
`and so on.
`1. A system that receives a video signal and a data signal
`The network layer 52 packetizes the data. The datalink
`from a broadcaster which inserts the data signal into a
`layer 54 serializes the packeted data for serial transmission
`vertical blanking interval of the video signal and transmits
`to the VBI inserter 16. In the preferred embodiment. the
`both the video and data comprising:
`associated data is provided to the V131 inserter in a
`a video receiver that receives the video and data signals;
`Vt_Frame 60 shown in FIG. 4. The Vt_Frame 60 includes
`a vertical blanking interval decoder that separates the data
`a start frame ?eld STX 62. a data packet ?eld Vt_Packet 64.
`signal from the video signal;
`a check ?eld CRC 66 and an end of frame character EI‘X 68.
`a video decoder that converts the video signal into a
`In the preferred embodiment, the Vt Jacket 64 is for
`television signal;
`matted as shown in FIG. 5. The nPacketProtocal ?eld 70 is
`a monitor that displays the television signal;
`a byte-length ?eld which identi?es the packet as one sup
`ported by the protocol. The nVersion ?eld 72 speci?es the
`a non-volatile storage device that contains a text/
`graphic signal that is resident within said non
`type of inserter that is transmitting the packet. The nChanlD
`volatiles storage device; and.
`?eld 74 is an integer value which provides the channel
`number of the packet in the serialized bitstream. The nMes
`a central processing unit which receives the data signal
`sagelD ?eld 76 speci?es the message number of the channel
`from said vertical blanking interval decoder and gen
`on which the packet is transmitted. The nPacketID ?eld 78
`erates a command to transfer said text/graphic signal
`de?nes the number of packets in a particular message. The
`from said non-volatile storage device to said monitor to
`display said text/graphic signal on said monitor.
`fMorePackets ?eld 80 speci?es whether there are any more
`packets in the message. The fReserved ?eld 82 is reserved
`2. The system as recited in claim 1. wherein said non
`for further use. The length of the data is speci?ed in the
`volatile storage device is an optical disk drive.
`nDataSize ?eld 84. The nData ?eld 86 contains the data.
`3. The system as recited in claim 1. wherein the data
`In the preferred embodiment. the data is typically pro
`signal includes a memory address.
`4. The system as recited in claim 1. wherein said central
`vided to the inserter 16 by the data source in the format
`processing unit commands a storage of said text/graphic
`shown in FIG. 6. The nMessageProtocal ?eld 88 is a byte
`signal into a ?le of said non-volatile storage device.
`length ?eld which identi?es the type of transmission being
`5. A system, comprising:
`provided. The nVersion ?eld 90 identi?es the version of the
`protocol. The ?sI-lint ?eld 92 indicates that the data is to
`a broadcaster that transmits a video signal and a data
`signal that is inserted into a vertical blanking interval of
`precede an event such as a television program. The ?sUP
`date ?eld 94 indicates whether the information is updating
`the video signal;
`transmitted information. The fReserved ?eld 96 is unused
`a video receiver that receives the video and data signals;
`and reserved for later use. The ?eld nDataSize 98 speci?es
`a vertical blanking interval decoder that separates the data
`the number of bytes in the data ?eld. The nOpcode ?eld 100
`signal from the video signal;
`de?ne a particular operation to be performed with the data.
`a video decoder that converts the video signal into a
`For example. the operation may be a read memory command
`television signal;
`and a display command. The fReservedZ ?eld 102 is not
`a monitor that displays the television signal;
`used and the bData ?eld 104 contains the data. The data may
`a non-volatile storage device that contains a text/graphic
`contain the speci?c addresses to be retrieved from the
`signal that is resident within said non-volatile storage
`storage device 38. The addresses requested typically corre
`device; and.
`spond to the physical address locations of the desired data on
`
`45
`
`55
`
`PETITIONER EX. 1004 Page 6
`
`

`
`5,724,103
`6
`5
`a central processing unit which receives the data signal
`video decoder means for converting the video signal into
`from said vertical blanking interval decoder and gen
`a television signal;
`erates a command to transfer said text/graphic signal
`monitor means for displaying said television signal;
`from said non-volatile storage device to said monitor to
`non-volatile storage means for storing a text/graphic
`display said text/graphic signal on said monitor.
`signal that is resident within said non-volatile storage
`6. The system as recited in claim 5. wherein said non
`means; and.
`volatile storage device is an optical disk drive.
`processing means for receiving the data signal from said
`7. The system as recited in claim 5. wherein the data
`vertical blanking interval decoder means and generat
`signal includes a memory address.
`ing a command to transfer said text/graphic signal from
`8. The system as recited in claim 5. wherein said central
`processing unit commands a storage of said text/graphic
`said non-volatile storage means to said monitor means
`signal into a ?le of said non-volatile storage device.
`to display said text/graphic signal or said monitor
`9. A method for displaying video and corresponding text
`means.
`on a computer monitor from a transmitted video signal
`14. The system as recited in claim 13. wherein said
`which has a data signal inserted into a vertical blanking
`non-volatile storage means is an optical disk drive.
`interval of the video signal. comprising the steps off
`15. The system as recited in claim 13. wherein the data
`a) receiving the video and data signals;
`signal includes a memory address.
`b) separating the data signal from the video signal;
`16. The system as recited in claim 13. wherein said
`c) converting the video signal into a television signal;
`processor means commands a storage of said text/graphic
`into a ?le of said non-volatile storage means.
`d) displaying the television signal on a monitor;
`17. A system comprising:
`e) generating a command from said separated data signal
`broadcaster means for transmitting a video signal and a
`which retrieves a text/graphic signal that is resident on
`data signal that is inserted into a vertical blanking
`a non-volatile storage device; and.
`interval of the video signal;
`i) displaying the text/graphic signal on said monitor.
`video receiver means for receiving the video and data
`10. The method as recited in claim 9. further comprising
`signals;
`the step of storing the text/graphic signal in a ?le.
`11. A method for displaying video and corresponding text
`vertical blanking interval decoder means for separating
`on a computer monitor. comprising the steps of:
`the data signal from the video signal;
`inserting a data signal into a vertical blanking interval of
`video decoder means for converting the video signal into
`a video signal;
`a television signal;
`b) transmitting the video and data signals;
`monitor means for displaying the television signal;
`c) receiving the video and data signals;
`non-volatile storage means for storing a text/graphic
`d) separating the data signal from the video signal;
`signal that is resident on said non-volatile storage
`d) converting the video signal into a television signal;
`means; and.
`e) displaying the television signal on a monitor;
`processor means for receiving the data signal from said
`generating a command from said separated data signal
`vertical blanking interval decoder means and generat
`which retrieves a text/graphic signal that is resident on
`ing a command to transfer said text! graphic signal from
`a non—volatile storage device; and.
`said non-volatile storage means to said monitor means
`g) displaying the text/graphic signal on said monitor.
`to display said textlgraphic signal on said monitor
`12. The method as recited in claim 11. fm'ther comprising
`means.
`the step of storing the text/graphic signals in a ?le.
`18. The system as recited in claim 17. wherein said
`13. A system that receives a video signal and a data signal
`non-volatile storage means is an optical disk drive.
`from a broadcaster which inserts the data signal into a
`19. The system as recited in claim 17. wherein the data
`vertical blanking interval of the video signal and transmits
`signal includes a memory address.
`both the video and data signals. comprising:
`20. The system as recited in claim 17. wherein the
`video receiver means for receiving the video and data
`processor means commands a storage of said text/graphic
`signals;
`into a ?le of said non-volatile storage means.
`vertical blanking interval decoder means for separating
`the data signal from the video signal;
`
`* * * * *
`
`45
`
`50
`
`10
`
`25
`
`35
`
`PETITIONER EX. 1004 Page 7