'
`United States Patent
`
`[191 l
`
`I|||ll|l|I||||||||||l|||l|||ll||l||||llllllllll|l||l|l||l||l|l|||||||lll|||
`US0051l3259A'
`_
`nu Patent Number:
`,- 5,113,259
`
`.
`
`
`Romesburg et a!.
`{45] Date of Patent: May 12, 19.92
`
`ng-1-A mmsrgn mom AN Exmmn
`COMPUTER TO A TELEVISION RECEIVER
`"
`_
`HC:P
`1:” PICTURE
`
`- 4,355,813
`
`[54]
`
`_
`[75]
`
`Inventors: Eric D. Romesburg: Kennefli _W.
`Mm; D'“‘i'5 '7' D"m°”* an °f
`I“,di"'“‘,P°“5? Mi°'‘‘'’ 3' Dd“!
`Zmnsvilles Bill! W- 391975; «In.
`,
`Greenfield; Kevin E. Bridgewat
`er
`Ind‘
`'
`1] "find.
`“mm” 3 °
`[73] Assignee: Thomson Consumer Electronics, Inc.
`Indianapolis, Ind.
`
`.
`
`-
`
`3/1939 Russell et a].
`Shfonodgeatdal.
`_ Ill
`.
`..
`,
`,
`_
`4,914,515 4/1990. Duifield
`4,913,531 4/1990 Johnson
`' 233%: $3?’ g‘e]I"]““"_$l‘:i's
`.' ass/133
`-4.994314 2/1991' Wiseman et-a.l
`353/133
`5.01-1:123
`s/1991 Chen
`5.016.106
`s/1991 Yong-Je et a]. .................... 353/131
`Primary-Examiner—James J. Groody
`Assistant Exami'nen—Jefi‘rey s. Murrell
`Arroniey. Agent, or F:'rm—Joseph S. Tripoli; Peter M.
`Emanuel: Thomas F. Lenilian
`
`ass/903
`
`ass/133
`358/133
`
`
`
`[21] App}. No.: 511.269
`[22] FM‘
`A1"-19: 199°
`[51] km CL,
`[52] us. C1.
`[53] Field of Search
`
`
`
`.
`HMN 5/272
`358/183; 353/903
`353/133, 132, 131, 190,
`353/903
`
`[55]
`
`Rafe]-encgg cued
`
`_
`_
`U.S. PATENT DOCUMENTS
`4,665,438
`s/1931 Miran et al.
`'4,-.-29.02-r
`3/1-ass
`I-Iakamada mi. .
`4,750,033
`6/1988 Wefles et al.
`4.809.069 2/1939 Meyer et al. -.
`4.812.909
`3/1939 ' Yokobaynslli et
`
`.
`
`353/133
`352/133
`353/133
`ass/153
`353/133
`
`[571
`A35T.RAC1'
`Thd"tizedvid '
`dates:
`:1‘
`‘:1 RAM
`in fa pig:-in-pix megigzfiermeivgr :o?li?ede°by the
`addition of digital data from an external computer vie a .
`°9"1P‘“°’ ““°1'_*"*°° °"°““ Md?’ °°""°19“'_’°_P‘*““'
`P“ °°'m'°“=I- Tl“ °°mP“‘°T '“‘¢”*°° °'1'°““ *5 ‘=‘.’“'
`pied to the internal controller of the televifion receiver
`via an external input terminal. The additional dig-ital
`' data received from the external computer may be repre- -
`sentative of either text or graphics to be merged into the
`stored video image_
`
`'
`
`-
`ts
`'_5C"'""’ 5 D“"”“g_
`Mlcrofichc Appendix Included
`(1 Microfiche. 12 Pages)
`
`Shae
`
`430
`
`
`uerscrons I
`
`mvF!ElglI£lgI§l»END
`
`NTFX-1005 I Page 1 of 9
`
`

`
`I US‘. Patent
`
`May 12, 1992
`
`Sheet 1 ‘of 5
`
`5,113,259
`
` IZNITERFACE
`
`-um %
`
`TRANSMIT om
`
`
`. REQUEST TO SEND f
`
`
`
`
`
`
`
`
`I -''-'|
`__ necvnm
`
`'
`
`om TERM nsnnv _
`
`~
`
`%
`
`SIGNALGND
`
`‘CONNECTOR DB9 (FEHfil.E)
`
`GND
`
`'
`
`awn sun
`
`sun
`
`NTFX-1005 I Page 2 of 9
`
`

`
`U.S. Patent
`
`' May 12,1992
`
`Sheet 2 of 5
`
`5,113,259
`
`"mass oouumn LINE
`
`305
`
`N0
`
`E
`
`312
`DISLAY USAGE
`MESSAGE AND
`Exit
`
`°°W*N° 0"’?
`2
`
`%
`vss '
`%
`INITIALIZE mrennuprs
`-AND men
`
`315
`
`31°
`
`2
`
`2
`
`2
`
`335
`
`E
`
`2
`
`EXIT
`
`'
`Fl '3
`
`2 320
`
`~
`
`2
`
`E
`
`325
`
`_
`
`PLACE CAV
`N RS 232 ?
`
`330
`
`-
`
`
`
`
`
`FREEZE A
`.‘ ICTUFIE ?
`
`
`YES
`DATA FROM DISK
`
`READA BLOCK or
`
`% 340
`
`
`
`SEND BLOCKOF ~
`DATA TO PIP
`
`SEND WHOLE
`PICTURE ?
`
`345
`
`-350
`
`7
`_. YES
`sea
`RESET MEHHUPTS “AND EXIT
`
`E
`
`2
`
`2
`
`INTFX-1005 I Page 3 of 9
`
`

`
`s”U
`
`f.H..waP.
`
`May 12, 1992
`
`Sheet 3 of 5
`
`' 5,113,259
`
`av
`
`uziO...
`
`m.m_>_E
`
`.
`
`mmq
`
`..<zo__mowns
`
`mommmoofi
`
`a.
`
`fie
`
`‘Immohommn
`
`
`%92mE_u__.::<_
`
`u__moz.‘“.5
`
`%24
`
`9?
`
`sq
`9&8.%
`
`_._% :z=%
`zocoudmn
`¢o§_¢&mE_._,2.5E.»
`
`
`2.
`
`.1.‘
`
`
`
`_H=H=U_UE>mom=
`
`%N...
`
`DUDE
`
`
`..oE.z8
`
`=ommm8Eo=_o_:%%
`
`NTFX-1005 I Page 4 of 9
`
`
`
`
`
`

`
`U.S. Patent
`
`May 12, 1992
`
`‘Sheet 4 ors -
`
`5,113,259
`
`RECEIVE 15
`' BYTES OF DATA
`AND ADDRESS
`
`' DATA" AND
`ADDRESS
`
`RECEIVE '
`REGISTER mm
`mo ADDRESS
`
`'
`
`wane om '
`TO MICRO RAM
`
`'
`
`STORE DATA _
`IN mono
`RAM
`
`RECEIVE
`_ ADDFIESSAND
`
`.
`
`F-IIEAD MICRO RAM,
`'SENDg!(I)AR_|FjS-232
`
`555
`
`no
`
`MD ‘I305?
`
`~
`
`YES
`
`RECEIVE VFW
`“D9935 ‘"9
`NWFBI-OGKS
`
`READ A BLOCK
`or nmrnou
`vamsauoour
`VIA as-232 PORT
`
`I NTFX-1005! Page 5 of9
`
`

`
`U.S. Patenf
`
`May 12, 1992
`
`Sheet-5.of 5
`
`5,113,259
`
`
`
`% INTERFACE
`unrr
`
` INTERFACE %
`
`um?
`
`NTFX-1005 I Page 6 of 9
`
`

`
`1
`
`5,113,259
`
`cars rnxnsrnn FROM AN EXTERNAL
`COMPUTER TO A TELEVISION RECEIVER
`HAVING PICTURE-IN-PICI'URE CAPABILITY
`
`REFERENCE TO A MICROFICHE APPENDIX
`A microfiche appendix containing computer program
`source code has been submitted to the United States
`Patent and Trademark Office for public access. There is
`one microfiche having 12 frames included therein. The
`microfiche contains source code written for an IBM PC
`and source code written for a Hitachi microcomputer.
`The source code written in Pascal language for an-IBM
`PC enables video data to be transferred to the subject
`PIX-IN-PIX television receiver from the hard disk unit
`of the PC. The source code for controller 4111, written
`in assembly language for the ED631301 YOP microcom-
`puter manufactured by Hitachi corporation is included
`after the IBM-source code.
`
`FIELD OF THE INVENTION
`
`This invention relates to the field of television receiv-
`ers having picture-in-picture capability.
`CROSS REFERENCE TO A RELATED
`APPLICATION
`
`-
`
`25
`
`' This application is related to-an application entitled
`DATA TRANSFER FROM A TELEVISION RE-
`CEIVER HAVING PICTURE-IN-PICTURE CA-
`PABILITY TO AN EXTERNAL COMPUTER,
`bearing application Ser. No. 510,711 filed Apr. 19, I990,
`and assigned to the same assignee as the subject applica-
`tion.
`
`BACKGROUND OF THE INVENTION
`Many modern television receivers have picture-in-
`picture (PIP, or PIX-IN-PIX) capability, that is, the
`capability to receive video signals from two different
`sources and combine them to produce a signal which
`when displayed includes a first program in a main view-
`ing area, and a second program in a secondary (inset)
`viewing area of the same display screen.
`It is known in the art to produce a still picture by
`storing a “snap-shot-like" still image '(i.e., freeze-frame)
`in a digitized'fonn in a video random access memory
`(video RAM or VRAM). Such a system is known from
`the RCA VPT-695 videocassette recorder, sold by
`Thomson Consumer Electronics,
`Inc.,
`Indianapolis,
`Ind.
`
`45
`
`SUMMARY OF THE INVENTION
`It is herein recognized that the stored digitized video
`image data stored in the video RAM in a pix-in-pix
`television receiver may be modified by addition of text
`or graphics written to selected memory locations of the
`pix-in-pix mory nnder control of the pix-in-pix con-
`trailer by an external computer via a computer interface '
`circuit.
`60
`
`BRIEF DESCRIPTION OF THE DRAWING
`
`FIG. 1 illustrates, in pictorial form. a system suitable
`for use with the invention.
`FIG. 2 illustrates in schematic form a communica-
`tions interface suitable for use with the invention.
`FIG. 3 is a flowchart showing a portion of a control
`program for the external computer of FIG. I.
`
`65
`
`2
`FIG. 4 illustrates, in block diagram form. an embodi-
`ment of the invention incorporated within the television-
`receiver of FIG. 1.
`- FIG. 5 is _a flowchart showing a portion of a control
`program for the microcomputer of FIG. 4.
`FIGS. 6:: and 6!: illustrate modification of the pix-in-
`pix image with graphics and text characters‘, respec-
`tively.
`
`DETAILEO DESCRIPTION OF THE
`EMBODIMENTS
`
`Referring to FIG. 1, a television receiver having
`pix-in-pix capability 100 is connected to an external
`computer 110 with an interface unit 120 for the inter-
`change of digital signals. While external computer 11!!
`need not be any specific one of the personal computers
`currtly available on the market, an IBM PC was used
`to develop the data transfer source code listed below.
`In an embodiment of the invention which was actu-
`ally built and evaluated, the communications port for
`the television receiver was a single wire bi-directional
`serial communications port. Such a port is known from
`the communications port used in the RCA DIMENSIA
`SYSTEM produced by Thomson Consumer Electron-
`ics Inc., Indianapolis
`Interface unit 120 converts
`signals between the DIMENSIA BUS format and the
`RS-232 format used by most ‘personal computers. Inter-
`face unit 120 may be eliminated by providing standard
`RS-232 communications capability in the television
`TEDEIVBI.
`
`Briefly, signals REQUEST TO SEND and TRANS-
`MIT DATA from personal computer" (PC) 110 are
`OR’_ed together by diodes D1 and D2 and applied to a
`DIMENSIA BUS phono connector J1 via open-colIee-
`tor transistor Q4. Data signals from the television -re-
`ceiver are applied to the RECEIVE DATA terminal of
`a DB9 connector Pl via transistors Q3, Q2, and Q1. The
`DATA "SET READY line is coupled to ‘The RE-
`CEIVE DATA line.
`External computer 110 initiates the data transfer to
`the video RAM under control of a program according
`to the flowchart of FIG. 3. Referring to FIG. 3, the
`program of FIG. 3 is entered at step 3ll5_in response to
`the tering of a computer command by a user. The
`phrase PARSE COMMAND LINE of step 305 means
`decode the command received. The command is exam-
`ined at step 310'to determine if it is an executable com-
`mand. If it is not, an error message is displayed and the
`program is exited (step 312). If the ‘command is execut-
`able, a procedure is begun in which interrupts are en-
`abled and the communication. the television receiver is
`attempted (steps 315-330). The DIMENSIA BUS is
`alsorefi.-rredtoastheC.AV BUS.andtheseten'nsas
`used herein have the same meaning. In the event that
`communication with television receiver 100 is not
`achieved th an error message is displayed and the
`program is exited (step 325); However, if communica-
`tion with television receiver lflll is achieved, then the
`transfer of data to be storedin the video RAM of the
`pix-in-pix circuitry of television 100 is begun in accor-
`dance with steps-340-350. When the transfer is com-
`plete. the interrupts are reset and the program is exited
`at step 360.
`'
`A simplified block diagram of an embodiment of the
`invention is shown in FIG. 4. Referring to FIG. 4, radio
`frequency (RF) signals are applied to an RF input tenni-
`nal IICII of-atuner assembly 402. Inner assembly 402
`selects a particttlar RF signal under control of a tuner
`
`NTFX-1005 I Page 7 of 9
`
`

`
`5,113,259
`
`3
`control unit 404 which applies a tuning control signal to
`tuner assembly 402 via a wire 403, and applies band-
`switching signals via a control bus 403’. ‘Inner control
`unit 404 is controlled by a controller 410. Controller
`410. which may be a microprocessor or microcomputer,
`includes a central processing unit (CPU) 412, a read-
`only memory (ROM) 414, and a random access memory
`416. Controller 410 receives user-entered control sig-
`nals from a local keyboard 420 and from an infrared
`(IR) receiver 422. IR receiver 422 receives and decodes
`remote control signals transmitted by a remote control
`unit 425.
`Tuner 402 produces a signal at an intermediate fre-
`quency (IF) and applies it to a processing unit 430 com-
`prising a video IF (VIF) amplifying stage an AFT cir-
`cuit, and video detector, and a sound IF (SIF) amplify-
`ing stage. Processing unit 439- produces a basehand
`video signal (TV), and a sound carrier signal. The sound
`carrier signal is applied to an audio signal processor unit
`435 which includes an audio detector and a stereo de-
`coder. Audio signal processor unit 435 produces base-
`band left and right audio signals and applies them a pair
`of speakers 433 for sound reproduction.
`The baseband video signal (TV) is coupled to a de-
`coder 440 which produces a l_uminance Y signal, and U
`and V color component signals. A video input terminal
`442 labelled AUX IN is provided for receiving a base-
`band video signal from an external source. The exter-
`nally supplied baseband video signal is applied to a
`second decoder unit 144 which also produces a lumi-
`nance Y signal, and U and V color component signals.
`' The derived Y, U, and V signals are applied to input
`terminals A and B of a picture-in-picture (PIP) process-
`ing unit 450, respectively.
`PIP unit 450 includes a processor unit 4504: for pro-
`cessing the applied video signals, and a control bus
`interface unit 4505 for communicating with controller
`410 over a serial control bus 418. Serial control bus 418
`
`couples command data from controller 410 which con-
`trols PIP unit 450 to -produce an image for display hav-
`ing a main (or primary) picture and an inset (or second-
`ary) picture. Under controller command, via serial bus
`418. the PIP function may be enabled and disabled, and
`the inset picture may be displayed, for example, in each
`of the fourcorners. or in several other areas. Also under
`controller command via serial bus 418, PIP unit 450
`may be controlled to interchange (swap) the main and
`inset television images.
`The output Y. U, and V signals of PIP unit 450 are
`applied to an encoder unit 452 which converts them
`into a composite video signal for further processing by
`video signal processor unit 455 and ultimately displayed
`on a display screen. of a display device (not shown).
`Video signals are also applied to a sync separator unit
`460 which derives vertical and horizontal synchroniz-
`ing signals therefrom. The derived vertical and horizon-
`tal signals are applied to controller 410 and to a dellec-Z
`tion unit 470 for the production of dellection signals for
`application to a yoke assembly (not shown).
`PIP unit 450 is coupled to a video memory unit
`(video-RAM) 480 by a data bus 482. Video RAM 480 is
`preferrably large enough to store one field of a televi-
`sion picture (i.e.. field store mory). The term field
`store memory as used herein means a memory capable
`of storing 220 television lines of data wherein each
`television line comprises 512 8-bit bytes of data. It
`should be noted.that controller 410 can cause PIP unit
`450 to address any individual memory location of video
`
`4
`RAM 480, by sending the appropriate command over
`serial control bus 418.
`Controller 410 is coupled to a serial digital communi-
`cations port 490 via a wire 492. The present invention is
`directed to that portion of FIG. 1 and to the program
`instructions of controller 410 which in combination
`allow the transfer of digital data to video RAM 480
`from an external computer via signal digital communi-
`cations port 490 under control of controller 410.
`. The RS-232 decoding portion of the control program
`for controller 410 is shown in FIG. 5. The routine is
`entered at step 500. and advances to step 505 where it
`determines if an external computer has communicated a
`command to enter the RS-232 mode. If not the routine
`.is exited at- step 530 to the remainder ofthe DIMENSIA
`SYSTEM decoding routines. _If an ENTER RS-232
`MODE_command has been received, then the program
`advances to step 510 in which the second byte of the
`command is used to set the baud rate for RS-232 type
`communication. At step 519, a 300 ms timer is set, and
`at step 520 a loop is entered in which the timer is repeat-
`edly checked (step 520) while the program waits for the
`start of an RS-232 message (step 525). If-the timer ex-
`pires then the routine is exited at step 530. If an RS-232
`message has begun then the program advances to the
`command parsing routines (steps 535. 540, 550, 560, and
`565). If the command received is the VIDEO RAM
`WRITE command then the program receives 16 bytes
`of data and the starting address of the memory locations
`in video RAM (VRAM) in which the data is to be
`stored. The data is then written to the VRan_1 at step 539
`and the program returns to step 515. At step 540, the
`program checks to see if the command received is the
`MICRO RAM WRITE command. If so._ then the pro-
`gram receives data and a starting address in the RAM-
`416 of controller 410. The data is written. to RAM 416
`
`5
`
`20
`
`25
`
`30
`
`35-
`
`40
`
`45
`
`55.
`
`in step 544. At step 550 the program determines if the
`command received is a subroutine call. If so, the pro-
`gram receives data for the setting of registers in CPU
`412 of controller 410, and an address in ROM 414 of the
`subroutine to execute (step 557). At step 558 the subrou-
`tine is called. At step 559 any data which resulted from
`the subroutine call is stored in a specific area of RAM
`416 so that the external computer can access it with a
`MICRO RAM READ command. At step 560. the pro-
`gram checks to see if the command received is a
`MICRO RAM READ command. If so, at step 562 it
`receives the address of the starting location in RAM 416
`from which to begin reading, and the number of bytes
`to be read. At step 564 the data is read from RAM 416
`and transmitted to the external computer via the serial
`communications port. At step 565, the program deter-
`mines if the command received is a VIDEO RAM
`READ command, if not the program returns to step
`515.'If so, then at step 567 the program receives the
`starting address of the VRAM read location and the
`number ofblocks N to be read, with each block being 16
`bytes long. At step 569 a block of data is read from the
`VRAM and sent to the external computer via the serial
`communications port. At step 570 a check is made to see
`if all requested blocks have been sent. If so, the the
`program returns to step 515. Ifnot, then the VRAM'
`starting location for the reading operation is adjusted by
`16 bytes and another block of data if read, until all
`requested data has been sent.
`By use of the above-described system, a user can
`modify a frozen field of video stored in the VRAM of
`- his pix-in-pix television receiver on a pixel-by-pixel
`
`NTFX-1005 I Page 8 of 9
`
`

`
`5,113,259
`
`6
`second input coupled to said input terminal for
`receiving said digital control signals and digital
`signals representative of a second video signal, and
`having an output for developing said first and sec-
`ond control signals in response to "one of said data
`signals input by said user and said digital control
`signals, -said control means applyingsaid digital
`signals representative of said second video signal
`received via said input terminal to said mory_
`means;
`
`5
`basis. The data sent to the VRAM can be either text
`characters or "graphics. A user can employ his pix-in-piit
`television to freeze a frame of video, and transfer it via
`a digital communications link to his personal computer.
`Once the digitized field of video is stored in the user‘s
`PC it can be “imported“ into "painting" programs (i.e.,
`graphics programs) to be modified as the user wishes.
`transfer it back to the VRAM of the television receiver.
`The modified video image can then be recorded on a
`video cassette recorder, if desired. It is also possible to
`create a library of video images by transferring them to
`the hard disk of a PC for later recall, transmission -back
`to the pix-in-pix television receiver, and display. A digi-
`tized video image may be received from other, distant
`computers via a standard computer "modem coupled to
`a telephone line, and transferred "to the pix-in-pix televi-
`sion receiver for display.
`_
`FIG. 6a shows a pix-in-pix television receiver 600;:
`having an inset video irnage_ which has been modified by
`the addition of graphics data in the form of large X
`drawn across the image. FIG. 6b shows a pix-in-pix
`television receiver 6005 having an inset video image
`which has been modified by the addition of a text char-
`acter (i.e.._ the numeral 8). While FIGS. 60 and 6!: show
`modification of an inset picture, it should be noted that
`a full size image can also be modified m deserilied
`above, or a complete full or reduced size image can be
`‘provided entirely from the external computer.
`It is also noted that when the television receiver is not
`using the piii-in-pix module, that data can be transferred
`to and from the VRAM without affecting the picture
`observed by the viewer.
`The term television receiver, as used herein, includes
`television receivers having it display device (commonly
`known as television sets) and television receivers with-
`out a display device," such as VCRs.
`What is claimed is:
`1. A picture-in-picture system for a television re-
`ceiver, comprising:
`'
`data entry means for generating data signals in re-
`sponse to activation by a user, said data entry
`means including local keyboard means and remote
`control receiver means;
`memory means for storing data;
`picture-in-picture procsing means. for receiving a
`first video signal, for storing a‘ predetermined
`amount of said first video signal in said memory
`means in response to a first control signal, and for
`producing a display signal for display on a display
`screen;
`
`said display signal being a combination of said first
`and second video signals.
`2. The picture-in-picture system of claim 1 wherein
`said digital signals representative of said second video
`signal correspond to graphics data.
`3. The picture"-in-picture system of claim 1 wherein
`said digital signals "representative of said second video
`‘signal correspond to text characters.-
`. 4. A picture-in-picture system for a- television re-
`ceiver, comprising:
`'
`a source 'ofa.first video signal corresponding to a first
`video image;
`an input terminal for receiving digital control signals
`and digital signal corresponding to a second video
`image from an external computer;
`memory means for storing data related to said video
`signals;
`picture-in-picture processing means coupled to said
`first source of video signals for receiving said video
`signals, _and coupled to said memory means for
`storing a predetermined amount of"said first video
`signals in response to a first control signal, for writ-
`ing said. digital signals corresponding -tosaid sec-
`ond video image to said memory means in response
`to a second control signal and for producing a
`display signal for display _on a display screen, said
`display comprising combination of said first and
`second images;
`'
`control means for controlling said picture-in-picture
`processing means, said control means having a first
`input coupled to said input terminal for receiving
`said digital control signals and said digital signals
`corresponding to said second video image, and
`- having an output for developing said first and sec-
`and control signals:
`said control means in response to said digital control
`signals generates said first control signal causing
`said picture-in—picture processing means to store
`said predetermined amount of said first video sig-
`nals, and generates said
`control signal caus-
`an input terminal for receiving digital control signals
`ing said picture-iii_-picture means to write said digi-
`and digital signals representative of a second video
`tal signals corresponding to said second video
`signal from an eirternal-source;
`image to said memory means.
`said picture-in-pictiire-processing means writing said '
`55
`5. The picture-in-picture system of claim 4 wherein
`digital signals representative of said second video
`signal to said mory means in response to a sec-
`said digital signals corresponding to said second video
`image correspond to" graphics data.
`ond control signal; and,
`:-
`6. The picture-in-picture system of claim 4 wherein
`control means for controlling said picture-in-picture
`said digital signals corresponding to said second video
`processing means, said control means having a first
`tilfli
`input coupled to said data entry means for receiv- 60 image correspondto text characters.
`ing sm'd data signals input by said user, having a
`
`15
`
`30
`
`45
`
`50
`
`NTFX-1005 I Page 9 of 9