wo 94117525
`
`P(.‘l‘l'!-'R_94!0008‘!
`
`11
`
`5. Dispositif
`
`selon l‘une quelconque
`
`des
`
`revendications
`
`précédentes, caractérisé en ce que des movens sont prévus pour
`
`numériser iedit signal source préalablement a son inscription sur ledit
`support.
`
`6. Dispositif selon la revendication 5. caractérisé en ce que
`
`lesdites tétes sont des tétes magnétiques munies d'entrefers.
`
`7. Dispositif selon la revendication 5, caractérisé en ce que
`
`lesdites tetes at iedit support formant un systems d'écriture-lecture de
`
`type magneto-optique.
`
`8. Dispositif selon i'une quelconque des revendications 1 at 2,
`
`oaractérisé en ce oue ladite mérnoire tampon est une mémoire a acces
`
`aleatoire gérée par microprocesseur.
`
`9. Dispositif seion la revendication 8. caractérisé en ce que les
`
`signaux représentatifs desdites images sont soumis 3 un traitement de
`
`compression préalablernent a leur écriture dans ladite mémoire et en ce
`
`que les signaux resultant de la lecture de ladite rnérnoire subissent un
`
`traitement inverse avant d'étre appliuués auxdits mavens d'affichage.
`
`10. Dispositif selon la revendication 8, caractérisé an ce que
`
`Padressage en écriture de ladite memoire est incrementé image par image
`
`ainsi que Padressage en lecture; l'écart entre les adresses d'écrirure Et
`
`de lecture étant rnodifié de plusieurs unites pour changer Ia vaieur dudit
`
`différé dans un sens et ensuite dans le sens inverse, unite par unité, 3
`
`une cadence sous-multiple cle la cadence de lecture desdites images.
`
`pour revenir a I'écart initial.
`
`11. Dispositif selon |‘une queiconque des revendications ‘let 2.
`
`caractérisé en ce que le signal source non différe est utilisé dans une
`
`phase the fonctionnement normal.
`
`tandis que des mavens sont prévus
`
`pour iui substituer le signal source différé dans une phase transit:-ire se
`
`terminanr par la reprise dudit fonctionnement normal.
`
`12. Dispositif salon l'une quelconque cies revendications 1 at
`
`2. caractérisé en ce oue ladite mémoire tampon est une rnemoire a
`semiconducteur.
`
`13. Dispositif selon Ia revendication 12. caractérisé en ce que
`
`Iadite mérnoire peut erre étendue par I‘ajout de modules.
`
`551
`
`

`
`W0 94117525
`
`PC'I‘fFR94!0BI]87
`
`J J KI N/'/OPE)?
`
`I XII
`
`FEUILLE DE HEMPLACEMENT [REGLE 26}
`
`552
`
`

`
`WO 94117626
`
`PC'I'r'FR94!0flfl8‘!
`
`FEUILLE DE REMPLACEMENT (HEGLE 26)
`
`553
`
`

`
`WO 94117626
`
`PCTlFR94!000B7
`
`FEUILLE DE REMPLACEMENT [HEGLE 26)
`
`554
`
`

`
`"INTERNATIONAL SEARCH REPORT
`
`‘-‘1”‘FP"°='“"‘ “"°
`“W”
`PCT/FR 94/00087
`
`A. Cl.ASSIl-'lCATIO?\ OF SUBJECT MATTER
`IPC 5
`HU4N5/44
`H04N5/76
`
`fiteoni-ding to Intcmaflnnaj Patemtjassiflcauun UPC) nr to truth natzonai ela-mficauon and II'C
`B. FIELDS 5|-ZAIICHED
`Minimum documentation searched (classification system followed by elasxificatmn xytnbulsj
`IPC 5
`HD4N
`
`Uaeumuiuuon seazwhed other tha.i;i mi.nti'nu'i'i'i doeturientaunn to the extent Ilut such documents are meluded in the fields searched
`
`i'¥l¢|‘-'|l'0t‘Iie data base eormitteii during the Interiiauonal search [name of data base and. where practical. search terms used}
`
`(I. D0(:UMl£N'l‘S DUN!-IDEILED TO BE RELEVANT
`
`(Zitation of documeiit, with riidication. where apptopnne. notthe relevant peerage:
`
`EP.K.0 279 549 (SONY) 24 August 1983
`
`see c}aims 1-ID; figure 3
`
`US.A.5 134 499 (SATA ET AL) 28 Ju1y 1992
`see co1umn 1,
`line 1% - column 4. Iine 25;
`figure 4
`
`GB,A,2 222 742 (HASHIMOTD) 14 March 1990
`see page 1,
`tine 19 - page 2. 1ine 6
`
`see page 3,
`
`line 17 - page 6, 1ine 5
`
`_/__
`
`E I-'unh1:r documents are listed In the eanunuanon of box C.
`' Special eatcgones of cited di:ici.ii-rieiits :
`
`'A' document defining the general state of the art which IE not
`mnsldered to be cil parttcttlar relevaiice
`‘Ii.’ earlier document but published on or nfler the intunauonal
`filing date
`' I.‘ d.0=I.ll'I'|¢I11. which may ‘|h.I'DIi‘ doubt: on pa-mi-iiy etami(:) or
`wrueh is cited to eiiiabtish the ptitzlieiition date of another
`eitalton or other special I-eiisuri (as meeiflld)
`‘U’ domit-ient refernng In an oral disdositrc. use. exhttimnn or
`other means
`'I‘' document. ptlbllflhcd prior to the iiiti:i-national filing date but
`later than the pnonty date claimed
`Date of the actual eoi'np-Eetion oi the mtematiuiiat search
`
`4 May 1994
`Nan-rie and mailing add;-as of the ISA
`limtipean l-'iIti:1IDl'|'ioi:. PB. SBI B Pateritlaan 2
`NI. - 1250 HV Klilwljk
`Td- (‘*- III-?D) 340-2D-$0. Tit. 3| 651 epo til.
`ia:(+MJm3uomm
`
`Form PCT,-'lSAJ'1lll (mound Inn!) (July I!!!)
`
`E Patent family merntlei-s are listed in annex.
`'1'’ later d9r:i.in-ieiit published alter the imzmataorial filing date
`or pi-i.ui-ity date and noun eonflict with the application but
`cited to urniei-stand the pnneipie or theory underlying the
`inwnucm
`‘X’ document oi particular II:1eva.nce; the claimed inveriuon
`eamot be considered novel or cannot be eon-iidu-ed to
`involve an inventive step when the dociiinent M taken alone
`‘Y’ document of pamettlar relevance; the claimed Ii-iveriooii
`cannot be consictei-ed to involve an iiivuiuve step when the
`dneurrieiiti: eombmed with one or more other such docu-
`rrrents. such Uumbinzuon being obvious to a person skilled
`in the an.
`‘E doctiment manner of 111: some patent [srmly
`Date at mailing ot the intiimahoriat seateh report
`
`1 1. D5. 3‘i
`
`Amhonzed uffieer
`
`Berwitz, P
`
`page 1 of 2
`
`555
`
`

`
`:3! Applicannn No
`inter
`PCT/FR 94/00087
`
`INTERNATIONAL SEARCH REPORT
`(_‘.((Ior1u11uaL1an) DOCUMENTS CONSIDERED TO BE RELEVJQNT
`
`Calzgary ‘
`
`(jmaunn ur document. mm mdicalmn. when appropnnte. oi mu relevant passages
`IEEE TRANSACTIONS ON COMMUNICATIONS
`V01. 26, no.
`5 , May 1973 , NEH veal: us
`pages 594 “ 600
`JOHNSTON ET AL.
`Sequence Stare‘
`see page 595. right column, 1ine 17 - page
`597. left column.
`line 35
`
`‘A Digital Television
`
`16TH
`SYMPOSIUM RECORD, BROADCAST SESSIONS;
`INTERNATIONAL TV SYMPOSIUM, 17 June 1989 ,
`MUNTREUX, CH
`pages 289 - 295
`HODDHAM ‘A Suiid State “Action Rep1ay"
`Recorder‘
`see the whole document
`
`Farm PCTnSIL‘1Il1¢cnnunuu.1nn or mama um!) [July I951}
`
`page 2 of 2
`
`556
`
`

`
`INTERNATIONAL SEARCH REPORT
`
`mu:
`
`nalnpphcanon No
`
`Patent. document
`mud in search rapurl
`
`Publication
`dz.-Le
`
`Patent fan-nily
`ms:mber(s)
`
`PCT/FR 94/00087
`Publication
`dam
`
`EP-A-02?9549
`
`24-08-88
`2201314
`10-02-94
`3886563
`18-08-89
`1205672
`02-01-90
`4391715
`__.______________________________________._........—..-—__.__—__________________
`
`24-08-88
`
`US-A-5134499
`
`28-07-92
`
`2044569
`2061859
`2060382
`
`2058984
`5018014
`
`14-02-90
`01-03-90
`28-02-90
`
`28-02-90
`21-05-91
`
`Form P|L'l'."|5A.|'.'llll Irllflllfllnili Illllli‘-J “H-1! 3993}
`
`557
`
`

`
`:RArPoR1' DE RECHERCHE INTERNATIONALE
`
`Inwmowewo
`Dem
`PCT/FR 94/00037
`
`A. CLASS!-ZMF.NT DI-E IICIH-JET DE LA DEMANDE
`IB 5
`HD4N5/44
`H04N5/76
`
`Salon la :13.-uuflcauon mnernanonale tlcs brevets [(5:13] on a la [ms salon la alas.-:!'1cauun nauonale 3!. la CIR
`13. DUMAINES SUR LESOUELS Lfi RF.CllERCI*ll:' A PORTE
`1)nl.'umcnI.anaI'I nurumale cansnuxcc (systcsnc dc ciassnflcanan aunt: ales sy-mbole: flu classemcnl}
`CIB 5
`H04N
`
`Docurncntahun cunsullae auln que la ¢n1:u.rn¢rIIauon I1'nn.1maI.: nan: la ms.-mar: all
`
`docmncms mieventdes clomau-ms sur iesquels a punt la mchuche
`
`Base dc dunntcn électmmqua mnsultba an noun d: I: retherch: Lnlamanunfle [mm da 1:. base dc donates. RL :1 azla an rcaJ.|u.bl:. Lcrrnu ne. recnarche
`uhlisés)
`
`C. DUCUM!-'.NT5 CONSIDER!-‘.5 CDMME PERTINENTS
`
`ldmlilicanon das docmncms cilbs. avcc. I: an militant. findiclnun dc: pasagcs pcrnncnls
`
`X
`
`EP,A,0 279 549 (SONY) 24 Aofit 1988
`
`voir revendications 1-10; figure 3
`
`US,A,5 134 499 {SATA ET AL) 28 Jui11et
`1992
`ligne 14 - colonne 4,
`vuir co1nnne 1,
`ligne 25; figure 4
`
`GB,A,2 222 742 (HASHIMOTD) 14 Mars 1990
`voir page 1,
`Iigne 19 - page 2, 1igne 6
`
`voir page 3,
`
`ligne 17 - page 6. Tigne 5
`
`../--
`
`en”:
`qwflnu spéualu de G“
`‘A’ dncm1entde[m1s:anI.I‘ua1 pneral de la lcchmque. non
`°°""d’"¢ “mm” p”“‘”1"'"“"“ Pm""""
`dOtL:1imcnI. mundguur. mass puhhé 5 la dale dc dcpct Intcrnanonal
`P": new
`document pnuvanl acts! in dnntl: aur unc rtvcndtcauon dc
`F"°'“'e °"' cl“ pm’ oewfiflm“ I‘
`'1' P“*“'°‘E‘.°“ ‘me
`“um “““'°"' W P°“" W‘ "3-"W" 59°C“-"‘ 1'-“"" 9“ ‘“d‘‘!“‘‘)
`'0‘ dncumcnl. se rereram A une dwulgauon oralc. 1 un usage. a
`um: I:1]:u:ml.|tIn nu mus aulns may-am
`‘P’ ducI.n1':emp1.|hIat avant Ia clan: :l¢ dcpot uncmaumal. mm
`pamtncurcrnenl A in date an pnonte rcvcnmquec
`Dale :1 Iaquv.-ll: la rudaudxc Intcnuuan/1|: a cue cl'i'I:cnv:m=n1 achcvte
`
`u|R;:rnanunRI an is
`'1‘ dncungent. umtricur ppflie apn‘.-A Ia dale
`‘E m:':“, p°u:'.::‘m’”n'::.¢ ,2, mm“
`ou la Sienna canmnnm la base de l'irwenu'on
`.x. document pamu_'mumm|_ Pemnmu rmvmuon mvmdlquu M pun
`we cwnsdcree comm: muvelie cu comm: mpllquantuna ncuvate
`mug;-mug par fgppuflau docnmgm mnndere gsoltmull
`‘Y’ document. pauuzlmumnlptrtmmu l‘1nw.'nt1on tcvu-Iduquée
`m: peux. cu-c cansdcrtc nornmc ampiiquam um aclmu‘. trwcnuw.
`Iorsque It docurnent an assault a 11.11 on plustcun auures
`docmnnu dc meme nalure. eem: cumhinamon euml. cndcnlc
`W” "" ""°""‘ ‘” ‘Mm’
`'&.' do-curmmt qul fan: pan: d: In meme fan-ullc dc hrcvels
`Data fl'€lp¢fll.lOli du prtscnt rapport dc recherche Inlarnauonalc
`
`4 Mai 1994
`Norn cl. ad:-ca: postal: dc 1'u1rrnnI:n-anon aha:-gen de la n:d1e|-aha mumahcmal:
`Officu Eu:-optcn nus Brcvuu, P.B. SSH! Palcndaan I
`NI. — 2230 HV Rllswljk
`'l‘d-(+1I-?lJ}1!D-201-0.Tx.3I 6Sl eponlu
`Fa.r.-.(+3|-70) 340.3015
`Fnnnulnirt PCTRSNIIO llauuino Iiuillcl lllllllll 1592)
`
`Fmmcmnun ulborlse
`
`'
`Ber'w11:z, P
`
`1 1. 95- 94
`
`page 1 de 2
`
`558
`
`

`
`RAPPORT DE RECHERCHE INTERNATIONALE
`
`Dem mmm.~.
`
`PCT/FR 94/00037
`
`Idcnlilicannn du document: cm‘.-x, avnc, I: can £x:l'IEanL I'indu:a.I:I0n I13 -passages pzmntnls
`IEEE TRANSACTIONS ON COMMUNICATIONS
`
`5 , Mai 1978 , NEW YORK US
`V01. 26, no.
`pages 594 - 600
`JOHNSTON ET AL.
`Sequence Store‘
`Iigne 17
`vair page 595, colonne de droite,
`- page 59?, colanne de gauche. Tigne 35
`
`‘A Digita1 Te1evision
`
`16TH
`SYMPOSIUM RECORD, BROADCAST SESSIONS;
`INTERNATIONAL TV SYMPOSIUM, 17 Juin 1989 .
`MONTREUK, CH
`pages 289 - 295
`NOODHAM ‘A So1id State “Action Replay"
`Recorder‘
`voir le document en entier
`
`Fnrmllaén PCTHSNJIII units do II dsumémn Ioullllniuiilul I993)
`
`page 2 de 2
`
`559
`
`

`
`uproar DE RECHERCHE INTERNATIONALE
`
`'“‘°“"“‘°““"”°
`”""
`PCT/FR 94/onus?
`
`Docurnnnl hrevcl ciu‘:
`a.u rapport dc recherche
`
`Dan: du:
`publicuian
`
`Membrqs) dc Ln.
`farnille tie brevcqs}
`
`Date de
`pubficaliun
`
`EP-A-0279549
`
`24-08-88
`2201314
`GB-A-
`10-02-94
`3886563
`DE-D-
`18-D8-89
`1205672
`JP-A-
`U2-01-90
`4891715
`US-A-
`__————______... ...-....———-...-...——__.......__..____ __...____—__—_——u-——— u--——_....____—.._-_
`
`24-08-88
`
`US-A-5134499
`
`28-07-92
`
`JP-A-
`JP-A-
`
`2044569
`2051859
`2060382
`
`2058984
`5018014
`
`14-02-90
`01-03-90
`28-02-90
`
`28-02-90
`21-D5-91
`
`Fommma PCT.-'!5N:lllI (mun Innmu do Iuwuu) uuhlli I992)
`
`560
`
`

`
`PCT
`
`WORLD IWIELLECIUAL PROPERTY ORGANIZATION
`Internarional Bureau
`
`INTBRNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT)
`
`(51) International Patent Classification 6 :
`
`(11) International Publication Number:
`
`W0 9533336
`
`H04N 51926, 51765, 7126
`
`(43) International Publication Date:
`
`7 December 1995 (m.12.9s)
`
`(21) [ternntional Application Number:
`
`PCTJUSQS/05951
`
`(81) Designated States: JP, KR, European patent (AT, BE. CH, DE.
`DE, ES. FR. GB. GR, IE. IT. LU. MC. NI... PT. SE].
`
`(22) International Filing Date:
`
`18 May 1995 08.05.95)
`
`Published
`With international Search report.
`Before the expiration of the time line‘: for amending the
`claims and to be republished in the even: of the receipt of
`atmendrnentr.
`
`{.30} Priority Data:
`OB.’250.!34
`
`26 May 1994 (2635.94)
`
`US
`
`(Tl) Applicant: H'U'GI-[ES AIRCRAFT COMPANY [US/US]; ‘T200
`Hughes Terrace. Los Angelee, CA 90045-0066 (US).
`
`(72) Inventors: YANG, Chao-Kong; 9041 Santiago Drive. Hunt-
`ington Beach, CA 92646 (US). W]1.LIAMS, Jim. (2.; 1755
`North Partridge Street. Anaheim. CA 92806 (US). ICRUT-
`SICK. Stanley; 1540 Domingo Rood, Fullerton, CA 92633
`(US).
`
`('74) Agents: GRUNEBACH, Gcorgann, S. et aI.; Hughes Aircraft
`Company. 72130 Hughes Tenace, Los Angeles, CA 90045-
`0066 (US).
`
`(54) Title: HIGH RESOLUTION DIGITAL SCREEN RECORDER AND METHOD
`
`(5?) Abstract
`
`A machine independent high resolution digital screen recorder (12) is disclosed for providing high quality video displays with
`manageable storage capacity and bandwidth. The screen recorder includes an analog to digital frame grabber (24) for converting a high
`resolution video slgna1(2D) that modulates a video display ([4) into RG3 sequences of digital frames (30). A video compression unit (26)
`separates the high and low variance portions of the digital frames, encodes them with respective losing (56) and lossless (54) compression
`algorithms and stores them in a mass storage device (23).
`
`561
`
`

`
`FOR THE PURPOSES OF LNFORADIIYON ONLY
`
`Codes used to identify States party In the PCT on the from pages of pamphlets publishing international
`a.ppIiI:a1.io:13 under the PCT.
`
`Ausu-in
`Australia
`Barbados
`I-cigiurn
`nwiina Fm:
`Bulgaril
`Benin
`Brazil
`Beiarnn
`Chanda
`(':nIml Ah-ican Republic
`Cons:
`Switzulsnd
`cue d'Ivoire
`
`AT
`AU
`BB
`BE
`
`§fi3E§E§B&§9§8f&95i'f='33E
`
`United Kingdum
`Georgia
`Guinea
`
`
`
`§§§§§EEE5fi§§§E‘?=“"=’ES%"§5-9
`
`562
`
`

`
`WO 95133336
`
`I'CTfUS95f05961
`
`HIGH REBOLUTIOH DIGITLL SCREEN RECORDER LED KETKOD
`
`ACKG
`
`D
`
`0
`
`Field of the Invention
`
`The present invention generally relates to the field
`
`of screen recorders, and more specifically to a high reso-
`
`lution digital screen recorder and recording method that
`
`converts an analog RGB video signal into a compressed digi-
`tal signal.
`
`8
`
`Conventional color televisions and display monitors
`
`produce relatively low resolution and bandwidth red, green
`
`and blue (RGB) video signals, e.g. 400-600 lines per.frame
`
`at 25-30 frames per second, that conform to one of several
`international standards.
`Three of the most common stan-
`
`dards are: the National Television Systems Comittee (NTSC)
`
`standard with 525 lines per frame at 30 frames per second,
`
`the Sequential chrominance signal 5 Memory (SECAH) standard
`
`with 625 lines per frame at 25 frames per second and the
`
`Phase Alternating Line (PAL) standard with 625 lines per
`
`frame at 25 frames per second. These standards are fairly
`
`flexible in that the number of scan lines actually used by
`
`a display may be significantly less than the specified sta-
`
`ndard.
`
`For these low resolution signals, video recording
`
`devices,
`
`such as video cassette recorders (vans),
`
`record
`
`the analog signal directly onto a magnetic tape.
`
`For high resolution workstation monitors having 1280
`
`or greater scan lines per frame, or high definition televi-
`
`563
`
`

`
`wo 95133336
`
`PCTfUS95lfl5961
`
`sion (HDTV),, it is not currently feasible to record the
`'analog video signals directly onto the tape in an analog
`
`format at the same high resolution.
`
`The storage require-
`
`ments and bandwidth of the video signals exceed the capa-
`
`bilities of analog recording devices. For example, a 1280
`
`x 1024 video signal at 3 bytes per pixel and 30 frames per
`
`second recorded for 3 hours would require 1274 Ghytes of
`
`memory and a bandwidth of 943 ups. As a result,
`
`in a cur-
`
`rent approach a scan converter is used to convert the high
`
`resolution video signal into one of the low resolution sta-
`
`ndard formats, which is recorded onto the tape.
`
`The down
`
`conversion is done by averaging adjacent scan lines or sim-
`
`ply skipping scan lines, and results in a substantial loss
`
`of resolution when the recorded tapes are played back. The
`
`"RGB/Videolink 16000“, RG3 Spectrum of Berkeley, Califor-
`
`nia, 1991 is described in a product bulletin as one example
`
`of a scan converter. The loss of resolution is particular-
`
`ly bothersome in a multimedia workstation, a typical dis-
`
`play includes text, graphics, pictures, audio and motion
`
`picture video data.
`
`The graphic icons and text are dist-
`
`orted hy converting the high resolution signal to the rela-
`
`tively coarse low resolution signal. This system sacrific-
`
`es image quality to maintain manageable storage require-
`
`ments and bandwidth.
`
`For applications where the replayed
`
`information is required to faithfully reconstruct events,
`
`e.g., air traffic control displays or is critical to liti-
`
`gation,
`able.
`
`the loss of dielity may make the recording unuse-
`
`OF
`
`HVENT ON
`
`The present invention seeks to provide a machine inde-
`
`pendent, high resolution digital screen recorder that pro-
`
`vides high guality video displays with manageable storage
`
`capacity and bandwidth, and can be implemented as a stand-
`
`alone or integrated unit.
`
`564
`
`

`
`W0 9533336
`
`PCTI'US95!0596l
`
`This is accomplished with a screen recorder for re-
`' cording a high resolution analog multi-color video signal,
`
`preferably RGB, that modulates a video display. An analog
`
`to digital frame grabber converts the high resolution video
`
`signal
`
`into sequences of digital
`
`frames.
`
`A video com-
`
`pression unit encodes the sequences of digital frames into
`
`compressed digital signals and stores them in a mass stor-
`
`age device.
`
`In a preferred embodiment, a high resolution multime-
`
`dia computer workstation displays high variance (motion
`
`picture video) and low variance (graphics, text,
`
`icons and
`
`background) data and produces the video signal to modulate
`
`its display. The video compression unit separates the high
`
`and low variance portions of the digital frames and encodes
`
`them with lossy and lossless compression algorithms, re-
`
`spectively.
`
`For a better understanding of the invention, and to
`
`show how the same may be carried into effect,
`
`reference
`
`will now he made, by way of example,
`
`to the accompanying
`
`drawings.
`
`EBIEI_DE§£RIEI1QE_QE_IEE_DBAflIH§fi
`
`FIG. 1 is a block diagram of a high resolution comput-
`
`er workstation and a digital screen recorder:
`
`FIG. 2 shows a typical multimedia display:
`
`FIG. 3 shows a difference frame for the display of
`FIG. 23
`.
`
`FIG.
`
`4 is a flowchart of a hybrid video compression
`
`algorithm:
`
`FIG. 5 is a flowchart of the lossless compression al-
`
`gorithm:
`
`FIG. 6 is a flowchart of the lossy compression algo-
`
`rithm:
`
`FIG. 7 is a flowchart of the video window detection
`
`35
`
`algorithm:
`
`565
`
`

`
`WO 95133336
`
`PC'I'fUS95!lJ596l
`
`FIGs. 8a-Bd illustrate the steps of the detection al-
`
`gorithm of FIG. 7; and
`
`FIGs. 9a-9d show typical video windows, their projec-
`
`tions and corresponding transition codes.
`
`DE1AILED_DE§QBIEIIQE_QE_IHE_IEEEEIIQH
`
`High resolution multimedia computer workstations oper-
`
`ate in a windowing environment that consists of several
`
`text and graphics windows, a motion video window,
`
`icons, a
`
`cursor and possibly audio signals, and in some applications
`
`live sensor images such as radar may also be present.
`
`A
`
`typical display has a resolution of at least 1280 x 1024
`
`pixels and is refreshed at a rate of 30 frames or higher
`
`per second, although the invention is applicable for arbi-
`
`trary resolutions and frame rates.
`
`FIG.
`
`1 shows a high
`
`resolution multimedia computer workstation 10 that is elec-
`
`trically conneoted through a screen recorder 12 to a moni-
`
`tor 14. Typical workstations have a single video output
`
`for the monitor, and thus to display the video signal
`
`in
`
`real time and store it as a compressed digital signal for
`
`future playback,
`
`the monitor and screen recorder are con-
`
`nected in a 'loopthrough" configuration that is similar to
`
`a TV/VCR connection. The workstation internally generates
`
`high resolution digital RG3 video frames 16 in a machine
`
`dependent format in response to the windowing environment,
`
`graphics, text or motion video applications and uses a D/A
`
`converter 18 to convert them into an analog RGB video sig-
`nal 20.
`The frames‘ resolution and rate are included as
`
`sideband information in the video signal. The analog video
`
`signal 20 modulates the intensity of a cathode ray tube
`
`(CRT) 22 in the monitor 14 to raster scan the RGB images 16
`
`onto the display at the given frame rate.
`
`The screen re-
`
`corder 12 captures the video signal 20 and stores it as a
`
`compressed digital signal.
`
`The screen recorder can be a stand-alone, workstation
`
`566
`
`

`
`WO 95733336
`
`PCTIUS95!0596l
`
`independent unit as shown, or it can be integrated into the
`
`workstation.
`
`In the preferred embodiment,
`
`the screen re-
`
`corder is connected to only video and/or audio outputs of
`
`the workstation's digital display system and does not re-
`
`quire any information from internal components of the dis-
`
`play. Thus,
`
`the invention is not dependent on the imple-
`
`mentation of any specific display and may be applied to a
`
`variety of devices.
`
`The screen recorder includes a high
`
`resolution frame grabber 24, a digital video compression
`
`unit 26 and a mass digital storage device 28.
`
`The frame
`
`grabber converts the analog RGB video signal 20 in real
`
`time into another sequence of RGB digital frames 30 in a
`
`machine independent
`
`format.
`
`A suitable frame grabber,
`
`is described in a new product bulletin from
`HI*DEF 111,
`IHAGRAPH of chelmsford. Hassachusetts and claims a 160 Hz
`
`bandwidth for capturing standard and non-standard video
`
`signals with up to 16K x 16K resolution.
`
`The compression
`
`unit 26 compresses the video frames 30 at a ratio of app-
`
`roximately 25u:1 with only slight visual degradation and
`
`maintains reasonable bandwidths and memory requirements.
`
`The compressed digital video signal is stored in the stor-
`
`age device 28, which can be a conventional hard drive, an
`
`optical drive or a portable device such as a digital tape.
`
`In the invention, the multimedia display is separated
`
`into high and low variance portions for the purpose of com-
`
`pressing the digital video signal 30.
`
`The high variance
`
`portion 32 is defined by a video window_34 in which some
`
`type of motion picture video signal is being played, while
`
`the low variance portion 35 is everything else: graphics
`
`36, text 38,
`
`icons 40 and the background 42.
`
`FIG. 2 shows
`
`a typical multimedia display and FIG. 3 shows the differ-
`
`ence between successive displays. As shown in these fig-
`
`ures,
`
`the high variance data 32 and low variance data 35
`
`are differentiated by a stark disparity in their temporal
`
`(interframe) and spatial
`
`(intraframe} variances. Because
`
`567
`
`

`
`W0 95133336
`
`PCT!US95l0596l
`
`the motion picture video data's spatial and temporal vari-
`
`ances are relatively high, at a given encoding rate the
`
`error in its reconstructed image will be greater than the
`
`reconstruction errors in the graphics,
`
`text,
`
`icons and
`
`background data. However, motion picture {high variance)
`
`imagery can be compressed such that encoding errors only
`
`slightly distort the image's visual quality or are com-
`
`pletely imperceptible. Conversely,
`
`the low variance data
`
`is highly correlated temporally and spatially and is thus
`
`easy to compress. but any encoding errors are easily visi-
`
`ble and distort the fine structure of the icons, graphics
`
`and text.
`
`Improved overall compression and reconstructed
`
`image quality is achieved by using separate lossy and loss-
`
`less encoding algorithms for the high and_low variance da-
`ta, respectively.
`In a perfect lossless algorithm the de-
`
`compressed images equal the digital images 30 provided by
`
`the frame grabber without distortion or error, while in a
`
`lossy algorithm the decompressed algorithm has some amount
`
`or distortion or error relative to the original.
`
`FIG. 4 is a flowchart of the hybrid video compression
`
`algorithm. To achieve the necessary speed, the video com-
`
`pression unit 26 implements the algorithm in hardware, al-
`
`though future advances in computer speed may allow a soft-
`
`ware implementation.
`
`In step 44, the frame grabber 24 pro-
`
`vides the compression unit with 24-bit RGB images 30 at 8
`
`bits per pixel for each frame extracted from the video sig-
`
`nal 20.
`
`In the next step 46,
`
`the capture errors of the
`
`frame grabber are eliminated by masking off a number of the
`
`least-significant-bits (lsbs), e.g., one lsb per image pix-
`el.
`Each successive set of I-163 frames 30 is then transf-
`
`ormed in step 48, using a YUV transform to produce '1 lumi-
`
`nance and UV chrominance components 49. The YUV transform
`
`is disclosed on pages 14 and 17 of a new product infome-
`
`tion sheet "Programable color space converter and color
`
`Corrector“, Brooktree corporation of San Diego, California.
`
`568
`
`

`
`W0 95133336
`
`PCT1'US95!0596l
`
`1990.
`
`an alternative or slightly modified transform denot-
`
`ed generally as an xrz transform could be used to provide
`
`an x intensity (luminance) component.
`
`In the next step 50,
`
`the 1!, U and V digital
`
`frames are subtracted from their
`
`respective successive YUV frames to form YUV difference
`
`images 51, which are integer valued and can be represented
`
`exactly by a digital codeword.
`
`In step 52,
`
`the video win-
`
`dow's boundary coordinates are extracted from the Y differ-
`
`ence image (see FIGs. 7-9 for details} and passed to the
`
`lossless and lossy compression algorithms.
`
`The window's
`
`boundary coordinates are also included in the sidehand in-
`formation 53 sent to the decoder. The video window can be
`
`computed for each difference image or for some period in
`
`accordance with the requirements of the specific compres-
`sion algorithms.
`
`In step 54 the low variance portion of each successive
`
`digital frame is compressed using a lossless algorithm (see
`
`FIG. 5 for details}, and in step 56 the high variance por-
`
`tion is encoded using a lossy algorithm (see FIG. 6 for
`
`details); these two steps produce respective bitstreams.
`
`The accompanying audio track is compressed in step 58 and
`
`its bitstream is multiplexed with the compressed high and
`low variance video data and sideband information bitstreams
`
`in step 60.
`
`In step 62, the multiplexed digital data 63 is
`
`written to the storage device 23.
`
`To playback the digitally stored video signal, a de-
`
`coder 64 demultiplexes the data 63, performs the respective
`
`inverse transforms on the compressed high and low variance
`
`signals and audio signals and adds the decoded images back
`
`together to produce a sequence of reconstructed digital
`
`images 65.
`
`A D/A converter 66 converts the reconstructed
`
`images 65 into an analog RGB video signal 67 to playback
`
`the stored multimedia session. Depending on the disp1ay's
`
`content and the size of the video window,
`
`the compression
`
`algorithm can realize compressions of approximately 250:1
`
`569
`
`

`
`W0 9533336
`
`PCT:’US95f0596l
`
`with no distortion in the low variance portions and only
`
`slightly perceptible distortion in the high variance motion
`
`picture portion.
`
`FIG. 5 illustrates a preferred embodiment of the loss-
`
`less compression algorithm for encoding the multimedia dis-
`
`play's low variance portion 35. The lossless algorithm is
`
`initiated by encoding the first YUV digital frames 49 and
`
`thereafter encoding the YUV difference frames 51 computed
`
`To decode the sequence of compressed frames,
`in step 50.
`the first frame must be transmitted so that the successive
`
`frames can be recursively reconstructed by decoding the
`
`next difference frame and adding it to the previous frame.
`
`It may be desirable to reset the encoding algorithm period-
`
`ically by directly coding a YUV frame every n frames, e.g.,
`200 frames, to prevent error propagation due to noise.
`
`Instead of completely removing the pixels in the video
`
`window and only encoding the low variance data pixels in
`
`the lossless algorithm, those pixels inside the video win-
`
`dow can be set equal to zero and encoded with the low vari-
`
`ance data. Since the pixels inside the window are all set
`
`to a constant value, their information content and effect
`
`on the compression of the frame is negligible. Alterna-
`
`tively, the windowed pixels could he removed and the algo-
`
`rithms modified to keep track of the video window. This
`
`approach. could provide slightly better compression. but
`would be substantially more complicated.
`
`In step 68 of FIG. 5, the algorithm checks the reset
`condition and selects either the YUV frames 49 or the dif-
`
`ference frames 51.
`
`In the former case,
`
`the pixel values
`
`for the respective YUV frames 49 inside the video window 34
`
`are set to zero (step 69) and each frame is compressed with
`
`the Joint Bi—level Image Group (JBIG) algorithm (step 70)
`
`to produce the encoded bitstream.
`
`The JBIG algorithm is
`
`described in the September 16, 1991 "CCIT Draft Recomenda-
`
`tion T.82 ISO/IEC committee Draft 11544 coded nepresenta-
`
`570
`
`

`
`W0 95!33336
`
`PCTIU 59510596 1
`
`tion of Picture and Audio Information-Progressive Bi-level
`
`Image Compression“. when the difference frames 51 are se-
`
`lected, the respective YIN difference frames‘ pixels inside
`
`the window are set to zero {step 71) and the frames are
`
`encoded using run length codes (step 72) to produce another
`
`bitstream.
`
`Run length codes are described in Jayant and
`
`Roll. Prentice-Hall, “Digital Coding of Waveforms", 1934,
`
`pp. 465-485.
`
`other lossless algorithms could be employed
`
`in place of the JBIG and run length codes without departing
`
`from the scope of the invention.
`
`FIG. 6 is a flowchart of the lossy compression algo-
`
`rithm for the high variance portion 32 of the multimedia
`
`display. The Motion Pictures Expert Group (HPEG) algorithm
`
`described in "Information Technology - Generic coding of
`
`Moving Pictures and Associated Audio, Recommendation H.262,
`
`ISO/IEC 13818-2", November 25, 1993 is preferably used to
`
`compress the data inside the video window and the audio
`
`track, although other video compression algorithms are also
`
`applicable.
`
`KPEG reads 16 x 16 blocks of pixel data and
`
`processes groups successive frames, e.g. , 12-15. To accom-
`
`modate the standard,
`
`the coordinates of the video window
`
`are updated for the first image of each group (FIG. 2, step
`
`52) and modified in step 74 to extend the window to fit the
`
`HPEG block size.
`
`In step 75, the YEN images 49 are conv-
`
`erted into a ham format to be compatible with the MPEG
`
`algorithm. For every 2 x 2 block of pixels the Y luminance
`
`value is the value of each pixel and the U and V values
`
`equal the average of the four pixels from their respective
`
`frames.
`
`In step 76, the 4:2:0 formatted pixels inside the
`
`extended window are encoded with the MPEG algorithm to pro-
`
`duce the bitstream. The decoder strips out the additional
`
`pixels prior to reconstructing the display.
`FIG. 7 is a flowchart of a suitable video window de-
`
`tection algorithm, which utilizes the standard rectangular
`
`geometry of the workstation's windowing environment and the
`
`571
`
`

`
`W0 9533336
`
`PCTfUS95.'0596l
`
`contrast in signal characteristics between the motion pic-
`
`ture and graphics, text,
`
`icons and background portions of
`
`the display.
`
`In step 78.
`
`the algoritlm receives the Y
`
`difference image. An activity measure. e.g. , the variance,
`
`is computed in step so for each row and each column, with
`
`the variances 81, B2 projected onto the vertical and hori-
`
`zontal axes,
`
`respectively.
`1:
`column is given by: o‘=%; (11-m)‘ where 0‘ is the variance,
`-1
`
`The variance of each row or
`
`In is the mean and n is the
`1, is the pixel luminance value,
`number of pixels in the row or column.
`The mean In of the
`
`difference images
`
`is normally approximately zero.
`
`The
`
`variance projections have sharp transitions at the video
`
`window's edge coordinates 83, 84, 85, 86 and at the interi-
`
`or coordinates 87, as where the window's width with respect
`
`to the particular axis changes, as shown in FIG. 8a.
`
`The
`
`coordinates of a bounding rectangle 89 for the video window
`
`34 are extracted from the first (33.85) and last (34.35)
`
`transitions of each projection (step 90) and specify the
`
`size and location of the window, as shown in FIG. 8b and
`
`denoted by a "0".
`
`In step 91, the interior row transition
`
`coordinates (87) are matched to the interior column transi-
`
`tion coordinates (88) to specify the interior corner points
`
`92, as shown in FIG. ac and denoted by an "X".
`
`To identify the window's shape, the projections‘ tran-
`
`sitions are coded (step 93) with a 10-digit transition code
`94. The cocle's format restricts the number of transitions
`
`to five or less in each projection 81 and 82: any projec-
`
`tion with more than five will default to the bounding rect-
`
`angle. A transition from low to high is indicated by a '1“
`
`and from high to low as a "0". Since the first transition
`
`of each projection is always a "1" and the last is always
`
`a "0", only the interior transitions are coded. The first
`
`two flags specify the number of row projection interior
`
`transitions (0-3) , flags 3-5 specify the transitions, flags
`
`572
`
`

`
`W0 95133336
`
`PCTI'US95!05961
`
`6-'7 specify the number of column. projection interior tran-
`
`sitions and flags 8-10 specify the transitions.
`
`any unused
`
`flags in positions 3-5 and 8-10 are designated by an "X".
`
`For example, a simple rectangular window would be coded
`"O0X.'!D{OO)DEX".
`
`In step 95, a look-up-table (LOT) which contains the
`
`codes for a number of common shapes outputs a shape identi-
`
`fier 96 and a set of instructions 97 that specify how the
`
`interior corner points 92 are used to define the video win-
`