Ulllted States Patent
`
`[19]
`
`[11] Patent Number:
`
`5,903,261
`
`Walsh et al.
`
`[45] Date of Patent:
`
`*May 11, 1999
`
`US005903261A
`
`........................ .. 395/162
`8/1994 Harney et al.
`5,335,321
`5,353,062 10/1994 Maeda ........... ..
`.. 348/412
`
`
`
`-- 348/445
`2/1995 K0Wek 61 31-
`.. 348/354
`3/199: Savatier
`.... ..
`~~
`fig“-V6“ t~~~~t-~~~1~~~
`6/1995 D:Il1fr]iagIer:t:l. ‘I..................2 395/325
`(List continued on next page.)
`
`5,387,940
`5,400,075
`
`5,428,751
`
`OTHER PUBLICATIONS
`International Search Report.
`
`Primary EXaminer—Richard Hjerpe
`Assistant Examiner—Ricardo Osorio
`
`Attorney, Agent, or Firm—Fish & Richardson P.C.
`_
`V ‘
`V ‘
`[37]
`ABSIRALI
`A computer based system for displaying and compressing
`video including a video capture card with a video compres-
`sor and a bus interface circuit that acts as a busmaster and
`
`outputs uncompressed video and compressed video to a
`computer bus for display of the uncompressed video on the
`computer monitor and storage of the compressed video on a
`memory of the computer. The system also includes a soft-
`Ware Vlmlal lmemlpt generamr thffi 11595 tlmer CVCIHS PTO‘
`Y1d?d by a Computerlsyslem Servlc‘? .and a transfer Slams
`indicator to generate interrupts to initiate transfer of a new
`block of video; an overlay controller implemented in soft-
`ware that transfers video from the video capture card over
`the gomputer bus to a graphics subsystem forldisplaydin a
`win ow on the computer monitor in an over ay mo e; a
`display controller inipleinented in software that causes dis-
`P133’ Of ““C°mPreS5ed V1990 from the Video Capture Card 0r
`Software deC0mi>ressed Vldeo;
`21 softwareeeontroller that
`compresses audlo 1n software and sends Vldeo data to be
`compressed across the computer bus to the compressor; a
`controller that
`is implemented in software and calibrates
`startup delay of the audio input subsystem and uses the delay
`to synchronize the audio and video; and a user interactive
`input mechanism for adjusting the rates of compression
`.th.
`f
`t bl
`t
`th t
`.
`fu
`t.
`f
`‘:1 “””a“g°° “CC? a E” cs
`a Varécsiisa
`“°1°“°
`t e output target me ium or compresse vi eo.
`
`8 Claims, 12 Drawing Sheets
`
`[54] COMPUTER BASED VIDEO sYsTEM
`
`[75]
`
`Inventors: Bruce E. Walsh, Londonderry, NH.;
`John Herdrich, Holden’ Mass‘;
`William Smith, Fitchburg, Mass.;
`Mark E‘ Vmbel’ Somhbridge’ Mass‘;
`Philip Borghesani, Acton, Mass.;
`Christine G. Hagberg, Hopkinton,
`Mass; Karen Champagne, Millbiirry,
`M ass.
`
`_
`.
`[73] Assignee: Data Translation, Inc., Marlboro,
`Mags,
`
`[*] Notice:
`
`This patent issued on a continued pros-
`ecntion application filed under 37 CFR
`1.53(d), and is subject to the twenty year
`patent
`term provisions of 35 U'S'C'
`154(a)(2)'
`
`[21] App1.N0.I03/555,950
`[22]
`Filed:
`Jun‘ 20’ 1996
`
`Int, CL5 ..................................................... ., G09G 5/00
`[51]
`[52] U.S. Cl.
`........................................... .. 345/302; 345/418
`[58] Field of Search ................................... .. 395/806, 807;
`382/235’ 236; 345/302’ 418; 348/423’ 8453;
`352/12
`
`References Cited
`U~S~ PATENT DOCUMENTS
`6/1985 Gueldenpfennig et al.
`............ .. 370/62
`4,521,879
`8/1991 Yoiieiiiitsu ............................ .. 358/135
`5,040,061
`9/1992 Ng ......................................... .. 358/135
`5,146,325
`3/1993 Yonemitsu
`358/335
`5,191,436
`5/1993 Normile et al.
`.. 382/56
`5,212,742
`11/1993 Dixit
`~~~~~~~~~~~ ~~
`358/135
`5,250,783
`5367334 “£1993 Nmmflle 6‘ ‘*1
`~- 382/'56
`5’287’178
`21994 Acampora et al‘
`348/384
`5,293,229
`3/1994 Iu ................... ..
`348/415
`5,301,242
`4/1994 Gonzales et al
`.. 382/56
`5,319,793
`61,1994 HanC0Cketa1_ __
`395/807
`5,327,248
`7/1994 Miller et al.
`..
`. 358/261.4
`5,329,365
`7/1994 U2 ......................................... .. 348/469
`
`
`
`
`
`[56]
`
`OPERATING SYSTEM
`
`
`
`VIDEO
`
`CAPTURE
`DRIVER
`
`
`
`DIGITAL
`VIDEO
`CARD
`RPX Exhibit 1043
` RPX Exhibit 1043
`RPX v. DAE
`RPX V. DAE
`
`12
`
`156
`
`28
`
`

`
`...................... .. 382/304
`10/1995 Normile et al.
`11/1995 Liohtbody etal.
`. 395/157
`‘’
`,
`2:’e1t‘t“““‘
`‘3379(;f9249'(1,
`/
`” 9‘ """"""""""""""""""" "
`>
`9
`4/1996 ”°'"“°/S ‘*1 a'-
`---------------------- -- 395/2-14
`5»505932
`4/1996 Rossmm 61 a1~
`~~~~~~~~~~~~~~~~~~~~ ~~ 395/806
`59508940
`8/1996 Hoarty eta].
`............................. .. 348/7
`5,550,578
`5,583,652 12/1996 Ware ..................................... .. 395/806
`
`
`
`5,461,679
`5,471,577
`
`5,903,261
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5,430,847
`5,438,663
`5,442,747
`5,444,575
`5,446,869
`5,443,310
`5,455,915
`
`....................... .. 395/325
`7/1995 Bradley et al.
`8/1995 Matsumotoetal.
`395/162
`
`8/1995 Chan etal.
`395/164
`8/1995 Augenbraun et al.
`.................. .. 360/64
`8/1995 Padgett et al.
`........................ .. 395/500
`
`.
`9/1995 Kopetet a1,
`,, 348/699
`10/1995 Coke ..................................
`395/325
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 1 of 12
`
`5,903,261
`
` 1/
`
`FIG.1
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 2 of 12
`
`162,309’5
`
`N_,
`
`E
`
`3
`
`Nv
`
`&_mam
`
`PE
`
`
`
`mo__._n_<mom_wn__mm_<m_-_on_
`om<oxma
`
`
`cmAomhzoo
`
`mam_On_Imam<m_
`
`omNN
`
`in
`
`>mos_m__>_
`
`n_m>z_>:>own.2.omo_>
`
`
`n_m<o_SoImmazzomgmmoooma
`
`mm\mm
`
`mm
`
`mmvm._.DOmmx<mn_m
`
`..20z_o_o:<
`
`3
`
`motzoz
`
`N.0_u_
`
`3>mo_2m__2
`
`xma
`
`
`

`
`tHEtaP&U
`
`May 11, 1999
`
`Sheet 3 of 12
`
`162,309’5
`
`m.0_u_
`
`mmu_”5m_
`
`9.
`
`mm
`
`mm
`
`mam_0n__
`
`__mmm_mon_<
`
`mmfizaoo
`
`mmFm<_>_
`
`mm8/
`
`
`
`.6528_On_3.
`
`
`
`momm._.n_=.._m
`
`._._mmmxv
`
`mmwzaoo
`
`___mum3oror
`
`89»mm
`
`_mum
`
`mmfizaoo
`
`
`
`
`
`n__mamomn___>ommmmmmzoo
`
`mmmmooomo
`
`
`
`
`
`s_oEwz_s_:omo_>omn__>>:>
`
`mm:m<s_
`
`
`
`vmbfim9:”.wF._oEzou_._o:>>m
`
`om
`
`Jomhzoo
`
`moi“.
`
`
`
`>Es_m5os_._<o”__u_
`
`mmfizsoo_wwmmoo<
`
`mm
`
`mm
`mm_5<s_<.EoowI
`
`
`
`
`
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 4 of 12
`
`5,903,261
`
`70
`
`APP
`
`70
`
`App
`
`154
`
`OPERATING SYSTEM
`
`72
`
`133
`
`VIDEO
`
`CAPTURE
`DRNER
`
`CODEC
`
`DRIVER
`
`RECODE
`DRIVER
`
`
`
`
`CARD
`
`HARDWARE INTERFACE
`
`135
`
`VIDEO
`CAPTURE
`
`12
`
`FIG. 3A
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 5 of 12
`
`5,903,261
`
`SYSTEM TIMER
`
`SCHEDULE TIMER
`
`I NTE RRU PT
`GENERATOR
`
`VIRTUAL
`
`WTERRUPT
`
`OVERLAY
`CONTROLLER
`
`HARDWARE
`CONTROLLER
`
` VIRTUAL
`
`VIDEO CAPTURE DRIVER
`
`TRANSFER DONE
`
`/142
`
`FIG. 3B
`
`FIG. 3C
`
`144
`
`SYSTEM TIMER
`
`__L___L__J_#
`
`VIRTUAL INTERRUPT
`
`142
`
`I
`
`144
`
`SYSTEM TIMER
`
`-
`
`FIG. 3D
`_J___L%_I_
`
`VIRTUAL INTERRUPT
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 6 of 12
`
`5,903,261
`
`><._mm>o
`
`>>on_z_>>
`
`m_o.:zos_
`
`
`
` om<om_3.mo_:n_<mo
`
`m_o<u_mm:.z_
`
`mo_:n_<mo
`
`><._mm_>o
`
`m_mu_n_:m
`
`omo_>
`
`mmooomo2
`
`Z_om_o_>
`
`E<moi".
`
`v.®_u_
`
`mobomzzoo
`
`mo_:n_<mo
`
`m~_:.2m"_
`
`oorZO_.._.<O_-_n_n_<
`
`E9
`
`NE.SEmm>_mo
`
`u=_mam
`
`_On_
`
`mam
`
`
`
`
`
`o~_<ommm:E<oOm_n=>
`
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`99911,1y3M
`
`.3
`
`ivnu9’
`
`11c2,
`
`manmo_:n_<mw
`
`782>w93om<om
`
`E<mO_m_n_
`
`
`
`<_..O_u_
`
`E9_o2zo:<o_._%_<
`
`esemm_>_mo
`
`n_\_mam
`
`><._mm>o
`
`
`
`nmm_n_u_:mmmooomo
`
`>>On_Z_>>
`
`motzoz
`
`mo<“_mm»z_
`
`mo_:n_<mo
`
`
`
`om<ommm:E<oomn__>
`
`
`
`
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 8 of 12
`
`5,903,261
`
`m.0_u_
`
`><._~_m>o
`
`>>On_2_>>
`
`motzos_
`
`NF
`
`E
`
`2
`
`s_m._m>m
`
`>mos_ms_
`
`>Smm>o
`
`._<fi:.m_>
`
`mmnmam
`
`
`
`
`
`><#m_>oQmm:E<oomo_>mm._._oEzoo
`
`_.N
`
`as'mm._._oEzooE3mm>_moms
`
`\_On_
`
`mam
`
`1
`
`an
`
`meu_\_mam
`
`Z_9
`
`omo_>
`
`ZO_._.<O_._n_n_<
`
`mm<>>om<:
`
`
`
`
`
`n_N_>._mUoomOomU_>
`
`
`
`
`
`am<omm_m:E<oom_o_>
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 9 of 12
`
`5,903,261
`
`VIEW SOURCE
`126
`
`v w
`SOURCE DURING
`CAPTURE
`
`120
`
`- OVERLAY
`
`124 0 PREVIEW"
`
`0 OVERLAY
`
`FIG. 6
`
`70
`
`70
`
`
`
`131
`
`SW
`
`CONTROLLER
`
`VIDEO CARD
`
`FIG. 7
`
`

`
` U.S.Patent
`
`
`
`mcozaoco_mmmEEoo
`
`
`
`
`
`
`
`>=_m:ccoawmaeooo_u:<a__m:c:o_mmwaEoo82>
`
`
`
`
`
`8.9mE_2.0
`
`
`
` O0._0«W©_0CCm£O_¢:DO
`
`May11,1999
`
`
`
`
`
`
`
`a__§o|9mm.28Emaclnemm9.8
`
`.051--ems.w.291--Ems.9.genEm:
`weoxwe24r__womwm..m.%.§
`
`
`
`_._m_Iu:m_Inm:o_m_._mE_uommE_
`
`mcEmtow
`
`
`
`
`
`ovmxmm©Eases.-.Em_>_toE:_um_>_--Ems.nA__mEwvomFxot0mixme?pEma...
`
`Sheet10of12
`
`
`
`
`
`
`
`<.\..O_u_
`
`
`
`5,903,261
`
`
`
` .Es:um.__:c9_momamxmfi9mE=mm
`
`$99xomn>m_n_
`
`

`
`C4|.aP3U
`
`m
`
`99w
`
`0H
`
`1M
`
`
`
`to_o:<ommmmmazoonFomn_s_
`
`
`
`u,mmx_s_o<m_m_¢o_o:<mo
`
`ys_<m_Emom_n__>z_23.mm
`
`
`
`om.s_<mEm
`
`
`
`n_<m~=¢o_o:<z_mznm
`
`ts_m:w>mmmAWN»w.m_Fn_.¢_.\m__\om_._2<m_Em
`
`
`
`
`
`
`mmommmumn_s_ooH:m_m<>>n_m<_._omo_>o<mm
`
`
`
`n_<mm:»omn_>z_wznm
`
`
`
`
`
`8&2Som_=om_.o_o:<n_<mm_
`
`m:.~.:2.omzomc
`
`EV
`
`
`
`
`
`zo_mmm_mn_s_ooomn__>mmw
`
`
`
`3,mmW,E5,>mosm__>_n_m>.
`
`wOE
`
`
`

`
`U.S. Patent
`
`May 11,1999
`
`Sheet 12 of 12
`
`5,903,261
`
`156
`
`28
`
`OPERATING SYSTEM
`
`VIDEO
`CAPTURE
`
`DRIVER
`
`FIG. 9
`
`

`
`5,903,261
`
`1
`COMPUTER BASED VIDEO SYSTEM
`
`BACKGROUND OF THE INVENTION
`
`The invention relates to a computer based video system.
`Computers have been used to edit and create video
`programs using digitized video. Owing to the large amount
`of data involved in even a single frame of video, it is typical
`to compress the digitized data. One compression format that
`is used is JPEG, which involves compressing the data on a
`frame-by-frame basis, permitting edits to be easily made at
`frame boundaries on the compressed data. Another com-
`pression format that has been used is MPEG, which can be
`used to partially compress the video into so-called I-frames,
`which rely only on the information in a single frame, or to
`fully compress the video into IBP frames, which rely on the
`information in subsequent and prior frames. Full MPEG
`encoding achieves a higher level of compression, though
`edits cannot be easily made on the fully-compressed videos.
`Some computer based video systems have relied on
`software compressors to compress video data while others
`have used compressors/decompressors implemented in inte-
`grated circuit chips.
`In computer based video systems, a hardware frame buffer
`is often connected by a direct hardwire connection to a
`graphics subsystem for display of the frame buffer contents
`in a window on a computer monitor in an overlay mode.
`SUMMARY OF THE INVENTION
`
`In one aspect the invention features in general circuitry
`for displaying and compressing video that includes an input
`port, a bus transmission buffer, and a bus interface circuit.
`The input port receives uncompressed video in real time, and
`the compressor receives the uncompressed video from the
`input port and outputs compressed video. The bus transmis-
`sion buffer is connected to receive either the uncompressed
`video from the input port or the compressed video from the
`compressor. The bus interface circuit controls the inputs to
`the bus transmission buffer and acts as bus master of a bus
`
`of the computer. The bus interface circuit also receives the
`uncompressed video and the compressed video from the bus
`transmission buffer and outputs the uncompressed video and
`the compressed video to the bus for display of the uncom-
`pressed video on the monitor of the computer and storage of
`the compressed video on a memory of the computer.
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations, the uncompressed Video includes frames
`of even and odd fields, and the bus transmission buffer
`receives one of the fields of each frame for output to the bus
`and receives the compressed video during the other field of
`each frame for output to the bus. The bus transmission buffer
`is a FIFO, and a multiplexer controls the input of uncom-
`pressed video or compressed video in response to control
`signals from the bus interface circuit. The multiplexer is
`controlled to latch a portion of the pixels in a line of a field
`of uncompressed video to decimate the uncompressed video
`in a horizontal direction. A decoder receives analog video
`from the input port and converts it to digitized video, e.g., in
`YUV format. The compressor is a video RISC processor,
`and the compressed video is in the form of MPEG I-frames.
`The bus interface circuit controls a first bus master channel
`for the uncompressed video and a second bus master channel
`for the compressed video and includes first and second
`address counters for specifying respective addresses in the
`memory for the first and second channels; the bus interface
`circuit also includes a master control state machine that
`
`5
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`controls the bus transmission buffer to provide the uncom-
`pressed video with an address from the first buffer and the
`compressed video with an address from the second buffer.
`The bus interface circuit also includes a FIFO input control
`that controls the multiplexer to alternatively output
`the
`uncompressed video from the input port and the compressed
`video from the compressor during alternate fields of uncom-
`pressed video The circuitry resides on a peripheral board for
`use in an expansion slot of a personal computer.
`Embodiments of this aspect of the invention may include
`one or more of the following advantages Because the data
`are not stored on the video capture board,
`the memory
`requirements for the board are reduced. The host computer’s
`monitor is used to view the source material avoiding the
`need for an external monitor. The display on the monitor is
`maintained in synchronization with the video being cap-
`tured.
`
`the invention features in general a
`In another aspect
`computer system for handling video that transfers blocks of
`Video to or from a picture buffer without relying on com-
`puter interrupts. A bus interface circuit of the computer
`system is operable to transfer video in blocks to or from a
`picture buffer under the control of a software controller. A
`software virtual interrupt generator uses timer events pro-
`vided by a computer system service and accesses a transfer
`status indicator indicating whether a transfer has been com-
`pleted to generate interrupts that cause the software control-
`ler to initiate a transfer of a new block of video.
`
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations the computer system service that provides
`the timer events is a multimedia timer provided by an
`operating system (e.g., Windows 95) of the computer. The
`picture buffer resides on a system memory on a bus of the
`computer; alternatively the picture buffer can reside on a
`peripheral board in an expansion slot of a personal computer.
`The video being transferred is compressed or uncompressed
`Video, and the blocks are frames or fields ofinterlaced video.
`The bus interface circuit acts as a busmaster; alternatively
`the bus interface circuit acts as a slave. The software
`
`controller and the software virtual interrupt generator are
`part of a video capture driver. In some implementations the
`timer events are scheduled to occur at regular intervals; in
`some other implementations the software virtual interrupt
`generator requests that the timer events occur at intervals
`matched to the rate at which transfers are completed, and
`requests a supplemental timer event a short time after a timer
`event if the transfer status indicator indicates that the trans-
`
`fer has not yet been completed.
`Embodiments of this aspect of the invention may include
`the following advantage. Because blocks of video data thus
`are transferred without relying on CPU interrupts, which are
`limited computer resources in high demand by other com-
`ponents of the computer system, installation of the video
`system on the computer is greatly simplified.
`In another aspect
`the invention features in general a
`computer system for displaying video on a computer moni-
`tor in an overlay mode. The system includes a video capture
`card with an input port receiving video in real
`time, a
`graphics subsystem connected to the monitor, and a graphics
`software driver that supports a standard bus interface to the
`graphics subsystem. The system also includes an overlay
`controller that is implemented in software and transfers the
`video from the video capture card over the bus to the
`graphics subsystem for display in a window on the monitor
`in an overlay mode.
`
`

`
`5,903,261
`
`3
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations there is a virtual overlay buffer provided in
`the computer’s system memory, and the overlay controller
`transfers video over the bus to the virtual overlay buffer and
`then transfers video from the overlay buffer to the graphics
`subsystem. Alternately the overlay buffer can reside on the
`graphics subsystem. The overlay controller includes an
`overlay application program and a video capture driver that
`supports a standard overlay interface to the application
`program.
`Embodiments of this aspect of the invention may include
`one or more of the following advantages. The graphics card
`can be used for prompt (virtually instantaneous) display of
`an overlay on the computer monitor without the need for a
`hardwired connection or the concern for compatibility of
`overlay hardware and graphics subsystem hardware.
`In
`addition, the software overlay controller is compatible with
`any standard graphics subsystem and can employ any stan-
`dard overlay application.
`In another aspect
`the invention features in general a
`computer system for displaying and compressing video on a
`computer in either a “preview model” or an “overlay mode.”
`The computer system includes an input port that receives
`uncompressed video in real time, a compressor that converts
`the uncompressed video from the input port to compressed
`video, and a bus interface circuit that is operable to transfer
`the uncompressed video and the compressed video to a
`computer bus. A display controller that is implemented in
`software is programmed to selectively display an image of
`the video being compressed either in a preview mode in
`which the compressed video transferred to the bus is decom-
`pressed and displayed on a computer monitor or in an
`overlay mode in which uncompressed video transferred to
`the bus is displayed on the monitor of the computer.
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations the user can select the preview mode or the
`overlay mode in both a “viewing without capture” mode and
`a “viewing” with capture mode. The user can also select a
`no-display mode during a capture mode. A dialog box
`displayed on the monitor and a selection control device are
`used to make the user selections. The display controller
`includes a video capture driver that controls the compressor
`and bus interface circuit. The display controller includes an
`application program and a software decompressor. The
`compressor is a video RISC processor, and the compressed
`video is in the form of MPEG I-frames.
`
`Embodiments of this aspect of the invention may include
`one or more of the following advantages. The overlay mode,
`which generally has better performance than the preview
`mode, can be employed on those platforms that have fast
`data transfer over the bus, while the preview mode can be
`used on systems that do not have sufficiently fast bus
`operation to support such transfer. Thus, the video system
`can he used on a variety of platforms having different levels
`of performance, and the best performance provided by the
`computer can be employed. In addition, in the preview mode
`the user can assess the quality of the captured and decom-
`pressed video The system permits on the fly switching
`between modes.
`
`the invention features in general a
`In another aspect
`computer system for deconipressing and displaying video on
`a computer monitor in either a hardware decompression
`mode or a software decompression mode. The system
`includes a decompressor circuit, a bus interface circuit, and
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`a display controller. The decompressor circuit converts
`compressed video to decompressed video. The bus interface
`circuit can operate in a playback mode in which compressed
`video is received from the computer bus and provided to the
`decompressor circuit, and decompressed video from the
`decompressor circuit is transferred to the computer bus. The
`display controller is implemented in software and is pro-
`grammed to selectively display an image of the video being
`decompressed in either a software mode in which the
`compressed video is decompressed in software and dis-
`played on a computer monitor or in an hardware mode in
`which compressed video that has been decompressed at the
`decompressor circuit and transferred to the bus is displayed
`on the monitor.
`
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations the display controller automatically deter-
`mines whether video transfers over the bus are sufliciently
`fast to support display of video that has been decompressed
`at the decompressor circuit, and automatically selects either
`the hardware mode or the software mode as a result of the
`
`determination. The preview mode can also be used to
`decompress and display video when the hardware is busy
`with other tasks. The decompressor circuit resides on a
`peripheral board that plugs into an expansion slot of the
`computer. The decompressor is a video RISC processor, and
`the compressed video is in the form of MPEG I-frames.
`In another aspect
`the invention features in general a
`computer system for compressing and storing video and
`audio on a computer. The system includes a compressor
`circuit that receives video and outputs compressed video,
`and a software controller that compresses audio in software
`and sends video data to be compressed across the bus to the
`compressor circuit.
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations the video received by the compressor cir-
`cuit is in the form of MPEG I-frames, and the compressed
`video is in the form of full MPEG IBP frames. Alternatively,
`the video received by the compressor circuit is in the form
`of uncompressed video. The software controller operates in
`threads and compresses audio in one thread and sends video
`data to be compressed across the bus to the compressor
`circuit in another thread.
`
`Embodiments of this aspect of the invention may include
`the following advantages. The use of hardware video com-
`pressor software audio compression permits parallel com-
`pression by two processors, which reduces the compression
`time without the need for additional circuitry.
`In another aspect
`the invention features in general a
`computer system for compressing and storing video and
`audio on a computer. The system includes an audio input
`subsystem that receives and digitizes audio, a video input
`port that receives uncompressed video in real time, a com-
`pressor circuit
`that compresses the uncompressed video
`from the input port, and a controller that is implemented in
`software and calibrates the startup delay of the audio input
`subsystem and uses the delay to synchronize the audio and
`video.
`
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations the controller calibrates startup delay of the
`audio input subsystem by sending an instruction to start
`digitizing audio and monitoring how long it takes the audio
`input subsystem to start digitizing after the instruction has
`been sent. The delay is determined by monitoring of a buffer
`
`

`
`5,903,261
`
`5
`in the audio driver. The audio input subsystem includes a
`sound card that plugs into an expansion slot of the computer
`system; alternatively the audio circuitry can reside on the
`motherboard. The video input port and the compressor
`circuit reside on a peripheral board that plugs into an
`expansion slot of the computer systems.
`Embodiments of this aspect of the invention may include
`one or more of the following advantages. The synchroniza-
`tion technique permits one to use a video card with any
`audio input subsystem without a direct connection. Also, one
`does not need to have an audio input subsystem on the video
`capture card, but can use the computer’s standard audio
`subsystem while still maintaining good audio/video syn-
`chronization.
`
`the invention features in general a
`In another aspect
`computer system for compressing video. The system
`includes a video compressor for compressing uncompressed
`input video at adjustable rates of compression, and a user
`interactive input mechanism for adjusting the rates of com-
`pression within a range of acceptable rates, the range of
`acceptable ratcs varying depending upon the output targct
`medium for compressed video.
`Certain implementations of this aspect of the invention
`include one or more of the following features. In certain
`implementations the user interactive input mechanism
`includes a mechanism for the user to select an output target
`medium and the ranges are based upon the selected target
`medium. Range adjusting sliders are displayed on a com-
`puter monitor to select a desired compression. Audio com-
`pression rates are similarly selected within varying ranges.
`Other advantages and features of the invention will be
`apparent from the following description of the preferred
`embodiment and from the claims. E.g., various combina-
`tions of the features described above can be combined in a
`
`computer video system to provide further improvements and
`advantages.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a diagram showing some of the components of
`a computer based video system according to the invention.
`FIG. 2 is block diagram of the FIG. 1 computer based
`video system.
`FIG. 3 is block diagram showing some of the components
`of the FIG. 1 computer based video system in more detail.
`FIG. 3A is a diagram showing layers of software control
`used to control a video capture card of the FIG. 1 system.
`FIG. 3B is a diagram of modules used to provide a
`no-interrupt video transfer in the FIG. 1 computer based
`video system.
`FIGS. 3C and 3D are timing diagrams for two different
`implementations of no—interrupt video transfer.
`FIGS. 4 and 4A are block diagrams illustrating prior art
`techniques for achieving an overlay of video in a computer
`monitor.
`
`FIG. 5 is a block diagram illustrating a technique for
`achieving a virtual overlay of video in a computer monitor
`of the FIG. 1 system.
`FIG. 6 is an illustration of display of a dialog box used to
`select various viewing and capturing options when using the
`FIG. 1 system.
`FIG. 7 is a diagram showing modules used in a playback
`mode of the FIG. 1 system.
`FIG. 7A shows a user interactive menu box used to select
`
`the video and audio compression rates.
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`FIG. 8 is a diagram of components used in combining
`audio and video streams in the FIG. 1 computer based video
`system.
`FIG. 9 is a diagram of modules used in synchronizing
`audio and video streams in the FIG. 1 computer based video
`system.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`After describing the major system components, the vari-
`ous modes of operation of the computer based video system
`will be reviewed, and then some of the operations will be
`discussed in detail.
`
`Major System Components
`Referring to FIG. 1, there is shown personal computer
`based video system 10 for capturing and digitizing video
`data from camera 11, VCR 13 or other source and then
`editing the digitized video to create short video programs,
`useful, e.g.,
`in business presentations or transfer over a
`network. System 10 is based upon personal computer 15,
`which has an associated monitor 16, keyboard 17, and
`speakers 23 and is provided with digital video card 12 and
`associated video control software 19 (both described in
`detail below) and commercially available graphics card 14
`(e.g., available from Diamond Multimedia under the Stealth
`64 trade designation) and sound card 28 (e.g., available from
`Creative Labs under the Sound Blaster trade designation)
`Computer 15 is Pentium based with 16 Mbytes RAM, 1 G
`bytcs disk, and a 133 MHz clock, Windows 95 with Video
`for Windows operating system, and appropriate multimedia
`sound drivers and graphic drivers for sound and graphics
`cards.
`
`Referring to FIG. 2, digital video card 12, graphics card
`14, CPU 25, and system memory 18 are shown connected to
`PCI system bus 21 of computer 15. PCI bus 21 is connected
`via PCI-ISA bridge 20 (e.g., an integrated circuit of a chip
`set available from Intel under
`the 82430FX trade
`
`designation) to ISA bus 22. The combination of bus 21 and
`bus 22 are sometimes referred to as a computer bus herein.
`Disk controller 24 is connected to ISA bus 22 and controls
`hard disk 26. Sound card 28 also is connected to ISA bus 22
`
`and has audio-in and speaker connections. Graphics card 14
`controls monitor 16 and performs YUV to RGB conversion
`and has standard overlay capability.
`Digital video card 12 has a video input port 30 connected
`to video decoder 32 (Philips SAA7111). Video decoder 32
`converts input analog video (in PAL or NTSC) and outputs
`its digital video in YUV (422) format to multiplexer 34 and
`multiplexer 36. The output of multiplexer 34 is connected as
`data-in input 37 to video RISC processor 38 (a compressor/
`dccomprcssor circuit available under the 4110 trade desig-
`nation from C-cube) Data-out output 39 of video RISC
`processor 38 is provided as a second input to multiplexer 36,
`which has an output to FIFO 40, which in turn is connected
`to bus interface circuit 42. Video RISC processor 38 also is
`connected to memory 44 (2 megabytes dynamic ram). Video
`RISC processor 38 also receives control signals over host
`interface control lines (not shown) via bus interface circuit
`42.
`Referring to FIG. 3, bus interface control circuit 42 is
`shown in more detail. It includes FIFO input control circuit
`46, master control state machine 48, 4x32 shift registers 50,
`multiplexer 52, size-I counter 54, size-II counter 56, master
`address-I counter 58, master address-II counter 60, and
`busmaster-I data input buifer 62. Circuit 42 is manufactured
`as an integrated circuit (e.g., a gate array) having the
`components and functionality described herein.
`
`

`
`5,903,261
`
`7
`Referring to FIG. 3A, the layers of software control are
`shown. Application programs 70 either operate through
`operating system 154 (Windows 95, including Video for
`Windows) on video capture driver 72 and codec driver 131
`or operate directly on Video capture driver 72 and recode
`driver 133. Video capture driver 72, codec driver 131, and
`recode driver 133 can control video capture card 12 through
`hardware interface 135, which permits control by one driver
`at a time.
`Modes of Operation
`Computer based video system 10 is capable of operating
`in a variety of different capture modes and playback modes
`depending upon user selections and data transfer capabilities
`of computer 15. The various modes of operation will first be
`generally discussed in turn before discussing details of the
`operations employed in the modes.
`1. Pass Through Mode
`In the pass through mode, video data are collected from
`the hardware and displayed to the user on system monitor
`16. The user typically uses this mode to initially view the
`source material before making any decisions as to what
`material may be included in the video clip being prepared.
`Referring to FIG. 2, analog video (PAL or NTSC) comes in
`through video input port 30 and is converted to a digital
`signal using video decoder 32. Digital video data then pass
`through multiplexer 36 and FIFO 40 to bus interface circuit
`42. The data are then transferred across PCI bus 21 to system
`memory 18. Video capture driver 72 (FIG. 5) then moves the
`image data in system memory 18 to graphics card 14 for
`display on analog monitor 16.
`2. I-frame Capture Mode
`In the I-frame capture mode, the input video data are
`compressed, sent across bus 21 to system memory 18 and
`typically saved on hard disk 26 for later use in creating a
`program In some cases compressed video might be sent
`across a LAN (not shown) for transmission to a remote
`device. The analog video signal comes through input port 30
`and is decoded into digital format with video decoder 32,
`and the digital video data are sent through multiplexer 34 to
`video RISC processor 38. Video RISC processor 38 converts
`the data into a compressed (MPEG, I-frame only) format and
`sends the data via its data output port 39 through multiplexer
`36 and FIFO 40 to bus interface circuit 42. Bus interface
`circuit 42 transfers the data to system memory 18. The data
`are sent via video capture driver 72 (FIG. 5) across PCI-ISA
`bridge 20 to disk controller 24 for storage on hard disk 26.
`3. Toggle mode
`It
`Another data capture mode is the “toggle” mode.
`combines the features of the previous two modes, pass
`through and I-frame capture. The input analog video signal
`passes through input port 30 to video decoder 32, and the
`digitized YUV video data pass through both multiplcxcrs 34
`and 36. From multiplexer 34, the YUV digital video data are
`sent to video RISC processor 38 for I-frame compression,
`and the compressed I-frame data that are output to one input
`of multiplexer 36. Multiplexer 36 thus has both compressed
`I-frame data and uncompressed YUV data which it alter-
`nately sends to FIFO 40. Bus interface circuit 42 takes the
`output of the