United States Patent
`
`[19]
`
`[11] Patent Number:
`
`5,546,103
`
`Rhodes et al.
`
`[45] Date of Patent:
`
`Aug. 13, 1996
`
`J||||l|l|||||||||lllllllllllllll|||l|||ll|l|||||||||||||||l||l||||l||||||l|
`US0055461D3A
`
`[54] METHOD AND APPARATUS FOR
`DISPLAYING AN IMAGE IN A WINDOWED
`
`[75]
`
`Inventors: Ken Rhodes, Portland; Rohan Coelho,
`H“l"b°"°; Dam Frank? Blake B°“d‘"'*
`b°‘h 91 B°‘“’°“°“- 3” °f (“'33-
`_
`_
`Intel Corporation, Santa Clara, Calif.
`
`_
`[73] Assignee:
`
`...................... .. 3451119
`
`1211937 Sciaoero et al.
`4.710.757
`1211991 Ya_maki -
`5.072.411
`2-as 1:233:
`::.n:;=;:;::.-.-
`:i:1:d§t‘:aI' """"""""""""""
`711993 Dinwiddie. '1'}:'££'§i:"."""""""'-
`....................... 3451119
`911993 Melntyre el :31.
`
`.. 3451115
`11.11993 Malcolm. 1:. et at.
`H1995 Gehmmn 1
`411995 Matsurnote .
`
`5’230'04:
`5:245:7o2
`5,253,750
`5,331,134
`5,404,445
`
`[21] A991‘ No‘: w3’256
`[22]
`Filed:
`Aug 5, 1993
`G09G 5114
`[51]
`Int. Cl.“
`3451119; 3951133; 395x157
`[52] U.S. Cl.
`.
`.
`3451113. 119,
`[53] Field of Search
`3451125. 121. 127. 112. 132. 133. 150.
`145. 153. 154. 200. 203; 3951129. 131.
`133. 134. 139. 153. 154. 155. 157. 151.
`154. 1155. 165; 3431557. 577. 573. 703
`
`[551
`
`References Cited
`
`U‘s‘
`
`Primary Examiner—Stcven Saras
`.;t;:lc;rr::y, Agent, or Fz'rm—S1eve Mendelsohn; Williaan H.
`
`ABSTRACT
`1571
`A method and apparatus for displaying an image in a system
`having an image generation subsystem and an image display
`subsystem.The image generation subsystem provides digital
`data corresponding to the image. The image generation
`subsystem then copies the digital data to a memory of the
`image display subsystem. The image display subsystem
`performs window management on the digital data and
`displays the
`data.
`
`4574,3154
`
`3.11986 Tabata et al.
`
`......................... .. 3451119
`
`18 Claims, 10 Drawing Sheets
`
`200
`
`30 2
`
`304
`
`APPLICRTIDN
`
`GRAPHIDDEVICE
`INTERFACE (om)
`
`MEDIA CONTROL
`INTERFACE (ucnwn
`
`303
`
`306
`
`II-|hGE DISPLAY
`SUBSYSTEH (IDS)
`
`IMAGE GENERATION
`SUBSYSTEI-I (I65)
`
`210
`
`1
`
`SAMSUNG 1008
`
`SAMSUNG 1008
`
`1
`
`

`
`U.S. Patent
`
`Aug. 13, 1995
`
`Sheet 1 of 10
`
`5,546,103
`
`..m.fi
`
`
`
`A~m<mo_mmvH.o~m
`
`=oHH¢uH4mm¢
`
`
`
`mw.u._Jozpzou¢_am=wU_>mQ-uH:m¢zm
`
`
`
`
`
`
`
`.u._flzoHp¢xmzwmmaczfi,¢gmm_awo¢=_
`
`
`
`
`
`
`
`
`
`Ampgwuzvmu¢mmw_=_A_aoWwuimmwkzw
`
`mw..._
`
`*uAu.H
`
`m..g,_
`
`
`
`fimmHV=mHm,mm=mAma_wzwpmymmzm
`
`
`
`
`
`moHop
`
`,mozm=
`
`2
`
`

`
`U
`
`.._...|a.
`
`IllPCON3N6:
`
`He
`
`/09Hn
`
`mhS
`
`301,6M
`
`5.,in
`
`cmNammm;:55m0
`
`
`m52.95:55:
`
`
`
`
`myad:=:m§__mA:553...:
`
`..lmomGON
`
`NON
`
` 3
`
`3
`
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 3 of 10
`
`5,546,103
`
`flu.
`
`m.or_
`
`men8528Em...
`
`
`
`22.5.:SEEEH
`
`mu_;mg-u_=m:¢D
`
`
`
`A_nmgmu{¢xm_z_
`
`won
`
`cfimfl
`
`OflNEOE:
`
`
`Amcwv
`
` xmhmymmamOHm=o:..$_mo$;__
`
`in
`
`
`
`5Among=mHm.mm=m
`
`
`
`,¢AmmHama¢=H
`
`mom
`
`EEQE...
`
`Non
`
`4
`
`
`
`
`

`
`US. Patent
`
`Aug. 13, 1996
`
`Sheet 4 of 10
`
`5,546,103
`
`200
`—‘
`
`FIG. 4
`
`402
`
`APPLICATION REQUESTS HCIAVI
`T0 DISPLAY IMAGE IN WINDOW.
`
`
`
`404
`
`HCIAVI OPENS WINDOW.
`
`MCIAVI OPENS IMAGE FILE.
`
`408
`
`MCIAV1 CAUSES IGS TO BE INITIALIZED.
`
`410
`
`I-ICIAVI SENDS DEVICE CONTEXT HANDLE
`
`
`
`AND WINDOW HANDLE T0 IGS.
`
`
`
`41 2
`
`IGS PROCESSES IMAGE DATA.
`
`41 4
`
`IGS MAKES IMAGE DATA AVAILABLE TO IDS
`
`IDS PROCESSES AND DISPLAYS IMAGE DATA
`
` 1 DISPLAY
`
`ANOTHER IMAGE
`7
`
`418
`
`N0
`
`422
`
`HCIAVI CLOSES IGS.
`
`424
`
`MCIAVI CLOSE IMAGE FILE.
`
`426
`
`HCIAVI CLOSES WINDOW.
`
`5
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 5 of 10
`
`5,546,103
`
`F IG . 5
`
`408
`
`504
`7"
`
`no
`
`INTERFACE
`suppoarsn IN
`
`ms?
`
`506
`
`£;()z3
`
`YES
`GET ALL INTERFACE FUNCTION ADDRESSES.
`
` 520
`
`FOR EACH SUPPORTED COLOR. QUERY IDS ACCESS.
`CLIPFING. CHROHAKEYING. AND STRETCHING SUPPORT.
`
`
`
`
`5522:!
`
`524
`
`526
`
`528
`
`DETERMINE DISPLAY WINDOW ENVIRONNENT.
`
`SELECT ACCESS MODE & COLOR FORMAT FROM PRIORITY LIST
`
`INITIALIZE LOCAL REPRESENTATION OF DISPLAY WINDOW.
`
`SET UP SYSTEM TO NOTIFY I65 OF CHANGES
`
`TO WINDOW SHAPE. POSITION. OR SIZE.
`
`6
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 6 of 10
`
`5,546,103
`
`512
`
`FIG. 6
`
`602
`
`604
`
`606
`
`608
`
`IGS INTERROGATES IDS TO
`
`DETERMINE IDS CAPABILITIES.
`
`FOR EACH PERMUTATION OF
`
`SUPPORTED CAPABILITIES.
`
`TIME THE DISPLAY OF
`
`TEST IMAGES.
`
`GENERATE PRIORITY LIST FOR
`
`EACH TYPE OF TEST IMAGE.
`
`SAVE CURRENT STATES
`
`OF IDS AND IGS.
`
`7
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 7 of 10
`
`5,546,103
`
`FIG. 7
`
`7021
`
`HHS
`WINDOW SHAPE
`
`N0
`
`412
`
`
`
`1 HAS
`
`WINDOW POSITION
`
`N0
`
`CHANGED
`
`?
`
`CHANGED
`
`?
`
`HANDLE SHAPE CHENGE .
`
`70 4
`
`706
`
`708
`
`HANDLE POSITION CHANGE.
`
`7 10-L
`
`HAS
`WINDOW SIZE
`
`CHANGED
`
`?
`
`N0
`
`7 1 2
`
`HANDLE SIZE CHANGE.
`
`714
`
`DECOMPRESS IMAGE DATA.
`
`716
`
`IF NECESSARY.
`
`CONVERT COLOR FOR!-IAT.
`
`8
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 3 of 10
`
`5,546,103
`
`FIG. 8
`
`800
`
` E3()Ez
`QUERY GDI FOR ORDER OF DISPLAY WINDOW
`
`804
`
`
`
`
`
`
`IN WINDOWS HIERARCHI.
`
` 806
`
`FOR EACH HIGHER ORDER WINDOW IN HIERARCHY.
`
`QUERY GDI FOR COORDINATES OF PORTION OF
`
`DISPLAY WINDOW OVERLAPPED BY THAT WINDOW.
`
`GENERWTE CLIP LIST FROM OVERLWP COORDINATES.
`
`9
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 9 of 10
`
`5,546,103
`
`FIG. 9
`
`414
`
`_
`
`9”
`
`9°‘
`
`9°“
`
`9”
`
`
`
`910
`
`IDS SENDS IGS: START ADDRESS. END ADDRESS.
`
`3; STRIDE.
`
`9”
`
` 7_
`
`MORE
`DATA IN CURRENT
`
`914
`
`FRAME
`
`?
`
`N0
`
`.
`
`916
`
`IF REQUIRED.
`
`IGS SENDS IDS END-FRAME MESSAGE.
`
`9 1 81 ANOTHER
`FRAME IN SAME
`
`YES
`
`9 201
`
`CHANNEL
`
`?
`
`HODIFY
`CHANNEL
`?
`
`no
`
`IGS MDDIFIES
`VIDEO
`
`CHANNEL.
`
`9 24
`
`IGS CLOSES mm CHANNEL.
`
`9 22
`
`10
`
`10
`
`

`
`U.S. Patent
`
`Aug. 13, 1996
`
`Sheet 10 of 10
`
`5,546,103
`
`FIG. 10
`
`414
`
`IGS OPENS VIDEO CHANNEL.
`
`
`
`IGS IMKES DhTfi AVAILABLE TO IDS.
`
`IGS STORES IMAGE DATA TO SYSTEM HENRY.
`
`1014
`
`11
`
`1004
`
`1006
`
`1°08
`
`1°16
`
`11
`
`

`
`5,546,103
`
`1
`l\r[ETHOD AND APPARATUS FOR
`DISPLAYING AN IMAGE IN A WJNDOVVED
`ENVIRONMENT
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to image processing. and. in
`particular, to methods and apparatuses for displaying images
`on a computer system operating in a windowed environ-
`ment.
`
`2. Description of the Related Art
`It is desirable to display images on a computer system
`operating in a windowed environment
`such as
`the
`Microsoft® Video for Windows (VFW) environment run-
`ning under a Microsoft® Windows 3.){ system. It is par-
`ticularly desirable to display high-quality motion video on
`the graphics monitor of a conventional WW-based personal
`computer (PC) system while minimizing the need for spe-
`cial-purpose hardware.
`Raw digitized motion video consumes a lot of storage
`space. For example, a l5-frame-per-second (FPS) video
`sequence, at a frame size of 320 pixels by 240 pixels and
`with 24-bit color, consumes about 3.5 MBytes per second.
`As a result, for conventional PC systems, digitai video is
`stored in a compressed fonnat.
`The amount of compression required is driven by band-
`width limitations of the storage devices used in PC systems.
`The only practical mass storage for motion video is CD-
`ROM. Access rates from single-spin CD-ROM devices is
`about 150 KBytes/sec. To play digital video back from
`CD-ROM devices, motion video is typically compressed to
`accommodate this access rate limitation. This compression
`of video data is typically perftinned on video data in a YUV
`format since compression ratios for YUV video data are
`typically greater than compression ratios for video data in
`other formats such as an RGB format.
`
`Decompressing and displaying video is a computation
`intensive task. Conventional window-based PC video sys-
`tems are capable of processing motion video data to com-
`bine video images with graphics for display on a display
`monitor. Like any system. conventional vidco systems have
`limits to their performance capabilities.
`It is desirable to
`provide an improved window-based video system that has
`greater performance capabilities than those of conventional
`video systems.
`Referring now to FIG. 1. there is shown a software block
`diagram of a conventional video system 100 for displaying
`video images under the Microsoft® Video for Windows
`environment. In FIG. 1:
`
`o Application 102 is an application program running in
`the \’fW enviromnent,
`
`o Graphic-display interface (GD!) 104 is part of
`Microsoft® Windows,
`0 Media control
`interface (MCIAVI) 103 is part of
`Microsoftfil Video for Windows,
`o lrnage generation subsystem (IGS) 110 is the software
`driver for the subsystem of video system 100 respon-
`sible for processing video image data to transform it
`from the format as received by the video system (e.g.,
`compressed YUV format) to a format acceptable for the
`image display subsystem software 106, and
`o Image display subsystem (IDS) 106 is the software
`driver for the subsystem of video system 190 that
`
`2
`handles the display of the video data processed by the
`IGS.
`Application 102, GD1104, MCIAVI 108, IGS 110, and IDS
`106 may all reside in system memory (not shown) and be
`implemented by a central processing unit
`(CPU)
`(not
`shown). Video system 100 also includes a mass storage
`device {not shown) for storing compressed YUV data, IDS
`memory for storing display bitmap (i.e., frame buficr) data,
`and a display monitor [not shown) for displaying decom-
`pressed video images.
`In conventional video system 100, after application 102
`tells Wdeo for Windows to open a particular video window,
`MCIAVI 108 defines the window in the display raster of the
`display monitor. MCIAVI 108 then accesses compressed
`YUV video data from the mass storage device and stores the
`compressed YUV video data to system memory. MCIAVI
`108 then instructs 1G5 110 to process the data for dispiay.
`IGS 110 accesses the compressed YUV video data from
`system memory, decompresses the compressed YUV video
`data, and stores the decompressed YUV video data back to
`system memory.
`Microsoft® Windows accepts video data in only RGB_S
`or RGB_24 format. Microsoftfis Video for Windows
`extends this capability by accepting video data in RGB_l6
`format. In conventional video system 100. therefore, [GS
`110 accesses the decompressed YUV video data from sys-
`tem memory, converts the decompressed YUV video data to
`an RGB format that Windows supports, and stores the RGB
`video data back to system memory. IGS 110 then passes the
`RGB video data to MCIAVI 108, which passes the RGB-
`format video data to GDI 104 and IDS 106 for storage to IDS
`memory and eventual display on the display monitor.
`In conventional video system 100, IGS Ill} spends most
`of its time decompressing compressed YUV video data,
`converting decompressed YUV video data to the desired
`RGB display format, and copying the convened RGB video
`data to MCIAVI 108. In addition, MCIAVI 108 and GD] 104
`spend time copying the image data to IDS 10G. Video
`system perfonnanee can be improved by accelerating the
`rate at which video system 100 can process video data.
`Conventional video systems like video system 100 work
`weil when the overhead required to display data is small
`compared with the overhead required to generate it. This is
`usually the case with computer graphics, where the same
`image components are used repeatedly. But in motion video,
`images can change as many as 30 times a second. The
`overhead associated with such image updates becomes sig-
`nificant, arid the display architecture that was satisfactory for
`computer graphics becomes a motion-video bottleneck.
`It is therefore desirable to accelerate video processing and
`thereby improve video system performance by reducing the
`amount of time spent by the image generation subsystem in
`copying converted video data to the display.
`In addition, at the frame rates associated with motion
`video, format conversion overhead becomes significant. It is
`therefore desirable to accelerate video processing and
`thereby improve video system performance by reducing the
`amount of time spent by the image generation subsystem in
`converting decompressed video data for display.
`It is accordingly an object of this invention to overcome
`the disadvantages and drawbacks of the known art and to
`provide a window-based video system with accelerated
`video processing and improved performance compared with
`conventional window—bascd video systems.
`It is a further object of the present invention to improve
`video system performance by reducing the amount of time
`spent by the image generation subsystem in converting
`decompressed video data for display.
`
`20
`
`30
`
`35
`
`45
`
`50
`
`55
`
`65
`
`12
`
`12
`
`

`
`5,546,103
`
`3
`It is a further object of the present invention to improve
`video system performance by reducing the amount of time
`spent by the image generation subsystem in copying con-
`verted video data to the display.
`Further objects and advantages of this invention will
`become apparent from the detailed description of a preferred
`embodiment which follows.
`
`SUMMARY OF THE INVENTION
`
`According to a preferred embodiment, the present inven-
`tion is a method and apparatus for displaying an image in a
`system having an image generation subsystem and an image
`display subsystem. The image generation subsystem pro-
`vides digital data corresponding to the image. The image
`generation subsystem then copies the digital data to a
`memory of the image display subsystem. The image display
`subsystem performs window management on the digital data
`and displays the digital data.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Other objects. features, and advantages of the present
`invention will become more fully apparent from the follow-
`ing detailed description of the preferred embodiment, the
`appended claims, and the accompanying drawings in which:
`FIG. 1 is a software block diagram of a conventional
`video system for displaying video images under
`the
`Microsoft® Video for Windows environment;
`FIG. 2 is a hardware block diagram of a video system for
`displaying video in the Microsoft-ill Video for Windows
`(VfW) environment, according to a preferred embodiment
`of the present invention;
`FIG. 3 is a software block diagram of the video system of
`FIG. 2;
`
`FIG. 4 is a process flow diagram of the top-level process-
`ing of the video system of FIGS. 2 and 3;
`FIG. 5 is a process flow diagram of the initialization of the
`image generation subsystem of the video system of FIGS. 2
`and 3;
`
`FIG. 6 is a process flow diagram of the processing
`implemented by the image generation subsystem to profile
`the video system of FIGS. 2 and 3;
`FIG.
`'7 is a process flow diagram of the video data
`processing implemented by the image generation subsystem
`of the video system of FIGS. 2 and 3;
`FIG. 8 is a process flow diagram of the generation of the
`clip list implemented by the image generation subsystem of
`the video system of FIGS. 2 and 3;
`FIG. 9 is a process flow diagram of the direct access mode
`implemented by the image generation subsystem and the
`image display subsystem of the video system of FIGS. 2 and
`3; and
`
`FIG. 10 is a process flow diagram of the indirect non-GDI
`access mode impiemented by the image generation sub-
`system and the image display subsystem of the video system
`of FIGS. 2 and 3.
`
`DESCRIPTION OF THE PREFERRED
`EM.BODIMENT(S)
`
`A. Video System Hardwa.re Configuration
`
`Referring now to FIG. 2, there is shown a hardware block
`diagram of video system 200 for displaying video under
`Microsoft® Video for Windows (WW) in the Microsoft®
`
`35
`
`45
`
`50
`
`65
`
`13
`
`4
`
`Windows 3.}( environment, according to a preferred
`embodiment of the present invention. In video system 200,
`compressed video data (e.g.. in a compressed YUV format)
`is either accessed from mass storage device 212 or received
`from network 214, and stored to system memory 204.
`Central processing unit (CPU) 202 retrieves the compressed
`YUV video data from system memory 204, decompresscs
`the compressed YUV video data. and stores the resulting
`decompressed YUV video data back to system memory 204.
`For purposes of this specification. the term “vidco" means
`digital video.
`Depending upon the capabilities of the software imple-
`mented in video system 200, in one mode of operation, CPU
`202 retrieves the decompressed YUV video data from sys-
`tem memory 204, converts the decompressed YUV video
`data to a display format (e.g., an RGB format). stores the
`converted RGB video data back to system memory 204, and
`then makes the converted RGB video data available for
`transfer to image display subsystem (IDS) memory 210.
`Image display subsystem 206 then accesses the converted
`RGB video data from IDS memory 210 for processing {e.g..
`digital to analog conversion) and display on display monitor
`203. In this mode of operation, video system 200 functions
`similarly to conventional video system 100 of FIG. 1.
`In another mode of operation, any necessary conversion
`of the decompressed YUV video data is implemented by
`image display subsystem 206. In this mode. CPU 202 either
`transmits the decompressed YUV video data directly to IDS
`memory 210 or makes the decompressed YUV video data
`available to IDS 206. In the latter case, IDS 206 copies the
`decompressed YUV video data from system memory 204 to
`IDS memory 210. In either case. once the decompressed
`YUV video data is stored in IDS memory 210, IDS 206
`performs any necessary conversion of the decompressed
`YUV video data for display on display monitor 208. In this
`mode of operation, the responsibility for format conversion
`is ofi’-loaded from CPU 202 to IDS 206.
`
`CPU 202 may be any suitable general purpose processor
`and is preferably an Intel® x86 processor, where an Intel®
`X86 processor is either an Intcl® 386. 486. or Pentium
`processor. Those skilled in the art will understand that higher
`powered processors are preferred to provide greater system
`performance.
`System memory 204 may be any suitable memory devices
`such as dynamic random access memory (DRAM) or static
`random access memory {SRAM) devices. and is preferably
`a DRAM device. IDS memory 210 may be any suitable
`memory devices such as DRAM, SRAM. or video random
`access memory (VRAM) devices, and is preferably a
`VRAM device. Mass storage device 212 may be any suitable
`storage device for compressed video data such as a hard
`drive, CD-ROM, or floppy disk. Network 214 may be a
`local-area or wide—area network suitable for teleconferenc-
`ing, cable television. telephony, or other similar application
`in which compressed video data is received from a remote
`source. Display monitor 208 may be any suitable device for
`displaying video images and is preferably a conventional PC
`graphics monitor.
`Processor local bus 216 is preferably the high-speed data
`bus for CPU 202. Expansion bus 229 may be any suitable
`digital data bus and is preferably a Peripheral Component
`Interconnect (PCI) bus. Expansion bus 220 may also be an
`Industry Standard Architecture (ISA) or Extended ISA
`(EISA) bus. Bus interface adapter 218 may be any interface
`suitable for connecting expansion bus 220 and processor
`local bus 216 such that buses 216 and 220 are allowed to
`operate at different speeds.
`
`13
`
`

`
`5
`
`6
`
`5,546,103
`
`5
`
`Image display subsystem 206 may be any suitable sub-
`system for displaying decompressed video data on a display
`monitor. In a preferred embodiment, IDS 206 is any image
`display subsystem that supports an enhanced interface
`between IDS 206 and the image generation subsystem of
`video system 200. Those skilled in the art will understand
`that IDS 206 may be implemented entirely in software.
`entirely in hardware, or a combination of both hardware and
`software. It will also be understood that some or all of the
`IDS software may be implemented on CPU 202. The pur-
`pose of the present invention is to provide video system
`hardware and software components that may be configured
`with any of a variety of suitable manifestations of image
`generation subsystems.
`Preferred embodiments of the enhanced interface and the
`
`image generation subsystem are described in the following
`sections of this specification.
`
`B. Video System Software Configuration
`
`Referring now to FIG. 3. there is shown a software block
`diagram of video system 200 of FIG. 2, according to a
`preferred embodiment of the present invention. Like con-
`ventional video system b0O of FIG. 1, video system 200 has
`an application program 302 that runs under the Microsoft®
`Video for Windows (VfW} environment [MCIAVI 308) in
`the Microsoft® Windows system (GDI 304). Video system
`200 also has image generation subsystem [IGS) 310, image
`display subsystem (IDS) 306, and an enhanced interface
`between IGS 310 and IDS 306 (represented in FIG. 3 by
`interface path 312, 314, and 316). IDS 306 of FIG. 3
`represents the software of IDS 206 of FIG. 2.
`The enhanced interface of video system 200 is an inte-
`grated digital media architecture interface that provides the
`capability for accelerating video processing on video system
`200 by (1) reducing the color format conversion responsi-
`bilities of IGS 310 andlor (2) reducing the volume and
`distance (i.e., number of layers) of data transfer from IGS
`310 to IDS 306. Both of these improvements are made
`possible by the ability of the enhanced interface to support
`the transfer of video data from IGS 310 to IDS 306 without
`going through Windows GDI 304.
`The enhanced interface takes advantage of the fact that
`IDS 306 may support more color formats than the two or
`three RGB formats supported by Microsoft® Video for
`Windows. If IDS 306 is capable of receiving video data in
`the same color format as the decornpressed video data (e.g.,
`YUV9), then IGS 310 may not need to convert the decom-
`pressed YUV9 video data to a different color
`format.
`Depending on the particular capabilities of IDS 306, IDS
`306 may then perform color conversion on the video data
`received from IGS 310.
`The enhanced interface reduces the distance of data
`transfer by avoiding GDI access (i.e., the transfer of video
`data from IGS 310 to MCIAVI 308 to GDI 304 to IDS 306
`to IDS memory 210 of FIG. 2). The enhanced interface
`provides this distance reduction when IDS 306 supports
`either direct access or indirect non-GDI access to IDS
`memory 210 by IGS 310.
`When IDS 306 supports direct access by IGS 310 to IDS
`memory 210, the enhanced interface reduces the volume of
`data transfer by taking advantage of the temporal redun-
`dancy typically found in video imagery. Since video imagery
`typically demonstrates a high degree of temporal redun—
`dancy (i.e., many corresponding pixels in consecutive video
`frames do not change significantly), direct access permits
`
`65
`
`14
`
`IGS 310 to update only those pixels components in IDS
`memory 210 that require update (i.c., only a subset of the
`pixels corresponding to the entire image). As a result, the
`volume of data transfer can be greatly reduced.
`These improvements reduce the percentage of CPU time
`IGS 310 spends for conversion and data transfer, leaving
`more time available for decompression. As a result, video
`system 200 may process video data at greater frame rates
`than would otherwise be available. thereby providing per-
`formance of video system 200 greater than that of conven~
`tional video systems such as video system 100.
`In a preferred embodiment, the enhanced interface estab-
`lishes a protocol by which IGS 310 interrogates IDS 306
`about the video processing capabilities supported by IDS
`306. Based on the results of those interrogations, IGS 310
`determines how to process and transfer video data to [D8
`306 for the current video window. The enhanced interface
`removes the need for IDS-specific coding in IGS 310. The
`result is an image generation subsystem that is compatible
`with more than one specific type of image display sub-
`system. Moreover, changes and improvements to the image
`display subsystem will not necessarily require changes to the
`image generation subsystem.
`The protocol for the enhanced interface is defined by a set
`of enhanced interface functions that are called by IGS 310
`and are recognized and properly handled by IDS 306. The
`syntax for these functions is defined in further detail later in
`this specification in Section D.
`Referring again to FIG. 3, the enhanced interface of video
`system 100 supports three different modes of access by IGS
`310 to IDS memory 210 of FIG. 2 (i.e., ways of getting video
`data from IGS 310 to IDS memory 210). GDI access [shown
`along path 312) is the same path used by conventional video
`system 100 of FIG. 1. In addition. the enhanced interface
`also supports direct access (shown along path 316) and
`indirect non—GDI access (shown along path 314}. [GS 310
`always has the GDI access mode available. In addition, IGS
`310 may have one of either the direct access mode or the
`indirect non—GDI access mode available or neither. When
`and how these various access modes are used are described
`in further detail later in this specification in conjunction with
`FIGS. 9 and 10.
`
`The enhanced interface is designed to be an extension of
`the interface between GDI 104 and IDS 106 of conventional
`video system 100 of FIG. 1. IGS 310 may be implemented
`entirely in software, entirely in hardware, or a combination
`of both hardware and software. It will also be understood
`that some or all of the IGS software may be implemented on
`CPU 101 of FIG. 2. In alternative preferred embodiments of
`the present invention, some or all of the IGS functions may
`be implemented using a special purpose processor, such as
`an Intel® 82’i50PB pixel processor.
`
`C. Video System Processing
`
`1. Top-Level Video System Processing
`Referring now to FIG. 4, there is shown a process flow
`diagram of the top—lcvel processing of video system 200 of
`FIGS. 2 and 3, according to a preferred embodiment of the
`present invention. After application 302 requests Video for
`Windows to display one or more images in a window [block
`402 of FIG. 4), MCIAVI 308 opens the window (block 404),
`opens the appropriate image file (block 406}, causes image
`generation subsystem 310 (block 408) to be initialized, and
`sends the device context handle and window handle to IGS
`310 (block 410). Those skilled in the art will understand that
`
`14
`
`

`
`5,546,103
`
`7
`the device context and window handles provide mechanisms
`for IGS 310 to determine the characteristics of the video data
`and display window,
`respectively.
`‘These characteristics
`include the size and position of the display window within
`the display raster of display monitor 208.
`IGS 310 processes the image data (block 412) and makes
`the processed image data available to image display sub~
`system 306 [by one of the three possible display access
`modes) (block 414). As described later in this specification
`in conjunction with FIG. 7,
`the processing of IGS 310
`includes decompression and if necessary color format con-
`version. IDS 306 then further processes and displays the
`image data (block 416). The processing of IDS 306 includes
`color format conversion (if necessary) and digita.l—to~analog
`conversion. Those skilled in the art will understand that
`some preferred embodiments of IDS 206 of FIG. 2 are
`capable of converting YUV video data directly to analog
`signals through hardware.
`then
`If another image is to be displayed (block 418},
`processing returns to process and display the next image
`(blocks 412-416). Otherwise. there is no other image to
`display (block 418) and MCIAVI 308 closes IGS 310 (block
`422), closes the image file {block 424), and closes the
`window (block 426),
`thereby ending processing for that
`video window request by application 302.
`Those skilled in the art will understand that video system
`200 is not limited to the sequence of steps presented in FIG.
`4. For example, the sequences of opening and closing the
`window, the image file, and IGS 310 may vary. In addition,
`application 302 may open and close the window rather than
`having MCIAVI 308 open and close the window {blocks 404
`and 426 of FIG. 4).
`
`2. Initialization of Image Generation Subsystem
`
`Referring now to FIG. 5, there is shown a process flow
`diagram of the initialization of image generation subsystem
`(block 408 of FIG. 4), according to a preferred embodiment
`of the present invention. IGS initialization preferably occurs
`only when the window is first opened or when the size ofthc
`window changes sufficiently to warrant reinitialization (as
`determined by IGS 310). Window size changes are described
`in further detail later in this specification in Section C.4.
`The enhanced interface of the present invention is pref-
`erably implemcnted in software, pan of which resides in the
`image generation subsystem and part of which resides in the
`image display subsystem of video system 200. Since IGS
`310 is compatible with all image display subsystems [i.e.,
`even those that do not support the enhanced interface), IGS
`310 determines whether the enhanced interface is supported
`by IDS 306 (block 504 of FIG. 5). IGS 310 preferably makes
`this determination by asking the Windows system for the
`library address of one of the enhanced interface functions
`using the Microsoft® GctProcAddress function.
`If the Windows system replies that that function does not
`exist in the Windows function library, then IGS 310 knows
`that IDS 306 does not support the enhanced interface. In that
`case, IGS 310 falls back to the default GDI access mode of
`transferring RGB video data to IDS 306 via GDI 304 (as in
`conventional video system of FIG. 1) [block 506 of FIG. 5).
`If the Windows system replies with a valid function address,
`then IGS 310 knows that IDS 306 does support the enhanced
`interface and IGS 310 then asks the Windows system for the
`addresses of the rest of the enhanced interface functions
`(block 508).
`
`The enhanced interface provides IGS 310 with the ability
`to optimize system performance for the particular capabili-
`
`10
`
`35
`
`50
`
`55
`
`15
`
`8
`ties supported by IDS 306 for the current video window by
`selecting the best (c.g., fastest) mode of operation from
`among the variety of permutations of capabilities supported
`by IDS 306. IGS 310 does this by profiling the system
`performance for each of the possible permutations of capa-
`bilities supported by IDS 306 and then using the results of
`that profiling to generation priority lists that tell IGS 310
`which permutation to select for a particular video window.
`Profiling only needs to be performed once for a particular
`configuration of IGS 310 and IDS 306.
`Thus. if IGS 310 determines that the state of either IGS
`310 or IDS 306 has changed (block 510 of FIG. 5). then IGS
`310 re-profiles system performance {block 512) and pro-
`cessing then continues to block 522 (described below). For
`example, the state of IGS 310 wili change if the speed of
`CPU 202 changes (e.g.. if CPU 202 is upgraded to a faster
`processor). Similarly, the state of IDS 306 will change if IDS
`306 is replaced with an improved image display subsystem.
`Profiling is described in further detail later in this specifi-
`cation in conjunction with FIG. 6.
`If the states of IGS 310 and IDS 306 have not changed
`(block 510), then IGS 310 interrogates IDS 306 to determine
`what color formats are supported by IDS 306 (block 514).
`IGS 310 performs this interrogation by calling the enhanced
`interface function Querycolorsupport with the appropriate
`parameters. IDS 306 responds to QueryColorSupport by
`returning to IGS 310 a pointer to a list of values correspond-
`ing to the color formats supported by IDS 306. QueryCol-
`orSup1:Ior1 and all the other enhanced interface functions are
`described in further detail
`later in this specification in
`Section D.
`
`If IDS 306 does not support any of the color formats
`recognized by IGS 310 (block 516). then IGS 310 defaults
`to the GDI access mode (block 518). Otherwise, [D8 306
`supports one or more of the color formats recognized by IGS
`310. If that case. for each color fonnat supported by IDS
`306, IGS 310 interrogates IDS 306 to determine the capa-
`bilities supported by IDS 306 for display access, clipping,
`chromalteying. and stretching (block 520 via QueryDispiay—
`Accesssupport. Queryclippingsupport, QueryChron1aKey-
`Support, Querystretchsupport functions, respectively).
`IDS 306 may support one of three different combinations
`of display access [i.e., modes of transferring video data from
`IGS 310 to IDS 306): (1) GDI access mode only, (2) either
`GDI access mode or direct access mode, and (3) either GDI
`access mode or indirect non-GDI access mode. The direct
`access and indirect non-GDI access modes are described in
`further detail later in this specification in conjunction with
`FIGS. 9 and 10. The GDI access mode is the conventional
`access mode employed by conventional video systems such
`as video system 100.
`IDS 306 either does not support clipping or supports one
`of two types of clipping. In one type of clipping support, IDS
`306 needs to be supplied with a clip list. In the other type of
`clipping support, IDS 306 does not need to be supplied with
`a clip list.
`IDS 306 either does not support chromakeying or sup-
`ports chromalreying in a particular color format. If IDS 306
`supports chrornakeying, the chromakey color format will be
`one of the color formats listed by [D8 306 in response to the
`QueryColorSupport fiinction. Those skilled in the art will
`understand that, in a preferred embodiment, IDS 306 may
`support either binary chromakeying or
`fractional alpha
`blending or both.
`-
`IDS 306 either does not sup