as) United States
`a2) Patent Application Publication 10) Pub. No.: US 2006/0107229 Al
`(43) Pub. Date: May18, 2006
`
`Matthewset al.
`
`US 20060107229A1
`
`(54) WORK AREA TRANSFORM IN A
`GRAPHICAL USER INTERFACE
`
`(75)
`
`Inventors: David A. Matthews, Seattle, WA (US);
`Charles W. Stabb, Seattle, WA (US);
`Kanwal Vedbrat, Seattle, WA (US);
`MarkR. Ligameri, Snohomish, WA
`(US)
`
`Correspondence Address:
`BANNER & WITCOFF LTD.,
`ATTORNEYS FOR MICROSOFT
`1001 G STREET , N.W.
`Suite 1100
`WASHINGTON, DC 20001-4597 (US)
`
`(73) Assignee: Microsoft Corporation, Redmond, WA
`
`(21) Appl. No.:
`
`10/986,950
`
`(22)
`
`Filed:
`
`Nov. 15, 2004
`
`Publication Classification
`
`(51)
`
`Int. CL
`(2006.01)
`G06F 17/00
`(52) U.S. CMe
`ecsccsssessssessecestensssessesestenesinseevesee 715/782
`
`(57)
`
`ABSTRACT
`
`A method and apparatus for transforming a work area and
`displaying an information component in a graphical user
`interface is provided. The graphical user interface utilizes a
`three-dimensional transformation to move a presently dis-
`played work area, for example a desktop with open win-
`dows, revealing a background presentation area behindit. In
`the portion of the presentation area revealed, an information
`component, such as a Start Menu, is displayed. Use of the
`three-dimensional transformation effectively decouples an
`operating system from the applications it hosts. The trans-
`formation further prevents visual clutter by separating the
`information component
`from the previously underlying
`desktop.
`
`SXdem
`
`& Thumbdrive
`
`a) irra lod
`corny Ce eR Tan
`Fetrt
`
`;
`
`d
`A
`‘|
`
`| é Cota Steele
`or ae)
`f
`‘th ASOrostd
`
`Hepteengert
`Le:
`ia aa be SY i bs ed
`Natoma0asm,
`ASO Pa Eee
`EeeaReR
`
`bo Sa fo
`
`APPLE 1019
`
`APPLE 1019
`
`1
`
`

`

`Patent Application Publication May 18,2006 Sheet 1 of 10
`
`susivadsAesthYaldvavLINNONLeanO
`
`
`
`
`
`
`WHOMLENMYOMLAN||LAdNIdasnFTEVAOWSYFTEVAOWSY-NONWYYDONdYSHLO
`
`|Wauv1¥001|OL\ZI0S}Ort
`MAINId[iLNdLNOZe}
`881|_Of
`SWVWY9ONd!NOILVONdd¥
`oeoeoweekyee:;"YwaziuioidKT¥OLINOW“OF
`
`
`
`
`
`FOVIYSALNI||JOVIYALNIAMOWSW“1OA-NONANOWAN“TOA-NONSaINGOW
`S8L78oor?;|
`
`US 2006/0107229 Al
`
`Og}
`
`JLOWSY
`
`YALNdNOO
`
`AdLOWSY
`
`NOILVONdd¥
`
`SWVYDONd
`
`
`
`FOVAYSLNIJOVAYSLNI
`
`
`
`VivaWweu90ud
`
`Suayvads
`
`681]I
`
`
`
`O3dIASNISSSOOUd
`
`€810z1
`
`
`
`AYXOWSWWALSAS
`
`2
`
`
`
`
`

`

`Patent Application Publication May 18,2006 Sheet 2 of 10
`
`US 2006/0107229 Al
`
`190
`
`192
`
`
`
`Legacy Graphical User
`
`Interface Subsystem
`190a
`190b
`
`
`
`Legacy Window)¢192a
`Subsystem
`Application
`og
`Application
`:
`.
`Programming
`Programming
`Manager
`
`
`
`Software
`Interface
`Interface
`
`
`
`
`194
`
`Compositing Desktop Window Manager
`(CDWM)
`
` Ul Object
`
`Universal Compositing Engine (UCE)
`
`Module
`
`Programming
`Interface
`
`Rendering
`
`Resource
`Manager
`
`Request
`Queue
`
`Device Interface
`
`Legacy Graphics
`
`}.192b
`
`193
`
`Theme Manager
`
`194
`
`3D Graphics Interface
`(Direct3D, OpenGL,etc)
`
`196
`
`Video Driver
`Device Object
`
`Video Driver
`Device Object
`
`19
`
`“EI
`
`Display Device
`
`Display Device
`
`FIG. 1B
`
`195
`
`196
`
`19
`
`7
`
`3
`
`

`

`Patent Application Publication May 18, 2006 Sheet 3 of 10
`
`US 2006/0107229 Al
`
`200
`
`201
`
`202
`
` :
`
`Search sicrosotecom for:
`
`- YOUR SERVER
`
`.
`
`:
`
`Protect your PC
`install Windows xP
`Service Pack 2 today.
`
`
`
`network-securitysolution 204
`
`MT Product Families
`Windows
`
`oe
`
`Servers
`Developer Tools
`Business Solutions
`Games & Xbox
`
`206
`
`?
`
`205
`
`FIG. 2
`(PRIOR ART)
`
`4
`
`

`

`Patent Application Publication May 18,2006 Sheet 4 of 10
`
`or) Beiteacd
`
`3) tae arlted
`ry
`rts ote ne
`ge Ramee
`\ Ka) Castelnatg
`rN leeRsold
`ae)
`[a (aur)
`q
`Siena ise h
`BRHeEN gag
`
`ee Peeeels
`
`Peete]
`
`Cee anne US 2006/0107229 Al
`
`
`oo
`oO Lorem Ipsum
`oO Lorem Ipsum
`oO Loram ipsum
`oO Lorem ipsum
`oO Lorem ipsum
`oO Lorem Ipsum
`oO Lorem Ipsum
`Oo Lorem Ipsum
`oO Lorem Ipsum
`oO Lorem Ipsum
`
`303
`
`oO Lorem Ipsum
`Oo Lorem Ipsum
`oO Lorem Ipsum
`oO Lorem Ipsum
`oO Lorem Ipsum
`O Lorem Ipsum
`Oo Lorem ipsum
`oO Lorem tpsum
`oO Lorem tpsum
`
`306
`
`5
`
`

`

`Patent Application Publication May 18, 2006 Sheet 5 of 10
`
`US 2006/0107229 Al
`
`401
`
`
`
`“we
`
`FIG. 4B
`
`ws
`
`6
`
`

`

`Patent Application Publication May 18, 2006 Sheet 6 of 10
`
`US 2006/0107229 Al
`
`7
`
`

`

`Patent Application Publication May 18, 2006 Sheet 7 of 10
`
`US 2006/0107229 Al
`
`III
`
`404
`
`406
`
`401~
`
`|U
`
`|!I|!ItIt}lI{t|I!IIl!|!
`
`512
`
`613
`
`SN Q407
`
`LOREM IPSUM|@ LOREM IPSUM
`
`8
`
`

`

`Patent Application Publication May 18, 2006 Sheet 8 of 10
`
`US 2006/0107229 Al
`
`401 CU
`
`706a
`
`704
`
`706b
`
`
`
` .
`
`FIG. 7A
`
`
`
`9
`
`

`

`Patent Application Publication May 18, 2006 Sheet 9 of 10
`
`US 2006/0107229 Al
`
`
`
`
`SET TIME
`(a)[24f{16]
`
`UM.-—__
`
`801
`
`901
`
`FIG. 8
`
`me
`
`@ LOREM rn —
`
`
`
`@ Lorem ipsum
`
`
`
`@ “OREM Ipsum
`
`@ LOREM ipsum
`
`@ (orem ipsum
`
`10
`
`10
`
`

`

`Patent Application Publication May 18, 2006 Sheet 10 of 10
`
`US 2006/0107229 Al
`
`START
`
`INFORMATION
`COMPONENT
`TRIGGERED
`
`TRANSFORM
`PRESENTLY DISPLAYED
`WORK AREA
`
`IN PORTION REVEALED
`
`REVEAL PORTION OF
`PRESENTATION AREA
`
`DISPLAY INFORMATION
`COMPONENT
`
`1001
`
`1002
`
`1003
`
`1004
`
`1005
`
`FIG. 10
`
`USER DIRECTS NEXT
`
`COURSE OF ACTION
`
`1006
` NEW
`
`
`WORK AREA
`
`REQ'D?
`1008
`1007
`
`
`
`RETURN
` DISPLAY NEW
`
`
`WORK AREA
`WORK AREA
`
`TO FOREFRONT
`
`END
`
`11
`
`11
`
`

`

`US 2006/0107229 Al
`
`May 18, 2006
`
`WORK AREA TRANSFORM IN A GRAPHICAL
`USER INTERFACE
`
`FIELD OF THE INVENTION
`
`[0001] The invention relates generally to computer oper-
`ating systems. More specifically, the invention provides a
`method for transforming a work area (e.g., desktop) of a
`graphical operating system in a virtual three-dimensional
`space to view an information component in the revealed
`presentation area.
`
`BACKGROUND OF THE INVENTION
`
`[0002] Computer operating systems have evolved signifi-
`cantly in recent years. Typically, these systems have a shell
`that provides a graphical user interface (GUI) to an end-user.
`The shell consists of one or a combination of software
`components that provide direct communication between the
`user and the operating system. Speed improvements in
`computer hardware, e.g., memory, hard drives, processors,
`graphics cards, system buses, and the like, have enabled
`richer GUIs that are drastically easier for users to compre-
`hend. Accompanying hardware price reductions have made
`computer systems more affordable, enabling broad adoption
`of computers as productivity tools and multimedia systems.
`GUIs have allowed users who may have been unschooled or
`unfamiliar with computers to quickly and intuitively grasp
`the meaning of desktops, icons, windows, and applications,
`and how the user can interact with each.
`
`older application software andthe limited capabilities of the
`affordable range of video hardware. More recently, however,
`real-time 3D computer games have overtaken operating
`systemsas the primary marketincentive for advancingretail
`video hardware, which has in a short
`time attained an
`exceptional
`level of sophistication. Real time, hardware-
`based 3D acceleration is now available to consumers at
`
`reasonable cost. Thus, graphics hardware features once
`considered highly advanced, such as accelerated texture and
`lighting algorithms as well as 3D transformationsare readily
`available. At present, generally only game software and
`highly specialized graphics applications actively exploit
`such features.
`
`[0005] An operating system, such as Microsoft Windows
`XP® brand or Windows 2000® brand operating systems,
`will typically comprise a graphical method for launching
`new software applications within its GUI. FIG.2 illustrates
`a well-known example of how this may be accomplished in
`the Windows 2000 operating system. The screenshot 200
`displays desktop 201, bordered on one side by taskbar 203,
`and featuring open window 202. When a user desires to
`launch a new application, the user moves a pointer (also
`knownas a cursor) controlled by a mouse and clicks on the
`appropriate menu item in the Start Menu 204, whichis itself
`first invoked by clicking on the Start button 205. The Start
`button 205 is generally located in a fixed location on the
`taskbar 204. A user may adjust the location of the taskbar
`203, but once in place,
`the Start button 205 becomes a
`constant and familiar starting point for the user to launch
`new applications.
`
`[0003] The desktop illustrated in FIG. 2 has become the
`standard graphical metaphor for modern GUIs. The interface
`[0006] Whenauserclicks on the Start button 205 in FIG.
`is designed to model the real world activity of working at a
`2, the Start Menu 204 appears as a floating list on top (i.e.,
`desk. The desktop typically occupies the entire surface of a
`has a higher Z-order value) of the currently open window
`single display device, or may span multiple display devices,
`202 and desktop 201. A subsequent submenu 206 ofthe Start
`and hosts subordinate user interface objects such as icons,
`Menu 204, here triggered whenthe user clicks on or hovers
`menus, cursors and windows. The desktop serves as a base
`over the “Programs”list item, appears on top of and to the
`work area, where multiple documents and applications can
`right of the original Start Menu in order to show more
`sit open. To draw this virtual work area, the operating system
`choices. Only when the user finally clicks on the desired
`uses a simulated three-dimensional layering of windows and
`application in the Start Menu 204 or submenu 206 do the
`desktop drawn in a two dimensional graphical space, some-
`Menu and submenus disappear. In the meantime, the user
`times referring to this layering as Z-ordering. The term
`may be confused by the flat and overlapping menus and
`Z-order is derived from three dimensional (3D) geometry,
`windows which together create a crowded stack of infor-
`where the horizontal axis is typically known as the X-axis,
`mation. In addition, any content under the Start Menu 204
`the vertical axis is the Y-axis, and the Z axis sits perpen-
`and submenu(s) 206 is completely hidden from the user,
`dicular to the plane formed by the X and Y axes. Hence, the
`Z-order value for each window refers to that window’s
`preventing viewing of and interaction with obscured con-
`tent.
`
`relative position along an axis perpendicular to the desktop.
`Ultimately, Z-ordering is used to draw the two dimensional
`display, by determining which object is on top when two
`objects overlap. The operating system shell draws the object
`with the higher Z-order value, and subsequently draws the
`area of the second object not covered by the first object.
`Although Z-ordering is nominally derived from 3D geom-
`etry, the method only minimally exploits in two dimensions
`the capabilities inherent in having a true third Z dimension.
`
`[0004] To someextent, this two-dimensional shortcoming
`has been driven by the video hardware available in personal
`computers. In the past, advancements in mid- and lower-end
`computer video hardware have been driven in large part by
`the graphical services available in popular operating sys-
`tems. However, the graphical services available in these
`systems have not significantly advanced for a variety of
`reasons, including the need to maintain compatibility with
`
`12
`
`[0007] Using a broader perspective, a program launching
`menu,like the Start Menu, occupying the same work area as
`the software applications inhibits a user’s fundamental
`understanding of the operating system. Manipulating appli-
`cation windows and the content therein can be viewed as
`
`tasks within and under the auspices of the operating system.
`For these tasks (e.g. editing a documentor clicking on a link
`in a web page) the operating system can be viewed as
`arbitrating communication between the user and the appli-
`cation, displaying application output for the user, and pass-
`ing user input to the application. Using this same perspec-
`tive,
`launching a new application can be viewed as a
`meta-task, or as making a direct request of the operating
`system which operates outside the normal user-input-appli-
`cation-output model. That being the case, a program launch-
`ing menu which occupies an existing work area inhabited by
`
`12
`
`

`

`US 2006/0107229 Al
`
`May 18, 2006
`
`other windowsand icons has the potential to confuse an end
`user, both visually and conceptually.
`
`[0019] FIG. 4B illustrates a frontal view of the virtual
`presentation area of FIG. 4A as viewed by a user.
`
`it would be an advancement in the art to
`[0008] Thus,
`provide for viewing a program launching menu in a way
`which does not clutter a work area such as a desktop, and
`also conceptually decouples the operating system from the
`applications it hosts.
`BRIEF SUMMARY OF THE INVENTION
`
`[0009] The following presents a simplified summary of the
`invention in order to provide a basic understanding of some
`aspects of the invention. The summary is not an extensive
`overview of the invention. It is not intended to identify key
`or critical elements of the invention or to delineate the scope
`of the invention. The following summary merely presents
`some concepts of the invention in a simplified form as a
`prelude to the more detailed description below.
`
`[0010] A first embodiment of the invention provides a
`method for displaying content to a user through a three-
`dimensional graphical user interface on a computer. The
`method comprises transforming a presently displayed work
`area, which includes desktops, windows, and the like. The
`transformation can involve rotating the work area away from
`the user and revealing a portion of a presentation area
`situated behind the workarea. Finally, an information com-
`ponent, such as a Start Menu, is displayed in the visible
`portion of the presentation area.
`
`[0011] A second embodiment of the invention provides a
`computer system comprising a pointing device, a processor,
`a display, and a memory,
`the memory storing computer
`executable instructions. The computer executable instruc-
`tions provide for a graphical user interface using three-
`dimensional graphics. In addition, the computer executable
`instructions provide for transforming a presently displayed
`workarea, and displaying an information component in the
`portion of the presentation area revealed behind the work
`area.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0012] A more complete understanding of the present
`invention and the advantages thereof may be acquired by
`referring to the following description in consideration of the
`accompanying drawings, in which like reference numbers
`indicate like features, and wherein:
`
`[0013] FIG. 1A illustrates an operating environment that
`may be used for one or more aspects of an illustrative
`embodimentof the invention.
`
`[0014] FIG. 1B illustrates a distribution of functions and
`services among components that may be used for one or
`more aspects of an illustrative embodimentof the invention.
`
`FIG.2 is ascreenshot depicting a prior art example
`[0015]
`of a program launching menu in a computer operating
`system graphical user interface.
`
`[0016] FIG. 3A is a screenshot depicting an illustrative
`embodiment of the invention.
`
`[0017] FIG. 3B illustrates a wire frame version of the
`screenshot in FIG.3A.
`
`[0018] FIG. 4A illustrates a top view of a virtual presen-
`tation area prior to utilizing an illustrative embodiment of
`the invention.
`
`[0020] FIG. 5Aillustrates a top view of a virtual presen-
`tation area showing an illustrative embodimentof the inven-
`tion.
`
`[0021] FIG. 5B illustrates a frontal view of the virtual
`presentation area of FIG. 5A as viewed by a user.
`
`[0022] FIG. 6Aillustrates a top view of a virtual presen-
`tation area showing an illustrative embodimentof the inven-
`tion.
`
`[0023] FIG. 6B illustrates a frontal view of the virtual
`presentation area of FIG. 6A as viewed bya user.
`
`FIG.7A illustrates a top view of a virtual presen-
`[0024]
`tation area showing an illustrative embodimentof the inven-
`tion.
`
`[0025] FIG. 7B illustrates a frontal view of the virtual
`presentation area of FIG. 6A as viewed bya user.
`
`FIG.8 illustrates a portion of a frontal view of an
`[0026]
`illustrative embodiment of the invention.
`
`FIG.9 illustrates a portion of a frontal view of an
`[0027]
`illustrative embodiment of the invention.
`
`[0028] FIG. 10 illustrates a method for displaying an
`information componentin a graphicaluser interface accord-
`ing to an illustrative aspect of the invention.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`In the following description of the various embodi-
`[0029]
`ments, reference is made to the accompanying drawings,
`which form a part hereof, and in which is shown by way of
`illustration various embodiments in which the invention
`
`may bepracticed. It is to be understood that other embodi-
`ments may be utilized and structural and functional modi-
`fications may be made without departing from the scope and
`spirit of the present invention.
`
`[0030]
`
`Illustrative Operating Environment
`
`[0031] FIG. 1 illustrates an example of a suitable com-
`puting system environment 100 in which the invention may
`be implemented. The computing system environment 100 is
`only one example of a suitable computing environment and
`is not intended to suggest any limitation as to the scope of
`use or functionality of the invention. Neither should the
`computing environment 100 be interpreted as having any
`dependencyor requirementrelating to any one or combina-
`tion of components illustrated in the exemplary operating
`environment 100.
`
`[0032] The invention is operational with numerous other
`general purpose or special purpose computing system envi-
`ronments or configurations. Examples of well known com-
`puting systems, environments, and/or configurations that
`maybe suitable for use with the invention include, but are
`not limited to, personal computers; server computers; por-
`table and hand-held devices such as personal digital assis-
`tants (PDAs),
`tablet PCs or laptop PCs; multiprocessor
`systems; microprocessor-based systems; set top boxes; pro-
`grammable consumerelectronics; network PCs; minicom-
`
`13
`
`13
`
`

`

`US 2006/0107229 Al
`
`May 18, 2006
`
`puters; mainframe computers; distributed computing envi-
`ronments that include any of the above systems or devices;
`and thelike.
`
`[0033] The invention may be described in the general
`context of computer-executable instructions, such as pro-
`gram modules, being executed by a computer. Generally,
`program modules include routines, programs, objects, com-
`ponents, data structures, etc. that perform particular tasks or
`implementparticular abstract data types. The invention may
`also be practiced in distributed computing environments
`wheretasks are performed by remote processing devices that
`are linked through a communications network. In a distrib-
`uted computing environment, program modules may be
`located in both local and remote computer storage media
`including memory storage devices.
`
`[0034] With reference to FIG.1, an illustrative system for
`implementing the invention includes a general purpose
`computing device in the form of a computer 110. Compo-
`nents of computer 110 may include, but are not limited to,
`a processing unit 120, a system memory 130, and a system
`bus 121 that couples various system components including
`the system memory to the processing unit 120. The system
`bus 121 may be any of several types of bus structures
`including a memory bus or memory controller, a peripheral
`bus, and a local bus using any of a variety of bus architec-
`tures. By way of example, and not limitation, such archi-
`tectures include Industry Standard Architecture (ISA) bus,
`Micro Channel Architecture (MCA) bus, Enhanced ISA
`(EISA) bus, Video Electronics Standards Association
`(VESA)local bus, Advanced Graphics Port (AGP) bus, and
`Peripheral ComponentInterconnect (PCI) bus also known as
`Mezzaninebus.
`
`[0035] Computer 110 typically includes a variety of com-
`puter readable media. Computer readable media can be any
`available media that can be accessed by computer 110 and
`includesboth volatile and nonvolatile media, removable and
`non-removable media. By way of example, and not limita-
`tion, computer readable media may comprise computer
`storage media and communication media. Computer storage
`media includes volatile and nonvolatile, removable and
`non-removable media implemented in any methodor tech-
`nology for storage of information such as computer readable
`instructions, data structures, program modulesor other data.
`Computer storage media includes, but is not limited to,
`RAM, ROM, EEPROM,flash memory or other memory
`technology, CD-ROM, DVDor other optical disk storage,
`magnetic cassettes, magnetic tape, magnetic disk storage or
`other magnetic storage devices, or any other medium which
`can be used to store the desired information and which can
`
`accessed by computer 110. Communication media typically
`embodies computer readable instructions, data structures,
`program modules or other data in a modulated data signal
`such as a carrier wave or other transport mechanism and
`includes any information delivery media. The term “modu-
`lated data signal” meansa signal that has one or moreofits
`characteristics set or changed in such a manneras to encode
`information in the signal. By way of example, and not
`limitation, communication media includes wired media such
`as a wired network or direct-wired connection, and wireless
`media such as acoustic, RF,
`infrared and other wireless
`media. Combinationsof the any of the above should also be
`included within the scope of computer readable media.
`
`[0036] The system memory 130 includes computer stor-
`age media in the form of volatile and/or nonvolatile memory
`such as read only memory (ROM) 131 and random access
`memory (RAM) 132. A basic input/output system 133
`(BIOS), containing the basic routines that help to transfer
`information between elements within computer 110, such as
`during start-up, is typically stored in ROM 131. RAM 132
`typically contains data and/or program modules that are
`immediately accessible to and/or presently being operated
`on by processing unit 120. By way of example, and not
`limitation, FIG. 1 illustrates operating system 134, applica-
`tion programs 135, other program modules 136, and pro-
`gram data 137.
`
`[0037] The computer 110 may also include other remov-
`able/non-removable, volatile/nonvolatile computer storage
`media. By way of example only, FIG. 1 illustrates a hard
`disk drive 141 that reads from or writes to non-removable,
`nonvolatile magnetic media, a magnetic disk drive 151 that
`reads from or writes to a removable, nonvolatile magnetic
`disk 152, and an optical disk drive 155 that reads from or
`writes to a removable, nonvolatile optical disk 156 such as
`a CD ROMorother optical media. Other removable/non-
`removable, volatile/nonvolatile computer storage media that
`can be used in the exemplary operating environment
`include, but are not limited to, magnetic tape cassettes, flash
`memory cards, DVD, digital video tape, solid state RAM,
`solid state ROM,and the like. The hard disk drive 141 is
`typically connected to the system bus 121 through a non-
`removable memory interface such as interface 140, and
`magnetic disk drive 151 and optical disk drive 155 are
`typically connected to the system bus 121 by a removable
`memory interface, such as interface 150.
`
`[0038] The drives and their associated computer storage
`media discussed above andillustrated in FIG. 1, provide
`storage of computer readable instructions, data structures,
`program modules and other data for the computer 110. In
`FIG. 1, for example, hard disk drive 141 is illustrated as
`storing operating system 144, application programs 145,
`other program modules 146, and program data 147. Note
`that these components can either be the sameasor different
`from operating system 134, application programs 135, other
`program modules 136, and program data 137. Operating
`system 144, application programs 145, other program mod-
`ules 146, and program data 147 are given different numbers
`here to illustrate that, at a minimum,
`they are different
`copies. A user may enter commandsand information into the
`computer 110 through input devices such as a keyboard 162
`and pointing device 161, commonlyreferred to as a mouse,
`trackball or touch pad. Other input devices (not shown) may
`include a microphone, joystick, game pad, satellite dish,
`scamner, or the like. These and other input devices are often
`connected to the processing unit 120 through a user input
`interface 160 that is coupled to the system bus, but may be
`connected by other interface and bus structures, such as a
`parallel port, gameport, universal serial bus (USB), or IEEE
`1394 serial bus (FireWire). A monitor 184 or other type of
`display device is also connected to the system bus 121 via
`an interface, such as a video adapter 183. The video adapter
`183 may comprise advanced 3D graphics capabilities, in
`addition to its own specialized processor and memory.
`Computer 110 may also include a digitizer 185 to allow a
`user to provide input using a stylus input device 186. In
`addition to the monitor, computers may also include other
`
`14
`
`14
`
`

`

`US 2006/0107229 Al
`
`May 18, 2006
`
`peripheral output devices such as speakers 189 and printer
`188, which may be connected through an output peripheral
`interface 187.
`
`[0039] The computer 110 may operate in a networked
`environment using logical connections to one or more
`remote computers, such as a remote computer 180. The
`remote computer 180 may be a personal computer, a server,
`a router, a network PC, a peer device or other common
`network node, and typically includes many or all of the
`elements described above relative to the computer 110,
`although only a memory storage device 181 has been
`illustrated in FIG. 1. The logical connections depicted in
`FIG.1 include a local area network (LAN) 171 and a wide
`area network (WAN) 173, but may also include other
`networks. Such networking environments are commonplace
`in offices, enterprise-wide computer networks, intranets and
`the Internet.
`
`[0040] When used in a LAN networking environment, the
`computer 110 is connected to the LAN 171 through a
`network interface or adapter 170. When used in a WAN
`networking environment, the computer 110 may include a
`modem 172 or other meansfor establishing communications
`over the WAN 173, such as the Internet. The modem 172,
`which maybeinternal or external, may be connected to the
`system bus 121 via the user input interface 160, or other
`appropriate mechanism. In a networked environment, pro-
`gram modules depicted relative to the computer 110, or
`portions thereof, may be stored in the remote memory
`storage device. By way of example, and notlimitation, FIG.
`1 illustrates remote application programs 182 as residing on
`memory device 181. It will be appreciated that the network
`connections shown are exemplary and other meansofestab-
`lishing a communications link between the computers may
`be used.
`
`ILLUSTRATIVE EMBODIMENTS
`
`[0041] The invention may use a compositing desktop
`window manager (CDWM), further described below and in
`co-pending application Ser. No. 10/691,450, filed Oct. 23,
`2003 and entitled “Compositing Desktop Window Man-
`ager.” The CDWMis used to draw and maintain the display
`using a composited desktop model,
`i.e., a bottom-to-top
`rendering methodology in a virtual
`three-dimensional
`graphical space, as opposed to simulated 3D in a two-
`dimensional graphical space. The COWM may maintain
`content in a buffer memory area (for future reference). The
`CDWMcomposesthe display by drawing from the bottom
`up, beginning with the presentation area background, then a
`desktop background and proceeding through overlapping
`windowsin reverse Z order. While composing a desktop, the
`CDWMmaydraw each window basedin part on the content
`in front of which the window is being drawn (e.g., trans-
`parency), and based in part on other environmental factors
`(e.g., light source, reflective properties, etc.). For example,
`the CDWM mayuse the alpha channel of an ARGB format
`texture to provide transparency to a window, and may
`selectively emphasize portions of window content(e.g., the
`frame) based on a virtual light source.
`
`[0042] The CDWM mayrely upon a lowerlevel graphics
`compositing subsystem, referred to herein as a Unified
`Compositing Engine (UCE), further described below and in
`co-pending application serial number(attorney docket num-
`
`ber 50037.201US01), filed Oct. 23, 2003, entitled “System
`and Methodfor a Unified Composition Engine in a Graphics
`Processing System”, herein incorporated by reference in its
`entirety for all purposes. In one illustrative embodiment the
`UCEis based on or uses Direct3D® and DirectX® tech-
`nology by Microsoft Corporation of Redmond, Wash. In
`alternative embodiments other graphics compositing sub-
`systems may be used, such as variations of the X Window
`platform based on the OpenGL® graphics engine by Silicon
`Graphics, Inc. of Mountain View, Calif., and the like. The
`UCEenables 3D graphics, animation, transparency, shad-
`ows,lighting effects, bump mapping, environment mapping,
`and other rich visual features on the desktop.
`
`a component architecture
`[0043] FIG. 1B illustrates
`according to an illustrative embodiment of a desktop com-
`position platform. A Compositing Desktop Window Man-
`ager (CDWM) 190 may include an Application Program-
`ming Interface 190a through which a composition-aware
`Application Software 191 obtains window and content cre-
`ation and managementservices; a Subsystem Programming
`Interface 1906,
`through which the Legacy Windowing
`Graphics Subsystem 192 interacts; and a UI Object Manager
`190c which maintains a Z-ordered repository for UI objects
`such as presentation areas, work areas, desktops, windows
`and their associated content. The UI Object Manager may
`communicate with a Theme Manager 193 to retrieve
`resources, object behavioral attributes, and rendering met-
`rics associated with an active interface theme. The Legacy
`Graphical User Interface Subsystem 192 may include a
`Legacy Window Manager 192a and Legacy Graphics
`Device Interface 1928.
`
`[0044] A Unified Compositing Engine (UCE) 194 may
`service rendering instructions and coalesce resources emit-
`ted from the CDWMvia a ProgrammingInterface 194a. The
`UCE Programming Interface 194a provides an abstract
`interface to a broad range of graphics services including
`resource management, encapsulation from multiple-display
`scenarios, and remote desktop support. Rendering desktops
`to multiple displays requires abstraction ofthe differences in
`refresh rate, pixel format support, and device coordinate
`mapping among heterogeneous display devices. The UCE
`may provide this abstraction.
`
`[0045] Graphics resource contention between write opera-
`tions and rendering operations may be arbitrated by an
`internal Resource Manager 1945. Requests for resource
`updates and rendering services are placed on the UCE’s
`Request Queue 194c by the Programming Interface subcom-
`ponent 194a. These requests may be processed asynchro-
`nously by the Rendering Module 194dat intervals coincid-
`ing with the refresh rate of the display devices installed on
`the system. Thus, the Rendering Module 194d of the UCE
`194 may access and manipulate resources stored in the
`Resource Manager 194as necessary, and assemble and
`deliver display-specific rendering instructions to the 3D
`Graphics Interface 195.
`
`[0046] The UCE mayalso beresponsible for delivering
`graphics data over a network connection in remote display
`configurations. In order to efficiently remote the display of
`one particular system to another, resource contention should
`be avoided, performance optimizations should be enacted
`and security should be robust. These responsibilities may
`also rest with the UCE.
`
`15
`
`15
`
`

`

`US 2006/0107229 Al
`
`May 18, 2006
`
`[0047] The 3D Graphics Interface 195 may include a
`low-level, immediate-mode(stateless) graphics service such
`as Direct3D®, OpenGL®, or the like. A purpose of the 3D
`Graphics Interface may be to provide an abstract interface
`over the features of the particular graphics hardware con-
`figuration. The 3D Graphics Interface may service a single
`display device; the UCE mayparse and distribute rendering
`instructions among multiple graphics output devices 197 in
`a multiple-display system via multiple device drivers 196.
`
`It should be noted that the componentarchitecture
`[0048]
`depicted in FIG. 1B is that of an illustrative embodiment.
`The figure is intended to illustrate functions that the inven-
`tion may include. These functions maybe distributed among
`a fewer or greater number of software components than
`those represented in the figure, according to the capabilities
`of the platform and the desired feature set. For example, a
`system that lacks theme management mightderive all stock
`resources from the system rather than from a separate theme
`manager. Anotherpossible variation may eliminate the Sub-
`system Programming Interface 1906 if legacy application
`compatibility is not required. The subcomponents of the
`UCE 194 depicted in FIG. 1B may be broken out into
`separate processes, folded into the CDWM,or integrated
`into the 3D Graphics Interface. Thus a wide range of
`particular componentdesignsare possible, each of which are
`capable offulfilling