fJl.ol4.o0
`
`jcS42 U.S. PTO
`\ \\\\\\ \\\\\ \\\\\ \\\\\ \\\\\ \\\\\ \\\\ \\\\
`02/11/00
`
`UTILITY PATENT APPLICATION
`UNDER 37 CFR 1.53(b)
`
`Box PATENT APPLICATION
`Assistant Commissioner for Patents
`Washington, DC 20231
`
`Sir:
`
`Case Docket No. 55218-012
`
`Transmitted herewith for filing is the patent application of:
`
`INVENTORS: ASHAR AZIZ, TOM MARKSON, MARTIN PATTERSON
`FOR:
`EXTENSIBLE COMPUTING SYSTEM
`
`~ o o o
`~ o
`
`Enclosed are:
`52 pages of specification, claims, abstract.
`Declaration and Power of Attorney.
`Priority Claimed:
`Certified copy of
`18 sheets of formal rlro:lUT11nOC
`An assignment of the invention to
`and the assignment recordation fee.
`Information Disclosure Statement, Form PTO-1449 and reference.
`Return Receipt Postcard.
`
`Respectfully submitted,
`
`MCD~ERY
`
`Edward A. Becker
`Registration No. 37,777
`
`600 13th Street, N.W.
`Washington, D. C. 20005-3096
`(408) 271-2300 EAB:ccf
`Date: February 11, 2000
`Facsimile: (408) 271-2310
`
`EXPRESS MAIL CERTIFICATE OF MAILING
`
`"Express Mail" mailmg label number
`
`EL402670905US
`
`Date of Deposit
`
`2·11·2000
`
`I hereby certify that this paper or fee is being deposited with the United States Postal Service "Express Mail Post
`Office to Addressee" service under 37 CFR 1.10 on the date indicated above and is addressed to the Commissioner
`of Patents and Trademarks, Washington, D.C. 20231.
`
`r nrn ilin
`
`

`
`055218-0012
`
`Patent
`
`UNITED STATES PATENT APPLICATION
`
`FOR
`
`EXTENSIBLE COMPUTING SYSTEM
`
`INVENTORS:
`
`ASHARAzIZ
`TOM MARKSON
`MARTIN PATTERSON
`
`PREPARED BY:
`
`MCDERMOTT, WILL & EMERY
`600 13TH STREET, N.W.
`WASHINGTON, DC 20005-3096
`(202) 756-8000
`
`"Express Mail" mailing label number
`
`EXPRESS MAIL CERTIFICATE OF MAILING
`
`E.1.. "(;).z.' 11J9 0 G '" 6
`II 10 /)
`
`. ; /
`Date of Deposit
`I hereby certify that this paper or fee is b;ing deposited with the United States Postal Service "Express Mail Post Office to
`Addressee" service under 37 CFR 1.10 on the date indicated above and is addressed to the Assistant Commissioner for Patents,
`Washington, D.C. 20231.
`
`C1.Are.. e. ~ ,."" e~
`
`(Typed or printed name of person maili g paper or fee)
`
`

`
`EXTENSIBLE COMPUTING SYSTEM
`
`FIELD OF THE INVENTION
`
`The present invention generally relates to data processing. The invention relates
`
`more specifically to methods, apparatus, and mechanisms providing an extensible,
`
`5
`
`flexible, and scalable computing system.
`
`BACKGROUND OF THE INVENTION
`
`Builders of Web sites and other computer systems today have to deal with many
`
`systems planning issues. These include capacity planning for normal growth, expected or
`
`unexpected peak demand, availability and security of the site, etc. Companies who wish
`
`10
`
`to provide services on the Web have new business and service models, which are the
`
`areas in which they want to innovate and lead, but in order to do so they have to deal with
`
`the non-trivial complexity of designing, building and operating a large-scale Web site.
`
`This includes the need to grow and scale the site while it is operational.
`
`Doing all this requires finding and hiring trained personnel capable of engineering
`
`15
`
`and operating such a site, which may be potentially large and complicated. This is
`
`creating difficulty for many organizations, because designing, constructing and operating
`
`such large sites is simply not their core competency.
`
`One response to these issues is to host an enterprise Web site at a third party site,
`
`co-located with other Web sites of other enterprises. Such outsourcing facilities are
`
`20
`
`currently available from companies such as Exodus, AboveNet, GlobalCenter, etc. These
`
`facilities provide physical space, and redundant network and power facilities so that the
`
`enterprise customer or user need not provide them. The network and power facilities are
`
`shared among many enterprises or customers.
`
`However, the users of these facilities are still required to do a lot of work relating
`
`25
`
`to their computing infrastructure in the course of building, operating and growing their
`
`facilities. Information technology managers ofthe enterprises hosted at such facilities
`
`055218-0012
`
`-2-
`
`

`
`remain responsible for selecting, installing, configuring, and maintaining their own
`
`computing equipment at the facilities. The managers must still confront difficult issues
`
`such as resource planning and handling peak capacity.
`
`Even when outsourcing companies also provide computing facilities (e.g., Digex),
`
`5
`
`the facilities are no easier to scale and grow for the outsourcing company, because
`
`growth involves the same manual and error-prone administrative steps. In addition,
`
`problems remain with capacity planning for unexpected peak demand.
`
`Further, each Web site may have different requirements. For example, particular
`
`Web sites may require the ability to be independently administered and controlled. Others
`
`10 may require a particular type or level of security that isolates the Web site from all other
`
`sites that are co-located at the service provider. Others may require a secure connection to
`
`an enterprise Intranet located elsewhere.
`
`Also, various Web sites differ in internal topology. Some sites simply comprise a
`
`row of Web servers that are load balanced by a Web load balancer. Suitable load
`
`15
`
`balancers are Local Director from Cisco Systems, Inc., BigIP from F5Labs, Web Director
`
`from Alteon, etc. Other sites may be constructed in a multi-tier fashion, whereby a row of
`
`Web servers handle Hypertext Transfer Protocol (HTTP) requests, but the bulk of the
`
`application logic is implemented in separate application servers. These application
`
`servers in tum may need to be connected back to a tier of database servers.
`
`20
`
`Some of these different configuration scenarios are shown in FIG. lA, FIG. IB,
`
`and FIG. I C. FIG. IA is a block diagram of a simple Web site, comprising a single
`
`machine 100 comprising a CPU 102 and disk 104. Machine 100 is coupled to the global,
`
`packet-switched data network known as the Internet 106, or to another network. Machine
`
`100 may be housed in a co-location service ofthe type described above.
`
`25
`
`FIG. IB is a block diagram ofa I-tier Web server farm 110 comprising a plurality
`
`of Web servers WSA, WSB, WSC. Each of the Web servers is coupled to a load-balancer
`
`055218-0012
`
`-3-
`
`

`
`112 that is coupled to Internet 106. The load balancer divides the traffic between the
`
`servers to maintain a balanced processing load on each server. Load balancer 112 may
`
`also include or may be coupled to a firewall for protecting the Web servers from
`
`unauthorized traffic.
`FIG. Ie shows a 3-tier server farm 120 comprising a tier of Web servers WI, W2,
`
`5
`
`etc., a tier of application servers AI, A2, etc., and a tier of database servers D1, D2, etc.
`
`The Web servers are provided for handling HTTP requests. The application servers
`
`execute the bulk of the application logic. The database servers execute database
`
`management system (DBMS) software.
`
`10
`
`Given the diversity in topology of the kinds of Web sites that may need to be
`
`constructed, it may appear that the only way for constructing large-scale Web sites is to
`
`custom build each one. Indeed, this is the conventional approach. Many organizations are
`
`separately struggling with the same issues, and custom building each Web site from
`
`scratch. This is inefficient and involves a significant amount of duplicate work at
`
`15
`
`different enterprises.
`
`Still another problem with the conventional approach is resource and capacity
`
`planning. A Web site may receive vastly different levels of traffic on different days or at
`
`different hours within each day. At peak traffic times, the Web site hardware or software
`
`may be unable to respond to requests in a reasonable time because it is overloaded. At
`
`20
`
`other times, the Web site hardware or software may have excess capacity and be
`
`underutilized. In the conventional approach, finding a balance between having sufficient
`
`hardware and software to handle peak traffic, without incurring excessive costs or having
`
`over-capacity, is a difficult problem. Many Web sites never find the right balance and
`
`chronically suffer from under-capacity or excess capacity.
`
`25
`
`Yet another problem is failure induced by human error. A great potential hazard
`
`present in the current approach of using manually constructed server farms is that human
`
`055218-0012
`
`-4-
`
`

`
`error in configuring a new server into a live server fann can cause the server fann to
`
`malfunction, possibly resulting in loss of service to users of that Web site.
`
`Based on the foregoing, there is a clear need in this field for improved methods
`
`and apparatus for providing a computing system that is instantly and easily extensible on
`
`5
`
`demand without requiring custom construction.
`
`There is also a need for a computing system that supports creation of multiple
`
`segregated processing nodes, each of which can be expanded or collapsed as needed to
`
`account for changes in traffic throughput. Other needs will become apparent in the
`
`disclosure provided in this document.
`
`055218-0012
`
`-5-
`
`

`
`SUMMARY OF THE INVENTION
`
`The foregoing needs and objects, and other needs and objects that will become
`
`apparent from the following description, are achieved by the present invention, which
`
`comprises, in one aspect, a method and apparatus for creating highly scalable, highly
`
`5
`
`available and secure data processing sites, based on a wide scale computing fabric
`
`("computing grid"). The computing grid is physically constructed once, and then
`
`logically divided up for various organizations on demand. The computing grid comprises
`
`a large plurality of computing elements that are coupled to one or more VLAN switches
`
`and to one or more storage area network (SAN) switches. A plurality of storage devices
`
`10
`
`are coupled to the SAN switches and may be selectively coupled to one or more of the
`
`computing elements through appropriate switching logic and commands. One port ofthe
`
`VLAN switch is coupled to an external network, such as the Internet. A supervisory
`
`mechanism, layer, machine or process is coupled to the VLAN switches and SAN
`
`switches.
`
`15
`
`Initially, all storage devices and computing elements are assigned to Idle Pools.
`
`Under program control, the supervisory mechanism dynamically configures the VLAN
`
`switches and SAN switches to couple their ports to one or more computing elements and
`
`storage devices. As a result, such elements and devices are logically removed from the
`
`Idle Pools and become part of one or more virtual server farms (VSFs). Each VSF
`
`20
`
`computing element is pointed to or otherwise associated with a storage device that
`
`contains a boot image usable by the computing element for bootstrap operation and
`
`production execution.
`
`By physically constructing the computing grid once, and securely and
`
`dynamically allocating portions of the computing grid to various organizations on
`
`25
`
`demand, economies of scale are achieved that are difficult to achieve when doing a
`
`custom build of each site.
`
`055218-0012
`
`-6-
`
`

`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention is illustrated by way of example, and not by way of
`
`limitation, in the figures of the accompanying drawings and in which like reference
`
`numerals refer to similar elements and in which:
`
`5
`
`FIG. lA is a block diagram ofa simple Web site having a single machine
`
`topology.
`
`FIG. IB is a block diagram of a one-tier Web server farm.
`
`FIG. IC is a block diagram of a three-tier Web server farm.
`
`FIG. 2 is a block diagram of one configuration of an extensible computing system
`
`10
`
`200 that includes a local computing grid.
`
`FIG. 3 is a block diagram of an exemplary virtual server farm featuring a SAN
`
`Zone.
`
`FIG. 4A, FIG. 4B, FIG. 4C, and FIG. 4D are block diagrams showing successive
`
`steps involved in adding a computing element and removing element from a virtual
`
`15
`
`server farm.
`
`FIG. 5A is a block diagram of an embodiment of a virtual server farm system,
`
`computing grid, and supervisory mechanism.
`
`FIG. 5B is a block diagram of a system in which a Supervisor or Control Plane
`
`server farm is protected by a firewall.
`
`20
`
`FIG. 6 is a block diagram of logical connections of a virtual server farm.
`
`FIG. 7 is a block diagram oflogical connections of a virtual server farm.
`
`FIG. 8 is a block diagram oflogical connections of a virtual server farm.
`
`FIG. 9 is a block diagram of a Control Plane server farm.
`
`FIG. lOis a block diagram showing connections of Control Plane machines to an
`
`25
`
`embodiment that uses a plurality of SAN switches ("SAN mesh").
`
`055218-0012
`
`-7-
`
`

`
`FIG. 11 is a block diagram of a plurality ofVSFs extended over WAN
`
`connections.
`
`FIG. 12 is a block diagram of a computer system with which an embodiment may
`
`be implemented.
`
`055218-0012
`
`-8-
`
`

`
`DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
`
`A method and apparatus for providing an extensible computing system is
`
`described. In the following description, for the purposes of explanation, numerous
`
`specific details are set forth in order to provide a thorough understanding of the present
`
`5
`
`invention. It will be apparent, however, to one skilled in the art that the present invention
`
`may be practiced without these specific details. In other instances, well-known structures
`
`and devices are shown in block diagram form in order to avoid unnecessarily obscuring
`
`the present invention.
`
`VIRTUAL SERVER FARM (VSF)
`
`10
`
`According to one embodiment, a wide scale computing fabric ("computing grid")
`
`is provided. The computing grid can be physically constructed once, and then logically
`
`divided up for various organizations on demand. A part of the computing grid is allocated
`
`to each of a plurality of enterprises or organizations. Each organization's logical pOliion
`
`of the computing grid is referred to as a Virtual Server Farm (VSF). Each organization
`
`15
`
`retains independent administrative control of its VSF. Each VSF can change dynamically
`
`in terms of number ofCPUs, storage capacity and disk and network bandwidth based on
`
`real-time demands placed on the server farm or other factors. Each VSF is secure from
`
`every other organizations' VSF, even though they are all logically created out of the same
`
`physical computing grid. A VSF can be connected back to an Intranet using either a
`
`20
`
`private leased line or a Virtual Private Network (VPN), without exposing the Intranet to
`
`other organizations' VSFs.
`
`An organization can access only the data and computing elements in the portion
`
`of the computing grid allocated to it, that is, in its VSF, even though it may exercise full
`
`(e.g. super-user or root) administrative access to these computers and can observe all
`
`25
`
`traffic on Local Area Networks (LANs) to which these computers are connected. This is
`
`055218-0012
`
`-9-
`
`

`
`accomplished using a dynamic fire-walling scheme, where the security perimeter of the
`
`VSF expands and shrinks dynamically.
`
`Each VSF can be used to host the content and applications of an organization
`
`which may be accessed via the Internet, Intranet or Extranet.
`
`5
`
`Configuration and control of the computing elements and their associated
`
`networking and storage elements is performed by a supervisory mechanism which is not
`
`directly accessible through any of the computing elements in the computing grid. For
`
`convenience, in this document the supervisory mechanism is called Control Plane and
`
`may comprise one or more processors or a network of processors. The supervisory
`
`10 mechanism may comprise a Supervisor, Controller, etc. Other approaches may be used,
`
`as described herein.
`
`The Control Plane runs on a completely independent set of computing elements
`
`assigned for supervisory purposes, such as one or more servers that may be
`
`;li:
`
`interconnected in a network or by other means. It performs control actions on the
`
`15
`
`computing, networking and storage elements of the computing grid through special
`
`control ports or interfaces of the networking and storage elements in the grid. The
`
`Control Plane provides a physical interface to switching elements of the system, monitors
`
`loads of computing elements in the system, and provides administrative and management
`
`functions using a graphical user interface or other suitable user interface.
`
`20
`
`Computers running the Control Plane are logically invisible to the computers in
`
`the computing grid (and therefore in any specific VSF) and cannot be attacked or
`
`subverted in any way via elements in the computing grid or from external computers.
`
`Only the Control Plane has physical connections to the control ports on devices in the
`
`computing grid, which controls membership in a particular VSF. The devices in the
`
`25
`
`computing can be configured only through these special control ports, and therefore
`
`055218-0012
`
`-10-
`
`

`
`computing elements in the computing grid are unable to change their security perimeter
`
`or access storage or computing devices which they are not authorized to do.
`
`Thus, a VSF allows organizations to work with computing facilities that appear to
`
`comprise a private server farm, dynamically created out of a large-scale shared
`
`5
`
`computing infrastructure, namely the computing grid. A Control Plane coupled with the
`
`computing architecture described herein provides a private server farm whose privacy
`
`and integrity is protected through access control mechanisms implemented in the
`
`hardware of the devices of the computing grid.
`
`The internal topology of each VSF is controlled by the Control Plane. The Control
`
`10
`
`Plane can take the basic interconnection of computers, network switches and storage
`
`network switches described herein and use them to create a variety of server farm
`
`configurations. These include but are not limited to, single-tier Web server farms front(cid:173)
`
`ended by a load balancer, as well as multi-tier configurations, where a Web server talks
`
`to an application server, which in tum talks to a database server. A variety ofload
`
`15
`
`balancing, multi-tiering and fire-walling configurations are possible.
`
`THE COMPUTING GRID
`
`The computing grid may exist in a single location or may be distributed over a
`
`wide area. First this document describes the computing grid in the context of a single
`
`building-sized network, composed purely oflocal area technologies. Then the document
`
`20
`
`describes the case where the computing grid is distributed over a wide area network
`
`(WAN).
`
`FIG. 2 is a block diagram of one configuration of an extensible computing system
`
`200 that includes a local computing grid 208. In this document "extensible" generally
`
`means that the system is flexible and scalable, having the capability to provide increased
`
`25
`
`or decreased computing power to a particular enterprise or user upon demand. The local
`
`computing grid 208 is composed of a large number of computing elements CPU1, CPU2,
`
`055218-0012
`
`-11-
`
`

`
`... CPUn. In an exemplary embodiment, there may be 10,000 computing elements, or
`
`more. These computing elements do not contain or store any long-lived per-element state
`
`information, and therefore may be configured without persistent or non-volatile storage
`
`such as a local disk. Instead, all long lived state information is stored separate from the
`
`5
`
`computing elements, on disks DISKl, DISK2, ... DISKn that are coupled to the
`
`computing elements via a Storage Area Network (SAN) comprising one or more SAN
`
`Switches 202. Suitable SAN switches are commercially available from Brocade and
`
`Excel.
`
`All of the computing elements are interconnected to each other through one or
`
`10 more VLAN switches 204 which can be divided up into Virtual LANs (VLANs). The
`
`VLAN switches 204 are coupled to the Internet 106. In general a computing element
`
`contains one or two network interfaces connected to the VLAN switch. For the sake of
`
`simplicity, in FIG. 2 all nodes are shown with two network interfaces, although some
`
`may have less or more network interfaces. Many commercial vendors now provide
`
`15
`
`switches supporting VLAN functionality. For example, suitable VLAN switches are
`
`commercially available from Cisco Systems, Inc. and Xtreme Networks. Similarly there
`
`are a large number of commercially available products to construct SANs, including
`
`Fibre Channel switches, SCSI-to-Fibre-Channel bridging devices, and Network Attached
`
`Storage (NAS) devices.
`
`20
`
`Control Plane 206 is coupled by a SAN Control path, CPU Control path, and
`
`VLAN Control path to SAN switches 202, CPUs CPUl, CPU2, ... CPUn, and VLAN
`
`Switches 204, respectively.
`
`Each VSF is composed of a set ofVLANs, a set of computing elements that are
`
`attached to the VLANs, and a subset ofthe storage available on the SAN that is coupled
`
`25
`
`to the set of computing elements. The subset of the storage available on the SAN is
`
`referred to as a SAN Zone and is protected by the SAN hardware from access from
`
`055218-0012
`
`-12-
`
`

`
`computing elements which are part of other SAN zones. Preferably, VLANs that provide
`
`non-forgeable port identifiers are used to prevent one customer or end user from
`
`obtaining access to VSF resources of another customer or end user.
`
`FIG. 3 is a block diagram of an exemplary virtual server farm featuring a SAN
`
`5
`
`Zone. A plurality of Web servers WS1, WS2, etc. are coupled by a first VLAN (VLAN1)
`
`to a load balancer(LB)lfirewall 302. A second VLAN (VLAN2) couples the Internet 106
`
`to the load balancer(LB)/firewall 302. Each ofthe Web servers may be selected from
`
`among CPU1, CPU2, etc., using mechanisms described further herein. The Web servers
`
`are coupled to a SAN Zone 304, which is coupled to one or more storage devices 306a,
`
`10
`
`306b.
`
`At any given point in time, a computing element in the computing grid, such as
`
`CPUl of FIG. 2, is only connected to the set ofVLANs and the SAN zone(s) associated
`
`with a single VSF. A VSF typically is not shared among different organizations. The
`
`subset of storage on the SAN which belongs to a single SAN zone, and the set ofVLANs
`
`15
`
`associated with it and the computing elements on these VLANs define a VSF.
`
`By controlling the membership of a VLAN and the membership of a SAN zone,
`
`Control Plane enforces a logical partitioning of the computing grid into multiple VSFs.
`
`Members of one VSF cannot access the computing or storage resources of another VSF.
`
`Such access restrictions are enforced at the hardware level by the VLAN switches, and by
`
`20
`
`port-level access control mechanisms (e.g., zoning) of SAN hardware such as Fibre
`
`Channel switches and edge devices such as SCSI to Fibre Channel bridging hardware.
`
`Computing elements that form part of the computing grid are not physically connected to
`
`the control ports or interfaces of the VLAN switches and the SAN switches, and therefore
`
`cannot control the membership ofthe VLANs or SAN zones. Accordingly, the computing
`
`25
`
`elements of the computing grid cannot access computing elements not located in the VSF
`
`in which they are contained.
`
`055218-0012
`
`-13-
`
`

`
`Only the computing elements that run the Control Plane are physically connected
`
`to the control ports or interface of the devices in the grid. Devices in the computing grid
`
`(computers, SAN switches and VLAN switches) can only be configured through such
`
`control ports or interfaces. This provides a simple yet highly secure means of enforcing
`
`5
`
`the dynamic partitioning of the computing grid into multiple VSFs.
`
`Each computing element in a VSF is replaceable by any other computing element.
`
`The number of computing elements, VLANs and SAN zones associated with a given
`
`VSF may change over time under control of the Control Plane.
`
`In one embodiment, the computing grid includes an Idle Pool that comprises large
`
`10
`
`number of computing elements that are kept in reserve. Computing elements from the
`
`Idle Pool may be assigned to a particular VSF for reasons such as increasing the CPU or
`
`memory capacity available to that VSF, or to deal with failures of a particular computing
`
`element in a VSF. When the computing elements are configured as Web servers, the Idle
`
`Pool serves as a large "shock absorber" for varying or "bursty" Web traffic loads and
`
`15
`
`related peak processing loads.
`
`The Idle Pool is shared between many different organizations, and therefore it
`
`provides economies of scale, since no single organization has to pay for the entire cost of
`
`the Idle Pool. Different organizations can obtain computing elements from the Idle Pool
`
`at different times in the day, as needed, thereby enabling each VSF to grow when
`
`20
`
`required and shrink when traffic falls down to normal. If many different organizations
`
`continue to peak at the same time and thereby potentially exhaust the capacity of the Idle
`
`Pool, the Idle Pool can be increased by adding more CPUs and storage elements to it
`
`(scalability). The capacity of the Idle Pool is engineered so as to greatly reduce the
`
`probability that, in steady state, a particular VSF may not be able to obtain an additional
`
`25
`
`computing element from the Idle Pool when it needs to.
`
`055218-0012
`
`-14-
`
`

`
`FIG. 4A, FIG. 4B, FIG. 4C, and FIG. 4D are block diagrams showing successive
`
`steps involved in moving a computing element in and out of the Idle Pool. Referring first
`
`to FIG. 4A, assume that the Control Plane has logically connected elements of the
`
`computing grid into first and second VSFs labeled VSFI, VSF2. Idle Pool 400 comprises
`
`5
`
`a plurality ofCPUs 402, one of which is labeled CPUx. In FIG. 4B, VSFI has developed
`
`a need for an additional computing element. Accordingly, the Control Plane moves
`
`CPUX from Idle Pool 400 to VSF 1, as indicated by path 404.
`
`In FIG. 4C, VSFI no longer needs CPUX, and therefore the Control Plane moves
`
`CPUX out ofVSFI and back into the Idle Pool 400. In FIG. 4D, VSF2 has developed a
`
`10
`
`need for an additional computing element. Accordingly, the Control Plane moves CPUX
`
`from the Idle Pool 400 to VSF2. Thus, over the course of time, as traffic conditions
`
`change, a single computing element may belong to the Idle Pool (FIG. 4A), then be
`
`assigned to a particular VSF (FIG. 4B), then be placed back in the Idle Pool (FIG. 4C),
`
`::::.
`
`and then belong to another VSF (FIG. 4D).
`
`15
`
`At each one of these stages, The Control Plane configures the LAN switches and
`
`SAN switches associated with that computing element to be part ofthe VLANs and SAN
`
`zones associated with a particular VSF (or the Idle Pool). According to one embodiment,
`
`in between each transition, the computing element is powered down or rebooted. When it
`
`is powered back up, it views a different portion of storage zone on the SAN, which
`
`20
`
`includes a bootable image of an operating system (e.g., Linux, NT, Solaris, etc.). The
`
`storage zone also includes a data portion that is specific to each organization (e.g., files
`
`associated with a Web server, database partitions, etc.). It is also part of another VLAN
`
`which is part ofthe VLAN set of another VSF, so it can access CPUs, SAN storage
`
`devices and NAS devices associated with the VLANs of the VSF into which it has been
`
`25
`
`transitioned.
`
`055218-0012
`
`-15-
`
`

`
`In a preferred embodiment, the storage zones include a plurality of pre-defined
`
`logical blueprints that are associated with roles that may be assumed by the computing
`
`elements. Initially, no computing element is dedicated to any particular role or task such
`
`as Web server, application server, database server, etc. The role of the computing element
`
`5
`
`is acquired from one of a plurality of pre-defined, stored blueprints, each of which
`
`defines a boot image for the computing elements that are associated with that role. The
`
`blueprints may be stored in the form of a file, a database table, or any other storage
`
`format that can associate a boot image location with a role.
`
`Thus, the movements of CPUX in FIG. 4A, FIG. 4B, FIG. 4C, FIG. 4D are
`
`10
`
`logical, not physical, and are accomplished by re-configuring VLAN switches and SAN
`
`Zones under control of The Control Plane. Further, each computing element in the
`
`computing grid initially is essentially fungible, and assumes a specific processing role
`
`only after it is connected in a virtual server farm and loads software from a boot image.
`
`No computing element is dedicated to any particular role or task such as Web server,
`
`15
`
`application server, database server, etc. The role of the computing element is acquired
`
`from one of a plurality of pre-defined, stored blueprints, each of which is associated with
`
`a role, each of which defines a boot image for the computing elements that are associated
`
`with that role.
`
`Since there is no long-lived state information stored in any given computing
`
`20
`
`element (such as a local disk), nodes are easily moved between different VSFs, and can
`
`run completely different OS and application software. This also makes each computing
`
`element highly replaceable, in case of planned or unplanned downtime.
`
`A particular computing element may perform different roles as it is brought into
`
`and out of various VSFs. For example, a computing element may act as a Web server in
`
`25
`
`one VSF, and when it is brought into a different VSF, it may be a database server, a Web
`
`load balancer, a Firewall, etc. It may also successively boot and run different operating
`
`055218-0012
`
`-16-
`
`

`
`systems such as Linux, NT or Solaris in different VSFs. Thus, each computing element in
`
`the computing grid is fungible, and has no static role assigned to it. Accordingly, the
`
`entire reserve capacity of the computing grid can be used to provide any of the services
`
`required by any VSF. This provides a high degree of availability and reliability to the
`
`5
`
`services provided by a single VSF, because each server performing a particular service
`
`has potentially thousands of back-up servers able to provide the same service.
`
`Further, the large reserve capacity of the computing grid can provide both
`
`dynamic load balancing properties, as well as high processor availability. This capability
`
`is enabled by the unique combination of diskless computing elements interconnected via
`
`10 VLANs, and connected to a configurable zone of storage devices via a SAN, all
`
`controlled in real-time by The Control Plane. Every computing element can act in the role
`
`of any required server in any VSF, and can connect to any logical partition of any disk in
`
`the SAN. When the grid requires more computing power or disk capacity, computing
`
`elements or disk storage is manually added to the idle pool, which may decrease over
`
`15
`
`time as more organizations are provided VSF services. No manual intervention is
`
`required in order to increase the number ofCPUs, network and disk bandwidth and
`
`storage available to a VSF. All such resources are allocated on demand from CPU,
`
`network and disk resources available in the Idle Pool by the Control Plane.
`
`A particular VSF is not subjected to manual reconfiguration. Only the machines
`
`20
`
`in the idle pool are manually configured into the computing grid. As a result, a great
`
`potential hazard present in current manually constructed server farms is removed. The
`
`possibility that human error in configuring a new server into a live server farm can cause
`
`the server farm to malfunction, possibly resulting in loss of service to users of that Web
`
`site, is virtually eliminated.
`
`25
`
`The Control Plane also replicates data stored in SAN attached storage devices, so
`
`that failure of any particular storage element does not cause a loss of service to any part
`
`055218-0012
`
`-17-
`
`

`
`ofthe system. By decoupling long-lived storage from computing devices using SANs,
`
`and by providing redundant storage and computing elements, where any computing
`
`element can be attached to any storage partition, a high degree of availability is achieved.
`
`A DETAILED EXAMPLE OF ESTABLISHING A VIRTUAL SERVER FARM,
`
`5 ADDING A PROCESSOR TO IT, AND REMOVING A PROCESSOR FROM IT
`
`FIG. 5A is a block diagram of a VSF system according to an embodiment. With
`
`reference to FIG. 5A, the following describes the detailed steps that may be used to create
`
`a VSF, add nodes to it and delete nodes from it.
`
`FIG. 5A depicts computing elements 502, comprising computers A through G,
`
`10
`
`coupled to VLAN capable switch 504. VLAN switch 504 is coupled to Internet 106, and
`
`the VLAN switch has ports VI, V2, etc. Computers A through G are f