I 1111111111111111 11111 111111111111111 IIIII IIIII IIIII 111111111111111 IIII IIII
`US008381209B2
`
`c12) United States Patent
`Reumann et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 8,381,209 B2
`Feb.19,2013
`
`(54) MOVEABLE ACCESS CONTROL LIST (ACL)
`MECHANISMS FOR HYPERVISORS AND
`VIRTUAL MACHINES AND VIRTUAL PORT
`FIREWALLS
`
`(75)
`
`Inventors: John Reumann, Croton on Hudson, NY
`(US); Debanjan Saha, Mohegan Lake,
`NY (US); Sambit Sahu, Hopewell
`Junction, NY (US); Dinesh Chandra
`Verma, Mount Kisco, NY (US)
`
`(73) Assignee: International Business Machines
`Corporation, Armonk, NY (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1429 days.
`
`(21) Appl. No.: 11/619,536
`
`(22) Filed:
`
`Jan.3,2007
`
`(65)
`
`(51)
`
`(52)
`(58)
`
`Prior Publication Data
`
`US 2008/0163207 Al
`
`Jul. 3, 2008
`
`Int. Cl.
`G06F 9/455
`(2006.01)
`U.S. Cl. ............................. 718/1; 709/250; 718/102
`Field of Classification Search .................. 709/250;
`718/1, 102
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,386,552 A
`1/1995 Garney
`6,496,847 Bl
`12/2002 Bugnion et al.
`6,691,146 Bl
`2/2004 Armstrong et al.
`6,795,966 Bl
`9/2004 Lim et al.
`2004/0015966 Al
`1/2004 MacChiano et al.
`8/2004 O'Brien
`2004/0158720 Al
`2006/0136653 Al *
`6/2006 Traut et al.
`2006/0143311 Al *
`6/2006 Madukkarumukumana
`et al.
`2006/0236127 Al* 10/2006 Kurien et al.
`2008/0034234 Al*
`2/2008 Shimizu et al.
`2008/0244569 Al* 10/2008 Challener et al.
`2009/0025007 Al*
`1/2009 Hara et al.
`2009/0119684 Al*
`5/2009 Mahalingarn et al.
`2009/0129385 Al*
`5/2009 Wray et al.
`2009/0249438 Al* 10/2009 Litvin et al.
`2009/0296726 Al* 12/2009 Snively et al.
`2009/0328074 Al* 12/2009 Oshins
`
`* cited by examiner
`
`711/6
`
`710/1
`713/193
`713/320
`718/1
`718/105
`719/324
`370/392
`726/1
`370/401
`719/321
`
`Primary Examiner - Mohamed Wasel
`(74) Attorney, Agent, or Firm -Eustus D. Nelson, Esq.;
`McGinn IP Law Group, PLLC
`
`ABSTRACT
`(57)
`A method (and system) which provides virtual machine
`migration with filtered network connectivity and control of
`network security of a virtual machine by enforcing network
`security and routing at a hypervisor layer at which the virtual
`machine partition is executed, and which is independent of
`guest operating systems.
`
`17 Claims, 18 Drawing Sheets
`
`START
`
`400
`
`copy network security and routing for the virtual machine to
`the hypervisor layer
`
`~ 401
`
`migrating the virtual machine from a first hardware device to
`a second hardware device
`
`~ 4 02
`
`updating routing controls for the virtual machine at the
`hypervisor level
`
`updating traffic filters for the virtual machine at the
`hypervisor level ( e.g .. by setting hypervisor firewalls to
`permit network traffic for the virtual machine to access the
`second hardware device)
`
`404
`
`advertising ( e.g., by said second hardware device) the
`migration of said virtual machine from the first hardware
`device to the second hardware device
`
`~ 405
`
`407
`
`routing network traffic for the
`virtual machine to the second
`hardware device based on the
`routing controls
`
`granting to the virtual machine on
`said second hardware device based
`on the traffic filters (e.g., ACLs).
`
`Microsoft Ex. 1001, p. 1
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 1 of 18
`
`US 8,381,209 B2
`
`100
`
`ISP
`
`...
`
`FIGURE 1
`
`Microsoft Ex. 1001, p. 2
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 2 of 18
`
`US 8,381,209 B2
`
`hypervisor
`<
`..-N-1 C-1- 2
`("')
`
`I Switch1 I
`
`I FW1 I
`
`200
`
`copy
`start
`
`hypervisor
`·····::l.
`NIC2 0
`Switch2
`
`NOACL
`
`~
`ALLOW WORLD ~
`
`FIGURE 2
`
`Microsoft Ex. 1001, p. 3
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 3 of 18
`
`US 8,381,209 B2
`
`hypervis
`
`NIC1
`I Switch1
`I FW1 I
`
`copy
`start
`
`E]
`
`hypervis
`
`NIC2
`
`ACL does not allow VM
`
`I Fw2 I ~ DENY
`
`FIGURE 3
`
`Microsoft Ex. 1001, p. 4
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 4 of 18
`
`US 8,381,209 B2
`
`START
`
`400
`
`copy network security and routing for the virtual machine to ~ 401
`the hypervisor layer
`
`migrating the virtual machine from a first hardware device to
`a second hardware device
`
`~402
`
`updating routing controls for the virtual machine at the
`hypervisor level
`
`~403
`
`updating traffic filters for the virtual machine at the
`hypervisor level (e.g., by setting hypervisor firewalls to
`permit network traffic for the virtual machine to access the
`second hardware device)
`
`404
`
`advertising (e.g., by said second hardware device) the
`migration of said virtual machine from the first hardware
`device to the second hardware device
`
`~4 0 5
`
`406
`
`407
`
`routing network traffic for the
`virtual machine to the second
`hardware device based on the
`routing controls
`
`granting to the virtual machine on
`said second hardware device based
`on the traffic filters (e.g., ACLs).
`
`FIGURE 4
`
`Microsoft Ex. 1001, p. 5
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 5 of 18
`
`US 8,381,209 B2
`
`VM
`
`lwinXPI
`
`vm
`
`ALLOW IN/OUT: MAC, IP, (5 tuples)
`Access policies
`IP address
`
`deployment editor
`Filters can be updated
`w/o running VM
`
`FIGURE 5
`
`Microsoft Ex. 1001, p. 6
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 6 of 18
`
`US 8,381,209 B2
`
`600
`
`VM
`
`ALLOW IN/OUT: MAC, IP, (5 tuples)
`Access policies
`IP address
`
`Stored in control center
`application (e.g. director)
`
`FIGURE 6
`
`Microsoft Ex. 1001, p. 7
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 7 of 18
`
`US 8,381,209 B2
`
`700
`
`VM
`
`vm
`
`FWinXP)
`Hypervisometwo!k
`serializeldeserialize
`mobility layer
`. - - - - - - - - - - -1 VNIC 1 - - - - -g . ._ - -~
`
`OSPF peer
`Deliver to/from VNIC
`
`Network
`ACL
`Editor
`
`~--U_p_da_~_r_ou_~_t_o_V_M_~v~N~G-~
`
`NIC
`
`FIGURE 7
`
`Microsoft Ex. 1001, p. 8
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 8 of 18
`
`US 8,381,209 B2
`
`800
`
`L2 control bloc::.k::_ __ ...1.._.=1P=H;-.F-IE_L_D_<_op_>_P_ATTT_E_R_N_,
`
`Navigable list for admin
`
`. - - - - - MAC<op>PATTERN
`
`!PH.FIELD <op> PATTERN
`
`ETH, TR,
`VMNETL,
`*
`
`IP, IPX,
`
`igmp
`
`UCP
`TCP
`ICMP,
`RTP,
`
`Policy
`ptr
`
`L3 PROTO
`
`L4 PROTO
`
`prevACL nextACL
`
`L2 PROTO
`
`Each.field is optional
`
`t /~
`
`..,_ _____ Named ACL directory
`
`ACL Head
`Identifies VM MAC to which AGL bound
`
`Maps human readable name.~ to ACLs
`
`FIGURE 8
`
`Microsoft Ex. 1001, p. 9
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb. 19, 2013
`
`Sheet 9 of 18
`
`US 8,381,209 B2
`
`Index ofACLs created
`using well-known boolean
`expression minimization,
`tries and the like
`
`ACL index
`
`900
`
`GetMatch
`Find ACL
`
`ApplyVLAN
`tag
`
`Remove VLAN
`tag
`
`(2)
`
`(1b)
`
`( 1)
`
`Real
`NIC
`
`(6)
`
`TAP
`incoming
`
`Hypervisor
`Network packe
`delivery code
`
`TAP
`outgoing
`
`Virtual
`NIC
`
`(8.1------,
`Guest
`VM
`
`(4)
`
`(5)
`
`Handle packet
`according to policy
`
`*NIC=network interface card
`Handle packet
`according to policy Outbound path is symmetrical
`
`FIGURE 9
`
`Microsoft Ex. 1001, p. 10
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 10 of 18
`
`US 8,381,209 B2
`
`1000
`
`Forward
`·paeke•·
`normally
`
`NO
`
`Return fixed IP as □ HCP
`lease
`
`Act as □ HCP proxy to
`real □ HCP server
`soecified in vNIC confio
`
`NO
`
`NO
`
`Emulate ARP with
`reduced timeout
`
`On timeout return MAC
`address of gateway
`according to OSPF
`
`FIGURE 10
`
`Microsoft Ex. 1001, p. 11
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 11 of 18
`
`US 8,381,209 B2
`
`1100
`
`Hypervisor network
`Control Window
`
`-
`I -
`I
`
`--
`
`Hypervisor FIB
`
`Virtual NIC
`ID
`
`IP address I
`Subnet pairs
`
`I
`I
`
`11
`
`11
`
`I
`
`Change trigger
`
`OSPF
`~ module
`.
`
`VLANID
`module
`
`-~
`
`Ir
`To OSPF peers
`Advertise any host/net on list
`
`FIGURE 11
`
`I Virtual NIC ID II TAG I
`I
`I
`
`I
`I
`
`I
`
`11
`II
`
`Microsoft Ex. 1001, p. 12
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 12 of 18
`
`US 8,381,209 B2
`
`Stop VM
`with ID X
`
`..
`
`Existing VM
`shutdown routine
`
`+
`Collect ACLs X I
`
`-..........
`Serialize ACLs for X in
`
`1200
`
`Collect FIB, TAG entries
`bound to VNICs, which
`belonq to VMID X
`
`Serialize FIB, TAG
`entries in data structure
`
`.
`Store S1 a
`... Associate
`nd S2
`stored file with VM ID X
`
`FIGURE 12
`
`data structure S1 •
`..
`..
`•
`
`Uniinstall FIB, TAG
`entries applicable to X
`
`Uninstall ACLs
`applicable to X
`
`Microsoft Ex. 1001, p. 13
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 13 of 18
`
`US 8,381,209 B2
`
`Start VM
`with IDX
`
`Find network information files
`Associated with VM ID X
`Load S1 (ACLs) and S2 (FIB) pertaining to
`VMIDX
`
`Create dummy virtual network interfaces that
`will be used by VM ID X when it loads. The
`number of dummy VNICs equals the number of
`unique VNICs mentioned in S1 and S2
`
`Deserialize ACLs for X from
`S1 and install in hypervisor
`Network AGL
`
`Deserialize VLAN TAG
`
`Deserialize FIB, TAG
`entries for VMID X from
`S2 and install in
`hvoervisor FIB
`
`1300
`
`Existing VM
`startup routine*
`
`* Modified to use dummy NICs created
`when ACLs were installed. Dummy NICs
`are fully configured into operational state
`using conventional startup
`
`FIGURE 13
`
`Microsoft Ex. 1001, p. 14
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 14 of 18
`
`US 8,381,209 B2
`
`1400
`
`············:,.························f"·········
`
`············f"···········
`
`( ....... , .... ............ ~ ........................ f .......... •
`
`i Point to or
`imbedd
`
`.......... I MAC11
`I VMID I i
`!. ........................ ! !. ........................ ! l ........................ 1 I MAC2 I
`
`j i Point to or
`i i
`imbedd
`
`j l Point to or j
`i
`i l
`imbedd
`
`MAC ..
`
`I
`
`I
`
`I VNet ptr ~
`I
`
`Main VM description file
`
`May be located on server or in file:
`
`Serialized ACL
`(e.g., XML description of data structure) -'
`
`Serialized FIB
`(e.g., XML description of data structure) -'
`
`FIGURE 14
`
`Microsoft Ex. 1001, p. 15
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 15 of 18
`
`US 8,381,209 B2
`
`.
`
`.
`
`' .
`
`.
`
`.
`
`. · Vi rl)Hil Mac~rie N~me . ; '
`· :l Redhat Lnu.x 91 .. Host A
`
`1500
`
`1
`
`L
`. i
`.J
`
`. . . O~!ete . .
`
`.: RetTIC'l&se!ected
`
`1.
`
`f 8indlngconligutetion
`"· ·
`I
`i VNIC
`r,, OSPF advertiZe
`IP Elil\dino
`I· ,...I v-N-,c-1-"----'--'--3]....,~ 119216s 123 012ss 2ss 2ss o.'.:J Ii
`l MAC
`!J '.•(. '? :,: 1;' ,, 1?
`VLan tag
`! 123
`·•.
`. .. ..
`1,• ·. __ ._,_, A_dd_.• _· _ _...l
`.
`·· _,.,
`s_.s--r:.r .. :·
`RreW\lsll
`,,
`.,, ... ·.
`·· ,,·
`• '''i' 'Y«««••""""""'""'""'m"•'•"••••• ", .•. ~-.........<»,,»».<w.-,.-,.-,.,.,,rr,,.,.,.,.,.,,.,•.,.,. -
`S6urce
`Destinat1or1
`11921681231
`
`Pfl)totoi
`
`. Poicv
`
`.:J -I 0-0-0-.0i-O ---.-.i:J j TCP ~JI accepl iJ I in.
`
`,.,•.-»•,.,,u•, • .,•,.,
`
`Oi redion
`
`Existing N&twork Access Controls
`
`.. Ack!:
`
`Done
`
`• j
`
`FIGURE 15A
`
`1501
`
`VMNet Config for
`
`window
`
`VMID X console r--. Layer of hypervisor
`
`Update VNET
`
`running VMX
`
`If VM running
`
`..
`.
`
`Update VNET
`In serialized
`representation of VM X
`
`always
`
`FIGURE 158
`
`Microsoft Ex. 1001, p. 16
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 16 of 18
`
`US 8,381,209 B2
`
`FOR EACH PORT +------ Read from: manual input or network c011figuration managrn1ent DB
`
`FOR EACH MAC ON PORT +--- Read.from: use SNMP, remote cmifiguration management
`(e.g., Cisco Works, CLI)
`
`TransferVLAN TAG
`
`Read.from: use SNMP, remote co1ifiguration
`management ( e.g .. Cisco Works, CL!)
`
`Capture VLAN Tag for port MAC pair
`
`Capture Network AGL installed in switch for port
`
`Save VLAN tag into VM descriptor
`
`Save VLAN tag into VM descriptor for VM X
`
`Obtain IP address for VM via SNMP query on VNIC
`Save in routing VNet data field
`By default enable OSPF advertize
`
`On next restart of VM proceed with installation of ACL and TAG
`Erase configuration in SWITCH (ask for confirmation)
`
`FIGURE 16
`
`Microsoft Ex. 1001, p. 17
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 17 of 18
`
`US 8,381,209 B2
`
`FOR EACH FIREWALL RULE
`
`1700
`
`SOURCE/DESTINATION
`APPLIES TO
`VMX DESTINATION Ives if N/A)
`
`Assume VM to be annotated with
`firewall ACLs is X
`
`Assume VM to be annotated with
`firewall ACLs is X
`
`Rewrite in specialized form by substituting
`matching VM X IP for destination
`
`Rewrite in specialized form by substituting
`matching VM X IP for source
`
`Store generated
`specialized rules in
`VM VNet descriptor
`
`Note: The VM will be fully
`protected after this p1 ocedure.
`It would be safe to delete the
`firewall rules. This is not
`recommended due to overall
`security implications
`
`FIGURE 17
`
`Microsoft Ex. 1001, p. 18
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`U.S. Patent
`
`Feb.19,2013
`
`Sheet 18 of 18
`
`US 8,381,209 B2
`
`1821
`
`1840
`
`1811
`
`1811
`
`1814
`
`1816
`
`1818
`
`1800
`
`NETWORK
`
`1834
`
`CPU
`
`CPU
`
`RAM
`
`ROM
`
`1/0
`ADAPTER
`
`COMMUNICATIONS
`ADAPTER
`
`1812
`
`KEYBOARD
`
`1824
`
`1838
`
`DISPLAY
`ADAPTER
`
`PRINTER
`
`1828
`
`1839
`
`1832
`
`1826
`
`FIGURE 18
`
`1900
`
`FIGURE 19
`
`Microsoft Ex. 1001, p. 19
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`US 8,381,209 B2
`
`1
`MOVEABLE ACCESS CONTROL LIST (ACL)
`MECHANISMS FOR HYPERVISORS AND
`VIRTUAL MACHINES AND VIRTUAL PORT
`FIREWALLS
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`The present invention generally relates to a method and
`system for providing control of network security of a virtual
`machine, and more particularly, to a method of virtual
`machine migration with filtered network connectivity which
`includes enforcing network security and routing at a hyper(cid:173)
`visor layer at which a virtual machine partition is executed
`and which is independent of guest operating systems.
`2. Description of the Related Art
`In a network-secured environment, host movement means
`moving its network entangled state, which includes routing
`(e.g., VLAN (virtual local area network) tags, OSPF (open 20
`shortest-path first) host route entries, etc.) and security (e.g.,
`firewall (FW) access control lists (ACLs), switch ACLs,
`router ACLs, VLAN tags, etc.) from one machine to another.
`That is, in order to perform maintenance on or provide a
`fail-over for a processor device or machine, it is desirable to
`move or migrate a virtual machine (VM) from one processor
`machine or device to another processor machine or device.
`For purposes of this disclosure, a virtual machine (VM)
`generally includes a virtual data processing system, in which
`multiple operating systems and programs can be run by the
`computer at the same time. Each user appears to have an
`independent computer with its own input and output devices.
`For purposes of this disclosure, logical partitioning
`(LPAR) generally means the capability to divide a single
`physical system into multiple logical or "virtual" systems,
`each sharing a portion of the server's hardware resources
`(such as processors, memory and input/output (I/O)). Each
`LPAR runs an independent copy of an operating system. They
`can even be different operating system versions or distribu(cid:173)
`tions.
`That is, LPAR generally allows customers to "slice-up" a
`machine into virtual partitions, and provides the flexibility to
`dynamically change the allocation of system resources for
`those environments, thereby providing the capability to cre-
`ate multiple virtual partitions within a processor. Spare capac- 45
`ity can be re-allocated to virtual partitions. Any of the virtual
`servers may run on any of the physical processors, meaning
`that the processor resources are fully shared, which makes it
`possible to run the physical server at very high utilization
`levels.
`For purposes of this disclosure, dynamic logical partition(cid:173)
`ing (DLPAR) generally
`increases flexibility, enabling
`selected system resources like processors, memory and I/O
`components to be added and deleted from dedicated parti(cid:173)
`tions while they are actively in use. The ability to reconfigure
`dynamic LPARs enables system administrators to dynami(cid:173)
`cally redefine all available system resources to enable opti(cid:173)
`mum capacity for each partition.
`For purposes of this disclosure, virtual local area network
`(VLAN or virtual LAN) generally allows clients to create 60
`virtual Ethernet connections to provide high-speed inter-par(cid:173)
`tition communication between logical partitions on a server
`without the need for network I/O adapters and switches. Con(cid:173)
`nectivity outside of the server can be achieved using the
`virtual I/O server partition that acts as an internet protocol 65
`(IP) forwarder to the Local Area Network (LAN) through an
`Ethernet I/O adapter.
`
`2
`For purposes of this disclosure, a hypervisor, sometimes
`referred to as a virtualization manager, includes a program
`that allows multiple operating systems, which can include
`different operating systems or multiple instances of the same
`5 operating system, to share a single hardware processor. A
`hypervisor preferably can be designed for a particular pro(cid:173)
`cessor architecture.
`Each operating system appears to have the processor,
`memory, and other resources all to itself. However, the hyper-
`10 visor actually controls the real processor and its resources,
`allocating what is needed to each operating system in tum.
`Because an operating system is often used to run a particu(cid:173)
`lar application or set of applications in a dedicated hardware
`server, the use of a hypervisor preferably can make it possible
`15 to run multiple operating systems ( and their applications) in a
`single server, reducing overall hardware costs. Production
`and test systems also preferably can run at the same time in
`the same hardware. In addition, different operating systems
`preferably can share the same server.
`Thus, a hypervisor generally means a scheme which allows
`multiple operating systems to run, unmodified, on a host
`computer at the same time. Such software lets multiple oper(cid:173)
`ating systems run on the same computer, a feature that is
`particularly useful for consolidating servers in order to save
`25 money, and for extracting as much work as possible from a
`single system.
`As mentioned above, in order to perform maintenance on
`or provide a fail-over for a processor device or machine, it is
`desirable to move or migrate a virtual machine (VM) from
`30 one processor machine or device to another processor
`machine or device.
`With reference to FIGS. 1-3, conventional approaches to
`migrating virtual machines from one device ( e.g., hardware
`device) to another device ( e.g., hardware device) will be
`35 described.
`FIG. 1 illustrates an exemplary system 100 which can
`include a plurality of virtual machines (VM) (101) controlled
`by a switches (e.g., SWA1-SWB5) (102) connected by an
`Internet Service Provider (ISP) (103) and protected by fire-
`40 walls FWl and FW2 (104).
`As mentioned above, in a network-secured environment,
`host movement means moving its network entangled state,
`which includes routing and security from one machine to
`another.
`In FIG. 2, the network entangled state of virtual machine
`VM 205 ( e.g., hypervisor 206; NI Cl 207, VNIC 210, switchl
`208, and firewall FWl 209) is copied to virtual machine VM'
`215 ( e.g., hypervisor 216; NIC2 217, VNIC (virtual network
`interface card) 210, switch2 218, and firewall FW2 219). In
`50 FIG. 2, there is noACL at switch2 (318), which means every
`virtual machine could be masqueraded. Also, at the firewall
`FW2 (219), there is no selection of which virtual machine can
`go where.
`As illustrated in FIG. 2, conventional systems (e.g., 200)
`55 generally do not include ACLs. Also, the firewall FW2 does
`not include a selection of which virtual machine can be
`accessed. Thus, the conventional systems provide very little
`security and routing generally is provided by OSPF adver(cid:173)
`tised host routes.
`FIG. 3 illustrates another conventional system in which
`routing is taken care ofby OSPF advertised host routes. FIG.
`3 illustrates a conventional system in which restrictive ACLs
`are included in the switch2 and the firewall FW2 includes
`restrictions for access.
`In FIG. 3, the network entangled state of virtual machine
`VM305 (e.g., hypervisor306; NICl 307, VNIC 310, switchl
`308, and firewall FWl 309) is copied to virtual machine VM'
`
`Microsoft Ex. 1001, p. 20
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`US 8,381,209 B2
`
`3
`315 (e.g.,hypervisor316; NIC2 317, VNIC310, switch2 318,
`and firewall FW2 319). As illustrated in FIG. 3, in the con(cid:173)
`ventional systems, the restrictive ACLs are provided, for
`example, at switch2 (318). The firewall FW2 also includes
`restrictions.
`Thus, the conventional systems and methods require a
`complex update scheme to update the ACLs in the real
`switches and the filters in the firewalls to migrate a virtual
`machine from one machine to another machine.
`Generally, conventional virtual machine systems and 10
`methods provide very little network security. In the conven(cid:173)
`tional systems and methods, routing generally is provided by
`open shortest-path first (OSPF) advertised host routes. Con(cid:173)
`ventional systems and methods generally do not include
`access control lists (ACLs) and security generally is only as 15
`good as security at each individual machine.
`For example, one conventional system and method relates
`to virtualizing computer systems on the same host practical.
`Some conventional methods relate to arbitration of access to
`shared resources on the same host when multiple operating 20
`systems attempt to access the shared resource. In particular,
`one conventional method focuses on the ability to virtualize
`shared memory page tables, which to date had not been suc(cid:173)
`cessfully addressed in direct execution virtual machines. The
`conventional method does not, however, address network 25
`virtualization, in which a virtual machine is to be network
`addressable, which is addressed herein below by the present
`invention. Instead, the conventional method merely relates to
`a virtual machine that is addressable but that does not migrate
`its network-entangled state.
`Another exemplary method and device relates to a mecha(cid:173)
`nism for restoring, porting, replicating and check pointing
`computer systems using state extraction. This conventional
`method covers the ability to initiate migration of a virtual
`machine from one system to another. Particularly, the con- 35
`ventional method and device discusses the migration of
`peripheral state in which the peripheral is assumed to be a
`hardware resource that is emulated. However, such conven(cid:173)
`tional methods and devices do not discuss the much more
`flexible and efficient possibility of capturing application 40
`state, such as the state of a firewall or routing that pertains to
`a particular movable partition, which is addressed herein
`below by the present invention. Instead, these conventional
`methods and devices merely focus on device control, which,
`as the ordinarily skilled artisan would know and understand, 45
`is not the same as ( or equivalent to) the establishment of
`logical rules that govern the interaction of a migrated virtual
`machine with the rest of the network infrastructure, as
`described herein below by the present invention. These con(cid:173)
`ventional methods and devices also do not disclose or sug- 50
`gest, however, that a logical device needs to be bootstrapped
`and/or that device state in the network needs to be revoked
`upon migration of a virtual machine partition, as described
`herein below by the present invention
`Other conventional systems and methods relate to a logical 55
`partition manager. These methods discuss the possibility of
`feeding information that is created within a logical partition
`(guest, or virtual machine) back to a partition manager. These
`conventional methods discuss the operating system (OS)
`itself applying security controls and routing in a special par(cid:173)
`tition. The crux of these conventional methods is so-called
`paravirtualization.
`In paravirtualization, the partition manager "trusts" the
`partition OS to cooperate with the other partitions. These
`conventional systems suffer from a serious security flaw that
`an undermined OS can disable access protection that prevents
`remote control software from manipulating an operating sys-
`
`30
`
`4
`tern instance running within a logical partition (guest or vir(cid:173)
`tual machine). These conventional methods, therefore, can(cid:173)
`not be used to implement access controls unless additional
`security inventions secure the shared state and control across
`5 partitions in reliable manner. These conventional methods do
`not discuss how the network access controls may have to be
`reset on copying a virtual machine from one computer to
`another, which is addressed herein below by the present
`invention. These conventional methods also do not discuss
`how network access control and routing is to be maintained.
`Other conventional systems and methods relate to virtual
`machine operating system local area networks (LAN s ), and
`describe a system for defining and creating virtual network
`adapters within a hypervisor for the use by guest virtual
`machines. These conventional systems and methods do not
`discuss access controls and routing problems pertaining to a
`virtual machine being copied across the network, which are
`addressed and solved herein below by the present invention.
`Other conventional systems and methods relate to preser-
`vation of a computer system processing state in a mass stor(cid:173)
`age device. These conventional systems and methods
`describe how the state of a computer should be stored in a
`mass storage device. These conventional systems and meth(cid:173)
`ods do not describe how the storage should be extended to
`also capture state that is external to the processor's address(cid:173)
`able memory, which is addressed herein below by the present
`invention.
`
`SUMMARY OF THE INVENTION
`
`In view of the foregoing and other exemplary problems,
`drawbacks, and disadvantages of the conventional methods
`and systems, an exemplary feature of the present invention is
`to provide a method and system for providing control of
`network security of a virtual machine, and more particularly,
`to a method of virtual machine migration with filtered net-
`work connectivity which includes enforcing network security
`and routing at a hypervisor layer at which a virtual machine
`partition is executed and which is independent of guest oper(cid:173)
`ating systems.
`As mentioned above, in order to perform maintenance on
`or provide a fail-over for a processor device or machine, it is
`desirable to move or migrate a virtual machine (VM) from
`one processor machine or device to another processor
`machine or device. However, conventional systems and meth(cid:173)
`ods require a complex scheme to update and install ACLs in
`the real switches of the machines and update and install
`firewalls. Also, the conventional systems and methods pro(cid:173)
`vide very little security.
`The exemplary method and system of the present invention
`can provide control of network security of a virtual machine
`by enforcing network security and routing at a hypervisor
`layer at which a virtual machine partition is executed and
`which is independent of guest operating systems.
`The exemplary aspects of the present application prefer-
`ably can provide a hypervisor security architecture designed
`and developed to provide a secure foundation for server plat(cid:173)
`forms, providing numerous beneficial functions, such as,
`strong isolation, mediated sharing and communication
`60 between virtual machines. These properties can all be strictly
`controlled by a flexible access control enforcement engine,
`which also can enforce mandatory policies.
`The exemplary features of the invention also can provide
`attestation and integrity guarantees for the hypervisor and its
`65 virtual machines.
`For example, the present invention exemplarily defines a
`computer implemented method of controlling network secu-
`
`Microsoft Ex. 1001, p. 21
`Microsoft v. Daedalus Blue
`IPR2021-00832
`
`

`

`US 8,381,209 B2
`
`5
`rity of a virtual machine, including enforcing network secu(cid:173)
`rity and routing at a hypervisor layer.
`Particularly, the present invention defines a computer
`implemented method of virtual machine migration with fil(cid:173)
`tered network connectivity, including enforcing network 5
`security and routing at a hypervisor layer which is indepen(cid:173)
`dent of guest operating systems.
`The exemplary method of the present invention can
`include, for example, copying network security and routing
`for the virtual machine to the hypervisor layer, migrating the 10
`virtual machine from a first hardware device to a second
`hardware device, updating routing controls for the virtual
`machine at the hypervisor level, updating traffic filters for the
`virtual machine at the hypervisor level, and advertising the
`migration of the virtual machine from the first hardware
`device to the second hardware device.
`On the other hand, an exemplary system for controlling
`network security of a virtual machine by enforcing network
`security and routing at a hypervisor layer, according to the
`present invention, includes a copying unit that copies network
`security and routing for the virtual machine to the hypervisor
`layer, a migrating unit that migrates the virtual machine from
`a first hardware device to a second hardware device, a first
`updating unit that updates routing controls for the virtual
`machine at the hypervisor level, a second updating unit that
`updates traffic filters for the virtual machine at the hypervisor
`level, and an advertising unit that advertises the migration of
`the virtual machine from the first hardware device to the
`second hardware device.
`As mentioned above, in the conventional methods and 30
`systems, it is difficult to move one virtual machine from one
`machine to another. Generally, in conventional systems, to
`move a virtual machine from one machine to another ( e.g.,
`from hardware 1 to hardware 2), the conventional methods
`and systems would merely shut down and copy from hard(cid:173)
`ware 1 to hardware 2. The conventional systems and methods
`have difficulties with security and routing.
`To solve the problems with the conventional systems and
`methods, the present invention copies security and routing,
`etc. for the virtual machine to the hypervisor layer so that the
`user will see no difference in operation between running the
`virtual machine on hardware 1 or hardware 2. That is, accord(cid:173)
`ing to the present invention, the first and second device ( e.g.,
`hardware 1 and hardware 2) would each act the same, and
`preferably, would each have the same internet protocol (IP)
`address.
`An important problem arises when networks are very large,
`such as Google and Yahoo, in which there could be a thousand
`servers, and no flat topography, switches and routers to pro(cid:173)
`tect the servers. That is, in such systems, the virtual system is
`run on top of the hypervisor such that each virtual system is
`only as good as the security at each machine.
`To migrate the virtual machine from a first hardware device
`to a second hardware device, the present invention routes
`network traffic for the virtual machine to the second hardware 55
`device at the hypervisor layer. The present invention also sets
`firewalls to permit network traffic for the virtual machine to
`go to the second hardware device at the hypervisor layer.
`According to the present invention, the hypervisor level
`provides traffic filtering and routing updating. Thus, the real
`switches do not need to be updated at the first and second
`hardware devices.
`Moreover, the present invention advertises the migration of
`the virtual machine from the first hardware device to the
`second hardware device using the second hardware device.
`Thus, the present invention has an important advantage of not
`requiring central control. The routers also do not need to be
`
`6
`updated because the migration is being advertised from the
`second hardware device ( e.g., hardware 2).
`The present invention decentralizes the updating scheme
`by using the hypervisor layer for security and routing. Thus,
`according to the present i