`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 1 of 28
`
`EXHIBIT 3
`EXHIBIT 3
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 2 of 28
`
`I 1111111111111111 11111 111111111111111 111111111111111 11111111111 111111111111
`US0083 81209B2
`
`c12) United States Patent
`Reumann et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 8,381,209 B2
`Feb. 19,2013
`
`(54) MOVEABLE ACCESS CONTROL LIST (ACL)
`MECHANISMS FOR HYPERVISORS AND
`VIRTUAL MACHINES AND VIRTUAL PORT
`FIREWALLS
`
`(75)
`
`Inventors: John Reumann, Croton on Hudson, NY
`(US); Dcbanjan Sah11, Mohegan Lake,
`NY (US); Samhit Sahu, Hopewell
`Junction, NY (US); Dinesh Ch11ndra
`Verma, Mount Kisco, NY (US)
`
`(73) Assignee: International Business Machines
`Corporation, Armonk, NY (US)
`
`( •) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1429 days.
`
`(21) Appl. No.: 11/619,536
`
`(22) Filed:
`
`Jan.3,2007
`
`(65)
`
`Prior Publication Data
`
`US 2008/0163207 Al
`
`Jul. 3, 2008
`
`(51)
`
`Int. CI.
`G06F 91455
`(2006.01)
`(52) U.S. CI .
`............................. 718/1; 709/250; 718/102
`(58) Field of Classification Search .................. 709/250;
`718/1, 102
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT OCX::UMENTS
`5,386,552 A
`1/ 1995 Garney
`12/2002 Bugnion et al.
`6,496,847 Bl
`6,691,146 Bl
`2/2004 Armstrong et al .
`6,795,966 Bl
`9/2004 Lim et al.
`2004/0015966 Al
`1/2004 MacChiano et al.
`2004/0158720 Al
`8/2004 O'Brien
`6/2006 Traut et al . . . . .. .... ... ... . .. . . .. . 711 /6
`2006/0136653 Al•
`2006/0143311 Al•
`6/2006 Madukkanunukumana
`et al .................................. 710/1
`10/2006 Kurien et al. ................. 713/193
`2/2008 Shimizu et al. ............... 713/320
`10/2008 Challener et al . ........ ... ... ... 718/1
`1/2009 Hara et al.
`.................... 718/105
`5/2009 Mahalingam et al ......... 719/324
`5/2009 Wray et al.
`.............. ..... 370/392
`10/2009 Litvin et al.
`.. . . ... . .. .. . . .. ... .. . 726/ I
`12/2009 Snively et al . ..... ... ........ 370/401
`12/2009 Oshins .. ...... .
`. .. 719/321
`
`2006/0236127 Al•
`2008/0034234 Al•
`2008/0244569 Al•
`2009/0025007 Al•
`2009/0119684 Al•
`2009/0129385 Al*
`2009/0249438 Al•
`2009/0296726 Al•
`2009/0328074 Al•
`
`* cited by examiner
`
`Primary Examiner - Mohamed Wasel
`(74) Attorney, Agent, or Finn - Eustus D. Nelson, Esq.;
`McGinn IP Law Group, PLLC
`
`(57)
`ABSTRACT
`A method (and system) which provides virtual machine
`migration with filtered network connectivity and control of
`network security of a virtual machine by enforcing network
`security and routing at a hypervisor layer at which the virtual
`machine partition is executed, and which is independent of
`guest operating systems.
`
`17 Claims, 18 Drawing Sheets
`
`START
`
`iQl!
`
`copy network security and routing for the virtual machine to
`the hypcrvisor layer
`
`,,,,,-- 401
`
`migrating the virtual machine from a first hardware-. device to
`a second hardware device
`
`,,,,,-- 402
`
`updating routing controls for the virtual machine at the
`hypervisor level
`
`updating traffic filters for the virtual machim: at the
`hypervisor level (e.g .. by setting hypervisor firewalls to
`permit network traffic for the vi rtual machine to access the
`second hardware device)
`
`-404
`
`odvcrtising (e.g., by so.id second hardware device) the
`migration of said virtual machine from the fin;t han.lwart:
`device to the second hardware device
`
`~ 405
`
`407
`
`routing network traffic for the
`virtual ma.chine lo the second
`hardware device based on the
`routing rontrols
`
`granting to the vinual machine on
`said second hardware device based
`on the traffic filters (e.g., AC Ls).
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 3 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 1 of 18
`
`US 8,381,209 B2
`
`100
`
`#IH
`
`ISP
`
`FIGURE 1
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 4 of 28
`
`U.S. Patent
`
`Feb.19, 2013
`
`Sheet 2 of 18
`
`US 8,381,209 B2
`
`hypervis
`
`NIC1
`I Switch1
`
`copy
`start
`
`hypervisor
`
`NIC2
`
`NOACL
`
`ALLOW WORLD I Fw2 I
`
`FIGURE 2
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 5 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 3 of 18
`
`US 8,381,209 B2
`
`Ll
`•
`.· . . copy
`start
`hypervisor
`
`NIC1
`I Switch1 I
`I FW1 I
`
`hypervis
`
`NIC2
`
`ACL does not allow VM
`
`I FW21 (9 DENY
`
`FIGURE 3
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 6 of 28
`
`U.S. Patent
`
`Feb. 19,2013
`
`Sheet 4 of 18
`
`US 8,381,209 B2
`
`START
`
`400
`
`copy network security and routing for the virtual machine to ~ 40 I
`the hypervisor layer
`
`migrating the virtual machine from a first hardware device to
`a second hardware device
`
`~402
`
`updating routing controls for the virtual machine at the
`hypervisor level
`
`~403
`
`updating traffic filters for the virtual machine at the
`hypervisor level (e.g., by setting hypervisor firewalls to
`permit network traffic for the virtual machine to access the
`second hardware device)
`
`404
`
`advertising (e.g., by said second hardware device) the
`migration of said virtual machine from the first hardware
`device to the second hardware device
`
`~405
`
`406
`
`407
`
`routing network traffic for the
`virtual machine to the second
`hardware device based on the
`routing controls
`
`granting to the virtual machine on
`said second hardware device based
`on the traffic filters (e.g., ACLs).
`
`FIGURE 4
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 7 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 5 of 18
`
`US 8,381,209 B2
`
`500
`
`VM
`
`lwinXPI
`
`vm
`
`ALLOW IN/OUT: MAC, IP, (5 tuples)
`Access policies
`IP address
`,vni , "N - - - - - - - - ,
`
`deployment editor
`Filters can be updated
`w/o running VM
`
`FIGURE 5
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 8 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 6 of 18
`
`US 8,381,209 B2
`
`VM
`
`ALLOW IN/OUT: MAC, IP, (5 tuples)
`Access policies
`IP address
`
`Stored in control center
`application (e.g. director)
`
`.
`tf ·:~~?::f{~;I)1~l;:
`
`.
`
`FIGURE 6
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 9 of 28
`
`U.S. Patent
`
`Feb. 19, 2013
`
`Sheet 7 of 18
`
`US 8,381,209 B2
`
`VM
`
`iw1iili,
`Hypervisometwork
`serializeldeserialize
`mobility layer ----------' VNIC ..,_ ___ ..,_ __ __,
`
`OSPF peer
`Deliver to/from VNIC
`
`___ u_pd_a_te_r_ou_te_t_o_V_M_----1 VLAN TAG
`~~•c
`
`Network
`ACL
`Editor
`
`FIGURE 7
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 10 of 28
`
`U.S. Patent
`
`Feb. 19, 2013
`
`Sheet 8 of 18
`
`US 8,381,209 B2
`
`800
`
`L2 control bloc;:.k:_ __ ..L_!:::IP=H;-.F-IE_L_D_<_op_>_P_ATTT_E_R_N ....
`
`Navigable list for admin
`
`.-----MAC~op>PATTERN
`
`!PH.FIELD <op> PATTERN
`
`E7H, TR,
`VMNET L ,
`
`. , ...
`
`IP, If'X,
`
`igmp
`
`UCP
`TCP
`ICMP,
`RTP,
`
`Policy
`ptr
`
`L2 PROTO
`
`i ~ Each field i.v optional
`
`L3 PROTO
`
`L4 PROTO
`
`prevACL nextACL
`
`.--------1 Named ACL directory
`
`ACL Head
`Identifies VM MAC to which ACL bound
`
`Map.1· human readable name.~ to ACLs
`
`FIGURE 8
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 11 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 9 of 18
`
`US 8,381,209 B2
`
`Index of ACL< created
`using wtll-known boolean
`txpression minimization,
`tries and the Iii«,
`
`ACL index
`
`900
`
`Remove VLAN
`tag
`
`GetMatch
`Find ACL
`
`Apply VLAN
`tag
`
`{1b)
`
`(2)
`,-------(3)
`
`(O)
`
`;.;;Jt~f&i;: (1
`
`f i~''r"-
`
`)
`
`TAP
`
`(6)
`
`incoming
`
`Hypervisor
`Networl< packe
`delivery code
`
`TAP
`outgoing
`
`rto'ili~
`.. ti
`
`(8 .
`
`'
`
`:·Gm,sf
`
`'
`
`. )~\
`·.-,.:
`
`(4)
`
`(5)
`
`Handle packet
`according to policy
`
`*NIC=network interface card
`Handle packet
`according to policy Outbound path is symmetrical
`
`FIGURE 9
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 12 of 28
`
`U.S. Patent
`
`Feb. 19,2013
`
`Sheet 10 of 18
`
`US 8,381,209 B2
`
`1000
`
`NO
`
`Emulate ARP with
`reduced timeout
`
`NO
`
`Return fixed IP as DHCP
`lease
`
`Act as DHCP proxy to
`real DHCP server
`specified in vNIC confio
`
`NO
`
`On timeout return MAC
`address of gateway
`according to OSPF
`
`FIGURE 10
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 13 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 11 of 18
`
`US 8,381,209 B2
`
`Hypervisor FIB
`
`1100
`
`Virtual NIC
`ID
`
`IP address/
`~ Subnet pairs
`
`'f-ty~ervisqr.1het"Yori<
`... -----t>·· cq~t~~J;f 1ryt1ow.
`' ~r;:ii: ''.·~t ... ~': ~r
`!=====~ =====::::::'''14------,
`·,1
`kl
`L:
`I
`r1!:::::::;:; ,.::=• •===!1
`i
`
`Change trigger
`
`., -~
`
`- '
`- . .,,
`OSPF
`...
`module
`
`,,
`
`VLANID
`module
`
`,.
`
`•• ~ ✓•
`
`,
`To OSPF peers
`Advertise any host/net on list
`
`"
`
`~ Virtual NIC ID II~
`II
`. I
`I::= =======11 , . I
`
`FIGURE 11
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 14 of 28
`
`U.S. Patent
`
`Feb. 19, 2013
`
`Sheet 12 of 18
`
`US 8,381,209 B2
`
`Stop VM
`with ID X
`
`•
`
`Existing VM
`shutdown routine
`
`♦
`
`Collect ACLs X I
`
`...........
`Serialize ACLs for X in
`
`data structure S1 •
`
`Collect FIB, TAG entries
`bound to VNICs, which
`belonQ to VMID X
`
`•
`
`Serialize FIB, TAG
`entries in data structure
`
`•
`•
`
`Uniinstall FIB, TAG
`entries applicable to X
`
`Uninstall ACLs
`applicable to X
`
`..
`.
`
`Store S1 an d S2
`Associates tared file with VM ID X
`
`FIGURE 12
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 15 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 13 of 18
`
`US 8,381,209 B2
`
`Start VM
`with IDX
`
`Find network information files
`Associated with VM ID X
`Load S1 (ACLs) and S2 (FIB) pertaining to
`VMIDX
`
`Create dummy virtual network interfaces that
`will be used by VM ID X when it loads . The
`number of dummy VNICs equals the number of
`unique VNICs mentioned in S1 and S2
`
`Deserialize ACLs for X from
`S, and install in hypervisor
`Network ACL
`
`Deserialize VLAN TAG
`
`Deserialize FIB, TAG
`entries for VMID X from
`S2 and install in
`hvoervisor FIB
`
`1300
`
`Existing VM
`startup routine*
`
`• Modified to use dummy N/Cs created
`when ACLs were installed. Dummy N/Cs
`are fully configured into operational state
`using conventional startup
`
`FIGURE 13
`
`
`
`, :;:r~ II :~f? II :~f? I
`........... ,. .......... , ............ f" .......... · ........... , ........... ·
`,, ........ ~ (" - . .... ·J, .......... I MAC11
`I VMID I . ~ 1 imbedd u 1mbedd n ,mbedd 1 I MAC2 I
`•. t ........................ j L ........................ ~ l .......... ~ .. ···········t
`
`,
`
`•
`
`MAC ...
`
`I-_./~ .
`
`\
`
`,f
`
`: , - . ·I
`-1 VNet ptr ~
`I
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 16 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 14 of 18
`
`US 8,381,209 B2
`
`, ........................ , , ........................ , ,, ........................ ,
`
`1400
`
`., . ; .. ,.,.~!< . i Point to or H Point to or H Point to or i
`
`, .......... . l,,
`
`.......... f .... -.. ..
`
`..
`
`·'
`
`Main VM description file
`
`May be located on server or in file:
`
`Serialized ACL
`(e.g ., XML description of data structure) -
`
`Serialized FIB
`(e.g., XML description of data structure) ~
`
`FIGURE 14
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 17 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 15 of 18
`
`US 8,381,209 B2
`
`1500
`
`1501
`
`FIGURE 15A
`
`VMNet Config for
`VMID X console
`window
`
`Update VNET
`
`r---. Layer of hypervisor
`
`running VMX
`
`If VM running
`
`Update VNET
`In serialized
`representation of VM X
`
`~
`
`always
`
`FIGURE 158
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 18 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 16 of 18
`
`US 8,381,209 B2
`
`FOR EACH PORT - - - - - - Read from: nuuwal input or neht.'ork configura tion manag~mc,11 DB
`
`FOR EACH MAC ON PORT +--- Readfrom: use SNMP, remote co11jiguration managemenr
`(e.g., Cisco Works, CL/)
`
`TransferVLAN TAG
`
`Read.from: use SNMP. remme Wf!{iguration
`111a11age111ent (e.g .. Cisco Works, CL/)
`
`Capture VLAN Tag for port MAC pair
`
`Capture Network ACL installed in switch for port
`
`Save VLAN tag into VM descriptor
`
`Save VLAN tag into VM descriptor for VM X
`
`1600
`
`Obtain IP address for VM via SNMP query on VNIC
`Save in routing VNet data field
`By default enable OSPF advertize
`
`On next restart ofVM proceed with installation of ACL and TAG
`Erase configuration in SWITCH (ask for confirmation)
`
`FIGURE 16
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 19 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 17 of 18
`
`US 8,381,209 B2
`
`FOR EACH FIREWALL RULE
`
`1700
`
`Assume VM to be annotated with
`firewall ACLs is X
`
`Assume VM to be annotated with
`firewall ACLs is X
`
`Rewrite In specialized form by substituting
`matching VM X IP for destination
`
`Rewrite in specialized form by substituting
`matching VM X IP for source
`
`Store generated
`specialized rules in
`VM VNet descriptor
`
`Note: The VM will be fully
`protected after this procedure.
`It would be safe to delete the
`firewall rules. This is not
`recommended due to overall
`security implications
`
`FIGURE 17
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 20 of 28
`
`U.S. Patent
`
`Feb.19,2013
`
`Sheet 18 of 18
`
`US 8,381,209 B2
`
`1821
`
`1840
`
`1800
`
`NETWORK
`
`1834
`
`1811
`
`1811
`
`1814
`
`1816
`
`CPU
`
`CPU
`
`RAM
`
`ROM
`
`1/0
`ADAPTER
`
`COMMUNICATIONS
`ADAPTER
`
`1812
`
`KEYBOARD
`
`USER
`INTERFACE
`ADAPTER
`
`DISPLAY
`ADAPTER
`
`1836
`
`1838
`
`PRINTER
`
`1839
`
`1900
`
`FIGURE 19
`
`1826
`
`FIGURE 18
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 21 of 28
`
`US 8,381,209 B2
`
`1
`MOVEABLE ACCESS CONTROL LIST (ACL)
`MECHANISMS FOR HYPERVISORS AND
`VIRTUAL MACHINES AND VIRTUAL PORT
`FIREWALLS
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`The present invention generally relates to a method and
`system for providing control of network security of a virtual
`machine, and more particularly, to a method of virtual
`machine migration with filtered network connectivity which
`includes enforcing network security and routing at a hyper(cid:173)
`visor layer at which a virtual machine partition is executed
`and which is independent of guest operating systems.
`2. Description of the Related Art
`In a network-secured environment, host movement means
`moving its network entangled state, which includes routing
`(e.g., \/LAN (virtual local area network) tags, OSPF (open 20
`shortest-path first) host route entries, etc.) and security (e.g.,
`firewall (FW) access control lists (ACLs), switch ACLs,
`router ACLs, \/LAN tags, etc.) from one machine to another.
`That is, in order to perform maintenance on or provide a
`fail-over for a processor device or machine, it is desirable to
`move or migrate a virtual machine (VM) from one processor
`machine or device to another processor machine or device.
`For purposes of this disclosure, a virtual machine (VM)
`generally includes a virtual data processing system, in which
`multiple operating systems and programs can be run by the
`computer at the same time. Each user appears to have an
`independent computer with its own input and output devices.
`For purposes of this disclosure, logical partitioning
`(LPAR) generally means the capability to divide a single
`physical system into multiple logical or "virtual" systems,
`each sharing a portion of the server's hardware resources
`(such as processors, memory and input/output (I/O)). Each
`LPAR runs an independent copy of an operating system. They
`can even be different operating system versions or distribu(cid:173)
`tions .
`That is, LPAR generally allows customers to "slice-up" a
`machine into virtual partitions, and provides the flexibility to
`dynamically change the allocation of system resources for
`those environments, thereby providing the capability to cre(cid:173)
`ate multiple virtual partitions within a processor. Spare capac- 45
`ity can be re-allocated to virtual partitions. Any of the virtual
`servers may nm on any of the physical processors, meaning
`that the processor resources are fully shared, which makes it
`possible to run the physical server at very high utilization
`levels .
`For purposes of this disclosure, dynamic logical partition(cid:173)
`ing (DLPAR) generally
`increases flexibility, enabling
`selected system resources like processors, memory and 1/0
`components to be added and deleted from dedicated parti(cid:173)
`tions while they are actively in use. The ability to reconfigure
`dynamic LPARs enables system administrators to dynami(cid:173)
`cally redefine all available system resources to enable opti(cid:173)
`mum capacity for each partition.
`For purposes of this disclosure, virh1al local area network
`(VLAN or virtual LAN) generally allows clients to create 60
`virtual Ethernet connections to provide high-speed inter-par(cid:173)
`tition communication between logical partitions on a server
`without the need for network 1/0 adapters and switches. Con(cid:173)
`nectivity outside of the server can he achieved using the
`virtual 1/0 server partition that acts as an internet protocol 65
`(IP) forwarder to the Local A.rea Network (LAN) through an
`Ethernet I/O adapter.
`
`2
`For purposes of this disclosure, a hypervisor, sometimes
`referred to as a virtualization manager, includes a program
`that allows multiple operating systems, which can include
`different operating systems or multiple instances of the same
`5 operating system, to share a single hardware processor. A
`hypervisor preferably can be designed for a particular pro(cid:173)
`cessor architecture.
`Each operating system appears to have the processor,
`memory, and other resources all to itself. However, the hyper-
`10 visor actually controls the real processor and its resources,
`allocating what is needed to each operating system in turn.
`Oecause an operating system is often used to run a particu(cid:173)
`lar application or set of applications in a dedicated hardware
`server, the use ofa hypervisor preferably can make it possible
`15 to run multiple operating systems (and their applications) in a
`single server, reducing overall hardware costs. Production
`and lest systems also preferably can run al the same time in
`the same hardware. In addition, different operating systems
`preferably can share the same server.
`Thus, a hypervisor generally means a scheme which allows
`multiple operating systems to run, unmodified, on a host
`computer at the same time. Such software lets multiple oper(cid:173)
`ating systems run on the same computer, a feature that is
`particularly useful for consolidating servers in order to save
`25 money, and for extracting as much work as possible from a
`single system.
`As mentioned above, in order to perform maintenance on
`or provide a fail-over for a processor device or machine, it is
`desirable to move or migrate a virtual machine (VM) from
`30 one processor machine or device to another processor
`machine or device.
`With reference lo FIGS. 1-3, conventional approaches to
`migrating virtual machines from one device (e.g., hardware
`device) to another device (e.g., hardware device) will be
`15 described.
`FIG. 1 illustrates an exemplary system 100 which can
`include a plurality of virtual machines (VM) (101) controlled
`by a switches (e.g., SWA1-SWB5) (102) connected by an
`Internet Service Provider (ISP) (103) and protected by fire-
`40 walls FWl and FW2 (104).
`As mentioned above, in a network-secured environment,
`host movement means moving its network entangled state,
`which includes routing and security from one machine to
`another.
`In FIG. 2, the network entangled state of virtual machine
`YM 205 (e.g., hypervisor 206; NICI 207, YNIC 210, switch!
`208, and firewall FWl 209) is copied to virtual machine YM
`215 (e.g., hypervisor 216; NIC2 217, VNIC (virtual network
`interface card) 210, switch2 218, and firewall FW2 219). In
`50 PIG. 2, there is no ACL at switch2 (318), which means every
`virtual machine could be masqueraded. Also, at the firewall
`FW2 (219), there is no selection of which virtual machine can
`go where.
`As illustrated in FIG. 2, conventional systems (e.g., 200)
`55 generally do not include ACLs. Also, the firewall FW2 does
`not include a selection of which virtual machine can be
`accessed. Thus, the conventional systems provide very little
`security and routing generally is provided by OSPF adver(cid:173)
`tised host routes.
`FIG. 3 illustrates another conventional system in which
`routing is taken care ofby OSPF advertised host routes. FIG.
`3 illustrates a conventional system in which restrictive ACLs
`are included in the swilch2 and the firewall FW2 includes
`restrictions for access.
`In FIG. 3, the network entangled state of virtual machine
`VM 305 (e.g., hypervisor 306; NI Cl 307, VNIC 310, switch!
`308, and firewall FWl 309) is copied to virtual machine VM'
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 22 of 28
`
`US 8,381,209 B2
`
`4
`tern instance running within a logical partition (guest or vir(cid:173)
`tual machine). These conventional methods, therefore, can(cid:173)
`not be used to implement access controls unless additional
`security inventions secure the shared slate and conlrul across
`partitions in reliable manner. These conventional methods do
`not discuss how the network access controls may have to be
`reset on copying a virtual machine from one computer to
`another, which is addressed herein below by the present
`invention. These conventional methods also do not discuss
`how network access control and routing is to be maintained.
`Other conventional systems and methods relate to virtual
`machine operating system local area networks (LANs), and
`describe a system for defining and creating virtual network
`adapters within a hypervisor for the use by guest virtual
`machines. These conventional systems and methods do not
`discuss access controls and routing problems pertaining to a
`virtual machine being copied across the network, which are
`addressed and solved herein below by the present invention.
`Other conventional systems and methods relate to preser(cid:173)
`vation of a computer system processing state in a mass stor(cid:173)
`age device. These conventional systems and methods
`describe how the state of a computer should be stored in a
`mass storage device. These conventional systems and meth(cid:173)
`ods do not describe how the storage should be extended to
`also capture state that is external to the processor's address(cid:173)
`able memory, which is addressed herein below by the present
`invention.
`
`SUMMARY OF THE INVENTION
`
`3
`315 (e.g., hypervisor316; NIC2 317, VNIC 310, switch2 318,
`and firewall FW2 319) . As illustrated in FIG. 3, in the con(cid:173)
`ventional systems, the restrictive ACLs are provided, for
`example, at switch2 (3 18). The firewall PW2 also includes
`restrictions.
`Thus, the conventional systems and methods require a
`complex update scheme to update the ACLs in the real
`switches and the tilters in the firewalls to migrate a virtual
`machine from one machine to another machine.
`Generally, conventional virtual machine systems and 10
`methods provide very little network security. In the conven(cid:173)
`tional systems and methods, routing generally is provided by
`open shortest-path first (OSPF) advertised host routes. Con(cid:173)
`ventional systems and methods generally do not include
`access control lists (ACLs) and security generally is only as 15
`good as security at each individual machine.
`For example, one conventional system and method relates
`to virtualizing computer systems on the same host practical.
`Some conventional methods relate to arbitration of access to
`shared resources on the same host when multiple operating 20
`systems attempt to access the shared resource. In particular,
`one conventional method focuses on the ability to virtualize
`shared memory page tables, which to date had not been suc(cid:173)
`cessfolly addressed in direct execution virtual machines. The
`conventional method does not, however, address network 25
`virtualization, in which a virtual machine is to be network
`addressable, which is addressed herein below by the present
`invention. Instead, the conventional method merely relates to
`a virtual machine that is addressable but that does not migrate
`its network-entangled state.
`Another exemplary method and device relates to a mecha(cid:173)
`nism for restoring, porting, replicating and check pointing
`computer systems using state extraction. This conventional
`method covers the ability to initiate migration of a virtual
`machine from one system to another. Particularly, the con- .15
`ventional method and device discusses the migration of
`peripheral state in which the peripheral is assumed to be a
`hardware resource that is emulated. However, such conven(cid:173)
`tional methods and devices do not discuss the much more
`flexible and efficient possibility of capturing application 40
`state, such as the state of a firewall or routing that pertains to
`a particular movable partition, which is addressed herein
`below by the present invention. Instead, these conventional
`methods and devices merely focus on device control, which,
`as the ordinarily skilled artisan would know and understand, 45
`is not the same as (or equivalent to) the establishment of
`logical rules that govern the interaction of a migrated virtual
`machine with the rest of the network infrastructure, as
`described herein below by the present invention. These con(cid:173)
`ventional methods and devices also do not disclose or sug- 50
`gest, however, that a logical device needs to be bootstrapped
`and/or that device state in the network needs to be revoked
`upon migration of a virtual machine partition, as described
`herein below by the present invention
`Other conventional systems and methods relate lo a logical 55
`partition manager. These methods discuss the possibility of
`feeding information that is created within a logical partition
`(guest, or virtual machine) back to a partition manager. These
`conventional methods discuss the operating system (OS)
`itself applying security controls and routing in a special par(cid:173)
`tition. The crux of these conventional methods is so-called
`paravirtualization.
`In paravirtualizalion, the partition manager "trusts" the
`partition OS to cooperate with the other partitions. These
`conventional systems suffer from a serious security flaw that
`an undermined OS can disable access protection that prevents
`remote control software from manipulating an operating sys-
`
`30
`
`In view of the foregoing and other exemplary problems,
`drawbacks, and disadvantages of the conventional methods
`and systems, an exemplary feature of the present invention is
`to provide a method and system for providing control of
`network security ofa virtual machine, and more particularly,
`to a method of virtual machine migration with filtered net-
`work connectivity which includes enforcing network security
`and routing at a hypervisor layer at which a virtual machine
`partition is executed and which is independent of guest oper(cid:173)
`ating systems.
`As mentioned above, in order to perform maintenance on
`or provide a fail-over for a processor device or machine, it is
`desirable to move or migrate a virtual machine (VM) from
`one processor machine or device to another processor
`machine or device. However, conventional systems and meth(cid:173)
`ods require a complex scheme to update and install ACLs in
`the real switches of the machines and update and install
`firewalls. Also, the conventional systems and methods pro(cid:173)
`vide very little security.
`The exemplary method and system of the present invention
`can provide control of network security of a virtual machine
`by enforcing network security and routing at a hypervisor
`layer at which a virtual machine partition is executed and
`which is independent of guest operating systems.
`The exemplary aspects of the present application prefer-
`ably can provide a hypervisor security architecture designed
`and developed to provide a secure foundation for server plat(cid:173)
`forms, providing numerous beneficial fonctions, such as,
`strong isolation, mediated sharing and communication
`60 between virtual machines. These properties can all be strictly
`controlled by a flexible access control enforcement engine,
`which also can enforce mandatory policies.
`The exemplary features of the invention also can provide
`attestation and integrity guarantees for the hypervisor and its
`65 virtual machines.
`For example, the present invention exemplarily defines a
`computer implemented method of controlling network secu-
`
`
`
`Case 6:20-cv-01152-ADA Document 1-3 Filed 12/16/20 Page 23 of 28
`
`US 8,381,209 B2
`
`5
`rity of a virtual machine, including enforcing network secu(cid:173)
`rity and routing at a hypervisor layer.
`Particularly, the present invention defines a computer
`implemented method of virtual machine migration with fil(cid:173)
`tered network connectivity, including enforcing network
`security and routing at a hypervisor layer which is indepen(cid:173)
`dent of guest operating systems.
`The exemplary method of the present invention can
`include, for example, copying network security and routing
`for the virtual machine to the hypervisor layer, migrating the
`virtual machine from a first hardware device to a second
`hardware device, updating routing controls for the virtual
`machine at the hypervisor level, updating traffic filters for the
`virtual machine at the hypervisor level, and advertising the
`migration of the virtual machine from the first hardware
`device to the second hardware device.
`On the other hand, an exemplary system for controlling
`network security of a virtual machine by enforcing network
`security and routing at a hypervisor layer, according to the
`present invention, includes a copying unit that copies network
`security and routing for the virtual machine to the hypervisor
`layer, a migrating unit that migrates the virtual machine from
`a first hardware device to a second hardware device, a first
`updating unit that updates routing controls for the virtual
`machine at the hypervisor level , a second updating unit that
`updates traffic filters for the virtual machine at the hypervisor
`level, and an advertising unit that advertises the migration of
`the virtual machine from the first hardware device to the
`second hardware device.
`As mentioned above, in the conventional methods and 30
`systems, it is difficult to move one virtual machine from one
`machine to another. Generally, in conventional systems, to
`move a virtual machine from one machine to another (e.g.,
`from hardware 1 to hardware 2), the conventional methods
`and systems would merely shut down and copy from hard(cid:173)
`ware 1 lo hardware 2. The conventional systems andmelhods
`have difficulties with security and routing.
`To solve the problems with the conventional systems and
`methods, the present invention copies security and routing,
`etc. for the virtual machine to the hypervisor layer so that the
`user will see no difference in operation between running the
`virtual machine on hardware 1 or hardware 2. That is, accord-
`ing to the present invention, the first and second device (e.g.,
`hardware 1 and hardware 2) would each act the same, and
`preferably, would each have the same internet protocol (IP)
`address.
`An important problem arises when networks are very large,
`such as Google and Yahoo, in which there could be a thousand
`servers, and no flat topography, switches and routers to pro(cid:173)
`tect the servers. That is, in such systems, the virtual system is
`run on top of the hypervisor such that each virtual system is
`only as good as the security at each machine.
`To migrate the virtual machine from a first hardware device
`to a second hardware device, the present invention routes
`nel work traffic for the virtual machine lo the second hardware 55
`device at the hypervisor layer. The present invention also sets
`firewalls to permit network traffic for the virtual machine to
`go to the second hardware device at the hypervisor layer.
`According to the present invention, the hypervisor level
`provides traffic filtering and routing updating. Thus, the real
`switches do not need to be updated at the first and second
`hardware devices.
`Moreover, the present invention advertises the migration of
`the virtual machine from the first hardware device to the
`second hardware device using the second hardware device. 65
`Thus, the present invention has an important advantage of not
`requiring central control. The routers also do not need to be
`
`6
`updated because the migration is being advertised from the
`second hardware device (e.g., hardware 2).
`The present invention decentralizes the updating scheme
`by using the hypervisor layer for secu