VMware and CPU
`Virtualization Technology
`
`Jack Lo
`Sr. Director, R&D
`
`Columbia Ex 2043-1
`Symantec v Columbia
`IPR2015-00375
`
`
`Virtualization Technology
`
`Jack Lo
`Sr. Director, R&D
`
`Columbia Ex 2043-1
`Symantec v Columbia
`IPR2015-00375
`
`
`
`This presentation may contain VMware
`confidential information.
`
`Copyright © 2005 VMware, Inc. All rights reserved. All other
`marks and names mentioned herein may be trademarks of their respective
`companies.
`
`Columbia Ex 2043-2
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Overview
`
`(cid:131) Emerging technologies that impact CPU
`virtualization
`(cid:131) Hardware assist (VT-x/Pacifica)
`(cid:131) 64-bit computing
`(cid:131) OS assist (paravirtualization)
`
`(cid:131) Today’s talk:
`(cid:131) Share our perspective on emerging
`technologies
`
`Columbia Ex 2043-3
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Agenda
`(cid:131) CPU virtualization technology overview
`(cid:131) Virtualizing the x86 architecture
`(cid:131) Trend No. 1: Hardware assist
`(cid:131) VT-x and Pacifica
`(cid:131) Trend No. 2: 64-bit computing
`(cid:131) Benefits of 64-bit architecture
`(cid:131) 64-bit guest support
`(cid:131) Trend No. 3: OS assist
`(cid:131) VMware and paravirtualization
`
`Columbia Ex 2043-4
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Full Virtualization Software Stack
`Third-party Solutions
`
`Management
`and
`Distributed
`Virtualization
`Services
`
`VMotion
`Provisioning
`Backup
`DRS
`DAS
`Distributed
`Services
`
`VirtualCenter
`
`SDK / VirtualCenter Agent
`
`VMX
`
`VMX VMX
`
`VMX
`
`VM
`
`VM VM VM
`
`Third-
`Party
`Agents
`
`Service
`Console
`
`I/O Stack
`Device Drivers
`
`Resource
`Management
`CPU Scheduling
`Memory Scheduling
`Storage Bandwidth
`Network Bandwidth
`ESX Server
`
`Hardware
`
`Virtual
`Machine
`Monitor
`
`Enterprise
`Class
`Virtualization
`Functionality
`
`VMM
`
`VMM VMM
`
`VMM
`
`Distributed
`Virtual Machine
`File System
`
`Virtual NIC
`and
`Switch
`
`Storage Stack Network Stack
`
`Device Drivers
`
`VMkernel Hardware Interface
`
`Columbia Ex 2043-5
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Today’s Focus
`Third-party Solutions
`
`Management
`and
`Distributed
`Virtualization
`Services
`
`VMotion
`Provisioning
`Backup
`DRS
`DAS
`Distributed
`Services
`
`VirtualCenter
`
`SDK / VirtualCenter Agent
`
`VMX
`
`VMX VMX
`
`VMX
`
`VM
`
`VM VM
`
`VM
`
`Third-
`Party
`Agents
`
`Service
`Console
`
`I/O Stack
`Device Drivers
`
`Resource
`Management
`CPU Scheduling
`Memory Scheduling
`Storage Bandwidth
`Network Bandwidth
`ESX Server
`
`Hardware
`
`Virtual
`Machine
`Monitor
`
`Enterprise
`Class
`Virtualization
`Functionality
`
`VMM
`
`VMM VMM
`
`VMM
`
`Distributed
`VM File System
`
`Virtual NIC &
`Switch
`
`Storage Stack Network Stack
`
`Device Drivers
`
`VMkernel Hardware Interface
`
`Columbia Ex 2043-6
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Virtualization SW Terminology
`
`VMM
`
`VMM
`
`VMM
`
`Enhanced
`Functionality
`
`Base Functionality (e.g. scheduling)
`Hypervisor
`(cid:131) Virtual Machine Monitor (VMM)
`(cid:131) SW component that implements virtual machine hardware abstraction
`(cid:131) Responsible for running the guest OS
`(cid:131) Hypervisor
`(cid:131) Software responsible for hosting and managing virtual machines
`(cid:131) Run directly on the hardware
`(cid:131) Functionality varies greatly with architecture and implementation
`
`Columbia Ex 2043-7
`Symantec v Columbia
`IPR2015-00375
`
`
`
`CPU Virtualization
`(cid:131) Three components to classical virtualization techniques
`(cid:131) Many virtualization technologies focus on handling privileged
`instructions
`
`Privileged instruction
`virtualization
`
`Memory virtualization
`
`De-privileging or ring
`compression to handle privileged
`instructions
`Memory partitioning and
`allocation of physical memory
`
`Device and I/O
`virtualization
`
`Routing I/O requests between
`virtual devices and physical
`hardware
`
`Columbia Ex 2043-8
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Apps
`
`Ring 3
`
`Guest OS
`
`Ring 0
`
`Handling Privileged Instructions
`(cid:131) In traditional systems
`(cid:131)OS runs in privileged mode
`(cid:131)OS “owns” the hardware
`(cid:131)Application code has less privilege
`(cid:131) VMM needs highest privilege level
`for isolation and performance
`(cid:131) Traditional VMM relies on “ring
`compression” or “de-privileging”
`(cid:131)Run privileged guest OS code at user-level
`(cid:131)Privileged instructions trap, and emulated by VMM
`
`Guest OS
`
`Apps
`
`Ring 3
`
`VMM
`
`Ring 0
`
`Columbia Ex 2043-9
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Virtualizing x86 Architecture
`(cid:131)De-privileging not possible with x86!
`(cid:131)Some privileged instructions have different
`semantics at user-level: “non-virtualizable
`instructions”
`(cid:131)VMware uses direct execution and
`binary translation (BT)
`(cid:131)BT for handling privileged code
`(cid:131)Direct execution of user-level code for
`performance
`(cid:131)Any unmodified x86 OS can run in virtual
`machine
`
`Columbia Ex 2043-10
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Protecting the VMM
`(cid:131) Need to protect VMM and ensure isolation
`(cid:131) Protect virtual machines from each other
`(cid:131) Protect VMM from virtual machines
`(cid:131) VMware relies on segmentation hardware
`to protect the VMM
`(cid:131) VMM lives at top of guest address space
`(cid:131) Segment limit checks catch writes to VMM area
`VMM
`
`0
`
`4GB
`
`Columbia Ex 2043-11
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Agenda
`(cid:131) CPU virtualization technology overview
`(cid:131) Virtualizing the x86 architecture
`(cid:131) Trend No. 1: Hardware assist
`(cid:131) Trend No. 2: 64-bit computing
`(cid:131) Trend No. 3: OS assist
`
`Columbia Ex 2043-12
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Trend No. 1: Hardware Assist
`(cid:131) CPU vendors are embracing virtualization
`(cid:131)Intel Virtualization Technology (VT-x)
`(cid:131)AMD Pacifica
`(cid:131) These CPU technologies are a series of
`enhancements to aid virtualization SW
`(cid:131)Initially focused on handling non-virtualizable
`instructions
`(cid:131)Use a trap-and-emulate model
`(cid:131)Alternative to using binary translation
`(cid:131) But hardware assist does not eliminate
`need for VMware technology
`
`Columbia Ex 2043-13
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Non-root mode
`
`Root mode
`
`Apps
`
`Apps
`
`Ring 3
`
`Guest OS
`
`Guest OS
`
`Ring 0
`
`VM
`exit
`
`VM
`enter
`
`Virtual Machine Monitor (VMM)
`
`VT-x/Pacifica Overview
`(cid:131) Key feature is new CPU
`execution mode (root mode)
`(cid:131) VMM executes in
`root mode
`(cid:131) Allows x86 virtualization
`without binary translation
`or paravirtualization
`(cid:131) Guest state stored in
`Virtual Machine Control
`Structures (VT-x) or Virtual
`Machine Control Block
`(Pacifica)
`
`Columbia Ex 2043-14
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Limitations of Hardware Assist
`(cid:131) Initial VT-x/Pacifica hardware does not include all
`components of CPU virtualization solution
`(cid:131) VT-x requires small emulator for real mode code
`(cid:131) Memory virtualization support lacking
`(cid:131) Not in VT-x; implementation-dependent for Pacifica
`(cid:131) Memory virtualization is key to performance!
`(cid:131) No device virtualization support
`Hardware
`Assist
`Yes
`No
`No
`
`Privileged instructions
`Memory virtualization
`Device and I/O virtualization
`
`Columbia Ex 2043-15
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Future of Hardware Assist
`(cid:131) CPU vendors will add more hardware capabilities
`in future
`(cid:131) Memory virtualization (Nested paging, EPT)
`(cid:131) VMware software will evolve to incorporate
`support for these new technologies
`(cid:131) Adopt technologies as they enable new capabilities
`Hardware Solution
`VT-x, Pacifica
`
`Extended Page
`Tables/Nested Paging
`Intelligent Devices
`
`Privileged
`instructions
`Memory virtualization
`
`Devices and I/O
`
`Columbia Ex 2043-16
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Trend No. 2: 64-bit Computing
`(cid:131) Progression of the x86 architecture
`(cid:131) 16-bit: 8086/8088 (1978)
`(cid:131) 32-bit: 80386 (1985)
`(cid:131) 64-bit: x86-64 (2003): a.k.a. AMD64, x64
`(cid:131) x86-64 architecture brings 64-bit
`computing to industry-standard
`systems
`(cid:131) Provides compatibility mode to run 32-bit
`x86 applications
`(cid:131) Extensions to x86 architecture
`
`Columbia Ex 2043-17
`Symantec v Columbia
`IPR2015-00375
`
`
`
`64-bit Transition Has Already Begun
`(cid:131) Apps exhausting limits of 32-bit address space
`(cid:131) Consuming 1 bit of address space / year
`(cid:131) Databases, Java app servers, other threaded
`applications
`(cid:131) Most new CPUs are 64-bit enabled
`(cid:131) AMD64, EM64T
`(cid:131) Major OSes have been ported
`(cid:131) Windows, Linux, Solaris 10, etc.
`(cid:131) Applications are being ported
`(cid:131) Databases, app servers, development tools,
`games, etc.
`
`Columbia Ex 2043-18
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Virtualization And x86-64
`(cid:131) Potential questions about 64-bit transition
`(cid:131) Do my apps run in 64-bit OS?
`(cid:131) Have drivers been ported?
`(cid:131) Are the 64-bit OSes robust?
`(cid:131) The solution: virtualization!
`(cid:131) Great aid for 64-bit transition
`(cid:131) Easy way to evaluate new
`OSes
`(cid:131) Can run 64-bit guest OSes
`on 32-bit host OS on 64-bit hardware)!
`
`32-bit or 64-bit
`Host OS
`
`64-bit Hardware
`
`Applications
`
`32-
`bit
`VM
`
`64-
`bit
`VM
`
`VMM
`
`VMM64
`
`Columbia Ex 2043-19
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Challenges of Virtualizing x86-64
`(cid:131) Initial AMD64 architecture did not include
`segmentation in 64-bit mode
`(cid:131) Segmentation also missing from EM64T
`How do we protect the VMM?
`(cid:131) 64-bit guest support requires additional
`hardware assistance
`(cid:131) Segment limit checks available in 64-bit mode
`on newer AMD processors
`(cid:131) VT-x can be used to protect the VMM on EM64T
`(cid:131) Requires trap-and-emulate approach instead of BT
`
`Columbia Ex 2043-20
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Flexible VMM Architecture
`(cid:131) Flexible “multi-mode” VMM architecture
`(cid:131) Separate VMM per virtual
`machine
`(cid:131) 32-bit: BT VMM
`(cid:131) 64-bit: BT or VT/Pacifica
`VMM depending on
`hardware
`(cid:131) Select mode that
`achieves best workload-
`specific performance
`(cid:131) Same VMM architecture for ESX Server,
`GSX Server, Workstation and ACE
`
`VM
`
`VM
`
`VM
`
`BT/VT
`VMM64
`
`BT
`VMM32
`
`BT/VT
`VMM64
`
`. . .
`
`. . .
`
`VM
`
`BT
`VMM32
`
`Columbia Ex 2043-21
`Symantec v Columbia
`IPR2015-00375
`
`
`
`64-bit Guests And WS 5.5
`(cid:131) Workstation 5.5 enables 64-bit guests
`(cid:131) Currently in beta
`(cid:131) Simultaneously run 32-bit and 64-bit guests
`(cid:131) Runs on 32-bit and 64-bit host OSes
`
`Applications
`
`32-bit or 64-bit
`Host OS
`
`32-
`bit
`VM
`
`64-
`bit
`VM
`
`VMM VMM64
`
`64-bit Hardware
`
`Columbia Ex 2043-22
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Requirements For 64-bit Guests
`(cid:131) Newer hardware required for 64-bit guest
`support
`(cid:131) AMD Opteron Rev. E or later
`(cid:131) AMD Athlon64 Rev. D or later
`(cid:131) Intel VT-enabled processor
`(cid:131) How to determine that you have a 64-bit
`capable system?
`(cid:131) Workstation 5.5 will automatically check to see if
`your CPU meets the requirements
`(cid:131) CPU check utility also available for download on
`WS5.5 beta web page
`(cid:131) http://www.vmware.com/products/beta/ws/
`
`Columbia Ex 2043-23
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Trend No. 3: OS Assist
`(cid:131) Three alternatives for handling non-virtualizable
`instructions
`(cid:131) Binary translation
`(cid:131) Hardware assist
`(cid:131) OS assist or paravirtualization
`
`Compatibility
`Performance
`VMM sophistication
`
`Para-
`virtualization
`
`Binary
`Translation
`Excellent
`Good
`High
`
`Hardware
`Assist
`Excellent
`Average
`Average
`
`Columbia Ex 2043-24
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Paravirtualization
`(cid:131) Paravirtualization can address same problem as
`hardware assist
`(cid:131) Modify the guest OS to remove non-virtualizable instructions
`(cid:131) Export a simpler architecture to OS
`(cid:131) Cannot support unmodified OSes (e.g. Windows 2000/XP)
`(cid:131) Paravirtualization not limited to CPU virtualization
`(cid:131) Higher performance possible
`(cid:131) Relatively easy to add paravirtualization support:
`very difficult to add binary translation
`Hardware
`Binary
`Assist
`Translation
`Excellent
`Excellent
`Average
`Good
`Average
`High
`
`Compatibility
`Performance
`VMM sophistication
`
`Para-
`virtualization
`Poor
`Excellent
`Average
`
`Columbia Ex 2043-25
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Paravirtualization Challenges
`(cid:131) XenoLinux paravirtualization approach
`unsuitable for enterprise use
`(cid:131) Relies on separate kernel for native and in
`virtual machine
`(cid:131) Guest OS and hypervisor tightly coupled
`(cid:131) Tight coupling inhibits compatibility
`(cid:131) Changes to the guest OS are invasive
`(cid:131) Inhibits maintainability and supportability
`(cid:131) Guest kernel must be recompiled when
`hypervisor is updated
`(cid:131) How can we deliver paravirtualization for
`enterprise customers?
`
`Columbia Ex 2043-26
`Symantec v Columbia
`IPR2015-00375
`
`
`
`VMI Paravirtualization API
`(cid:131) VMware proposal: Virtual machine Interface API
`(cid:131) VMI provides maintainability & stability
`(cid:131) API supports low-level and higher-level interfaces
`(cid:131) Allows same kernel to run natively and in a
`paravirtualized virtual machine: “transparent
`paravirtualization”
`(cid:131) Allows for replacement of hypervisors without a guest
`recompile
`(cid:131) Preserve key virtualization functionality: page sharing,
`VMotion, etc.
`(cid:131) We are gathering feedback on the API from many kernel
`developers and OSVs
`(cid:131) http://www.vmware.com/vmi
`(cid:131) http://www.vmware.com/standards/hypercalls.html
`
`Columbia Ex 2043-27
`Symantec v Columbia
`IPR2015-00375
`
`
`
`VMI Paravirtualization
`(cid:131) VMI approach to paravirtualization improves
`compatibility
`(cid:131) API need not compromise performance
`compared to invasive paravirtualization
`
`Compatibility
`Performance
`VMM sophistication
`
`Binary
`Translation
`Excellent
`Good
`High
`
`Hardware
`Assist
`Excellent
`Average
`Average
`
`Para-
`virtualization
`Good
`Excellent
`Average
`
`Columbia Ex 2043-28
`Symantec v Columbia
`IPR2015-00375
`
`
`
`VMware Paravirtualization Performance
`
`Performance Relative to Native
`Bigger is Better
`
`1
`0.9
`0.8
`0.7
`0.6
`0.5
`0.4
`0.3
`0.2
`0.1
`0
`
` compile
`
`Dbench/1client [cpu
`microbenchmark]
`
`Dbench/1client
`HIGHMEM [cpu
`microbenchmark]
`
`Database kernel
`
`Xen2
`
`Xen3-devel
`
`VMware-para
`
`Columbia Ex 2043-29
`Symantec v Columbia
`IPR2015-00375
`
`
`
`VMware And Paravirtualization
`(cid:131) VMware will support paravirtualized
`Linux OSes
`(cid:131) Another guest type when such
`OS’s commercially available
`(cid:131) Flexible architecture
`(cid:131) Use most efficient
`technique for the
`guest OS type
`(cid:131) BT, VT/Pacifica, or
`paravirtualization
`
`VM
`
`VM
`
`VM
`
`VM
`
`Para-
`VMM
`
`VMM32
`
`VMM64
`
`VMM64
`
`Distributed
`VM File
`System
`
`Virtual
`NIC &
`Switch
`
`Storage
`Stack
`
`Network
`Stack
`
`Resource
`Management
`
`Device Drivers
`ESX Server
`
`Hardware
`
`Columbia Ex 2043-30
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Summary
`(cid:131) 64-bit transition happening now
`(cid:131) Virtualization can assist with transition
`(cid:131) 64-bit guests supported in WS5.5
`(cid:131) VMware provides flexible architecture to support
`emerging virtualization technologies
`(cid:131) Multi-mode VMM utilizes binary translation, hardware
`assist and paravirtualization
`(cid:131) Select best operating mode for the workload
`(cid:131) VMware will support paravirtualized guests as they
`appear in enterprise distributions
`(cid:131) VMI offers superior maintainability/flexibility
`(cid:131) Performs as well as invasive paravirtualization
`
`Columbia Ex 2043-31
`Symantec v Columbia
`IPR2015-00375
`
`
`
`PAC346
`VMware and CPU
`Virtualization Technology
`
`Jack Lo
`Sr. Director, R&D
`
`Columbia Ex 2043-32
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Backup slides
`
`Columbia Ex 2043-33
`Symantec v Columbia
`IPR2015-00375
`
`
`
`Performance of Binary Translation
`(cid:131) BT provides many performance optimization
`opportunities
`(cid:131) Fault elimination
`(cid:131) Avoid costs of repeated virtual machine exits
`(cid:131) Binary translator identifies faulting instructions and
`replaces them with special translations
`(cid:131) Jump directly to appropriate handlers without
`an expensive fault
`(cid:131) Guest and VMM share an address space:
`reduces context switch costs
`
`Columbia Ex 2043-34
`Symantec v Columbia
`IPR2015-00375
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
