`(12) Patent Application Publication (10) Pub. No.: US 2012/0317568 A1
`Aasheim
`(43) Pub. Date:
`Dec. 13, 2012
`
`US 20120317568A1
`
`(54) OPERATING SYSTEM DECOUPLED
`HETEROGENEOUS COMPUTING
`
`(75) Inventor:
`(73) Assignee:
`
`Jered Aasheim, Bellevue, WA (US)
`MICROSOFT CORPORATION,
`Redmond, WA (US)
`
`(21) Appl. No.:
`
`13/155,387
`
`(22) Filed:
`
`Jun. 8, 2011
`Publication Classification
`
`(51) Int. Cl
`we
`(2006.01)
`G06F 9/455
`(52) U.S. Cl. ............................................................ 718/1
`
`ABSTRACT
`(57)
`A heterogeneous processing system is described herein that
`provides a software hypervisor to autonomously control oper
`ating system thread Scheduling across big and little cores
`without the operating system's awareness or involvement to
`improve energy efficiency or meet other processing goals.
`The system presents a finite set of virtualized compute cores
`to the operating system to which the system schedules threads
`for execution. Subsequently, the hypervisor intelligently con
`trols the physical assignment and selection of which core(s)
`execute each thread to manage energy use or other processing
`requirements. By using a software hypervisor to abstract the
`underlying big and little computer architecture, the perfor
`mance and power operating differences between the cores
`remain opaque to the operating system. The inherent indirec
`tion also decouples the release of hardware with new capa
`bilities from the operating system release schedule.
`
`420
`
`440
`
`S
`
`
`
`y
`
`
`
`S
`
`SSSSSS
`
`SSSSSS
`
`3SY
`
`SSSSS
`
`3.
`
`S
`
`l
`
`S
`
`:S
`
`- - -
`
`- - - - - - - - - -
`
`- -
`
`S.
`
`Coherent interconnect
`
`Petitioner Mercedes Ex-1023, 0001
`
`
`
`Patent Application Publication
`
`Dec. 13, 2012 Sheet 1 of 4
`
`US 2012/0317568 A1
`
`100
`
`
`
`Heterogeneous Processing System
`110
`120
`
`Central
`Processing
`Unit(s)
`
`Operating
`System
`Interface
`Component
`
`Virtual COre
`Mgmt.
`Component
`
`Policy
`Engine
`Component
`
`Policy Data
`Store
`
`Scheduling
`Component
`
`Capability
`Mgmt.
`Component
`
`Hardware
`Interface
`Component
`
`FIG. I.
`
`Petitioner Mercedes Ex-1023, 0002
`
`
`
`Patent Application Publication
`
`Dec. 13, 2012 Sheet 2 of 4
`
`US 2012/0317568 A1
`
`Initialize Virtual COres
`
`Receive Startup Request
`
`210
`
`Enumerate Physical
`COres
`
`Determine COre
`Capabilities
`
`220
`
`230
`
`ldentify Operating System
`
`240
`
`Access Hypervisor Policy
`
`250
`
`Create Virtual COres
`
`260
`
`Invoke Operating System
`
`270
`
`DOne
`
`FIG. 2
`
`Petitioner Mercedes Ex-1023, 0003
`
`
`
`Patent Application Publication
`
`Dec. 13, 2012 Sheet 3 of 4
`
`US 2012/0317568 A1
`
`Schedule Thread
`
`Receive Thread
`Scheduling Request
`
`Determine Thread
`Processing Needs
`
`310
`
`320
`
`Access Scheduling Policy
`
`330
`
`Select Physical
`Processing Core
`
`Manage Core Capability
`Differences
`
`340
`
`350
`
`Schedule Thread to
`Execute On Selected COre
`
`360
`
`FIG. 3
`
`Petitioner Mercedes Ex-1023, 0004
`
`
`
`Patent Application Publication
`
`Dec. 13, 2012 Sheet 4 of 4
`
`US 2012/0317568 A1
`
`y^y Y.
`
`420
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`440
`
`
`
`
`
`Coherent interconnect
`
`FIG. 4
`
`Petitioner Mercedes Ex-1023, 0005
`
`
`
`US 2012/0317568 A1
`
`Dec. 13, 2012
`
`OPERATING SYSTEM DECOUPLED
`HETEROGENEOUS COMPUTING
`
`BACKGROUND
`Energy efficiency is increasingly becoming an
`0001
`important differentiator from mobile phones to datacenters.
`Customers are willing to pay a premium for longer lasting
`mobile device experiences but also are anxious to get increas
`ing performance from these same devices. On the other end of
`the scale, datacenters continue to scale up compute power but
`face thermal limits for what can be efficiently cooled. In
`addition, the public is increasingly more conscious of energy
`usage and environmental impact of energy use. Making effi
`cient use of energy is therefore a higher priority design goal in
`many types of computing systems.
`0002 These technically opposing agendas—delivering
`more performance but using less power—have resulted in the
`industry experimenting with heterogeneous designs of “big”
`compute cores closely coupled with “little' compute cores
`within a single system or silicon chip, called heterogeneous
`cores or processing herein. The big cores are designed to offer
`high performance in a larger power envelope while the little
`cores are designed to offer lower performance in a smaller
`power envelope. The conventional wisdom is that an operat
`ing system's scheduler will then selectively schedule threads
`on the big or little cores depending upon the workload(s).
`During at least some times of the day, the operating system
`may be able to turn off the big core(s) entirely and rely on the
`power sipping little cores.
`0003 Big and little cores may or may not share the same
`instruction set or features. For example, little cores may
`include a reduced instruction set or other differences that
`involve further decision making by the operating system to
`schedule processes on a compatible core. One traditional
`example is a system that includes a central processing unit
`(CPU) and graphics-processing unit (GPU) and allows the
`GPU to be used for computing tasks when it is idle or
`underutilized.
`0004 Existing and present solutions depend on modifying
`the operating system's kernel in order to “enlighten' the
`operating system to the presence of big and little cores, their
`respective performance and power characteristics, and which
`facilities in the system (e.g. CPU performance counters,
`cache miss/hit counters, bus activity counters, and so on) the
`operating system can monitor for determining on which core
`(s) to schedule a particular thread. This approach has several
`drawbacks: 1) it involves modifying the kernel for all Sup
`ported operating systems, 2) it requires the modified kernel to
`understand differences in big/little designs across potentially
`different architectures (e.g., Supporting N different imple
`mentations), and 3) it tightly couples the release schedule of
`the operating system kernel and the underlying computer
`architecture. Changes to the computer architecture then
`involve waiting for the next scheduled operating system
`release (i.e., potentially several years or more) before the
`kernel can Support new cores commercially (or vice versa).
`
`SUMMARY
`0005. A heterogeneous processing system is described
`herein that provides a software hypervisor to autonomously
`control operating system thread scheduling across big and
`little cores without the operating system's awareness or
`involvement to improve energy efficiency or meet other pro
`
`cessing goals. The system presents a finite set of virtualized
`compute cores to the operating system to which the system
`schedules threads for execution. Subsequently, underneath
`the Surface, the hypervisor intelligently controls the physical
`assignment and selection of which core(s)—big or little—
`execute each thread to manage energy use or other processing
`requirements. By using a software hypervisor to abstract the
`underlying big and little computer architecture, the perfor
`mance and power operating differences between the cores
`remain opaque to the operating system. The inherent indirec
`tion also decouples the release of hardware with new capa
`bilities from the operating system release schedule. A hard
`ware vendor can release an updated hypervisor, and allow
`new hardware to work with any operating system version the
`Vendor chooses.
`0006. This Summary is provided to introduce a selection
`of concepts in a simplified form that are further described
`below in the Detailed Description. This Summary is not
`intended to identify key features or essential features of the
`claimed subject matter, nor is it intended to be used to limit
`the scope of the claimed subject matter.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`0007 FIG. 1 is a block diagram that illustrates compo
`nents of the heterogeneous processing system, in one embodi
`ment.
`0008 FIG. 2 is a flow diagram that illustrates processing
`of the heterogeneous processing system to initialize a com
`puting device with heterogeneous processing cores using a
`hypervisor between the cores and an operating system, in one
`embodiment.
`0009 FIG. 3 is a flow diagram that illustrates processing
`of the heterogeneous processing system to schedule one or
`more operating system threads through a hypervisor that
`manages heterogeneous processing cores, in one embodi
`ment.
`0010 FIG. 4 is a block diagram that illustrates an operat
`ing environment of the heterogeneous processing system, in
`one embodiment.
`
`DETAILED DESCRIPTION
`0011. A heterogeneous processing system is described
`herein that provides a Software hypervisor to autonomously
`control operating system thread scheduling across big and
`little cores without the operating system's awareness or
`involvement to improve energy efficiency or meet other pro
`cessing goals. The system presents a finite set of virtualized
`compute cores to the operating system to which the system
`schedules threads for execution. Subsequently, underneath
`the Surface, the hypervisor intelligently controls the physical
`assignment and selection of which core(s)—big or little—
`execute each thread to manage energy use or other processing
`requirements. By using a software hypervisor to abstract the
`underlying big and little computer architecture, the perfor
`mance and power operating differences between the cores
`remain opaque to the operating system. The inherent indirec
`tion also decouples the release of hardware with new capa
`bilities from the operating system release schedule. A hard
`ware vendor can release an updated hypervisor, and allow
`new hardware to work with any operating system version the
`Vendor chooses.
`0012. The hypervisor implementation is tightly coupled to
`the underlying computer architecture and uses the available
`
`Petitioner Mercedes Ex-1023, 0006
`
`
`
`US 2012/0317568 A1
`
`Dec. 13, 2012
`
`system feedback (e.g., CPU utilization, bus/cache activity,
`and so forth) to autonomously assign the appropriate cores for
`the requested workloads. This approach allows the underly
`ing computer architecture to change frequently in coopera
`tion with the software hypervisor and decouple this evolution
`from the above operating system(s). The heterogeneous pro
`cessing system provides simple, course-grained power man
`agement without modifying the operating system kernel
`itself. Thus, the heterogeneous processing system allows for
`more rapid hardware innovation, and allows existing data
`center and other installations to benefit today from available
`heterogeneous processing hardware.
`0013 Heterogeneous computing is an emerging field
`within the industry with the goal of optimizing the execution
`of workloads based on different types of computing cores
`(e.g., CPUs, GPUs, accelerators, and so on) available in the
`system. Optimization can be for performance, power, latency,
`or other goals. The heterogeneous processing system, while
`applicable to these more general cases, is also targetable at
`systems with cores that have identical functional equivalence
`but differing performance/power operating characteristics.
`Typically, these systems have one or more big cores and one
`or more little cores. The big cores typically have deep pipe
`lines, out-of-order execution, large caches, high clock speeds,
`and are manufactured using higher leakage processes (e.g.
`40G). The little cores typically have shorter pipelines, smaller
`caches, lower clock speeds, various power levels, and are
`manufactured using low leakage processes (e.g. 40LP).
`0014. In some embodiments, the big and little cores may
`have architecture equivalence, micro-architecture equiva
`lence, a global interrupt controller, coherency, and virtualiza
`tion. Architecture equivalence may include the same Instruc
`tion Set Architecture (ISA), Single Instruction Multiple Data
`(SIMD), Floating Point (FP), co-processor availability, and
`ISA extensions. Micro-architecture equivalence may include
`difference in performance but the same configurable features
`(e.g. cache line length). A global interrupt controller provides
`the ability to manage, handle, and forward interrupts to all
`cores. Coherency means all cores can access (cache) data
`from other cores with forwarding as needed. Virtualization is
`for Switching/migrating workloads from/to cores.
`0015. In some embodiments, the heterogeneous process
`ing system may be able to handle minor differences in cores.
`For example, a little core that does not support Streaming
`Single Instruction, Multiple Data (SIMD) Extensions (SSE)
`(now existing in four iterations, SSE1, SSE2, SSE3, and
`SSE4), may still handle other Intel x86-based software code.
`The hypervisor may detect unsupported instructions in the
`instruction stream, and wake up an appropriate core to which
`to assign such streams. Other instruction streams may operate
`faithfully on any core. In some cases, such as where only a
`handful of unsupported instructions are used, the hypervisor
`may include Some level of emulation to emulate the unsup
`ported instructions on the available instruction set. For
`example, operations such as Vector math can often be broken
`down and implemented at lower efficiency using standard
`math instructions.
`0016. The software hypervisor installs itself during the
`device boot process prior to operating system (OS) initializa
`tion. After completing specified hardware configuration (i.e.,
`configuring memory, initializing the virtualization facilities,
`and so on), the hypervisor then configures the big and little
`processing cores installed in the computing device via policy.
`For example, if the device is a mobile phone, the policy could
`
`dictate that the hypervisor start the operating system with a
`minimal amount of performance available and optimize for
`battery life; the hypervisor would subsequently schedule
`operating system threads to one or more little cores. Alterna
`tively, if the device is a datacenter blade, the policy could
`dictate that the hypervisor start the operating system with the
`maximal amount of available performance and sacrifice
`energy efficiency; the hypervisor would Subsequently sched
`ule operating system threads to the available big cores—as
`well as possibly the little cores depending on the available
`thermal budget. After completing initialization, the Software
`hypervisor loads the operating system boot manager, which
`then loads the operating system.
`0017. During runtime, the heterogeneous processing sys
`tempresents a virtualized set of cores to the operating system.
`The operating characteristics and differences between the
`cores are opaque to the operating system and managed pri
`vately by the software hypervisor based upon the defined
`operating policy. The operating policy may be set during
`system initialization or dynamically during runtime.
`0018. The hypervisor uses the operating policy in con
`junction with available system facilities (e.g. CPU perfor
`mance counters, cache miss/hit counters, bus activity
`counters, and so on) to determine to which cores to schedule
`the operating system threads. The hypervisor will use this
`information to understand CPU core utilization, trends over
`time, locality of information, and input/output (I/O) patterns.
`From this information, the hypervisor can dynamically and
`speculatively migrate the operating system threads across the
`big and little cores as appropriate. Additionally, the hypervi
`Sor may also control dynamic frequency and Voltage scaling
`(DFVS) on behalf of the operating system depending on the
`system implementation.
`0019 Here is a sampling of available operating policies
`they hypervisor may control: Minimum Power (MiPo), Maxi
`mum Performance (MaPe), Minimal Power, Performance on
`Demand (MiPoD), and Maximum Performance, Power
`Down on Idle (MaPel). Each of these is described in the
`following paragraphs. However, additional, more advanced
`operating policies can be implemented as chosen by any
`particular implementation.
`(0020 Minimum Power (MiPo) schedules threads to the
`minimal set of cores. This typically will mean the hypervisor
`schedules threads to the little cores and uses DVFS as needed
`to control the power and performance operating point for the
`core. Additional little cores can be powered and scheduled as
`needed.
`0021 Maximum Performance (MaPe) schedules threads
`to the maximal set of cores. This typically will mean the
`hypervisor Schedules threads to all available cores—starting
`with the big cores—and use DVFS as needed to control the
`power and performance operating point for the cores. The
`little cores are also powered and scheduled as much is allowed
`by the available thermal budget.
`(0022. Minimal Power, Performance on Demand (MiPoD)
`normally operates at the lowest available power state (e.g., on
`one or more little cores) but boosts performance as workloads
`demand. This is commonly referred to as a “turbo' or “boost
`mode of operation and is enabled by dynamically allocating
`and Scheduling to big cores. Once the workload is completed,
`the system returns to the minimal power state (e.g. on a little
`core).
`0023 Maximum Performance, Power Down on Idle
`(MaPel) normally operates at the maximal available perfor
`
`Petitioner Mercedes Ex-1023, 0007
`
`
`
`US 2012/0317568 A1
`
`Dec. 13, 2012
`
`mance state (e.g. on one or more big cores) but acquiesces to
`lower power states once an idle threshold is reached. The idle
`threshold in this case is not the typical near-zero CPU utili
`zation but can be arbitrarily defined at some Dhrystone Mil
`lion Instructions per Second (DMIPS) or CPU utilization
`percentage as defined by the policy. When going to idle, the
`hypervisor dynamically allocates and schedules to little cores
`and puts the unused big cores into standby/parked States.
`Policy and/or future workloads determine when the system
`returns to the maximum available performance State (e.g. on
`big cores).
`0024 FIG. 1 is a block diagram that illustrates compo
`nents of the heterogeneous processing system, in one embodi
`ment. The system 100 includes one or more central process
`ing units 110, an operating system interface component 120,
`a virtual core management component 130, a policy engine
`component 140, a policy data store 150, a scheduling com
`ponent 160, a capability management component 170, and a
`hardware interface component 180. Each of these compo
`nents is described in further detail herein. The following
`components may be implemented within a software hypervi
`Sor that sits between an operating system and hardware
`resources of a computing device.
`0025. The one or more central processing units 110
`include one or more processing cores that have heterogeneous
`processing capabilities and power profiles. Typically, each
`CPU complex is located on a single silicon die and each core
`of a CPU complex shares a silicon die. Hardware can be
`implemented in a variety of packages for a variety of types of
`devices. For example, newer mobile devices and even some
`recent desktop processors include a CPU and GPU on the
`same chip for efficient communication between the two and
`lower power usage. Each CPU complex may include one or
`more big and little cores. Alternatively or additionally, one
`CPU complex may include all big cores while another CPU
`complex includes all little cores. CPU complexes as used here
`applies to GPUs and other hardware that can execute software
`instructions.
`0026. The operating system interface component 120
`communicates between a hypervisor and an operating system
`to receive instructions for delivering to hardware resources
`and for receiving output from the hardware resources. The
`operating system may schedule threads, provide a pointer to
`an instruction stream (e.g., a program counter (PC)), write to
`memory areas that pass instructions to hardware, and so forth.
`An operating system typically interacts directly with the
`hardware on a computing device. However, a hypervisor
`inserts a layer of indirection between the operating system
`and hardware for a variety of purposes. Often, hypervisors are
`used to provide virtualization so that multiple operating sys
`tems can be run contemporaneously on the same hardware. A
`hypervisor can also be used to present virtual hardware to the
`operating system that differs from the actual hardware
`installed in a computing device. In the case of the heteroge
`neous processing system 100, this can include making big and
`little cores appear the same to the operating system. The
`system 100 may even present a different number of cores to
`the operating system than actually exist in the device.
`0027. The virtual core management component 130 man
`ages one or more virtual cores that the hypervisor presents to
`the operating system. A virtual core appears to the operating
`system as a CPU core, but may differ in characteristics from
`available physical hardware in a computing device. For
`example, the virtual cores may hide differences in processing
`
`or power capabilities from the operating system, so that an
`operating system not designed to work with heterogeneous
`big and little cores can operate in a manner for which the
`operating system was designed. In such cases, the hypervisor
`provides any specialized programming needed to leverage the
`heterogeneous computing environment, so that the operating
`system need not be modified.
`0028. The policy engine component 140 manages one or
`more policies for scheduling operating system threads and
`presenting virtual cores to the operating system based on the
`available one or more central processing units. The policy
`engine component 140 may include hardcoded policies spe
`cific to a particular hypervisor implementation or may
`include administrator-configurable policies that can be modi
`fied to Suit the particular installation goals. Policies may
`determine which cores are scheduled first, tradeoffs between
`power usage and processing goals, how cores are shut off and
`awoken to save power, how virtual cores are presented to the
`operating system, and so forth.
`(0029. The policy data store 150 stores the one or more
`policies in a storage facility accessible to the hypervisor at
`boot and execution times. The policy data store 150 may
`include one or more files, file systems, hard drives, databases,
`or other storage facilities for persisting data across execution
`sessions of the system 100. In some embodiments, the admin
`istrator performs a setup step that takes the system 100
`through a configuration phase to store an initial set of policies
`for use by the hypervisor.
`0030 The scheduling component 160 schedules one or
`more instruction streams received as threads from the oper
`ating system to one or more of the central processing units
`installed in the computing device. The scheduling component
`receives a virtual core identification from the operating sys
`tem that identifies the virtual core to which the operating
`system requests to schedule the thread. The scheduling com
`ponent 160 examines the schedule request and determines a
`physical core on which to schedule the thread to execute. For
`example, the component 160 may determine if power or
`processing is more relevant for the thread, and schedule to an
`appropriate little or big core in response. In some cases, the
`component 160 may avoid scheduling threads to certain cores
`to allow those cores to be powered down to save power.
`0031. The capability management component 170 option
`ally manages one or more differences between big and little
`processing cores. In some cases, the system 100 may only
`operate on processing units in which the big and little cores
`share the same capabilities, and the capability management
`component 170 is not needed. In other cases, the system 100
`handles minor or major differences between available pro
`cessing cores. For example, the system 100 may watch for
`instructions that are not supported by some cores and sched
`ule the corresponding threads on cores that do Support those
`instructions. In more Sophisticated implementations, the
`component 170 may virtualize or emulate big core capabili
`ties on little cores (or vice versa) to satisfy a power or other
`profile goal.
`0032. The hardware interface component 180 communi
`cates between the hypervisor and central processing units to
`schedule Software instructions to run on available physical
`cores. The hardware interface component 180 may include
`real memory addresses or other facilities for accessing real
`hardware that are hidden from other components and in par
`ticular from the guest operating system(s) managed by the
`hypervisor.
`
`Petitioner Mercedes Ex-1023, 0008
`
`
`
`US 2012/0317568 A1
`
`Dec. 13, 2012
`
`0033. The computing device on which the heterogeneous
`processing system is implemented may include a central pro
`cessing unit, memory, input devices (e.g., keyboard and
`pointing devices), output devices (e.g., display devices), and
`storage devices (e.g., disk drives or other non-volatile storage
`media). The memory and storage devices are computer-read
`able storage media that may be encoded with computer-ex
`ecutable instructions (e.g., Software) that implement or
`enable the system. In addition, the data structures and mes
`sage structures may be stored or transmitted via a data trans
`mission medium, Such as a signal on a communication link.
`Various communication links may be used, such as the Inter
`net, a local area network, a wide area network, a point-to
`point dial-up connection, a cell phone network, and so on.
`0034 Embodiments of the system may be implemented in
`various operating environments that include personal com
`puters, server computers, handheld or laptop devices, multi
`processor systems, microprocessor-based systems, program
`mable consumer electronics, digital cameras, network PCs,
`minicomputers, mainframe computers, distributed comput
`ing environments that include any of the above systems or
`devices, set top boxes, systems on a chip (SOCs), and so on.
`The computer systems may be cell phones, personal digital
`assistants, Smartphones, personal computers, programmable
`consumer electronics, digital cameras, and so on.
`0035. The system may be described in the general context
`of computer-executable instructions, such as program mod
`ules, executed by one or more computers or other devices.
`Generally, program modules include routines, programs,
`objects, components, data structures, and so on that perform
`particular tasks or implement particular abstract data types.
`Typically, the functionality of the program modules may be
`combined or distributed as desired in various embodiments.
`0036 FIG. 2 is a flow diagram that illustrates processing
`of the heterogeneous processing system to initialize a com
`puting device with heterogeneous processing cores using a
`hypervisor between the cores and an operating system, in one
`embodiment.
`0037 Beginning in block 210, the system receives a star
`tup request to initialize a computing device. For example, a
`basic input/output system (BIOS), extensible firmware inter
`face (EFI), boot loader, or other initial device software may
`load and invoke a hypervisor that implements the heteroge
`neous computing system. In some cases, the administrator
`will have previously performed an installation phase to install
`the hypervisor on the computing device, although the system
`can also support networkboot and other non-installation sce
`narios commonly offered for computing devices.
`0038 Continuing in block 220, the system enumerates
`two or more physical processing cores of the computing
`device. In some embodiments, at least two cores offer differ
`ent performance and power usage characteristics. However,
`the system may also be used where asymmetry is not present.
`For example, using a Software hypervisor for power manage
`ment could still be applicable in scenarios where you have N
`physical CPUs on die but that only K can be operated based
`upon externalities such as: ambient temperature, form factor
`enclosure, cost of available power, etc. At boot, the hypervi
`sor can use this “policy' information to report a virtualized set
`of K cores to the operating system and this could vary upon
`each boot cycle. The hypervisor would be performing the
`same task in this scenario for symmetric cores. The system
`may invoke the BIOS or other underlying layer to determine
`how many and what kind of processors the computing device
`
`has installed, and may execute a CPUID or other similar
`instruction to determine information about the processing
`capabilities of the processors. In some embodiments, the
`system may include an extensibility interface through which
`drivers or other hypervisor extensions can be implemented
`and added by the hypervisor manufacturer or a third party to
`add Support for new processing hardware to the hypervisor,
`without necessarily updating the hypervisor itself.
`0039 Continuing in block 230, the system determines
`capabilities of each enumerated processing core. The capa
`bilities may include one or more power profiles offered by
`each core, one or more instruction sets Supported by each
`core, performance characteristics of each core, and so forth.
`The system may leverage informational interfaces (such as
`the previously mentioned CPUID instruction) of the core
`itself or information provided by a driver or other extension to
`the hypervisor, to determine each core's capabilities. The
`system uses the determined capabilities to assign threads to
`each core that are compatible with the core, and to perform
`scheduling in a manner consistent with received policies and
`processing goals.
`0040 Continuing in block 240, the system identifies one
`or more operating systems for which the hypervisor will
`manage access and Scheduling for the enumerated physical
`cores. The system may access a hard drive, flash drive, or
`other storage of the computing device to determine which
`operating system to invoke after the hypervisor is initialized.
`The hypervisor may be designed with information about vari
`ous operating systems, and may include extensibility so that
`new operating systems can be supported without updating the
`hypervisor itself. Each operating system and operating sys
`tem version may have different scheduling semantics or other
`nuances that the hypervisor handles to allow the operating
`system to execute correctly on virtualized processing
`resources. In some cases, the hypervisor may be requested to
`allow multiple operating systems to share the enumerated
`physical processing cores, and policy may dictate how that
`sharing is handled.
`0041 Continuing in block 250, the system accesses hyper
`visor policy information that specifies one or more goals for
`scheduling operating system threads on the enumerated
`physical processing cores. The goals may include perfor
`mance goals, power usage goals, or other directions for deter
`mining which core or cores on which to execute operating
`system threads. The policy may be stored in a storage device
`associated with the computing device, hardcoded into a
`hypervisor implementation, and so forth. The hypervisor may
`receive updates to the policy through an administrative inter
`face provided to administrators.
`0042 Continuing in block 260, the system creates one or
`more virtual cores to expose to the identified operating sys
`tem, wherein each virtual core isolates the operating system
`from determined differences in capabilities among the physi
`cal processing cores. For example, the heterogeneous pro
`cessing system may present two or more big and little cores as
`a single type of uniform virtual core to the operating system.
`Upon receiving a scheduling request from the operating sys
`tem to execute a thread on a virtual core, the system deter
`mines which physical core to select for the job based on the
`accessed hypervisor policy. The hypervisor policy may
`specify that the hypervisor present a different number of
`virtual cores than physical cores, such as when it is a goal to
`be able to seamlessly power down at least Some higher power
`demanding cores in favor of using lower power demanding
`
`Petitioner Mercedes Ex-1023, 0009
`
`
`
`US 2012/0317568 A1
`
`Dec. 13, 2012
`
`cores. Alternatively, the system may still power down cores
`the operating system is aware of but wake up the cores if the
`operating system chooses to use them or to use a quantity of
`cores that cannot be satisfied by lower powered cores alone.
`0043 Continuing in block 270, the system invokes the
`identified operating system