US 6,711,691 B1
`(10) Patent No.:
`a2) United States Patent
`Howardetal.
`(45) Date of Patent:
`Mar.23, 2004
`
`
`US006711691B1
`
`(54) POWER MANAGEMENT FOR COMPUTER
`SYSTEMS
`
`(75)
`
`Inventors: Brian D. Howard, Portola Valley, CA
`wi
`(US);Michael F. Culbert, San Jose,
`(
`)
`.
`.
`(73) Assignee: Apple Computer, Inc., Cupertino, CA
`(US)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`US.C, 154(b) by 0 days.
`
`(21) Appl. No.: 09/567,201
`
`(22)
`
`Filed:
`
`May8, 2000
`
`(60)
`
`Related U.S. Application Data
`Provisional application No. 60/133,918, filed on May 13,
`1999,
`
`(SL) Unt. C0 eee ecccccccsecseeseeseeseereeseeseeneesees GO06F 1/32
`(52) US. Ch. cece 713/300; 713/320; 713/322
`(58) Field of Search oo...eee 713/300, 320
`713/322 323 304
`
`(56)
`
`References Cited
`
`8/1998 Fung
`5,799,198 A
`9/1998 Carmeanetal. ............ 713/322
`5,809,314 A *
`9/1998 Broedneretal.
`5,812,796 A
`87000 rung
`ene ‘
`tal
`arky et al.
`105,
`9/2000. Mirandaet al.
`6,119,194 A
`6,141,762 A * 10/2000 Nicol et al. we. 713/300
`6,230,277 B1
`5/2001 Nakaokaet al.
`6.272.644 Bl
`8/2001. Uradeetall.
`6,308,278 B1
`10/2001 Khouli etal.
`6,460,143 B1 * 10/2002 Howard et al.
`
`.......0.0... 713/323
`
`OTHER PUBLICATIONS
`“Universal Serial Bus Specification”, Revision 1.0, Jan. 15,
`1996.
`
`* cited by examiner
`
`Primary Examiner—Dennis M. Butler
`(74) Attorney, Agent, or Firm—Beyer Weaver & Thomas
`LLP
`
`(57)
`
`ABSTRACT
`
`18 Claims, 22 Drawing Sheets
`
`100
`
`Power management approaches for computer systems hav-
`ing one or more processors are disclosed. One power man-
`agement approach provides hierarchical power manage-
`ment. The hierarchical nature of the power management
`provided by the invention has various levels of power
`managementsuch that power consumption of the computer
`system is dependent upon the amount of work placed on the
`U.S. PATENT DOCUMENTS
`processing resources of the computer system. Another
`5,167,024 A
`11/1992 Smith etal.
`power management approach pertains to deterministic hand-
`5,239,652 A
`8/1993 Seibertetal.
`shaking provided between a power manager and one or more
`5,254,928 A
`10/1993 Youngetal.
`controller units. The deterministic handshaking provides for
`5,396,635 A
`3/1995 Fung
`more reliable and controllable transitions between power
`5,483,656 A
`1/1996 Oprescuetal.
`managementstates which have associated power manage-
`5,557,777 A
`9/1996 Culbert
`ment taking place in the controller units. The power man-
`ean ‘ afoes cnet I et al.
`.
`+
`.
`.
`?
`2
`Uber
`
`5,632,037 A * s.escccsesssee- 713/322+*8ement approaches are suitable for use with a single5/1997 Maheret al.
`
`5708816 A
`1/1998 Culbert
`processor computer system or a multi-processor computer
`5,710,929 A
`1/1998 Fung
`system.
`5,724,591 A *
`3/1998 Hara etal. we. 713/322
`5,737,615 A *
`4/1998 Tetrick oes 713/324
`
`OPERATING
`
`
`SYSTEM 18
`f
`
`
`
`
`
`
`|
`INTA
`PROCESSOR
`SRST-A
`
`
`
`
`pS
`>|
`‘
`GAGA
`puuca
`
`
`2
`.
`
`dig!
`lee.
`INT-B.
`
`SRST-B]__,| PROCESSOR
`-t
`
`
`
`PLLC-B| “s
`|
`° oe
`
`
`Lo
`Lee
`INTC
`
`
`
`
`SMI-G.
`|
`QACK-C_
`srstcl PROCESSOR lQREG-C
`iNT-D
`
`
`
`
`
`LLC-C) 4os[~|LT >| et
`
`
`
`
`srstpl>| PROCESSOR l"
`P| MEMORY
`ONTROLLER/
`D
`
`SMD}
`»|
`BUS
`
`IPLLC-O soe |-BRECD>|MANAGER>
`
`
`
`
`HRST
`a2
`SUSREQ1
`
`
`POWER
`SUSACKt
`SRS’
`MANAGER
`
`
`
`
`
`
`
`
`
`
`
`140
`
`*
`
`SREQ
`
`PINT
`susREG?
`SUSACK2OSOSC—~—SSSOSOSCOCSCSC‘“
`
`VO - INTERRUPT CONTROLLER|
`416
`
`1
`
`APPLE 1021
`
`APPLE 1021
`
`1
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 1 of 22
`
`US 6,711,691 B1
`
` MONITOR WORKLOAD
`
`?
`
`
`IS
`WORKLOAD
`HEAVY
`
`
`
`
`
`
`
`
`
`ACTIVATE ONE OR MORE
`PROCESSORS
`
`
`IS
`
`WORKLOAD
`LIGHT
`?
`
`20
`
`
`
`DEACTIVATE ONE OR
`MORE PROCESSORS
`
`
`
`2
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 2 of 22
`
`US 6,711,691 B1
`
`rc
`
`
`
`>1
`
`
`ACTIVE
`PROCESSORS
`
`?
`PLACE ONE OR MORE(BUT
`
`NOT ALL) PROCESSORSIN
`SLEEP MODE BASED ON
`WORKLOAD
`
`
`
`
`
`
`
`IN FIRST LOW
`
`
`POWER
`
`MODE
`
`PLACE LAST PROCESSOR
`?
`
`IN FIRST LOW POWER
`MODE
`
`
`PLACE LAST PROCESSOR
`IN SECOND LOW POWER
`MODE
`
`SHUT DOWN UNNEEDED
`SUBSYSTEMS
`
`3
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 3 of 22
`
`US 6,711,691 B1
`
`OPERATING
`
`SYSTEM
`
`100
`
`J
`
`MANAGER
`
`MEMORY
`ONTROLLER/
`BUS
`
`FIG. 10
`
`4
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 4 of 22
`
`US 6,711,691 B1
`
`
`
`
`
`
`WAKEUP
`PROCESSING
`
`FOR LAST ACTIVE
`
`
`
`SHUTDOWN
`PROCESSOR
`
`PROCESSING
`FOR LAST PROCESOR
`
`238
`
`224
`
`214
`
`STARTUP PROCESSING
`
`204
`
`
`
`
` IDLE
`PROCESSING
`PRODESSOR
`
`
`
`
`
`
`
`
`ACTIVATION PROCESSING
`
`FOR ALL BUT LAST
`ACTIVE PROCESSOR
`
`
` NAP
`
`
`210
`
`
`
`PROCESSING
`FOR LAST
`228
`
`ACTIVE
`
`iP|PROCESSOR
`930
`218
`
`
`
`222
`
`SHUTDOWN PROCESSING
`FOR EXTRA PROCESSORS
`
`208
`
`
`
`ACTIVATION
`
`PROCESSING
`FOR LAST PROCESSOR
`
`
`
`NAP SINGLE
`INTERRUPT
`PROCESSING
`
`
`220
`
`FIG. 2
`
`5
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 5 of 22
`
`US 6,711,691 B1
`
`STARTUP PROCESSING
`
`[300
`
`PMGR ASSERTS SUSREQ2, SREQ AND SUSREQ1
`
`302
`
`PMGR ASSERTS ALL RESETS (EXCEPT POC)
`
`304
`
`308
`
`
`PMGR ASSERTS POC
`
`PMGR ACTIVATES POWER AND CLOCKS
`
`310
`
`MCBM ASSERTS SUSACK1
`
`312
`
`314
`
`PMGR DE-ASSERTS POC AND ALL NON-PROCESSOR RESETS
`
`PMGR DE-ASSERTS SUSREQ1 AND SUSREQ2
`
`316
`
`MCBM AND IOIC BEGIN INTERNAL WAKEUP SEQUENCES
`
`318
`
`320
` -SUSACK1
`
`
`AND SUSACK2
`DE-ASSERTED
`?
`
`YES
`
`NO
`
`FIG. 3A
`
`6
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 6 of 22
`
`US 6,711,691 B1
`
`PMGR DE-ASSERT RESETS FOR PROCESSORS
`
`324
`
`PROCESSORS EXECUTE RESET VECTORS
`
`326
`
`GOMPLETE WARM OR COLD BOOT SEQUENCE
`
`328
`
`MAP ALL INTERRUPTSTO LAST
`ACTIVE PROCESSOR
`
`PMGR DE-ASSERTS SREQ
`
`330
`
`332
`
`ENABLE INTERRUPTS ON ALL PROCESSORS
`
`334
`
`FIG. 3B
`
`7
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 7 of 22
`
`US 6,711,691 B1
`
`
`
`
`
`
`
`
`(LAST ACTIVE PROCESSORIS IN A RUN MODE)
`
`SAVE STATE OF ALL BUT LAST
`ACTIVE PROCESSOR
`
`FLUSH AND THEN DISABLE CACHES
`ON ALL BUT LAST ACTIVE PROCESSOR
`
`SET SLEEPBITS IN ALL BUT
`LAST ACTIVE PROCESSOR
`
`336
`
`338
`
`340
`
`ALL BUT LAST ACTIVE PROCESSOR ASSERT
`THEIR QREQs
`
`342
`
`MCBM ASSERTS ALL QACKs
`
`344
`
`346
`
`ALL BUT LAST ACTIVE PROCESSOR ENTERS SLEEP MODE
`
`8
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 8 of 22
`
`US 6,711,691 B1
`
`
`
`
`ACTIVATION PROCESSING
`
`FOR ALL BUT LAST ACTIVE
`PROCESSOR
`
`
`MAP INTERRUPTS TO ALL PROCESSORS
`
`402
`
`ALL BUT LAST ACTIVE PROCESSOR RECEIVE
`INTERRUPTS AND THEN CLEAR SLEEP
`BITS AND DE-ASSERT QREQs
`
`ACTIVE PROCESSOR
`
`ENABLE CACHESON ALL BUT LAST
`
`
`
`ALL BUT LAST ACTIVE PROCESSOR
`ENTER RUN MODE(LAST ACTIVE
`PROCESSOR REMAINS IN RUN MODE)
`
`
`
`
`404
`
`406
`
`408
`
`FIG. 4
`
`9
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 9 of 22
`
`US 6,711,691 B1
`
`
`
`
`SHUTDOWN PROCESSING
`
`FOR EXTRA PROCESSORS
`
`l 500
`
`502
`
`504
`
`506
`
`508
`
`510
`
`511
`
`514
`
`MAP ALL INTERRUPTS TO LAST ACTIVE
`PROCESSOR
`
`SAVE STATE OF ALL BUT LAST ACTIVE
`PROCESSOR
`
`FLUSH AND THEN DISABLE CACHES ON
`ALL BUT LAST ACTIVE PROCESSOR
`
`SET SLEEP BIT IN ALL BUT LAST ACTIVE
`PROCESSOR
`
`ALL BUT LAST ACTIVE PROCESSOR ASSERT
`THEIR QREQs
`
`MCBMASSERTSALL QACKs
`
`
`
`512
`
`ALL
`
`QACKs
`ASSERTED
`
`
`
`?
`
`YES
`
`NO
`
`ALL BUT LAST ACTIVE PROCESSOR ENTERS
`SLEEP MODE(LAST ACTIVE PROCESSOR
`
`‘REMAINS IN RUN MODE)
`
`FIG. 5
`
`10
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 10 of 22
`
`US 6,711,691 B1
`
`l 600
`
` NAP SINGLE INTERRUPT
`
`PROCESSING
`
`LAST ACTIVE PROCESSOR RECEIVES
`INTERRUPT WHICH CLEARS NAP BIT
`
`602
`
`LAST ACTIVE PROCESSOR DE-ASSERTSITS QREQ
`
`604
`
`MCBM DE-ASSERTS QACK
`
`LAST ACTIVE PROCESSOR ENTERS RUN MODE
`(OTHER PROCESSORS REMAIN IN SLEEP MODE)
`
`LAST ACTIVE PROCESSOR
`PROCESSES INTERRUPT
`
`606
`
`608
`
`610
`
`FIG. 6
`
`11
`
`11
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 11 of 22
`
`US 6,711,691 B1
`
`
`
`
`NAP PROCESSING
`
`FOR LAST ACTIVE
`~ PROCESSOR
`
`
`[ 700
`
`SET NAPBIT IN LAST ACTIVE PROCESSOR
`
`702
`
`LAST ACTIVE PROCESSORASSERTS ITS QREQ
`
`704
`
`706
`
`NO
`
`
` QACK
`ASSERTED
`
`
`
`FOR LAST
`ACTIVE
`
`PROCESSOR
`?
`
`YES
`
`708
`
`LAST ACTIVE PROCESSOR ENTERS NAP MODE
`(ALL OTHER PROCESSORS REMAININ SLEEP
`
`MODE)
`
`FIG. 7
`
`12
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 12 of 22
`
`US 6,711,691 B1
`
`Lo800
`
`
`IDLE PROCESSING
`
`
`FOR LAST PROCESSOR
`
`SET IDLE BIT INMCBM
`
`802
`
`SET IDLE COMMAND TO PMGR
`
`804
`
`PMGR ASSERTS SUSREQ1, SUSREQ2 & SREQ
`
`806
`
`FLUSH AND THEN DISABLE CACHES ON
`LAST ACTIVE PROCESSOR
`
`808
`
`SET SLEEPBIT IN LAST ACTIVE PROCESSOR
`
`812
`
`LAST ACTIVE PROCESSOR ASSERTS
`QREQ AND ENTERS DOZE MODE
`
`814
`
`816
` QACK
`
`ASSERTED
`
`
` NO
`TO LAST
`ACTIVE
`
`PROCESSOR
` 818
`?
`
`YES
`
`LAST ACTIVE PROCESSOR ENTERS SLEEP MODE
`(ALL OTHER PROCESSORS ALREADY SLEEPING)
`
`(c)
`
`FIG. 8A
`
`13
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 13 of 22
`
`US 6,711,691 B1
`
`
`
`AND SUSACK2
`ASSERTED
`?
`
` SUSACK1
`
`14
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 14 of 22
`
`US 6,711,691 B1
`
`
`
` ACTIVATION PROCESSING
`
`FOR LAST PROCESSOR
`
`PMGR STARTS PROCESSOR CLOCKS
`
`PMGR STARTS PROCESSORPLLs
`
`[ 900
`
`902
`
`904
`
`PMGR DE-ASSERTS SUSREQ1 AND SUSREQ2
`
`906
`
`MCBM CLEARSITS IDLE BIT
`
`908
`
`910
`
`SUSACK1
`AND SUSACK2
`DE-ASSERTED
`NO
`
`
`
`
`YES
`
`PMGR DE-ASSERTS SREQ
`
`912
`
`LAST ACTIVE PROCESSOR RECEIVES INTERRUPT,
`CLEARS SLEEPBIT IN LAST ACTIVE PROCESSOR,[~~914
`AND DE-ASSERTSITS QREQ
`
`
`
`MCBM DE-ASSERTS ALL QACKs
`
`ENABLE CACHES ON LAST ACTIVE PROCESSOR
`
`LAST ACTIVE PROCESSOR ENTERS RUN MODE
`(OTHER PROCESSORS REMAIN IN SLEEP MODE)
`
`916
`
`918
`
`920
`
`CEND)
`
`FIG. 9
`
`15
`
`15
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 15 of 22
`
`US 6,711,691 B1
`
` SHUTDOWN PROCESSING
`
`FOR LAST PROCESSOR
`
`l 7000
`
`SET SLEEP BIT IN MCBM
`
`1002
`
`SEND SLEEP OR OFF COMMAND TO PMGR
`
`1004
`
`PMGR ASSERTS SUSREQ1, SUSREQ2 & SREQ
`
`1006
`
`SAVE STATE OF LAST ACTIVE PROCESSOR
`
`1008
`
`FLUSH AND THEN DISABLE CACHES ON
`LAST ACTIVE PROCESSOR
`
`1010
`
`SET SLEEPBIT IN LAST ACTIVE PROCESSOR
`
`1012
`
`LAST ACTIVE PROCESSOR ASSERTS
`ITS QREQ
`
`4014
`
`1016
`
`
`ALL
`
`
`QACKs
`
`ASSERTED
`?
`
`YES
`
`NO
`
`LAST ACTIVE PROCESSOR ENTERS SLEEP MODE
`(ALL OTHER PROCESSORSALREADY SLEEPING)
`
`1018
`
`FIG. 10A
`
`16
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 16 of 22
`
`US 6,711,691 B1
`
`MCBM AND IOIC BEGIN INTERNAL SHUTDOWN
`SEQUENCE
`
` 1020
`
`
`SUSACK1
`AND SUSACK2
`ASSERTED
`?
`
`
`
`
`
`PMGR STOPS ALL CLOCKS,
`ASSERTS PROCESSOR RESETS,
`AND REMOVES POWER TO
`PROCESSORS
`
`
`
`1024
`
`
`
`
`
` OFF
`
`
`REQUESTED
`?
`
`ASSERT ALL RESETS
`AND REMOVE ALL POWER
`
`FIG. 10B
`
`17
`
`

`

`U.S. Patent
`
`Mar. 23,2004
`
`Sheet 17 of 22
`
`US 6,711,691 B1
`
`WAKEUP PROCESSING
`
`FOR LAST PROCESSOR
`
`1100
`
`[
`
`4402
`
`1104
`
`1106
`
`4108
`
`PMGR TURNS-ON POWER TO PROCESSORS
`AND RESTARTS CLOCKS
`
`START PROCESSORPLLs, THEN DE-ASSERT
`SUSREQ1 AND SUSREQ2
`
`MCBM CLEARSITS SLEEPBIT
`
`MCBM AND IOIC BEGIN INTERNAL
`WAKEUP SEQUENCE
`
`1110
`
`
`
`SUSACK4
`AND SUSACK2
`
`
`NO
`DE-ASSERTED
`
`
`
`
`YES
`
`DE-ASSERT RESETS FOR PROCESSORS
`
`1112
`
`PROCESSORS EXECUTE RESET VECTORS
`
`1114
`
`RESTORE STATE TO ALL PROCESSORS
`
`1116
`
`ENABLE INTERRUPTS ON ALL PROCESSORS
`
`1120
`
`FIG. 11
`
`18
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 18 of 22
`
`US 6,711,691 B1
`
`
`
`
`SNOOP PROCESSING
`
`
`l 1200
`
`MCBM DE-ASSERTS ALL QACKs
`
`1202
`
`1204
`
`ANY NAPPING PROCESSORS ENTER DOZE MODE
`(SLEEPING PROCESSORS REMAIN SLEEPING)
`
`SNOOP OPERATION COMPLETED
`
`1206
`
`
`
`YES
`
`ALL DOZING PROCESSORSRE-ENTER
`NAP MODE
`
`1210
`
`Cnn
`
`FIG. 12
`
`19
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 19 of 22
`
`US 6,711,691 B1
`
`NAP PROCESSING
`
`1300
`
`‘4
`
`
`
`YES
`
`1312
`
`ANY PROCESSOR IN DOZE MODE ENTERS NAP MODE
`
`FIG. 13A
`
`20
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 20 of 22
`
`US 6,711,691 B1
`
`INTERRUPT
`
`PROCESSING
`
`1326
`
`‘4
`
`ASSOCIATED PROCESSOR CLEARSITS NAPBIT
`
`1328
`
`ASSOCIATED PROCESSOR DE-ASSERTSITS QREQ[-~1330
`
`MCBM DE-ASSERTSALL QACKs
`
`1332
`
`
`
`
`
`ASSOCIATED PROCESSOR ENTERS RUN MODE
`(ANY OTHER PROCESSORS THAT WERE
`NAPPING ENTER DOZE MODE, AND SLEEPING
`PROCESSORS REMAIN SLEEPING)
`
`1334
`
`ASSOCIATED PROCESSOR HANDLES THE
`INTERRUPT
`
`1336
`
`FIG. 13B
`
`21
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 21 of 22
`
`US 6,711,691 B1
`
`l 1400
`
`COMPLETE ANY PREVIOUSLY QUEUED
`TRANSACTIONS
`
`1402
`
`TAKE CONTROL ALL BUSES
`
`1404
`
`PLACE DRAM INTO SELF-REFRESH MODE
`
`1406
`
`DRIVE ADDRESS AND DATA LINES LOW
`
`1408
`
`DE-ASSERT AND TRI-STATE
`CONTROLLINES
`
`4410
`
`BYPASS AND SHUT DOWNINTERNALPLLs
`
`1412
`
`ASSERT SUSACK1
`
`1414
`
`FIG. 14A
`
`22
`
`

`

`U.S. Patent
`
`Mar. 23, 2004
`
`Sheet 22 of 22
`
`US 6,711,691 B1
`
`l 1450
`
`CLEAR SLEEP AND IDLE BITS
`
`1452
`
`ENABLE AND SWITCH TO
`INTERNAL PLL
`
`4454
`
`RETURN DRAM TO NORMAL MODE
`
`1456
`
`STOP DRIVING DATA & ADDRESS LINES LOW,
`STOP TRI-STATING CONTROLLINES, AND DE-
`
`ASSERT CONTROL LINES
`
`1458
`
`RELEASE CONTROL OF BUS
`
`1460
`
`DE-ASSERT SUSACK1
`
`1462
`
`FIG. 14B
`
`23
`
`

`

`US 6,711,691 B1
`
`1
`POWER MANAGEMENT FOR COMPUTER
`SYSTEMS
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`This application claims the benefit of U.S. Provisional
`Application No. 60/133,918,
`filed May 13, 1999, and
`entitled “POWER MANAGEMENT FOR COMPUTER
`SYSTEMS”, the content of which is hereby incorporated by
`reference.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to computer systems and,
`more particularly, to power management for computer sys-
`tems.
`
`2. Description of the Related Art
`Computer systems include electrical components that
`consume power whenis active. These electrical components
`include processors, controllers, buses, and various sub-
`systems. Limited power management has been performed to
`reduce the power consumption of these electrical compo-
`nents. Often, with portable computers where power con-
`sumption is a major concern, the portable computers can
`enter a sleep mode in which the processor is slowed or
`stopped and in which the controllers, buses and various
`subsystems are also shutdown. The sleep mode thus pro-
`vides a state which the portable computeris able to enter to
`conserve power when processing resources are not needed.
`In the sleep mode, processors, controllers and subsystems
`are able to be shutdown. Typically little power management
`is offered with desktop computers.
`Although there are various disadvantages of conventional
`power management, one disadvantage is that
`the power
`managementis primarily processor power management and
`not system level. As a result, the overall power management
`is not very efficient
`in reducing power consumption.
`Typically, many electrical components that consumesignifi-
`cant amounts of power are either not power managed or
`crudely power managedto have only an onstate (i.¢., active)
`and anoff state (i.e., shutdown). Also, in the case of desktop
`computers, conventional power management has been even
`less efficient.
`
`Still further, conventional power management for multi-
`processor computer systems has not been efficient. Hence,
`large amounts of power are consumed by these multi-
`processor computer systems even whenthereis no activity.
`Thus, there is a need for improved power managementin
`computer systems.
`
`SUMMARYOF THE INVENTION
`
`Broadly speaking, the invention relates to power manage-
`ment for computer systems having one or more processors.
`Oneaspect of the invention pertains to providing hierarchi-
`cal power management. The hierarchical nature of the power
`management provided by the invention has variouslevels of
`power management such that power consumption of the
`computer system is dependent upon the amount of work
`placed on the processing resources of the computer system.
`Anotheraspect lie of the invention pertains to deterministic
`handshaking provided between a power manager and one or
`more controller units. The deterministic handshaking pro-
`vides for more reliable and controllable transitions between
`power management states which have associated power
`managementtaking place in the controller units. The inven-
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`tion is suitable for use with a single-processor computer
`system or a multi processor computer system.
`The invention can be implemented in numerous ways,
`including as a computer system, an apparatus, a method, and
`a computer readable medium. Several embodiments of the
`invention are discussed below.
`
`As a power management method for a multi-processor
`computer system having a plurality of processors, one
`embodimentof the invention includes: monitoring workload
`on the multi-processor computer system; and, placing the
`multi-processor computer system in one of a plurality of
`predetermined power managementstates based on the work-
`load.
`
`As a method for managing power consumption of a
`multi-processor computer system having a plurality of
`processors, one embodiment of the invention includes:
`determining a processing workload for the multi-processor
`computer system; awakening one or more of the processors
`from an inactive mode to an active mode whenthe process-
`ing workload is heavy; and transitioning one or more of the
`processors from the active modeto the inactive mode when
`the processing workloadis light. The power consumption of
`the multi-processor computer system is managed such that
`those of the processors that are not needed to process the
`processing workload are transitioned into the inactive mode
`to conserve power, yet one or more of the processors can
`awaken from the inactive mode to the active mode to
`provide additional processing capabilities as needed to
`handle the processing workload.
`As a method for providing deterministic state changes
`within a computer system having a plurality of processors,
`a bus controller and a power manager, one embodiment of
`the invention includes: determining when a lower power
`state of the multi-processor computer system should be
`entered to reduce power consumption, and entering the
`lower power state. The lower power state is entered by
`performing the following operations: receiving at the bus
`controller a state change request from the power manager;
`initiating a state change sequenceat the bus controller upon
`receiving the state change request; and notifying the power
`manager when the bus controller has completed the state
`change sequence.
`As a computer system, one embodimentof the invention
`includes: at
`least one processor,
`the processor executes
`operations in accordance with a processor clock; a bus
`controller operatively connected to the processor, the bus
`controller controls bus activity on a bus; and a power
`manager operatively connected to the processor and the bus
`controller, the power manager controls the processor clock
`and shutdownof the bus controller, and the power manager
`provides a first handshaking between the power manager
`and the bus controller to provide deterministic mode
`changes for the computer system so as to manage power
`consumption.
`The advantages of the invention are numerous. Different
`embodiments or implementations may have one or more of
`the following advantages. One advantage of the invention is
`that it offers improved power managementthat is obtained
`by a layered approach. Another advantage of the invention
`is that
`the system power management provided by the
`invention includes power management(i.e., shutdown) for
`not only processors but also related control circuitry (e.g.,
`bus controllers, I/O controllers, memory controllers, inter-
`rupt controllers). Still another advantage of the invention is
`that deterministic control over power managementof control
`circuitry provides proper sequencing of shutdown opera-
`24
`
`24
`
`

`

`US 6,711,691 B1
`
`3
`tions. Yet another advantage of the invention is that more
`aggressive power management of a computer system is
`provided, whether for a single processor or a multi-processor
`system.
`Other aspects and advantages of the invention will
`become apparent from the following detailed description,
`taken in conjunction with the accompanying drawings,
`illustrating by way of example the principles of the inven-
`tion.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention will be readily understood by the
`following detailed description in conjunction with the
`accompanying drawings, wherein like reference numerals
`designate like structural elements, and in which:
`FIG. 1A is a flow diagram of power managementpro-
`cessing according to a basic embodiment of the invention;
`FIG. 1B is a flow diagram of processor deactivation
`processing according to one embodiment of the invention;
`FIG. 1C is a block diagram of a multiple processor
`computer system according to one embodimentofthe inven-
`tion;
`FIG. 2 is a flow diagram of power management process-
`ing according to one embodimentof the invention;
`FIGS. 3A-3C are flow diagrams of start processing
`according to one embodiment of the invention;
`FIG. 4 is a flow diagram of activation processing for all
`but the last active processor according to one embodiment of
`the invention;
`FIG. 5 is a flow diagram of the shutdown processing for
`extra processors according to one embodimentof the inven-
`tion;
`FIG. 6 is a flow diagram of nap single interrupt processing
`according to one embodiment of the invention;
`FIG. 7 is a flow diagram of nap processing for the last
`active processor according to one embodimentof the inven-
`tion;
`FIGS. 8A and 8Bare flow diagramsofidle processing for
`the last active processor according to one embodimentof the
`invention;
`FIG. 9 is a flow diagram of activation processing for the
`last active processor according to one embodiment of the
`invention;
`FIGS. 10A and 10B are flow diagrams of shutdown
`processing for the last active processor according to one
`embodiment of the invention;
`FIG. 11 is a flow diagram of wakeup processing for the
`last active processor according to one embodiment of the
`invention;
`FIG. 12 is a flow diagram of snoop processing according
`to one embodimentof the invention;
`FIG. 13A illustrates a flow diagram of nap processing
`according to one embodiment of the invention;
`FIG. 13B illustrates a flow diagram of interrupt process-
`ing according to one embodimentof the invention;
`FIG. 14A is a flow diagram of memory controller/bus
`manager (MCBM)sleep shutdown processing according to
`one embodiment of the invention; and
`FIG. 14B is a flow diagram of MCBM sleep wakeup
`processing according to one embodiment of the invention.
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`The invention relates to power management for computer
`systems having one or more processors. One aspect of the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`invention pertains to providing hierarchical power manage-
`ment. The hierarchical nature of the power management
`provided by the invention has various levels of power
`managementsuch that power consumption of the computer
`system is dependent upon the amount of work placed on the
`processing resources of the computer system. Another
`aspect of the invention pertains to deterministic handshaking
`provided between a power manager and one or more con-
`troller units. The deterministic handshaking provides for
`more reliable and controllable transitions between power
`management states which have associated power manage-
`ment taking place in the controller units. The invention is
`suitable for use with a single-processor computer system or
`a multi-processor computer system.
`Embodiments of the invention are discussed below with
`reference to FIGS. 1A-14B. However, those skilled in the
`art will readily appreciate that the detailed description given
`herein with respect
`to these figures is for explanatory
`purposes as the invention extends beyond these limited
`embodiments.
`
`FIG. 1A is a flow diagram of power management pro-
`cessing 10 according to a basic embodiment of the inven-
`tion. The power managementprocessing 10 is performed by
`a multi-processor computer system to reduce the power
`consumption of any processing resources whenever those
`resources are not needed.
`
`The power management processing 10 monitors 12 the
`workload of the computer system or the processors within
`the computer system. The workload reflects how busy the
`computer system or the processors therein are in processing
`useful tasks. Once the workload is obtained,it is determined
`14 whether the workload is heavy.
`Whenit is determined 14 that the workload is heavy, then
`one or more processors of the computer system are activated
`16. Here,
`the heavy workload indicates that additional
`processing resources are needed. By activating 16 one or
`more of the processors (that were previously inactive), the
`computer system obtains the additional processing resources
`to help process the heavy workload.
`Assume a simplistic example in which the workload is
`1000 units of work that is waiting to be processed by the
`computer system. If only one processor is active, the work-
`load of the one processor can be considered 1000 units. If a
`heavy workload is deemed anything over 600 units, then the
`power managementprocessing 10 will operate to activate
`one or more additional processors. If one additional proces-
`sor were to be activated 16, then the average workload for
`each of the two activated processors would drop to 500
`units.
`
`On the other hand, when it is determined 14 that the
`monitored workload is not heavy,
`it
`is determined 18
`whether the monitored workload is light. Whenit is deter-
`mined 18 that the monitored workloadis light, then one or
`more processors of the computer system are deactivated 20.
`By deactivating 20 one or more of the processors,
`the
`computer system conserves powerby placing the computer
`system, or its processors, into a lower poweredstate.
`Following blocks 16 and 20, as well as following the
`decision block 18 when the workloadis not light, the power
`management processing 10 is complete and ends. However,
`typically,
`the power management processing 10 continu-
`ously repeats such that
`the power management for the
`computer system is ongoing and dynamically performed.
`According to the invention, the deactivation 20 of one or
`more of the processors can be performed in a wide variety
`of ways. In particular, the deactivation 20 can be performed
`25
`
`25
`
`

`

`US 6,711,691 B1
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`6
`5
`placed in the first low powerstate. Still further, if further
`over a series of low power states into which the computer
`power reduction is further desired,
`then the last active
`system can enter (or low power modes into which its
`processor can by placed in the second low power mode.
`processors can enter) to conserve power. The deactivation 20
`Typically,
`the second power mode would placed the last
`can also be considered to be performed over a series of
`active processor in even a greater power saving state than
`layers such that different layers offer different tradeoffs of
`provided by the first low power mode. Additionally, since
`processing resources verses powerefficiency.
`the second low power mode essentially deactivates the last
`FIG. 1B is a flow diagram of processor deactivation
`active processor, various other electrical components or
`processing 30 according to one embodiment of the inven-
`units within the multi-processor computer system can also
`tion. The processor deactivation processing 30 is,
`for
`be placed in a low-power state (or shut-down) to further
`example, suitable for use as the deactivation 20 of the one
`enhance the power reductions. For example, the electrical
`or more processors such as shown in FIG. 1A.
`components or units can include controllers (e.g., controller
`The processor deactivation processing 30 initially deter-
`chips) or unneeded subsystems. Often, some subsystem can
`mines whether there is more than one active processor.
`be shutdown in other earlier layers.
`Whenit is determined 32 that there is more than one active
`A third low power mode could also be provided by the
`processor, the processor deactivation processing 30 places
`processor deactivation processing 30. The third low power
`34 one or more (but not all) of the processors in a sleep
`mode would offer even greater power reductions than pro-
`mode. The number of the processors being placed in the
`videdin the first and second low power modes. For example,
`sleep mode is based on the workload. For example, in a
`in the third low power mode, the last processor could be
`computer system having five processors, the processor deac-
`placed in the sleep modeand,thus, with all processors in the
`tivation processing 30 could shutdown up to four of the
`sleep mode, the computer system would itself be in a sleep
`processors by placing them in the sleep mode when the
`mode. More than three low power modes could also be
`workloadis light.
`provided. The various low power modes can include, for
`On the other hand, whenit is determined 32 that there is
`example, nap, doze or sleep modes for a processor as well
`only one active processor, then the processor deactivation
`as also remove clocks and power from a processor and/or
`processing is performed for the last processor in a layered
`other electrical components in certain of the various low
`fashion. In particular, it is initially determined 36 whether
`powerstates.
`the last processor is in a run mode. The run mode is a mode
`Although the invention is described herein largely in
`in which the last active processor is active such that
`it
`terms of a multi-processor computer system, the invention is
`processes instructions. When it is determined that the last
`also applicable to single processor computer systems. As an
`processoris in the run mode, then the last processor is placed
`example, blocks 3444 of FIG. 1B in effect pertain to power
`38 inafirst low power mode. In this embodiment, the first
`management for single processor computer systems.
`low power modeis a mode in which the processor conserves
`FIG. 1C is a block diagram of a multiple processor
`some powerbut has a loss of performance as compared to
`computer system 100 according to one embodimentof the
`the run mode.
`invention. The multiple processor computer system 100
`includes a plurality of processors, including processor A
`102, processor B 104, processor C 106 and processor D 108.
`Eachof the processors 102-108 operates to execute instruc-
`tions under the control of an operating system (not shown).
`The multiple processor computer system 100 also includes
`a power manager 110 that provides power management
`functions for the multiple processor computer system 100.
`In general,
`the power manager 110, provides soft resets
`(SRST), hard resets (HRST), system managementinterrupts
`(SMIs), and configuration control signals for phase lock
`loops (PLLs). In one embodiment, the power manager is an
`embedded controller or processor. The multiple processor
`computer system 100 also includes a memorycontroller/bus
`manager (MCBM) 112, a dynamic random access memory
`(DRAM)114, an I/O-interrupt controllerOIC) 116, and an
`operating system 11. Although not shown in FIG. 1C, the
`multiple processor computer system 100 would also include
`other components such as Read Only Memory (ROM),
`system bus, display controller, etc.
`The processors 102-108 operate to perform program
`instructions under the control of operating system 118. For
`processor mode control, processors 102-108 communicate
`with the memory controller/ous manager 112 using a
`hardware-handshake. The hardware-handshake controls
`
`the last
`is determined 36 that
`Alternatively, when it
`processoris not in the run mode, it is determined 40 whether
`the last processor is in the first low power mode. Whenitis
`determined 40 that the last processor is already in thefirst
`lower power mode, then the processor deactivation process-
`ing 30 can perform additional operations to provide greater
`reductions in power consumption. Specifically, the last pro-
`cessor is placed 42 in a second low power mode. The second
`low power modeis a modethat offers less power consump-
`tion (as well as less performance) than does the first low
`power mode. Next, any unneeded electrical components or
`units of the computer system are shut down 44. On the other
`hand, whenit is determined 40 that the last processoris not
`in the first low power mode, it is assumed that the last
`processoris already in the second low power modesuch that
`additional processing by the processor deactivation process-
`ing 30 is not available and, thus, the processor deactivation
`processing 30 is complete and ends. In addition, following
`blocks 34, 38 and 44, the processor deactivation processing
`30 is complete and ends.
`The processor deactivation processing 30 can be consid-
`ered to have multiple hierarchical levels. For example, with
`respect to a multi-processor computer system, the processor
`deactivation processing 30 offers three layer of power man-
`agement. A first layer is provided by block 34 such that the
`number of processors that are active (and not sleeping) is
`determined based on workload. Within the first layer there
`are various different levels of power management based
`largely on the numberof processors activated. In any case,
`once only one of the processorsis active, than a second layer
`can be entered for further power reduction. In the second
`layer provided by block 38,
`the last active processor is
`
`45
`
`50
`
`55
`
`60
`
`65
`
`whenthe processors 102-108 are able to enter a lower power
`state. The processor A 102 supplies a quiescent request
`(QREQ-A) to the memory controller/ous manager 112.
`Likewise, the processor B 104, the processor C 106, and the
`processor D 108 respectively supply quiescent signals
`(QREQ-B, QREQ-C, and QREQ-D)
`to the memory
`controller/bus manager 112. When the memorycontroller/
`bus manager 112 determines that all of the quiescent
`26
`
`26
`
`

`