US 6,711,691 B1
`(16) Patent No.:
`(12) United States Patent
`
`Howard et al. Mar. 23, 2004 (45) Date of Patent:
`
`
`USOO6711691B1
`
`(54) POWER MANAGEMENT FOR COMPUTER
`SYSTEMS
`
`(75)
`
`Inventors: Brian D. Howard, Portola Valley, CA
`.
`-
`(CUAS)(’U1§[)IChael F‘ Culbert, san Jose”
`.
`.
`(73) Ass1gnee: Apple Computer, Inc., Cupertino, CA
`(US)
`.
`.
`.
`.
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U’S’C’ 154(b) by 0 days.
`
`.
`( * ) Notice:
`
`(21) Appl. No.: 09/567,201
`
`(22)
`
`Filed:
`
`May 8, 2000
`
`(60)
`
`Related US, Application Data
`Provisional application No. 60/133,918, filed on May 13,
`1999.
`
`Int. Cl.7 .................................................. G06F 1/32
`(51)
`(52) US. Cl.
`........................ 713/300; 713/320; 713/322
`(58) Field of Search ................................. 713/300 320
`713/322 323’ 324;
`
`(56)
`
`References Cited
`
`............ 713/322
`
`8/1998 Fung
`5,799,198 A
`9/1998 Carmean et al.
`5,809,314 A *
`9/1998 Broedner et al.
`5,812,796 A
`3:33?) Eunkg
`t
`1
`231333933: 2
`ar y e a .
`,
`,
`9/2000 Miranda et al.
`6,119,194 A
`6,141,762 A * 10/2000 Nicol et al.
`................. 713/300
`6,230,277 B1
`5/2001 Nakaoka et al.
`6,272,644 B1
`8/2001 Urade et al.
`6,308,278 B1
`10/2001 Khoulietal.
`6,460,143 B1 * 10/2002 Howard etal.
`
`............. 713/323
`
`OTHER PUBLICATIONS
`“Universal Serial Bus Specification”, Revision 1.0, Jan. 15,
`1996.
`
`* cited by examiner
`
`Primary Examiner—Dennis M. Butler
`(74) Attorney, Agent, or Firm—Beyer Weaver & Thomas
`LLP
`
`(57)
`
`ABSTRACT
`
`Power management approaches for computer Systems haV-
`ing one or more processors are disclosed. One power man-
`agement approach provides hierarchical power manage-
`ment. The hierarchical nature of the power management
`provided by the invention has various levels of power
`management such that power consumption of the computer
`system is dependent upon the amount of work placed on the
`processing resources of the computer system. Another
`power management approach pertains to deterministic hand-
`shaking provided between a power manager and one or more
`controller units. The deterministic handshaking provides for
`more reliable and controllable transitions between power
`management states Which have associated power manage-
`ment taking place in the controller units. The power man-
`-
`-
`-
`_
`agemem approaehes are sultable for. use web a smgle
`processor computer system or a multi-processor computer
`SYStem
`
`18 Claims, 22 Drawing Sheets
`
`U~S- PATENT DOCUMENTS
`5 167 024 A
`11/1992 Smith et a1.
`5:239:652 A
`8/1993 Seibert et a1.
`5,254,928 A
`10/1993 Young et a1.
`5,396,635 A
`3/1995 Fung
`5,483,656 A
`1/1996 Oprescu et a1.
`5,557,777 A
`9/1996 Clflbeft
`3,238,221 2
`12/1337 21011:)0‘1611: H et a1~
`,
`,
`u er
`5,632,037 A *
`5/1997 Maher et al.
`............... 713/322
`5 708 816 A
`1/1998 Culbert
`5,710,929 A
`1/1998 Fung
`5,724,591 A *
`3/1998 Hara et al. .................. 713/322
`5,737,615 A *
`4/1998 Tetrick ....................... 713/324
`
`OPERATING
`
`
`SYSTEM m
`f
`
`
`_,
`INT-A
`PROCESSOR
`SRST-A
`
`5W“
`:
`A
`gig:
`PLLC A
`m ‘__
`_
`
`
`
`
`
`
`
`P,
`‘
`|NT—B
`
`
`
`SRST‘B > PROCESSOR
`_
`
`
`SMIVB
`:
`B lawful/:51:
`PLLO—B
`
`
`,_,.
`‘
`1NT-C
`
`
`
`SMI—C
`.,
`QACK-C
`SRST-C , PROCESSOR “ch
`\NT-D
`
`
`
`-2
`>
`105 ‘—
`my?”
`
`
`
`
`
`D
`ONTRCLLERI
`SRSTvDr: PROCESSOR ‘
`_: MEMOR‘I
`SMl-D
`>
`BUS
`
`
`
`_
`103 -Q—>«;.§E%g MANAGER
`PLLC—D
`HRST
`1.1.2
`
`SUSREQ1
`SUSACK1
`POWER
`
`
`SRST
`MANAGER
`
`SREQ
`
`
`mm
`SUSREQZ
`”(3- INTERRUPTCONTROLLER
`
`
`SUSACK2——>
`L‘Lfi
`
`10°
`
`
`
`
`
`use
`
`
`
`
`
`m
`
`A
`
`1
`
`APPLE 1021
`
`APPLE 1021
`
`1
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 1 0122
`
`US 6,711,691 B1
`
`
`
`MONITOR WORKLOAD
`
`
`
`[10
`
`12
`
`
`
` IS
`WORKLOAD
`
`HEAW
`1;
`
`ACTIVATE ONE OR MORE
`
`PROCESSORS
`
`16
`
`14
`
`
`
`
`IS
`WORKLOAD
`
`LIGHT
`
`?
`
`
`20
`
`DEACTIVATE ONE OR
`MORE PROCESSORS
`
`2
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 2 0f 22
`
`US 6,711,691 B1
`
`[30
`
`>1
`
`
`
`ACTIVE
`
`
`PROCESSORS
`
`?
`
`PLACE ONE OR MORE (BUT
`
`NOT ALL) PROCESSORS IN
`SLEEP MODE BASED ON
`
`WORKLOAD
`
`
`36
`
`
`IS
`LAST
`
`
`PROCESSOR
`IN RUN
`
`MODE
`
`7
`
`
`IN FIRST LOW
`
`POWER
`
`
`MODE
`
`
`PLACE LAST PROCESSOR
`?
`IN FIRST LOW POWER
`
`MODE
`
`
`PLACE LAST PROCESSOR
`IN SECOND LOW POWER
`MODE
`
`
`
`
`3
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 3 0f 22
`
`US 6,711,691 B1
`
`OPERATING
`
`SYSTEM
`
`100
`
`[
`
` MEMORY
`ONTROLLER/
`
`BUS
`MANAGER
`
`
`.112
`
`
`POWER
`
`MANAGER
`
`
`13!! F1/0 - INTERRUPT CONTROLLER
`
`1.1.6
`
`
`
`FIG. 1C
`
`4
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 4 0f 22
`
`US 6,711,691 B1
`
`
`
`238
`
`214
`
`
`
`WAKEUP
`STARTUP PROCESSING
`
`
`PROCESSING
`
`
`FOR LAST ACTIVE
`
`SHUTDOWN
`
`
`PROCESSOR
`
`PROCESSING
`FOR LAST PROCESOR
`
`
`
`204
`
`224
`
` IDLE
`
`PROCESSING
`
`
`13:82:93ng
`
`
`
`210
`
`
`
`
`ACTIVATION PROCESSING
`FOR ALL BUT LAST
`ACTIVE PROCESSOR
`
`
`
`222
`
`
`
`NAP
`
`PROCESSING
`
`223
`FOR LAST
`
`ACTIVE
`
`n
`PROCESSOR
`230
`218
`
`
`
`
`
`
`SHUTDOWN PROCESSING
`FOR EXTRA PROCESSORS
`
`208
`
`‘
`
`ACTIVATION
`
`
`PROCESSING
`
`FOR LAST PROCESSOR
`
`
`NAP SINGLE
`
`INTERRUPT
`PROCESSING
`
`
`22°
`
`FIG. 2
`
`5
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 5 0f 22
`
`US 6,711,691 B1
`
`STARTUP PROCESStNG
`
`[300
`
`PMGR ASSERTS SUSREQZ, SREQ AND SUSREQ1
`
`302
`
`PMGR ASSERTS ALL RESETS (EXCEPT FCC)
`
`304
`
`306
`
`
`
`308
`
`
`
`PMGR ASSERTS POC
`
`PMGR ACTIVATES POWER AND CLOCKS
`
`310
`
`MCBM ASSERTS SUSACK1
`
`312
`
`_
`
`314
`
`PMGR DE-ASSERTS POC AND ALL NON-PROCESSOR RESETS
`
`PMGR DE-ASSERTS SUSREQ‘I AND SUSREQZ
`
`316
`
`MCBM AND 1010 BEGIN INTERNAL WAKEUP SEQUENCES
`
`313
`
`320
`
`
`
`
`
`
`AND SUSACKZ
`DE-ASSERTED
`
`
`
` 'SUSACK1
`
`
`
`
`N0
`
`FIG. 3A
`
`6
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 6 0f 22
`
`US 6,711,691 B1
`
`PMGR DE-ASSERT RESETS FOR PROCESSORS
`
`324
`
`PROCESSORS EXECUTE RESET VECTORS
`
`326
`
`COMPLETE WARM 0R COLD BOOT SEQUENCE
`
`328
`
`MAP ALL INTERRUPTS TO LAST
`
`ACTIVE PROCESSOR
`
`PMGR DE—ASSERTS SREQ
`
`330
`
`332
`
`ENABLE iNTERRUPTS ON ALL PROCESSORS
`
`334
`
`FIG. 3B
`
`7
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 7 0f 22
`
`US 6,711,691 B1
`
`
`
`
`
`
`
`
`
`
`
`SAVE STATE OF ALL BUT LAST
`
`ACTIVE PROCESSOR
`
`FLUSH AND THEN DISABLE CACHES
`ON ALL BUT LAST ACTIVE PROCESSOR
`
`SET SLEEP BITS IN ALI. BUT
`
`LAST ACTIVE PROCESSOR
`
`
`
`
`
`336
`
`338
`
`340
`
`ALL BUT LAST ACTIVE PROCESSOR ASSERT
`
`THEIR QREQS
`
`MCBM ASSERTS ALL QACKS
`
`
`
`342
`
`344
`
`346
`
`QACKS
`
`ASSERTED N0
`?
`
`
`
`
`
`
`
`
`ALL BUT LAST ACTIVE PROCESSOR ENTERS SLEEP MODE
`(LAST ACTIVE PROCESSOR IS IN A RUN MODE)
`
`8
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 8 0f 22
`
`US 6,711,691 B1
`
`
`ACTIVATION PROCESSING
`
`
`
`FOR ALL BUT LAST ACTIVE
`
`PROCESSOR
`
`
`MAP INTERRUPTS TO ALL PROCESSORS
`
`402
`
`404
`
`406
`
`408
`
`ALL BUT LAST ACTIVE PROCESSOR RECEIVE
`INTERRUPTS AND THEN CLEAR SLEEP
`
`BITS AND DE-ASSERT QREQs
`
`ACTIVE PROCESSOR
`
`ENABLE CACHES ON ALL BUT LAST
`
`ALL BUT LAST ACTIVE PROCESSOR
`
`
`
`ENTER RUN MODE (LAST ACTIVE
`PROCESSOR REMAINS IN RUN MODE)
`
`
`
`
`
`FIG. 4
`
`9
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 9 0f 22
`
`US 6,711,691 B1
`
`
`
`
`
`SHUTDOWN PROCESSING
`FOR EXTRA PROCESSORS
`
`
`[ 500
`
`502
`
`504
`
`506
`
`508
`
`510
`
`511
`
`MAP ALL INTERRU PTS TO LAST ACTIVE
`PROCESSOR
`
`SAVE STATE OF ALL BUT LAST ACTIVE
`PROCESSOR
`
`FLUSH AND THEN DISABLE CACHES ON
`ALL BUT LAST ACTIVE PROCESSOR
`
`SET SLEEP BIT IN ALL BUT LAST ACTIVE
`PROCESSOR
`
`ALL BUT LAST ACTIVE PROCESSOR ASSERT
`THEIR QREQs
`
`MCBM‘ ASSERTS ALL QACKS
`
`
`
`512
`
`
`ALL
`QACKS
`
`ASSERTED
`
`
`?
`
`
`YES
`
`NO
`
`ALL BUT LAST ACTIVE PROCESSOR ENTERS
`SLEEP MODE (LAST ACTIVE PROCESSOR
`
`REMAINS IN RUN MODE)
`
`514
`
`FIG. 5
`
`10
`
`10
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 10 0f 22
`
`US 6,711,691 B1
`
`f 600
`
` NAP SINGLE INTERRUPT
`
`PROCESSING
`
`LAST ACTIVE PROCESSOR RECEIVES
`INTERRUPT WHICH CLEARS NAP BIT
`
`602
`
`LAST ACTIVE PROCESSOR DE-ASSERTS ITS QREQ
`
`604
`
`MCBM DE-ASSERTS QACK
`
`LAST ACTIVE PROCESSOR ENTERS RUN MODE
`(OTHER PROCESSORS REMAIN IN SLEEP MODE)
`
`LAST ACTIVE PROCESSOR
`PROCESSES INTERRUPT
`
`605
`
`608
`
`610
`
`FIG. 6
`
`11
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 11 0122
`
`US 6,711,691 B1
`
`
`
`
`NAP PROCESSING
`
`
`‘ PROCESSOR
`
`
`FOR LAST ACTIVE
`
`[ 700
`
`SET NAP BIT IN LAST ACTIVE PROCESSOR
`
`702
`
`LAST ACTIVE PROCESSOR ASSERTS ITS QREQ
`
`704
`
`706
`
`
`
` QACK
`ASSERTED
`
`FOR LAST
`
`
`
`
`
`YES
`
`ACTIVE
`
`PROCESSOR
`
`7
`
`NO
`
`708
`
`LAST ACTIVE PROCESSOR ENTERS NAP MODE
`
`(ALL OTHER PROCESSORS REMAIN IN SLEEP
`
`MODE)
`
`FIG. 7
`
`12
`
`12
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 12 0f 22
`
`US 6,711,691 B1
`
`/ 800
`
`IDLE PROCESSING
`FOR LAST PROCESSOR
`
`
`SET IDLE BIT IN MCBM
`
`802
`
`SET IDLE COMMAND TO PMGR
`
`804
`
`PMGR ASSERTS SUSREQI. SUSREQZ & SREQ
`
`305
`
`FLUSH AND THEN DISABLE CACHES ON
`LAST ACTIVE PROCESSOR
`
`808
`
`SET SLEEP BIT IN LAST ACTIVE PROCESSOR
`
`312
`
`LAST ACTIVE PROCESSOR ASSERTS
`QREQ AND ENTERS DOZE MODE
`
`814
`
`816
`
`
`QACK
`
`
`ASSERTED
`
`TO LAST
`ACTIVE
`
`
`
`
`PROCESSOR
`
`?
`
`YES
`
`NO
`
`818
`
`LAST ACTIVE PROCESSOR ENTERS SLEEP MODE
`
`(ALL OTHER PROCESSORS ALREADY SLEEPING)
`
`0
`
`FIG. 8A
`
`13
`
`13
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 13 0f 22
`
`US 6,711,691 B1
`
`SUSACK1
`
`AND SUSACK2
`
`ASSERTED
`
`
`
`
`
`
`822
` 824
`
`?
`
`PMGR STOPS PROCESSOR PLLS
`
`PMGR STOPS PROCESSOR CLOCKS
`
`FIG. BB
`
`14
`
`14
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 14 0f 22
`
`US 6,711,691 B1
`
`
`
` ACTIVATION PROCESSXNG
`
`FOR LAST PROCESSOR
`
`[— 900
`
`PMGR STARTS PROCESSOR CLOCKS
`
`902
`
`PMGR STARTS PROCESSOR PLLs
`
`904
`
`PMGR DE-ASSERTS SUSREQ1 AND SUSREoz
`
`906
`
`MCBM CLEARS ITS IDLE BIT
`
`908
`
`910
`SUSACK1
`
`AND SUSACK2
`DE-ASSERTED
`NO
`
`
`
`YES
`
`PMGR DE-ASSERTS SREQ
`
`912
`
`LAST ACTIVE PROCESSOR RECEIVES INTERRUPT,
`CLEARS SLEEP BIT IN LAST ACTIVE PROCESSOR.
`AND DE—ASSERTS ETS QREQ
`
`
`
`914
`
`MCBM DE-ASSERTS ALL QACKS
`
`916
`
`ENABLE CACHES ON LAST ACTIVE PROCESSOR
`
`918
`
`LAST ACTIVE PROCESSOR ENTERS RUN MODE
`(OTHER PROCESSORS REMAIN IN SLEEP MODE)
`
`920
`
`@ FIG. 9
`
`15
`
`15
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 15 0f 22
`
`US 6,711,691 B1
`
` SHUTDOWN PROCESSING
`
`FOR LAST PROCESSOR
`
`[ 1000
`
`SET SLEEP BIT IN MCBM
`
`1002
`
`SEND SLEEP OR OFF COMMAND TO PMGR
`
`1004
`
`PMGR ASSERTS SUSREO1, SUSREoz & SREQ
`
`1006
`
`FLUSH AND THEN DISABLE CACHES ON
`LAST ACTIVE PROCESSOR
`
`1010
`
`SET SLEEP BIT IN LAST ACTIVE PROCESSOR
`
`1012
`
`LAST ACTIVE PROCESSOR ASSERTS
`ITS QREQ
`
`1014
`
`1016
`
`ALL
`
`-
`
`QACKs
`ASSERTED
`?
`
`N0
`
`
`YES
`
`LAST ACTIVE PROCESSOR ENTERS SLEEP MODE
`(ALL OTHER PROCESSORS ALREADY SLEEPING)
`
`1018
`
`FIG. 10A
`
`16
`
`16
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 16 0f 22
`
`US 6,711,691 B1
`
`1020
`
`MCBM AND IOIC BEGIN INTERNAL SHUTDOWN
`
`SEQUENCE
`
`SUSACK1
`
`
`AND SUSACK2
`
`
`ASSERTED
`?
`
`ASSERTS PROCESSOR RESETS.
`AND REMOVES POWER TO
`PROCESSORS
`
`
`
`1024
`
`
`
`
`
`
`PMGR STOPS ALL CLOCKS.
`
`
`
` OFF
`
`REQUESTED
`
`7
`
`ASSERT ALL RESETS
`AND REMOVE ALL POWER
`
`1028
`
`
`
`FIG. 1GB
`
`17
`
`17
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 17 0f 22
`
`US 6,711,691 B1
`
`WAKEUP PROCESS:NG
`
`FOR LAST PROCESSOR
`
`1100
`
`[
`
`PMGR TURNS-ON POWER TO PROCESSORS
`AND RESTARTS CLOCKS
`
`1102
`
`START PROCESSOR PLLs, THEN DE-ASSERT
`SUSREQ‘I AND SUSREQZ
`
`1104
`
`MCBM CLEARS TTS SLEEP BIT
`
`1106
`
`MCBM AND IOIC BEGIN INTERNAL
`WAKEUP SEQUENCE
`
`1103
`
`1110
`
`
`
`
`SUSACK‘!
`AND SUSACKZ
`DE-ASSERTED
`
`YES
`
`
`NO
`
`
`DE-ASSERT RESETS FOR PROCESSORS
`
`1112
`
`PROCESSORS EXECUTE RESET VECTORS
`
`1114
`
`RESTORE STATE TO ALL PROCESSORS
`
`1116
`
`ENABLE INTERRUPTS ON ALL PROCESSORS
`
`112“
`
`FIG. 11
`
`18
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 18 0f 22
`
`US 6,711,691 B1
`
`
`
`
`SNOOP PROCESSING
`
`
`[ 1200
`
`MCBM DE-ASSERTS ALL QACKS
`
`1202
`
`
`
`ANY NAPPING PROCESSORS ENTER DOZE MODE
`(SLEEPING PROCESSORS REMAIN SLEEPING)
`
`1204
`
`SNOOP OPERATION COMPLETED
`
`1206
`
`1208 ‘
`
`
`ASSERTED NO
`
`
`YES
`
`ALL DOZING PROCESSORS RE-ENTER
`NAP MODE
`
`.
`
`1210
`
`@
`
`FIG. 12
`
`19
`
`19
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 19 0f 22
`
`US 6,711,691 B1
`
`NAP PROCESSING
`
`1300
`
`‘\
`
`SCHEDULE NAP TASK TO ASSOCIATED PROCESSOR
`
`1302
`
`SET NAP BIT IN ASSOCIATED PROCESSOR
`
`1304
`
`ASSOCIATED PROCESSOR ASSERTS ITS OREO
`
`1306
`
`ASSOCIATED PROCESSOR ENTERS DOZE MODE
`
`I308
`
`
`
`
`
`ASSERTED NO
`
`YES
`
`1312
`
`ANY PROCESSOR IN DOZE MODE ENTERS NAP MODE
`
`FIG. 13A
`
`20
`
`20
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 20 0f 22
`
`US 6,711,691 B1
`
`INTERRUPT
`
`PROCESSING
`
`1326
`
`[
`
`ASSOCIATED PROCESSOR CLEARS ITS NAP BIT
`
`1328
`
`ASSOCIATED PROCESSOR DE-ASSERTS ITS QREQ
`
`1330
`
`MCBM DE-ASSERTS ALL QACKS
`
`1332
`
`ASSOCIATED PROCESSOR ENTERS RUN MODE
`
`
`
`
`
`(ANY OTHER PROCESSORS THAT WERE
`NAPPING ENTER DOZE MODE. AND SLEEPING
`PROCESSORS REMAIN SLEEPING)
`
`1 334
`
`ASSOCIATED PROCESSOR HANDLES THE
`INTERRUPT
`
`1335
`
`FIG. 138
`
`21
`
`21
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 21 0f 22
`
`US 6,711,691 B1
`
`[ 1400
`
`COMPLETE ANY PREVIOUSLY QUEUED
`TRANSACTIONS
`
`1402
`
`TAKE CONTROL ALL BUSES
`
`1404
`
`PLACE DRAM INTO SELF-REFRESH MODE
`
`1405
`
`DRIVE ADDRESS AND DATA LINES LOW
`
`1408
`
`DE-ASSERT AND TRI-STATE
`CONTROL LINES
`
`1410
`
`BYPASS AND SHUT DOWN INTERNAL PLLs
`
`1412
`
`ASSERT SUSACKI
`
`1414
`
`FIG. 14A
`
`22
`
`22
`
`

`

`US. Patent
`
`Mar. 23, 2004
`
`Sheet 22 0f 22
`
`US 6,711,691 B1
`
`[ 1450
`
`CLEAR SLEEP AND IDLE BITS
`
`1452
`
`ENABLE AND SWITCH TO
`INTERNAL PLL
`
`1454
`
`RETURN DRAM TO NORMAL MODE
`
`1455
`
`STOP DRIVING DATA & ADDRESS LINES LOW.
`STOP TRI-STATING CONTROL LINES, AND DE-
`
`ASSERT CONTROL LINES
`
`1458
`
`RELEASE CONTROL OF BUS
`
`1460
`
`DE»ASSERT SUSACK‘I
`
`1462
`
`FIG. 14B
`
`23
`
`23
`
`

`

`US 6,711,691 B1
`
`1
`POWER MANAGEMENT FOR COMPUTER
`SYSTEMS
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`This application claims the benefit of US. Provisional
`Application No. 60/133,918,
`filed May 13, 1999, and
`entitled “POWER MANAGEMENT FOR COMPUTER
`
`SYSTEMS”, the content of which is hereby incorporated by
`reference.
`
`10
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention relates to computer systems and,
`more particularly, to power management for computer sys-
`tems.
`
`2. Description of the Related Art
`Computer systems include electrical components that
`consume power when is active. These electrical components
`include processors, controllers, buses, and various sub-
`systems. Limited power management has been performed to
`reduce the power consumption of these electrical compo-
`nents. Often, with portable computers where power con-
`sumption is a major concern, the portable computers can
`enter a sleep mode in which the processor is slowed or
`stopped and in which the controllers, buses and various
`subsystems are also shutdown. The sleep mode thus pro-
`vides a state which the portable computer is able to enter to
`conserve power when processing resources are not needed.
`In the sleep mode, processors, controllers and subsystems
`are able to be shutdown. Typically little power management
`is offered with desktop computers.
`Although there are various disadvantages of conventional
`power management, one disadvantage is that
`the power
`management is primarily processor power management and
`not system level. As a result, the overall power management
`is not very efficient
`in reducing power consumption.
`Typically, many electrical components that consume signifi-
`cant amounts of power are either not power managed or
`crudely power managed to have only an on state (i.e., active)
`and an off state (i.e., shutdown). Also, in the case of desktop
`computers, conventional power management has been even
`less efficient.
`
`Still further, conventional power management for multi-
`processor computer systems has not been efficient. Hence,
`large amounts of power are consumed by these multi-
`processor computer systems even when there is no activity.
`Thus, there is a need for improved power management in
`computer systems.
`
`SUMMARY OF THE INVENTION
`
`Broadly speaking, the invention relates to power manage-
`ment for computer systems having one or more processors.
`One aspect of the invention pertains to providing hierarchi-
`cal power management. The hierarchical nature of the power
`management provided by the invention has various levels of
`power management such that power consumption of the
`computer system is dependent upon the amount of work
`placed on the processing resources of the computer system.
`Another aspect lie of the invention pertains to deterministic
`handshaking provided between a power manager and one or
`more controller units. The deterministic handshaking pro-
`vides for more reliable and controllable transitions between
`
`power management states which have associated power
`management taking place in the controller units. The inven-
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`tion is suitable for use with a single-processor computer
`system or a multi processor computer system.
`The invention can be implemented in numerous ways,
`including as a computer system, an apparatus, a method, and
`a computer readable medium. Several embodiments of the
`invention are discussed below.
`
`As a power management method for a multi-processor
`computer system having a plurality of processors, one
`embodiment of the invention includes: monitoring workload
`on the multi-processor computer system; and, placing the
`multi-processor computer system in one of a plurality of
`predetermined power management states based on the work-
`load.
`
`As a method for managing power consumption of a
`multi-processor computer system having a plurality of
`processors, one embodiment of the invention includes:
`determining a processing workload for the multi-processor
`computer system; awakening one or more of the processors
`from an inactive mode to an active mode when the process-
`ing workload is heavy; and transitioning one or more of the
`processors from the active mode to the inactive mode when
`the processing workload is light. The power consumption of
`the multi-processor computer system is managed such that
`those of the processors that are not needed to process the
`processing workload are transitioned into the inactive mode
`to conserve power, yet one or more of the processors can
`awaken from the inactive mode to the active mode to
`
`provide additional processing capabilities as needed to
`handle the processing workload.
`As a method for providing deterministic state changes
`within a computer system having a plurality of processors,
`a bus controller and a power manager, one embodiment of
`the invention includes: determining when a lower power
`state of the multi-processor computer system should be
`entered to reduce power consumption, and entering the
`lower power state. The lower power state is entered by
`performing the following operations: receiving at the bus
`controller a state change request from the power manager;
`initiating a state change sequence at the bus controller upon
`receiving the state change request; and notifying the power
`manager when the bus controller has completed the state
`change sequence.
`As a computer system, one embodiment of the invention
`includes: at
`least one processor,
`the processor executes
`operations in accordance with a processor clock; a bus
`controller operatively connected to the processor, the bus
`controller controls bus activity on a bus; and a power
`manager operatively connected to the processor and the bus
`controller, the power manager controls the processor clock
`and shutdown of the bus controller, and the power manager
`provides a first handshaking between the power manager
`and the bus controller to provide deterministic mode
`changes for the computer system so as to manage power
`consumption.
`The advantages of the invention are numerous. Different
`embodiments or implementations may have one or more of
`the following advantages. One advantage of the invention is
`that it offers improved power management that is obtained
`by a layered approach. Another advantage of the invention
`is that
`the system power management provided by the
`invention includes power management (i.e., shutdown) for
`not only processors but also related control circuitry (e.g.,
`bus controllers, I/O controllers, memory controllers, inter-
`rupt controllers). Still another advantage of the invention is
`that deterministic control over power management of control
`circuitry provides proper sequencing of shutdown opera-
`24
`
`24
`
`

`

`US 6,711,691 B1
`
`3
`tions. Yet another advantage of the invention is that more
`aggressive power management of a computer system is
`provided, whether for a single processor or a multi-processor
`system.
`Other aspects and advantages of the invention will
`become apparent from the following detailed description,
`taken in conjunction with the accompanying drawings,
`illustrating by way of example the principles of the inven-
`tion.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present invention will be readily understood by the
`following detailed description in conjunction with the
`accompanying drawings, wherein like reference numerals
`designate like structural elements, and in which:
`FIG. 1A is a flow diagram of power management pro-
`cessing according to a basic embodiment of the invention;
`FIG. 1B is a flow diagram of processor deactivation
`processing according to one embodiment of the invention;
`FIG. 1C is a block diagram of a multiple processor
`computer system according to one embodiment of the inven-
`tion;
`FIG. 2 is a flow diagram of power management process-
`ing according to one embodiment of the invention;
`FIGS. 3A—3C are flow diagrams of start processing
`according to one embodiment of the invention;
`FIG. 4 is a flow diagram of activation processing for all
`but the last active processor according to one embodiment of
`the invention;
`FIG. 5 is a flow diagram of the shutdown processing for
`extra processors according to one embodiment of the inven-
`tion;
`FIG. 6 is a flow diagram of nap single interrupt processing
`according to one embodiment of the invention;
`FIG. 7 is a flow diagram of nap processing for the last
`active processor according to one embodiment of the inven-
`tion;
`FIGS. 8A and 8B are flow diagrams of idle processing for
`the last active processor according to one embodiment of the
`invention;
`FIG. 9 is a flow diagram of activation processing for the
`last active processor according to one embodiment of the
`invention;
`FIGS. 10A and 10B are flow diagrams of shutdown
`processing for the last active processor according to one
`embodiment of the invention;
`FIG. 11 is a flow diagram of wakeup processing for the
`last active processor according to one embodiment of the
`invention;
`FIG. 12 is a flow diagram of snoop processing according
`to one embodiment of the invention;
`FIG. 13A illustrates a flow diagram of nap processing
`according to one embodiment of the invention;
`FIG. 13B illustrates a flow diagram of interrupt process-
`ing according to one embodiment of the invention;
`FIG. 14A is a flow diagram of memory controller/bus
`manager (MCBM) sleep shutdown processing according to
`one embodiment of the invention; and
`FIG. 14B is a flow diagram of MCBM sleep wakeup
`processing according to one embodiment of the invention.
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`The invention relates to power management for computer
`systems having one or more processors. One aspect of the
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`invention pertains to providing hierarchical power manage-
`ment. The hierarchical nature of the power management
`provided by the invention has various levels of power
`management such that power consumption of the computer
`system is dependent upon the amount of work placed on the
`processing resources of the computer system. Another
`aspect of the invention pertains to deterministic handshaking
`provided between a power manager and one or more con-
`troller units. The deterministic handshaking provides for
`more reliable and controllable transitions between power
`management states which have associated power manage-
`ment taking place in the controller units. The invention is
`suitable for use with a single-processor computer system or
`a multi-processor computer system.
`Embodiments of the invention are discussed below with
`reference to FIGS. 1A—14B. However, those skilled in the
`art will readily appreciate that the detailed description given
`herein with respect
`to these figures is for explanatory
`purposes as the invention extends beyond these limited
`embodiments.
`
`FIG. 1A is a flow diagram of power management pro-
`cessing 10 according to a basic embodiment of the inven-
`tion. The power management processing 10 is performed by
`a multi-processor computer system to reduce the power
`consumption of any processing resources whenever those
`resources are not needed.
`
`The power management processing 10 monitors 12 the
`workload of the computer system or the processors within
`the computer system. The workload reflects how busy the
`computer system or the processors therein are in processing
`useful tasks. Once the workload is obtained, it is determined
`14 whether the workload is heavy.
`When it is determined 14 that the workload is heavy, then
`one or more processors of the computer system are activated
`16. Here,
`the heavy workload indicates that additional
`processing resources are needed. By activating 16 one or
`more of the processors (that were previously inactive), the
`computer system obtains the additional processing resources
`to help process the heavy workload.
`Assume a simplistic example in which the workload is
`1000 units of work that is waiting to be processed by the
`computer system. If only one processor is active, the work-
`load of the one processor can be considered 1000 units. If a
`heavy workload is deemed anything over 600 units, then the
`power management processing 10 will operate to activate
`one or more additional processors. If one additional proces-
`sor were to be activated 16, then the average workload for
`each of the two activated processors would drop to 500
`units.
`
`On the other hand, when it is determined 14 that the
`monitored workload is not heavy,
`it
`is determined 18
`whether the monitored workload is light. When it is deter-
`mined 18 that the monitored workload is light, then one or
`more processors of the computer system are deactivated 20.
`By deactivating 20 one or more of the processors,
`the
`computer system conserves power by placing the computer
`system, or its processors, into a lower powered state.
`Following blocks 16 and 20, as well as following the
`decision block 18 when the workload is not light, the power
`management processing 10 is complete and ends. However,
`typically,
`the power management processing 10 continu-
`ously repeats such that
`the power management for the
`computer system is ongoing and dynamically performed.
`According to the invention, the deactivation 20 of one or
`more of the processors can be performed in a wide variety
`of ways. In particular, the deactivation 20 can be performed
`25
`
`25
`
`

`

`US 6,711,691 B1
`
`5
`over a series of low power states into which the computer
`system can enter (or low power modes into which its
`processors can enter) to conserve power. The deactivation 20
`can also be considered to be performed over a series of
`layers such that different layers offer different tradeoffs of
`processing resources verses power efficiency.
`FIG. 1B is a flow diagram of processor deactivation
`processing 30 according to one embodiment of the inven-
`tion. The processor deactivation processing 30 is,
`for
`example, suitable for use as the deactivation 20 of the one
`or more processors such as shown in FIG. 1A.
`The processor deactivation processing 30 initially deter-
`mines whether there is more than one active processor.
`When it is determined 32 that there is more than one active
`
`processor, the processor deactivation processing 30 places
`34 one or more (but not all) of the processors in a sleep
`mode. The number of the processors being placed in the
`sleep mode is based on the workload. For example, in a
`computer system having five processors, the processor deac-
`tivation processing 30 could shutdown up to four of the
`processors by placing them in the sleep mode when the
`workload is light.
`On the other hand, when it is determined 32 that there is
`only one active processor, then the processor deactivation
`processing is performed for the last processor in a layered
`fashion. In particular, it is initially determined 36 whether
`the last processor is in a run mode. The run mode is a mode
`in which the last active processor is active such that
`it
`processes instructions. When it is determined that the last
`processor is in the run mode, then the last processor is placed
`38 in a first low power mode. In this embodiment, the first
`low power mode is a mode in which the processor conserves
`some power but has a loss of performance as compared to
`the run mode.
`
`the last
`is determined 36 that
`Alternatively, when it
`processor is not in the run mode, it is determined 40 whether
`the last processor is in the first low power mode. When it is
`determined 40 that the last processor is already in the first
`lower power mode, then the processor deactivation process-
`ing 30 can perform additional operations to provide greater
`reductions in power consumption. Specifically, the last pro-
`cessor is placed 42 in a second low power mode. The second
`low power mode is a mode that offers less power consump-
`tion (as well as less performance) than does the first low
`power mode. Next, any unneeded electrical components or
`units of the computer system are shut down 44. On the other
`hand, when it is determined 40 that the last processor is not
`in the first low power mode, it is assumed that the last
`processor is already in the second low power mode such that
`additional processing by the processor deactivation process-
`ing 30 is not available and, thus, the processor deactivation
`processing 30 is complete and ends. In addition, following
`blocks 34, 38 and 44, the processor deactivation processing
`30 is complete and ends.
`The processor deactivation processing 30 can be consid-
`ered to have multiple hierarchical levels. For example, with
`respect to a multi-processor computer system, the processor
`deactivation processing 30 offers three layer of power man-
`agement. A first layer is provided by block 34 such that the
`number of processors that are active (and not sleeping) is
`determined based on workload. Within the first layer there
`are various different levels of power management based
`largely on the number of processors activated. In any case,
`once only one of the processors is active, than a second layer
`can be entered for further power reduction. In the second
`layer provided by block 38,
`the last active processor is
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`placed in the first low power state. Still further, if further
`power reduction is further desired,
`then the last active
`processor can by placed in the second low power mode.
`Typically,
`the second power mode would placed the last
`active processor in even a greater power saving state than
`provided by the first low power mode. Additionally, since
`the second low power mode essentially deactivates the last
`active processor, various other electrical components or
`units within the multi-processor computer system can also
`be placed in a low-power state (or shut-down) to further
`enhance the power reductions. For example, the electrical
`components or units can include controllers (e.g., controller
`chips) or unneeded subsystems. Often, some subsystem can
`be shutdown in other earlier layers.
`A third low power mode could also be provided by the
`processor deactivation processing 30. The third low power
`mode would offer even greater power reductions than pro-
`vided in the first and second low power modes. For example,
`in the third low power mode, the last processor could be
`placed in the sleep mode and, thus, with all processors in the
`sleep mode, the computer system would itself be in a sleep
`mode. More than three low power modes could also be
`provided. The various low power modes can include, for
`example, nap, doze or sleep modes for a processor as well
`as also remove clocks and power from a processor and/or
`other electrical components in certain of the various low
`power states.
`Although the invention is described herein largely in
`terms of a multi-processor computer system, the invention is
`also applicable to single processor computer systems. As an
`example, blocks 34—44 of FIG. 1B in effect pertain to power
`management for single processor computer systems.
`FIG. 1C is a block diagram of a multiple processor
`computer system 100 according to one embodiment of the
`invention. The multiple processor computer system 100
`includes a plurality of processors, including processor A
`102, processor B 104, processor C 106 and processor D 108.
`Each of the processors 102—108 operates to execute instruc-
`tions under the control of an operating system (not shown).
`The multiple processor computer system 100 also includes
`a power manager 110 that provides power management
`functions for the multiple processor computer system 100.
`In general,
`the power manager 110, provides soft resets
`(SRST), hard resets (HRST), system management interrupts
`(SMIs), and configuration control signals for phase lock
`loops (PLLs). In one embodiment, the power manager is an
`embedded controller or processor. The multiple processor
`computer system 100 also includes a memory controller/bus
`manager (MCBM) 112, a dynamic random access memory
`(DRAM) 114, an I/O-interrupt controller (IOIC) 116, and an
`operating system 11. Although not shown in FIG. 1C, the
`multiple processor computer system 100 would also include
`other components such as Read Only