[19]
`United States Patent
`[11] Patent Number:
`5,978,594
`Bonnell et al.
`[45] Date of Patent:
`Nov. 2, 1999
`
`US005978594A
`
`[54]
`
`SYSTEM FOR MANAGING COMPUTER
`RESOURCES ACROSS A DISTRIBUTED
`COMPUTING ENVIRONMENT BY FIRST
`READING DISCOVERY INFORMATION
`ABOUT HOW TO DETERMINE SYSTEM
`RESOURCES PRESENCE
`
`[75]
`
`Inventors: David N. Bonnell, Houston; Kirill L.
`Tatarinov; Martin W. Picard, both of
`Bellaire, all of Tex.
`
`[73] Assignee: BMC Software, Inc., Houston, Tex.
`
`[21] Appl. No.: 08/812,860
`
`[22]
`
`Filed:
`
`Mar. 6, 1997
`
`Related US. Application Data
`
`[63] Continuation of application No. 08/316,034, Sep. 30, 1994,
`abandoned.
`
`Int. Cl.6 ...................................................... G06F 13/10
`[51]
`[52] US. Cl.
`............................................. 395/837; 395/825
`[58] Field of Search ......................... 395/200.53, 200.54,
`395/500, 712, 825, 837; 364/242
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`10/1988 Crossley .................................. 364/200
`4,780,821
`9/1989 Burkhardt, Jr. et al.
`............... 364/900
`4,866,664
`5/1990 Hershey et al.
`.......
`.. 364/200
`4,924,378
`.. 364/200
`5,062,037 10/1991 Shorter et al.
`.....
`5,079,695
`1/1992 Dysart et al.
`..
`.. 395/700
`
`5,109,486
`4/1992 Seymour
`395/200
`7/1992 Coyle, Jr. et al.
`.....
`.. 395/650
`5,129,086
`
`5,142,683
`8/1992 Burkhardt, Jr. et al.
`.. 395/725
`5,167,035
`11/1992 Mann et al.
`...........
`.. 395/575
`.....
`.. 395/650
`5,237,684
`8/1993 Record et al.
`
`
`.. 395/600
`5,291,591
`3/1994 Kawano et al.
`.....
`.. 395/650
`5,291,597
`3/1994 Shorter et al.
`
`.. 395/650
`5,310,323
`5/1994 Maeurer et al.
`.....
`.. 395/275
`5,315,711
`5/1994 Barone et al.
`
`5,317,742
`5/1994 Bapat
`.............
`395/700
`5,367,635
`11/1994 Bauer et al.
`.. 395/200
`5,369,570
`11/1994 Parad ...................................... 364/401
`
`5,442,791
`5,463,775
`5,471,617
`5,483,468
`5,491,796
`
`..
`
`8/1995 Wrabetz et al.
`395/650
`10/1995 DeWitt et al.
`395/184.01
`11/1995 Farrand et al.
`........ 395/700
`1/1996 Chen et al.
`.......
`364/551.01
`
`................. 395/200.09
`2/1996 Wanderer et al.
`
`
`
`FOREIGN PATENT DOCUMENTS
`
`Japan .
`of 0000
`57146361
`Japan .
`60252952 of 0000
`Japan .
`61286959
`of 0000
`OTHER PUBLICATIONS
`
`DDS/Patrol, ver. 1.3 Installation and User’s Guide; Patrol
`Software Pty. Ltd.; Jun. 10, 1993.
`Communicating Event Information Between Applications in
`A Multitasking Environment;
`IBM Technical Disclosure
`Bulletin, p. 4832 (Apr, 1987).
`
`(List continued on next page.)
`
`Primary Examiner—Thomas C. Lee
`Assistant Examiner—Chien Yuan
`
`Attorney, Agent, or Firm—Jenkens & Gilchrist, PC.
`
`[57]
`
`ABSTRACT
`
`A method and apparatus are disclosed for managing a
`computer network. A manager software system is installed
`on a network management computer system within the
`network, and one agent software system is installed on each
`of the server computer systems in the network. Aknowledge
`module in the form of a text fie is stored on the network
`
`manager computer system so that the manager software
`system can transmit knowledge to the various agent software
`systems throughout the network, for use by the agents in
`monitoring and managing the server on which they are
`installed. Interpretable script language programs are present
`on all computers in the network, expanding and customizing
`the functionality of the agent software systems. Amethod is
`disclosed for using the high level interpretable script lan-
`guage programs in connection with the agent software
`systems for discovering resources on the network, monitor-
`ing aspects of resources, and taking recovery actions auto-
`matically in the event of an alarm condition.
`
`14 Claims, 11 Drawing Sheets
`
`_‘
`.
`[Ac—EETFTKAER'] F'UARAZET'
`I
`|
`INDICATES
`|l
`SDFWIARE
`max] DISCOVERT
`|
`'SENDS SIGNAL'l/HB
`. PROCEDURE
`.
`.
`To INITIATE
`.
`:
`SHOULD BE
`:
`:
`DISCOVERY :
`L EXECUTED J L PROCEDURE J
`
`READ RNOULEDCE DATABASE ON
`___l______I
`SERVER To FIND NAME OF A
`RESOURCE CLASS THAT
`SHOULD BE SEARCHED EOR
`
`EOUND A
`RESOURCE CLASS
`
`TO SEARCH FOR
`7
`156
`WERE Am
`
`READ KNOWLEDGE DAIAEASE ON
`RESOURCES DISCOVERED
`“0
`ON THE SERVER
`SER/ER TO FIND NAME
`AND LOCATION OF SCRIPT
`PROGRAM THAT WILL
`
`SEND MESfiGE TO
`LOCATE THE SCRIPT PROGRAM
`SEARCH EUR THTs RESOURCE
` WHAT RESOURCES 138
`
`MANAGER INDICATINC \
`T26/
`WERE DISCOVERED
`
`
`I
` HAS SCRIPT
`PROGRAM BEEN
`RECEIVE MESSACE(S)
`
`FROM MANAGER
`CDNPILED YET
`a
`CONTAINING MORE
`CUNPILE
`ANDRLEDCE AND
`
`
`YES
`SCRIPT PROGRAM
`SCRIPT PROGRAMS
`USEFUL IN MANAGING
`
`
`THE DISCUMERED
`I30
`RESOURCES
`INTERPRET SCRIPT PROGRAM
`
`TO SEARCH FOR
`132/
`PR:SENCE DE THE RESOURCE
`
`.—I
`STDRE RESULTS OF SEARCH
`[32/
`
`
` I
`
`1247
`
`120
`
`‘40
`
`
`
`
`
`001
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`001
`
`

`

`5,978,594
`Page 2
`
`OTHER PUBLICATIONS
`
`Having a Network Nightmare? Let the Software Take Over,
`Fisher, Lawrence M.; The New York Times, Sep. 18, 1994;
`p. F8.
`The Simple Book, An Introduction to Internet Management;
`Rose, Marshall T.; 2d ed.; 1994.
`DEC, Development and distribution agreement between
`DEC and Patrol Software, Oct. 1993.
`Graeme, Philip, Developer Down Under Hits Distributed
`Shore, Software Magazine, v 13 n9, Jun. 199.
`Digital Equipment Corp., “Polycenter System Watchdog
`(V.1.0)”, Computer Select, Nov. 1993.
`“Automatic System Configuration Determination Program”,
`IBM TDB vol. 13, pp. 2722—2724, Feb. 1971.
`Yemini, Yechiam, Netmate: Management of Complex Dis-
`tributed Networked Systems, IEEE, p. 173, 1991.
`Drake, Peter, Using SNMP to Manage Networks, IEEE, pp.
`2/1—2/4, 1991.
`Kiriha, Yoshiaki, An Automatic Generation of Management
`INformation Base for OSI based Network Management
`System, IEEE, pp. 0649—0653, 1991.
`
`Agoulmine, Nazim, A System Architecture for Updating
`Management
`INformation in Heterogeneous Networks,
`IEEE, pp. 999—1003, 1992.
`Ahmed M., B—ISDN Customer Network Management
`Framework, IEEE, pp. 36—41, 1992.
`Zhang, XinXin, X500 Directory and OSI Management,
`IEEE, pp. 1106—1110, 1992.
`Zhang, XinXin, Supporting Network Management Through
`Distributed Directory Service, IEEE, pp. 1000—1010, May
`1994.
`
`Wu, Shyhtsun F., EMOSY: An SNMP Protocol Object
`Generator for the Protocol Independent MIB, IEEE, pp.
`133—144, 1994.
`O’Mahony, Donal, Security Considerations in a Network
`Management Enviornment, IEEE, pp. 12—17, 1994.
`Leon, Benjamin J. Leon, Network Management Information
`for System Control, IEEE, pp. 1553—1557, 1993.
`Abeck, Sebastian, Implementing OSI and Internet Managed
`Object Libraries, IEEE, pp. 1197—1201, 1993.
`Miyauchi, Naoto, An Implementation of Management Infor-
`mation Base, IEEE, pp. 318—321, 1991.
`
`002
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`002
`
`

`

`US. Patent
`
`Nov. 2, 1999
`
`Sheet 1 0f 11
`
`5,978,594
`
`on
`
`
`
`mm>mmm”EEO
`
`
`
`mzmzmammszou
`
`_ll.l.l.lll_SLazéoog_EEom_FIIIIIL —IIIIIII-_misEom_\_Ema“_.llllIII—
`
`292223228
`
`mm<aom<z
`
`N 6
`
`5%
`
`S
`
`r____________
`
`"'""'"----'I
`
`5385;;
`
`om
`
`_l|.l.l|ull_
`
`.l-IIIIIJ
`2858?:S_Exam_\
`
`
`
`IIIIIIIIIIIIIIIIII.III.I...I|.II.|_
`
`2222232200
`
`mmssom<z
`
`_ 3
`
`_I.|I..||I.J
`:382_
`
`on
`
`310
`
`§<m
`
`m:
`
`mm
`
`
`
`83mg5&on
`
`_I.I||.lll_
`
`—II.|III.L
`_II.I.II.I.I.__magma__$37227:,
`
`vm
`
`
`
`55%mmgmzommvmam¢z<§xmoEsz/S
`
`IIL
`
`003
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`003
`
`
`
`
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 2 0f 11
`
`5,978,594
`
`47
`
`FIG 2
`
`I'_"'_‘-,'-'I
`I' ------ 1
`I
`IIINANAGERS
`I
`49\I
`OBJECT
`I
`OWLEDGE I/
`.
`I
`,
`DATABASE
`.
`DATABASE
`L
`_,
`------ MANAGER '- ------ -'
`34
`l.__\ __________
`
`44
`
`KNIOOILIPIGE
`PARSER
`
`SCRIPT
`PROGRAM
`COMPILER
`
`KNOWLEDGE
`DATABASE
`MANAGER
`
`OBJECT
`DATABASE
`MANAGER
`
`”I???“
`INTERFACE
`
`INTERFACE
`TO OTHER
`NETWORK
`MANAGEMENT
`SOFTWARE
`
`
`
`l '
`
`'
`I
`I
`
`IlI
`
`'
`I 58
`I
`I
`I
`I
`I
`
`I l L
`
`________________________________________ J
`
`75\.'""A<';EIF3'""I
`I
`I
`LKI‘EWEEPEEDL‘LABI‘SEJ
`AGENT
`SOFTWARE SYSTEM
`
`8.
`
`36
`
`SCRIPT
`PROGRAM
`COMPILER
`
`SCRIPT
`PROGRAM
`INTERPRETER
`
`66
`APPLICATION
`DISCOVERY
`MANAGER
`
`COMMUNICATIONS
`
`62
`
`DISPATCHER _
`
`72
`
`
`
`KNOWLEDGE
`DATABASE
`MANAGER
`
`
`PROCESS
`CACHE MANAGER
`
`76
`
`r---- ----------------------------- 1
`I
`64
`68
`I
`I
`I
`I
`I
`.
`.
`I
`I
`I
`l
`'
`I
`'
`I I‘""I
`I
`'
`I QE‘EUE I
`I
`I
`I__ -4
`I
`I
`'\
`I
`I
`71
`I
`I
`I
`.
`I
`I
`I
`I
`8O
`78
`I
`.
`I_ __________________________________ _l
`l—__"' —"'—'|
`
`FIG 3
`
`COMMAND
`EXECUTION
`MANAGER
`
`RUN
`OUEUE
`SCHEDULUER
`
`70
`PARAMETER
`AND RECOVERY
`ACTION MANAGER
`
`LPBOEESE SABER 77
`
`004
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`004
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 3 0f 11
`
`5,978,594
`
`38
`\‘
`
`FIG. 4
`
`INFORMATION APPLICABLE TO ALL CLASSES OF COMPUTERS
`
`- ENVIRONMENT
`- PARAMETER
`- COMMAND TYPES
`
`- COMMANDS
`- SETUP COMMANDS
`- INFOBOX COMMANDS
`
`
`
`INFORMATION APPLICABLE
`TO COMPUTER CLASS 1
`
`ENVIRONMENT
`- PARAMETERS
`' COMMAND TYPES
`- COMMANDS
`- SETUP COMMANDS
`'
`INFOBOX COMMANDS
`
`INFORMATION
`APPLICABLE TO
`COMPUTER
`INSTANCE I
`CLASS I
`
`IN
`
`- ENVIRONMENT
`- PARAMETERS
`- COMMANDS
`'
`INFOBOX
`COMMANDS
`
`INFORMATION
`APPLICABLE TO
`COMPUTER
`INSTANCE ’n'
`IN CLASS I
`
`' ENVIRONMENT
`- PARAMETERS
`- COMMANDS
`'
`INFOBOX
`COMMANDS
`
`INFORMATION APPLICABLE
`TO APPLICATION CLASS 1
`ENVIRONMENT
`- PARAMETERS
`' COMMAND TYPES
`‘ COMMANDS
`-
`INFOBOX COMMANDS
`' DISCOVERY
`
`INFORMATION APPLICABLE
`TO COMPUTER CLASS ’n’
`
`ENVIRONMENT
`- PARAMETERS
`- COMMAND TYPES
`- COMMANDS
`- SETUP COMMANDS
`- INFOBOX COMMANDS
`
`INFORMATION
`APPLICABLE TO
`COMPUTER
`INSTANCE I
`CLASS 'n'
`
`IN
`
`- ENVIRONMENT
`- PARAMETERS
`- COMMANDS
`'
`INFOBOX
`COMMANDS
`
`INFORMATION
`APPLICABLE TO
`COMPUTER
`INSTANCE 'n'
`IN CLASS ’n'
`
`- ENVIRONMENT
`- PARAMETERS
`- COMMANDS
`-
`INFOBOX
`COMMANDS
`
`INFORMATION APPLICABLE
`TO APPLICATION CLASS 'n'
`ENVIRONMENT
`- PARAMETERS
`- COMMAND TYPES
`- COMMANDS
`' INFOBOX COMMANDS
`' DISCOVERY
`
`
`
`
`INFORMATION
`APPLICABLE TO
`APPLICATION
`
`INSTANCE I
`CLASS I
`
`IN
`
`' ENVIRONMENT
`- PARAMETERS
`- COMMANDS
`
`INFORMATION
`APPLICABLE TO
`APPLICATION
`
`INSTANCE 'n'
`IN CLASS I
`
`~ ENVIRONMENT
`~ PARAMETERS
`- COMMANDS
`
`INFORMATION
`APPLICABLE TO
`APPLICATION
`
`INSTANCE I
`CLASS 'n'
`
`IN
`
`~ ENVIRONMENT
`- PARAMETERS
`' COMMANDS
`
`INFORMATION
`APPLICABLE TO
`APPLICATION
`
`INSTANCE 'n'
`IN CLASS ’n'
`
`- ENVIRONMENT
`- PARAMETERS
`- COMMANDS
`
`005
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`005
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 4 0f 11
`
`5,978,594
`
`!PATROLV2.0.21
`!++
`
`3973F830585795A8
`
`PATROL Session Knowledge Module
`
`I !
`
`I I
`
`..—
`VERSIONZ
`.
`COMPUTERS = I
`3 NAME = "ALL_COMPUTERS".
`COMMANDS = I
`I NAME = “Login”,
`AVAILABILITY = AVAILABLE_ONLINE.
`SECURITY = SECURITY_INHERIT,
`BASE_COMMAND = I
`I COMPUTER_TYPE = "ALLCOMPUTERS".
`COMMAND_TYPE
`"OS".
`COMMAND_TEXT
`769753920 ”ZI/terminoIEmulatori —s| 500 —5b —n ZI/ho
`
`FIG' 5a
`
`I
`
`I
`
`I
`
`I.
`
`INFO_BOX = 3
`3 NAME = ”Ip Address".
`AVAILABILITY = AVAILABLE_ONLINE.
`SECURITY = SECURITY_INHERIT,
`BASE_COMMAND = 3
`I COMPUTER_TYPE = "ALL_COMPUTERS",
`'
`COMMAND_TYPE = "OS",
`COMMAND_TEXT = 759992599 “7.ECHO Zf/ipAddressf"
`
`;
`
`I
`
`I
`
`I.
`SETUP = 5
`I NAME = "Terminal Emulater",
`AVAILABILITY = AVAILABLE_ALWAYS.
`SECURITY = SECURITY_INHERIT,
`BASE_COMMAND =
`PE = "ALLCOMPUTERS",
`I COMPUTER_
`COMMAND_TYPE
`"PSL".
`f COMMAND_TEXT
`759992599 ”set (\“/termino|Emqutor\", \“xterm\");"
`I COMPUTER_TYPE = ”ULTRIX”,
`COMMAND_TYPE
`“PSL”,
`COMMAND_TEXT
`759992599 "set (\“IerminoIEmulotor\". \"dxterm\");"
`
`I
`
`I
`
`E
`I
`ENVIRONMENT = I ”PATH" = "/bin:/usr/bin:/etc:/usr/etcz/usr/ucbz/usr/bsd/quotoz/usr/sl
`PARAMETERS = g
`INAME
`"PrinterDIscovery".
`PARAM_TYPE = STANDARD,
`ACTIVE = True,
`MONITOR = False,
`CHECK = False,
`BASE_COMMAND = I
`I COMPUTER_TYPE = ”ALL_COMPUTERS".
`COMMAND_TYPE
`“PSL”.
`I COMMAND_TEXT = 766297889 “set (\”/prnDiscoveryPoIlTime\". 5500);"
`
`s»
`START = "ASAP",
`EXECUTION = ”I",
`
`006
`
`ServiceNow's Exhibit No
`
`.1001
`
`ServiceNow's Exhibit No. 1001
`
`006
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 5 0f 11
`
`5,978,594
`
`FIG. 5b
`
`3 NAME = "CPU Utilization".
`AVAILABILITY = AVAILABLE_0NLINE.
`SECURITY = SECURITY_INHERIT,
`BASE_COMMAND =
`PE = "ALLCOMPUTERS".
`I COMPUTER_
`COMMAND_TYPE
`“PSL”.
`COMMAND_TEXT
`LOAD "cpu_trend_onolysis.psl"
`
`I
`
`£1
`TASK = :
`SHOW_TIMER = TRUE.
`INTERACTIVE = FALSE.
`ATTN_POPUP = TRUE
`
`;
`
`I
`
`I
`
`I
`
`I.
`INFO_BOX = I
`3 NAME = ”System Load".
`AVAILABILITY = AVAILABLE_0NLINE.
`SECURITY = SECURITY_INHERIT,
`BASE_COMMAND = [W
`PE = ”ALL_COMPUTER5",
`I COMPUTER_
`COMMAND_TYPE
`”PSL".
`COMMAND_TEXT
`759992618 ”print (get (\"/SYSTEM/SYSTEM/uptLood\"))
`
`I
`
`I
`
`I
`
`I.
`PARAMETERS = I
`3 NAME = ”USRPROCCoII",
`PARAM_TYPE = COLLECTOR,
`ACTIVE = True.
`MONITOR = False.
`CHECK = False.
`BASE_COMMAND = I”
`PE = "ALL_COMPUTERS",
`I COMPUTER_
`COMMAND_TYPE
`"PSL".
`I COMMAND_TEXT = LOAD "usr_proc_collector.psl"
`
`I.
`START = "ASAP",
`POLLTIME = “600".
`EXTERNALPOLLING = False
`
`I
`
`I
`
`I!
`
`44656
`
`007
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`007
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 6 0f 11
`
`5,978,594
`
`# PSL Serial No.: 765775959
`
`E usr_proc_collector.psl
`
`FIG. 6a
`
`lnc. as an unpublished licensed work.
`If Copyright 1993-94 Patrol Software.
`If Copyright 1994 BMC Software.
`Inc. as an unpublished licensed work.
`If
`is a trade secret which is the property
`fit The source code created in 1994.
`# of BMC Software.
`Inc. All use. disclosure. and/or reproduction not specifically
`# authorized by BMC Software.
`lnc..
`is prohibited.
`
`laws
`
`# {
`
`I This program may also be protected under the copyright and trade secret
`# of non—U.S. countries. All
`right reserved.
`
`# #
`
`(c) 1993. 1994 Patrol Software
`Copyright
`# Copyright (c) 1994 BMC Software.
`Inc.
`
`# #
`
`: usr_proc_collector.psl
`File
`:H KuSumo
`# Author
`{I Platform IALL
`retrieve user information and
`# Description: The script
`process information from who and psl process
`function. This script calculate number of user
`login. session. process. as well as average
`process per user
`
`ZOMBIE = ”<defunct>";
`
`# get machine type
`machine_type = get (”/appType");
`
`cmd = “who":
`
`# execute user info command
`buf = system (cmd);
`
`# check if buffer is empty or command not
`
`found
`
`a;
`
`(buf =exi=t;“ ") 3
`
`elsif
`
`(index (buf. "not
`found")){
`pript
`(”SYSTEM:SYSTEMzUSRColl: Warning environment
`exr ;
`
`is not set properly. switch to sil
`
`t n
`
`o_session = 0;
`If process user information
`no_user = 0;
`user_|ist = " "'
`
`foreach entry (buf)l
`# get user name
`name = ntharg (entry. 1);
`
`is root.
`it
`{I check if
`if (name !=.“root")i
`no_sessron ++;
`
`ignore it
`
`this user is not
`# check if
`in the user list
`if
`(lindex (user_|ist. name
`user_|ist = user_|ist . name . “\n";
`no_user ++;
`
`008
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`008
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 7 0f 11
`
`5,978,594
`
`FIG. 6b
`
`g
`
`t
`
`{I process process information
`process_info = process (“¢");
`no_process = 0:
`no_user__process = 0;
`no_zomble_process = 0;
`
`foreoch entry (process_info)§
`I} check if
`this is zombie process
`if
`(index (entry. ZOMBIE»!
`‘
`no_zombie_process ++;
`
`if (nthorg (entry, 3)
`!= "root")l
`‘
`no_user_process ++;
`no_process + +;
`
`if calculate average user process
`if ((no_user !=0 && (no_,use _process !=0))§
`i
`ovg_usr_proc = mt no_user_process / no_user);
`else}
`,
`BXIt;
`
`# set global variables
`set ("/SYSTEM/SYSTEM/ueroUser", no_user);
`set ("/SYSTEM/SYSTEM/ueroSession", no_session);
`set ("/SYSTEM/SYSTEM/usrAngsrProc". avg_usr_proc);
`set
`("/SYSTEM/SYSTEM/procNumProcs". no_process);
`set
`("/SYSTEM/SYSTEM/procUserProcs”, no_user_process);
`set
`("/SYSTEM/SYSTEM/procNoZombieProcess”. no_zombie_process);
`
`# get global parameters
`if (exists ("/USERS/USERS"))§
`set ("/USERS/USERS/USRNoUser/value", no_user);
`set ("/USERS/USERS/USRNoSession/value", no_session);
`set ("/USERS/USERS/USRUserProcs/volue", no_user_process);
`set ("/USERS/USERS/USRAvgUsrProc/value". avg_usr_proc);
`
`i
`
`(exists "/SYSTEM/SYSTEM")){
`set
`("/SYSTEM/SYSTEM/SYSNoUser/vaIue". no_user):
`set
`("/SYSTEM/SYSTEM/SYSNumProcs/volue". no_process);
`set
`('“/SYSTEM/SYSTEM/SYSAvgUsrProc/vatue". ovg_ usr_proc);
`
`(exists (“/PROCESS/PROCESS"))§
`set ("/PROCESS/PROCESS/PROCNUmProcs/volue". no_process);
`
`if
`
`t i
`
`f
`
`i
`
`009
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`009
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 8 0f 11
`
`5,978,594
`
`FIG. 7
`
`MANAGER SOFTWARE PARSES KNOWLEDGE
`
`MODULE, EXTRACTING INFORMATION AND
`CREATING KNOWLEDGE DATABASE.
`
`MANAGER SOFTWARE ACCEPTS INPUT
`INFORMATION INDICATING WHAT TYPES
`
`OF COMPUTERS ARE TO BE MANAGED.
`
`TO COMPUTERS THAT ARE TO BE MANAGED.
`
`MANAGER SOFTWARE SEARCHES KNOWLEDGE
`DATABASE FOR KNOWLEDGE PERTINENT
`
`
`
`
`
`MANAGER SOFTWARE SENDS MESSAGE(S) TO
`AGENT CONTAINING: KNOWLEDGE PERTINENT
`
`TO COMPUTER IN WHICH AGENT IS INSTALLED.
`
`WHICH RESOURCE CLASSES TO LOOK FOR ON THAT
`COMPUTER, AND WHICH SCRIPT PROGRAMS SHOULD BE
`USED TO LOOK FOR THE RESOURCE CLASSES.
`
`
`
`
`
`
`
`
`
`
`
`MANAGER SOFTWARE ALSO SENDS THE SCRIPT
`
`
`PROGRAMS THAT WILL BE NECESSARY.
`
`
`AGENT SOFTWARE RECEIVES THE MESSAGE(S),
`PARSES THE INFORMATION, CREATES A
`KNOWLEDGE DATABASE WITH THE INFORMATION,
`
`AND STORES THE SCRIPT PROGRAMS.
`BE DONE FOR PARTICULAR RESOURCES
`
`KNOWLEDGE DATABASE MANAGER WITHIN AGENT
`
`SOFTWARE CREATES JOBS AND PLACES THEM IN
`THE RUN QUEUE IF RECEIVED KNOWLEDGE INDICATES
`THAT PERIODIC MONITORING PROCEDURES SHOULD
`
`96
`
`98
`
`100
`
`110
`
`112
`
`II4
`
`010
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`010
`
`

`

`
`
` FOUND A
`RESOURCE CLASS
`TO SEARCH FOR
`?
`
`NO
`
`
`
`
`
`US. Patent
`
`N0V.2, 1999
`
`Sheet 9 0f 11
`
`5,978,594
`
`115
`
`I-_MANA-(;I-:-R__I
`FAG—ENT-S-TTMER—I
`:
`INDICATES
`::
`SOFTWARE
`I
`DISCOVERY
`SENDS SIGNAL
`\: PROCEDURE
`I
`I
`TO INITIATE If
`:
`SHOULD BE
`:
`:
`DISCOVERY :
`LELEEUJEL
`f‘ioEEB‘iREJ
`
`118
`
`FIG. 8
`
`READ KNOWLEDGE DATABASE 0N
`SERVER TO FIND NAME OF A
`RESOURCE CLASS THAT
`
`SHOULD BE SEARCHED FOR
`
`120
`
`
`
`YES
`
`SEND- MESSAGE TO
`MANAGER INDICATING
`WHAT RESOURCES
`
`WERE ANY
`RESOURCES DISCOVERED
`
`
`ON THE SERVER
`
`
`?
`WERE DISCOVERED
`
`
`
`”0
`
`138
`
`
`
`
`
`
`I40
`
`RECEIVE MESSAGE(S)
`FROM MANAGER
`CONTAINING MORE
`KNOWLEDGE AND
`SCRIPT PROGRAMS
`USEFUL IN MANAGING
`THE DISCOVERED
`RESOURCES
`
`
`
`
`
`
`
`
`READ KNOWLEDGE DATABASE ON
`SERVER TO FIND NAME
`
`AND LOCATION OF SCRIPT
`
`PROGRAM THAT WILL
`SEARCH FOR THIS RESOURCE
`
`
`
`LOCATE THE SCRIPT PROGRAM
`
`
`
` HAS SCRIPT
`PROGRAM BEEN
`
`COMPILED YET
`
`
`I) COMPILE
`
`
`SCRIPT PROGRAM
`
`
` INTERPRET SCRIPT PROGRAM
`TO SEARCH FOR
`
`PRESENCE OF THE RESOURCE
`
` STORE RESULTS OF SEARCH
`
`01’]
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`011
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 10 0f 11
`
`5,978,594
`
`142
`
`FAGENT'S TIMER INDICATES THAT}
`MONITORING PROCEDURE
`I
`SHOULD BE EXECUTED
`J'
`
`:-
`
`I
`I
`:-
`
`MANAGER SOFTWARE SENDS 1I
`SIGNAL TO INITIATE
`If144
`MONITORING PROCEDURE
`J'
`
`AGENT’S RUN OUEUE SENDS JOB TO COMMAND EXECUTION MANAGER
`
`COMMAND EXECUTION MANAGER DETERMINES THAT JOB IS FOR
`PURPOSE AND MONITORING A RESOURCE AND THEREFORE
`PASSES IT TO PARAMETER AND RECOVERY ACTION MANAGER
`
`146
`
`148
`
`
`'F QCEFEJ'NG
`PARAMETER AND RECOVERY
`
`
`
`ACTION MANAGER DECIDES:
`TYPE
`
`
`IS JOB A SCRIPT LANGUAGE
`
`
`COMMAND TYPE OR AN OPERATING
`
`
`SYSTEM COMMAND TYPE?
`
`
`
`FIND AND EXECUTE
`OPERATING SYSTEM
`
`
`COMMANDS DESIGNATED
`
`IN KNOWLEDGE DATABASE
`
`
`AS BEING NECESSARY
`
`
`FOR COLLECTING THIS DATA
`
`
`[FRISTCRAIMT
`TYPE
`
`154
`
`FIND THE PERTINENT
`SCRIPT PROGRAM
`
`1
`
`56
`
`
`
`SCRIPT PROGRAM
`BEEN COMPILED
`o
`
`COMPILE
`SCRIPT PROGRAM
`
`
`
`
`152
`
`158
`
`160
`
`INTERPRET SCRIPT PROGRAM TO COLLECT DATA
`
`STORE VALUE OF COLLECTED DATA IN A VARIABLE
`
`152
`
`PARAMETER AND RECOVERY ACTION MANACER
`CHECKS VALUE AGAINST THRESHOLD
`INFORMATION FROM KNOWLEDGE DATABASE
`
`164
`
`
`
`IS
`
`RECOVERY
`
`PROCECURE INDICATED
`
`
` FIG. 9
`BASED ON VALUE AND
`THRESHOLDS
`
`
`?
`
`166
`
`EXECUTE RECOVERY
`PROCEDURE INDICATED IN
`KNOWLEDGE DATABASE
`
`
`
`SEND MESSAGE TO MANAGER INDICATING
`
`STATUS OF MONITORED RESOURCE
`
`170
`
`168
`
`012
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`012
`
`

`

`US. Patent
`
`N0V.2, 1999
`
`Sheet 11 0f 11
`
`5,978,594
`
`FIG. 70
`
`DETERMINE FROM READING
`KNOWLEDGE DATABASE WHICH
`RECOVERY ACTION
`
`SHOULD BE EXECUTED
`
`172
`
`
`
`COMMAND EXECUTION MANAGER
`DETERMINES IF INDICATED
`
`
`RECOVERY ACTION IS AN OPERATING
`
`
`SYSTEM COMMAND TYPE OR A
`SCRIPT LANGUAGE COMMAND TYPE
`
`
`IF
`OPERATING
`SYSTEM
`TYPE
`
`I74
`
`IF
`SCRIPT
`PROGRAM
`TYPE
`
`I78
`
`FIND THE PERTINENT
`SCRIPT PROGRAM
`
`
`
`SCRIPT PROGRAM
`
`BEEN COMPILED
`9
`
`FIND AND EXECUTE OPERATING
`SYSTEM COMMANDS INDICATED IN
`KNOWLEDGE DATABASE AS BEING
`NECESSARY TO TAKE
`
`
`
`THIS RECOVERY ACTION
`
`176
`
`,
`
`I
`
`NO
`
`COMPILE
`SCRIPT PROGRAM
`
`
`
`
`180
`
`
`
`
`
`182
`
`INTERPRET SCRIPT
`PROGRAM TO TAKE
`RECOVERY ACTION
`
`184
`
`013
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`013
`
`

`

`5,978,594
`
`1
`SYSTEM FOR MANAGING COMPUTER
`RESOURCES ACROSS A DISTRIBUTED
`COMPUTING ENVIRONMENT BY FIRST
`READING DISCOVERY INFORMATION
`ABOUT HOW TO DETERMINE SYSTEM
`RESOURCES PRESENCE
`
`RELATED APPLICATIONS
`
`This application is a continuation of application Ser. No.
`08/316,034, filed Sep. 30, 1994, now abandoned.
`
`FIELD OF THE INVENTION
`
`This invention relates generally to computer networks.
`More specifically,
`the invention relates to a method and
`apparatus for centrally monitoring and managing the
`computers, applications and other resources present in a
`distributed computing environment.
`
`BACKGROUND
`
`The data processing resources of business organizations
`are increasingly taking the form of a distributed computing
`environment in which data and processing are dispersed
`over a network comprising many interconnected, heteroge-
`neous and geographically remote computers. Among the
`reasons for this approach are: to offload non-mission-critical
`processing from the mainframe;
`to provide a pragmatic
`alternative to centralized corporate databases; to establish a
`single computing environment;
`to move control into the
`operating divisions of the company; and to avoid having a
`single point of failure. For example, many business entities
`have one client/server network installed in each regional
`office, in which a high-capacity computer system operates as
`the server supporting many lower-capacity desktop comput-
`ers. The servers in such a business entity are also commonly
`connected to one another by a higher-level network known
`as a wide area network. In this manner, users at any location
`within the business entity can theoretically access resources
`present in the company’s network regardless of where the
`resource is located.
`
`The flexibility gained for users with this type of arrange-
`ment comes with a price, however. It is very difficult to
`manage such a diverse and widely-dispersed network for
`many reasons. Servers installed in the wide area network are
`frequently not all of the same variety. One regional office
`may be using an IBM machine with a UNIX operating
`system, while another regional office may be using a WANG
`machine with a VMS operating system. Also, applications
`present on the servers throughout the network vary not only
`in terms of type, but also version number within an appli-
`cation type. Moreover, the applications present are changed
`frequently by users throughout the network, and failure
`events in such a network are usually difficult to catch until
`after a failure has already occurred.
`Therefore, a need exists for a network management sys-
`tem that will provide an increase in automation and effi-
`ciency for network management and a decrease in the
`complexity of such management. Also, a need exists for a
`solution that is easy to implement and maintain as installed
`applications and computers change.
`
`SUMMARY OF THE INVENTION
`
`The invention is a novel method and apparatus for man-
`aging a computer network. The method is intended to be
`used in any distributed computing environment in which two
`or more computer systems are connected by a network,
`
`65
`
`2
`including environments in which the networked computers
`are of different types. Amanager software system is installed
`on and runs on one of the networked computer systems
`designated as the network management computer system.
`The network management computer system acts as a central
`console for managing the entire network. An agent software
`system is installed on and runs on each of the other computer
`systems in the network. Each respective agent software
`system carries out tasks on the computer system in which it
`is installed, such as discovering which resources and appli-
`cations are present on that computer system, monitoring
`particular aspects of the resources and applications present
`on that computer system, and executing recovery actions
`automatically when such actions are warranted. Each agent
`also carries on a dialog of communication with the manager
`software system via the network, so that the central console
`on the network management computer system can provide a
`continuously updated display representing all resources and
`applications present throughout the network, as well as the
`state of each such resource or application. A knowledge
`module is installed on the central network management
`computer system, and then that computer system distributes
`knowledge throughout the network to the agent software
`systems. Such knowledge allows the agent software systems
`to operate in a semi-autonomous and automatic manner,
`relieving burdens from the central console operator. Also,
`the agent software systems make use of script programs
`written in a high-level interpretable language in order to
`execute certain procedures. Thus, only one agent software
`system need be installed on each server. As needs for agent
`software functionality changes or increases over time with
`changes in the network, new script programs may be devel-
`oped and distributed throughout the network to customize
`the agent software systems instead of replacing, modifying
`or duplicating the agent software systems themselves.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`FIG. 1 is a block diagram of a computer network config-
`ured to implement the method of the invention in a preferred
`manner.
`
`40
`
`FIG. 2 is a block diagram showing the preferred compo-
`nents of a manager software system for implementing the
`method of the invention.
`
`45
`
`FIG. 3 is a block diagram showing the preferred compo-
`nents of an agent software system for implementing the
`method of the invention.
`
`FIG. 4 is a diagrammatic illustration of the types of
`information that are preferably stored in a knowledge mod-
`ule.
`
`50
`
`FIGS. 5a and 5b comprise an exemplary listing of por-
`tions of an actual knowledge module.
`FIGS. 6a and 6b comprise an exemplary listing of por-
`tions of an actual script program.
`FIG. 7 is a flow diagram illustrating a preferred procedure
`for initializing a network for operating according to the
`method of the invention.
`
`55
`
`60
`
`FIG. 8 is a flow diagram illustrating a preferred procedure
`for discovering resources according to the method of the
`invention.
`
`FIG. 9 is a flow diagram illustrating a preferred procedure
`for monitoring resources according to the method of the
`invention.
`
`FIG. 10 is a flow diagram illustrating a preferred proce-
`dure for executing recovery actions according to the method
`of the invention.
`
`014
`
`ServiceNow's Exhibit No. 1001
`
`ServiceNow's Exhibit No. 1001
`
`014
`
`

`

`5,978,594
`
`3
`DETAILED DESCRIPTION OF THE DRAWINGS
`
`The components of the preferred apparatus for imple-
`menting the invention will now be described in detail with
`reference to FIGS. 1—6. Note that the word “resource,”
`wherever it is used in this specification, is intended in its
`broad sense to include, without limitation, hardware such as
`computers, printers, memory or other network devices,
`applications such as database management systems, and
`logical devices such as logical disk drives or filing systems.
`Note also that the words “agent” and “manager” refer to the
`agent software system and the manager software system,
`respectively.
`FIG. 1 illustrates a computer network configured to
`implement the method of the invention. A network manage-
`ment computer system 10 is coupled via network 12 to
`server computer system 14 and a plurality of other server
`computer systems. The hardware present
`in each of the
`computer systems may be of any conventional type such as
`is typically found on server computers in a client/server
`network environment. Moreover, the hardware configura-
`tion of each of the computer systems need not be the same.
`For example, network management computer system 10
`might be built around a computer sold by International
`Business Machines Corporation operating with the well-
`known UNIX operating system, while server computer
`system 14 might be built around a computer sold by Wang
`Corporation operating with the well-known VMS operating
`system. The other server computer systems in the network
`might be built around yet other platforms. In addition, all of
`the server computers in the network might be coupled to a
`variety of supported client computers such as desk-top
`computers and perhaps other resources. It is anticipated,
`however, that network management computer system 10 and
`each of the server computer systems in the network will be
`equipped with some sort of CPU 16, 18, some sort of
`conventional input/output equipment 20, 22 such as a key-
`board and a display monitor, some sort of conventional data
`storage device 24, 26 such as a disk or tape drive or CD
`ROM drive, some sort of random access memory (“RAM”)
`28, 29, and some sort of conventional network communi-
`cation hardware 30, 32 such as an ETHERNET interface
`unit for physically coupling the computer system to network
`12. Network 12 may be implemented using any conven-
`tional network protocol such as TCP/IP. In order to imple-
`ment the method of the invention, however: a manager
`software system 34 should be stored on storage device 24 in
`network management computer system 10; and one agent
`software system should be installed on each of the server
`computer systems in the network, such as agent software
`system 36 shown stored on storage device 26 in server
`computer system 14; at least one knowledge module 38
`should be stored on storage device 24 in network manage-
`ment computer system 10; and at least one script program
`40, 42 should be stored on each of the storage devices 24, 26
`throughout the computer network.
`FIG. 2 illustrates the preferred main components for
`implementing manager software system 34. Knowledge
`module parser 44 is responsible for accessing knowledge
`module 38 and parsing the information therein for use by
`knowledge database manager 46, which in turn creates and
`maintains a database 47 of knowledge that is more readily
`useable by manager software system 34 than would be the
`data stored in knowledge module 38. Object database man-
`ager 48 creates and maintains a database 49 representing all
`of the resources and applications (collectively, “objects”)
`present on the computer network, as well as information
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`pertaining to the state of those objects, in a form that will be
`readily useable by graphical user interface module 50.
`Databases 47 and 49 may be stored in RAM or on a storage
`device such as a hard disk. Graphical user interface 50 is
`responsible for communicating with display driver software
`in order to present visual representations of objects on the
`display of network management computer system 10. Such
`representations typically take the form of icons for objects.
`Also, graphical user interface module 50 coordinates the
`representation of pop-up windows for command menus and
`the display of requested or monitored data. Event manager
`52 is responsible for keeping a record of various occurrences
`throughout the computer network, such as the occurrence of
`alarm conditions and their resolution, for the purpose of
`record keeping and management convenience. Interface 54
`is for the purpose of interfacing with network management
`software other than the manager software system 34 and
`agent software system 36. For example, users of network
`management computer system 10 may make use of software
`such as Hewlett Packard Corporation’s OPENVIEW prod-
`uct for the purpose of monitoring low-level network condi-
`tions such as broken physical connections. While using such
`a third-party product,
`the user may open a window and
`request information from manager software system 34, in
`which case interface 54 will coordinate communication
`
`between manager software system 34 and such third party
`product. Communications module 56 is responsible for
`handling all communications to and from agent software
`systems installed throughout the computer network. Script
`program compiler 58 is used when the user of manager
`software system 34 wishes to develop script programs for
`use in managing a computer network according to the
`method of the invention. Kernel 60 represents all other
`miscellaneous functions within manager software system
`34, such as coordinating the action of the above-named
`modules and the communications between them.
`
`FIG. 3 illustrates the preferred main components of agent
`software system 36, and is representative of all such agent
`software systems installed throughout
`the computer net-
`work; provided, however, that before physically installing
`an agent software system on a particular server computer in
`executable form, the agent software system must of course
`