Special Publication 800-92
`
`Guide to Computer Security
`Log Management
`
`Recommendations of the National Institute
`of Standards and Technology
`
`
`
`Karen Kent
`Murugiah Souppaya
`
`
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 1
`
`

`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 2
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 2
`
`

`
`Guide to Computer Security Log
`Management
`
`Recommendations of the National
`Institute of Standards and Technology
`
`Karen Kent
`Murugiah Souppaya
`
`
`
`
`
`NIST Special Publication 800-92
`
`C O M P U T E R S E C U R I T Y
`
`Computer Security Division
`Information Technology Laboratory
`National Institute of Standards and Technology
`Gaithersburg, MD 20899-8930
`
`September 2006
`
`
`
`
`
`
`
`U.S. Department of Commerce
`
`Carlos M. Gutierrez, Secretary
`
`Technology Administration
`
`Robert C. Cresanti, Under Secretary of Commerce
`for Technology
`National Institute of Standards and Technology
`William Jeffrey, Director
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 3
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`
`Reports on Computer Systems Technology
`
`The Information Technology Laboratory (ITL) at the National Institute of Standards and Technology
`(NIST) promotes the U.S. economy and public welfare by providing technical leadership for the nation’s
`measurement and standards infrastructure. ITL develops tests, test methods, reference data, proof of
`concept implementations, and technical analysis to advance the development and productive use of
`information technology. ITL’s responsibilities include the development of technical, physical,
`administrative, and management standards and guidelines for the cost-effective security and privacy of
`sensitive unclassified information in Federal computer systems. This Special Publication 800-series
`reports on ITL’s research, guidance, and outreach efforts in computer security and its collaborative
`activities with industry, government, and academic organizations.
`
`
`
`
`National Institute of Standards and Technology Special Publication 800-92
`Natl. Inst. Stand. Technol. Spec. Publ. 800-92, 72 pages (September 2006)
`
`
`
`
`
`
`Certain commercial entities, equipment, or materials may be identified in this
`document in order to describe an experimental procedure or concept adequately.
`Such identification is not intended to imply recommendation or endorsement by the
`National Institute of Standards and Technology, nor is it intended to imply that the
`entities, materials, or equipment are necessarily the best available for the purpose.
`
`
`
`
`
` ii
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 4
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`Acknowledgements
`
`The authors, Karen Kent and Murugiah Souppaya of the National Institute of Standards and Technology
`(NIST), wish to thank their colleagues who reviewed drafts of this document and contributed to its
`technical content, especially Bill Burr, Elizabeth Chew, Tim Grance, Bill MacGregor, Stephen Quinn,
`and Matthew Scholl of NIST, and Stephen Green, Joseph Nusbaum, Angela Orebaugh, Dennis Pickett,
`and Steven Sharma of Booz Allen Hamilton. The authors particularly want to thank Anton Chuvakin of
`LogLogic and Michael Gerdes for their careful review and many contributions to improving the quality of
`this publication. The authors would also like to express their thanks to security experts Kurt Dillard of
`Microsoft, Dean Farrington of Wells Fargo Bank, Raffael Marty of ArcSight, Greg Shipley of Neohapsis,
`and Randy Smith of the Monterey Technology Group, as well as representatives from the Department of
`Energy, the Department of Health and Human Services, the Department of Homeland Security, the
`Department of State, the Department of Treasury, the Environmental Protection Agency, the National
`Institutes of Health, and the Social Security Administration, for their valuable comments and suggestions.
`
`
`
`All names are registered trademarks or trademarks of their respective companies.
`
`Trademarks
`
`
`
` iii
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 5
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`Table of Contents
`
`Executive Summary............................................................................................................ES-1
`
`2.
`
`1.
`
`Introduction ................................................................................................................... 1-1
`1.1 Authority................................................................................................................ 1-1
`1.2 Purpose and Scope............................................................................................... 1-1
`1.3 Audience ............................................................................................................... 1-1
`1.4 Publication Structure ............................................................................................. 1-1
`Introduction to Computer Security Log Management ................................................ 2-1
`2.1 The Basics of Computer Security Logs.................................................................. 2-1
`2.1.1 Security Software....................................................................................... 2-2
`2.1.2 Operating Systems..................................................................................... 2-4
`2.1.3 Applications................................................................................................ 2-4
`2.1.4 Usefulness of Logs..................................................................................... 2-6
`2.2 The Need for Log Management............................................................................. 2-7
`2.3 The Challenges in Log Management..................................................................... 2-8
`2.3.1 Log Generation and Storage ...................................................................... 2-8
`2.3.2 Log Protection............................................................................................ 2-9
`2.3.3 Log Analysis............................................................................................. 2-10
`2.4 Meeting the Challenges....................................................................................... 2-10
`2.5 Summary............................................................................................................. 2-11
`3. Log Management Infrastructure................................................................................... 3-1
`3.1 Architecture........................................................................................................... 3-1
`3.2 Functions............................................................................................................... 3-3
`3.3 Syslog-Based Centralized Logging Software......................................................... 3-5
`3.3.1 Syslog Format............................................................................................ 3-5
`3.3.2 Syslog Security .......................................................................................... 3-7
`3.4 Security Information and Event Management Software ......................................... 3-9
`3.5 Additional Types of Log Management Software................................................... 3-10
`3.6 Summary............................................................................................................. 3-11
`4. Log Management Planning........................................................................................... 4-1
`4.1 Define Roles and Responsibilities ......................................................................... 4-1
`4.2 Establish Logging Policies..................................................................................... 4-3
`4.3 Ensure that Policies Are Feasible.......................................................................... 4-7
`4.4 Design Log Management Infrastructures............................................................... 4-9
`4.5 Summary............................................................................................................. 4-10
`5. Log Management Operational Processes.................................................................... 5-1
`5.1 Configure Log Sources.......................................................................................... 5-1
`5.1.1 Log Generation .......................................................................................... 5-1
`5.1.2 Log Storage and Disposal.......................................................................... 5-2
`5.1.3 Log Security............................................................................................... 5-4
`5.2 Analyze Log Data.................................................................................................. 5-5
`5.2.1 Gaining an Understanding of Logs............................................................. 5-5
`5.2.2 Prioritizing Log Entries ............................................................................... 5-6
`5.2.3 Comparing System-Level and Infrastructure-Level Analysis....................... 5-7
`
` iv
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 6
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`5.3 Respond to Identified Events................................................................................. 5-8
`5.4 Manage Long-Term Log Data Storage .................................................................. 5-9
`5.5 Provide Other Operational Support...................................................................... 5-10
`5.6 Perform Testing and Validation ........................................................................... 5-10
`5.7 Summary............................................................................................................. 5-11
`
`
`List of Appendices
`
`Appendix A— Glossary ........................................................................................................A-1
`
`Appendix B— Acronyms ......................................................................................................B-1
`
`Appendix C— Tools and Resources....................................................................................C-1
`
`Appendix D— Index ..............................................................................................................D-1
`
`
`
`List of Figures
`
`Figure 2-1. Security Software Log Entry Examples ................................................................ 2-3
`Figure 2-2. Operating System Log Entry Example ................................................................. 2-4
`Figure 2-3. Web Server Log Entry Examples ......................................................................... 2-6
`Figure 3-1. Examples of Syslog Messages ............................................................................ 3-6
`
`
`
`Table 4-1. Examples of Logging Configuration Settings......................................................... 4-6
`
`List of Tables
`
`
`
` v
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 7
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`This page has been left blank intentionally.
`
` vi
`
`
`
`
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 8
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`Executive Summary
`
`A log is a record of the events occurring within an organization’s systems and networks. Logs are
`composed of log entries; each entry contains information related to a specific event that has occurred
`within a system or network. Many logs within an organization contain records related to computer
`security. These computer security logs are generated by many sources, including security software, such
`as antivirus software, firewalls, and intrusion detection and prevention systems; operating systems on
`servers, workstations, and networking equipment; and applications.
`
`The number, volume, and variety of computer security logs have increased greatly, which has created the
`need for computer security log management—the process for generating, transmitting, storing, analyzing,
`and disposing of computer security log data. Log management is essential to ensuring that computer
`security records are stored in sufficient detail for an appropriate period of time. Routine log analysis is
`beneficial for identifying security incidents, policy violations, fraudulent activity, and operational
`problems. Logs are also useful when performing auditing and forensic analysis, supporting internal
`investigations, establishing baselines, and identifying operational trends and long-term problems.
`Organizations also may store and analyze certain logs to comply with Federal legislation and regulations,
`including the Federal Information Security Management Act of 2002 (FISMA), the Health Insurance
`Portability and Accountability Act of 1996 (HIPAA), the Sarbanes-Oxley Act of 2002 (SOX), the
`Gramm-Leach-Bliley Act (GLBA), and the Payment Card Industry Data Security Standard (PCI DSS).
`
`A fundamental problem with log management that occurs in many organizations is effectively balancing a
`limited quantity of log management resources with a continuous supply of log data. Log generation and
`storage can be complicated by several factors, including a high number of log sources; inconsistent log
`content, formats, and timestamps among sources; and increasingly large volumes of log data. Log
`management also involves protecting the confidentiality, integrity, and availability of logs. Another
`problem with log management is ensuring that security, system, and network administrators regularly
`perform effective analysis of log data. This publication provides guidance for meeting these log
`management challenges.
`
`Implementing the following recommendations should assist in facilitating more efficient and effective log
`management for Federal departments and agencies.
`
`Organizations should establish policies and procedures for log management.
`
`To establish and maintain successful log management activities, an organization should develop standard
`processes for performing log management. As part of the planning process, an organization should define
`its logging requirements and goals. Based on those, an organization should then develop policies that
`clearly define mandatory requirements and suggested recommendations for log management activities,
`including log generation, transmission, storage, analysis, and disposal. An organization should also
`ensure that related policies and procedures incorporate and support the log management requirements and
`recommendations. The organization’s management should provide the necessary support for the efforts
`involving log management planning, policy, and procedures development.
`
`Requirements and recommendations for logging should be created in conjunction with a detailed analysis
`of the technology and resources needed to implement and maintain them, their security implications and
`value, and the regulations and laws to which the organization is subject (e.g., FISMA, HIPAA, SOX).
`Generally, organizations should require logging and analyzing the data that is of greatest importance, and
`also have non-mandatory recommendations for which other types and sources of data should be logged
`and analyzed if time and resources permit. In some cases, organizations choose to have all or nearly all
`log data generated and stored for at least a short period of time in case it is needed, which favors security
`
`ES-1
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 9
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`considerations over usability and resource usage, and also allows for better decision-making in some
`cases. When establishing requirements and recommendations, organizations should strive to be flexible
`since each system is different and will log different amounts of data than other systems.
`
`The organization’s policies and procedures should also address the preservation of original logs. Many
`organizations send copies of network traffic logs to centralized devices, as well as use tools that analyze
`and interpret network traffic. In cases where logs may be needed as evidence, organizations may wish to
`acquire copies of the original log files, the centralized log files, and interpreted log data, in case there are
`any questions regarding the fidelity of the copying and interpretation processes. Retaining logs for
`evidence may involve the use of different forms of storage and different processes, such as additional
`restrictions on access to the records.
`
`Organizations should prioritize log management appropriately throughout the organization.
`
`After an organization defines its requirements and goals for the log management process, it should then
`prioritize the requirements and goals based on the organization’s perceived reduction of risk and the
`expected time and resources needed to perform log management functions. An organization should also
`define roles and responsibilities for log management for key personnel throughout the organization,
`including establishing log management duties at both the individual system level and the log management
`infrastructure level.
`
`Organizations should create and maintain a log management infrastructure.
`
`A log management infrastructure consists of the hardware, software, networks, and media used to
`generate, transmit, store, analyze, and dispose of log data. Log management infrastructures typically
`perform several functions that support the analysis and security of log data. After establishing an initial
`log management policy and identifying roles and responsibilities, an organization should next develop
`one or more log management infrastructures that effectively support the policy and roles. Organizations
`should consider implementing log management infrastructures that includes centralized log servers and
`log data storage. When designing infrastructures, organizations should plan for both the current and
`future needs of the infrastructures and the individual log sources throughout the organization. Major
`factors to consider in the design include the volume of log data to be processed, network bandwidth,
`online and offline data storage, the security requirements for the data, and the time and resources needed
`for staff to analyze the logs.
`
`Organizations should provide proper support for all staff with log management responsibilities.
`
`To ensure that log management for individual systems is performed effectively throughout the
`organization, the administrators of those systems should receive adequate support. This should include
`disseminating information, providing training, designating points of contact to answer questions,
`providing specific technical guidance, and making tools and documentation available.
`
`Organizations should establish standard log management operational processes.
`
`The major log management operational processes typically include configuring log sources, performing
`log analysis, initiating responses to identified events, and managing long-term storage. Administrators
`have other responsibilities as well, such as the following:
`
`(cid:31) Monitoring the logging status of all log sources
`(cid:31) Monitoring log rotation and archival processes
`
`ES-2
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 10
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`(cid:31) Checking for upgrades and patches to logging software, and acquiring, testing, and deploying
`them
`(cid:31) Ensuring that each logging host’s clock is synched to a common time source
`(cid:31) Reconfiguring logging as needed based on policy changes, technology changes, and other factors
`(cid:31) Documenting and reporting anomalies in log settings, configurations, and processes.
`
`
`
`ES-3
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 11
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`This page has been left blank intentionally.
`
`ES-4
`
`
`
`
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 12
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`1.
`
`Introduction
`
`1.1 Authority
`
`The National Institute of Standards and Technology (NIST) developed this document in furtherance of its
`statutory responsibilities under the Federal Information Security Management Act (FISMA) of 2002,
`Public Law 107-347.
`
`NIST is responsible for developing standards and guidelines, including minimum requirements, for
`providing adequate information security for all agency operations and assets; but such standards and
`guidelines shall not apply to national security systems. This guideline is consistent with the requirements
`of the Office of Management and Budget (OMB) Circular A-130, Section 8b(3), “Securing Agency
`Information Systems,” as analyzed in A-130, Appendix IV: Analysis of Key Sections. Supplemental
`information is provided in A-130, Appendix III.
`
`This guideline has been prepared for use by Federal agencies. It may be used by nongovernmental
`organizations on a voluntary basis and is not subject to copyright, though attribution is desired.
`
`Nothing in this document should be taken to contradict standards and guidelines made mandatory and
`binding on Federal agencies by the Secretary of Commerce under statutory authority, nor should these
`guidelines be interpreted as altering or superseding the existing authorities of the Secretary of Commerce,
`Director of the OMB, or any other Federal official.
`
`1.2 Purpose and Scope
`
`This publication seeks to assist organizations in understanding the need for sound computer security log
`management. It provides practical, real-world guidance on developing, implementing, and maintaining
`effective log management practices throughout an enterprise. The guidance in this publication covers
`several topics, including establishing log management infrastructures, and developing and performing
`robust log management processes throughout an organization. The publication presents log management
`technologies from a high-level viewpoint, and it is not a step-by-step guide to implementing or using log
`management technologies.
`
`1.3 Audience
`
`This publication has been created for computer security staff and program managers; system, network,
`and application administrators; computer security incident response teams; and others who are responsible
`for performing duties related to computer security log management.
`
`1.4 Publication Structure
`
`The remainder of this publication is organized into four major sections. Section 2 provides an
`introduction to computer security log management, including an explanation of log management needs an
`organization might have and the challenges involved in log management. Section 3 discusses the
`components, architectures, and functions of log management infrastructures. Section 4 provides
`recommendations for planning log management, such as defining roles and responsibilities and creating
`feasible logging policies. Section 5 explains the processes that an organization should develop and
`perform for log management operations.
`
`The publication also contains several appendices with supporting material. Appendices A and B contain a
`glossary and acronym list, respectively. Appendix C lists tools and online and print resources that are
`
` 1-1
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 13
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`useful references for gaining a better understanding of log management. Appendix D contains an index
`for the publication.
`
`
`
` 1-2
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 14
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`2.
`
`Introduction to Computer Security Log Management
`
`A log is a record of the events occurring within an organization’s systems and networks. Logs are
`composed of log entries; each entry contains information related to a specific event that has occurred
`within a system or network. Originally, logs were used primarily for troubleshooting problems, but logs
`now serve many functions within most organizations, such as optimizing system and network
`performance, recording the actions of users, and providing data useful for investigating malicious activity.
`Logs have evolved to contain information related to many different types of events occurring within
`networks and systems. Within an organization, many logs contain records related to computer security;
`common examples of these computer security logs are audit logs that track user authentication attempts
`and security device logs that record possible attacks. This guide addresses only those logs that typically
`contain computer security-related information.1
`
`Because of the widespread deployment of networked servers, workstations, and other computing devices,
`and the ever-increasing number of threats against networks and systems, the number, volume, and variety
`of computer security logs has increased greatly. This has created the need for computer security log
`management, which is the process for generating, transmitting, storing, analyzing, and disposing of
`computer security log data. This section of the document discusses the needs and challenges in computer
`security log management. Section 2.1 explains the basics of computer security logs. Section 2.2
`discusses the laws, regulations, and operational needs involved with log management. Section 2.3
`explains the most common log management challenges, and Section 2.4 offers high-level
`recommendations for meeting them.
`
`2.1 The Basics of Computer Security Logs
`
`Logs can contain a wide variety of information on the events occurring within systems and networks.2
`This section describes the following categories of logs of particular interest:
`
`(cid:31) Security software logs primarily contain computer security-related information. Section 2.1.1
`describes them.
`(cid:31) Operating system logs (described in Section 2.1.2) and application logs (described in Section
`2.1.3) typically contain a variety of information, including computer security-related data.
`Under different sets of circumstances, many logs created within an organization could have some
`relevance to computer security. For example, logs from network devices such as switches and wireless
`access points, and from programs such as network monitoring software, might record data that could be of
`use in computer security or other information technology (IT) initiatives, such as operations and audits, as
`well as in demonstrating compliance with regulations. However, for computer security these logs are
`generally used on an as-needed basis as supplementary sources of information. This document focuses on
`the types of logs that are most often deemed to be important by organizations in terms of computer
`security. Organizations should consider the value of each potential source of computer security log data
`when designing and implementing a log management infrastructure.
`
`Most of the sources of the log entries run continuously, so they generate entries on an ongoing basis.
`However, some sources run periodically, so they generate entries in batches, often at regular intervals.
`
`1
`For the remainder of this document, the terms “log” and “computer security log” are interchangeable, except where
`otherwise noted.
`If the logs contain personally identifiable information—information that could be used to identify individuals, such as social
`security numbers—the organization should ensure that the privacy of the log information is properly protected. The people
`responsible for privacy for an organization should be consulted as part of log management planning.
`
`2
`
` 2-1
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 15
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`This section notes any log sources that work in batch mode because this can have a significant impact on
`the usefulness of their logs for incident response and other time-sensitive efforts.
`
`2.1.1
`
`Security Software
`
`Most organizations use several types of network-based and host-based security software to detect
`malicious activity, protect systems and data, and support incident response efforts. Accordingly, security
`software is a major source of computer security log data. Common types of network-based and host-
`based security software include the following:
`
`(cid:31) Antimalware Software. The most common form of antimalware software is antivirus software,
`which typically records all instances of detected malware, file and system disinfection attempts,
`and file quarantines.3 Additionally, antivirus software might also record when malware scans
`were performed and when antivirus signature or software updates occurred. Antispyware
`software and other types of antimalware software (e.g., rootkit detectors) are also common
`sources of security information.
`(cid:31) Intrusion Detection and Intrusion Prevention Systems. Intrusion detection and intrusion
`prevention systems record detailed information on suspicious behavior and detected attacks, as
`well as any actions intrusion prevention systems performed to stop malicious activity in progress.
`Some intrusion detection systems, such as file integrity checking software, run periodically
`instead of continuously, so they generate log entries in batches instead of on an ongoing basis.4
`(cid:31) Remote Access Software. Remote access is often granted and secured through virtual private
`networking (VPN). VPN systems typically log successful and failed login attempts, as well as
`the dates and times each user connected and disconnected, and the amount of data sent and
`received in each user session. VPN systems that support granular access control, such as many
`Secure Sockets Layer (SSL) VPNs, may log detailed information about the use of resources.
`(cid:31) Web Proxies. Web proxies are intermediate hosts through which Web sites are accessed. Web
`proxies make Web page requests on behalf of users, and they cache copies of retrieved Web
`pages to make additional accesses to those pages more efficient. Web proxies can also be used to
`restrict Web access and to add a layer of protection between Web clients and Web servers. Web
`proxies often keep a record of all URLs accessed through them.
`(cid:31) Vulnerability Management Software. Vulnerability management software, which includes
`patch management software and vulnerability assessment software, typically logs the patch
`installation history and vulnerability status of each host, which includes known vulnerabilities
`and missing software updates.5 Vulnerability management software may also record additional
`information about hosts’ configurations. Vulnerability management software typically runs
`occasionally, not continuously, and is likely to generate large batches of log entries.
`(cid:31) Authentication Servers. Authentication servers, including directory servers and single sign-on
`servers, typically log each authentication attempt, including its origin, username, success or
`failure, and date and time.
`
`
`
`3
`See NIST SP 800-83, Guide to Malware Incident Prevention and Handling, for more information on antivirus software.
`The publication is available at http://csrc.nist.gov/publications/nistpubs/.
`For more information on intrusion detection systems, see NIST SP 800-94 (DRAFT), Guide to Intrusion Detection and
`Prevention Systems, which is available at http://csrc.nist.gov/publications/nistpubs/.
`5 NIST SP 800-40 version 2, Creating a Patch and Vulnerability Management Program, contains guidance on vulnerability
`management software. SP 800-40 version 2 can be downloaded from http://csrc.nist.gov/publications/nistpubs/.
`
`4
`
` 2-2
`
`BLUE COAT SYSTEMS - Exhibit 1049 Page 16
`
`

`
`GUIDE TO COMPUTER SECURITY LOG MANAGEMENT
`
`(cid:31) Routers. Routers may be configured to permit or block certain types of network traffic based on
`a policy. Routers that block traffic are usually configured to log only the most basic
`characteristics of blocked activity.
`(cid:31) Firewalls. Like routers, firewalls permit or block activity based on a policy; however, firewalls
`use much more sophisticated methods to examine network traffic.6 Firewalls can also track the
`state of network traffic and perform content inspection. Firewalls tend to have more complex
`policies and generate more detailed logs of activity than routers.
`(cid:31) Network Quarantine Servers. Some organizations check each remote host’s security posture
`before allowing it to join the network. This is often done through a network quarantine ser