Log Management

Best practices frameworks are emerging to help IT organizations meet compliance objectives. Industry experts agree on the importance of log management as a cornerstone of any organization’s risk management strategy. Log data can provide a complete real-time and historical record of access, activity, and configuration changes for applications, servers, and network devices. It can also be used to aid with security and business policy validation. Lastly, IT managers can use log data to receive early warning of potential security and performance problems and can mine log data for root cause analysis to aid in system recovery and damage cleanup after a security or performance incident. Considering all of its uses, log data management not only assists in achieving corporate compliance, it also reduces the risks of legal exposure from security breaches and of costly network downtime.

Log Management (LM) comprises an approach to dealing with large volumes of computer-generated log messages(also known as audit records, audit trails, event-logs, etc). LM covers log collection, centralized aggregation, long-term retention and log analysis (in real-time and in bulk after storage).

Systems administrators usually perform LM analysis for reasons of security, of operations (such as system or network administration or of regulatory compliance.

Effectively analyzing large volumes of diverse logs can pose many challenges — such as huge log-volumes (reaching hundreds of gigabytes of data per day for a large organization), log-format diversity, undocumented proprietary log-formats (that resist analysis) as well as the presence of false log records in some types of logs (such as intusion-detection logs).

Users and potential users of LM can build their own log management and intelligence tools, assemble the functionality from various open-source components, or acquire (sub-)systems from commercial vendors.

Deployment life-cycle

One view] of assessing the maturity of an organization in terms of the deployment of log-management tools might use] successive categories such as:

1. Level 1: in the initial stages, organizations use different log-analyzers for analyzing the logs in the devices on the security-perimeter. They aim to identify the patterns of attack on the perimeter infrastructure of the organization.

2. Level 2: with increased use of integrated computing, organizations mandate logs to identify the access and usage of confidential data within the security-perimeter.

3. Level 3: at the next level of maturity, the log analyzer can track and monitor the performance and availability of systems at the level of the enterprise — especially of those information-assets whose availability organizations regard as vital.

4. Level 4: organizations integrate the logs of various business-applications into an enterprise log manager for better value proposition.

5. Level 5: organizations merge the physical-access monitoring and the logical-access monitoring into a single view.