Keeping security logs is a balance between wanting to have documentation if it’s needed and keeping way too much for too long. Fortunately, we have some guidance that considers best-practices security without the need to build your own data center to house the logs.
What is log retention?
Log files are detailed, text-based records of events. Many devices and applications generate them, including anti malware, system utilities, firewalls, intrusion detection and prevention systems (IDSes/IPSes), servers, workstations, and networking equipment.
Log retention is the regular archiving of event logs, particularly those significant to cyber security. The logs provided are essential for security monitoring. While retaining logs for extended periods provides more historical information, holding on to that data can become expensive.
Log files provide a crucial audit trail and can help monitor activity within the IT infrastructure, identify policy violations, pinpoint fraudulent or unusual activity, and highlight security incidents. Security teams can use them to detect and respond to indicators of compromise, investigate and analyze where an attack is coming or came from, and establish how it has affected IT resources.
Log retention guidance
Developing a cyber security log policy is an integral part of your security program. Log retention varies wildly, with some companies keeping them only for a month and others holding on to them for seven years.
One year is a commonly agreed upon standard for long retention, meeting most regulations, including the Federal Information Security Management Act of 2002 (FISMA), the Health Insurance Portability and Accountability Act of 1996 (HIPAA), the Sarbanes-Oxley Act of 2002 (SOX), the Gramm-Leach-Bliley Act (GLBA), and the Payment Card Industry Data Security Standard (PCI DSS).
NIST 800-171 offers general guidance for contractors regarding logging requirements:
Basic Security Requirements:
3.3.1 Create and retain system audit logs and records to the extent needed to enable the monitoring, analysis, investigation, and reporting of unlawful or unauthorized system activity.
3.3.2 Ensure that the actions of individual system users can be uniquely traced to those users so they can be held accountable for their actions.
Derived Security Requirements:
3.3.3 Review and update logged events.
3.3.4 Alert in the event of an audit logging process failure.
3.3.5 Correlate audit record review, analysis, and reporting processes for investigation and response to indications of unlawful, unauthorized, suspicious, or unusual activity.
3.3.6 Provide audit record reduction and report generation to support on-demand analysis and reporting.
3.3.7 Provide a system capability that compares and synchronizes internal system clocks with an authoritative source to generate timestamps for audit records.
3.3.8 Protect audit information and audit logging tools from unauthorized access, modification, and deletion.
3.3.9 Limit management of audit logging functionality to a subset of privileged users.
Although NIST explains the basic security requirements, the guidance does not explicitly mention the length of time needed for log retention. Therefore, the best practice for contractors is to follow the requirements of their specific agency.
For example, the log retention requirements for DoD contractors that may process, store, or transmit DoD CUI come from DFARS 252.204-7012. It says that all DoD contractors in this category must comply with all controls in 800-171, and Paragraph E within that clause requires contractors to retain logs for at least 90 days. Additionally, it says that if the contractor is using a cloud-based service to retain logs, that cloud vendor must also retain the logs it stores relevant to the client’s in-scope CUI network for 90 days.
How to manage cyber security logs
Security log management comprises the generation, transmission, storage, analysis, and disposal of security log data, ensuring its confidentiality, integrity, and availability.
Organizations that fail to collect, store, and analyze system events are missing a critical piece of modern security practice. Log management allows companies to perform general audits, establish baselines, and identify operational trends and longer-term problems. This is why various laws and standards, such as HIPAA and DFARS 252.204-7012, require log management for compliance and reporting.
Because logs come from multiple endpoints and different sources and formats, they require normalizing for easy searching, comparison, and readability. The systems and media which share and retain logs must be highly secure with tightly controlled access. In addition, they must be capable of processing large amounts of data without impairing overall system performance.
The security events an organization captures depend on the industry and relevant legal requirements. However, several events should always be captured and logged to ensure user accountability and help companies detect, understand, and recover from an attack, including:
- Authentication successes and failures
- Access control successes and failures
- Session activity, such as files and applications used, particularly system utilities
- Changes in user privileges
- Processes starting or stopping
- Changes to configuration settings
- Software installed or deleted
- Devices attached or detached
- System or application errors and alerts
- Alerts from security controls, such as firewalls, IDSes, and antimalware
Fault logging—faults generated by the system and the applications running on it—can be used to find issues with a system or application and identify indications of faulty equipment and should include:
- Date and time
- User and/or device ID
- Network address and protocol
- Location when possible
- Event or activity
Compromised or inaccurate logs can hamper investigations into suspicious events, undermine their credibility, and invalidate disciplinary and court actions.
One way to ensure trustworthy logs is to use synchronized system clocks, giving every log entry an accurate timestamp. This involves obtaining a reference time from an external source, combined with a network time protocol, to sync internal clocks. Always record the time of an event in a consistent format, such as Coordinated Universal Time. For additional security, add a checksum.
Here are three other best practices to follow:
1. Actively use logs for monitoring
Collected data is worthless unless it is monitored, analyzed, and acted on. Logging and auditing ensure that users are only performing authorized activities. These processes also play a role in preventing inappropriate activity, as well as ensuring hostile actions are tracked down and stopped.
2. Give admins and sysops extra scrutiny
One area that requires extra consideration is administrator and system operator (sysop) activities. These users have powerful privileges, and their actions must be carefully recorded and checked. To that end, these users should not be allowed physical or network access to logs of their activities. Additionally, those tasked with reviewing logs should be independent of the people, activities, and logs under review.
3. Use logging tools
Due to the volume of incoming data organizations confront daily, most need a dedicated log management system to improve management, event correlation, and analysis. A specialized system also increases dashboard data and report quality.
Security Information and Event Management (SIEM) solutions are a common approach used to aggregate log data from multiple sources. SIEM systems can parse and analyze data in real-time to identify deviations from common actions in the network’s systems. If an anomaly is detected, SIEM systems can generate alerts, possibly activating additional security mechanisms. They can be rules-based, often employing a statistical correlation engine to establish relationships between event log entries. Advanced systems further rely on user and entity behavior analytics, security orchestration, automation, and response tools.
Security monitoring policies and log retention are just two elements of a comprehensive cybersecurity plan. DTS can assess your unique needs and industry requirements to determine how your company should address log retention. We offer GCC High licensing and integration as well as other secure data storage solutions.
