ManageEngine Log360 (2024)

A security information and event management (SIEM) solution ensures a healthy security posture for an organization's network by monitoring different types of data from the network. Log data records every activity happening on the device, and applications across the network. To assess the security posture of a network, SIEM solutions must collect and analyze different types of log data.

This article elaborates the different types of log data that you should collect and analyze using a SIEM solution to ensure network security.

Perimeter devices monitor and regulate traffic to and from the network. Firewalls, virtual private network (VPNs), intrusion detection systems (IDSs), and intrusion prevention systems (IPSs) are some of the perimeter devices. These devices generate logs containing a large amount of data, and perimeter device logs are vital for understanding the security events occurring in the network. Log data in the syslog format helps IT admins perform security audits, troubleshoot operational issues, and better understand the traffic passing through and from the corporate network.

2015-07-0611:35:26 ALLOW TCP 10.40.4.182 10.40.1.11 63064 135 0 - 0 0 0 - - - SEND

The log entry above specifies the time stamp of the event, followed by the action. In this instance, it indicates the day and time the firewall allowed traffic. It also contains information about the protocol used, as well as the IP addresses and port numbers of the source and destination. From log data like this, you can detect attempts to connect to ports that you do not use, indicating that the traffic is malicious.

2. Windows event logs

Windows event logs are a record of everything that happens on a Windows system. This log data is further classified into:

Windows application logs: These are events logged by the applications in the Windows operating system. For example, an error forcing the application to close is recorded in this application log.
Security logs: These are any events that may affect the security of the system. It includes failed login attempts and file deletion instances.
System logs: It contains events that are logged by the operating system. The logs indicate if processes and drivers were loaded successfully.
Directory service logs: It contains events logged by the Active Directory (AD) service. It records AD operations such as authentication and modification of privileges. These logs are available only for domain controllers.
DNS server logs: These are logs from domain name system (DNS) servers with information such as client IP addresses, the domain queried, and the record requested. It is available only for DNS servers.
File replication service logs: It contains events of domain controller replication. It is available only for domain controllers.

Why do you need to monitor Windows event logs?

To ensure server security: Most critical servers, such as file servers, and AD domain controllers, run on the Windows platform. It is essential to monitor this log data to understand what is happening to your critical resources.
Windows workstation security: Event logs provide valuable insights into the functioning of a workstation. By monitoring Windows event logs generated from a device, user activities can be monitored for anomalous behavior which can help detect attacks in the early stages. In case of an attack, the logs can help reconstruct the user's activities for forensic purposes.
To monitor hardware components: An analysis of Windows event logs helps diagnose problems with malfunctioning hardware components of a workstation by indicating the cause for malfunction.

Dissecting a typical Windows event log

3. Endpoint logs

Endpoints are devices that are connected across the network and communicate with other devices across servers. Some examples include desktops, laptops, smartphones, and printers. With organizations increasingly adopting remote work, endpoints create points of entry to the network that could be exploited by malicious actors.

Why do you need to monitor endpoint logs?

To monitor activities on removable disk drives: Removable disk drives are often vulnerable to malware installations and data exfiltration attempts. By monitoring endpoint logs, these attempts can be detected.
To monitor user activity: Users are required to abide by their organization's internal as well as external regulatory policies related to installation and use of software on their workstations. Endpoint logs can be used to monitor these policies and provide notifications if violations occur.

Dissecting a typical endpoint device log

Error 6/20/2019 5:00:45 PM Terminal Services- Printers 1111 None

The log above specifies that an error has occurred with the Terminal Services Easy Print driver. This is indicated by the error source, and the Event ID (1111). If a user faces issues while printing a file, the logs can be checked to understand the exact cause for the issue and resolve it.

4. Application logs

Businesses run on various applications such as databases, web server applications, and other in-house apps to perform specific functions. These applications are often vital for the effective functioning of the business. All of these applications generate log data that provide insights about what is happening within the applications.

5. Proxy logs

Proxy servers play an important role in an organization's network by providing privacy, regulating access, and saving bandwidth. Since all web requests and responses pass through the proxy server, proxy logs can reveal valuable information about usage statistics and the browsing behavior of endpoint users.

Why do you need to monitor proxy logs?

To baseline user behavior: Analyzing users' browsing activities from the proxy logs collected can help form a baseline of their behavior. Any deviation from the baseline might reveal a data breach, and indicate that further inspection is required.
To monitor the length of packets: Proxy logs can help monitor the length of packets exchanged through the proxy server. For example, a user repeatedly sending or receiving packets of the same length within a given interval of time might indicate a software update, or uncover malware exchanging signals with control servers.

Dissecting a typical proxy log

4/8/2020 2:20:55 PM User-001 192.168.10.10 GET https://wikipedia.com/

The log above specifies that User-001 requested pages from Wikipedia.com on the date and time indicated in the log. Analyzing the requests, URLs, and time stamps in the logs help detect patterns, and aids in evidence recovery in case of an event.

6.IoT logs

Internet of Things (IoT) refers to a network of physical devices that exchange data with other devices on the internet. These devices are embedded with sensors, processors, and software to enable data collection, processing, and transmission. Like endpoints, devices that make up an IoT system generate logs. Log data from IoT devices provides insights into the functioning of hardware components, such as microcontrollers, the firmware update requirements of the device, and the flow of data in and out of the device. A crucial part of logging data from IoT systems is the storage location of log data. These devices do not possess sufficient memory to store the logs. So, the logs must be forwarded to a centralized log management solution where they can be stored for extended periods of time. The SIEM solution then analyzes the logs to troubleshoot errors and detect security threats.

The logs from all of the above sources are usually forwarded to the centralized logging solution that correlates and analyzes the data to provide a security overview of your network. The logs are stored and transmitted in different formats, such as CSV, JSON, Key Value Pair, and Common Event Format.

CSV

CSV is a file format that stores values in a comma-separated format. It is a plain-text file format, which allows CSV files to be easily imported into a storage database, regardless of the software used. Because CSV files are not hierarchical or object-oriented, they are also easier to convert to other file types.

JSON

JavaScript Object Notation (JSON) is a text-based format for storing data. It is a structured format, which makes it easier to analyze the stored logs. It can also be queried for specific fields. These additional features make JSON a very reliable format for log management.

Key Value Pair

A key-value pair consists of two elements: a key and a value mapped to it. The key is a constant, and the value is variable across different entries. The formatting involves grouping similar sets of data under a common key. By running the query for a specific key, all the data under that key can be extracted.

Common Event Format

Common Event Format, commonly referenced as CEF, is a log management format that promotes interoperability by making it easier to collect and store log data from different devices and applications. It uses the syslog message format. The most widely used logging format, it is supported by a variety of vendors and software platforms, and consists of a CEF header and a CEF extension that contains log data in key-value pairs.

These are the different types of log data and their formats. Manually collecting these logs from all the different sources in a network and correlating them is a tedious and time-consuming process. A SIEM solution can help you with this. A SIEM solution analyzes the logs collected from different sources, correlates the log data, and provides insights to help organizations detect and recover from cyberattacks.