Going Beyond the Basics: Specific Metrics to Evaluate SIEM Solutions

SIEM technology has become an essential part of modern cybersecurity operations, enabling organizations to detect and respond to security incidents in real-time and ensuring that data is protected against malicious attacks.

However, most articles or blogs about SIEM technology discuss topics in very general terms, such as data collection, normalization, correlation, analysis, alerting, and reporting, integration with other security tools, user and entity behavior analytics (UEBA), centralized monitoring, threat detection, incident response, compliance management, and forensic analysis. While these topics are important, they do not provide specific metrics or examples to help readers better understand the concepts.

To truly understand the capabilities of a SIEM solution, it is essential to examine its features and capabilities in more detail. This requires a deeper understanding of the underlying technology and the specific needs of your organization.

As the SIEM market continues to evolve, it is becoming more challenging for organizations to choose the right solution that meets their specific needs. Many articles and blogs provide a generic overview of SIEM features, but they do not provide enough information to differentiate between SIEM solutions effectively. Therefore, it is crucial to look for specific and measurable metrics to evaluate SIEM products.

One metric to consider is the storage efficiency of the SIEM solution. For example, if three SIEM products have the same EPS values and log sources, but one uses 200 TB disk, another uses 80 TB disk, and the third uses only 5 TB disk but keeps the logs for at least one year or longer, the third product would be more efficient for incident response and research purposes.

Another metric to consider is the detection capability of the SIEM solution. Different SIEM researchers may suggest various indicators for detecting attacks and suspicious events. For example, some researchers may suggest capturing only brute-force attempts or escalating user privileges, while others may suggest advanced use cases such as:

• Detect an abnormal number of failed login attempts during a day

• Detect an abnormal ratio of total failed sessions to successful ones in daily network events

• Detect if the user logging into the system at this hour is abnormal

• Detecting data loss: Monitor the logs of a database table for any anomalies between the number of inserts in the logs and the number of rows added to the table. If the number of inserts in the logs is significantly lower than the number of rows added to the table, this could indicate potential data loss or deletion.

• Detecting unauthorized data access: Use logs to track access to a sensitive database table and compare the number of inserts in the logs to the number of rows added to the table. If there is a significant difference between the two, generate an alert to indicate potential unauthorized access to the data.

• Detecting data tampering: Monitor the logs of a database table for any anomalies between the number of inserts in the logs and the number of rows added to the table. If there is a discrepancy between the two numbers, generate an alert to notify security teams of potential data tampering.

• Generate an alert if a file containing personal data is copied to a shared path that is accessible to everyone, or if personal data is added to an existing file in the shared path that is being edited.

•Calculate the ratio of unsuccessful DNS requests to the total number of successful requests. A higher-than-normal ratio may indicate operational issues, and an alert should be generated.

•Compare the number of NX domain name responses or admin logins between different days or weeks. If the ratio exceeds 3, generate an alert

•Compare the error rates and page loading times over the past day or hour. If the ratio exceeds 3, generate an alert.

•Compare the number of failed logins on the same day last week with the number of failed logins over the last month. If the number exceeds 30%, generate an alert.

•Compare the number of user signups on the same day last week with the number of user signups over the last month. If the number exceeds 30%, generate an alert

•Compare the number of error logs in the last hour to the number in the previous hour. If the ratio exceeds 1.5, generate an alert.

•If an account has not been used in at least the last 30 days (31, 40, 60, 90, 180 days, etc.), send a notification and either lock or delete the account.

Therefore, it is essential for SIEM researchers who write articles to research and find these indicators and enlighten end-users.

Overall, when evaluating a SIEM solution, it is important to look beyond the general topics discussed in most articles and blogs and to focus on specific features and capabilities that are relevant to your organization’s needs. By doing so, you can ensure that you select a SIEM solution that will provide the level of security and protection that your organization requires. While many SIEM solutions provide the same essential features, it is crucial to evaluate them based on specific and measurable metrics to determine which solution best meets your organization’s needs. Storage efficiency and detection