The Math of SIEM Comparison.

11 min readMay 19, 2020

There are many comparisons and scoring reports like Gartner. But a small part of their scoring is technical capacity. Other comparisons available on the web or magazines are marketing, sales, and presales documents. They do not include extensive technical analysis.

In today’s ever-evolving cybersecurity climate, businesses face more threats than ever before. Finding the right SIEM is crucial in protecting against the latest risks and equipping your organization with a robust security strategy.

A SIEM’s power is in its correlation. %80 percent of SIEM is correlation. if you are spending 80 percent of your time within a SIEM tool doing alert review and analysis, then you are on the right track. [SANS Your SIEM Questions Answered]

A detailed comparison of the correlation capacity of SIEM products technically will be given. The comparison based on the most critical correlation and detection capabilities:

Rule Chain (Multi-Stage Rules):

Rule chain is the ability to combine multiple steps (rules) of a use case without any restrictions. This type of rule detects a sequence of events occurs.

SureLog multi-stage rule sample: “if a firewall admin login has occurred and after this login action there is no configuration change immediately (wait for 15 minutes) but if there is a change in the firewall after this 15 minutes within 12 hours, notify”, there are many other use case examples [1].

Most of the SIEM tools like Micro Focus ArcSight, Logrthym, Qradar, Securonix, and SureLog support multi-stage rules.

AlienVault , McAfee, FireEye, FortiSIEM , Solarwinds LEM, ManageEngine SIEM are the other SIEM tools that support multi-stage rules with some limits.

SureLog Rule Chain Sample

has a restriction on the rule chain capability. For example: is not possible with McAfee. It is not possible to develop this type of rule chain because it is not possible to define “if a firewall admin login has occurred and after this login action there is no configuration change immediately (wait for 15 minutes) but if there is a change in the firewall after this 15 minutes within 12 hours, notify”, “wait 15 minutes” then check for “later 12 hours”.

McAfee Rule Chain Editor

Since there are two or more actions that require time windows, the 10 minutes must be divided between them. For this example, five minutes is the period for each action. Once the unsuccessful attempts have occurred in five minutes, the system begins to listen for a successful logon from the same IP source in the next five minutes. .so there is no chance to implement wait logic between actions (rules)

FireEye has the same restrictions as McAfee.

Solarwinds LEM has the same restrictions as McAfee.

ManageEngine SIEM has the same restrictions as Solarwinds LEM. In ManageEngine SIEM, there is no chance to define a new rule type to chain. Also, there are schema fields restrictions to link rule chains.

only uses None, Plugin_sid, SRC_IP, DST_IP, SRC_Port, DST_Port, AlienVault has the same restrictions as Solarwinds LEMAlso w hen chaining rules, Alienvault Protocol, and Sendor. There is no way for other schema fields to link rule chains.

FortiSIEM also does not have “wait for 15 minutes” kind of capability to chain rules.

Logpoint does not have this kind of correlation capability.

Rapid7 does not have this kind of correlation capability.

One another requirement when chaining rules, is cross-linking of rule fields. As an example: If a device is the destination of a brute force attack and then this destination device is the source of the port scan, detect this device.

Micro Focus ArcSight, FortiSIEM, Logrthym, Qradar, and SureLog support cross-linking of rule fields.

Alienvault only uses SRC_IP, DST_IP, SRC_Port, DST_Port, Protocol rule fields.

McAfee does not support cross-linking of rule fields.

Logpoint does not have this kind of correlation capability.

Rapid7 does not have this kind of correlation capability.

Exabeam and Securonix are UEBA tools. They are not correlation based solutions.

Correlation Logic:

Rules are discriminators used to find a certain behavior. If their designer knows what it’s searching for, they will be invaluable tools. To design a rule without any limits or barriers, the correlation logic of the rule engine must be very powerful and flexible. It is hard to test the correlation logic of the SIEM tools. One of the simplest ways is to try to implement a discriminator use case (correlation rule). For example :

“Detects more than three authentication failures from the same user within five minutes without any successful login in-between.”

This logic seems simple but “without any successful login in-between” is different. SureLog correlation engine can detect this use case.

Micro Focus ArcSight also can detect similar use cases.

If you want to detect this use case with Splunk, it might be possible to do with “transaction” events. But those searches are very taxing in the search head.

Rapid7 and Logpoint have the same issues with Splunk.

AlienVault, FortiSIEM, ManageEngine SIEM, McAfee, Solarwinds LEM could not detect the above use case.

Another test use case is detecting changes. Rapid7 has a change detection capability.

Rapid7 Change Detection Wizard

Qradar also has a change detection capability.

SureLog also has a similar capability with an expert rule option.

Another example is “Never Seen Before Type Of Rules” [2,3]. While Micro Focus ArcSight, Exabeam, Qradar, Rapid7, Securonix, and SureLog have this capability, AlienVault, McAfee, FireEye, Solarwinds LEM, and ManageEngine do not.

SureLog has some additional rule chain capability, chaining multiple rules with “ at the same time” logic. Chaining rule samples:

If a user failed to authenticate a server at the same time the same user authenticates to another server, then notify.
If a user accesses sensitive files and at the same time he has a connection to file sharing sites, then notify.
If there is authentication failure at the same time from user interface(Oracle Management Studio) and console (SQL*Plus), notify

List/Watchlist Management:

has a strong list management feature. Both products support simple lists, multi-dimensional lists, complex lists, lists with 20 columns. Also, those products add, Micro Focus ArcSight, Logrthym, Qradar, Securonix, and SureLog delete, modify, list items dynamically, or manually. ANET SureLog has additional list operators like count, sum, compare, check case sensitivity in lists, tables, and cells. .

SureLog List Management

SureLog also, updates, modifies multiple lists at the same time.

AlienVault:

Dynamic list usage in correlation rules is not supported in AlienVault. It is not possible to develop a rule like If a VPN user connected after business hours and the user is not in VPN white list, alert.

The only way to implement a simple Active Lists is to develop a code.

https://

www.alienvault.com/blogs/security-essentials/how-to-use-ossim-usm-active-lists-with-python-scripts FortiSIEM:

But even if you can develop a Python Scripts, there is no key: value, reference set, reference map, multi-dimensional type of lists. AlienVault SIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, AlienVault does not support list operators like count, sum, compare, check case sensitivity.

Dynamic list usage in correlation rules is limited to one dimension.

McAfee:

There is no key: value, reference set, reference map, multi-dimensional type of lists. The only available operators are “IN, NOT IN”. Also the only way of removing items from a watchlist is time based. Also, FortiSIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, FortiSIEM does not support list operators like count, sum, compare, check case sensitivity.

LogPoint:

There is no key: value, reference set, reference map, multi-dimensional, type of lists. McAfee SIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, McAfee SIEM does not support list operators like count, sum, compare, check case sensitivity.

LogPoint supports two kinds of lists; Static List and Dynamic List. Also LogPoint supports tables, but there is no reference set, reference map, multi-dimensional type of lists.

Also, if you are looking for a GUI for list/watchlist management, LogPoints works over queries. Dynamic lists and table updates are the only query-based. Also, LogPoint SIEM does not support updating multiple lists at the same time (more than one list) by a query. Also, LogPoint SIEM does not support list operators like count, sum, compare, check case sensitivity.

RSA NetWitness Platform:

RSA has a limited list management capability. There is no key: value, reference set, reference map, multi-dimensional, type of lists. Also, RSA SIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, RSA SIEM does not support list operators like count, sum, compare, check case sensitivity.

There are many other correlation features to check [1] . But without an advanced list/watchlist management, it is not possible to detect advanced attacks.

Real Time Correlation:

AlienVault, Micro Focus ArcSight, Fireye, FortiSIEM, Logrthym, ManageEngine SIEM, McAfee, Qradar, RSA NetWitness, Solarwinds LEM, SureLog has a realtime correlation capability. if you use Splunk ES for real time detection, you have to consider “Each realtime search unpreemptively locks 1 core on EVERY INDEXER and on your Search Head”.

Elastic also has no realtime correlation feature.

Cross Correlation:

Micro Focus ArcSight, FortiSIEM, Logrthym, McAfee, Qradar, SureLog has a cross correlation capability. AlienVault crosss-orrelation can only run on (just) IPS and Vulnerability Scanner logs and the combining on just IP addresses.

RSA NetWitness utilizes ESPER CEP, and there is no GUI for cross-correlation rule development.

Logpoint does not have this kind of correlation capability. Mainly it is a search based tool.

Rapid7 does not have this kind of correlation capability. Mainly it is a search based tool.

Exabeam and Securonix are UEBA tools. They are not correlation based solutions.

Correlation Operators:

Micro Focus ArcSight, Logrthym, Qradar, SureLog has a strong correlation operator support like

SureLog also has some edditional correlation operators like:

McAfee has some missing operators like “At the Same Time”, “Before”, “Not Fallowed by Within”

Solarwinds LEM documents mention some other correlation limits.. For example you cannot create a rule using “NOT FALLOWED BY” operator.”At the Same Time”, “Before” are an example of ther missing correlation operators.

Only “AND”, “OR” Operator supported. “NOT” Operator is not supported. Also, other operators listed above are not supported.

ManageEngine Eventlog analyzer correlation has only one operator “Fallowed by Within”. Many operators are missing like “Not Fallowed by Within”.

Also, other operators listed above are not supported like “At the Same Time”, “Before”.

FortiSIEM does not support “At the Same Time”, “Before”.

does not support RSA NetWitness “At the Same Time”, “Before”.

Logpoint does not have this kind of correlation capability. Mainly it is a search based tool.

Rapid7 does not have this kind of correlation capability. Mainly it is a search based tool.

Exabeam and Securonix are UEBA tools. They are not correlation based solutions.

Correlation Field Operators:

Micro Focus ArcSight, Logrthym, Qradar, SureLog has a strong correlation operator support like:

Link Fields
Check Base64
Count Characters
In List
Not In List
Count
Sum
Regex Matches
Matches,
Not Maches,
Entropy Bigger Than
Entropy SmallerThan
Is null,
Is not null,
IP Range Equals,
IP Range Not Equals,
In list,
Not in list,
Starts with in list,
Starts with in list case insensitive,
Not starts with in list,
Not starts with in list case insensitive,
Contains list key in data,
Not contains string in list,
Not contains string in list case insensitive,
Is contained in string,
Regex in list,
Check data in regex list,
Contains in list,
Not contains in list,
Contains credit card number

McAfee has some of those correlation operators but less than the above list.

McAfee Operators

AlienVault, FortiSIEM, ManageEngine and Solarwinds LEM does not support most of the above list.

RSA NetWitness utilizes ESPER CEP, and there is no GUI for rule development. ESPER languase does not support all of the operators.

Logpoint does not have this kind of correlation capability. Mainly it is a search based tool.

Rapid7 does not have this kind of correlation capability. Mainly it is a search based tool.

Exabeam and Securonix are UEBA tools. They are not correlation based solutions.

Correlation Field Restrictions:

Micro Focus ArcSight, Logrthym, Qradar, SureLog has no restrictions on fields. All the available fields on search and report schema will be available for correlation.

McAfee has also some limitations. if I see a user attempt to login to our VPN from two different “regions” within a three-hour window. I have the logic built but in the correlation rules “Advanced Options” I try to set a ‘Distinct values’ of 2 but the monitored fields only seems to provide a ‘Source Geo location’ option, and not ability to select state, region, country, etc.

AlienVault correlation engine has sticky diff restrictions.

Solarwinds LEM does not use all the report fields on correlation. Also correlation cannot fire on raw log data that is received

ManageEngine SIEM has correlation field restrictions. It is not possible to use all the available report and search schema in correlation.

Machine Learning:

SureLog, IBM QRadar, Microfocus, LogRhythm, Exabeam, Securonix, NetWitness Platform has NLP/ML/AI features like DGA detection, outlier detections, rarity detection, similarity detection. LogPoint uses 3rd party UEBA tool Fortscale (RSA Now).

Rare and Abnormal events are common use cases for UEBA. Exabeam and Securonix support that kind of event. SureLog also detects rare and abnormal events.

References

Spike, suspicious (Outliers) events are common use cases for UEBA. Exabeam and Securonix support that kind of event. SureLog, IBM QRadar, Microfocus, LogRhythm, Exabeam, Securonix, NetWitness Platform also detects outliers.

Originally published at https://www.peerlyst.com on May 19, 2020.

The Math of SIEM Comparison.

Rule Chain (Multi-Stage Rules):

only uses None, Plugin_sid, SRC_IP, DST_IP, SRC_Port, DST_Port, AlienVault has the same restrictions as Solarwinds LEMAlso w hen chaining rules, Alienvault Protocol, and Sendor. There is no way for other schema fields to link rule chains.

Correlation Logic:

List/Watchlist Management:

SureLog also, updates, modifies multiple lists at the same time.

RSA NetWitness Platform:

Real Time Correlation:

Cross Correlation:

Correlation Operators:

Correlation Field Operators:

Correlation Field Restrictions:

Machine Learning:

Written by Ertugrul Akbas

No responses yet