Mastering SIEM: Key Questions and Leading Use Cases from Giants like Splunk and IBM QRadar
Q: In urgent cases, such as “China having ‘persistent’ access to U.S. critical infrastructure, with China-backed hackers maintaining access to major U.S. critical infrastructure for ‘at least five years,’ according to an intelligence advisory released Wednesday [1]” how quickly can archived data be restored?
A: In all SIEM solutions, archived storage will need to be thawed. We will use Splunk calculations and values for the required disk size calculation.
Assume the Daily Data Volume is 5GB, which means a maximum of 250 EPS. The Raw Compression Factor is 0.15, and the Metadata Size Factor is 0.35. The number of Days in Hot/Warm is 30 days, in Cold is 60 days, and in Archive is 270 days. Therefore, the total 1-year disk requirement for 250 EPS or a daily data volume of 5GB is 427.5 GB [2].
In the mentioned case, logs are only available for 90 days (30 days in Hot/Warm and 60 days in Cold). To access the Archive, you need to thaw it, but this is not a straightforward and can be very time-consuming [3,4,5,6].
Considering the same scenario for a larger network with 10,000 EPS or a 500 GB daily log volume and a 5-year span for a case like the one mentioned, the complexities and time required for data restoration would significantly increase.
In cloud environments, it will be more critical to manage retention policies. We will provide examples from QRadar Cloud. “The default retention period is 30 days; then, the data is immediately deleted.” [7].
Q: Is real-time correlation the same as search-based (Batch Analytics) detection/correlation?
A: Real-time correlation is designed to identify security incidents as they happen.
Search-based detection or correlation, on the other hand, involves querying and analyzing historical data to identify patterns or incidents that may have occurred in the past
Q: Are there any limitations on real-time correlation capabilities in the SIEM products on the market?
A: Yes, There are some real time detection limits. And it changes from product to product. For example Splunk[8,9,10,11,12], and Microsoft Sentinel[13], have limited real-time detection features
References
1. https://www.axios.com/2024/02/07/china-volt-typhoon-critical-cyberattacks
2. https://community.splunk.com/t5/Deployment-Architecture/Hot-Warm-and-Cold-Storage/m-p/539207
6. https://community.splunk.com/t5/Splunk-Cloud-Platform/Data-restore/m-p/673462
8. https://community.splunk.com/t5/Splunk-Search/Real-Time-Search-Issues/m-p/423805
9. https://answers.splunk.com/answers/433872/why-are-real-time-searches-not-running-and-getting.html
10. https://medium.com/@clong/splunk-building-dynamic-lookup-tables-a593261569
11. https://docs.splunk.com/Documentation/Splunk/latest/Search/Realtimeperformanceandlimitations
12. https://answers.splunk.com/answers/671819/real-time-alert-1.html
13. https://docs.microsoft.com/en-us/azure/sentinel/near-real-time-rules
