DATA and Log Enrichment as Code in SureLog SIEM

Data & Log Enrichment

Data and log enrichment is the process of adding contextual information to raw data and logs collected from various sources, such as network devices, servers, applications, and security tools. This additional context makes the data more informative and actionable, helping organizations better understand and respond to security events, operational issues, or other incidents.

Key Aspects of Data and Log Enrichment:

1. Enhancing Raw Data:

• Raw logs and data typically contain basic information like timestamps, IP addresses, usernames, and event details. Enrichment involves augmenting this raw data with additional details that provide context and meaning.

2. Adding Contextual Information:

• Geo-location Data: For IP addresses, enrichment might include adding geographical information like country, city, and even organization (using GeoIP services).
• Threat Intelligence: Enriching data with threat intelligence involves cross-referencing IPs, domains, or file hashes with known threat databases to identify whether they are associated with malicious activity.
• User Information: Logs might be enriched with information about the user involved in an event, such as their role, department, or permissions within the organization.
• Asset Information: Enriching data with information about the assets involved, such as their location, owner, operating system, and criticality to the business.
• Vulnerability Data: Linking logs to vulnerability data can highlight whether an affected system is known to have specific vulnerabilities, increasing the urgency of the event.

3. Benefits of Enrichment:

• Improved Incident Detection: By providing additional context, enriched logs make it easier to identify suspicious activity or patterns that might indicate a security threat.
• Faster Response: Enrichment helps security teams prioritize and respond to incidents more effectively by giving them the context they need to understand the severity and potential impact of an event.
• Reduced False Positives: With more context, it’s easier to filter out benign events and focus on those that truly pose a risk, reducing the number of false positives.
• Better Analysis and Reporting: Enriched data is more useful for generating meaningful reports, dashboards, and visualizations that can guide strategic decision-making.

4. Common Enrichment Techniques:

• Lookup Tables: Using pre-built tables to add information like department names or device types based on known IP addresses or usernames.
• API Integrations: Fetching real-time information from external services (e.g., threat intelligence providers) to enrich logs.
• Correlation: Associating related events across different log sources, helping to identify complex attack patterns or operational issues.
• Tagging and Categorization: Automatically tagging events with labels (e.g., “malware,” “failed login,” “suspicious IP”) to make them easier to analyze.

Common and Well Known Example Scenario:

Imagine a raw log entry showing a user login attempt from a specific IP address at a certain time. Without enrichment, this log might only show:

• Timestamp: 2024–08–16 12:34:56
• Username: jdoe
• IP Address: 192.168.1.1
• Event Type: Login Attempt

After enrichment, this log could include additional information like:

• Geo-location: San Francisco, USA
• User Role: Administrator
• Asset Criticality: High
• IP Reputation: Blacklisted (indicating potential malicious activity)

SureLog SIEM Difference: Data and Log Enrichment as Code

Besides Lookup Tables, API Integrations, Correlation, Tagging and Categorization, SureLog SIEM introduce advanced data and log enrichment method “Data and Log Enrichment as Code”.

With SureLog SIEM, it is possible to enrich logs and data in real time by writing code in Java, beyond the standard enrichment methods mentioned above. This method not only removes all barriers to enrichment but also opens the door to all kinds of advanced analysis. Additionally, the ability to operate in real time is crucial for threat detection.

Data and Log Enrichment as Code Example Scenarios:

VPN usage is vital and is a fact of life for most organizations. VPN is key for being connected, even partners and third party vendors may get VPN to access to enterprise networks. VPN is also popular among cyber threat actors, as they can use it to gain privileged access to the network for the price of a username and password.

Being able to audit and monitor user activity across a Windows Server based Network and heterogeneous network is key to knowing what is going on in your Windows environment and heterogeneous environment. Monitoring user activity is vital in helping mitigate increasing insider threats.

As many members of the IT staff used the VPN to access the network remotely, this access could really have been from anywhere.

A common question taken from one of the well know global SIEM vendor blog site. “We have a requirement to track users that logon via vpn and then go on to logon to servers on our environment, we can see the separate events but have not had success in getting a correlation rule with both of these to trigger or an alarm at the very least.”. There is no reply for the last 2 years for this requirement. SureLog has an answer for this requirement.

Do you really know what they are doing and what has happened?

A virtual private network (VPN) is a network that uses primarily public telecommunication infrastructure, such as the Internet, to provide remote offices or travelling users access to a central organizational network. Nowadays VPN is widely used in many sectors for different reasons. Some use it to secure their end-user access to the corporate access

others use it to give secure access to their outsource partners. Examples are widely available and create a growing demand for VPN-like services.

Control and Monitor VPN Access

While secure VPN solutions are a very good method to address different issues, it is very hard to control and monitor what is really happening inside the connection. When you provide VPN access to someone, you usually trust them. However, blind trust conceals the truth. When the VPN user can access critical systems or sensitive data, you definitely want to monitor and control their activities.

Automatic VPN tracking can help reveal malicious activities from insiders and outsiders. SureLog is uniquely suited to automate this massive task

It’s hard to gather information about VPN activity; what was really happening: An authorized user is logging in over VPN, escalating privileges, and shipping data out over SSH. Figure 1 is a diagram of the steps the admin took.

The administrator connected to the VPN from her home machine and logged in with her normal User ID of Banu. She was then granted a DHCP address of 172.16.23.21 to her home system’s MAC address of 00:0c:76:8b:c4:16. She then logged in as root, using SSH to the application server at IP address 172.16.99.99. A short time later, the firewall logged a successful outbound SSH/Secure Copy (SCP) session to address 1.2.3.4 from the app server, where the admin was using SCP to send data outbound. Sounds simple enough to catch, right? If the proper correlation and monitoring capabilities are in place, it could be. Usually, however, this is not the case.

ANET SureLog SIEM solution has proper correlation and monitoring capabilities for VPN monitoring without installing any agent.

Some sample monitoring scenarios:

· A company uses secure VPN connection to grant access to their internal IT environment to an outsource IT partner. The outsource IT partner has access to all critical environments and the company can only hope that they will not do anything harmful.

· A company uses secure VPN connection to grant access to their internal application to some users. The users can access the application remotely anytime without further control. If the application cannot monitor user activity, no one would know what happens exactly during a connection.

· A company uses secure VPN connection to grant access to their IT environment to the internal IT team. System administrators can manage the whole IT environment without strict control and the company can only hope that they can trust their employees. In a forensics situation it is difficult to detect who did what and when on a server.

· One rogue admin had logged in from a VPN connection, accessed these systems by logging in as root or Administrator directly, and copying sensitive files elsewhere. How could this have been detected?

Many organizations are facing both government and industry compliance requirements that involve implementing policies, audit processes, and security controls. Several of these call out privileged user management and monitoring specifically. Two examples are the Payment Card Industry Data Security Standard (PCI DSS) and the Federal Financial Institutions Examination Council (FFIEC) Information Security Booklet.

The PCI DSS is comprised of 12 sections, each focusing on a major aspect of information security programs. Section 10 is labeled “Track and monitor all access to network resources and cardholder data,” and contains two subsections that require privileged user monitoring:

Section 10.1:

“Establish a process for linking all access to system components (especially access done with administrative privileges such as root) to each individual user”

Section 10.2.2:

“Implement automated audit trails . . . [for] all actions taken by privileged users”

These requirements directly follow best practices, namely, to disallow the use of generic privileged user accounts such as root and Administrator directly (with tools like su and sudo), and also to generate and maintain logs related to all privileged user activity

SureLog VPN Tracking capability is not limited to firewall logs. SureLog will monitor FW, Servers and processes, File System logs. Automatic tracing algorithm has

· VPN synchronization module: Monitor VPN connection and VPN close logs firewalls

· Synchronize all the RDP, SSH, Process and File System events with VPN close event in order not to produce redundant reports

· Monitor Multiple VPN connections to the same Server

· Correlate all the RDP, SSH, Process and File Access events according to VPN source, destination IP pair, Server IP and User Name pair

Tracing

· VPN ->RDP

· VPN->RDP->RDP

· VPN ->RDP->Process

· VPN ->RDP->File Access

· VPN ->RDP->(Internet&WEB) Access

· VPN->RDP->RDP->Process

· VPN->RDP->RDP->File Access

· VPN ->RDP-> RDP-> (Internet&WEB) Access

VPN Multiple RDP Reports

SureLog SIEM VPN tracking reports are not limited with just first RDP connection. It’s tracing capability allows administrators to track multiple RDP records. SureLog will depict the previous RDP connection with parentrdpsource field since usernames are same

VPN Process Monitoring Reports

SureLog SIEM VPN tracking module reports running processes which was started by the VPN user. Process tracing has the ability to monitor deep RDP track. Process activities for both first and multiple RDP connections will be available. For multiple RDP connection previous RDP connection will be depicted as parentrdpsrc

VPN File Access Reports

SureLog SIEM VPN tracking module reports file access activities which was started by the VPN user. File activity traking has the ability to monitor deep RDP tracking. File activities for both first and multiple RDP connections will be available. For multiple RDP connection previous RDP connection will be depicted as parentrdpsrc

VPN (Internet&Web) Access Reports

SureLog SIEM VPN tracking module reports internet &web access activities which was started by the VPN user. Internet&web activity traking has the ability to monitor deep RDP tracking. Internet&web for both first and multiple RDP connections will be available. For multiple RDP connection previous RDP connection will be depicted as parentrdpsrc

Statistical Reports

Statistical reports also available like top most reports.

Example Reports

· Top VNP Users

· Top RDP Servers

· Top RDP Users

· Top VPN Processes

· Top File Acces

· Top VPN Internet Access

Alarms:

Monitoring VPN activity and creating automated alarms is vital in some cases ad important for most of the time. Correlation rules:

· If VPN user ertugrul will access computers other than x,y,z (whitelist), notify

· If any VPN connection is open more than 6 hours (Suspect long flow) , notify

· VPN servers whitelist

· VPN servers blacklist

· If a VPN user executes a nmap.exe (any process), notify

· If a VPN user access hosts.conf file, notify

· If a VPN user starts a traffic to IP X.Y.W.Z, notify

· If a VPN user starts a traffic other than A.B.C.D (Blacklist), notify

· If a VPN user starts a web request to anetusa.net, notify

· If a VPN user starts a web request other than anetusa.net(Blacklist), notify

There are many reasons to pay attention to VPN user activity. Aside from the risk of malicious behavior from insiders, even accidental activities can have disastrous consequences due to excessive privilege use. Many compliance mandates are now also stipulating the management and monitoring of privileged user activities, ranging from policy definition to implementation of least privilege and logging requirements.

References

https://surelogsiem.com/data-and-log-enrichment-as-code/