Effective Incident Response Starts with Detection: What Is Your Team Missing?

It’s nearly impossible respond effectively to something you don’t see. Many organizations invest heavily in a range of controls across the enterprise to detect potentially malicious activity. These tools help an organization’s leaders figure out what’s going on and what to do about it.

The more confident you are that you understand what’s happening, the clearer the appropriate response, whether that’s a fully automated response to known malware, notifying investigators that something unusual has happened and may be worth investigating, or doing nothing at all. The more you know, the more likely you can act quickly and correctly. The less you know, the more likely you will have to spend valuable time and resources finding out whether you should do anything at all.

Unfortunately, it can be difficult to catch everything, and it’s likely you will experience incidents where you simply didn’t see enough. So, how can you tell what you’re missing?

It may seem obvious, but incident response is all about response. Its success or failure is measured by the quality of the response. Did we do the right thing? Could we have done it better? Better detection typically means a better response.

It’s often helpful to think of incident response in terms of a series of actions you take to respond to an incident, and the decisions you have to make to act in that way. What would you need to know, for example, to decide to block that email, isolate that machine or quarantine that file? The more confident you are in your detection, the more confident you will likely be in your response. It also follows that being able to automate (or semi-automate) that response will depend on how confident you are that your detective controls are giving you the full picture. 

That said, it’s important to understand that more detection does not necessarily mean better detection. Many organizations are awash in logs, but unless this data can be turned into an effective response, they may be wasting resources on collecting and storing it to little purpose. So, how do you know you are detecting things that are valuable in response?

When you are trying to figure out what you may be missing, incident response playbooks are often the best place to start. Playbooks typically describe the steps you would take to respond to common scenarios. These can range from standard malware to ransomware and advanced persistent threats. They are often used to provide guidance on how typical threats can be detected and investigated.

When it comes to detection, your playbooks should help focus on triggers that would indicate a specific type of incident. These include both higher-confidence triggers (where little additional investigation or analysis should be required before declaring an incident and responding to it) and lower-confidence triggers (usually reports of unusual activity where more investigation or analysis would likely be required). 

Work through your playbooks to identify areas where you may be missing potential high-confidence triggers. For example, a simple system malware playbook may call out higher-confidence triggers such as:

• Malware alerts from endpoint anti-malware or endpoint detection and response (EDR) solutions
• Alerts generated by malware rules and offenses in a security information and event management (SIEM) solution
• Malware alerts from internal intrusion detection/prevention systems (IDS/IPS) or firewall systems
• Malware alerts from web firewall, proxy or gateway systems 
• Malware alerts from VPN clients and gateways
• Malware alerts from deception sensors

You may, however, need to pull telemetry from multiple systems to understand what’s going on. Traditionally, organizations have relied on security suites (and their associated management consoles) or SIEM systems to help pull data together to correlate detection across multiple sources. These systems tend to focus on use cases (sometimes also referred to as detection logic or analytical plays) to use the combined telemetry from multiple source systems as well as enrichment (additional data to help provide context) to help detect incidents across a wider range of detective controls. The continuous improvement of such use cases is a typical way that we can improve detection in simpler cases.

Unfortunately, not all triggers come in the form of clear-cut alerts. For system malware, for example, you may also find several lower-confidence triggers, such as: 

• Employee or customer reports of unusual system behavior
• Anomalous traffic alerts from network protections such as IPSs, firewalls, proxies and VPNs
• Notifications of unusual or new processes from endpoint protection systems
• Web proxy logs and network metadata indicating connections to known malicious sites

These are much more challenging to work with. Investigating low-confidence triggers requires a combination of human skill, advanced analytics and some deep sources of data.

Unfortunately, the most effective way to tell what you are missing is to go through the pain of a breach or even multiple breaches. Certainly, some of the most widespread or damaging incidents in the past few years started as something that may have been recognized as unusual but not necessarily malicious. 

A vital part of incident response is in post-incident analysis, where you can identify opportunities to improve detection. In some cases, these may be simpler fixes where a specific type of detection is missing or defective, but in most cases, you likely missed something because automated systems didn’t have all the information and analysis necessary to understand what was going on.

Lessons learned from past incidents can be valuable for improving detection, but thankfully you can also learn from others. Your organization should run regular tabletop exercises in which you talk about how you might detect and respond to specific types of incidents. Also, you may need to leverage external testing firms to help evaluate your ability to detect simulated attacks intended to look like real-world threats. You can even take advantage of information sharing through industry peers. All of these will help you to ensure that your suite of detection capabilities is sound.

Even with the best set of detective controls, you still may not see enough to catch all threats, particularly advanced attacks that take advantage of undisclosed vulnerabilities over a long period of time.

In these cases, the most advanced organizations have turned to a combination of investments in three key areas to improve detection. 

• Automation: Organizations have invested in automating as much of the response as possible to free up time for human analysts to work on investigating lower-confidence indicators such as anomaly alerts. This typically requires investment in a security orchestration, automation and response (SOAR) platform as well as the resources required to integrate it into the enterprise and fully take advantage of it. 
• Data: Investments in wide and deep stores of telemetry data (such as logs) as well as context data (such as identity, asset information, vulnerability data and the like) can take the form of large, on-premises data lakes associated with a SIEM solution or large-scale cloud data environments. This provides the wide range of data available to correlate detection, but also typically requires additional time to generate and run queries across that data.
• Analytics: Organizations are investing in the application of machine learning models to more quickly identify anomalous activity that is probably malicious. This has started to show some promise, but early efforts also demonstrate that organizations still need human effort to maintain such models and explain their results. 

Most organizations, however, can rely on technology providers and managed detection and response service providers who have already made this investment to improve response across their managed customers.

Story by Gary McIntyre, the managing director of cyber defense at Focal Point Data Risk, a CDW company, focused on customer cybersecurity operations and defenses. He is a seasoned information security professional with over 20 years of experience focusing on the development and operation of large-scale information security programs. As an architect, manager and consultant, he has worked with a wide range of public and private sector organizations around the world to design, build and maintain small and large security operations teams. He is co-author of Security Operations Center: Building, Operating and Maintaining your SOC.