Detection Engineering in the Clouds

Threat actors are evolving at a faster rate than ever before. Companies, meanwhile, are migrating workloads to public cloud resources. Now, security teams must create detection content and infrastructure that is mature, flexible, and scalable to identify and respond to advanced threats. Engineering those detections effectively for the public cloud requires a baseline posture, robust logging capabilities, reliance on vendor/third-party tooling, and more. Below is a high-level overview of key areas that organizations need to address in order to be able to detect disruptive and destructive threat activity. 

By the way, infrastructure-as-a-service (IaaS), software-as-a-service (SaaS), and platform-as-a-service (PaaS) models are connected to enterprise architecture more than ever. Implementing a zero trust approach to cybersecurity is crucial due to workload migrations, public cloud multitenancy, remote work cultures, and many other reasons. In a future post, we’ll discuss incorporating a zero trust strategy into current and future cloud detection workflows.

In addition to detecting threat actors in the cloud, many organizations continue to struggle with baseline posture. What’s most important about moving detection capabilities away from atomic indicators and toward statistical and behavioral activity as stated in the pyramid of pain is understanding where the current posture resides. In this post, baselines mean historical insight into previous normal, day-to-day behaviors, and benchmarks are simply the concepts for establishing a minimum security profile for assets in the cloud. Without these two foundations, writing detections will become exponentially more difficult as migrations mature and vendor services increase in count. 

Here are some ideas for baselining activity: 

• Noting application programming interfaces (API) that the entity or identity and access management (IAM) user doesn’t normally call 
• Abnormal spike in the number of API calls (failed and successful) 
• Amount of data normally exfiltrated from data buckets 
• Days of the week high value targets are accessed and by what users 
• User agent anomalies 
• Compute instances stood up (or autoscaled) on average 

Depending on the organization, having data for about 45 to 60 days (in some cases 6 to 12 months) should give an accurate picture of what activity is normal and what activity is unusual. The maturity level of your baseline should also mature over time. 

To develop benchmarks, organizations can leverage standards such as those from the Center for Internet Security (CIS), Security Technical Implementation Guide (STIG), and the National Institute of Standards and Technology (NIST). These standards serve as a manual for security configuration best practices for public cloud vendors. Although security issues caused by user error and misconfigurations have decreased over the years, partly due to vendors attaching "dummy-proof" security measures to resources (e.g., AWS S3 now automatically encrypts all new objects added on buckets on the server side, using AES-256 by default), they still pose a large threat to resources in the cloud. 

Together, baselines and benchmarks can help organizations deploy mature detections, minimize analysts’ fatigue, and develop effective and efficient response playbooks.

Even though most organizations are partially reliant on public cloud workloads, very few have completed a full migration. These transition stages leave gaps in long-term detection maturity workflows that can turn into issues if not addressed as a joint architecture. 

Some questions to ask may include: 

• Is the asset management posture mature enough to respond to IAM detection events associated with user workstation activity? 
• How are the detection findings in the cloud aiding in the maturity of the organization’s enterprise architecture and design? 
• Does the vulnerability management/endpoint detection and response (EDR) program take into consideration users who have access to high-value cloud resources that may be storing secrets locally? 
• Are we conducting red team exercises on-premise as well as in the cloud? 

Logging in the cloud can get complex. Unfortunately, there’s not one special button that will give a security team everything needed to carry out analysis and investigation. Since the cloud is compiled of services, each with its own logs and API calls, each service’s logging capabilities must be addressed separately. Knowing what logs to collect, how they're being collected/parsed, and where they're being stored (and for how long) are some key avenues to analyze. 

When uncovering what logs are needed to build out efficient detections in the public cloud, ask questions such as the following: 

• What services are in use and what logs are available? 
• Are all the logs needed or not? And why? Maybe only certain fields will give you what you need to know.
• How and what services are used to house other applications? (e.g., how are you obtaining logs from services running on compute instances such as MYSQL, Redis, and Tomcat? Are those services running in any containers inside Kubernetes?) 

When determining how to collect and store these logs, ask questions such as the following:

• How is the team accessing these logs? Is it via security information and management (SIEM) or a vendor-specific tool?
• How long is the team storing the logs? Does the team need to retain certain data sources for longer than others? Does it give the team enough to meet its baseline criteria? The SolarWinds breach, for instance, showed customers that a log retention period dating back a year was crucial for analysis.

Also, keep in mind that not all service logs are on and active by default. It is important to know what services are running and where (regions/availability zones) in order to receive important data sources. 

Using vendor tools to compare and contrast against external logging can help build mature detections. Here is a list of some tools available to detect threats:

•  CloudTrail, CloudWatch, GuardDuty, Detective, Inspector, Athena, and more

• StackDriver, Cloud Monitoring & Logging, BigQuery, Cloud Armor, and more

• Azure Monitor, Azure Data Lake, Azure Sentinel, Azure Security Center, and more

Detection engineering in the cloud is exciting and complex work. Although there are many areas of focus, some of the groundwork and implementation techniques described can help save time and drive focus toward engineering robust detections. It's encouraged to promote open discussion around what works for certain detection teams so that others may benefit from it as well. In addition, the techniques can also assist in increasing IAM maturity which is a major cornerstone for zero trust architecture in the cloud. Engineering detections for IaaS, SaaS, and PaaS models that are connected to enterprise architecture via human or system (e.g., serverless functions) are important for decreasing the size of the threat landscape. In a future post, we’ll discuss incorporating a zero trust strategy into current and future cloud detection workflows.