IDPS in the Public Cloud

What is an IDPS?

The network infrastructure is the center of any company’s IT operations. Since the first networks were introduced in the 1960’s there has been a potential for intrusion. As networks continued to evolve and become more sophisticated the types of network attacks increased as well. Intrusion Detection Systems (IDS) are built around detecting anonymous, suspicious, and incorrect activity on a network. Intrusion Prevention Systems (IPS) are built around actively scanning packets of data which contain unauthorized or unauthenticated data. An IDPS is the combination of both an IDS and IPS system.

How does an IDPS work?

IDPS are installed on a network to analyze data packets.  An agent, or sometimes called a sensor, is installed physically on the network.  These agents analyze and monitor the activity on the network.  There are many capabilities which can be configured and tuned with an IDPS such as thresholds, blacklists, whitelists, and alert settings.  IDPS utilize three methods of detection: signature-based, statistical anomaly-based, and stateful protocol analysis.

Signature-Based Detection:

Signature-based detection uses a pre-defined list of threat signatures to observe while scanning packets on the network.  This detection method is similar to how most modern anti-malware programs work in the detection of viruses on a desktop computer.  Updates to the signature lists are usually automated and can be patched very quickly.

New vulnerabilities and attacks are usually undetected through this type of detection.  The IDPS relies upon constant updates in order to stay current.  Furthermore, signature-based detection does not keep track of the communication events, so it is unable to detect attacks that compromise multiple systems.

Some network attacks flood the network with too many packets in which the IDPS cannot process.  The IDPS can be configured to handle this attack two different ways, it will ‘fail-closed’ in which all data packets are dropped until the queue is relieved.  Or the IDPS will ‘fail-open’ in which all data packets are allowed to traverse the network.

Statistical Anomaly-Based Detection:

Statistical anomaly-based detection compares the current network activities against what is considered ‘normal’ activity. A baseline of normal activity is gathered through observation (“listening”) on the network over a period of time. After the baseline is created, all network activity is analyzed against it. Any discrepancies found in the network traffic will be responded to by the IDPS. This type of detection is extremely effective against unknown threats.

However, there are many issues with this type of detection method. The number of false positives generated by this method is very high. The IDPS must wait in a ‘listening’ mode for a long period of time (generally greater than 90 days) in order to ‘learn’ what is considered ‘normal’ network activity. There is a concern that these types of detection may drop data packets from legitimate authenticated customers. Also, some applications may not generate consistent network data in which the IDPS cannot set an acceptable baseline.

Stateful Protocol Analysis Detection:

This is accomplished by comparing predetermined profiles of what is known to be generally accepted definitions of standard protocol activity against previously observed abuse to identify deviations. In short, the IDPS is configured to know how a particular protocol should work, such as FTP, so it can respond when it detects anomalous behavior.

As with the other detection modes, stateful protocol analysis has some limitations. It can be very resource intensive as there can be a lot of processing overhead required to performing the session-based assessments on multiple connections. If an attack never displays a protocol violating its intended behavior, the IDPS cannot detect the attack.

Why the Public Cloud is different?

Unlike most companies internal networks the Public Cloud operates differently in a virtual infrastructure and is often referred as an ‘overlay network.’ Which is a network that is built on top of another network – packets are encapsulated one packet inside of another packet to provide separation and a demarcation. Installing an IDPS on a traditional internal network involves placing a physical device or sensor in-line of our network traffic to analyze packets.

This is not possible in Public Cloud environments due to the fact that we do not own the physical network environment. We share this Public Cloud environment with hundreds of thousands of other customers and their data. We utilize encryption in transit protections to ensure our data is protected from all of the other occupants within the Public Cloud (yay!). Yet we all conduct our business on the same physical network infrastructure. Public Cloud overlay networks give us some visibility into the Port used (e.g., 22, 443, etc.) through Security Groups and VPC flow logs. However, they can only provide us the Internet Protocol used (i.e., TCP, UDP) - so even though we opened up port 53 for DNS resolution we do not know if a malicious threat actor is abusing a protocol and pumping out an encrypted SSH session out via DNS. In short, we (and the industry as a whole) struggle to have the necessary visibility regarding malicious inter-network traffic within our Public Cloud infrastructure.

It would be very difficult for us to install an IDPS physically within all of the data centers used by a Public Cloud Provider. We would then need the IDPS to have the capability to read and inspect ONLY our packets and ignore everyone else’s. The physical infrastructure of Public Cloud providers is locked down - even to customers (e.g., no datacenter walk throughs). Most major Public Cloud providers do not allow the installation of company specific hardware within their data centers.

What can we do?

Fortunately, there are a number of solutions available that help us tell a good story when it comes to monitoring events within our Public Cloud and stopping threats once detected.

Amazon GuardDuty

Amazon GuardDuty is a threat detection service that continuously monitors for malicious activity and unauthorized behavior to protect AWS accounts and workloads. With the cloud, the collection and aggregation of account and network activities is simplified, but it can be time consuming for security teams to continuously analyze event log data for potential threats.

The service uses machine learning, anomaly detection, and integrated threat intelligence to identify and prioritize potential threats. GuardDuty analyzes tens of billions of events across multiple AWS data sources, such as AWS CloudTrail, Amazon VPC Flow Logs, and DNS logs.

Amazon Detective

Amazon Detective makes it easy to analyze, investigate, and quickly identify the root cause of potential security issues or suspicious activities. Amazon Detective automatically collects log data from AWS resources and uses machine learning, statistical analysis, and graph theory to build a linked set of data that enables you to easily conduct faster and more efficient security investigations.

Determining the root cause of security findings can be a complex process that often involves collecting and combining logs from many separate data sources, using extract, transform, and load (ETL) tools or custom scripting to organize the data, and then security analysts having to analyze the data and conduct lengthy investigations.

Amazon Detective simplifies this process by enabling a CSOC to easily investigate and quickly get to the root cause of a finding. Amazon Detective can analyze trillions of events from multiple data sources such as Virtual Private Cloud (VPC) Flow Logs, AWS CloudTrail, and Amazon GuardDuty, and automatically creates a unified, interactive view of your resources, users, and the interactions between them over time.

Conclusion

While we cannot achieve a perfect 1:1 IDPS solution with most popular Public Cloud providers at this time (and that's okay).  There are more capabilities coming down the pipe from our Public Cloud Providers as well as third-party products that will continue to give your company a good story to tell regarding Threat Detection and Prevention.  I look forward to seeing what else is coming...