Esphion

The black-market for vulnerabilities

My apologies for not posting a new blog entry in such a long time. A busy work schedule, business trips and the holidays thrown in for good measure prevented me from paying as much attention to the blog as I should have.

Well, we are one month into the new year now. Worms and viruses keep coming at us. One vulnerability, which was discussed in great length in recent weeks was the so-called WMF (Windows Meta File) flaw. Merely by visiting a compromised web-site, a user's PC could already be infected.

Interestingly, in September of last year I wrote an article about the possibility of an emerging black-market for vulnerabilities. The idea is that certain individuals are willing to pay money to get their hands on exploits, which allow them to compromise more machines. These machines can then be used for lucrative businesses, such as spam, p0rn hosting,  DDoS attacks, click-fraud, etc.

Today eWeek reported that the WMF exploit was available for money, in exactly this black-market for vulnerabilities, weeks before security researchers even knew about it. For $4000 the exploit was offered in the middle of December last year by Russian hacker groups. Here is an interesting quote from the article:

There are dozens of these sites with hackers offering zero-day code for sale all the time. They even have a mechanism to test the code to make sure it is legitimate and will get past anti-virus software.

The lesson we can learn from this is: Zero-day attacks will remain a threat to our network and computer security. Therefore, we will continue to see attacks that manage to evade signature based security solutions.

Juergen

Worms exploring new territories

Several publications today commented on the new SANS Top-20 report, which was just published. Noteworthy about this year's report is the fact that in 2005 the authors of worms have moved their attention from operating system bugs to vulnerabilities in application code and even network devices.

Largely, this can be attributed to increased security efforts by the OS vendors, such as Microsoft's now regularly scheduled 'patch Tuesdays'. But while a lot of attention has been paid to OS security and the automatic and regular application of necessary patches, much less has happened for application software. For the most part, patching of applications is still irregular, slow, manual, and basically ... patchy.

In 2005 there was a remarkable number of highly critical vulnerabilities in various networking devices, such as switches and routers, but also in security software. We all remember the Witty worm, which so elegantly took advantage of a protocol-parsing vulnerability in firewall code. It demonstrated that even with a relatively small vulnerable population, it is possible to accumulate significant numbers of zombies, if the worm is well written.

What are the consequences of these trends? With increased attention to the non-OS layers of the overall computation stack, we can expect to see more diverse worms. An individual worm may not infect many millions of systems, but can still deliver a sizable population of zombies to its creators.

Since worms are now focusing on applications and network devices as well, there are obviously many more potential points at which a network may become compromised. If patching and securing a network felt like a never-ending nightmare before - apparently it is going to get worse now. I think it is safe to assume that the developments, which are reflected in the Top-20 report point towards a heightened possibility for any organization to experience a worm outbreak on their internal networks, and are going to result in ever larger windows between discovery of a vulnerability and patches eventually being available and applied.

What can be done? Well, we have to accept the fact that outbreaks are going to happen. There are too many attack points to guard them all. The famous dissolving perimeter has just become even more of a problem. While a key focus for every organization has to remain on outbreak prevention, it is vitally important also to devote resources to outbreak management. Assume that an outbreak will happen - what are you going to do about it? Signature based systems, which may be great at preventing known exploits from passing through, will be powerless when faced with a zero-day vulnerability.

An ideal, towards which every organization could strive, is the self-defending network. In such a network, anomaly detection systems are deployed to ensure that any outbreak, even of a zero-day worm, can be detected, analyzed and mitigated quickly.

In general, we can say this: Outbreaks will happen, but only if you see them, and understand them, will you be able to do something against them. Anomaly detection solutions provide instant visibility and analysis of suspicious behavior on the network. Therefore, they are a vital component in any outbreak management strategy.

Juergen

Your private threat-detector

We all know that it is a good idea to place smoke detectors throughout our houses. In theory, though, we could also wait for the neighbors, or the community in general, to call the fire-department once they see flames coming out of our house. Or if there is a larger fire in our neighborhood, we could assume that the sound of the sirens will be enough to alert us.

But we don't think that way. While community action is good, and emergency broadcasts about approaching fires are certainly very necessary, we still also place our own smoke-detectors in our house. Why is that?

The reason of course is that our own smoke-detectors are best situated to detect a fire in the very early stages, right where it matters - in our house. At such an early stage, there is only very little smoke. No neighbor will see this, no fire sirens will sound, no fire-trucks will come rushing down our driveway. There is too little smoke for anyone else to see. Instead, we very much need to have our own personal alarm in our own, private space. Nobody else has the same insight that we have, nobody else can see or smell the air in our own house, and nobody else can detect this fire, our very own personal problem, as fast as we ourselves can.

This is quite obvious, and I am sure that nobody will really disagree with these statements.

Therefore, it is surprising to me that many network operators and corporations are placing the security of their network into the hands of the community, if you will, without allowing for the presence of their own smoke detectors. What I am referring to is the tendency of many organizations to feel safe and secure, once they have installed IPSs (Intrusion Prevention Systems) in their network, which receive updates and new signatures from the global data center of the IPS vendor. Let me ask you this:

What does the global data center know about localized anomalies and issues, that are specific to your own network?

If your network is targeted by a specific DDoS attack, or if there is a zero day worm spreading in your network, or if there is a traffic anomaly caused by equipment failure or misconfiguration, or if there are failing or misbehaving applications or users... how can some global data center, which does not see any of this be of help to you?

The answer, of course, is that it cannot help. To get back to the analogy of the smoke and fire, these data centers are great to inform you of fires in your community. They can sound the sirens to alert you to approaching storms, which also have affected others already. In some cases, they may even be able to prevent issues on your network, by uploading a signature for one of those global anomalies before your network is hit.

However, these global data centers are quite useless when it comes to detecting the first wisps of smoke, indicating something that is specific to your network alone.

So, to be truly protected, you see that you need your own smoke-detector for your network. These network smoke-detectors are called anomaly detection systems. Every mission critical network has to have one of those, since signature-based IPSs alone cannot help at all with any issue that is localized, and that is affecting your network, either by accident or by design.

The good thing is, though, that you don't need to throw away your investment in IPSs or deep-inspection firewalls. Quite the opposite. A good anomaly detection system will be able to produce fine-grained signatures out of the first traces of an anomaly, which can then be fed to the IPSs for filtering. See my articles here and here.

For fire-protection, we rely on a community effort along with local smoke detection. We intuitively know that this is best practice. The same holds true for our networks. Having IPSs, which can protect against known threats, is necessary. But at the same time, having an anomaly detection solution as the local smoke detector is equally necessary, and equally needs to be considered as best practice. Without it, our threat detection and management capabilities are simply not complete.

Juergen

Of butterflies and raindrops...

Please allow me to get a bit philosophical today, about highly complex, dynamic systems. I promise, at the end there is a rather important network angle to all of this...

The Butterfly Effect

In chaos theory, the term butterfly effect describes a particularly interesting observation: Given a sufficiently complex and dynamic system, even the smallest variation of the starting conditions will result in unpredictable long-term behavior. The effect probably got its name from the example that even the gentle flapping of a butterfly's wings will potentially influence the earth's weather and climate (a phenomenally complex system) in the long run. In other words, if this butterfly would not have flapped its wings, then maybe we would not have had a storm on the other side of the world a few months later.

Incredible as this may sound, such is the behavior of highly complex, dynamic systems: Changing just a single parameter may eventually result in a completely different and unexpected outcome. The butterfly effect can be demonstrated with various mathematical formulas. Real world systems usually 'suffer' from permanent additional random input, which makes accurate predictions about their behavior even more difficult. On the most basic level of matter, the Brownian motion of particles is completely unpredictable. If you allow me to stretch the euphemism further, the Brownian motion essentially transforms every single atom into a randomly flapping little butterfly. Keeping this picture in mind, you can see that any attempt of accurate long-term prediction is doomed to failure. Our notoriously inaccurate long-term weather forecasts are a good example.

Interestingly, we are surrounded by such complex systems in our daily life. The air that flows over our cars causes chaotic vortices behind it, raindrops run down our windows in unpredictable paths. Yet, albeit often entirely unpredictable, we are very rarely surprised by these effects. Why is that?  Because we have learned to deal with them on a macroscopic, rather than microscopic level.

Take the example of the raindrops on the window: We may not be able to predict their exact path, but we do know that eventually the water will flow down. The drop may zig or zag, but unless it gets stuck somewhere half way, it will eventually make it's way downward. Intuitively, we realize that any attempt to predict the exact motion of those drops is entirely futile. Our understanding of the world is sufficiently confirmed by the simple fact that the drops will eventually find their way down. That's all we need to know and all we can know.

The whole is more than the sum of the parts

Here is another interesting observation, which contributes to the unpredictable behavior of complex systems. 1+1 is 2. In math this is simple. In the real world, however, we have cases where the whole certainly is different than the sum of the parts.

For example, take two sufficiently sized lumps of weapons-grade Uranium. Put them together. What do you get? Not one big lump of Uranium, but instead an entirely surprising crater in the ground. Another good example is our brain. It is made up of a huge number of very simple cells, the Neurons, which are connected via the Synapses. Put it all together, and you don't have a blob of cells, but something rather wonderful and astonishing: A brain capable of thoughts and memories.

Again, we are surrounded by examples of this. In fact we are an example of this effect. Yet, we tend to not think about this at all. Why? Because in many cases, we see the result of the whole before we even realize that there are all those parts playing a role in it.

And what does this have to do with networks?

Glad you asked...

Consider today's networks: They are getting more complex, no doubt about it. There are more vendors, more pieces of equipment, more architectures, more people and computers, more use models and applications. So, not only are the individual networks more complex than in the past, they are also more unique. The uniqueness is not surprising, considering that there is this increasing number of variables, which defines the network. No two organizations will have the exact same network.

So, now imagine one of those overtaxed corporate or provider networks, operating more or less within acceptable boundaries. Suddenly, and quite possibly completely out of the control of the network operator, a new application is unleashed onto the network. Skype is an application like that. A worm outbreak is an extreme case of such an application. P2P traffic is another example, which has been building up and morphing and shifting over the last couple of years. So, what happens when something new is added to the network, may it be another application or another piece of equipment, or another batch of users? The truth is that very often, not even the network operator will know...

Many operators try to deal with this on a macroscopic level. As we have seen, this is the natural tendency for us. So they add large amounts of excess capacity to the network, hoping it will be able to deal with whatever comes their way. But of course, this is inefficient. Corporate network operators don't even have that option, since they need to worry about more than just bandwidth and availability - they also need to ensure the security of the network and its attached computers.

Chaotic complexity of networks

Quite often, we hear from potential customers that they don't even know anymore what exactly is happening on their network. That is how complex these systems have become. The fact that there is some excess capacity has often been the saving grace of those installations. Just like a huge number of chaotic moving water drops can be controlled on a macroscopic level, by forcing them all through a water pipe, many network operators have taken a step back, and simply hope that the access capacity will have the same effect: It all keeps flowing, even though we have no idea what is really going on within the pipe.

But at the moment a more fine-grained control is required, for example for detailed SLAs, or just for network security considerations, this approach fails. Then we are suddenly back to rules, signatures and policies. What do they represent? A microscopic approach to network and traffic management.

Remember the cases of the raindrops, running down the window? We have seen that their motion is unpredictable. If you observe a single drop, you can probably make a pretty good prediction what the motion will be like in the next tenth of a second (a short-term forecast), but after that, predictions will become inaccurate.

Networks have demonstrably reached a point where exact predictions about their behavior is not possible anymore. Change a little bit, add a little bit, and the outcome is unpredictable and often surprising.

The futility of rules

Why then, do we still rely on rules, policies and signatures in our attempt to control those networks? It is essentially a law of nature that a microscopic control approach is not well suited at all for a complex, and possible chaotic system.

Firstly, it has become quite impossible to write enough rules to cover all the use cases and situations for a network. Secondly, once there is just a slight change to the network, the unpredictable nature of its behavior may render many of those rules useless. As a result, maintaining rule sets and policies for complex networks quickly becomes a never ending Sisyphean task.

Taking an intelligent macro-view

Networks are becoming more complex and more unique. The number of variables that describe those complex systems are always increasing. In light of this, I propose that an operators of complex networks should not rely on controls on the microscopic level. Instead, a macro-view of the network needs to be taken. Instead of providing rules for every network condition, the overall behavior of the network should be considered. This leads us then to the field of behavioral anomaly detection.

I am not proposing that all rule and signature-based systems should be torn out of a network installation. These systems are useful to provide some basic boundaries within which the network traffic has to operate. Almost like the water pipes for the chaotic moving water. However, it is not possible to exhaustively describe the behavior of the network with those rules. Instead of wasting time and resources in attempting the impossible, it makes much more sense to complement the network architecture with a rule-less behavioral anomaly detection system. This system will be able to detect macro-trends (comparable to an observation such as raindrops run downwards), without having to know the exact detail about every single packet.

If the anomaly detection system is good, it will manufacture detailed rules and signatures on the fly, which allow the operator to handle anomalies as they happen. This allows mitigation, without having to express in rules and signatures about what to look for, ahead of time. These on-the-fly rules apply to the behavior of the network right now. Comparable to more accurate short-term forecast vs. much less accurate long-term forecast.

Conclusion

I hope I was able to provide some food for thought and highlight some aspects of the nature of complex systems. Today's networks approach complexity levels, which already result in unpredictable, chaotic behavior. The butterfly effect and the simple statement that the whole is often more than the sum of the parts, nicely describe the situation.

Chaotic systems are inherently unpredictable in their behavior. Therefore, any attempt to express predictions is doomed from the onset. If we can agree that many networks may exhibit complex and possibly chaotic behavior, then it is instantly obvious that some aspects cannot possible be covered by rule and signature sets. Writing those sets and maintaining them can be utterly frustrating and, in the end, useless.

As a solution to this dilemma, I propose the deployment of behavioral anomaly detection systems, which are able to observe the network and provide more accurate rules on-the-fly and in realtime, taking into account the current network condition and actually observed anomalies.

So, next time your network faces a meltdown, remember the butterfly and the raindrops. This is probably a good time then to remember that pre-supplied rules and signatures are not any more accurate than long-term weather forecasts, and that something more intelligent is needed in the network.

Juergen

What is Zero Day protection?

Let me try to clear up some confusion about the meaning of zero day protection. Unfortunately, many vendors of security solutions modify the definition of this term as needed, to make their products appear in the most positive light. After all, they all want to be able to say: We offer zero day protection!

Well, not so quick, please.

First of all, it is important to distinguish between two different concepts: The zero day vulnerability on one hand, and the zero day exploit on the other. Too often, those two terms are used interchangeably, even though they mean something very different.

Zero Day Vulnerability

If there is some vulnerability in a system, which nobody except the discoverer of that vulnerability knows about, then we talk about a zero day vulnerability. What this implies is that the security community and public do not know about the vulnerability, at all. Therefore there are no signatures for it and no patches. If the discoverer of the vulnerability is set on compromising other computers, then they may start to attack systems at their leisure. Everyone will be taken by surprise.

Zero Day Exploit

This is something entirely different: A zero day exploit is the term used to describe the attempt to take advantage of a known vulnerability, but with a new kind of exploit. If an IPS or IDS has rules to detect the attempt to take advantage of this known vulnerability, then they may detect this event. In effect, even though we have not seen a particular exploit before, we may still be protected, because just trying to take advantage of the underlying vulnerability will result in some specific session / packet content that can be discovered via signatures. Examples here are different mutations of the same worm, which all take advantage of the same vulnerability, but have somewhat modified code to do so.

Something in-between

One may discuss the theoretical definitions all day long. In the end, for the individual network operator, all that matters is whether their network is protected. The best signature-based system does not help, if it does not have the latest signatures, yet. For example, the Witty worm was, in effect, exploiting a zero day vulnerability for most networks. The particular vulnerability it exploited had been announced about one day before the worm broke out. Thus, the vulnerability was known to the security community. However, most networks did not have signatures for it, yet, and thus, for all practical purposes, it was a zero day vulnerability as far as they were concerned.

Zero Day Protection

So, what is zero day protection then? For vendors of traditional signature based systems (most IDSs and IPSs), zero day protection is the ability to protect against zero day exploits. They rely on the fact that they know ahead of time of a particular vulnerability. This allows them to provide signatures for the mere attempt to take advantage of the vulnerability. As we have seen with the Witty worm, this approach does not guarantee protection, or even detection of a new worm. Other trends are further contributing to the ever shrinking time window between discovery of a vulnerability and the release of a new worm trying to take advantage of it.

True zero day protection therefore cannot ever rely on any prior knowledge. For true zero day protection, a security solution needs to be able to discover abnormal behavior of hosts or networks, without needing any signatures databases. In addition, such a solution needs to be able to extract fine-grained signatures from the observed anomaly. Only if both conditions are met it is possible to architect self-defending networks, which can deal even with a true zero day vulnerability.

Juergen

Deep packet inspection - A security risk?

Yesterday, CERT issued this warning. It describes a vulnerability in a special protocol processing module of the popular SNORT intrusion detection software. Using a maliciously crafted packet, an attacker may gain control over the machine that runs SNORT (!). It is expected that the simplicity of this exploit (a buffer overflow) means that a worm will be appearing soon.

A very similar scenario took place with the Witty worm earlier last year. What do these scenarios have in common? In both cases, a deep-packet inspection security solution, performing protocol analysis, actually became a security weakness. Somewhat similar cases have also repeatedly happened with various anti-virus software packages from different vendors.

Ok, I admit it: The title of this article is probably more controversial than necessary. Firewalls and IPS solutions, which are capable of deep-packet inspection and protocol analysis, by themselves are quite useful. They do not represent a security risk any more than any normal server or desktop does...

Wait a minute...

We do assume that servers and desktops are a security risk of sorts, or else why would we deploy a deep-packet inspection firewall? Why should servers or desktops be a security risk? Because those machines need to parse the various application level requests that come their way over the network. And in doing so, they may be vulnerable to maliciously malformed requests and packets, if the code performing the parsing contains a bug. Buffer overflow exploits are quite common, caused by missing or incorrect range checking of the parameters that are contained within the request.

So, what do we do to combat the weakness introduced by code that has to perform the complex request parsing? We introduce even more code that performs complex request parsing, in the form of a deep-packet inspection firewall or protocol parsing IPS/IDS. Somehow, this does not seem to make any sense. Now, instead of one potentially vulnerable system (the server, for example), we have two potentially vulnerable systems on the network.

Of course, it can be argued that the vendors of the security solutions will be extra careful in writing their protocol parsers. But in the end, the people writing them are still just that: People. Human beings, who are prone to make mistakes once in a while, just like the authors of server and client software sometimes make mistakes.

In general, the more complex the solution, the more room there is for potential vulnerabilities. In this case, the parsing of potentially maliciously formed packets and requests can be quite complex. The attacker is in control here, because the security solution has to parse whatever was sent their way.

There are a class of security solutions, which do not suffer from that particular weakness. Ordinary firewalls, IPSs and IDSs that perform simple signature matching, anomaly detection systems that work on traffic meta data, and so on. All of these solutions have in common that they don't really parse packets or application requests. They look at fixed data fields in packet headers, for example. Not much parsing going on in that case. Or they count packets and packet types. Also, nothing the attacker sends needs to be parsed.

The moral of the story is a well-known security architecture paradigm: Don't trust a single-layer or single-solution approach to security. In this case, the ideal solution would be to pair the protocol-analyzing deep-inspection firewall or IPS/IDS with an anomaly detection system. The former takes care of well-known exploits, the latter of all the zero-day exploits and site specific anomalies. It may even produce signatures for zero-day vulnerabilities on the fly and feed them to the deep-packet inspection firewall or IPS (see here for more on that). In addition, the anomaly detection solution may also notice when the protocol-parsing solution has been commandeered by an attacker for different tasks.

Put in other words: If one needs the analysis and filtering of protocol-parsing and deep-packet inspection, it would be a good idea to have something else watching the watch-man, so to speak.

Juergen

Fine-grained filtering - revisited

Departing from the usual text-only style of my articles, today I would like to share a picture with you. It came out of an attempt to find a graphical representation of the necessity for fine-grained filters when it comes to the mitigation of network anomalies. I talked about that topic before.

The point I repeatedly made in this blog is: Unless you have fine-grained filtering capabilities in place, any attempt to mitigate a network anomaly may be comparable to doing open-heart surgery with an axe. In particular, Netflow based solutions are always severely limited in the accuracy of any filter recommendations they can produce. This is caused by the fact that they don't have access to any of the information needed to produce such fine-grained signatures. Instead, they see an abstraction of the traffic (flow-records) rather than the traffic itself.

Packet-based anomaly detection solutions, however, can see all the information they need to produce truly fine-grained signatures, because they have access to the raw packet data.

The concept I would like to introduce then is the Smallest Possible Superset (SPS) of an anomaly. The SPS describes how well the recommended mitigation filter matches the anomalous traffic. Of course, the filter should ideally cover at least 100% of the anomaly. However, if the filter is too broad, it will cover more than necessary, resulting in innocent traffic being filtered as well. The smaller the SPS the better. This graphic illustrates the point:

Illustration of the Smallest Possible Superset (SPS) of an anomaly

We can see the overall traffic in a network, illustrated as the gray area. The anomaly, for example a DDoS attack on a web-server, is presented in red. The purple area is the SPS, which is described by the mitigation filter that was recommended by an anomaly detection solution.

In case of a flow-based system, the SPS may be quite large. Imagine the web-server is under TCP-SYN attack to port 80 from random source addresses. If the anomaly detection solution does not see the raw packets, all it can do is to recommend that all SYN packets to port 80 of that web-server be filtered. Clearly, that would shut down any further activity for that server. Even perfectly legitimate connection requests would be denied.

A packet-based anomaly detection solution, however, can look for identifying characteristics in the SYN packets that make up the attack. Thus, once identified, those characteristics can be used to describe the SPS much more accurately.

Especially in conjunction with an in-line filtering system, for example a good IPS, such real-time and fine-grained mitigation filters can be very effectively implemented. Overall network impact on the legitimate operation of the network will be significantly reduced, due to the fact that the SPS is much smaller, and very closely matches the anomalous traffic.

Juergen

Anomaly Detection 101

In past postings to this blog, I have often talked about the merits of anomaly detection: How it can proactively protect networks against the unknown and how it can improve ROI on existing investments in security and infrastructure. Nevertheless, even though anomaly detection should by now be part of a best-practices approach to any network security architecture, there still is the need for customer education.

The reason for this is that traditional security solutions typically were based on deterministic  rules. Firewalls blocked specified ports. IPSs/IDSs blocked/detected certain pre-specified signatures. We now know that such deterministic approaches are not sufficient anymore when faced with modern threads such as zero-day worms or rapidly changing DDoS attacks. Therefore, anomaly detection has become a necessary addition to any multi-layered security approach.

Because the deterministic security solutions have dominated our thinking whenever network security was considered for such a long time, it is sometimes difficult to appreciate how anomaly detection differs from that approach, even if the business values are in theory obvious.

Therefore, in this article, I would like to give a brief introduction into how we at Esphion perform anomaly detection.

What Esphion's anomaly detection is NOT

First, it is important to understand what we do not do:

Our anomaly detection does not rely on any pre-specified rules, any baselines, any models, any signatures, or any other prior knowledge.

This is a very important point. In the moment you use prior knowledge for anything, you are faced with two problems:

1. How do you configure that knowledge? Do you sit down and write rules? Do you personally tell the system what those rules are? How long does that take you and how flexible is this approach?
2. What happens when things change? Do you have to re-configure the system? How long will it be inoperable when that has to happen?

IPSs and IDSs require constant updates to their signature databases. Firewalls need updated lists of ports to block or allow. This configuration needs to be performed regularly. If a new threat or anomaly emerges, then those solutions are blind to this until they have been updated.

Please note that even many vendors in the anomaly detection market are still requiring prior knowledge. This often does not come in the form of explicit rules and signatures, but instead requires those solutions to baseline the normal behavior of the network. This usually takes some time during which the system cannot report on anomalies, but instead simply observes the traffic in the network. This time is usually called the bedding-in period. How long this period is depends on the vendor's specific approach. However, many vendors recommend several days or even weeks. Obviously, if the usage profile of your network changes considerably, this bedding-in period needs to be repeated. Consequently, such solutions are not very scalable in real-world network environments.

Esphion's approach to anomaly detection is completely independent of any prior knowledge. Not only do we not use any established rules or signatures, but we also do not require a bedding-in period. Once installed, Esphion's solution is virtually instantaneously ready to report on observed network anomalies. We recommend for around two-hours to pass after installation, but that is all. This time is also not used for any baselining activity, but instead to prime a statistical pre-processing pipeline in our system. Once this is done, however, this step does not need to be repeated, even if the network environment should change.

So, what's important to remember from all of this? In the moment you rely on any prior knowledge of any kind, whether configured or learned, you are already at risk. Esphion's solution fortunately does not have this problem.

The basis of detection: Traffic meta-data

Our approach to anomaly detection is scalable, by restricting heavy-duty traffic analysis to when it is really needed. Therefore, much higher packet rates can be supported than in the case of IPSs or IDSs, which need to perform in-depth scanning of every packet.

Instead, our anomaly detection utilizes data about the network traffic. In effect, the presence of anomalies is detected by means of traffic meta-data. As network packets pass by our listening sensors (we call them agents), we merely record statistics about this traffic. For example, how many TCP packets, how many UDP packets, etc. Slightly more detailed, we may record how many TCP-Syn packets we see, how many TCP-Fin packets, and so on. We keep track of a few thousand such statistics.

This data is constantly collected and forms the foundation on which our anomaly detection is build. How so? As it turns out, one can detect even the onset of network-impacting anomalies by careful examination of those statistics. Under normal usage conditions, these statistics behave in certain ways relative to each other. In the face of an anomaly, for example a DDoS attack or a worm outbreak, the way those statistics relate changes subtly.

The brain of detection: Neural networks

If you were to plot the various statistics that we collect about the network traffic, you would see how they are changing during times of network anomalies. The changes would be subtle, and it would not look very specific to the casual observer. Even so, you would probably be able to realize that something strange is going on. You can do that, because every human being has a great pattern recognition engine: The brain. It may not be good at quantifying (what exactly is going on and to what degree), but certainly good at qualifying that there is something amiss.

To provide this capability in an automated fashion, Esphion utilizes neural networks to observe the traffic meta-data. Neural networks, as we know, are a computer-based emulation of the brain's neurons. These neural networks know how to recognize that the network traffic is changing in ways that are not seen during normal operation. So, in effect, the neural networks act as a 24/7 intelligent observer of network meta-data.

The footwork of anomaly detection: Zero-day signature extraction

Remember that so far we have only worked with traffic meta-data: Light-weight statistics about the network traffic. However, in order to properly mitigate an anomaly a little bit more information is needed. Since Esphion's agents are listening to the raw network traffic, we have access to all the information we need, including data contained in the packet headers and payloads.

Once the neural networks have detected the presence of a network anomaly, the agents are instructed to capture actual sample traffic. We then perform a more CPU intensive analysis only on this truly relevant, pre-qualified traffic. This analysis looks to find any patterns, which uniquely distinguish this traffic from other, normal traffic on the network.

Note that this is not the comparison of observed traffic to pre-configured signature databases. As discussed, that would be the wrong approach. Instead, the analysis starts with no assumptions at all and simply detects if there are elements in the observed traffic, which are unique to the packets that are part of the anomaly. As a result, we get very fine-grained zero-day signatures, usually within seconds after the onset of an anomaly. No matter if it is a DDoS attack, a worm outbreak, or a network malfunction - very quickly the network operator has a detailed signature in their hands, which can then be used to surgically remove the offending traffic.

Conclusion

Well, there it is: A high-level overview of how our anomaly detection works. The key points are that there is absolutely no prior knowledge required, and thus, the system can not be blind-sided by zero-day anomalies, changing network conditions, or long bedding-in periods. Combine this with intelligent neural networks, and the capability to observe the raw network traffic to extract zero-day signatures, and the result is a truly powerful additional layer of security, which should be present in any mission critical network.

Juergen

Rescuing the IPS

Lately, as we are talking to customers and partners, we are increasingly hearing one particular message more and more often: Disappointment in the performance of IPSs ... budgets being reallocated from IPS deployment projects to NBAD projects. In this article, I would like to explore where the disappointment comes from, and how IPS deployments (and investments) can be rescued.

The wonderful world of IPS marketing

Unhappyness with IPSs. Where does this remarkable turn of events come from? After all, not too long ago, IPSs were heralded as the be-all and end-all of network security. The one-stop-shop for all that is required to keep an enterprise's data, hosts and networks safe and sound.

Even then, though, many security analysts already pointed out the accepted best-practices approach: There should always be multiple layers to any network security architecture. Do not rely on a single point solution. However, there is something inherently attractive about an IPS for any organization. and this message proofed to be more powerful than any warning: Here is one box, which can act as a firewall, but also as a much smarter filter on the data that needs to be let through. It may even scan e-mail for viruses. And it is all updated automatically, remotely, right? No problem then! If a new worm or virus should come around, there will be new signatures for it uploaded on the device in a hurry, and my network will be safe, right? One box to secure it all...

Too much hype

Ironically, this simple and effective marketing message now turns out to be the IPS's undoing. Even more ironically, the technology of the IPSs is fundamentally sound. These are good devices, which really work. The problem is that claims for their capabilities have been blown out of proportion. Therefore, customer expectations have been elevated to levels, which the technology cannot meet in the real world.

The prime example for this is the claim that IPSs can protect a network against zero day attacks or anomalies. But the simple fact is that a signature-based pattern matching mechanism, the core technology of an IPS, can never detect zero day exploits or anomalies. Even those IPSs which can detect RFC violations in protocols are essentially matching a signature (the description of the protocol in the RFC) against the observed traffic.

What's the problem?

If your network security relies on signatures then you have to ask yourself: Where do those signatures come from? In the case of IPSs, the signatures are generated by human beings in the IPS vendor's data center. Skilled security specialists are working there to examine any newly found type of malware or anomaly, and generate signatures suitable for their device. These are then uploaded to all the installations out there. But there are of course two significant shortcomings to this approach:

1. The vendor's data center only sees those anomalies and traffic samples that are reported to it, or which they managed to collect themselves. If your enterprise is hit with a very specific attack or anomaly, then the vendor's data center does not see this traffic. Therefore, there will not be any signature for this anomaly forthcoming. The IPS in your network will remain blind to it.
2. In the moment humans are involved, things take time. Typically, we can expect vendors to have signatures for new anomalies that they know about (!) available within a few hours or days. So, even under the best of circumstances, in case of a true zero day anomaly, the network will face hours without any protection at all.

So, what happened in many of those networks in which shiny, new IPSs were installed? Time and again, even those networks protected by IPSs were taken down or were affected by zero day outbreaks, anomalies or attacks. And obviously, any organization that bought into the marketing message of the IPSs, is disappointed (to say the least) when this happens after a significant investment into IPS solutions.

What IPSs are good at

I said earlier that I believe IPSs are in fact good devices, with good technology. You may wonder how I can say that in light of the disadvantages I just listed. Well, I also said that the problem was mostly with overblown customer expectations. I really do think that IPSs are very good at what they are designed to do: Look for patterns, even deep inside of packets or connections, match those against some known set of signatures, and perform some specified action based on that. This is all, no more no less. Many IPSs have become very good at this, and provide wonderfully fine-grained means to examine and filter traffic.

Clearly, though, this impressive ability does not help with any zero day anomaly, anything that is specific to the network in which the IPS is deployed, or anything the IPS does not know about in advance.

How to rescue the IPS investment

Many organizations that have invested heavily in IPSs are now wondering how they could better leverage this investment. Is there a way to effectively use IPSs even when faced with true zero day anomalies? Glad you asked, because as you can imagine, I think we have an answer...

As we have seen, IPSs are very good when they know what they are looking for. If they have accurate signature databases, then they will be able to find whatever matches those signatures. If it is not in there, however, then the IPS is effectively blind. So, the key obviously is to get the signatures for any zero day anomaly into the IPS as quickly as possible.

This cannot be achieved if we have to rely on some remote data center, in which humans work at human speed on a selective set of anomalies that they have been made aware of. We need something that can look at whatever anomaly or issue your network is facing right now. The anomaly must not only be detected, but also must be analyzed to the point where an accurate signature for this anomaly can be provided. Automatically, and rapidly.

The solution you are looking for is a packet-based network anomaly detection solution. In Esphion's netDeFlect, we do not only have the ability to detect anomalies such as DDoS attacks, worm outbreaks, misbehaving applications and other network impacting events in seconds, thanks to our specially trained neural networks. In addition, we also have zero day signature extraction modules. These analyze the anomalous traffic right there, in your network, and extract the characterizing signature for the anomaly. All of this takes place fully automatically, and within just seconds.

The network operator can then choose to have this signature translated into a variety of formats, router ACLs, IDS signatures but also IPS signatures. These are ready to be applied to the IPS, which then becomes an instantaneous mitigation device, even for zero day anomalies, that are entirely local to your network. For additional information about how this works, and how you can get a powerful, self-defending network security architecture out of this, please see my article here.

Conclusion

So, what do we get out of all of this? I think the message I would like to get across is this: IPSs are good devices, but even they need to be part of a multi-layer security architecture. This is best practice, and has held true in the past, and continues to hold true, even in the age of powerful, multi-function devices, such as IPSs. IPSs are good at what they are doing: Finding what they know about. But for zero day anomalies, an IPS can only be effective if it is supported by one of those additional layers in the security architecture: An intelligent anomaly detection system, which has the ability to extract fine-grained signatures even for zero day anomalies.

The capabilities of such an anomaly detection solution and of IPSs are the perfect match: Fine-grained signatures are derived out of the anomaly detection and analysis. And only IPSs have the fine-grained filtering capabilities to really take advantage of those signatures.

So, if you have already invested in IPSs, but would like to really leverage and protect this investment, by using those IPSs even during zero day anomalies, do consider the deployment of a network anomaly detection solution. That is how to get the most out of your IPS.

Juergen

What is 'proactive detection'?

We like to describe netDeFlect, our anomaly detection solution, as proactive. However, you might ask, how can detection be proactive? After all, detection by definition takes place as or after something happens. Being proactive, however, implies that something is done before an event takes place. The dictionary definition of proactive (according to dictionary.com) is:

pro·ac·tive or pro-ac·tive
adj.
Acting in advance to deal with an expected difficulty; anticipatory...

Clearly, therefore, detection cannot be proactive.

In our case, though, the word proactive does not refer to the actual detection of an anomaly. As good as it is, even netDeFlect cannot detect an anomaly before it happens, of course. Our research department always works on great new technologies to be added into our products, but the ability to see into the future has not been implemented, yet.

In the context of netDeFlect, the word proactive means several things:

• The solution continuously adapts autonomously and does not require ongoing maintenance or continuous uploading of updated signatures. In other words, no specific action needs to be taken before an anomaly takes place. Without updates or ongoing maintenance and configuration, netDeFlect is always ready to detect a wide variety of network anomalies. In that respect, netDeFlect is proactive: It constantly adapts its detection mechanisms to the current network conditions. This is an always ongoing process, independent of the presence of any anomalies. Therefore, netDeFlect actively adapts. It acts in advance (constant adaptation) to deal with an expected difficulty (a network anomaly). This, of course, matches the dictionary definition of proactive. As a result, even previously unknown types of anomalies (zero-day) can be detected, analyzed and characterized.
• But not only netDeFlect is proactive. The network operators and organizations, which utilize netDeFlect, now can become proactive in their response to security incidences as well. Rather than waiting for a DDoS attack or a worm to take down servers, routers, or the entire network, they can now start to mitigate the anomaly before this happens. For example, netDeFlect can detect worm outbreaks or DDoS attacks within seconds. Also within seconds, the anomaly has been characterized and signatures have been extracted from the observed packets. Therefore, the anomalous traffic can now be filtered. From detection to successful mitigation, only seconds will have elapsed. So, the network operators proactively shut down the anomaly, before it could bring down the network, or the servers, or before a worm could infect a large number of hosts.

The detection of something can never be proactive. But as we can see, the way netDeFlect operates certainly is proactive, and the new capabilities that it provides to network operators, are so as well.

Juergen