Posted by: Greg Ness | April 18, 2008

Taking the Pain out of Network IPS

Last month I blogged about virtsec being the final straw for tired deep packet inspection-based IPS. Let’s now take a step back from the future disruption posed by mutating pools of VMs moving undetected behind static pattern match IPS and address the core challenges facing data center intrusion prevention today.


Almost two years ago the network security industry noticed the emergence of new classes of highly effective IPS evasions.  It became obvious that hackers had gone pro and were learning how to disguise their exploits in ways that could evade pattern match (signature or anomaly) detection.


Polymorphic Hacker Bonanza

Patterns depend on prior knowledge.  Global networks of security experts still watch for new patterns to quickly deploy protection before too much damage is done; but mutation renders pattern match security worthless. I discussed this last year in Where’s Waldo Goes Polymorphic. 


The rise of exploit mutation is also feeding the rise of voluminous libraries of short-life libraries, the rise of processing requirements for traffic inspection, the rise in signature tuning requirements and draconian tradeoffs between different kinds of intrusion prevention, latency and service impacts.  Every credible review of deep packet intrusion prevention systems includes asterisks (qualified comments) and/or frank discussions of latency and service disruption.  Many emphasize the network IPS ability to manage the noise produced by false alarms or praise the cottage industry of noise management solutions.


IPS solutions can also be evaded by the likes of IP fragmentation, buffer overflow attacks (with disguised payloads) and SQL injection.  While heuristics and packet scanners can help to identify some forms of these attacks, hackers have found ways to evade them through alphanumeric, metamorphic and mutating shell codes that cannot be easily identified; they can pass through the perimeter blending in with the vast majority of innocent traffic.


Enterprise CSOs depending on pattern match (plus this and/or that add on) are about to learn a new reality.  The old world devices aren’t working like they used to work.  At the dawn of production virtualization (server mutation in the data center) the network IPS is already crumbling under the pressure of exploit mutation outside the perimeter.  See my blog from February 2007- Virtsec: the Beginning of the End of Static Security.


Security Needs to Migrate from Layer 4 to Layer 7


All of these evasions can work because deep packet pattern match doesn’t have the protocol intelligence to recognize the attacks with high enough levels of accuracy.  Another problem is that these “one size fits all enforcement systems” can only alert (on) or block suspicious traffic.  Besides the latency incurred because inspecting traffic for more patterns ties up more processing resources; blocking terminates server sessions reducing system availability.  Security has never been important enough to impact service availability; and IPS has never been accurate enough to trust more than a small population of signatures for blocking.  So most protective signatures are disabled as not to mistakenly disrupt services.


So round and round we go watching the hacker and marketing arms race escalate between tired hardware, tired architectures and mired security pros fighting off innovative “open source” professionals probing perimeters for wealth and information assets.  One side gets perpetual defense and the other perpetual offense.


There is a Better Way: Protocol Fluency


Data Centers can use more than a hundred protocols between dozens of different operating systems, applications and databases and users.  Most IPS systems understand only a fraction of those protocols.   Every protocol not covered is a vector that can be exploited by a hacker.  It is a point of pattern match evasion.


Protocols, Proactive Protection and Promises


That’s why I predict that we are about to see a netsec protocol and vulnerability race that will be like nobody’s business.  As security teams realize that deep packet is in deep trouble (as far as netsec is concerned) more vendors will start touting protocol coverage and vulnerability intelligence.  Who was it this year at RSA who talked about security needing to think inside out?


There are about 130 data center protocols, depending on what you’ve deployed.  How many does your IPS understand? If you’ve standardized that number may be smaller.  But it’s a fair question to ask your vendor, especially if you’re experiencing successful evasions.


There are also hundreds of unique software vulnerabilities across leading data center operating systems, applications and databases.  Don’t confuse signatures with vulnerabilities, although some vendors do try to blur the line with the “virtual patch” marketing concept.  One security reseller determined that an entire library of signatures with one leading system only protected about 30% of known data center vulnerabilities across Microsoft, Oracle, Solaris, Linux, Apache, etc.  And that’s with the rare instance of all signatures turned on!


That means assuming that a large signature library means comprehensive protection is a head fake at best.  It does explain CSOs who testify about millions of false alarms a week being an excuse for missing a real attack: lots of noise from false alarms (the large library identifying suspicious traffic) while hackers evade the static pattern match systems with mutation and exploit unprotected vulnerabilities.


Then there is an architecture consideration.  If an IPS is using brute force pattern match on all traffic in combination with some protocol decoding, it is very likely that the decoding will be “slow path”.  That means that it will require more processing resources and may force additional latency/coverage tradeoffs.  Many network IPS architectures already require Draconian tradeoffs between latency and protection because they have to equally inspect all traffic.  Additional specialized processing of traffic has the effect of extra cars on the rush hour freeway; each one beyond a point adds a disproportionate influence on delay.


That’s why the next front in the IPS war will be about accuracy-enabled exception-based systems.  By exception based I mean that they will be able to quickly parse out innocent traffic and focus controlled code on the traffic heading to known vulnerabilities.  An example of this approach is in this 19 page white paper by Blue Lane CTO Allwynn Sequeira.  It is a vendor paper (my apologies) but I think it is the most brilliant articulation thus far of what we’ll call a data center IPS.  And the industry needs fresh thinking.


Last year I predicted the rise of virtsec as an issue.  Now I’ll go on the record and announce the race to proactively secure the next generation data center (physical and virtual infrastructure) against new classes of attacks.  That race will be won by the vendors who understand all key protocols, all key vulnerabilities and can act with precision on exploits, and in ways that have very little impact on latency.  From Oracle databases to Windows servers and hypervisors holding pools of VMs, the industry needs to shift its traditional focus away from exploits and desktops and turn it to core vulnerabilities and protocols.  Until then we’re all a hack away from losing trust in yet another institution, or having to testify why the system you bought simply couldn’t keep up.




Disclosure: I’m the VP Marketing for Blue Lane Technologies, a winner of the 2007 InfoWorld Technology of the Year for security, Best of Interop 2007 in security and the AO 100 Top Private Company award for 2006 and 2007. Blue Lane is also a 2007 Best of VMworld Finalist in data protection. I’ve been a marketing executive at Juniper Networks, Redline Networks, IntruVert Networks and ShoreTel. I’ve been an Always On blogger/columnist since 2004. My recently launched personal blog is: .  These are all my opinions, and do not represent the opinions of employers, spouses, kids, etc.


Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: