VMware and Deep Packet Apocalypse

Earlier this week I spoke at a database security event about the inevitable virtsec architectural shift from (layer 4) core deep packet inspection to application protocol context (layer 7). A few hours after my preso a netsec vendor presented a virtsec vision of agents/sensors deployed at numerous key checkpoints across a virtualized infrastructure. It was indeed a beautiful slide. One of the attendees asked if the architecture was layer 7 and he replied “yes, deep packet inspection, layers 1-7, we do everything.”

Rather than single out a particular vendor, let me describe what I see taking place in coming months/years for the network IPS space. Those first generation IPS deep packet architectures charged with inspecting ALL traffic passing through them (and warning of suspicions and terminating sessions based on qualified suspicions) will have to really scramble to keep up with the demands of server and VM security. Doing everything, as the vendor claimed, is enigmatic when your core deep packet architecture is based on pattern matching. There are at least two fundamental problems, from which many others spring.

The first problem is driven by a fundamental requirement for any core pattern recognition system: accuracy and reliability depend upon stability. A suspicious attack needs to stay suspicious. And past suspicious attacks need to represent most of the future suspicious attacks. Mutation/change when it comes to attacks means an increasing likelihood of obfuscation and evasion for pattern matching architectures.

Yet exploits are now mutating at a fast pace; some can even mutate in the midst of an attack. Mutation accelerates the obsolescence of static pattern recognition.   I talked about this in Attack of the Mutant Bots months ago at Always On.

The second problem with exploit pattern recognition is that it can become resource-intensive. All traffic (“everything” as the vendor commented) passing through an appliance (or checkpoint) is inspected. As more patterns (and permutations) are added more processing is required to keep up. Now imagine that deep packet inspection IPS deployed via scattered agents at an array of key checkpoints between VLANS, hypervisors and servers in a partially virtualized data center.  Larger and larger libraries are pattern matched in multiple checkpoints against all traffic pulsing back and forth through meshes of VMs and servers.

Rhetorical Questions Worth Asking

Exactly how many processor cycles will be needed to operate these scattered full traffic inspection points? How much latency will be produced at how many concomitant places? How many false alarms will ripple through these multiple points and create more noise and confusion? How many sensors will be required between each zone (or fuzzy perimeter) to keep each isolated to its own level of security?

This “chokepoint architecture” is a substantial barrier to both the accrual of the benefits of virtualization (movement, flexibility, mutation, utilization, etc) and the effective protection of the data center. It will force Draconian tradeoffs of a massive scale relative to the single point tuning at the outer perimeter, where firewalls function well to limit access to particular ports and segments.

>>Deep packet inspection within the server or VM mesh means mushrooming processing and security management requirements in a scenario now having to inspect and track mutation inside AND outside multiple new fuzzy perimeters. <<

Deep packet-centric inspection intrusion prevention, for this reason, is the enemy of VMware and every vendor betting on the fast adoption of virtualization in the data center. The Draconian tradeoff of the pattern match requirement in a fluid environment is, however, the best friend of an appliance-centric old guard that has monetized specialized hardware and security as a service (complexity equals revenue) models that understands the impact of virtsec on their business models.   I see a similar “when do you virtualize” struggle about to take place between the vendors of commodity and specialized processors.

Everyone agrees that virtualization of the data center is inevitable. The question is when. And the answer to that question will have a substantial impact on market caps across many appliance categories in levels that we perhaps haven’t seen for awhile.

That is why VMsafe at Cannes drew such a reaction. It is a declaration of war on a tired status quo that has produced minimal innovation in recent years (especially relative to the black hats/malicious hackers). It is security’s next big hope or failure, on multiple fronts; and as VMware plans next steps the enemies of virtualization will naturally assemble armies of experts, pundits and partners in defense of a past that is already dying. They will do their best to distract the market with rumors of hypervisor attacks and ongoing risk debates, until they are ready to take the plunge.

That is why one blog’s recent description of virtsec as a defibrillator for the security industry resonated soon after VMworld.

I also think that it is inevitable that the network intrusion prevention space will bifurcate into packet inspection at the perimeter (integration into exploit-centric next generation firewalls) and advanced layer 7 server IPS systems (in front of servers, databases and VMs) that are more accurate, use less processor cycles and can protect systems without heightened availability risks, latency and detection obfuscation headaches.

Now that VMware has crossed the Rubicon and put the data center on notice, every deep packet-centric network IPS architecture (with or without some layer 7 add-ons) will face a choice of where to go (outer or inner perimeter), and it will make all the difference.

Comments

[nick gogerty]

counter mutation will unfortunately always be a responsive action. In theory the next game beyond defensive mutations will be speed. The ability to rapidly transmit attack profiles across networks. Curious has anyone charted the growth rate of mutations and counter defenses relative to absolute traffic growth. Will the ecosystem grow faster than the function set?

[GNESS]

Nick: great question. We've charted a single mutation against MS 06-040 since MSFTs announcement almost two years ago and we showed dozens of mutating bot attacks (zero day exploits) and of course corresponding signatures from the deep packet pattern match vendors. I think speed is a must but at the end of the day countering mutations with growing pattern histories is still a reactive posture that accepts a level of vulnerability and requires "fire hose" enforcement (compute-intensive) versus layer 7 complete app/protocol decode and exception processing.

Then there are the other issues that are problemmatic from the standpoint of pattern match defense: SQL injection; cross-site scripting;and layer 2 evasions (like IP fragmentation).

The data center that has both servers and VMs in a mesh will require more flow visibility, more vulnerability intelligence and more app knowledge than the traditional systems are capable of delivering. I think these devices will stay at the perimeter (because they're exploit-focused) while a new data center (or server/VM) IPS category will take shape as meshes replace pipes.

Thanks for your comment./
Greg

[steveballmer]

Microsoft has gone as deep as 12!