Mais conteúdo relacionado Semelhante a Tolly Group Report: IBM Security Network IPS GX7800 Appliance (20) Tolly Group Report: IBM Security Network IPS GX7800 Appliance1. #212148
November 2012
Commissioned by
IBM Security Systems Division
IBM Security Network Intrusion Prevention System GX7800
Comparative Efficacy and Performance Evaluation
Executive Summary The Bottom Line
Enterprise-class networks today are facing more advanced threats from a
multitude of sources than ever before. Effective threat protection solutions must The IBM Network Security IPS GX7800:
defend against real-world threats that are evolving quickly, and at the same time
deliver high levels of performance and availability. IBM commissioned Tolly to
evaluate their protocol-based Network Intrusion Prevention System (IPS) GX7800 1 Delivers superior protection from evolving
threats with high levels of performance
and compare its efficacy to that of a Snort-based device, a signature-based
platform.
2 Stopped 99% of tested, publicly-available attacks
Tolly engineers conducted many different performance tests with the GX7800
and achieved a maximum of 35.7 Gbps throughput under mixed traffic
loads. This demonstrates a great tolerance for network surges, growth and 3 Was nearly twice as effective as Snort at
stopping mutated attacks
capacity over IBM's published performance characteristics. Tolly also evaluated
the IBM IPS GX7800’s efficacy and functionality.
Tests showed the IBM IPS GX7800 to be more effective blocking publicly-
available exploits than Snort and dramatically more effective when blocking
4 Protected streams of 100% HTTP traffic at
speeds of 20 Gbps and mixed traffic loads at
mutated exploits - blocking 100% compared to 52% for Snort. See Figure 1. over 35 Gbps
Inline IPS System Efficacy Against Publicly-Available (PA) and Mutated Exploits
IBM IPS GX7800 vs. Snort IPS
Publicly-Available Exploits Blocked Mutated Exploits Blocked
(Out of 74) (Out of 31)
99% 100 %
91 %
Exploits Blocked (%)
Exploits Blocked (%)
52%
IBM IPS GX7800 Snort IBM IPS GX7800 Snort
IBM IPS GX7800 Snort
Source: Tolly, October 2012 Figure 1
© 2012 Tolly Enterprises, LLC Tolly.com Page 1 of 6
2. IBM IPS GX7800 #212148
As enterprise IT has evolved, network Mutated Threats Blocked
security should keep pace. Today’s threats are
As with the AV industry, the Internet is host
IBM Security
more refined, diverse, and potentially
harmful than ever—and as a result they to an ever-expanding number of threats. Systems
require new and intuitive solutions to offset You can think of signature-based solutions as Division
their negative impact. a face recognition system and the mutation
Traditional signature-based IPS solutions as a mask that “mutates” the face and can IPS GX7800
don’t protect against the evolving threats confuse the face recognition system.
that are ever-present in today’s enterprise Signature-based solutions have difficulty Efficacy and
environment. Signature-based IPS solutions keeping pace when threats are mutating by Performance Tested
can protect against an exploit once it is the thousands. In order to replicate these
Evaluation October
known, but offers less protection against mutations, engineers deliberately altered the
threats that have mutated . payloads of the tested exploits. This was
2012
Using its protocol analysis module (PAM), the accomplished in most cases by changing the
IBM GX7800 is able to decode the name of a single variable within the exploit With 44K objects, the IBM GX7800 delivered
application traffic and identify malicious code. over 19 Gbps in “drop” mode and more than
code in any form, helping to maintain a more The IBM GX7800 stopped 100% of mutated 24 Gbps in “forward” mode.
secure network than signature-based IPS threats, while the signature-based Snort The IBM GX7800 delivered identical results in
alone. Furthermore, the engine is extensible solution stopped half (16 out of 31) of the both modes for Core IPS and Enterprise IPS
and can cover more than just vulnerabilities mutated exploits. See Figure 1. traffic profiles, demonstrating 35.7 Gbps of
(e.g SQL injection and shell code). The IBM throughput for all four scenarios (Core IPS
GX7800 is only part of the solution that IBM Performance Test Results drop/forward and Enterprise IPS drop/
provides. Behind the scenes, IBM’s X-Force forward). See Figure 2.
Research and Development Team In today’s enterprise environment, security is
a must. However, performance is just as
proactively seeks out new threats,
important for large deployments.
Features/Functionality
incorporating this insight back to the
appliance via software updates. Organizations need to remain online and Though some features can be viewed as
secure at multiple 10GbE speeds. “nice to have”, a certain usability of an
Test Results Engineers verified the performance of the
IBM GX7800 using Ixia’s BreakingPoint
effective system should not be overlooked.
The IBM GX7800 provides a variety of
features/functions that make its deployment
Efficacy Test Results FireStorm in both “drop” and “forward”
modes across a range of object sizes that
and management intuitive and easy-to-use.
Publicly-Available Threats Blocked included streams of pure HTTP traffic as well From the dashboard, administrators are
as streams containing mixes of enterprise greeted with an “at-a-glance” look into the
IBM X-Force gathered exploits from the X-
and core traffic types. overall network security, including recent
Force Database where they publish all
events, and general statistics for many of the
disclosed vulnerabilities and exploits from Tolly test results show that the IBM GX7800
modules and threats.
many sources. can maintain high levels of performance in
both “drop” and “forward” modes. Testing By default, the IBM GX7800 is equipped with
Tolly engineers tested the IBM GX7800 and
with “drop” mode enabled disallows any powerful policies in the form of X-Force
the open-source Snort1 device against a
traffic beyond what the device could scan at Virtual Patch, a collection of protection
corpus of 74 such threats. The IBM GX7800
a given time, whereas with “forward” mode algorithms which are specifically designed to
stopped 99% (73 out of 74) of the exploits,
enabled, excess traffic is forwarded to the not generate false positives and act as a
while the open-source Snort device blocked
network without having been scanned, but stand-in for many application patches.
only 67 out of 74. See Figure 1.
is important for business continuity.
1 Snort® is an open source network intrusion prevention and detection system (IDS/IPS) developed by Sourcefire.
© 2012 Tolly Enterprises, LLC Tolly.com Page 2 of 6
3. IBM IPS GX7800 #212148
IBM IPS GX7800 Aggregate System Throughput
8X10GbE Ports In Drop Mode and Forward Mode
As reported by Ixia BreakingPoint FireStorm
Aggregate Data Rate (Gbps)
35.7 35.7 35.7 35.7
20.8 24.1
19.5
12.0 10.3
7.8
10K 21K 44K Core IPS Enterprise IPS
HTTP Traffic (100%) Mixed Traffic (HTTP, FTP, SMTP, DNS, etc)
Source: Tolly, October 2012 Drop Mode Testing Forward Mode Testing Figure 2
In addition to providing high-performance Testing was conducted within a VMware ESXi The Snort instance was configured with the
security, the GX7800 is also host to an array 5.0.0 environment. Multiple clients were latest VRT (Vulnerability Research Team)
of other functions including a simple deployed and configured to be vulnerable to updates as of October 16, 2012, and was
Network Data Loss Prevention (NDLP) the exploits tested. Snort was deployed on a running Snort Engine version 2.9.3.1, set to
module and web application protection. CentOS client with 1vCPU and 2GB RAM. inline mode. To ensure the highest security
These inclusions transform the GX7800 into Multiple VM Networks were created within settings were used, engineers configured all
an all-purpose network security appliance. the host to segment the “Attacker” network Snort Rules and SO_Rules to “Block” by
The GX7800 can also be deployed in high from the “Vulnerable” network. default. Both the plaintext rules and
availability scenarios within an organization obfuscated, shared object rules were used
to further increase the amount of stateful
traffic which can be inspected. Likewise,
multiple appliances can be deployed in IBM Security Network Intrusion Prevention Solutions
geographically disparate sites within an Product Specifications
organization. In this topology, appliances Performance Characteristics
share information with each other and can
• 200 Mbps to 20 Gbp + aggregated throughput (depending on model)
be centrally managed from a single console.
• 1.3M to 21M simultaneous connections (depending on model)
• Less than 150 microseconds of latency (less than 75 microseconds for GX7 models)
Test Setup & Core Capabilities Availability
• Virtual Patch technology • Active/active high availability
Methodology • Web application protection • Redundant hard drives and power supplies
• Protection from client-side attacks Research/Updates
Test Bed Setup • Data and content security
• Application awareness
• Updates powered by IBM X-Force research team
Engineers deployed an environment • X-Press Updates - automated updated delivery
Protection Modes Management Options
consisting of one IBM Network Security
• In-line protection • Local web-based management
Intrusion Prevention System GX7800, which
• In-line simulation • Centralized management via IBM SiteProtector
was equipped with 8x10GbE ports, running • Passive monitoring
firmware v4.5 with security content XPU For more information, call 1-877-426-3774 or visit:
version 32.090. http://www-01.ibm.com/software/tivoli/products/security-network-intrusion-prevention/
Source: IBM
© 2012 Tolly Enterprises, LLC Tolly.com Page 3 of 6
4. IBM IPS GX7800 #212148
for the test. Snort claims coverage for all headers, with a time delay of 2 seconds Tests were run for 5 minutes in both Drop
common vulnerabilities and exposures (CVE) between trace replays. and Forward mode configured on the IBM
used in testing. GX7800. Due to the nature of the
All 74 base exploits and 31 mutations were
BreakingPoint traffic, certain DOS attacks
run through the environment and
Test Methodology repeatable results were reported by Traffic IQ.
would occasionally trigger when tests were
running. For the purpose of the performance
For the efficacy testing, Metasploit
For the Performance tests, Tolly utilized an testing, these rules were disabled to allow
Framework 4.5.0-dev-15713 was used to
Ixia BreakingPoint FireStorm system version the traffic to flow without error.
create payloads and deliver the exploits to
3.0 b105019. Engineers tested three HTTP
vulnerable hosts. Publicly-available exploits Engineers also injected a 6-attack StrikePack
object sizes (10K, 21K, 44K) with 250 clients/
were used for all CVEs. ‘Mutants’ were to verify that attacks were being detected
servers per AppSim profile, with two on each
exploits with various changes to the code while under heavy load. In no scenario were
port to get bidirectional transactions.
(such as changing variable/function names) attacks allowed through the GX7800.
Additionally, engineers used the Core and
which produced the same outcome as the
Enterprise IPS traffic mixes to stress the
original exploit. This is a common approach
GX7800. These mixes contained HTTP, SMTP,
used to attack systems. See Figure 4.
SIP, FTP, DNS and other stateful traffic.
Initially, exploits were run without any
security solution inline and packet captures Test Equipment Summary
were created for both sides of the
Vendor Product Web
conversation.
Idappcom Traffic IQ Professional v2.0.299
was used to replay both sides of the Ixia BreakingPoint
Ixia
conversation through interfaces on the FireStorm V3.0 http://www.ixiacom.com
vulnerable and attacker networks while each
IPS device was connected in inline mode.
Traffic IQ was configured to rewrite HTTP
Intrusion Prevention System Test Environment
Source: Tolly, October 2012 Note: For Snort testing, the IBM IPS was replaced by the device running Snort. Figure 3
© 2012 Tolly Enterprises, LLC Tolly.com Page 4 of 6
5. IBM IPS GX7800 #212148
Test Data: Mutation Examples and Common Vulnerabilities
Mutation Example 1: Renaming Variables
Many of the tested exploits contain variable names. These variable names are irrelevant to successful exploitation, and therefore
cannot be depended upon for detection of exploit attempts. In order to test mutated versions of the exploits, we simply altered the
variables names, as shown in the examples below:
Original Variable Names Mutated Variable Names
Shellcode somecode
Block brick
heapLib badLib
While these changes had no impact on the effectiveness of the exploits, they did allow the exploits to go undetected by the signature-
based Snort solution.
Mutation Example 2: Renaming Class References
Many of the tested exploits contain references to classes contained within Java archives. Because class filenames within an archive
are variable and arbitrary, they should not be depended upon for detection of malicious activity. In order to test mutated versions of the
exploits, we simply altered the referenced class names, as shown in the example below:
Original Class Reference Mutated Class Reference
<html><head></head> <html><head></head>
<body><applet archive="jmBXTMuv.jar" <body><applet archive="eXRZLr.jar" code="msf.x.badguy.class"
code="msf.x.Exploit.class" width="1" height="1"><param width="1" height="1"><param name="data" value=""/><param
name="data" value=""/><param name="jar"> name="jar">
While these changes had no impact on the effectiveness of the exploits, they did allow the exploits to go undetected by the signature-
based Snort solution.
Mutation Example 3: Adding Comments
In order to test mutated versions of some of the exploits, we simply added comments to the exploit code as shown in the example
below:
Original Code Mutated Code
var t = unescape; var t = unescape <!— Comment -->;
While these changes had no impact on the effectiveness of the exploits, they did allow the exploits to go undetected by the signature-
based Snort solution.
Tested Server Vulnerabilities
CVE-2012-0002 CVE-2011-4191 CVE-2011-3192 CVE-2011-1248 CVE-2011-1206
CVE-2011-0807 CVE-2011-0654 CVE-2011-0267 CVE-2011-0266 CVE-2010-3972
CVE-2010-2729 CVE-2010-1555 CVE-2010-0478 CVE-2009-3103 CVE-2009-3023
CVE-2009-1429 CVE-2008-4250 CVE-2008-1697
Tested Client Vulnerabilities
CVE-2012-1889 CVE-2012-1875 CVE-2012-0779 CVE-2012-0507 CVE-2012-0500
CVE-2012-0158 CVE-2012-0013 CVE-2011-3544 CVE-2011-3400 CVE-2011-2462
CVE-2011-1260 CVE-2011-0611 CVE-2011-0609 CVE-2011-0105 CVE-2011-0073
CVE-2011-0065 CVE-2011-0041 CVE-2011-0027 CVE-2010-4452 CVE-2010-3971
CVE-2010-3970 CVE-2010-3962 CVE-2010-3654 CVE-2010-3653 CVE-2010-3552
CVE-2010-3333 CVE-2010-3148 CVE-2010-2883 CVE-2010-2568 CVE-2010-1885
CVE-2010-1423 CVE-2010-1297 CVE-2010-1240 X2 CVE-2010-0842 CVE-2010-0840
CVE-2010-0806 CVE-2010-0805 CVE-2010-0249 CVE-2010-0248 CVE-2010-0188
CVE-2010-0094 CVE-2010-0033 CVE-2010-0027 CVE-2009-4324 CVE-2009-3459
CVE-2009-2477 CVE-2009-1534 CVE-2009-1136 CVE-2009-0927 CVE-2009-0658
CVE-2009-0075 CVE-2008-4844 CVE-2008-4037 CVE-2008-2992 CVE-2008-0015
Note: To view details of a given CVE, use the following format with the CVE name:: http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-1206
Source: Tolly, October 2012 Figure 4
© 2012 Tolly Enterprises, LLC Tolly.com Page 5 of 6
6. IBM IPS GX7800 #212148
About Tolly
Interaction with Competitors
The Tolly Group companies have been
delivering world-class IT services for In accordance with Tolly’s Fair Testing Charter, Tolly personnel invited
more than 20 years. Tolly is a leading
representatives from Sourcefire, Inc, developer of Snort, to participate in
global provider of third-party
validation services for vendors of IT
the testing. Sourcefire reviewed the test plan and declined to participate.
products, components and services.
For more information on the
You can reach the company by E-mail Tolly Fair Testing Charter, visit:
at sales@tolly.com, or by telephone at http://www.tolly.com/FTC.aspx
+1 561.391.5610.
Visit Tolly on the Internet at:
http://www.tolly.com
Terms of Usage
This document is provided, free-of-charge, to help you understand whether a given product, technology or service merits additional
investigation for your particular needs. Any decision to purchase a product must be based on your own assessment of suitability
based on your needs. The document should never be used as a substitute for advice from a qualified IT or business professional. This
evaluation was focused on illustrating specific features and/or performance of the product(s) and was conducted under controlled,
laboratory conditions. Certain tests may have been tailored to reflect performance under ideal conditions; performance may vary
under real-world conditions. Users should run tests based on their own real-world scenarios to validate performance for their own
networks.
Reasonable efforts were made to ensure the accuracy of the data contained herein but errors and/or oversights can occur. The test/
audit documented herein may also rely on various test tools the accuracy of which is beyond our control. Furthermore, the
document relies on certain representations by the sponsor that are beyond our control to verify. Among these is that the software/
hardware tested is production or production track and is, or will be, available in equivalent or better form to commercial customers.
Accordingly, this document is provided "as is", and Tolly Enterprises, LLC (Tolly) gives no warranty, representation or undertaking,
whether express or implied, and accepts no legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness
or suitability of any information contained herein. By reviewing this document, you agree that your use of any information
contained herein is at your own risk, and you accept all risks and responsibility for losses, damages, costs and other consequences
resulting directly or indirectly from any information or material available on it. Tolly is not responsible for, and you agree to hold Tolly
and its related affiliates harmless from any loss, harm, injury or damage resulting from or arising out of your use of or reliance on any
of the information provided herein.
Tolly makes no claim as to whether any product or company described herein is suitable for investment. You should obtain your
own independent professional advice, whether legal, accounting or otherwise, before proceeding with any investment or project
related to any information, products or companies described herein. When foreign translations exist, the English document is
considered authoritative. To assure accuracy, only use documents downloaded directly from Tolly.com. No part of any document
may be reproduced, in whole or in part, without the specific written permission of Tolly. All trademarks used in the document are
owned by their respective owners. You agree not to use any trademark in or as the whole or part of your own trademarks in
connection with any activities, products or services which are not ours, or in a manner which may be confusing, misleading or
deceptive or in a manner that disparages us or our information, projects or developments.
212148 jwft1 jt-mts-wt-2012-11-29-VerO
© 2012 Tolly Enterprises, LLC Tolly.com Page 6 of 6