@ICSA_Labs Brian Monkman discusses how he is working with Spirent's latest testing solution to help with #performance testing of #security devices at scale. This presentation was shared during #RSAC and #Interop 2014.

1. © 2014. All Rights Reserved.
Brian Monkman
Programs Manager, ICSA Labs
April 7, 2014
Scalable Performance Testing
How Spirent Makes That Possible

2. 2 Copyright 2014 . All Rights Reserved.
About ICSA Labs
Best Known For
Providing 3rd Party Assurance
– Through ISO-Accredited Testing
Testing Security Products
~25 Years
Testing Since 1991
– Anti-virus Products
– Network Firewalls
– Etc.
ISO Accredited
– ISO 9001 & 17025
Seal of Approval
Newest Initiatives
Healthcare Testing
– EHR & IHE USA
Mobility Testing
– Platform Security for Verizon Wireless
– Enterprise Mobile Apps

3. 3 Copyright 2014 . All Rights Reserved.
About Reservoir Labs
Founded in 1990
20 employee
– More then half hold a Ph.D. in either Engineering or Computer Science
Significant work in “extreme scale” or “exascale” computing technologies
Next generation security monitor
Extends the state-of the art Bro
analytics language
Real-time threat detection and
network monitoring solution
Scalable to 100 Gbps and beyond
R-Scope Network Security

4. 4 Copyright 2014 . All Rights Reserved.
Reservoir Labs
ICSA Labs Test Engagement
Purpose of this test
– empirically measure the performance of Reservoir Lab’s R-Scope appliance and
consequently the bro software
Performance definition
– the percentage of HTTP transactions analyzed
– A transaction consists of a single request packet (HTTP GET) and one or more
response packets (HTTP 200 OK)
– number of response packets is a function of the size of the HTML document that is
included in the response
Variables and domains used during this test
– R-Scope variables
»Number of analyzers (a)
a = {4, 8, 12, 16, 20, 24, 28, 32}
»Number of forwarders (f)
f = {4, 8}
»*a + f must be <= 36

5. 5 Copyright 2014 . All Rights Reserved.
Reservoir Labs
ICSA Labs Test Engagement
Variables and domains used during this test (continued)
– HTTP Traffic Variables:
»Payload size (p)
p = {8kB, 16kB, 32kB, 64kB, 128kB, 256kB, 512kB, 1024kB}
»Bandwidth (n)
n = {100Mbps, 500Mbps, 1Gbps, 2Gbps}
Variables and constraints create over 350 unique combinations
– Number will grow as bandwidth ceiling increased
Attempting to discover the optimal R-Scope configuration given the
payload size and bandwidth
– also interested in seeing how payload size impacts the performance of the R-Scope

6. 6 Copyright 2014 . All Rights Reserved.
Test Environment

7. 7 Copyright 2014 . All Rights Reserved.
Why Spirent
Provide purpose-built testing solutions that permit us to recreate a mix of
application traffic based on real applications
Scalable and flexible
Given the volume of tests and scenarios we need to execute scripting
capabilities are required - Spirent delivers
Ability to craft traffic mix from own pcaps

8. 8 Copyright 2014 . All Rights Reserved.
www.icsalabs.com