Security Testing for Testing Professionals

TL
PM Tutorial
10/1/2013 1:00:00 PM

"Security Testing for Testing
Professionals"
Presented by:
Jeff Payne
Coveros, Inc.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Jeff Payne
Coveros, Inc.
Jeff Payne is CEO and founder of Coveros, Inc., a software company that builds secure
software applications using agile methods. Since its inception in 2008, Coveros has become a
market leader in secure agile principles and has been recognized by Inc. magazine as one of
the fastest growing private US companies. Prior to founding Coveros, Jeff was chairman of the
board, CEO, and cofounder of Cigital, Inc., a market leader in software security consulting.

8/20/2013

Security Testing for Test
Professionals

© Copyright 2011 Coveros, Inc.. All rights reserved.

1

Trainer

Jeffery Payne
Jeffery Payne is CEO and founder of Coveros, Inc., a software company that
helps organizations accelerate the delivery of secure, reliable software. Coveros
uses agile development methods and a proven software assurance framework to
build security and quality into software from the ground up. Prior to founding
Coveros, Jeffery was Chairman of the Board, CEO, and co-founder of Cigital, Inc.
Under his direction, Cigital became a leader in software security and software
quality solutions, helping clients mitigate the risk of software failure. Jeffery is a
recognized software expert and popular speaker at both business and technology
conferences on a variety of software quality, security, and agile development
topics. He has also testified before Congress on issues of national importance,
including intellectual property rights, cyber-terrorism, Software research funding,
and software quality.


2

1

8/20/2013

About Coveros

 Coveros helps organizations accelerate the delivery of secure, reliable
software
 Our consulting services:
–
–
–
–

Agile software development
Application security
Software quality assurance
Software process improvement

Corporate Partners

 Our key markets:
–
–
–
–

Financial services
Healthcare
Defense
Critical Infrastructure


3


4

Agenda

 Introduction to Security Testing
 Security Testing Framework
– Steps in security testing
– Security test planning
– Security test tools

 Wrap up

2

8/20/2013

Expectations
 What are your expectations for this tutorial?
 What do you wish to learn?
 What questions do you want answered?


5

Introduction to Security Testing


6

3

8/20/2013

What is Information Security?

When you hear the term “Information Security” and
“Security Testing”:
What do you think they mean?
What comes to mind?


7

What is Information Security?
Definition of Information Security
 Information Security means protecting information and
information systems from unauthorized access, use,
disclosure, disruption, modification, perusal, inspection,
recording or destruction.
 The key concepts of Information Security include:
– Confidentiality – prevent the disclosure of information to
unauthorized individuals or systems
– Integrity – data cannot be modified undetectably
– Availability – data and systems are available in an uninterrupted
manner
– Authenticity – ensure that data, transactions, communications or
documents (electronic or physical) are genuine
– Non-Repudiation – ensure that someone cannot deny something


8

4

8/20/2013

The Software Security Problem
Our IT systems are not castles any longer!


9


10

Why Software Security is Important

5

8/20/2013

Understanding Risk
How to Define Security Risk in Software
 Common Security Nomenclature
– Risk: a possible future event which, if it occurs, will lead to an
undesirable outcome
– Threat: A potential cause of an undesirable outcome
– Vulnerability: Any weakness, administrative process, or act of
physical exposure that makes an information asset susceptible to
exploit by a threat.
– An exploit is a piece of software, a chunk of data, or sequence of
commands that takes advantage of a vulnerability in order to cause
unintended or unanticipated behavior to occur on computer
software, hardware, or something electronic.
– Attack: the approach taken by a threat to exploit a vulnerability
 Denial of service, spoofing, tampering, escalation of privilege

11

Security Testing
What? How?
 Security Testing is testing used to determine whether an
information system protects its data from its threats.
 Security Testing is not a silver bullet for your enterprise
security. Security Testing doesn’t fix your security, it only
makes you aware of it. Security must be built into your
software
 A sound Security Testing process performs testing
activities:
–
–
–
–
–

Before development begins
During requirements definition and software design
During implementation
During deployment
During maintenance and operations


12

6

8/20/2013

Exercise
Security Testing Case Study
 Your company, SecureTelco, has developed an instant
messaging program to be used for private use in customers
homes and for companies and government agencies.
 SecureChat requires users to sign up with an account prior
to using the system. After authenticating with a username
and password, each user can message other users and
expect their conversations to be private.
 Users have the ability to add/remove friends from their
contact list, search for friends based on their email, block
users from IMing them, become “invisible” to all users on
demand.
 Messages archives and activities logs document user
behavior and can be retrieved by the user or a SecureTelco
Administrator through the application or by the
administrative console, respectively.

13

Security Testing Framework


14

7

8/20/2013

Security testing before development begins
Overview
 “Testing” before development begins is really a QA function
to assess the readiness of the organization to build secure
software applications.
 Always remember that security testing evaluates the
security posture of your applications, it does not build
security in.
 Irrespective of your findings, do not become the “quality
police”.


15

Review Security Policies and Standards
 Understand the policies and standards that have been
adopted by the organization and their relationship to
software security
 Examples:
–
–
–
–

Privacy policies regarding your customer data
Service level agreements with clients
IT security standards you must adhere to
PCI compliance activities for credit card transactions

 Your goal is to understand these policies and standards to
the level that will allow you to validate security requirements
and effectively test the end product against them

16

8

8/20/2013

Review Secure Software Development Lifecycle
 If the security of your software is an enterprise concern, the
development team should be adhering to a defined secure
software development lifecycle model.
– Defines development activities that builds security in
– Defines security testing activities performed by appropriate parties
(development, testing, security org, operations, etc.)

 Common secure software development models
– Microsoft’s Secure Development Lifecycle (SDL)
– Coveros SecureAgile process
– There are others as well

 Secure software standards
– Secure coding standard

17

Security testing during definition and design
Overview
 Testing activities during requirements definition and
software design focus on assuring that security has been
effectively integrated into software requirements and the
overall architecture and design of the product
 Typical activities include:
–
–
–
–

Security requirements development/validation
Architecture and design reviews
Threat modeling
Test strategy and planning


18

9

8/20/2013

Software Requirements
 Functional Requirements: These are statements of
services the system should provide, how the system should
react to particular inputs and how the system should behave
in particular situations.
 What each feature within the software should do

 Non-Functional Requirements: These statements
describe additional requirements that are not associated
with individual functional behaviors. These statements
include information about: reliability, configurability,
availability, performance, etc.
 What quality goals must the entire software system achieve


19

Security Requirements
 Security Requirements describe functional and nonfunctional requirements that need to be satisfied in order to
achieve the security attributes of an IT system or
application.

What does that mean?
 Functional Security Requirements
 Additions to functional requirements that define what
the software should not do.
 Non-Functional Security Requirements
 Additional non-functional requirements that define what
overall security the system must provide

20

10

8/20/2013

Example Security Requirement
Functional requirement:
SecureChat login screen shall accept a valid
username/password pair and allow system access
Functional requirement that includes security:
SecureChat login screen shall accept valid
username/password pairs and allow system access.
•

Entering either an invalid username or invalid password will result in the
display of the message “Invalid username or password” on a redisplay
of the login screen after both a username and password are entered

•

Three successive invalid login attempts from a particular machine will
lock the user’s account and display the message “User Account
Locked, Call System Administrator” on a redisplay of the login screen.
Subsequent valid login/password pairs will not allow system access
until the account is unlocked by the system administrator

21

Example Security Requirement
Functional requirements:
SecureChat user shall choose a userid and a password for
their account during registration
Functional security requirement:
SecureChat user shall choose a userid and a password for
their account during registration
• Userid shall be unique within the system
• Userid shall consist of alphanumeric characters
• Password shall be at least 12 characters long and include
at least one capital letter, one special character, and one
whole number

22

11

8/20/2013

Examples of Non-Functional Security Requirements
 SecureChat shall ensure that data is protected from
unauthorized access at all times.
 SecureChat shall have an availability of 99.9%.
 SecureChat shall process a minimum of 8 transactions per
second.
 Each SecureChat build shall undergo secure code review
prior to release.
 All communications between the SecureChat client
application and the SecureChat central servers shall be
encrypted.

23

Architectural and Design Reviews
 Architectural and design reviews focus on determining
whether the stated architecture / design enforces the
appropriate level of security as defined in the requirements.
 Typically performed by security architects and/or other
software leads within the organization.
 Examines these artifacts for flaws such as:
– Violation of trust boundaries
– Distributed control of authorization
– Custom algorithms for cryptography / random number generation


24

12

8/20/2013

Design Flaws vs. Implementation Bugs
 Flaws (Design Defects)
– Misuse of cryptography
– Compartmentalization problems
in design
– Privileged block protection failure
– Type safety confusion error
– Insecure auditing
– Broken or illogical access control
– Method over-riding problems

 Bugs (Implementation Defects)
–
–
–
–

Buffer overflows
Cross site scripting
Race conditions
SQL Injection


25

Threat modeling for risk assessment
 Threat modeling – a process by which any risks to a piece of
software are identified and mitigated
 A variety of approaches exist for doing threat modeling
 Microsoft STRIDE model
–
–
–
–

Diagram your system – high level dataflow diagrams
Identify threats (risks) – each type of entity/interaction has enemies
Mitigate threats (risks) – determine security controls
Validate mitigations – test effectiveness of these controls

 Spoofing, Tampering, Repudiation, Information Disclosure,
Denial of Service, Escalation of Privilege (STRIDE)

26

13

8/20/2013

Fixing the Problem – One DoD Initiative
60.00

Critical/High Vulnerabilities Per 1,000 Lines of Code
Initial
Follow-On

40.00

20.00

0.00
App1

App2

App3

App4

App5

App6

But there are 1,000’s of apps … do the math

27

Assessing your risk – Answers the ‘so what?’ question
 Identifying threats and flaws in your design only result in
better security if the flaws are mitigated to minimize the
threat.
 But at what cost to the organization?
 What benefit?
 How do you convince management to fund mitigation
efforts?

28

14

8/20/2013

Risk Assessments
 Information on design flaws/vulnerabilities and known
threats from our threat model are often combined together
to estimate the likelihood and consequence of a flaw/defect
resulting in significant business impact
Highly Likely

Likely

Unlikely

Business-critical

High priority

Priority

Priority

Business concern

High priority

Priority

Not a Priority

Minor or cosmetic

Not a Priority

Not a Priority

Not a Priority


29

Risk Assessments Results
 Risks are placed in appropriate categories based upon
understood consequence and likelihood of occurrence
– Consequence – depends upon your business and market
– Likelihood – depends upon your risks and threats
Highly Likely

Business-critical
Business concern
Minor or cosmetic

Likely

Unlikely

High priority

Priority
Stealing of
secrets

Priority

High priority

Priority
Tampering

Not a Priority

Not a Priority

Not a Priority

Not a Priority

Denial of
service

Inappropriate
access


30

15

8/20/2013

Exercise
Functional Security Requirement
 SecureChat Authentication Requirements
– When a user attempts to authenticate with a valid username and an
invalid password, the application shall not authenticate the user and
return them to the authentication page.
– The system must alert the user that their attempt to authenticate has
failed due to an incorrect password (“Invalid Password”) utilizing the
standard error text formatting.
– When a user attempts to authenticate with a invalid username, the
application shall not authenticate the user and return them to the
authentication page.
– The system must alert the user that their attempt to authenticate has
failed due to an incorrect username (“Invalid Username”) utilizing
the standard error text formatting.
– What a user attempts to authenticate using a username and a valid
password, the application shall authenticate the user and redirect
them to the homepage.

 What risks/attacks might be possible?

31

Test strategy and planning
Security test strategy
 Security is one aspect of testing that must be incorporated
into your test strategy and is typically included in a master
test plan
 Typical master test plan format
–
–
–
–
–
–

Overview of system
High level risks and threats to quality
Types of testing that will help mitigate risks and threats
Roles and responsibilities
Automation approach
Test infrastructure and schedule


32

16

8/20/2013

Developing a security test plan
What should be included?
 Describe and detail your process and procedures for
security testing
–
–
–
–

When should testing begin?
How are test results reported?
Who validates and verifies findings/results?
When are vulnerabilities addressed?

 Types of tests you should include in your test plan:
–
–
–
–

Security Feature Testing
Risk Based Testing of functional and non-functional requirements
Internal Penetration Tests
External (Independent) Penetration Tests

 Security test plans are usually separate test plans (for
compliance / audit reasons)

33

Integrating security requirements in test plans
Know your Security Requirements – Requirements analysis
 It is important that each tester understand the security
requirements for your application and what they imply.
 Often Security requirements may come in conflict with another type
of requirement. If there are conflicts, it is important that you identify
those concerns and the requirements are clarified by a Business
Analyst.

 In most organizations, security requirements are not well
defined if it all.
 A general rule of thumb: Make sure your core information security
concepts are all covered. If not, request that they are.

 Understand which security requirements are functional and
which are non-functional, this will have an impact how you
plan to test them.

34

17

8/20/2013

Integrating security requirements in test plans
Testing Security Requirements
 Feature testing covers positive security requirements. This
typically ensures the software behaves according to
customer expectations.
 Example – If security requirements state that the length of any user
input must be validated, then a feature test suite should be created
to exercise the application inputs and verify that this requirement is
implemented correctly.

 Testers should also cover negative security testing or RiskDriven testing. Each test is intended to probe for a specific
risk or vulnerability. These risk may have been identified
during your risk assessment.
 Example – Cross Site Scripting and SQL Injection;
These vulnerabilities are not obviously features of the
application, therefore the fall under the negative security
requirements umbrella.
 Security testing tools provide out of the box testing for
common web security issues


35

Security testing during implementation
Overview
 Testing activities during implementation focus on assuring
that the software is implemented properly according to its
requirements and design
 Key activities during implementation include:
– Secure code review – identifying security vulnerabilities in source
code
– Testing individual components/features for security
– Testing requirements at the appropriate level


36

18

8/20/2013

Secure code review
 Secure code review identifies vulnerabilities within source
code that potentially impact system security.
 Examples
– Buffer overflows
– Race conditions

 Secure code review is a combination of manual and
automated analysis
 Secure code review is typically done by developers or a
dedicated security team

37

Testing components and features
 The testing of components and individual features will
identify code that improperly implements functionality
against its requirements.
 While some feature testing has historically been done at the
system level, more and more of this type of testing today is
done on individual units / stories by either a developer or
code savvy test engineer.
 Review of tests performed at this level should look for
common gaps that lead to security issues:
– Inadequate testing of error handling routines
– Insufficient protection during system reboot
– Forgetting to test administrative capabilities


38

19

8/20/2013

Testing common security controls
 Due to the security-critical nature of many of our
applications, it is common to see the following security
controls implemented within our software.
 Each must be validated in order to work!
 Authentication & Access Control
 Input Validation & Encoding
 Encryption
 User and Session Management
 Error and Exception Handling
 Auditing and Logging
 Test catalog’s can assure security controls are tested
adequately.


39

Common Approaches to Authentication
All About Authentication
 When we refer to authentication in computer security, we
refer to the process of attempting to verify the digital identity
of the sender of a communication.
– A common example of such a process is the login process.
– Authentication always depends upon using one or more
authentication category: something I know, I have, I am

 Two-factor authentication: factors from two categories
– Multi-factor authentication: more than one authentication factor but
can be from the same category

 Testing authentication schemas means understanding how
the process works and using that information to circumvent
the authentication mechanism.

40

20

8/20/2013

Authentication Test Catalog
 Credentials transport over an encrypted channel
– The tester must try to understand if the data inputted by the user is
transmitted using secure protocols that protect them from an
attacker or not.

 Testing for user enumeration
– The tester must verify if it is impossible to collect a set of valid users
by interacting with the authentication mechanism of the application.
This will become useful for brute force testing.

 Testing for guessable (dictionary) user accounts
– The tester must validate that there are no default user accounts or
guessable username/password combinations

 Brute force testing
– When dictionary attacks don’t succeed, the tester can attempt brute
force methods to gain access. This is not often easy to accomplish
because of time constraints.

41

Authentication Test Catalog (cont.)
 Testing for bypassing authentication schema
– The tester must validate that other application resources are
adequately protected, and can’t be used to bypass authentication
using those other resources.

 Testing for vulnerable remember password and password
reset features
– The tester must analyze how the application manages the process
of “password resets”. The tester must check whether the application
allows the user to store passwords in the browser.

 Testing for logout and browser cache management
– The tester must check that the logout and caching functions are
properly implemented.


42

21

8/20/2013

 Testing for CAPTCHA
– Used by many applications to ensure the response is not generated
by a computer, CAPTCHA (“Completely Automated Public Trust test
to tell Computers and Humans Apart”) implementations are often
vulnerable to various kinds of attacks.

 Testing multiple factor authentication
– The tester must test the following scenarios:





One Time Password Generator Tokens
Crypto devices like USB tokens or smart cards
X.509 Certificates
Random OTP sent via SMS

 Testing for race conditions
– The tester must ensure that an unexpected result on a multithread
application doesn’t create an authentication flaw. By their nature,
Race Conditions are difficult to test for


43

 Testing for session management schema
– The tester must test the security of a session tokens issues to the
client browser:
 How to reverse engineer a cookie
 How to manipulate cookies to hijack a session

 Testing for cookie attributes
– The tester must check if an application can take the necessary
precautions when assigning cookies and test the cookie attributes.

 Testing for session fixation
– The tester must validate that an application renews the cookie after
a successful user authentication, so that an attacker could not utilize
a session fixation vulnerability.


44

22

8/20/2013

 Testing for exposed session variables
– The tester must validate that it is not possible to create a replay
session attack utilizing exposed session information.

 Testing for CSRF (Cross Site Request Forgery)
– The tester must ensure that there is not a way to force an
unknowing user to execute unwanted actions on a web application
they are authenticated on.


45

Tools to Support Authentication Testing
Password Crackers/Brute Force Tools
 Where to use?
– When you want to break the default credentials or test your
authentication mechanisms against common security tools.

 Free Tools
– THC Hydra
– Cain and Abel
– Wfuzz

 Paid Tools
– John the Ripper


46

23

8/20/2013

Risk-based Testing
 Risk-based Testing focuses on testing that the risks
identified during threat modeling, design reviews, code
reviews were properly mitigated in the code
 Define negative tests that validate these issues have been
mitigated.
 Perform these tests at whatever level is appropriate to
identify any remaining vulnerabilities.
 Typically performed at the integration / system level

47

Top 25 Most Dangerous Software Errors
 SQL Injection
 OS Command Injection
 Buffer Overflow

 Cross site scripting
 Missing authentication
 Missing authorization

 Untrusted inputs in a
security decision
 Unnecessary privileges
 Cross-site request forgery
 Improper limitation of a
restricted file path

 Hard-coded credentials

 Download of code without
integrity checks

 Missing encryption

 Risky crypto algorithms

 Upload of dangerous files

 Use of potentially
dangerous functions

48

24

8/20/2013

Integration and Systems Testing
 Testing non-functional security requirements that span
features within the system
 Includes Web Application Security testing of any web-based
interfaces
– Learn to read the output of these tools and understand how the
vulnerabilities identified can be mitigated!

 Often includes internal Penetration Testing type activities to
“test like a hacker”
–
–
–
–

Fuzzing
Password crackers
Network port scanners
Dynamic input strings


49

Tools to Support Web Security Testing
Web Application Scanners
 Where to use?
– Looking for XSS, Injection and input validation vulnerabilities; some
tools will attempt to actively exploit vulnerabilities.

 Free Tools
–
–
–
–
–
–
–

Zap
Nikto
W3af
Paros
Skipfish
Wfuzz
ratproxy

 Paid Tools
– Netsparker
– WebSecurify

50

25

8/20/2013

Zaproxy
 Intercepting Proxy


Active scanner



Passive scanner



Brute Force scanner



Spider



Fuzzer



Port Scanner



Dynamic SSL certificates



API



Beanshell integration

Continuous Integration

$

51

$

Management

Engineer

IntelliJ IDEA/
Eclipse
JDepend

Hudson

subversion


52

26

8/20/2013

Tools to Support Penetration Testing
Network Security Tools
 Where to use?
– Scanning for mis-configurations
– Testing for OS, application and network vulnerabilities

 Free Tools
– OpenVAS

 Paid Tools
– Nessus
– Core Impact


53

Security testing during deployment
Overview
 Testing during the deployment process focuses on those
tests that cannot be adequately completed within a
development/QA environment plus any third party IV&V
– Red Team Penetration Testing
– Load and performance testing (for availability)
– Configuration testing

 Red Team Penetration Testing is typically done by a team
of security experts and includes both network and
application testing


54

27

8/20/2013

Security testing during maintenance / support
Overview
 Testing during maintenance and support focuses on:
– Assuring that any identified vulnerabilities within the application,
supporting software, or network configuration are patched and
revalidated

 Based upon the identified vulnerability and patch, a wide
variety of testing activities may be performed again to
assure the patch operations properly and also does not
break something else!


55

Questions?
Contact Information:
Jeffery Payne
Jeff.payne@coveros.com
703.431.2920

56

28

Security Testing for Testing Professionals

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (15)

Semelhante a Security Testing for Testing Professionals

Semelhante a Security Testing for Testing Professionals (20)

Mais de TechWell

Mais de TechWell (20)

Último

Último (20)

Security Testing for Testing Professionals