1.
Software Program Security

2.
Program Security
Protecting programs is the heart of computer security
• All kinds of programs: applications, operating systems, database systems, etc.
• A program is secure if it provides/enforces confidentiality, integrity, and
availability
Program security is difficult:
• Security often conflicts with usefulness and performance
• It is essentially impossible to ensure that a program does only what it is
designed to do, and nothing more
• Programming and software engineering techniques change faster than
computer security techniques
• Even a single flaw can be catastrophic!

3.
Program Security
• Fault tolerance terminology:
• Error - may lead to a fault
• Fault - a deviation from intended functionality
• Failure - system malfunction caused by fault
• Two categories of faults (classified by duration)
• Permanent faults
• Transient faults - can be much more difficult to diagnose

4.
Incomplete Mediation
• Incomplete mediation: Sensitive data are in an exposed, uncontrolled
condition
• Usually inadvertent (non-malicious), but has serious security consequences
• Example:
• URL generated by client's browser during online purchase
http://www.~~.com/order/final&custID=101&part=55A&qty=20&price=10&shipcost=5&total=2
05
• Instead, user edits URL directly, changing price and total cost as follows:
• http://www.~~~.com/order/final&custID=101&part=55A&qty=20&price=1&shipcost
=5&total=25
• User uses forged URL to access server
• The server takes 25 as the total cost!

5.
Incomplete Mediation
• Unchecked/unvalidated data are a serious vulnerability!
• Possible solution: anticipate problems
• Don't let client return a sensitive result (like a total) that can be easily
recomputed by the server
• Use drop-down boxes / choice lists for data input
• Prevent user from editing input directly
• Check validity of data values upon receipt from the client
• Do not rely solely on client-side validation

6.
Time-of-Check to Time-of-Use (TOCTTOU)
• TOCTTOU: mediation with "bait and switch" in the middle
• Non-computing example:
• Swindler shows buyer real Rolex watch (bait)
• After buyer pays, switches real Rolex to a forged one
• In computing:
• Data are changed between the time that they are checked/validated and the time that they
are used
• Prevention of TOCTTOU errors
• Be aware of time lags!
• Use digital signatures and certificates to "lock" data values after checking them
• So no one can modify them after check & before use

7.
Undocumented Access Points
• An undocumented access point (also called a backdoor or a trapdoor)
is a method of accessing or manipulating internal values within a
software program that circumvents or sidesteps the program's usual
security mechanisms
• Often an artifact of system development
• Commonly created by programmers to:
• Facilitate testing
• Facilitate debugging

8.
Types of Program Flaws
1. Intentional
• Malicious or non-malicious
2. Inadvertent
• Commonly more numerous than -- and cause just as much damage as -- intentional flaws
• Validation error (incomplete or inconsistent)
e.g., incomplete or inconsistent input data
• Domain error
e.g., using the value of a variable outside of its domain
• Serialization and aliasing
Serialization-e.g., in DBMSs or OSS
Aliasing-one value, when changed, has an indirect (usually unexpected)
effect on some other value
• Inadequate identification and authentication

9.
Penetrate and Patch
• A method of judging software security in which a Red Team / Tiger
Team intentionally tries to crack or break a software program
• Theory: If the program withstands the attack, then security is adequate
• Is this true? Rarely.
• Developers often try to quickly fix problems discovered by the Tiger
Team
• Quick patches often introduce new faults due to:
• Pressure - causes developers to focus on the fault, not context
• Non-obvious side effects

10.
Software Development Security Controls
Hazard analysis
• A set of systematic techniques designed to reveal hazardous system states
• Involves the development of hazard lists and what-if scenarios in order to identify security flaws
Prediction
• Predicting which security problems are most dangerous or most likely to occur
• These predictions allow us to manage risk by targeting our limited resources to where they will be
most useful
Static analysis
• Examining a program's code and design specifications for evidence of potential security threats
• Conducted before a system is deployed - focuses on control flows, data flows, and data structures
Analysis of mistakes
• Learning from security breaches in order to avoid making the same mistakes in the future
• Requires that we document our design decisions so that we can review them when security
breaches are identified.

11.
Testing
• Programs should be extensively tested prior to deployment in order to ensure
quality, stability, and accuracy
• Unit testing
• Testing (in isolation) the functionality and behavior of each component that comprises the
program
• Integration testing
• Testing the extent to which system components interact with one another as intended
• Function testing
• Evaluating the ability of the system to perform the functions for which it was designed
• Performance testing
• Evaluating the ability of the system to achieve the performance standards for which it was
designed
• Acceptance testing
• Evaluating the system’s capabilities against the requirements provided by the customer.

12.
Testing (Cont’d)
• Installation testing
• Evaluating whether the system functions correctly in its final production
environment
• Black-box testing
• Ensuring that a system generates correct output values in light of known input values
• Does not consider the internal workings of the system
• Clear-box (white-box) testing
• Testing a program based on knowledge of its internal structures/design
• Penetration testing
• Testing a program by simulating an attack from external or internal threats
• Regression testing
• Testing where all system functions are still working properly after a change has been
made.

13.
Security Vulnerabilities

14.
Security Vulnerabilities
• Security Problems in the TCP/IP Protocol Suite
• Attacks on Different Layers
• IP Attacks
• ICMP Attacks
• Routing Attacks
• TCP Attacks
• Application Layer Attacks

15.
Why?
• TCP/IP was designed for connectivity
• Assumed to have lots of trust
• Host implementation vulnerabilities
• Software “had/have/will have” bugs
• Some elements in the specification were left to the implementers

16.
Security Flaws in IP
• The IP addresses are filled in by the originating host
• Address spoofing
• Using source address for authentication
• r-utilities (rlogin, rsh, rhosts etc..)
Internet
2.1.1.1 C
1.1.1.1 1.1.1.2
A B
1.1.1.3 S
•Can A claim it is B to the server
S?
•ARP Spoofing
•Can C claim it is B to the server
S?
•Source Routing

17.
Security Flaws in IP
• IP fragmentation attack
• End hosts need to keep the fragments till all the fragments arrive
• Traffic amplification attack
• IP allows broadcast destination
• Problems?

18.
Ping Flood
Attacking System
Internet
Broadcast
Enabled
Network
Victim System

19.
ICMP Attacks
• No authentication
• ICMP redirect message
• Can cause the host to switch gateways
• Benefit of doing this?
• Man in the middle attack, sniffing
• ICMP destination unreachable
• Can cause the host to drop connection
• ICMP echo request/reply
• Many more…
• http://www.sans.org/rr/whitepapers/threats/477.php

20.
Routing Attacks
• Distance Vector Routing
• Announce 0 distance to all other nodes
• Blackhole traffic
• Eavesdrop
• Link State Routing
• Can drop links randomly
• Can claim direct link to any other routers
• A bit harder to attack than DV
• BGP
• ASes can announce arbitrary prefix
• ASes can alter path

21.
TCP Attacks
Issues?
• Server needs to keep waiting for ACK y+1
• Server recognizes Client based on IP address/port and y+1
Client
Server
SYN x
SYN y | ACK x+1
ACK y+1

22.
TCP Layer Attacks
• TCP SYN Flooding
• Exploit state allocated at server after initial SYN packet
• Send a SYN and don’t reply with ACK
• Server will wait for 511 seconds for ACK
• Finite queue size for incomplete connections (1024)
• Once the queue is full it doesn’t accept requests

23.
TCP Layer Attacks
• TCP Session Hijack
• When is a TCP packet valid?
• Address/Port/Sequence Number in window
• How to get sequence number?
• Sniff traffic
• Guess it
• Many earlier systems had predictable ISN
• Inject arbitrary data to the connection

24.
TCP Layer Attacks
• TCP Session Poisoning
• Send RST packet
• Will tear down connection
• Do you have to guess the exact sequence number?
• Anywhere in window is fine
• For 64k window it takes 64k packets to reset
• About 15 seconds for a T1

25.
Application Layer Attacks
• Applications don’t authenticate properly
• Authentication information in clear
• FTP, Telnet, POP
• DNS insecurity
• DNS poisoning
• DNS zone transfer

26.
An Example
Shimomura (S) Trusted (T)
Mitnick
Finger
• Finger @S
• showmount –e
• Send 20 SYN packets to S
• Attack when no one is around
• What other systems it trusts?
• Determine ISN behavior
Showmount -e
SYN

27.
An Example
Shimomura (S) Trusted(T)
Mitnick
• Finger @S
• showmount –e
• Send 20 SYN packets to S
• SYN flood T
• Attack when no one is around
• What other systems it trusts?
• Determine ISN behavior
• T won’t respond to packets
Syn flood
X

28.
An Example
Shimomura (S) trusted (T)
Mitnick (M)
• Finger @S
• showmount –e
• Send 20 SYN packets to S
• SYN flood T
• Send SYN to S spoofing as T
• Send ACK to S with a guessed
number
• Attack when no one is around
• What other systems it trusts?
• Determine ISN behavior
• T won’t respond to packets
• S assumes that it has a session with T
X
SYN
SYN|ACK
ACK

29.
An Example
Shimomura (S) Trusted (T)
Mitnick
• Finger @S
• showmount –e
• Send 20 SYN packets to S
• SYN flood T
• Send SYN to S spoofing as T
• Send ACK to S with a guessed
number
• Send “echo + + > ~/.rhosts”
• Attack when no one is around
• What other systems it trusts?
• Determine ISN behavior
• T won’t respond to packets
• S assumes that it has a session with T
• Give permission to anyone from
anywhere
X
++ > rhosts

30.
Outline
• Security Vulnerabilities
• DoS and D-DoS
• Firewalls
• Intrusion Detection Systems
You are here

31.
Denial of Service
• Objective  make a service unusable, usually by overloading the
server or network
• Consume host resources
• TCP SYN floods
• ICMP ECHO (ping) floods
• Consume bandwidth
• UDP floods
• ICMP floods

32.
Denial of Service
• Crashing the victim
• Ping-of-Death
• TCP options (unused, or used incorrectly)
• Forcing more computation
• Taking long path in processing of packets

33.
Simple DoS
Attacker
Victim Victim Victim
• The Attacker usually spoofed
source address to hide origin
• Easy to block

34.
Coordinated DoS
Attacker
Victim Victim Victim
Attacker Attacker
• The first attacker attacks a different victim to cover up the real attack
• The Attacker usually spoofed source address to hide origin
• Harder to deal with

35.
Distributed DoS
Attacker
Handler Handler
Agent Agent Agent Agent Agent
Victim

36.
Distributed DoS
• The handlers are usually very high volume servers
• Easy to hide the attack packets
• The agents are usually home users with DSL/Cable
• Already infected and the agent installed
• Very difficult to track down the attacker
• How to differentiate between DDoS and Flash Crowd?
• Flash Crowd  Many clients using a service legimitaly
• Slashdot Effect
• Victoria Secret Webcast
• Generally the flash crowd disappears when the network is flooded
• Sources in flash crowd are clustered

37.
Outline
• Security Vulnerabilities
• DoS and D-DoS
• Firewalls
• Intrusion Detection Systems
You are here

38.
Firewalls
• Lots of vulnerabilities on hosts in network
• Users don’t keep systems up to date
• Lots of patches
• Lots of exploits in wild (no patch for them)
• Solution?
• Limit access to the network
• Put firewalls across the perimeter of the network

39.
Firewalls (contd…)
• Firewall inspects traffic through it
• Allows traffic specified in the policy
• Drops everything else
• Two Types
• Packet Filters, Proxies
Internet
Internal Network
Firewall

40.
Packet Filters
• Packet filter selectively passes packets from one network interface to
another
• Usually done within a router between external and internal networks
• screening router
• Can be done by a dedicated network element
• packet filtering bridge
• harder to detect and attack than screening routers

41.
Packet Filters Contd.
• Data Available
• IP source and destination addresses
• Transport protocol (TCP, UDP, or ICMP)
• TCP/UDP source and destination ports
• ICMP message type
• Packet options (Fragment Size etc.)
• Actions Available
• Allow the packet to go through
• Drop the packet (Notify Sender/Drop Silently)
• Alter the packet (NAT?)
• Log information about the packet

42.
Packet Filters Contd.
• Example filters
• Block all packets from outside except for SMTP servers
• Block all traffic to a list of domains
• Block all connections from a specified domain

43.
Typical Firewall Configuration
• Internal hosts can access DMZ and
Internet
• External hosts can access DMZ only,
not Intranet
• DMZ hosts can access Internet only
• Advantages?
• If a service gets compromised in
DMZ it cannot affect internal hosts
Internet
Intranet
DMZ
X
X

44.
Example Firewall Rules
• Stateless packet filtering firewall
• Rule  (Condition, Action)
• Rules are processed in top-down order
• If a condition satisfied – action is taken

45.
Sample Firewall Rule
Dst
Port
Alow
Allow
Yes
Any
> 1023
22
TCP
22
TCP
> 1023
Ext
Int
Out
SSH-2
Int
Ext
In
SSH-1
Dst
Addr
Proto
Ack
Set?
Action
Src
Port
Src
Addr
Dir
Rule
• Allow SSH from external hosts to internal hosts
– Two rules
• Inbound and outbound
– How to know a packet is for SSH?
• Inbound: src-port>1023, dst-port=22
• Outbound: src-port=22, dst-port>1023
• Protocol=TCP
– Ack Set?
– Problems?
SYN
SYN/ACK
ACK
Client Server

46.
Default Firewall Rules
• Egress Filtering
• Outbound traffic from external address  Drop
• Benefits?
• Ingress Filtering
• Inbound Traffic from internal address  Drop
• Benefits?
• Default Deny
• Why?
Any
Dst
Port
Any Deny
Any
Any
Int
Any
Int
In
Ingress
Deny
Any
Any
Ext
Any
Ext
Out
Egress
Any Deny
Any
Any
Any
Any
Any
Any
Default
Dst
Addr
Proto
Ack
Set?
Action
Src
Port
Src
Addr
Dir
Rule

47.
Packet Filters
• Advantages
• Transparent to application/user
• Simple packet filters can be efficient
• Disadvantages
• Usually fail open
• Very hard to configure the rules
• Doesn’t have enough information to take actions
• Does port 22 always mean SSH?
• Who is the user accessing the SSH?

48.
Alternatives
• Stateful packet filters
• Keep the connection states
• Easier to specify rules
• More popular
• Problems?
• State explosion
• State for UDP/ICMP?

49.
Alternatives
• Proxy Firewalls
• Two connections instead of one
• Either at transport level
• SOCKS proxy
• Or at application level
• HTTP proxy
• Requires applications (or dynamically linked libraries) to be modified
to use the proxy

50.
Proxy Firewall
• Data Available
• Application level information
• User information
• Advantages?
• Better policy enforcement
• Better logging
• Fail closed
• Disadvantages?
• Doesn’t perform as well
• One proxy for each application
• Client modification

51.
Outline
• Security Vulnerabilities
• DoS and DDoS
• Firewalls
• Intrusion Detection Systems
You are here

52.
Intrusion Detection Systems
• Firewalls allow traffic only to legitimate hosts and services
• Traffic to the legitimate hosts/services can have attacks
• CodeReds on IIS
• Solution?
• Intrusion Detection Systems
• Monitor data and behavior
• Report when identify attacks

53.
Types of IDS
Host-based
Network-
based
Signature-
based
Anomaly-
based

54.
Signature-based IDS
• Characteristics
• Uses known pattern matching
to signify attack
• Advantages?
• Widely available
• Fairly fast
• Easy to implement
• Easy to update
• Disadvantages?
• Cannot detect attacks for which it has no signature

55.
Anomaly-based IDS
• Characteristics
• Uses statistical model or machine learning engine to characterize normal
usage behaviors
• Recognizes departures from normal as potential intrusions
• Advantages?
• Can detect attempts to exploit new and unforeseen vulnerabilities
• Can recognize authorized usage that falls outside the normal pattern
• Disadvantages?
• Generally slower, more resource intensive compared to signature-based
IDS
• Greater complexity, difficult to configure
• Higher percentages of false alerts

56.
Network-based IDS
• Characteristics
• NIDS examine raw packets in the network
passively and triggers alerts
• Advantages?
• Easy deployment
• Unobtrusive
• Difficult to evade if done at low level of
network operation
• Disadvantages?
• Fail Open
• Different hosts process packets differently
• NIDS needs to create traffic seen at the end
host
• Need to have the complete network topology
and complete host behavior

57.
Host-based IDS
• Characteristics
• Runs on single host
• Can analyze audit-trails, logs, integrity of files and directories, etc.
• Advantages
• More accurate than NIDS
• Less volume of traffic so less overhead
• Disadvantages
• Deployment is expensive
• What happens when host get compromised?