Delivered on 10 November 2011 Previously delivered at OWASP Chapter Netherlands Chapter Meeting on 30 June 2010

1. skipfish
10 November 2011
Ernst & Young, Sydney Australia
Previously presented at:
OWASP NL
30 June 2010

2. Overview
Not an OWASP Project
By Michal Zalewski
Major contributions to webappsec with Google
RatProxy;
Browser Security Handbook;
“Rise of the Robots” i.e. The inspiration for the OWASP
“Google Hacking” Project

3. Overview
Fast webappsec scanner which“spiders” using word lists
Could be used to test www DOS

4. Overview
Fast webappsec scanner which“spiders” using word lists
Similar to Burp Scanner, etc
Does not satisfy WASC Security Scanner Evaluation Criteria
I don’t think lcamtuf intends too either :)

5. Overview
3.Fast webappsec scanner which“spiders” using word lists
Similar to DirBuster maybe Nikto, etc
“2007 entries resulting in about 42K HTTP Requests”
Based on the recommended *minimal* Word List
i.e. bigger wordlist = bigger number of HTTP Requests

6. Build/Install
From Source Code
Doesn’t build on OpenBSD (issue noted)
Dependency on libidn
Builds on backtrack

7. Release Cycle
lcamtuf rapidly updates via minor releases
i.e. RatProxy followed same development
Insert http://
vis.cs.ucdavis.edu/
~ogawa/codeswarm/

8. Build/Install
http://www.shortinfosec.net/2010/03/compiling-latest-skipfish-for-windows.html
Not mantained with each release i.e. v1.29b
No mention of support on code.google.com
i.e. Use at your own risk

9. Spidering
./skipfish -W /dev/null -LV ...

10. Word List
keywords and extensions
type hits total_age last_age keyword

11. Supplied Word Lists
1. Empty
2. extensions-only.wl
Must be used in conjunction with ./skipfish -Y

12. Word List
The following all contain 1.7K keywords:

13. Word List
minimal.wl
~50,000 HTTP Requests
medium.wl
~50,000 HTTP Requests x 2
complete.wl
~50,000 HTTP Requests x 3

14. Word List
Insert sh script
1.Select wordlist from ./dictionaries/
2.Copy as ../skipfish.wl
*copy* .wl as skipfish may append skipfish.wl
may depends on cmd line i.e. ./skipfish -V ...

15. Wordlist
Custom Wordlist
./skipfish -W custom_wl ...
Suppress Automatic Learning
./skipfish -L ...
Suppress Amending Wordlist
./skipfish -V ...

16. Lightweight Brute Force
~1,700 HTTP Requests
cp ./dictionaries/complete.wl dictionary.wl
./skipfish -W dictionary.wl -Y ...

17. Word List
Limit Keyword Guess Size Jar
./skipfish -G ...
Drop Old Dictionary Entries
./skipfish -R ...
Don’t fuzz $keyword.$extension
./skipfish -Y ...

18. Basic Usage
Output Directory
./skipfish -o output_dir URL ...
Suppress Real-Time Statistics
./skipfish -u ...

19. Usage - Scheduling
Percentage of links and directories
./skipfish -p percentage ...
Repeat previous scan
./skipfish -q seed ...

20. Usage - Authentication
HTTP Auth
./skipfish -A user:pass ...
Cookie
./skipfish -C name=value ...
Autocomplete Forms
./skipfish -T form_field=value ...

21. Usage - Cookie
Cookie
./skipfish -C name=value ...
Ignore new set-cookies from specific locations
i.e. prevent URIs from being fetched, such as logout.aspx
./skipfish -X ...
Ignore new set-cookies from all locations
./skipfish -N ...

22. Usage - HTTP Headers
User Agent
./skipfish -b ffox or ie or phone...
Custom HTTP Header
./skipfish -H Header ...

23. Usage - Scoping
Spider from
./skipfish -I URI ...
Parameters not to Fuzz, such as SessionID
./skipfish -K SessionID_parameter ...
Include Domain
./skipfish -D FQDN...
Exclude URI
./skipfish -S URI or -X URI ...

24. Usage - Scoping
Limit crawl depth to number of sub directories/folders
./skipfish -d number ...
Limit the number of child directories per parent
./skipfish -c number ...
Limit Total HTTP Requests
./skipfish -r number ...

25. Usage - Scoping
No parsing of Form
./skipfish -O ...
No parsing of HTML
./skipfish -P ...

26. Usage - Low Impact
Mixed TLS/SSLv3 and HTTP (i.e. Cleartext)
./skipfish -M ...
Low severity i.e. images are out of scope
Caching Directives of HTTP 1.0 vs 1.1
./skipfish -E ...
Information Leakage i.e. E-mail Addresses and URL
./skipfish -U ...

27. Usage - Reporting
Suppress reporting of duplicates hosts
./skipfish -Q ...
Suppress warning of “trusted” domains
./skipfish -B ...
Purge binary content without affecting report quality
./skipfish -e ...

28. Delta Reporting
sfscandiff
non-destructively annotated by adding red
background to all new or changed nodes; and blue
background to all new or changed issues found

29. Issues
Won’t detect common low risks, such as:
cookie without HTTPonly or secure flags
autocomplete enabled Forms

30. Issues (Credit ‘FX’)
High Number of False Positives
ASCII txt interpreted as JSON reply with XSSI
Deviation between charset and MIME type
Note ./skipfish -J ...
No wordlist generation based on robots.txt

31. Issues (Credit ‘FX’)
Resolved
Does not write output while the tool is executing
Total Size of HTTP Request vs File System Image

32. Issues
Does not support intercepting web proxy
No supporting log entires that skipfish was used
Use wireshark instead i.e. TCP/80 and TCP/443

33. Benefits (Credit ‘FX’)
Will display the source of CGI script
Can detect IPS
HTTP 500 for ASP.NET HttpRequestValidationException

34. Performance Tuning
Number of connections to all hosts
./skipfish -g ...
Recommended to be < 50
Per IP
./skipfish -m number ...
2 - 4 localhost
4 - 8 local network
10 - 20 external
30 - 50 hosts which lag or slow connections

35. Performance Tuning
I/O Timeout
./skipfish -w number ...
Total Request Timeout
./skipfish -t number ...
Number of HTTP Errors before Terminating
./skipfish -f number ...
Truncate HTTP Response
./skipfish -s number ...

36. Q&A
Thanks Wouter - Ernst & Young
Latest slides available from
http://slideshare.net/cmlh
http://github.com/cmlh/skipfish
http://cmlh.id.au/contact