An open and accurate accounting of the available intelligence for an individual, organization, or business is typically an undervalued component of both offensive and defensive information security activities. From the defender.s perspective, it is important to understand how the source, content, and fidelity of publicly available data can affect the overall security posture of the organization. For the attacker, the gathering and analysis of publicly available data, which often includes usernames, emails, hostnames, subnets, technologies deployed, new product initiatives, employee habits, hobbies, and relationships, will provide actionable intelligence products that can be leveraged to gain a foothold in the target organization and provide the foundation for a successful attack. This presentation will cover intelligence sources, gathering and analysis methods, and the supporting toolset. Individual use cases will highlight how a specific piece of information can be developed into an actionable intelligence product that can then be incorporated into a larger attack plan. This presentation also provides suggestions for limiting, detecting, and mitigating against the information that is made available to the public.
Passive Recon: Collapsing your target's wavefunction.
1. Passive Recon: Collapsing your
target’s wavefunction.
2013.10.17
Charleston ISSA
Gabe LeBlanc @gabeleblanc
Philip Hartlieb @pjhartlieb
Black Lantern Security Group
2. caveats / notes
1.
2.
3.
“We are standing on the shoulders of giants.” Numerous references have
been provided throughout the talk. Additional materials will be provided
for further reading in an appendix.
This talk is about the principles, methodology, process for performing
passive reconnaissance using tools and methods developed by a
community of researchers.
The tools, artifacts/raw data, and intelligence products presented are not
intended to be comprehensive. Every customer provides a new and
interesting challenge.
4. Terminology / context
• Vulnerability assessment
• Penetration Test
• Full Scope Red Team engagement
5. Test purpose and objectives
• Acquiring information that would significantly impact the operational
effectiveness of the business or organization.
• intellectual property
• trade secrets 1
• PHI 2
• PII 3
• mergers and acquisitions 4
• troop movements 5
• diplomatic cables
• Gaining elevated privileges on critical systems, applications, and
infrastructure in order to demonstrate the potential for impacting the
operational effectiveness of the business or organization.
1.
2.
3.
4.
5.
http://www.wishtv.com/news/local/two-accused-of-selling-eli-lilly-secrets-to-chinese-company
http://www.secureworks.com/resources/blog/general-hackers-sell-health-insurance-credentials-bank-accounts-ssns-and-counterfeit-documents/
http://krebsonsecurity.com/2012/06/carderprofit-forum-sting-nets-26-arrests/
http://www.imdb.com/title/tt0094291/
http://www.timesofisrael.com/israel-tracked-russian-navy-in-syria/?utm_source=dlvr.it&utm_medium=twitter
7. passive recon – focus on test objectives
What would most adversely impact the mission/ business
/organization? [CRITICAL HIT]
• Future earnings
PHI
PII
Access to critical
resources
Classified Materials
•
•
•
•
•
•
•
fines ($$$)
Faith (customers)
Lives
Political stability
Force projection
Diplomacy
Negotiation
Intellectual Property
8. passive recon – objective
• Gather, organize, and analyze data in order
to create actionable intelligence product(s)
that will support
– target identification;
– exploitation; and
– post exploitation activities
9. passive recon – case study
Actionable Intelligence
products
smtp security controls
pub/priv facebook,
linkedin profiles
Tools /
Manual Labor
Adv. Google Searches
Social media
chatter/comments
activity timelines
ATTACK
PLAN
spear phish
vector n ...
Raw Data
Brain
Maltego
Keyboard
Org. structure and
personnel
News media
Document Archives
Business
processes
MX records
Facebook Graph
search
Verified email
addresses
Documents and
metadata
FOCA
Brain
10. passive recon – establish baseline
Grunt Work
Search Engines
•
•
•
•
Google
Yandex 2
Yahoo
Blekko
•
•
•
•
•
•
•
•
Metagoofil
FOCA *
Exiftool
SearchDiggity
Doc archives
Network Resources
Specialty Sources4
Documents and
Metadata
EDGAR database
SEC filings
www.defense.gov/contract
s/
•
•
•
•
•
•
Whois
Fierce.pl*
Dnsrecon*
Pentbox
Centralops
Robtex.com
OSINT Frameworks
•
•
Maltego*
Recon-ng* 3
Raw Data 1
Key
Public points of
contact (POCs) Partnerships
1.
2.
3.
4.
Market
Vertical
Key
Network/Physical Products, services, Mission statement
Leadership
footprint
and offerings
and purpose
http://www.pentest-standard.org/index.php/Intelligence_Gathering
http://www.irongeek.com/i.php?page=videos/derbycon3/2304-practical-osint-shane-macdougall
http://www.irongeek.com/i.php?page=videos/derbycon3/1104-look-ma-no-exploits-the-recon-ng-framework-tim-lanmaster53-tomes
http://http://rr.reuser.biz/
11. passive recon – establishing baseline
• Let’s not forget passive physical/human
engineering (yes you can!)
• Recon Routes
– Smoking area
– Gym
– Local eatery
– After hours hot spots (dig, madra..anybody?)
– Parking lot
12. passive recon – establishing baseline
• What you ‘need’ (this is the short list)
– Camera (duh!)
– Monocular (depth perception and peripheral)
– Proper bag
– Space pen
– Waterproof notebook
– Street smarts
• Optional - Attire
13. passive recon – establishing baseline
• What you ‘need’ to do (this is the short list)
– Camera && be natural/use cover + conceal
– Monocular (depth perception and peripheral)
&& see camera + consider surroundings
– Proper bag && (I’m biased but this IS REALLY
important)
– Space pen && no-brainer
– Waterproof notebook && see pen + learn
sniper/infantry techniques
– Street smarts
• Optional - Attire
14. passive recon – establishing baseline
RESOURCE: Warrick
RAW DATA: Archived web resources and documents
INTELLIGENCE PRODUCT(S): Descriptions of products / services, culture, customers, technologies
used, financials, etc.
NOTES: The number of archived resources is heavily dependent on target
Mirror (approximate) web sites for viewing offline
•
•
A utility for reconstructing or recovering a website when a back-up is not
available.
Downloads the pages and images and will save them to your filesystem.
> ./warrick.pl -D ~/Desktop/cisco -k http://www.cisco.com/
1.
http://warrick.cs.odu.edu//about.php
15. passive recon – establishing baseline
RESOURCE: Warrick
RAW DATA: Archived web resources and documents
INTELLIGENCE PRODUCT(S): Descriptions of products / services, culture, customer profiles,
technologies used, financial outlook, etc.
NOTES: The number of archived resources is heavily dependent on target
original resource
archived resource
new local file
16. passive recon – establishing baseline
RESOURCE: Search engines
RAW DATA: inbound links from partners and customer organizations
INTELLIGENCE PRODUCT(S): Key Partnerships and customer profiles
NOTE: Search string = inanchor: <target site> -site: <target site> keyword
“The anchor text, link label, link text, or link title is the visible, clickable text in a hyperlink.” – wikipedia.org
17. passive recon – establishing baseline
RESOURCE: Maltego / Website Incoming Links Transform
RAW DATA: inbound links from partners and customer organizations
INTELLIGENCE PRODUCT(S): Key Partnerships and customer profiles
NOTE: Mixed Success
18. passive recon – establishing baseline
RESOURCE: Search engines
RAW DATA: Documents and Metadata
INTELLIGENCE PRODUCT(S): Organizational structure (usernames) and emails, passwords,
software, OS, major/minor version numbers, internal IP address space, PII, PHI, financial
outlook, etc.
NOTE: Search string = filetype: <ppt, pdf, xls, doc> -site: <target site> <keyword>
1.
2.
3.
4.
http://www.sans.org/reading-room/whitepapers/privacy/document-metadata-silent-killer-32974?show=document-metadata-silent-killer-32974&cat=privacy
http://jwebnet.net/advancedgooglesearch.html
http://www.irongeek.com/i.php?page=videos/derbycon3/2304-practical-osint-shane-macdougall
http://www.spylogic.net/2009/10/enterprise-open-source-intelligence-gathering-%E2%80%93-part-2-blogs-message-boards-and-metadata/
19. passive recon – establishing baseline
RESOURCE: Document Archives1
RAW DATA: Documents and Metadata
INTELLIGENCE PRODUCT(S): Organizational structure (usernames) and emails, passwords,
software, OS, major/minor version numbers, internal IP address space, PII, PHI, financial
outlook, etc.
NOTE: Search string = filetype: <ppt, pdf, xls, doc> -site: <target site> <keyword>
Docstoc http://www.docstoc.com/
Scribd http://www.scribd.com/ (RSS feed of results)
SlideShare http://www.slideshare.net/ (RSS feed of results)
PDF Search Engine http://www.pdf-search-engine.com/
Toodoc http://www.toodoc.com/
http://www.docs-archive.com/
1.
http://www.spylogic.net/2009/10/enterprise-open-source-intelligence-gathering-%E2%80%93-part-2-blogs-message-boards-and-metadata/
20. passive recon – establishing baseline
RESOURCE: FOCA
RAW DATA: Documents and Metadata
INTELLIGENCE PRODUCT(S): categorized usernames and emails, passwords, software, OS,
major/minor version numbers, internal IP address space, PII, PHI, financial outlook, organizational
structure, etc
NOTE: Provides document paths, OS, software used, email, usernames, printers, etc.
21. passive recon – establishing baseline
RESOURCE: SearchDiggity
RAW DATA: Misconfigurations, default web pages, login pages, user credentials, leakage, etc
INTELLIGENCE PRODUCT(S): vulnerable web applications, collections of valid default user
credentials, back-up data, etc.
NOTE:
22. passive recon – establishing baseline
RESOURCE: Fierce Domain Scanner 1
RAW DATA: subnets, IPs, hostnames, FQDNs
INTELLIGENCE PRODUCT(S): Systems categorized according to name/function, network footprint
NOTE: Designed to locate hosts in non-contiguous IP space
>./fierce.pl -wide -dns <target domain> -dnsserver <dns server> -wordlist <custom wordlist> -file
<output file>
brute
list
attempts
domain
xfer
no
joy?
OUTPUT
file
http://ha.ckers.org/fierce/
hit?
no
yes!
1.
yes!
forward
lookup
additional
reverse
lookups
23. passive recon – establishing baseline
RESOURCE: Robtex.com
RAW DATA: subnets, IPs, hostnames, FQDNs
INTELLIGENCE PRODUCT(S): Systems categorized according to name/function, network footprint
NOTE: describe scripted approach
>for i in {0..255}; do wget https://route.robtex.com/72.23.${i}.0-24.html#sites; sleep
2; done
24. passive recon – establishing baseline
RESOURCE: Yatedo.com / Advanced Google searching “site:linkedin.com cisco administrator”
RAW DATA: human targets / seed accounts
INTELLIGENCE PRODUCT(S): Organizational Structure (usernames)
NOTE: A small perl script will quickly return csv formatted first name, last name, org, role
Seed Names
25. passive recon – case study
RESOURCE: LinkedIn / Facebook Account creation and data mining
RAW DATA: human targets / seed accounts
INTELLIGENCE PRODUCT(S): Organizational structure and personnel, key relationships,
culture, friendships, insider bullshit jargon, speech patterns.
NOTE: ** May violate ToS.
....
Recursively Harvest and
Catalogue Key groups of
individuals
INTEL:
System
Admins
INTEL: Help
Desk
INTEL:
Sharepoint
Admins
INTEL:
Database
Admins
INTEL: Mgmt
C-level
26. passive recon – case study
View Contacts:
Recursive
Harvesting
new target 1
new target 2
INTEL: New
LinkedIn
Connection
new target 3
Request
Connection(s)
new target n ...
Senior
leadership
“About”
section
INTEL: work email
format !
LastFiMi@x.y.z
27. passive recon – case study
RESOURCE: namechk.com
RAW DATA: user footprint, account enumeration
INTELLIGENCE PRODUCT: Relationships, friendships, hobbies, speech patterns, bad behavior
NOTE:
28. passive recon – case study
INTEL: John D.
LinkedIn
account
Facebook
Graph Search
John D.
Facebook
account
Recursively
Harvesting
Friends
Gerry L. (mgr.)
Facebook
account
Monitoring
Monitoring
John Posts Link
to article
describing
upgrade
INTEL: Activity
timeline,
resources, and
leadership for
upgrade
Link
Public Article
INTEL: Every
Windows
Admin on FB
Gerry posts
congrats! to
team and tags
all direct
reports
29. passive recon – case study
INTEL: work email
format !
LastFiMi@x.y.z
TO: <TARGETS>
FROM: <SENDER>
EMAIL BODY
INTEL: Every
Windows Admin on
FB
INTEL: Activity
timeline,
leadership, and
resources for
upgrade.
PHISH!
ATTACHMENT LINK
30. passive recon – case study
Actionable
Intelligence
products
Raw Data
pub/priv facebook,
linkedin profiles
Tools /
Manual Labor
Adv. Google Searches
activity timelines
ATTACK
PLAN
spear phish
vector n ...
Social media
chatter/comments
Brain
Maltego
Keyboard
Org. structure and
personnel
News media
Document Archives
Business
processes
MX records
Facebook Graph
search
Verified email
addresses
Documents and
metadata
FOCA
Brain
31. passive recon: process notes
• Native search functions will miss data (Facebook graph and LinkedIn
search)
• Hacker tools will miss data
• Take ridiculously detailed notes
• Don’t underestimate the importance of taking the time to use
Google/Bing advanced search functions in new and creative ways
• Be prepared to change objectives based on newly returned data
• Take ridiculously detailed notes
• Always be working towards an intelligence-product
• Organize your notes so they will still make sense 30 days from now [
Evernote (local), Zim, Keepnote, etc.]
• Some of our most interesting finds have fallen out of extremely tedious
long term manual search methods.
32. passive recon – mitigations [org.]
- Be at least as knowledgeable as the attacker.
- Perform passive recon against your own organization.
- Do you know how you make money?
- Where are your critical resources? What would be the death blow for the
organization?
- How would you plan an attack?
- Acceptable Use policy (AUP) for social media
- Monitoring of Social Media 1,2
- Public Affairs Office (PAO)
- Is there a process for the public release of information. Are there people
involved other than sales and marketing? How do they handle metadata?
- Use the free monitoring tools:
- google alerts, yahoo pipes, RSS readers
- twitter search, social media APIs
- SearchDiggity
- Consider one or more paid services 3
1.
2.
3.
http://sproutsocial.com/features/social-media-monitoring
http://www.cnn.com/2013/09/14/us/california-schools-monitor-social-media/index.html
https://pwnedlist.com/services
33. passive recon – mitigations [individual]
- LinkedIn security settings
- Keep your connections private. [Really annoying when enumerating]
- Avoid connections with people you have never met. [mutual connections !=
trust ]
- Do not publish email information. [Make it difficult to map out your digital
footprint]
- Facebook privacy settings
- Don’t allow followers
- Avoid public posts like the plague. [ We personally monitor and analyze these
daily for long term engagements]
- Avoid routinely checking in at your work address!
- Avoid those hookah pictures [ No one will ever believe that it was flavored
tobacco anyway .. cmon man ]
- Vanity is an attackers best friend ... truly my favorite sin.
- Forums
- How much are you revealing about technologies you use?
- Bugs in the software?
- Maintenance periods?
- Organizational deficiencies?
35. What else are we working on?
November 1-2 2013
RYAN WINCEY - Java Shellcode Execution
MICHEAL RESKI - Using MLP to classify Encrypted
Network Traffic
37. passive recon – baseline data
Gathering baseline information for understanding the organization / business
–
ACTION: Scouring publicly available web resources to gather:
–
• Mission statement and purpose
• Products and services available
• Key Leadership [Command Structure / C-level executives]
• POC information [ Public facing contacts or forms]
• Key partnerships
• Market Vertical
• Network Footprint
• Documents and metadata
• Web resources
• Seed accounts for personnel
TOOLS / RESOURCES:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Public facing web pages and portals
Corporate pages
EDGAR online database
SEC filings
Org charts
Maltego
Search engines (advanced operators)
Robtex / CentralOps / deepmagic (coming soon!)
Warrick
Internet Archives
Document archives
FOCA
Facebook graph search
Yatedo.com
Spokeo.com
38. osint - human targets
Harvesting, mapping, and categorizing human targets
– ACTION: gathering and analyzing data to create target packages
•
•
•
•
•
Publicly available social media profiles [pedigree, private email, role, responsibility, org, etc.]
Existing connections within artificial accounts [seed accounts]
News articles [recent projects, milestones, promotions, awards etc.]
Blogs and other forms of online publications [information leakage, physical addresses, phone
#s]
Alumni pages [friendships, hobbies, habits, sports]
– TOOLS
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Search engines (Google, Bing, Baidu, Duck Duck Go, Blekko, Yandex, etc.)
Facebook graph and LinkedIn search functions
Automated scripts
Yatedo
Spokeo
PiPL
Recon-ng
Foca
Scythe
Maltego
Namechk.com
Wayback machine
Google cache
SearchDiggity
Paste sites
TheHarvester
Uberharvester
39. osint – detailed data
Gathering detailed information for understanding products, services, processes,
technologies used, critical resources, markets, partnerships, and competitors
•
ACTION: Gathering and analyzing data from:
–
–
–
–
–
–
–
•
Spidered web content [services/products offered, external links (partners), etc.]
Publicly available documents [metadata: users, IPs, OS, email, printers, etc.)]
Social media pages [latest product offerings and announcements, partners, fans, key
personnel]
News releases and marketing announcements [ new products, defective products, lawsuits,
hirings.firings, acquisitions]
Trade publications [employee/departmental highlights, technical product specifications,
products or technologies used]
Job announcements [technologies used, skill shortages, under staffed departments]
Forum postings [email addresses, technologies used, information leakage, deficient areas]
TOOLS / RESOURCES:
–
–
–
–
–
–
–
–
–
Search Engines (Google, Bing, Baidu, Duck Duck Go, Blekko, etc.)
FOCA (document collection and metadata analysis)
SearchDiggity (google dorks, document collection and analysis)
SiteDigger (google dorks, document collection and analysis)
Recon-Ng
Goofile
Metagoofil
Httrack / ZED attack proxy / Burp / wget / curl
Maltego
40. osint - products
•
Products include
– Users categorized according to:
•
•
•
•
Role / Responsibility
Organization
Time in position
Physical location
– Email addresses
– Vulnerable product/technology used
– Spear fishing themes [recent promotion, new requirement, gossip, new
acquisition etc.]
– Communication patterns amongst employees or partners
– Social Engineering script based on good/bad user habits/interests
– Target subnets, hosts, applications
– Vulnerable web page / form
– Protected or default web pages
– Sensitive documents
– Building layouts
– Cohabitants
– Threat vectors / Agents
– Password policy
– Hub users
– Bridged users
41. footprinting – process notes
• Don’t underestimate the importance of **native** administrative tools
• Understand exactly what a tool will do before you run it
–
–
–
–
–
What are you after?
What Snort signatures may fire?
What kind of load does it put on the target system
What is the frequency of requests?
For web requests, what User agents are used?
• Investigate **every** finding no matter how esoteric
• Take ridiculously detailed notes [ date, time, tool used, command
run, switches used, file saved ]
• Organize your notes so they will still make sense 30 days from now [
Evernote (local), Zim, Keepnote, etc.]
• Mind your surroundings
–
–
–
–
Is this system in scope?
What makes this system an attractive target?
Should I trust my results? Do they make sense?
What do I hope to gain? PHI, PII, beachhead, user credentials?
42. footprinting – detailed data
Foot printing the organization and its partners (external / internal)
– ACTION: gathering and analyzing data from:
•
•
•
•
•
Discovered subnets and hosts
Running services and applications
Open ports
Hostnames (forward/reverse DNS)
Protection mechanisms
– TOOLS
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Maltego [ hostnames, IPs, subnets, and much more ]
WHOIS / WHOIS by IP
nslookup / dig / fierce / dnsrecon / dnsenum / deepmagic / robtex [ DNS ]
Goohost [ target hosts ]
recon-ng [ target hosts, subnets, users, and more ]
Portqry [ port scanning ldap, smb, smtp, mssql, netbios, rpc, isa ]
nmap / nse scripts [ port scanning, enumeration, banner grabbing ]
Msf [ port scanning, enumeration ]
Sqlmap / burp suite / zed attack proxy / nikto / w3af / skipfish / dirbuster [ web apps ]
Nessus / OpenVAS [ vulnerability scanning ]
Winfo / enum / nbtscan / nbtdump / nbtenum / net commands [ smb enumeration ]
Ike scan [ vpn scanning and enumeration ]
Smtp_enum_user [ smpt user identification ]
Blue Pill / Red Pill
43. footprinting - products
•
Products include
– Hostnames
– Hosts categorized according to:
•
•
•
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Program of record (PORs)
Function (workstation, database, application, name server, mail server, etc.)
Trust relationships
Open ports
Misconfigured services
Interesting error messages
Unpatched systems and/or applications
Vulnerable web applications
Lockout thresholds
Major/Minor version numbers
Email addresses
Outdated systems
Test systems
Default credentials
Virtualization platforms / systems
Load Balancers
Web application firewalls
Internal IP address space
Trust relationships
Nature and frequency of communications between systems
Host and Network based protection mechanisms