This document discusses various methods for measuring criminal and illicit activities that cannot be directly observed, such as fraud, cash movement, and cybercrimes. It provides examples of direct measurement techniques including surveys and samples, as well as indirect methods like accounting gaps and system statistics. Specific measurement approaches are examined for crimes like fraud, cash usage, and cybercrimes including spam, botnets, and malware. The document advocates testing simple metrics and aggregating existing data to better estimate underground and illicit activities.
15. NCVS is the Nation's
primary source of
information on criminal
victimization.
Sample of 76,000
households & ~135,300
persons
Frequency,
characteristics and
consequences (crimes in
the US)
The survey enables BJS
to estimate the likelihood
of victimization via
categories of violent &
property crimes for the
population as a whole
Population segments:
gender, age, ethnicity,
geography
http://bjs.ojp.usdoj.gov/index.cfm?ty=dcdetail&iid=245
Thursday, April 28, 2011
20. Cybercrime against Businesses,
2005
7,818 businesses in 2005
Data on:
Monetary loss and system
downtime
Types of offenders, types of
systems affected,
vulnerabilities, whether
incidents were reported to LE
Highlights:
3,247 businesses incurred loss
totaling $867M
Majority of attacks went
unreported to LE
http://bjs.ojp.usdoj.gov/index.cfm?ty=pbdetail&iid=769
Thursday, April 28, 2011
26. Method Approach
Direct methods Surveys
Audits
Indirect methods
Via national accounting
Gap between production & expenditure
Via national accounting Gap between official & actual laborVia national accounting
Gap between official & actual income
Monetary statistics
Velocity of M1 (cash/currency)
Monetary statistics
Velocity of major bills
Monetary statistics
Transactions approach
Monetary statistics
Currency demand
Physical input consumption Electricity consumption
Soft modeling Cause/effect (DYMIMIC)
The Shadow Economy: An International Study. Cambridge Press. Schneider & Enste (2002)
Thursday, April 28, 2011
27. Changes over time
0
7.5
15
22.5
30
Belgium Sweden Ireland France Netherlands Germany GB USA
Size of shadow economy as a % of official GNP (cash approach)
Data Source: Schneider & Enste (1998)
1970
1980
1994
1995
1996
1997
Thursday, April 28, 2011
28. Comparing results
0
7.5
15
22.5
30
Belgium Sweden Ireland France Netherlands Germany GB USA
Size of the shadow economy as % of official GNP
Cash approach (Johnson 1990/93)
Cash approach (Schneider 1989/90)
Cash approach (Schneider 1990/93)
Electricity Consumption (1989/90)
Data Source: Schneider & Enste (1998)
Thursday, April 28, 2011
29. Method Example
Direct methods
Samples/Surveys Crime surveys
Intrusive observation Tax Audits
Passive observation Bill tracking
Indirect methods
Gap accounting Income vs expenditure
System statistics Velocity of money
Impact indicators Energy consumption
Qualitative modeling DYMIMIC
Thursday, April 28, 2011
33. Spam & Phishing
Majority of email is “bad” (~90%
Q1‘2010)
Malware taking share from spam
Crafted attacks as well as blitzes
Most campaigns are short (<24 hours)
Botnets
Virus &
Malware
Thursday, April 28, 2011
35. Custom malware
Social networks: Infection
mechanism & targets
Drive-bys
Mobile & POS devices
Spam & Phishing
Botnets
Virus & Malware
Thursday, April 28, 2011
36. ISPs, independent researchers
Mechanisms of communication,
control
Profiling & tracking (network,
victims, targets)
Feature analysis
Performance (attack metrics)
Spam & Phishing
Botnets
Virus &
Malware
Thursday, April 28, 2011
37. Packet, Flow, Log (app, A/V, spam) analysis
Machine learning algorithms for IRC-based C&C botnet traffic
(Strayer et al)
Clustering analysis for P2P botnet detection (Zeidanloo et al)
DNS analysis & monitoring
Changes in DNS traffic patterns (volume, errors)
Sinkholing (domain name takeovers)
IRC & P2P infiltration
Honeypots Spam & Phishing
Botnets
Virus &
Malware
Thursday, April 28, 2011
38. useful.
Spam & Phishing
Botnets Virus &
Malware
Google Postini Services Spam Trend & Analysis (July
2010, >3B email connections/day)
McAfee Quarterly Threats Report, (>20M new malware
samples in 2010)
Symantec State of Spam & Phishing, 300M email
addresses
Trustwave Global Security Report 2011 (15 billion
emails from 2006-10, 220 breach investigations)
ENISA: Botnets: Measurement, Detection, Disinfection
and Defence
Thursday, April 28, 2011
39. Method Example
Direct methods
Samples/Surveys Spam & Phishing, Virus & Malware
Intrusive observation Sinkholing, Audits
Passive observation Honeypots, Flow analysis
Indirect methods
Gap accounting “Cuckoo’s Egg”
System statistics
Impact indicators Breach investigations
Qualitative modeling
Thursday, April 28, 2011
40. More opportunities for data
aggregation
System accounting
Test simple metrics, data
sets in experimental
models
For existing data-sets:
Opportunities to move
from transactional to flow-
based
Questions?
Allison Miller
@selenakyle
Thursday, April 28, 2011