This is an empirical investigation into the reasons why many developers misuse the Java cryptography APIs. Such API misuses can lead to non-intentional security vulnerabilities. We triangulate data from three different sources of information: StackOverflow, GitHub, and developer surveys.
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Why Java Developers Struggle With Cryptography APIs?
1. “Jumping Through Hoops”
Why do Java Developers Struggle With
Cryptography APIs?
Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden
sarahnadi.org@sarahnadi
ICSE ’16 — May 20th, 2016
4. Sarah Nadi
User accounts
Payment info.
How to encrypt data?
Encryption vs
Hashing?
Encryption mode?
Salted hashing?
How to securely connect
to a server?
An Application Developer’s World
2
Application
Sensitive user
documents
5. Sarah Nadi
Application Developers in The Wild
3
83% of 269 Vulnerabilities are due to misuse of crypto libraries
[Lazar et al., APSys ’14]
Even Amazon & Paypal misuse SSL certificate validation
[Georgiev et al., CCS ‘12]
88% of ~12,000 Android apps misuse crypto APIs
[Egele et al., CCS ‘13]
7. Sarah Nadi
Example of an API Misuse
5
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
8. Sarah Nadi
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
Example of an API Misuse
6
“Algorithm/Mode/Padding”
9. Sarah Nadi
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
Example of an API Misuse
6
“Algorithm/Mode/Padding”
In some API implementations,
default mode for AES is
Electronic Codebook (ECB) —
which is insecure
10. Sarah Nadi
Sarah Nadi
Example of an API Misuse
5
SecretKey secretKey = …
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
[https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation]
Example of an API Misuse
6
“Algorithm/Mode/Padding”
In some API implementations,
default mode for AES is
Electronic Codebook (ECB) —
which is insecure
12. Sarah Nadi 7
Why do Developers Struggle
With Cryptography APIs?
What obstacles do
developers face?
13. Sarah Nadi 7
Why do Developers Struggle
With Cryptography APIs?
What are the common
cryptography tasks
developers perform?
What obstacles do
developers face?
14. Sarah Nadi 7
Why do Developers Struggle
With Cryptography APIs?
What are the common
cryptography tasks
developers perform?
What tools/methods would help
developers use cryptography
more effectively?
What obstacles do
developers face?
20. Sarah Nadi
Study 2 (S2) Design
10
random
100
repos
javax.crypto
Goal: Find tasks
21. Sarah Nadi
Study 2 (S2) Design
10
random
100
repos
What crypto task
is performed?
javax.crypto
Goal: Find tasks
22. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Goal: Find obstacles, tasks, & suggestions
23. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Goal: Find obstacles, tasks, & suggestions
24. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Goal: Find obstacles, tasks, & suggestions
25. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Goal: Find obstacles, tasks, & suggestions
26. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Frequently-used Crypto
APIs/libraries & ease of use
Goal: Find obstacles, tasks, & suggestions
27. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Frequently-used Crypto
APIs/libraries & ease of use
Obstacles (free-text +
rated)
Goal: Find obstacles, tasks, & suggestions
28. Sarah Nadi
Study 3 (S3) & Study 4 (S4) Survey Design
11
Background
Frequency of Cryptography Use
Never Rarely
…
Occasionally
…
Frequently
…
Cryptography Tasks Used/Needed
Frequently-used Crypto
APIs/libraries & ease of use
Obstacles (free-text +
rated)
Goal: Find obstacles, tasks, & suggestions
37. Sarah Nadi
What Obstacles do Developers Face?
15
Obstacle % Posts
API use 57%
Domain knowledge 15%
Provider & Setup 15%
Library Identification 7%
Domain knowledge + API use 6%
38. Sarah Nadi
What Obstacles do Developers Face?
15
S3 participants mention:
poor documentation, bad API design,and missing cryptography knowledge
Obstacle % Posts
API use 57%
Domain knowledge 15%
Provider & Setup 15%
Library Identification 7%
Domain knowledge + API use 6%
39. Sarah Nadi
What Obstacles do Developers Face?
15
S3 participants mention:
poor documentation, bad API design,and missing cryptography knowledge
65% of S4 Participants find the
APIs hard to use
Obstacle % Posts
API use 57%
Domain knowledge 15%
Provider & Setup 15%
Library Identification 7%
Domain knowledge + API use 6%
41. Sarah Nadi
Obstacle Category 1: Domain Knowledge
17
11%
24%
38%
27%
Frequently Occasionally Rarely Never Don't know
Identify correct cryptography algorithm
14%
41% 43%
3%
Frequently Occasionally Rarely Never Don't know
Identify relevant cryptography concepts
Developers may not always know the correct
cryptography algorithm to use
42. Sarah Nadi
Obstacle Category 2: Setting Up
18
27% 32% 27%
11% 3%
Frequently Occasionally Rarely Never Don't know
Identify relevant Java API
22%
41%
30%
5% 3%
Frequently Occasionally Rarely Never Don't know
Select provider
22% 27% 32%
16%
3%
Frequently Occasionally Rarely Never Don't know
Setup environment
43. Sarah Nadi
Obstacle Category 2: Setting Up
19
27% 32% 27%
11% 3%
Frequently Occasionally Rarely Never Don't know
Identify relevant Java API
22%
41%
30%
5% 3%
Frequently Occasionally Rarely Never Don't know
Select provider
22% 27% 32%
16%
3%
Frequently Occasionally Rarely Never Don't know
Setup environment
Finding the right API/library to use & setting it
up is often an obstacle in itself
44. Sarah Nadi
19%
54%
22%
5%
Frequently Occasionally Rarely Never Don't know
32%
43%
19%
5%
Frequently Occasionally Rarely Never Don't know
35%
46%
11% 8%
Frequently Occasionally Rarely Never Don't know
Obstacle Category 3: API Use
20
Identify sequence of method calls
Identify parameters
Understand API error messages
45. Sarah Nadi
19%
54%
22%
5%
Frequently Occasionally Rarely Never Don't know
32%
43%
19%
5%
Frequently Occasionally Rarely Never Don't know
35%
46%
11% 8%
Frequently Occasionally Rarely Never Don't know
Obstacle Category 3: API Use
21
Identify sequence of method calls
Identify parameters
Understand API error messages
The most frequently faced obstacle is
identifying the sequence of API method calls
47. Sarah Nadi
Common Cryptography Tasks
23
10% of analyzed GitHub repos
Top task by 64% of S3 participants
Avg. Rank of 3.95 by 28 S4 participants
6% of analyzed GitHub repos
Top task by 34% of S3 participants
Avg. Rank 2.22, 35 S4 participants
64% of analyzed GitHub repos
37% of analyzed StackOverflow posts
Avg. rank 5.03, 25 S4 participants
User Authentication
Secure Communication
Symmetric Encryption
49. Sarah Nadi
What do Developers Want?
25
Better Documentation
“Better documentation with examples
and deeper layers of documentations
with theoretical knowledge” (S4-P23)
“better examples” (S4-P2)
“More/better example code, access to API source code and
high quality JavaDoc with cross references” (S4-P12)
“A real documentation would be very helpful. Some kind of best
practice methodology for a crypto API usage could also help” (S4-P18)
50. Sarah Nadi
What do Developers Want?
26
Higher Abstraction Level
“Make [the] API simpler, e.g., have a class with
the name AES or RSA[;] getInstance(‘whatever’) is
bad API design” (S4-P36)
“[I want something that] just takes input
[…]. Short and sweet” (S3-P3)
“Higher level task oriented APIs for things like
public key crypto, key exchanges, ..” (S4-P27)
“High level APIs [that] can't be used
incorrectly..” (S4-P8)
“A library providing simple API calls (one
or two methods and simple parameters)
for different use cases.” (S4-P7)
“Standardized task-based API” (S4-P1)
“…Providing higher-level APIs, .. would go
along way to making sure developers don't
do something dumb.” (S4-P27)
51. Sarah Nadi
What do Developers Want?
27
“Test tooling that understands encryption and
verifies if encryption is used where it should be.
Source scanners that identify configuration
mistakes, weak algorithms etc.” (S4-P17)
Tool Assistance
“Some kind of testing tool…” (S4-P13)
“Special CryptoDebugger” (S4-P11)
“Templates for common used patterns” (S4-P7)
“IDE Plugin generating code pattern
for specific use cases” (S4-P6)
53. Sarah Nadi 29
Better Documentation
Higher Abstraction Level
Tool Assistance
How Can We Move Forward?
54. Sarah Nadi
The Ideal Tool Assistance
30
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
55. Sarah Nadi
The Ideal Tool Assistance
30
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
56. Sarah Nadi
The Ideal Tool Assistance
30
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
57. Sarah Nadi
The Ideal Tool Assistance
30
BlockCipher
AES
keySize (128, 192, 256)
mode (ECB, CBC, …)
padding (PKCS5Padding, NoPadding)
DES
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
58. Sarah Nadi
The Ideal Tool Assistance
30
BlockCipher
AES
keySize (128, 192, 256)
mode (ECB, CBC, …)
padding (PKCS5Padding, NoPadding)
DES
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey secretKey = keyGen.generateKey();
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
59. Sarah Nadi
The Ideal Tool Assistance
30
BlockCipher
AES
keySize (128, 192, 256)
mode (ECB, CBC, …)
padding (PKCS5Padding, NoPadding)
DES
COMMON
TASKS
TO
SUPPORT
STEPS
TO
SUPPORT
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey secretKey = keyGen.generateKey();
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7PADDING");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
cipher.doFinal(inputMsg);
60. Sarah Nadi
“The ability to just perform some
simple cryptographic tasks in Java
without jumping through hoops
would be brilliant…” (S4-P10)
31
61. “Jumping Through Hoops”
Why do Java Developers Struggle With
Cryptography APIs?
Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden
sarahnadi.org@sarahnadi