Talk from ICSE 2012

1. Synthesizing API Usage Examples
Ray Buse Wes Weimer ICSE 2012
Zurich, Switzerland
2

2. Java

3. Simple Bank Withdraw Function
void withdraw( double amount )
{
double currentBal = getBalance();
double newBal = currentBal - amount;
if( newBal < 0 )
throw new OverDraftException();
account.setBalance(newBal);
}
4

4. Simple Bank Withdraw Function
void withdraw( double amount )
{
double currentBal = getBalance();
double newBal = currentBal - amount;
if( newBal < 0 )
throw new OverDraftException();
Initial
account.setBalance(newBal); Balance = 2
}
withdraw(1.10);
Final Balance = ?
5

5. Simple Bank Withdraw Function
void withdraw( double amount )
{
double currentBal = getBalance();
double newBal = currentBal - amount;
if( newBal < 0 )
throw new OverDraftException();
Initial
account.setBalance(newBal);Balance = 2
}
withdraw(1.10);
Final Balance = ?
Final Balance = 0.8999999999999999
6

6. 7

7. 8

8. “Improved” Withdraw Function
void withdraw( double amount )
{
BigDecimal currentBal = getBalance();
BigDecimal newBal
= currentBal.subtract(new BigDecimal(amount));
if( newBal.compareTo(BigDecimal.ZERO) < 0 )
throw new OverDraftException();
account.setBalance(newBal);
}
9

9. “Improved” Withdraw Function
void withdraw( double amount )
{
BigDecimal currentBal = getBalance();
BigDecimal newBal
= currentBal.subtract(new BigDecimal(amount));
if( newBal.compareTo(BigDecimal.ZERO) < 0 )
throw new OverDraftException();
Initial
Balance = 2
account.setBalance(newBal);
}
withdraw(1.10);
Final Balance = ?
Final Balance = 0.899999999999999911182158029987
10

10. BigDecimal
public BigDecimal(double val)
Translates a double into a BigDecimal which is the exact
decimal representation of the double's binary floating-
point value. The scale of the returned BigDecimal is the
smallest value such that (10scale × val) is an integer.
http://docs.oracle.com/javase/6/docs/api/java/math/BigDecimal.html
11

11. “Improved” Withdraw Function
void withdraw( double amount )
{
BigDecimal currentBal = getBalance();
BigDecimal newBal
= currentBal.subtract(new BigDecimal(amount));
if( newBal.compareTo(BigDecimal.ZERO) < 0 )
throw new OverDraftException();
Initial Balance
account.setBalance(newBal); = 1.10
}
withdraw(1.10);
Final Balance = ?
OverDraftException
12

12. One Solution: String Constructor
void withdraw( String amount )
{
BigDecimal currentBal = getBalance();
BigDecimal newBal
= currentBal.subtract(new BigDecimal(amount));
if( newBal.compareTo(BigDecimal.ZERO) < 0 )
throw new OverDraftException();
account.setBalance(newBal);
Initial
Balance = 2
}
withdraw(“1.10”);
Final Balance = 0.90
13

13. “The greatest obstacle to learning an
API … is insufficient or inadequate
examples”
Martin Robillard. What Makes APIs Hard to Learn? Answers
from Developers. IEEE Software, 26(6):27-34, 2009.

14. The Rest of this Talk
What makes a
good example?
Motivation
Approach: Example Synthesis Evaluation
15

15. The Rest of this Talk
What makes a
good example?
Approach: Example Synthesis Evaluation
16

16. What makes a good example?

17. Some Common Sources of Examples
18

18. Research Tools
• MAPO
– H. Zhong, T. Xie, L. Zhang, J. Pei, and H. Mei,
“MAPO: Mining and recommending API usage
patterns,” in European Conference on Object-
Oriented Programming, 2009.
• eXoaDocs
– J. Kim, S. Lee, S. Hwang, and S. Kim, “Towards an
intelligent code search engine,” in AAAI
Conference on Artificial Intelligence, 2010.
19

19. Research Tools
• MAPO
– H. Zhong, T. Xie, L. Zhang, J. Pei, and H. Mei,
“MAPO: Mining and recommending API usage
patterns,” in European Conference on Object-
Oriented Programming, 2009.
• eXoaDocs
– J. Kim, S. Lee, S. Hwang, and S. Kim, “Towards an
intelligent code search engine,” in AAAI
Conference on Artificial Intelligence, 2010.
20

20. Gold-Standard Human-Written Examples
Tutorials JavaDocs
21

21. Search-based examples
BufferedReader reader = new BufferedReader(new
InputStreamReader(page));
try {String line = reader.readLine();
while (line != null) {
if (line.matches(substituteWikiWord(wikiWord,newTopicPattern)))
{
JExamples: BufferedReader
22

22. Hand-Crafted Examples
Example Data
FileOutputStream fos = new FileOutputStream("t.tmp");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeInt(12345);
oos.writeObject("Today");
oos.writeObject(new Date());
oos.close();
JavaDoc: java.util.ObjectOutputStream
Complete
23

23. Hand-Crafted Examples
Abstract
Initialization
int glyphIndex = ...;
GlyphMetrics metrics =
GlyphVector.getGlyphMetrics(glyphIndex);
int isStandard = metrics.isStandard();
float glyphAdvance = metrics.getAdvance();
JavaDoc: java.awt.font.GlyphMetrics
24

24. Hand-Crafted Examples
for(char c = iter.first();
c != CharacterIterator.DONE;
c = iter.next()) {
processChar(c);
} Hole
JavaDoc: java.text.CharacterIterator
try {
file.delete(); Hole
} catch (IOException exc) {
// failed to delete, do error handling here
}
return FileVisitResult.CONTINUE;
JavaDoc: java.nio.FileVisitor
25

25. Our API Examples
FileReader fReader; //initialized previously
BufferedReader br = new BufferedReader(fReader);
while(br.ready()) {
String line = br.readLine();
//do something with line
}
br.close();
Query: java.util.BufferedReader
26

26. Our API Examples
Common Abstract
variable names Initialization
FileReader fReader; //initialized previously
BufferedReader br = new BufferedReader(fReader);
while(br.ready()) {
String line = br.readLine();
//do something with line
}
br.close(); “Holes”
Query: java.util.BufferedReader
Complete
27

27. Automatic Documentation
• Cheap
• Always up-to-date
• Complete
• Well-defined trust properties
• Structured (searchable)
28

28. Our Approach

29. Specification Mining
• ``Common behavior is correct behavior.''
Software
Corpus
Miner
Common Patterns
30

30. Specification Mining
• ``Common behavior is correct behavior.''
Software
Corpus
Bug Finder
Miner
Common Patterns
31

31. API Example Synthesis
• ``Common behavior makes the best examples.''
Software
Corpus
Miner Code Gen
Common Patterns
32

32. Two Key Insights
Dataflow Analysis Abstraction

33. Two Key Insights
Abstraction
Operator
Dataflow Analysis Abstraction

34. Search Term
Software “BufferedReader”
Corpus Control Flow
<init> Graph
Representation
ready
Miner
readLine
print
close
35

35. argument
Search Term argument
“foo.txt”
Software FileReader
“BufferedReader”
Corpus
<init> identifier
br
ready
Miner predicate
ready()
readLine
returns
String:
line
print
With Dataflow argument
Annotations Relevant line
to Documentation close
36

36. Search Term
Software “BufferedReader”
Corpus
Miner
...
Many
Concrete Uses
37

37. Multiple Common Patterns
BufferedReader br = new BufferedReader(new FileReader("foo.in"));
while( br.ready() )
{
String s = br.readLine();
//Do something with s
}
BufferedReader br = new BufferedReader(new FileReader("foo.in"));
String s;
while( ( s = br.readLine() ) != null )
{
//Do something with s
}
38

38. Clustering: Group Similar Patterns
Many Concrete
Uses
… Grouped Into
Clustering
A Few Different
Operator Patterns
39

39. Clustering
• k-medoids
• Distance metric captures
difference in order of
statements and types of
objects
40

40. Abstraction
Many
Concrete
Examples
Abstraction
Operator
… Into One
Abstract Example
41

41. Concrete Concrete
if(iter.hasNext()) { if(iter.hasNext()) {
set.add( iter.next() ); print( iter.next() );
} }
Abstraction
Least-upper- Operator
bound types
if(iter.hasNext()) {
Object o = iter.next(); Insert Hole
//Do something with o
}
42

42. Concrete
Iterator iter = set.iterator();
...
Concrete
Iterator iter = list.iterator();
...
Abstraction
Operator Abstract
Initialization
Iterator iter = SOMETHING.iterator();
...
43

43. Recap
Software
Corpus
Mine Cluster Abstract
Search Term
44

44. Recap
Software
Corpus
Mine Cluster Abstract
Search Term
Yield Well-Formed Code Gen
Source Code
45

45. Recap
Software
Corpus
Mine Cluster Abstract
Search Term
Yield Well-Formed Code Gen
Source Code
46

46. 47

47. Examples
Calendar calendar = Calendar.getInstance();
Date d = calendar.getTime();
//Do something with d
Query: java.util.Calendar
48

48. Examples
String regex; //initialized previously
String input; //initialized previously
Pattern pattern = Pattern.compile(regex);
Matcher m = pattern.matcher(input);
//Do something with m
Query: java.util.regex.Pattern
49

49. Limitations
• Can’t always be perfectly precise
– E.g., Aliasing, Types
– Conservative analysis preserves correctness
• Common usage is not always best
– E.g., poor exception handling
– Guarantee representative examples
• Not all APIs have indicative patterns
• Some patterns are difficult to find
– Message passing over network etc.
50

50. Evaluation

51. Name of Class
Two examples
randomly drawn from
{Our Tool, Human-
Written, eXoaDoc}
Participant specifies
preference 52

52. Comparison to Code Search
53

53. Comparison to Human-Written
54

54. Potential Use Cases

55. Warnings for Likely Mistakes
56

56. Auto Completion
57

57. Conclusion

58. Future Work
• More Complex Queries
– How do I use X with Y ?
– Kuat Yessenov, Zhilei Xu, and Armando Solar-
Lezama, Data-driven synthesis for object-oriented
frameworks, OOPSLA, 2011.
• What does this example do?
– Symbolic execution of the example itself
• Additional Guarantees
– Proof Carrying Examples
59

59. Our Implementation
60

60. Documents how a source code change affects
program behavior.
arrestedcomputing.com/deltadoc
61

61. 62

62. 63

63. 64

64. Thank You!

65. Questions?
66