The work presentation on APSEC 2011

1. Recommending Proper API Code Examples
for Documentation Purpose
Lee Wei Mar, Ye-Chi Wu, and Hewijin Christine Jiau
DBSE Lab @ Inst. CCE & Dept. EE
National Cheng Kung University (NCKU), Taiwan

2. Outline
• Introduction
• Code Search Engines for Code Example Collection
• The Proposed Methodology - PropER-Doc
– Execution Process
– Recommendation Assistants
• Evaluation
• Conclusion

3. Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works

4. Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works
Power ↑ API Complexity ↑ Learning Barrier ↑

5. Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works
Hamper
Power ↑ API Complexity ↑ Learning Barrier ↑

6. Introduction
• Nowadays, object-oriented frameworks & libraries offer powerful APIs
that facilitate programmers in accomplishing their works
Hamper
Power ↑ API Complexity ↑ Learning Barrier ↑
To ensure the facilitation, offering effective API learning resources
is essential for API providers

7. Introduction (cont.)
• Code examples (CEs) are effective resources for API learning

8. Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
Eclipse SWT
public class Snippet26 {
public static void main (String [] args) {
Display display = new Display ();
Shell shell = new Shell (display);
Combo combo = new Combo (shell, SWT.READ_ONLY);
combo.setItems (new String [] {"Alpha", "Bravo", "Charlie"});
Rectangle clientArea = shell.getClientArea ();
combo.setBounds (clientArea.x, clientArea.y, 200, 200);
shell.pack ();
shell.open ();
….
}
Independent CE repository

9. Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
Eclipse SWT Eclipse JDT
public class Snippet26 {
public static void main (String [] args) {
Display display = new Display ();
Shell shell = new Shell (display);
Combo combo = new Combo (shell, SWT.READ_ONLY);
combo.setItems (new String [] {"Alpha", "Bravo", "Charlie"});
Rectangle clientArea = shell.getClientArea ();
combo.setBounds (clientArea.x, clientArea.y, 200, 200);
shell.pack ();
shell.open ();
….
}
Independent CE repository Embedding CE in API reference

10. Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
Guide programmers to properly use API by demonstration

11. Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
Guide programmers to properly use API by demonstration
• However, many APIs fail to offer sufficient CEs

12. Introduction (cont.)
• Code examples (CEs) are effective resources for API learning
Guide programmers to properly use API by demonstration
• However, many APIs fail to offer sufficient CEs
API providers must invest large human effort for CE
construction from scratch

13. CSEs for Code Example Collection
• Code Search Engines (CSEs) are potentially useful in CE collection
query
API provider
source files potentially
use the API element

14. CSEs for Code Example Collection
• Code Search Engines (CSEs) are potentially useful in CE collection
query
API provider
source files potentially
use the API element
The signature of a specific API element
is given as the query keyword

15. CSEs for Code Example Collection
• Code Search Engines (CSEs) are potentially useful in CE collection
query
API provider
source files potentially
use the API element
The signature of a specific API element
is given as the query keyword • These files contain “client code” that
demonstrates the use of the API element
• The client code can be treated as
“candidates” of CEs for documentation

16. CSEs for Code Example Collection - Challenges
• Code Search Engines (CSEs) are potentially useful in CE collection
query
API provider
source files potentially
use the API element
API providers must manually

17. CSEs for Code Example Collection - Challenges
• Code Search Engines (CSEs) are potentially useful in CE collection
query
API provider
source files potentially
use the API element
API providers must manually
1. Locate API usage code scattered in source files as candidates
2. Reorganize candidates according to different usages
3. Check appropriateness of candidates for documentation

18. CSEs for Code Example Collection - Challenges
• Code Search Engines (CSEs) are potentially useful in CE collection
query
API provider
source files potentially
use the API element
API providers must manually
1. Locate API usage code scattered in source files as candidates
2. Reorganize candidates according to different usages
3. Check appropriateness of candidates for documentation
These manual efforts must be reduced to make such practice applicable

19. The Proposed Methodology – PropER-Doc
• Proper code Example candidates Recommendation for Documentation
– Use API element links to guide the recommendation from CSE results

20. The Proposed Methodology – PropER-Doc
• Proper code Example candidates Recommendation for Documentation
– Use API element links to guide the recommendation from CSE results
Links between API elements to indicate the joint use relationship
• Structural Links from structural dependencies in API implementation
• Conceptual Links from cross references in API documentation

21. The Proposed Methodology – PropER-Doc
• Proper code Example candidates Recommendation for Documentation
– Use API element links to guide the recommendation from CSE results
Links between API elements to indicate the joint use relationship
• Structural Links from structural dependencies in API implementation
• Conceptual Links from cross references in API documentation
– Offer recommendation assistants for candidate inspection & selection

22. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
candidates
check API call
Candidate significance
candidate suggestion Annotation
annotated
candidates
Candidate
Recommendation

23. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider

24. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
The API provider gives a name
of the API type that needs CEs

25. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
The API provider gives a name
of the API type that needs CEs The query is delegated to CSEs

26. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
candidates
The API provider gives a name
of the API type that needs CEs The query is delegated to CSEs
• PropER-Doc collects candidates from CSE results
• Candidate: a method implementation that refers targetType
usage

27. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
candidates
check API call
Candidate significance
Annotation

28. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
candidates
check API call
For each API call, annotate it with a
Candidate significance
significance degree Annotation
6 0
(relevant) (irrelevant)

29. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
candidates
check API call
For each API call, annotate it with a
Candidate significance
significance degree Annotation
6 0
(relevant) (irrelevant)
annotated
candidates

30. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
candidates
check API call
candidate suggestion
Candidate significance
Annotation
annotated
candidates
Candidate
Recommendation

31. PropER-Doc Execution Process
perform
targetType query Candidate code search
Collection
API Provider
candidates
check API call
candidate suggestion
Candidate significance
Annotation
recommendation
with
assistants annotated
candidates
Candidate
Recommendation

32. Candidate Recommendation
Four Recommendation Assistants are offered to guide the recommendation

33. Candidate Recommendation
Four Recommendation Assistants are offered to guide the recommendation
[All candidates]
Grouping for
distinguishing API usages

34. Candidate Recommendation
Four Recommendation Assistants are offered to guide the recommendation
[All candidates] [A selected group]
Grouping for Diagramming API
distinguishing API usages types interaction

35. Candidate Recommendation
Four Recommendation Assistants are offered to guide the recommendation
[All candidates] [A selected group] [Candidates in a group]
Grouping for Diagramming API Ranking based on
distinguishing API usages types interaction appropriateness metrics

36. Candidate Recommendation
Four Recommendation Assistants are offered to guide the recommendation
[All candidates] [A selected group] [Candidates in a group] [A selected candidate]
Grouping for Diagramming API Ranking based on Presenting the
distinguishing API usages types interaction appropriateness metrics candidate for inspection

37. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[All candidates] – Distinguish different API usages

38. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[All candidates] – Distinguish different API usages
Rationale: In different usages, targetType tends to interact with different API types

39. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[All candidates] – Distinguish different API usages
Rationale: In different usages, targetType tends to interact with different API types
• PropER-Doc extracts iTypes of each
candidate
– iTypes: the set of API types where
targetType directly interacts with
• Candidates with the same iTypes tend
to demonstrate the same usage and are
grouped together

40. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[All candidates] – Distinguish different API usages
Rationale: In different usages, targetType tends to interact with different API types
• PropER-Doc extracts iTypes of each
candidate
– iTypes: the set of API types where
targetType directly interacts with
• Candidates with the same iTypes tend
to demonstrate the same usage and are
grouped together

41. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]

42. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage

43. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar

44. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type

45. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls

46. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls
– API call popularity

47. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls
– API call popularity

48. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[One group is selected for inspection]
Use visual notation to summarize API type interaction of a specific usage
• Visual Grammar
– Box: API Type
– Edge / Label: interaction / API calls
– API call popularity
• Selection of interested API calls for
filtering candidates

49. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]

50. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]
Which one is more appropriate for documentation and should be inspected first?

51. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]
Define three appropriateness metrics to rank the candidates

52. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A set of similar candidates demonstrating the same usage is chosen]
Define three appropriateness metrics to rank the candidates
Significance Density Cohesiveness
Importance of referred API calls Portion of relevant API calls Degree of relevant call aggregation
> > >
more more
representative comprehensible easier extraction
Candidate Statements with significance degree
6 0

53. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A set of candidates is chosen]
Rank function: f(c) = Significance(c) + Density(c) + Cohesiveness(c)

54. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]

55. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Highlight relevant code place to facilitate code inspection & extraction

56. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Highlight relevant code place to facilitate code inspection & extraction

57. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Highlight relevant code place to facilitate code inspection & extraction
Important code blocks are highlighted

58. Recommendation - Clustering
Grouping Diagramming Ranking Presenting
[A candidate is selected for inspection & CE construction]
Highlight relevant code place to facilitate code inspection & extraction
Key API elements are
marked as bold
Important code blocks are highlighted

59. Evaluation
• Objective: check PropER-Doc effectiveness on
A. Distinguishing different API usages among candidates
B. Suggesting proper candidates for CEs construction
• Procedure
1. For an API type, manually identify its typical usages (U1) that need CEs
2. Use PropER-Doc to collect candidates and distinguish the API type usages (U2)
3. Map between U1 and U2 (objective A)
4. For each usage in U1, use PropER-Doc to select the top-ranked candidate
5. Evaluate the effort for constructing CE from top-ranked candidate (objective B)
• Subject: ASTParser class (A complex API type in Eclipse JDT framework)

60. Evaluation Result – Mapping Result
U1 U2 (GID)
Get AST from ICompilationUnit 1
Get AST from a string of class body 2
Get AST from a string of statements 3
Get AST from a string of expression 4
Get AST from ITypRoot 5
Get IJavaElement binding info. 6
Get AST from source code 7
Get ASTs from ICompilationUnits 8
Get AST from IClassFile 9

61. Evaluation Result – Mapping Result
U1 U2 (GID) • Mostly 1 : 1 mapping
Get AST from ICompilationUnit 1
Get AST from a string of class body 2
Get AST from a string of statements 3
Get AST from a string of expression 4
Get AST from ITypRoot 5
Get IJavaElement binding info. 6
Get AST from source code 7
Get ASTs from ICompilationUnits 8
Get AST from IClassFile 9

62. Evaluation Result – Mapping Result
U1 U2 (GID) • Mostly 1 : 1 mapping
Get AST from ICompilationUnit 1 – Confirm the effectiveness of the
Get AST from a string of class body 2
grouping assistant in PropER-Doc
Get AST from a string of statements 3
Get AST from a string of expression 4
Get AST from ITypRoot 5
Get IJavaElement binding info. 6
Get AST from source code 7
Get ASTs from ICompilationUnits 8
Get AST from IClassFile 9

63. Evaluation Result – Mapping Result
U1 U2 (GID) • Mostly 1 : 1 mapping
Get AST from ICompilationUnit 1 – Confirm the effectiveness of the
Get AST from a string of class body 2
grouping assistant in PropER-Doc
Get AST from a string of statements 3
Get AST from a string of expression 4
• For the 3 : 1 mapping, the
diagramming assistant is useful in
Get AST from ITypRoot 5 distinguishing different usages
Get IJavaElement binding info. 6
Get AST from source code 7
Get ASTs from ICompilationUnits 8
Get AST from IClassFile 9

64. Evaluation Result – Mapping Result
U1 U2 (GID) • Mostly 1 : 1 mapping
Get AST from ICompilationUnit 1 – Confirm the effectiveness of the
Get AST from a string of class body 2
grouping assistant in PropER-Doc
Get AST from a string of statements 3
Get AST from a string of expression 4
• For the 3 : 1 mapping, the
diagramming assistant is useful in
Get AST from ITypRoot 5 distinguishing different usages
Get IJavaElement binding info. 6
Get AST from source code 7
Get ASTs from ICompilationUnits 8
Get AST from IClassFile 9
The Grouping and Diagramming assistants are useful
in distinguishing different API usages

65. Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh.
Get AST from ICompilationUnit 0/0 1.00
Get AST from a string of class body 0/0 1.00
Get AST from a string of statements 0/0 1.00
Get AST from a string of expression 4/0 1.00
Get AST from ITypRoot 0/1 1.00
Get IJavaElement binding info. 3/0 1.00
Get AST from source code 0/0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0/0 1.00
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)

66. Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh. • 5 of 9 top candidates can be
Get AST from ICompilationUnit 0/0 1.00 directly used as CEs
Get AST from a string of class body 0/0 1.00
Get AST from a string of statements 0/0 1.00
Get AST from a string of expression 4/0 1.00
Get AST from ITypRoot 0/1 1.00
Get IJavaElement binding info. 3/0 1.00
Get AST from source code 0/0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0/0 1.00
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)

67. Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh. • 5 of 9 top candidates can be
Get AST from ICompilationUnit 0/0 1.00 directly used as CEs
Get AST from a string of class body 0/0 1.00
Get AST from a string of statements 0/0 1.00 • Only 1 top candidate misses 1
statement for CE construction
Get AST from a string of expression 4/0 1.00
– Top candidates contains almost all
Get AST from ITypRoot 0/1 1.00 required API calls for CE construction
Get IJavaElement binding info. 3/0 1.00
Get AST from source code 0/0 1.00
Get ASTs from ICompilationUnits 16 / 0 0.63
Get AST from IClassFile 0/0 1.00
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)

68. Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh. • 5 of 9 top candidates can be
Get AST from ICompilationUnit 0/0 1.00 directly used as CEs
Get AST from a string of class body 0/0 1.00
Get AST from a string of statements 0/0 1.00 • Only 1 top candidate misses 1
statement for CE construction
Get AST from a string of expression 4/0 1.00
– Top candidates contains almost all
Get AST from ITypRoot 0/1 1.00 required API calls for CE construction
Get IJavaElement binding info. 3/0 1.00
Get AST from source code 0/0 1.00 • High cohesiveness in all top
candidates
Get ASTs from ICompilationUnits 16 / 0 0.63
– Effort for relevant code extraction
Get AST from IClassFile 0/0 1.00 from top candidates is quite low
∆Stmt (+/-): #(extra statements) / #(missed statements)
Coh.: Cohesiveness (between 0 and 1)

69. Evaluation Result – Constructing Effort
U1 ∆Stmt (+/-) Coh. • 5 of 9 top candidates can be
Get AST from ICompilationUnit 0/0 1.00 directly used as CEs
Get AST from a string of class body 0/0 1.00
Get AST from a string of statements 0/0 1.00 • Only 1 top candidate misses 1
statement for CE construction
Get AST from a string of expression 4/0 1.00
– Top candidates contains almost all
Get AST from ITypRoot 0/1 1.00 required API calls for CE construction
Get IJavaElement binding info. 3/0 1.00
Get AST from source code 0/0 1.00 • High cohesiveness in all top
candidates
Get ASTs from ICompilationUnits 16 / 0 0.63
– Effort for relevant code extraction
Get AST from IClassFile 0/0 1.00 from top candidates is quite low
PoperER-Doc suggests high quality candidates
for CEs construction

70. Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API
code examples using code search engines
1. Candidate grouping algorithm
2. API-type interaction diagram
3. Metrics-based ranking mechanism

71. Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API
code examples using code search engines
1. Candidate grouping algorithm Distinguish different API usages
2. API-type interaction diagram
3. Metrics-based ranking mechanism

72. Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API
code examples using code search engines
1. Candidate grouping algorithm Distinguish different API usages
2. API-type interaction diagram Inspect specific usage & filter candidates
3. Metrics-based ranking mechanism

73. Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API
code examples using code search engines
1. Candidate grouping algorithm Distinguish different API usages
2. API-type interaction diagram Inspect specific usage & filter candidates
3. Metrics-based ranking mechanism Select high quality candidates

74. Conclusion
• PropER-Doc is proposed to assist API providers in constructing proper API
code examples using code search engines
1. Candidate grouping algorithm Distinguish different API usages
2. API-type interaction diagram Inspect specific usage & filter candidates
3. Metrics-based ranking mechanism Select high quality candidates
• Evaluation on Eclipse JDT framework has been conducted to confirm the
effectiveness of PropER-Doc

75. Thank You
Lee Wei Mar (馬立偉)
lwmar@nature.ee.ncku.edu.tw
Database and Software Engineering Laboratory,
Institute of Computer and Communication Engineering &
Department of Electrical Engineering,
National Cheng Kung University, Tainan, Taiwan