1. Recommendation Systems for Code Reuse Tao Xie Department of Computer Science North Carolina State University Raleigh, USA

3. Example Task from Eclipse Programming Task: How to parse code in a dirty editor of Eclipse? ? Query: “ IEditorPart -> ICompilationUnit ” Open Source Projects 1 2 N … … Extract MIS 1 MIS 2 ... … MIS k *MIS: Method-Invocation sequence, FMIS: Frequent MIS FMIS 1 FMIS 2 … FMIS n Recommend Mine PARSEWeb [Thummalapenta&Xie ASE 07]

14. What are Developers Searching for? Assieme [Hoffmann et al. UIST 07] 339 sessions related to Java programming 15 million queries of Windows Live Search from May 2006. 117 API sessions (34.2%); 70 trouble-shooting sessions (20.6%)

17. Why Not Use Web Search Engines? Only compatible with new Java versions Requires installation of external library, but no link Code on pages essentially the same Contains no code examples parse xml java ©Raphael Hoffmann Assieme [Hoffmann et al. UIST 07]

18. Code Search Engines import javax.xml.parsers.*; import org.w3c.dom.*; public class JAXPSample { public static void main(String[] args) { String filename = "sample.xml"; try { DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); DocumentBuilder parser = factory.newDocumentBuilder(); Document d = parser.parse(filename); } catch ( Exception e ) { System.err.println("Exception: " + e.getMessage()); } } } Index source code of open-source Projects (from compressed archive Files and CVS repositories) Code is parsed and terms in type names, variable names, etc. are weighted differently. ©Raphael Hoffmann Assieme [Hoffmann et al. UIST 07]

19. Why not use code search engines only? Irrelevant (An Emacs Lisp File!?!) Code is complicated, contains no comments related to query, and is more than 300(!) lines long Requires installation of external library, but no link Code on pages essentially the same parse xml java ©Raphael Hoffmann Assieme [Hoffmann et al. UIST 07]

20. Why not use code search engines only? MAPO [Xie&Pei MSR 06]

30. Jungloids with Downcasts IDebugView debugger = ... Viewer viewer = debugger.getViewer(); IStructuredSelection sel = (IStructuredSelection) viewer.getSelection(); JavaInspectExpression expr = (JavaInspectExpression) sel.getFirstElement(); Prospector [Mandelin et al. PLDI 05 ] IDebugView Viewer ISelection IStructuredSelection JavaInspectExpression Object getViewer() getSelection() getFirstElement() getInput() downcast downcast

33. Association Rules KApplication reuse patterns CodeWeb [Michail ICSE 00]

34. #include <abcdef.h> void p ( ) { b ( ) ; c ( ) ; } void q ( ) { c ( ) ; b ( ) ; } void r ( ) { e ( ) ; f ( ) ; } void s ( ) { f ( ) ; e ( ) ; } int main ( ) { int i, j, k; a ( ); if ( i == 1) { f ( ) ; e ( ) ; c ( ) ; exit ( ); } else { if ( j == 1 ) p ( ); else q ( ); d ( ) ; if ( k == 1 ) r ( ); else s ( ); } } Frequent SubSeq/Partial Order Consider APIs a, b, c, d, e, and f Apiator [ Acharya et al. FSE 07 ]

35. #include <abcdef.h> void p ( ) { b ( ) ; c ( ) ; } void q ( ) { c ( ) ; b ( ) ; } void r ( ) { e ( ) ; f ( ) ; } void s ( ) { f ( ) ; e ( ) ; } int main ( ) { int i, j, k; a ( ); if ( i == 1) { f ( ) ; e ( ) ; c ( ) ; exit ( ); } else { if ( j == 1 ) p ( ); else q ( ); d ( ) ; if ( k == 1 ) r ( ); else s ( ); } } 1 a  f  e  c 2 a  b  c  d  e  f 3 a  c  b  d  e  f 4 a  b  c  d  f  e 5 a  c  b  d  f  e a d c e b f a  b  d  e a  b  d  f a  c  d  e a  c  d  f (b) Static program traces (c) Frequent sequential patterns Support 4/5 (d) Frequent partial order R (a) Example code Consider APIs a, b, c, d, e, and f Frequent SubSeq/Partial Order Apiator [ Acharya et al. FSE 07 ]

36. #include <abcdef.h> void p ( ) { b ( ) ; c ( ) ; } void q ( ) { c ( ) ; b ( ) ; } void r ( ) { e ( ) ; f ( ) ; } void s ( ) { f ( ) ; e ( ) ; } int main ( ) { int i, j, k; a ( ); if ( i == 1) { f ( ); e ( ); c ( ); exit ( ); } else { if ( j == 1 ) p ( ); else q ( ); d ( ) ; if ( k == 1 ) r ( ); else s ( ); } } 1 a  f  e  c 2 a  b  c  d  e  f 3 a  c  b  d  e  f 4 a  b  c  d  f  e 5 a  c  b  d  f  e a d c e b f a  b  d  e a  b  d  f a  c  d  e a  c  d  f (b) Static program traces (c) Frequent sequential patterns support, 4/5 (d) Frequent partial order R (a) Example code Frequent SubSeq/Partial Order Consider APIs a, b, c, d, e, and f Apiator [ Acharya et al. FSE 07 ]

37. 1 a  f  e  c 2 a  b  c  d  e  f 3 a  c  b  d  e  f 4 a  b  c  d  f  e 5 a  c  b  d  f  e a d c e b f a  b  d  e a  b  d  f a  c  d  e a  c  d  f (b) Static program traces (c) Frequent sequential patterns support, 4/5 (d) Frequent partial order R (a) Example code #include <abcdef.h> void p ( ) { b ( ) ; c ( ) ; } void q ( ) { c ( ) ; b ( ) ; } void r ( ) { e ( ) ; f ( ) ; } void s ( ) { f ( ) ; e ( ) ; } int main ( ) { int i, j, k; a ( ); if ( i == 1) { f ( ); e ( ); c ( ); exit ( ); } else { if ( j == 1 ) p ( ); else q ( ); d ( ) ; if ( k == 1 ) r ( ); else s ( ); } } Frequent SubSeq/Partial Order Apiator [ Acharya et al. FSE 07 ] MAPO [Xie&Pei MSR 05] MAPO Apiator

48. Strathcona Strathcona [Holmes et al. 05]

49. XSnippet XSnippet [ Sahavechaphan&Claypool OOPSLA 06 ]

50. PARSEWeb PARSEWeb [Thummalapenta&Xie ASE 07]

51. PARSEWeb http://news.google.com/

52. MAPO (new) MAPO [Xie&Pei MSR 06]

53. MAPO (new) MAPO [Xie&Pei MSR 06]

56. Assieme Assieme [Hoffmann et al. UIST 07] links to pages with snippets group pages with similar snippets links to required libraries

68. Comparison with Other Tools Percentage of tasks successfully completed by PARSEWeb, Prospector, and XSnippet

69. Significance of Internal Techniques *Legend: Method inline: Method inlining Post Process: Sequence Post Processor Query Split: Query Splitter

71. Mining Partial Orders Consider APIs a, b, c, d, e, and f Partial Order Partial Order with Transitive Reduction The extracted scenarios are fed to a partial order miner The partial order miner mines frequent closed partial order Closed Partial Order Apiator [ Acharya et al. FSE 07 ]

72. XOpenDisplay XCloseDisplay XCreateWindow XGetWindowAttributes XCreateGC XSetForeground XGetBackground XMapWindow XChageWindowAttributes XMapWindow XSelectInput XGetAtomName XFreeGC XNextEvent Example Partial Order A usage scenario around XOpenDisplay API as a partial order. Specifications are shown with dotted lines. Apiator [ Acharya et al. FSE 07 ]