Presentation at the 9th International Conference on the Principles and Practice of Programming in Java.

1. The SOUL Tool Suite for
Querying Programs
in Symbiosis with Eclipse
Coen De Roover, Carlos Noguera, Andy Kellens, Viviane Jonckers
Principles and Practice of Programming in Java (PPPJ11)

2. Querying Programs?
... commonly implemented by tool builders
{
identify code with structural
characteristics of interest control flow
data flow
e.g. plugin
find and highlight
idiomatic uses of reflection

3. Logic Program Querying
... specify characteristics through logic conditions
... leave operational search to logic evaluator
quantify over
reified program
representation
AST, CFG, PTA

4. Logic Program Querying
... specify characteristics through logic conditions
... leave operational search to logic evaluator
variable binding
reified AST node
single result
variable bindings

5. Logic Program Querying
... many instantiations, yet seldom exploited by tools
eepWeaver
ASTLog D
GenT
JTran L
sform
TyRuBa er
PQ Co
L deQ Lo
JT ues gEn
adoption hurdles?
L t
Soul
H1: queries difficult to specify LogicA
H2: results difficult to exploit J

6. H1 Illustrated intricate details
program representation + reification
... specifying queries is difficult
implementation variants
data flow + control flow

7. H2 Illustrated deifying a variable binding
finding corresponding element
... exploiting results is difficult
code changes
representation vs reified representation
prevalent reification of AST
= transcription to logic facts
e.g. node(a,<node(b,<>),
a node(c,<node(d,<>)])).
b c variable
e.g. node(a,<b,c>). binding
d node(b,<>).
corresponding node(c,<d>).
node in IDE node(d,<>).

8. The SOUL Tool Suite
... an architectural overview
logic predicates
reify AST node relations
plugin framework
on-demand
+ scheduled JDT quantifies over Cava library
Barista interacts with
Soul evaluator
UI for
Annotations
Barista UI
prototyping queries
Smart
case
study
Eclipse platform

9. deifying a variable binding
Addressing H2 finding corresponding element
... exploiting results is difficult
code changes
representation vs reified representation

10. Look Ma, No Fact Base!
... reify(ASTNode)=ASTNode
objects as
through linguistic symbiosis first-class
logic values
Cava library MethodDeclaration
quantifies over if ?m methodDeclarationHasName: ?n
SimpleName
JDT

11. Look Ma, No Fact Base!
... reify(ASTNode)=ASTNode
expressions
through linguistic symbiosis in logic
queries

12. Look Ma, No Fact Base!
al
... reify(ASTNode)=ASTNode
vi
deifying a variable binding
t ri
finding corresponding element
addresses H2
code changes
representation vs reified representation

13. intricate details
Addressing H1 program representation + reification
... specifying queries is difficult
implementation variants
data flow + control flow

14. Addressing H1 developer exemplifies
implementation of characteristics
... example-driven specification
evaluator finds implicit
implementation variants
e.g. if jtExpression(?exp){Class.forName(?string)}
but if jtClassDeclaration(?class){
also class ?className {
private ?fieldDeclaration ?fieldName;
?modList ?returnType ?methodName(?parameterList) {
return ?fieldName;
}
}
}
or if jtMethodDeclaration(?m){
?modList ?returnType ?methodName(?paramList) {
?class := Class.forName(?string);
?instance := ?class.newInstance();
(?type) ?casted;
}
},
?casted mustAliasWith: ?instance

15. Addressing H1
... demystified
lenient matching of code templates
varies from AST-based to flow-based
if jtMethodDeclaration(?m){
public static void main(String[] args) {
?scanner := new Scanner(?argList);
?scanner.close();
?scanner.next();
}
}
domain-specific unification procedure
occurrences of variable can be bound to implementation variants
an ASTNode an ASTNode identical
a Type a Type same or co-variant return types
a MethodInvocation a MethodDeclaration may invoke
an Expression an Expression may-alias

16. Case Study co-evolve code and annotations
embed usage constraints
... revisiting Smart Annotations
all annotated
should be result
@Target(ElementType.METHOD)
@TargetVariable("method")
public @interface Getter {
@Necessary
public static final String NAMING_CONVENTION
= "?method methodDeclarationHasName: {get*}";
@Sufficient
public static final String STRUCTURAL
= "jtClassDeclaration(?class){
class ?className {
private ?fieldDeclaration ?fieldName;
?modList ?returnType ?methodName(?parameterList){
return ?fieldName;
}
}
}";
}
all results should
be annotated
A. Kellens, C. Noguera, K. De Schutter, C. De Roover, and T. D’Hondt.
Co-evolving annotations and source code through smart annotations.
Conference on Software Maintenance and Re-engineering (CSMR10)

17. 37 }
29 for ( Object res : requestor . getResult ()) {
30
31
32
33
Case Study
Figure 7. Gathering constraints using Eclipse search API
IAnnotation ann = ( IAnnotation ) res ;
IField field = ( IField ) ann .
getAncestor ( IJavaModel . FIELD );
IType annotation = ( IType ) ann .
using logic program queries
to gather usage constraints
34 getAncestor ( IJavaModel . TYPE );
35
36
... revisiting Smart Annotations
String rule = ( String ) field . getConstant ();
// handle c o n s t r a i n t
37 } of fields annotated with
@neccesary or @sufficient within
Figure 7. Gathering constraints using Eclipse search API
Figure 7. Gathering constraints using Eclipse search API declaration:
an annotation type
- name
- initializer
- annotation type declaration
1 IEvaluator eval = barista
2 . query (
3 " if ? type a n n o t a t i o n T y p e D e c l a r a t i o n H a s N a m e : ? annotation , "
4 + " ? type de fi n es Va ri a bl e : ? field , "
5 + " ? field v a r i a b l e D e c l a r a t i o n F r a g m e n t H a s I n i t i a l i z e r : ? rule , "
6 + " ? field v a r i a b l e D e c l a r a t i o n F r a g m e n t H a s N a m e : ? rulename , "
7 + " ? field v a r i a b l e H a s A n n o t a t i o n : ? named : ? annotationType , "
8 + " or (
9 equals (? annotationType ,{ Necessary }) ,
10 equals (? annotationType ,{ Sufficient })) " ,
11 javaProject , " Evaluator " , " JavaEclipse " );
1 IEvaluator eval = barista result variables
2 . query (
13 IResults iresults = eval . getAllResults (); bound to
3 " if ? type a n n o t a t i o n T y p e D e c l a r a t i o n H a s N a m e : ? annotation , "
15+ " ? type de, List < Object e : ? field , = iresults . toMap ();
4 Map < String fi n es Va ri a bl > > results "
annotation type
5 + " ? field v a r i a b l e D e c l a r a t i o n F r a g m e n t H a s I n i t i a l i z e r : ? rule , " declaration
17+ " ? field i a= i0; l e D e c l a r a t i o . getSize (); s N a m e :
6 for ( int v r a b i < iresults n F r a g m e n t H a i ++) { ? rulename , "
18+ " ? field vannType H a s A n n o t a t ) o results . get: " ? ? annotationType , "
7 ASTNode a r i a b l e = ( ASTNode i n : ? named ( type " ). get ( i );
AST node
19+ " or (
8 String type = results . get ( " ? ann otationT ype " ). get ( i ). toString ();
20
9 equals (? annotationType ,{ Necessaryrule " ). get ( i ). toString ();
String ruleString = results . get ( " ? }) ,
21
10 // Handle c o n s t r a i n t
equals (? annotationType ,{ Sufficient })) " ,
11 }
22 javaProject , " Evaluator " , " JavaEclipse " );

18. 1
2
3
Case Study
SearchPattern allNecesary = SearchPattern . createPattern (
I J a v a S e a r c h C o n s t a n t s . ANNOTATION_TYPE ,
using org.eclipse.jdt.core.search
to gather usage constraints
" be . ac . vub . s m a r t _ a n n o t a t i o n s _ l i b r a r y . Necessary " ,
4 I J a v a S e a r c h C o n s t a n t s . ANNOTATION_TYPE_REFERENCE ,
5
... revisiting Smart Annotations
SearchPattern . R_EXACT_MATCH );
7 SearchPattern allSufficient = SearchPattern . createPattern (
" be . ac . vub . s m a r t _ a n n o t a t i o n s _ l i b r a r y . Sufficient " ,
8
19SearchPatternC o n s t a n t s . ANNOTATION_TYPE ,
I J a v a S e a r c h allNecesary = SearchPattern . createPattern ( search patterns for
2 I" be . S e a r c h C o nm a r t _ a n n o t a t i o n s _ l i b r a r y . Necessary " ,
J a v a ac . vub . s s t a n t s . ANNOTATION_TYPE_REFERENCE ,
10
3
11 I J a v a S e a r c h C o n s t a n t s . ANNOTATION_TYPE ,
SearchPattern . R_EXACT_MATCH ); @necessary and @sufficient
4 I J a v a S e a r c h C o n s t a n t s . ANNOTATION_TYPE_REFERENCE ,
5
13
14
SearchPattern . R_EXACT_MATCH );
SearchPattern allConditions = SearchPattern .
c re at e Or Pa tt e rn ( allNecesary , allSufficient );
inter-node relations difficult
7 SearchPattern allSufficient = SearchPattern . createPattern (
16 I J" be . ac r c h S c o p e r t _ a n n o t a t i o n= _SearchEngine
8 a v a S e a . vub . s m a projectScope s l i b r a r y . Sufficient " ,
17
9 I cre S archCons hScope (
.J a v a t e J a v a S e a r ct a n t s . ANNOTATION_TYPE ,
18
10 new IJavaElement [] { jProject } , false );
I J a v a S e a r c h C o n s t a n t s . ANNOTATION_TYPE_REFERENCE ,
11 SearchPattern . R_EXACT_MATCH );
20 S i m p l e S e a r c h R e q u e s t o r requestor = new S i m p l e S e a r c h R e q u e s t o r ();
13 SearchPattern allConditions = SearchPattern .
22 SearchEngine engine = new SearchEngine ();
14 c re at e Or Pa tt e rn ( allNecesary , allSufficient ); limited pattern composition
24 engine . search ( allConditions ,
16 I J a v a S e a r c h S c o p e projectScope = SearchEngine
25 new S e a r c h P a r t i c i p a n t []
17
26
. c r e a t e J a v a S e a r c h S c o p e ( u l t S e a r c h P a r t i c i p a n t ()
{ SearchEngine . g e t D e f a },
18
27
new IJavaElement [] { jProject } , false );
projectScope , requestor , null );
20 S i m p l e S e a r c hres u e s t o r requestor = new S i m p l e S e a r c h R e q u e s t o r ();
29 for ( Object R e q : requestor . getResult ()) {
30 IAnnotation ann = ( IAnnotation ) res ;
22 SearchEngine engine = new ann .
31 IField field = ( IField ) SearchEngine ();
32 getAncestor ( IJavaModel . FIELD );
24 engine . search ( allConditions , ann .
33 IType annotation = ( IType ) filter out non-fields
25
34
35
26
new S e a r c h P agetAncestor ( IJavaModel . TYPE );
r t i c i p a n t []
String rule = ( String D e f a u l t. getConstantc i p a n t ()
{ SearchEngine . g e t ) field S e a r c h P a r t i (); }, gather results manually
36 projectScope o n s t r a i n t
27 // handle c , requestor , null );
37 }
29 for ( Object res : requestor . getResult ()) {
30
Figure 7. Gathering constraints using Eclipse search API
IAnnotation ann = ( IAnnotation ) res ;
31 IField field = ( IField ) ann .
32 getAncestor ( IJavaModel . FIELD ); manually navigate
33 IType annotation = ( IType ) ann .
34 getAncestor ( IJavaModel . TYPE ); upwards to field declaration
35 String rule = ( String ) field . getConstant ();
36 // handle c o n s t r a i n t
37 }

19. Conclusions
... tool suite for querying Java programs
SOUL Cava Barista
logic program predicate library Eclipse plugin
query language
H1: queries difficult to specify
address adoption hurdles
H2: results difficult to exploit
example- reification extension points
driven through +
specification symbiosis query prototyping

20. ent classes change together. The predicate wasChanged/1 all methods considered as a zombie method. T
?end will be bound to the first version w
Want to know more?
(lines 7 and 9) check if its argument got changed in the
current version of the path.
• Lines 10–14 consume one or more versions in which
method gets called again. Notice that we also
for the methods ?invoker and ?newInvoker
example-driven pattern detection@ICSM changed
classes ?classA and ?classB are consistently UML templatesinformationofregarding the
not only provide
ascallers such methods
but also indicates all
queries@ICSM
1
together (lines 10–12) or not changed at all (lines 13 and
if jtClassDeclaration(?subjectClass){
2 14). ?subjectName {
class
V. D ISCUSSION AND F UTURE W
?mod1List ?t1 ?observers = ?init;
• Finally, the path expression succeeds if we reach the end
3
4 public ?t2 ?addObserver( ?observerType ?observer ) {
of a?observers .?add( ?observer ); 15).
path (terminal on line a) Performance of A BSINTHE: As th
5
}
on introducing the A BSINTHE tool and in
Notice that line 12 binds the current version at that point suitability of quantified regular path expre
6
7 public ?t3 ?removeObserver( ?observerType ?otherObserver) {
8 in time to the logic variable ?changedVersion. By doing this, the history of a system, we have not perf
?observers .?remove(?otherObserver);
9
10
solutions to this query will contain bindings for all versions benchmarking of our tool.
}
?mod2List ?t4 ?notifyObservers(?param1List) {
11 in which ?observers ;
the two classes changed together. However, in order to obtain indications of
?observer . ?update (?argList);
of our tool, we have applied all the quer
12
13 }
14 C. Temporal Bad smell: Zombie Code
} Section IV to the last two years of developm
},
S OUL program query language. Our reposito
15
16
17
1 if
jtClassDeclaration(?observerClass){
history querying @WCRE
class ?observerName { download@soft.vub.ac.be/SOUL/ vers
of this part of the history includes 179
18
2 ?start isOrigin,
?mod3List ?t5 ?update (?argList) {} each version contains 244 classes and 2369 m
3 e(
19 } per version). As our representation only st
4 (true)*<,
},
20
have changed (see Section III-A), it only con
5 and(?m isMethodWithName:?name inClass: ?c,
21
6
jtExpression(?register){ ?subject.?addObserver( ?lapsed ) },
?invoker methodSendsMessage:?name), 43Mb of memory.
22 not(jtExpression(?unregister){ ?subject.?removeObserver( ?lapsed ) }),
7 (and(?m isMethodWithName:?name inClass: ?c, Early results show that the tool performs
jtExpression(?alloc){ ?lapsed := new ?observerName (?argList) }
23
8 (? methodSendsMessage:?name) not))+<, example, the query in Section IV-B1 (Figure
9 and(?m isMethodWithName:?name inClass: ?c,
classes that the first author of the paper has b
10 ?newInvoker methodSendsMessage:?name))
gure 6: Example-based specification for the Observer design pattern [GHJV94] (lines 1–that and the lapsed listener by other develope
and 21) were also modified
matches: ?path
tfall [Liv05] in its implementation (lines 1–25).
11
12 start: ?start Solutions for most of the other queries in this
13 end: ?end within one minute.
etecting Lapsed Listeners
Note that the time it takes to evaluate a q

21. One more thing
;)
inspector on
Smalltalk proxy
for Java object