1. Automatically
RepairingTest Cases
for Evolving Method Declarations
Mehdi Mirzaaghaei
Fabrizio Pastore
Mauro Pezzè
Università
della
Svizzera
italiana
http://swiss-landmarks.ch/panos/Lugano9.jpg

2. Software evolves

3. public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance
};
}
Software evolves

4. testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance
};
}
Software evolves

5. testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance
};
}
Software evolves

6. public class .....
public class AccountUtil {
private double dailyInterestRate = 0.00005;
private int interestTerm=365;
public double interest(BankAccount account
return account.getBalance()
*dailyInterestRate*interestTerm;
}
...
public class AccountFactory {
public static BankAccount create( AccountContext ctx, boolean special ){
BankAccount account = new BankAcccount( ctx.amount );
if ( special ){
account.setInterestRate(0.0001);
}
...
public class .....
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance
};
}
Software evolves

7. public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance
};
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
};
}
Original classes are modified

8. public class .....
public class .....
Test cases need maintenance
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance
};
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
};
}
Compilation error:
Method deposit(Money) in the type
BankAccount is not applicable for
the arguments (int)

9. changes in method declarations:
23% changes
that impact on compilation

10. changes in method declarations:
23% changes
that impact on compilation
> 80% in maintenance releases

11. changes in method declarations:
23% changes
that impact on compilation
> 80% in maintenance releases
Automatic Repair to Save Effort

12. AutomaticTest Repair

13. [Memon et. al. 2008, Grechanik 2009 ]
Repair GUI tests
Well suited only for GUI tests
AutomaticTest Repair

14. [Memon et. al. 2008, Grechanik 2009 ]
Repair GUI tests Repair Oracles
[B. Daniel et al 2010]
testInterest(){
...
assertEquals( 50, result );
}
FAILURE: expected 50, found: 40
assertEquals( 40, result );
Well suited only for GUI tests Focus on oracles only
AutomaticTest Repair

15. [Memon et. al. 2008, Grechanik 2009 ]
Repair GUI tests
Refactoring Techniques
[ReBa, Eclipse]
Repair Oracles
[B. Daniel et al 2010]
testInterest(){
...
assertEquals( 50, result );
}
FAILURE: expected 50, found: 40
assertEquals( 40, result );
//calculate one year interest
result = account.interest();
result = account.interest( 0 );
Well suited only for GUI tests Focus on oracles only
Prevent only some compilation errors
AutomaticTest Repair

16. TestCareAssistant
repairs test case compilation errors
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes

17. TestCareAssistant
repairs test case compilation errors
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes

18. TestCareAssistant
repairs test case compilation errors
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes

19. TestCareAssistant
repairs test case compilation errors
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
Money amount = new Money(500);
account.deposit(amount);
assertEquals(500, account.getBalance());
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes

20. TestCareAssistant
repairs test case compilation errors
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
Money amount = new Money(500);
account.deposit(amount);
assertEquals(500, account.getBalance());
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes

21. TestCareAssistant
repairs test case compilation errors
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
Money amount = new Money(500);
account.deposit(amount);
assertEquals(500, account.getBalance());
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes

22. TestCareAssistant
repairs test case compilation errors
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
Money amount = new Money(500);
account.deposit(amount);
assertEquals(500, account.getBalance());
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
• Parameter add
• Parameter type change • Return type change
• Parameter remove
caused by method declaration changes

23. Repair parameter type change
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.CentsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

24. Which parameter to initialize?
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.CentsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

25. Which parameter to initialize?
Use code diff to identify the parameter1
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.CentsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

26. How to initialize a complex object?
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

27. How to initialize a complex object?
Find first uses of parameter fields inVersion12
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

28. How to determine fields values?
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

29. How to determine fields values?
Diff to identify corresponding variable inVersion 03
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

30. What was the original value?
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.CentsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

31. What was the original value?
Analyze the def-use chain of the variable back to
the definition in the test4
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.CentsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

32. What was the original value?
Analyze the def-use chain of the variable back to
the definition in the test4
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.CentsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

33. How can we repair the test?
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

34. How can we repair the test?
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

35. How can we repair the test?
Instantiate the new object,
use original values to initialize fields5
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

36. How can we repair the test?
Instantiate the new object,
use original values to initialize fields5
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
public class BankAccount {
private int balance;
public void deposit(int cents){
balance += cents;
}
public int getBalance(){
return balance;
}
}
public class BankAccount {
private int balance;
public void deposit(Money money){
balance += money.centsValue;
}
public int getBalance(){
return balance;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals( 500, account.getBalance());
}
Version 0 Version 1

37. To what extentTestCareAssistant can
fix test cases automatically?

38. To what extentTestCareAssistant can
fix test cases automatically?
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters
removed, 4 return types changed

39. To what extentTestCareAssistant can
fix test cases automatically?
Results Total Repaired %
Test Cases 22 16 72
Compilation Errors 24 18 75
Values Initialized 36 29 80
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters
removed, 4 return types changed

40. To what extentTestCareAssistant can
fix test cases automatically?
Results Total Repaired %
Test Cases 22 16 72
Compilation Errors 24 18 75
Values Initialized 36 29 80
• Repair 22 test cases of 6 open source systems
• Continuum, Geronimo, xml-security, PMD, POI, Shindig
• 24 compilation errors caused by different changes
• 9 parameter types changed, 8 parameters added, 3 parameters
removed, 4 return types changed

41. To what extentTestCareAssistant can
fix test cases automatically?
• Static data flow analysis not always effective
• Use of complex data structures, e.g. hash tables
• Changes in method logic
• Changes in interfaces
Results Total Repaired %
Test Cases 22 16 72
Compilation Errors 24 18 75

42. Conclusions
public class BankAccount {
public void deposit(int cents){
balance += cents;
}
}
public class BankAccount {
public void deposit(Money money){
balance += money.CentsValue;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());

43. Conclusions
public class BankAccount {
public void deposit(int cents){
balance += cents;
}
}
public class BankAccount {
public void deposit(Money money){
balance += money.CentsValue;
}
}
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
Money amount = new Money(500);
account.deposit(amount);
assertEquals(500, account.getBalance());

44. Conclusions
public class BankAccount {
public void deposit(int cents){
balance += cents;
}
}
public class BankAccount {
public void deposit(Money money){
balance += money.CentsValue;
}
}
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
Money amount = new Money(500);
account.deposit(amount);
assertEquals(500, account.getBalance());
TestCareAssistant

45. Future Work
CopticChronology EthiopicChronology
Chronology
+ withUTC()
TestCopticUTC(){
....
assertEquals("AM",
copticChronology.withUTC() );
}

46. Future Work
CopticChronology EthiopicChronology
Chronology
+ withUTC()
TestCopticUTC(){
....
assertEquals("AM",
copticChronology.withUTC() );
}
TestEthiopicUTC(){
....
assertEquals("EE",
ethiopicChronology.withUTC()
}
CopticChronology EthiopicChronology
Chronology
+ withUTC()

47. Future Work
CopticChronology EthiopicChronology
Chronology
+ withUTC()
TestCopticUTC(){
....
assertEquals("AM",
copticChronology.withUTC() );
}
TestEthiopicUTC(){
....
assertEquals("EE",
ethiopicChronology.withUTC()
}
CopticChronology EthiopicChronology
Chronology
+ withUTC()

48. Future Work
CopticChronology EthiopicChronology
Chronology
+ withUTC()
TestCopticUTC(){
....
assertEquals("AM",
copticChronology.withUTC() );
}
TestEthiopicUTC(){
....
assertEquals("EE",
ethiopicChronology.withUTC()
}
CopticChronology EthiopicChronology
Chronology
+ withUTC()
AutomateTest Generation by Identifying and
ImplementingTest Adaptation Patterns

49. Questions?
public class BankAccount {
public void deposit(int cents){
balance += cents;
}
}
public class BankAccount {
public void deposit(Money money){
balance += money.CentsValue;
}
}
Replace:
int amount = 500;
with:
Money amount = new Money(500);
account.deposit(amount);
testDeposit(){
BankAccount account = new BankAccount();
int amount = 500;
account.deposit(amount);
assertEquals(500, account.getBalance());
testDeposit(){
BankAccount account = new BankAccount();
Money amount = new Money(500);
account.deposit(amount);
assertEquals(500, account.getBalance());
TestCareAssistant