T# @ Agile Tour Montreal 2009 En

Ludovic Dubois
ldubois@prettyOBJECTS.com

www.prettyOBJECTS.com

©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009

Pretty Objects – Visual T#
 Survey
 A good test is…
 Conclusion


• Mission: help OO developers (1993)
• Three service offers:
– Training(1993)
o In partnership with CRIM
o Trained over 10,000 participants
– Consultation – Coaching (1994)
o Object oriented – automated unit tests
o Speciality: Microsoft.NET
– Development of development tools(1995)
o POCMock: VS addin for whitebox testing (2003)
o T#: .NET language to simplify writing unit tests (2009)


• 2005: Idea to simplify writing tests, in particularly when
using mocks
• 2005: T# documentation
• 2006: Beginning of development
• 2007: C# compiler done (over 8,000 tests)
Beginning of Visual Studio integration
Complete rewrite of T# documentation
• 2008: T# compiler done (over 12,000 tests),
Visual Studio integration still going!
• 05/2009: T# v1.0 (almost 18,000 tests)
• 09/2009: T# v1.5 (almost 20,000 tests)
• 10/2009: T# v1.6 (over 22.000 tests)

Pretty Objects – Visual T#
Survey
 A good test is…
 Conclusion


• Who knows about unit testing theory?
• Who develops unit tests?
• Who uses NUnit? MSTests? Other?
• Who has written over:
– 100 tests?
– 1,000 tests?
– 10,000 tests?


Survey
A good test is…
 Conclusion


 No test = the worst of tests!
 Unless it is…
 … the test which verifies the contrary of what we want!
 Test = a will to test
– Is the good will to test sufficient?

 Test written after the code
– « Does this test actually tests something? »
 Test written before the code
– Think about the method’s objective prior to creating it

 Never fix a bug…
 Write a test which proves the existence of the bug
 Correct the code to make the test pass
 Verify that the bug has disappeared from the original
context

 Never change a test…
 Unless the business objective or need has changed
 In case of a new bug: duplicate the test and change the
copy


 Generalize whenever possible
– Define general contexts for your tests
– Separate tests in many classes (because the contexts
are different):
o Constructors
o Class declarations (statics)
o Instance declarations (non-statics)
 The context creates an instance

 Limit code length
– Create high-level Assert… methods
Writing, maintaining and refactoring becomes much
easier

protected void AssertLinked( Invoice invoice,
params object[] products )
{
Assert.IsTrue(products.Length % 2 == 0, "Bad test!");
Assert.AreEquals( products.Count / 2, invoice.Count );
for( int i = 0; i < invoice.Count; i += 2 )
{
Item item = invoice[ i / 2 ];
Assert.AreSame( products[i], item.Product );
Assert.AreEqual( products[i + 1], item.Quantity );
}
}
[TestMethod]
public void Purchase()
{
…
AssertLinked( invoice, product1, 2, product2, 4 );
}


 Think about the objective prior to coding
it!
– « What does it test? »
o Which declaration? Of which class?

– « Under which situation? »
o States which influences the execution of the tested
declaration?
 An instance’s state
 A class’ state
 Other instances/class’ states
 The environment’s state: files, DB…
o What are the parameters during the invocation?

 Non formalized objectives
– « What does it test? »: name of a method!
– « Under which situation? »: name of a method!
Not easily manageable!
Rests on the developer’s discipline


Formalized objectives
– « What does it test? »: indicate the tested declaration
– « Under which situation? »: make use of criteria
Knowledge of what is being tested and by which tests
Compiler validation
Missing tests notification at compilation
Execution of tests from your tested declaration


[TestClass]
public class ProductTest
{
[TestMethod]
public void PriceChange_SomeRandomValue()
{
…
}

[TestMethod]
public void PriceChange_MinimumValue()
{
…
}
}


testclass for Product
{
test Price set
when MinIncluded.IsAboveMin
{
…
}

test Price set
when MinIncluded.IsMin
{
…
}
}

// The compiler will warn you of a missing
// test for MinIncluded.BelowMinCase


Determine the test cases for the
declaration
– Normal cases (such as IsAboveMin, IsMin)
– Error cases (such as BelowMinCase)

 Détermine the cases for each parameter

 Write a test for each possible combination
– Combine valid cases
– Isolate error cases

 Test coverage
– Technical, not logical!
o Doesn’t indicate missing tests, only code that hasn’t been
used by the tests
o Doesn’t properly cover all cases
 if( M1() && M2() )
with M1() always true, M2() variable

– Must run all the tests to know which ones are missing
– If the coverage isn’t complete: there’s a problem
– If the coverage is complete: it doesn’t prove anything!


 Using tests criteria
– Definition of normal cases (IsAboveMin, IsMin)
– Definition of error cases (BelowMinCase)
– Logical definitions, not technical ones

 Write a test per possible combination
– Define case expressions for each test
o Use the following operators &&, ||, => and ()
The compiler will warn you about the missing tests


criteria MinIncludedCriteria
{
IsAboveMin,
IsMin,
[Error]
BelowMinCase
}

criteria NotifyPropertyChangedCriteria
{
HasNoSubscriber,
HasSubscribersSetSameValue,
HasSubscribersSetOtherValue
}


test Price set
when MinIncluded.IsAboveMin &&
NotifyPropertyChanged.HasNoSubscriber
{ … }

test Price set
NotifyPropertyChanged.HasSubscribersSetOtherValue
{ … }

test Price set
NotifyPropertyChanged.HasSubscribersSetSameValue
{ … }

test Price set
when MinIncluded.BelowMinCase
{ … }


 A couple of lines of code
– « What does it test? » « Under which situation? »
– « For what outcome? »
 Separate it in 3 distinctive parts:
– Execution: « What does it test? »
o Write a single line of code
– Preparation: « Under which situation? »
o Write the most necessary code in the tests’ context
o Validate the situation at the end of each preparation
– Verification: « For what outcome? »
o Correspond the assertions before and after the execution
The effect of the tests code becomes evident

 Context in two methods
– A method before running the test
– Another one after running the test
Instance variables instead of local ones
Impossible to protect the code
o No try…catch or finally blocks.
Impossible to repeat tests


 Context defined in only one method
– runtest keyword used like a placeholder for the test
Local variables instead of instance variables
Possibility to protect the code
o try…catch or finally around runtest
Easy repetition of tests
o Use of multiple runtest
o Loop over runtest
o Define a new situation before each call to runtest


[TestClass]
public class ProductTest
{
private Product p;

[TestInitialize]
public void Setup()
{
p = new Product( "T#", 123 );
}

…


testclass for Product
{
private Product p;

testcontext
{
p = new Product( "T#", 123 );
runtest;
}

…


 3 parts well distinctive?
– With comments in the code
Discipline of each developer

Assertions by methods
– Lots of different Assert…
– Not always natural
o Expected value as first parameter
o No automatic casting of types (MSTests)


 Keyword: runtest
– Execution: identified by the runtest keyword
– Preparation: before runtest
– Verification: after runtest
Validation done by the compiler
 Keyword: assert
– Followed by a boolean expression
– Whichever order of expected and actual values
– Automatically generated error messages
More natural to use


[TestMethod]
public void PriceChange_SomeValue_NoSubscribers()
{
// Preparation
Assert.AreNotEqual( 234.0, p.Price );

// Execution
p.Price = 234;

// Verification
Assert.AreEqual( 234.0, p.Price );
}


test Price set
NotifyPropertyChanged.HasNoSubscriber
{
assert p.Price != 234;

runtest p.Price = 234;

assert p.Price == 234;
}


 Verify all wanted effects
– Each test has assertions for every wanted effect
– Relative vs absolute assertions
o Relative: « 1 more than before »
 Clearly shows the change of value
 More code, more complex, less maintenance
o Absolute: « now equals to 1 »
 Ensures fixed values
 Less code, more simpler, more maintenance

 Verify all unwanted effects
– Each test has assertions for every unwanted effects


 Verify all wanted effects
– One Assert. … for each wanted effect
– More complex scenarios: events
o TONS of code
Objective of the test is flooded in the code
– Sometimes not well tests: exceptions
o Makes use of attributes
Test passes if any line of code raises an exception!
Impossible to add further assertions


 Verify all wanted effects
– One assert keyword used for any verification
o assert <expression booléenne>
o assert [!] raised <event>
 1 single line of code!
o assert thrown <exception>
 1 single line of code!
 Assertion for the « Execution » part only
 Other possible assertions
o Automatic generated message (more detailed)


[TestMethod]
public void PriceChange_SaveValue_HasSubscriber()
{
// Preparation
p.PropertyChanged += PropChanged;
eventRaised = false;

// Execution
p.Price = p.Price;

// Verification
Assert.IsFalse( eventRaised );
}
private bool eventRaised;
private void PropChanged( object sender, EventArgs e )
{
eventRaised = true;
}


test Price set
NotifyPropertyChanged.HasSubscriberSetSameValue
{
runtest p.Price = p.Price;

assert !raised p.PropertyChanged;
}


[TestMethod]
public void PriceChange_SomeValue_HasSubscriber()
{
// Preparation
p.PropertyChanged += PropChanged;
eventRaised = false;
Assert.AreNotEqual( 234.0, p.Price );

// Execution
p.Price = 234;

// Verification
Assert.IsTrue( eventRaised );
Assert.AreEqual( 234.0, p.Price );
}


test Price set
NotifyPropertyChanged.HasSubscriberSetOtherValue
{


assert raised p.PropertyChanged;
assert p.Price == 234;
}


[ExpectedException(typeof(ArgumentOutOfRangeException)]
public void PriceChange_ImpossibleValue()
{
// Preparation
// If the preparation code throws an expected
// exception, the test passes!!!
Assert.AreNotEqual( -1.0, p.Price );

// Execution
p.Price = -1;

// Verification
// Instruction never reached (else: problem!)
// Impossible to add further assertions
// If reached and throws expected exception,
// the test passes!!!
}


test Price set
{
// If the preparation code throws the expected
//exception, the test fails
assert p.Price != -1;

runtest p.Price = -1;

assert thrown ArgumentOutOfRangeException;
// Add as many assertions as needed
// If the verification code throws the expected
// exception, the test fails
}


 Verify all wanted effects…cont’d…
– Relative assertions in 1 single line of code
o assert changed <expression>
 assert changed collection.Count++
 assert changed collection.Count += 1
 assert changed collection.Count =
collection.Count + 1


[TestClass]
public class InvoiceTest
{
private Invoice invoice;
private Product product;
…
[TestMethod]
public void Add_New_MultipleProducts_Test()
{
// Preparation
int totalBefore = invoice.Total;
Assert.AreEqual( 0, invoice.Count );
// Execution
invoice.Add( product, 2 );
// Verification
Assert.AreEqual( 1, invoice.Count );
Assert.AreEqual( totalBefore + 2 * product.Price,
invoice.Total );
}

testclass for Invoice
{
private Invoice invoice;
private Product product;
…
test Add( Product p, int qty )
when p: ValidReference.IsNotNull,
qty: MinIncluded.IsAboveMin,
Collection.IsEmpty &&
CollectionItem.IsNotInCollection
{
assert invoice.Count == 0;

runtest invoice.Add( product, 2 );

assert changed invoice.Count++;
assert changed invoice.Total += 2 * product.Price;
}


 Verify what you do not want
Lots of code…blurry
Code in « Preparation » to save
Code in « Verification » to compare
Objective of the test is flooded in the code


 Verify what you do not want
– Always the same keyword: assert !changed
o For only public properties
assert !changed obj.*;
o For all variables
assert !changed obj.-*;
– No changes except…: …except…
o Some properties (variables)
assert !changed obj.* except Property;
o An absolute assertion
assert !… except Property == 1;
o A relative assertion
assert !… except changed Property++;

[TestMethod]
public void PriceChange_ImpossibleValue()
{
// Preparation
double actualPrice = p.Price;
Assert.AreNotEqual( -1.0, actualPrice );
// Execution
try
{
p.Price = -1;
// Verification
Assert.Fail("ArgumentOutOfRangeException
expected!");
}
catch( ArgumentOutOfRangeException )
{
Assert.AreEqual( actualPrice, p.Price );
}
}

test Price set
{
assert p.Price != -1;

runtest p.Price = -1;

assert thrown ArgumentOutOfRangeException;
assert !changed p.*;
// Verifies the constancy of Price, Number,
// Description, etc.
}


test Price set
NotifyPropertyChanged.HasSubscriberSetOtherValue
{


assert raised p.PropertyChanged;
assert !changed p.*
except Price == 234;
}


 Prefer « black box » tests
– Test only available services
– Not the internal organization of classes
 Not always thinkable
– Simple invocation of the service, but complex to handle
o A compiler has only one method:
Compile(sourceFiles, destinationFiles)
– Test hidden parts of higher level
Changing code so that it can be testable
is…detestable!


• Test hidden parts
 Don’t test them
o Doesn’t correspond to our need!
o Who is going to use those parts? For what outcome?
 Use meta-programming in the test
o typeof(…).GetMethod(…).Invoke(…)
Blurs the test code
Harder to read and understand
Error at runtime instead of compile time
– Hide the meta inside extension methods in the test
module
o Less bad solution!


• Test hidden parts
 .- operator
o As simple as the . operator
o Access any declaration, even private ones
o Access any type, even internal ones
o Doesn’t change the tested code
o More effective than reflection


 Pass the tests one by one
– If you’re writing many tests, ignore all but one
– Then make another one pass…

 Instantly find tests regarding a declaration

 Easily find what’s going on when a test
fails


 Find the tests
– 1 class to test = many test classes
– 1 declaration to test = many test methods

 Find out what’s wrong
– 3 states for a test:
o Ignored: temporarily on hold
o Correct: it passes, all is good
 Uh…really?
o Failed: the code is not good
 Which code? The tested code? The test code?


 Find the tests
– Organize the tests according to the tested code
– Execute the tests from the tested code
 Find out what’s wrong
– 4 states for a test:
o Ignored: temporarily on hold
o Correct: it passes, all is good
o Invalid: the test code is not good (for sure)
o Failed: the tested code is not good (probable)


Survey
A good test is…
Conclusion


 Visual T# is free!

http://forum.prettyobjects.com/forum2-t-download.aspx
http://fr.wikipedia.org/wiki/Visual_T_Sharp

 What are you waiting for to write good
tests?


Thanks! :)
Pretty Objects Computers inc.
550 Sherbrooke Ouest, suite 100
Montreal, H3A 1B9
TÉL.: 514-840-1253
FAX: 514-840-1244
www.prettyOBJECTS.com
info@prettyobjects.com

T# @ Agile Tour Montreal 2009 En

Recommended

Recommended

More Related Content

Similar to T# @ Agile Tour Montreal 2009 En

Similar to T# @ Agile Tour Montreal 2009 En (20)

T# @ Agile Tour Montreal 2009 En