Smolder And Buildbot

What is Smolder?
●
Will it automate my build/test cycle?
– No.
●
Will it run my test suite?
– No.
●
Will it checkout my code from svn, run my build
and then run my test suite?
– Hello!

Smolder takes the information about a test run and
makes it pretty.

Michael Peters, Plus Three LP (http://plusthree.com)

History of Smolder
●
Plus Three, LP
●
Long running test suites for multiple projects
– 5,460 tests in 28 minutes (510K HTML report)
– 18,000+ tests in 50 minutes
●
Stole lots of ideas from the Pugs smoke server.
●
Added lot
– Comments, tags, invalidation
– Pretty graphs
– Multiple projects (public and private)
– Atom data feeds

TAP
●
Test Anything Protocol
●
Created by Test::Building, Test::More, etc
●
Consumed by Test::Harness, prove, etc
1..5
ok 1 – foo is fine
ok 2 – testing bar value
not ok 3 – The object isa Frobnicate
# this is some useful
# comment thing
not ok 4 # SKIP Foo::Bar is not instal
led
not ok 5 # TODO future feature


TAP
●
How do you create it?
use Test::More (tests => 5)
ok($foo, 'foo is fine');
is($bar, 'right', 'testing bar value');
isa_ok($bar, 'Frobnicate');
diag(“this is some usefulncomment thing”);
SKIP: {
  skip('Foo::Bar is not installed' 1);
  ok($foo>bar, 'has a bar');
}
TODO: {
  local $TODO = 'future feature';
  ok($foo>feature, 'some feature');
}


TAP
●
Started in Perl-Land
●
We have since created colonies
– C
– PHP
– Python
– Java
– JavaScript
– Parrot
– Postgres
– .... more


TAP Archives
●
A single file (.tar or .tar.gz)
●
Multiple recorded TAP streams (files)
●
Some meta information (META.yml)
– Stream (test) order
– Start/Stop times
– Future expansion (server info, log files, etc)
●
Easy to create with TAP::Harness::Archive
– prove -a


Continuous Integration Cycle
●
Automated builds
– Simple bash script
#!/bin/bash
if svn update | grep 'Updated to'
then
make build && make test
else
echo quot;No updatesquot;
fi
●
Get fancier
– SmokeRunner::Multi
– Custom harness written with Test::Harness
– BuildBot

Buildbot
●
Most projects don't have multi-platform, multi-
architecture needs
– Anything written in pure Perl
– Most $work applications
●
Some projects do
– Languages (Perl, Python)
– VMs (Parrot)
– System Tools (SCM, etc)


What is Buildbot?
●
Will it automate my build/test cycle?
– Yes.
●
Will it run my test suite?
– Yes.
●
Will it checkout my code from svn, run my build
and then run my test suite?
– Absolutely!


Other Buildbot Spiffyness
●
Master configuration, multiple slaves do the
work
– Lots of architectures and platforms
●
Arbitrary build commands
– Not a language specific system
– Doesn't care if you use make, Build.PL, ant, maven
●
Portable
– It's Python and works in lots of places despite all
the whitespace
●
Very configurable

So Why Do I Need Smolder?
●
Buildbot gives just a single pass/fail for each
run
●
All other details are just dumped to the screen
●
No good way to organize your test runs
●
....


●
It's UGLY!


●
Oooh-la-la!



Buildbot => Smolder



Buildbot => Smolder

=>


●
Information is more organized
– Access to the raw TAP Archives
– Search interface and Graphs
●
Let's you answer questions like
– When was the last time our test suite passed on
platform X?
– Who broke the test suite?
– How many tests have we added for this release?
– How did checkin X affect the speed of the test
suite?


Buildbot Talking to Smolder
●
Remember Buildbot allows arbitrary build
commands
●
So your project just needs a command that can
run the suite, create a TAP archive and send it
to a Smolder server
– prove + smolder_smoke_signal
– make smoke_tests
●
Buildbot doesn't know that it's being cheated
on!


Scary Live Demos!
●
Parrot
●
Smolder


Smolder's Future
●
Test Coverage
●
Tracking individual test assertions
●
Better visual display of diagnostics (new TAP
extensions)
●
More and better ways to search old reports


Smolder And Buildbot

Recomendados

Recomendados

Mais conteúdo relacionado

Último

Último (20)

Destaque

Destaque (20)

Smolder And Buildbot