What is continuous deployment? What are the challenges to getting it right? How does Perl meet/destroy those challenges?

1. Continuous
Deployment in Perl
Presentation by Alex Balhatchet (KAORU)

2. Intro

3. What is continuous deployment?
Commit Changes
Deploy Changes
As quickly as
possible! :-)
Test Changes

4. Committing Changes
● Use your favourite VCS
● Push the changes somewhere centralized
(staging/testing environment)
● This should kick off a "build and test" cycle if
one is not currently running

5. Testing Changes
Perl is absolutely fantastic for testing.
● TAP
● TAP::Harness / App::Prove
● Test::Builder / Test::More

6. Deploying Changes
● Pick your favourite deployment method
● Make it as fast as possible
● Make it as easy as possible

7. Challenges
● Fast tests
● Reliable tests
● Fast deployment

8. Challenges
● Fast tests
Focus of this talk
● Reliable tests
● Fast deployment

9. Perl Testing

10. TAP::Harness
and App::Prove

11. TAP::Harness and App::Prove
/usr/bin/prove
uses App::Prove
uses TAP::Harness

12. App::Prove features - basics
● Run lots of tests with one command
● Hide non-failure output
● Run tests in parallel
● Show timing information

13. App::Prove features - basics
alex@karin$ prove -l -r -j 16 t/
t/00-load.t .......... ok
t/01-backup.t ........ ok
t/02-delete_files.t .. ok
All tests successful.
Files=3, Tests=51, 1 wallclock secs ( 0.06 usr
0.01 sys + 0.78 cusr 0.20 csys = 1.05 CPU)
Result: PASS

14. App::Prove features - advanced
● Load plugins from App::Prove::Plugin::*
● Save the TAP results to a .tgz archive
● Save state between runs and use that state
to change the order for future runs

15. App::Prove features - advanced
alex@karin$ prove -l -r --state=hot,all,save t/
No saved state, selection will be empty
t/00-load.t .. ok
t/01-good.t .. ok
t/02-bad.t ... 1/1
alex@karin$ prove -l -r --state=hot,all,save t/
t/02-bad.t ... 1/1 <===== Runs first!
t/00-load.t .. ok
t/01-good.t .. ok

16. App::Prove features - advanced
Other "state" options:
● slow - very useful with -j N
● fresh - only run the tests you need to
● last - same as last run, even with --shuffle

17. Test::Aggregate

18. Test::Aggregate
● Combine multiple .t files into a single file
● Avoids compilation overhead
● Therefore speeds up your tests!

19. Test::Aggregate::Nested
● Alpha code
● Uses the fairly recently added syntax for
subtests in TAP
● Works really well for us!

20. Test::Aggregate caveats
● Breaks BEGIN and END blocks
(use Scope::Guard instead!)
● exit() is also a bad idea!
● Be aware of Test modules which use these
(Test::NoWarnings is the most common!)

21. Parallelisation of Aggregated Tests
Currently closed source - sorry! The logic is:
i. Split up tests into N groups
ii. Copy .t files into temporary directory
iii. Write temporary .t file that uses Test::
Aggregate::Nested
iv. Run those tests with prove -j N

22. TAP::Harness
::Archive

23. TAP::Harness::Archive
● "isa" TAP::Harness
● Stores the TAP output in a tar archive
● Gives you full TAP even when running in
parallel
● Allows you to run your tests once and then
format the results in multiple ways

24. How we format our TAP
● Process the output from our parallelised
aggregated test runs
● Match up the failures with the .t files
● Write a summary in Markdown format that's
later converted to HTML and emailed to us

25. How you might format your TAP
● TAP::Formatter::JUnit (eg. for Jenkins)
● TAP::Formatter::TeamCity
● TAP::Formatter::HTML

26. TAP::Formatter::HTML example

27. TAP::Formatter::HTML example

28. Putting it all
together

29. Continuous Deployment at Lokku
svn commit On test success new code is
automatically pushed
Deployment is one command
and very quick
Perl daemon notices commit
Parallelised and aggregated test run
starts running
Nicely formatted results are emailed
on success or failure

30. What's next?

31. What's next?
● Test::WWW::Selenium::More
● Faster tests - more unit tests, fewer
integration tests
● Parallelised deployment

32. Thanks!
Any questions?