A brief presentation on scientific software development - "How to grow a sustainable software development process."

1. How to grow a sustainable software
development process (for scientific
software)
C. Titus Brown
6/8/15

2. Questions and suggestions and
skepticism welcome!

3. The khmer project
• ~13k lines of C++, ~7k lines of Python
• About 8 years old
• ~5 regular developers, ~60 contributors total
• Formal dev/contrib process (khmer.rtfd.org)
• We took a circuitous path… see
“Walking the talk”
http://dx.doi.org/10.6084/m9.figshare.791567

4. Example
(Creating a new pull request)

5. A simple three step process.
1. Use version control and create branches for
each feature set.
2. Make up a checklist to fill out at each merge.
3. Adhere to the checklist :)

6. Version control
With distributed VCS like git or mercurial, there
is no good reason not to use version control
even for small personal projects.
Version control will save your bacon.

7. A checklist

8. What goes on the checklist?!
Ideas:
• Two-person merge rule;
• “I ran the tests, and they succeeded”
• “Someone else ran the tests, and they
succeeded”
• “A computer ran the tests automatically, and
they succeeded.”

9. What goes on the checklist?!
Ideas:
• Code formatting guidelines;
• Code coverage guidelines;
• Spellcheck;
• Changes described in ChangeLog;
• Commit messages make sense;
• Specific feature checks (“command line scripts
should be documented”; “CPython objects should
have XXX”);

10. Important thing about these rules:
Target things that are causing you pain.
• There are an infinite number of edges to sand
down on any reasonably involved software
project!
• Focus on the painful edges.

11. (Running the tests.)
(Continuous Integration)

12. Testing
I don’t personally do Test Driven Development,
although I’m happy to work with people who do.
“Test-Enhanced Development” –
1. Write really simple tests that cover most of the
new code, along with any tricky bits that you
were worried about when writing the code
(~TDD-lite).
2. Then do SDD.

13. “Stupidity Driven Development”
• Wait until something breaks (user report, bug
found, whatever).
• Write a test that triggers that bug.
• Then, and only then, fix the bug.
That exact bug will never reappear guise in the
codebase :)

14. Code coverage
Use code coverage as a guide to writing new
tests.
If a line if code isn’t executed by your tests, then
it’s not tested by your tests.
(Just because it’s successfully executed by your
tests doesn’t mean it’s correct!)

15. Random advice

16. Automate your tests
• Anything that isn’t automated won’t be run
regularly, if at all.

17. Keep many tests lightweight
• Only tests that run quickly will be run
regularly.
• E.g. Save obscure cross-platform tests for
releases.

18. Set up continuous integration, maybe?
• Very useful for cross-platform testing,
integration testing…
• …but people should be running the tests
before they commit, and certainly before
they push.

19. Have a release process, if you do
releases.
…yet another checklist.
Releases are usually done infrequently, so
remove need for good memory.

20. Never rewrite both your tests and your
code at the same time.
Otherwise you’ll not only invent exciting new
bugs, but you’ll reimplement boring old ones!