1. Sumatra: an electronic labook. Git it right
Michele Mattioni
Computational Neurobiology group
mattioni@ebi.ac.uk
EBI is an Outstation of the European Molecular Biology Laboratory.
2.
3. Open questions which I had no answer:
• Where are my results?
• Which parameters I used on that simulation?
http://www.flickr.com/photos/valeriebb/3006348550/
• Which one was that simulation?
• Why it worked yesterday and not today?
From Valerie Everett's photostream
• At exactly which point it worked yesterday?
• Where is my mind stuff?
4. How the experimental people solve that?
• Write down on a labbook everything they do :
• reagents
• times
• protocols
why!
http://www.flickr.com/photos/mwichary/3249202959/
•
• what worked, what didn't
• ...
Marcin Wichary's photostream
5. We need an electronic labbook
• Able to track (automatically) our 'experiments' details:
• script launched
• parameters used
• reason
• outcome
• But we can also:
• search them
• organize them
• Rerun them!
6. Say hello to Sumatra
http://neuralensemble.org/trac/sumatra
Slide from Andrew Davison (main developer),
http://neuralensemble.org/meetings/talks/CodeJam3_Davison_Sumatra.pdf
7. Sumatra: what I need to use it
• Code under version control
• Accepted version control:
• Mercurial (hg)
• Subversion (svn)
• Git (git) (I patched it! Power of the opensource..)
• your choice (exstensible architeture)
• Scripts organized as Main class and parameter_file
8. smt Usage
$ cd myproject
$ smt init MyProject
Normal way you launched the script:
$ python main.py default.param
New way using Sumatra:
$ smt run simulator=python main=main.py default.param
9. Shorter cmdline to launch the job
From
$ smt run simulator=python main=main.py default.param
Configure it (it has to be done once.)
$ smt configure simulator=python main=main.py
To
$ smt run default.param
10. What if I use java?
$ java main default.param
$ smt run simulator=java main=main default.param
11. Parameter file? Excuse me?
Basically a name <-> parameter pair
dt = 0.05 # Dt for the integration
tstop = 10 # [ms]
Not obligatory, but it's very handy because:
• Sumatra can store your parameters in the record
• If Sumatra understand them then You can change
parameter on the fly
$ smt run default.param tstop=20 dt=0.01
12. Where is my stuff?
smt monitor a directory (default Data).
• Any new file/image saved there is automatically added
to the simulation record
You save the results
Sumatra remembers where is it
My way: I create a dir for each day and then save all my stuff in
dedicated folder
• Data/19042010/Sim_1/, Data/19042010/Sim_2/
• Data/20042010/Sim_1/, Data20042010/Sim_2/
13. Access the results: smt
$ smt list
default_20100407163228
default_20100407171249
$ smt list l
Label : default_20100420170241
Reason : Testing the tagging from the cmdline
Outcome : It works!
Duration : 4.38211798668
Repository : GitRepository at
/home/mattions/Work/model/MSN/TimeScales
Main_File : spineIntegration.py
Version : 8a61ecdd999e30d25c29620a87d9f4536aa67e9e
Executable : Python (version: 2.6.4) at /usr/bin/python
Timestamp : 20100420 17:02:41.445588
Tags : test, test2
17. How to install
ATM: Git support only available from the latest code:
hg clone https://neuralensemble.org/hg/sumatra
git clone http://github.com/mattions/sumatra_on_github
18. How sumatra saved the day (yesterday)
Scenario:
• Made some massive restructuring/refactoring on neuronvisio
• Get lost on the way
• Nothing worked anymore.
But ...
23. Merged restart back into master
kind of apply twice the same patch, but you
know it was worth it.
24. Summary
• Sumatra can help you to:
• keep your stuff organized
• not lose your work
• speed up the recovery time when somethig get wrong
• remember what you used ages ago, to make that figure
• be more focus on your science.
• You can help sumatra
• Using it
• Asking for request (ok, send a patch. It's quicker and easier)
25. Acknowledgement
• Andrew Davison (main sumatra developer)
• people got the picture from (link on each picture)
• Nicolas Le Novère (to let me work on this)
• Computational Neurobiology group @ EBI.