4. Windows
An error has occurred. To continue:
Press Enter to return to Windows, or
Press CTRL+ALT+DEL to restart your computer. If you do this, you will loose any
unsaved information in all open applications
Error: 0E : 016F : BFF9B3D4
Press any key to continue _
11. Best Practices
Write programs for people, not computers
Automate repetitive tasks
Use the computer to record history
Make incremental changes
Use version control
Don't repeat yourself (or others)
Plan for mistakes
Optimize software only after it works correctly
Document the design and purpose of code
Conduct code reviews
Wilson et al 2012. Best Practices for Scientific Computing. arXiv:1210.0530v3
12. Best Practices
Write programs for people, not computers
Automate repetitive tasks
Use the computer to record history
Make incremental changes
Use version control
Don't repeat yourself (or others)
Plan for mistakes
Optimize software only after it works correctly
Document the design and purpose of code
Conduct code reviews
Wilson et al 2012. Best Practices for Scientific Computing. arXiv:1210.0530v3
13. Best Practices 1: Automation
Write programs for people, not computers
Automate repetitive tasks
Use the computer to record history
Make incremental changes
Use version control
Don't repeat yourself (or others)
Plan for mistakes
Optimize software only after it works correctly
Document the design and purpose of code
Conduct code reviews
Altintas et al 2004. Kepler: an extensible system for design and execution of scientific workflows. Proc 16th ICSSDM
14. Parameter Uncertainty: Test Case
Single Analysis:
Contribution of parameter uncertainty to
uncertainty in Switchgrass Yield prediction.
LeBauer, Wang, Richter, Davidson, and Dietze 2013.
Facilitating Feedbacks between ecological models and data. Ecological Monographs
15. Parameter Uncertainty: Automated
* 17 Plant functional types
* 6 biomes
* 8 scientists
* 6 Months
Dietze, Serbin, LeBauer, Davidson, Desai, Feng, Kelly, Kooper, LeBauer, Mantooth, McHenry, and Wang. submitted
A quantitative assessment of a terrestrial biosphere model's data needs across North American biomes. JGR
% SD Explained
Contribution of parameter uncertainty
to model uncertainty.
16. Best Practices 2:
Iteration with Testing
Write programs for people, not computers
Automate repetitive tasks
Use the computer to record history
Make incremental changes
Use version control
Don't repeat yourself (or others)
Plan for mistakes
Optimize software only after it works correctly
Document the design and purpose of code
Conduct code reviews
Wilson et al 2012. Best Practices for Scientific Computing. arXiv:1210.0530v3
26. Conclusions
* Best practices lead to more effective and efficient modeling
* Applied integration tests to support model development
* Controlling technical error produces more robust and accurate inference
27. Future Directions
* Track benchmark metrics for specific model runs
* Maintain ability to reproduce published results
* Automated testing with each code commit or major release
* Current Metrics to define limits of model credibility
