A public lecture given at the e-Science Institute and online on 29th October 2009.

1. Software Sustainability: Neil Chue Hong Director, OMII-UK e-Science Institute, Edinburgh, 29 October 2009 Thanks to: Rob Baxter for the original idea and discussion; Malcolm Atkinson, Jim Austin, David De Roure, Carole Goble, and Jen Schopf for their input and slides I’ve borrowed; and especially Simon Hettrick for the imagery and extra research

2. The challenges of enduring

4. Why do myths begin? Miscommunication Uncertainty Misunderstanding Doubt Misinformation Fear

5. “one Spartan was worth several men of any other state”

6. Collecting software in repositories sustains it www.omii.ac.uk

7. Preserving something …

8. … is not the same as sustaining it

9. Comparing Apples and Oranges? www.omii.ac.uk

10. Comparing Apples and Oranges? www.omii.ac.uk

11. Comparing Apples and Books! Storage lifespan: ~12 months Storage lifespan: ~ 50 years Which one is closer to the lifespan of software? www.omii.ac.uk

12. Reproducible Research www.omii.ac.uk

13. Research software should be given to software experts to sustain it www.omii.ac.uk

14. Some things are best kept together

15. Can we throw things over the wall? “Sustainability is not an issue for researchers, we want others to take the software over subject to IP issues” “Sustainability is a big issue, we are producing complex tools we want to continue to use but it’s not clear how they’ll be sustained” - Quotes taken from ENGAGE interviews www.omii.ac.uk

16. Best practice development models www.omii.ac.uk Agile Working together Rigid Photo by HamedSaber

17. The Rise and Fall of Netscape www.omii.ac.uk

18. CASTEP: keeping up with the community Individual Group Consortium W/ industry Community Active www.omii.ac.uk www.omii.ac.uk

19. I can program– I can look after it www.omii.ac.uk

20. A little knowledge is a dangerous thing

22. Average training in S/W Dev in UG Geography: 0 hourswww.omii.ac.uk

23. The Bus Factor Photo by express000 www.omii.ac.uk

24. I’ve maintained software in this domain and it’s just the same in any other www.omii.ac.uk

28. Requirements for e-Research Embedding e-Infrastructure in the research process There is no single common e-Infrastructure Ease of use is the initial barrier Dealing with complexity is complex Trust is important Open development is necessary Give credit for digital creation Attitudes must be changed ~70 attendees (developers, PIs, managers, researchers and funders) http://www.nesc.ac.uk/technical_papers/UKeS-2009-01.pdf www.omii.ac.uk

30. Cohesive communities

32. Isolated yet multidiscliplinary

34. Build it and they won’t always come...

35. Open Source Software is Free... www.omii.ac.uk

36. Free as in Puppy... www.omii.ac.uk Long term costs Needs love and attention May lose charm after growing up Occasional clean-ups required Many left abandoned by their owners

37. Business Modelsare confusing Development Model ([vendor|community|mixed]/[open source|hybrid]) Licensing Strategy (Dual | Open-Core | Open and Closed | Single | Assembled | Closed) Revenue Trigger Commercial License, Subscriptions, Service/Support, Embedded Hardware, Embedded Software, Software as a Service (SaaS), Advertising, Custom Development, Other Products and Services http://blogs.the451group.com/opensource/2009/03/12/a-classification-of-open-source-business-strategies/ www.omii.ac.uk

38. Open Source Business Models www.omii.ac.uk

39. Sustainability Models for Research Software Grant Mosaic Flagship (e.g. CCPs: DL_POLY) Institutional (e.g. Subject repositories, CNX) Fully Costed (e.g. HECTOR CSE Support) Mixed Enterprise / Consultancy (e.g. SugarCRM) Foundation (e.g. Sakai, R) T-shirt UK e-Science Core Programme: Business Models for Sustainability (2007) http://www.jisc.ac.uk/media/documents/programmes/einfrastructure/day2_breakoutbusinessmodels.pdf www.omii.ac.uk

40. Collecting software in repositories sustains it Research software should be given to software experts to sustain it I can program – I can look after it I’ve maintained software in this domain and it’s just the same in any other Outsourcing my software will provide me with free effort www.omii.ac.uk

42. Rememberyour users

43. Engaging Research with e-Infrastructure Interviews Wider deployment Projects Dissemination Adoption New requirements

45. Understand the value

46. Slide from Jim Austin

47. Novel reuse of public sector data http://www.mysociety.org

48. Pick the right team

49. CPOSS: Crystal Energy Landscapes Understanding polymorphism in drugs E.g. Dosage profile Chemists Computational Experimental Developers Domain S/W Engineers Integrators Research Computing Services Facilitator http://www.youtube.com/watch?v=bkbRwOWmiwo www.omii.ac.uk

50. Take difficult tasks…

51. …and make them easy

53. Why do they use it?

54. What do they value from it?

55. What is their relationship between developers and users?

56. What do people want to do?

58. guaranteed long-term support

59. stable APIs as well as interoperable standards

60. sharing of best practice and issuesClear understanding of the requirements that make them a distinct community

61. The R Project for Statistical Computing 1993: First public release; 2 devs 1995: Code open sourced; 3 devs 1996: r-testers list set up 1997: lists split: r-announce, r-help, r-devel; public CVS; 11 devs 2000: CRAN split and mirror 2001: BioConductor 2003: Namespaces 2005: I8n, L8n 2007: R-Forge Today: BioConductor (33 core devs), R-Forge (532 projects, 1562 devs), CRAN (1400+ packages) http://cran.r-project.org/doc/html/interface98-paper/paper_2.html www.omii.ac.uk

62. GridSAM: GrowingCommunity www.omii.ac.uk

63. OGSA-DAI R and OGSA-DAI? www.omii.ac.uk

64. Nominate your successor

65. The Trade-Offs of a Larger Community “Connected, distributed systems, from power grids to business firms to even entire economies, are both more fragile and more robust than populations of isolated entities.” Duncan J. Watts Professor of Sociology, Columbia University Sometimes the thing that kills software is that the community becomes too fragile www.omii.ac.uk

66. Beat the bus and save your software http://video.google.com/videoplay?docid=-4216011961522818645# www.omii.ac.uk

67. Mnemosyne: Remember your Users Zeus: Understand the Value Jason: Pick the right Team Sisyphus: Take difficult tasks and make them Easy Sibilla de Fortia: Nominate your Successor www.omii.ac.uk

68. Software Sustainability: a new dawn? www.omii.ac.uk

69. Software Sustainability: Thanks to: Rob Baxter for the original idea and discussion; Malcolm Atkinson, Jim Austin, David De Roure, Carole Goble, and Jen Schopf for their input and slides I’ve borrowed; and especially Simon Hettrick for the imagery and extra research

70. www.omii.ac.uk Software development comes in stages (and it takes time)

71. Software development comes in stages Idea Prototype Research Idea Prototype Idea Idea Prototype Research Supported Product Idea Prototype Research Supported An idea to solve a problem Understand the functionality Scaling to work for others Allow othersto participate www.omii.ac.uk

72. Commissioned Software Programme @ Q1 ‘08 Supporting Developing Evaluating AHE GridSAM OGRSH BPEL Designer GridBS Broker Grimoires KNOOGLE SAGA RAPID VIC Open Grid Manager PAG RAVE NGS JSDL App Rep WHIP RAT SCAMP OMII-AuthZ NDG Security WSRF:: Lite www.omii.ac.uk

73. Commissioned Software Programme @ Q3 ‘08 Supporting Developing Evaluating AHE SAGA GridSAM GridBS Broker RAPID BPEL Designer ICT GridSAM Grimoires KNOOGLE PAG OGRSH VIC Open Grid Manager RAVE NGS JSDL App Rep WHIP RAT SCAMP OMII-AuthZ NDG Security WSRF:: Lite Commissioned Software Projects progress through the software lifecycle www.omii.ac.uk