make is a basic tool to define pipelines of shell commands.
It is useful if you have many shell scripts and commands, and you want to organize them.
Even if it has been written to automatize the build of compiled language programs, make is also useful in bioinformatics and other fields.
31. It allows you to use command-line tools which are more flexible;
32. Combined with a revision control software, it makes possible to reproduce all the operations made to your data;
33. Second part A closer look at make syntax (target and commands)
34.
35.
36. Everytime you call a make rule (example: 'make all'), the program looks for a file called like the target name (e.g. 'all', 'clean', 'inputdata.txt', 'results.txt')
37. The rule is executed only if that file doesn't exists.
38.
39.
40. When we call ' make testfile.txt ', make checks if a file called 'testfile.txt' already exists.
41. Filename as target names The commands associated with the rule ' testfile.txt ' are executed only if that file doesn't exists already
42.
43. You can retrieve the matched target with the special variable $@
62. Example: result1.txt : data1.txt data2.txt cat data1.txt data2.txt > result1.txt @echo 'result1.txt' has been calculated'
63. Prerequisites are files (or rules) that need to exists already in order to create the target file.
64. If 'data1.txt' and 'data2.txt' don't exist, the rule 'result1.txt' will exit with an error (no rule to create them)
65.
66.
67.
68. what happens if we remove the file 'result1.txt' we just created?
69.
70. what happens if we remove the file 'result1.txt' we just created?
71. The second time we run the 'make result1.txt' command, it is not necessary to create data1.txt again, so only a rule is executed
72.
73. Make all will calculate result1.txt and result2.txt, if they don't exist already (and they are older than their prerequisites)
74.
75. We can do better: create or update a file only if it is newer than its prerequisites
76.
77. A great feature of make is that it execute a rule not only if the target file doesn't exist, but also if it has a 'last modification date' earlier than all of its prerequisites
78.
79. $: touch data1.txt calculate_result.py $: make result1.txt result1.txt has been calculated $: make result1.txt result1.txt is already up-to-date $: touch data1.txt $: make result1.txt result1.txt has been calculated
80.
81. Let's say you discover an error in one of your input data: you will be able to repeat the analysis by executing only the operations needed
82. You can also use it to re-calculate results every time you modify a script: result.txt : scripts/calculate_result.py python calculate_result.py > result.py
90. They are called by including their name in ' $() ' WORKING_DIR is a variable
91.
92. '$@' always corresponds to the target name: $: cat >Makefile %.txt : echo $@ $: make filename.txt echo filename.txt filename.txt $: $@ took the value of 'filename.txt'
93. Other special variables $@ The rule's target $< The rule's first prerequisite $? All the rule's out of date prerequisites $^ All Prerequisites
94.
95. Most frequently used functions: $(addprefix <prefix>, list) -> add a prefix to a space-separated list example: FILES = file1 file2 file3 $(addprefix /home/user/data, $(FILES)
124. Study it and understand its logic. Use it in the most basic way, without worrying about prerequisites and special variables. Later you can look for easier tools (biomake, rake, taverna, galaxy, your own, etc..)
125.
126. A Makefile is a pipeline http://www.nodalpoint.org/2007/03/18/a_pipeline_is_a_makefile