See also
- @transform in the Ruffus Manual
- Decorators for more decorators
@transform( input, filter, output, [extras,...] )ΒΆ
- Purpose:
Applies the task function to transform data from input to output files.
Output file names are specified from input, i.e. from the output of specified tasks, or a list of file names, or a glob matching pattern.
String replacement occurs either through suffix matches via suffix or the formatter or regex indicators.
Only out of date tasks (comparing input and output files) will be run
Simple Example
Transforms *.c to *.o:
@transform(input = ["1.c", "2.c"], filter = suffix(".c"), output = ".o") def compile(infile, outfile): passSame example with a regular expression:
@transform(["1.c", "2.c"], regex(r".c$"), ".o") def compile(infile, outfile): passBoth result in the following function calls:
# 1.c -> 1.o # 2.c -> 2.o compile("1.c", "1.o") compile("2.c", "2.o")Escaping regular expression patterns
A string like universal.h in add_inputs will added as is. r"\1.h", however, performs suffix substitution, with the special form r"\1" matching everything up to the suffix. Remember to ‘escape’ r"\1" otherwise Ruffus will complain and throw an Exception to remind you. The most convenient way is to use a python “raw” string.Parameters:
- filter = suffix(suffix_string)
must be wrapped in a suffix indicator object. The end of each input file name which matches suffix_string will be replaced by output.
Input file names which do not match suffix_string will be ignored
The non-suffix part of the match can be referred to using the "\1" pattern. This can be useful for putting the output in different directory, for example:
@transform(["1.c", "2.c"], suffix(".c"), r"my_path/\1.o") def compile(infile, outfile): passThis results in the following function calls:
# 1.c -> my_path/1.o # 2.c -> my_path/2.o compile("1.c", "my_path/1.o") compile("2.c", "my_path/2.o")For convenience and visual clarity, the "\1" can be omitted from the output parameter. However, the "\1" is mandatory for string substitutions in additional parameters,
@transform(["1.c", "2.c"], suffix(".c"), [r"\1.o", ".o"], "Compiling \1", "verbatim") def compile(infile, outfile): passResults in the following function calls:
compile("1.c", ["1.o", "1.o"], "Compiling 1", "verbatim") compile("2.c", ["2.o", "2.o"], "Compiling 2", "verbatim")Since r”1” is optional for the output parameter, "\1.o" and ".o" are equivalent. However, strings in other parameters which do not contain r”1” will be included verbatim, much like the string "verbatim" in the above example.
- filter = regex(matching_regex)
is a python regular expression string, which must be wrapped in a regex indicator object See python regular expression (re) documentation for details of regular expression syntax Each output file name is created using regular expression substitution with output
- filter = formatter(...)
a formatter indicator object containing optionally a python regular expression (re).
- output = output
Specifies the resulting output file name(s) after string substitution
- extras = extras
Any extra parameters are passed verbatim to the task function
If you are using named parameters, these can be passed as a list, i.e. extras= [...]
Any extra parameters are consumed by the task function and not forwarded further down the pipeline.
If regex(matching_regex) or formatter(...)` is used, then substitution is first applied to (even nested) string parameters. Other data types are passed verbatim.
For example:
@transform(["a.c", "b.c"], regex(r"(.*).c"), r"\1.o", r"\1") def compile(infile, outfile): passwill result in the following function calls:
compile("a.c", "a.o", "a") compile("b.c", "b.o", "b")
See here for more advanced uses of transform.