Hi,
I’m testing the renku workflow iterate
in a test project where I’m working on a workflow which takes a file and a parameter as input and creates a file out of it:
Id: /plans/3743bc29ee7a4ba78c08489a15b8b48b
Name: test_iter
Command: Rscript --vanilla script.R --input_folder data/input1 --input_param 1 --model_name test
Success Codes:
Inputs:
- vanilla-1:
Default Value: script.R
Position: 1
Prefix: --vanilla
- input_folder-2:
Default Value: data/input1
Position: 2
Prefix: --input_folder
Outputs:
- output-0639:
Default Value: data/output/test.txt
Position: None
Parameters:
- input_param-3:
Default Value: 1
Position: 3
Prefix: --input_param
- model_name-4:
Default Value: test
Position: 4
Prefix: --model_name
I’ve tried renku workflow iterate
in several manners to create multiple outputs, without success.
Case 1
I don’t force any naming of the output, simply iterate over the parameters, stored in a param-yaml
:
renku workflow iterate --mapping param.yaml test_iter
But only 1 file is created, although I specified 3 variations of the parameter.
Case 2
I try to specify explicitly the output with {iter_index}
renku workflow iterate --mapping param.yaml test_iter --map "output-0639=data/output/test_{iter_index}.txt"
But CWL stops, telling me that
Did not find output file with glob pattern: '['data/output/test_0.txt']'.", {})
.
Case 3
I try to name the outputs inside my script, based on the parameters used:
renku run --name test_iter2 Rscript --vanilla script2.R --input_folder input1 --input_param 1 --model_name test
renku workflow iterate --mapping param.yaml test_iter2
But again, CWL cannot find file with glob pattern: '['data/output/testinput11.txt']'.", {})
Summary
So I’m a bit puzzled about how I should use renku workflow iterate. I guess something is wrong about how I define the outputs but I tried to copy as closely as possible the examples (although I’m working here with R). Ultimately, I would like to :
- iterate over parameters
- iterate over input files
- map the outputs to a specific name