Hello, I have created a Renku pet project to test renku dataset
.
I initially created a dataset, added a file and run a script to clean data that takes 2 parameters:
- an
--input-path
to read the dirty data - an
--output-path
to write a cleaned csv file.
The (here simplified) command was:
renku run python clean_data.py --input-path air-quality/file.csv --output-path processed/file.csv
Then I re-run a similar command including also an --input
directory to force renku updating when adding any file to the folder:
renku run python clean_data.py --input-path air-quality/file.csv --output-path processed/file.csv --input air-quality/
At this point, I tried to add another file to the dataset, that is downloaded and added to the input folder air-quality/
. I expected the update to run automatically, but this didn’t happen. Is this correct?
Then I typed renku update
but I got an error.
I believe I created extra dirty workflows that should be removed. Does this make sense? How can I get rid of the redundant workflow form .renku/workflows
in a clean way?