Since it was implemented, I find that the most convenient way for me to create the lineage for a file is by defining in the scripts the inputs and outputs relative paths, and saving them into the files
.renku/tmp/outputs.txt, respectively. This has worked already in the past quite neatly.
However, I tried to do the same now, and even though all seems to work correctly, the files do not have any lineage. No idea which could be the reasons for that. Please, let me know who could take a look, and I will add you to the repo, as it is a private one.
Thank you so much
it should be a yaml file like
.renku/tmp/inputs.yml that looks like
i.e. name:path pairs.
Out of curiosity, did you try the Renku API for this? It might be more convenient than using the files.
Ohh, I see, so this changed! Cool then, I try with
yaml, should be quick an easy. I guess no indent is required, each line is just an input, or an output, with the key-value pair, is that right? And regarding the API, I remember I tried in the past, but honestly, back then it did not work.
Yes, no indentation is needed and each line is for a file (or folder).
You could also use a workflow file which might be easier to use
My reason to not use the Workflow files is that some of the input and output files have variable names that are defined inside the script, and using
workflow files is not compatible with manual definition of inputs and outputs.
In any case, I gave it another try, using the API, and well, all went well (it seems…). But again, I got some output files that do not have any lineage. Hence, I don’t know what happened with all those
The repo status presents a couple of errors, but it does not provide any further information.
Besides, there seems to be something weird with the datasets. The repo is not that old (less than 1 year), and in the beginning they were being shown correctly
Again, if at the end the problem is related with incompatible versions, it is a bit unfortunate, as the repo is not that old. How could I solve all these problems?
Thank you so much fellas for all your help!
When you say “do not have any lineage”, do you mean in the UI? Do you see the lineage in the CLI using
renku workflow visualize <path> ?
As to the UI issues, I guess you’re using renku CLI> 2.x on renkulab.io(not limited)? We still have to roll out Renku The Platform on renkulab.io so it properly supports Renku The CLI (should happen soon-ish) and until then, project using the newer version of the CLI have limited support
Thanks for the reply again! This
renku workflow visualize <path> worked! So yep, now I know at least the information is stored
Then, waiting for the new fancy roll out in order to visualize the graph in all its glory on the UI
Thank you so much!