How to resolve circular workflows?

I forked an old project (from 2021), successfully ran renku migrate, but now if I run renku status, it shows a lot of outdated files, but I cannot update them, as:

$ renku workflow visualize
Error: Cycles detected in execution graph: (/activities/615a8439d53c8d899f6ae07c6660df421988c52d, notebooks/theory/pyFile_storage/theory_variable.py, /activities/bf074d031839a74fce6b2f93ec92fe6e4ba9f2d8, notebooks/theory/pyFile_storage/theory_equation.py)
...

Could anyone help me resolve the alleged cycles? Here is the project:

Hi @schymans, As far as I understand, the way the workflows were defined, Renku may be erroneously detecting an input as an output. Basically, I see that at least two files (theory_equation.py and theory_variable.py) are inputs and outputs of you workflows (run renku workflow inputs and renku workflow outputs for checking input and output files). I assume that they should not be part of the outputs, since scripts are usually input files. Please, specify the wrongly detected outputs as an explicit input using ‘–input’ when defining the workflows.

Thanks, @elisabetc! In fact both files are outputs and inputs, but this should not be a circularity. Essentially the files are created out of notebooks where new mathematical formulations are derived, then written to .py files and imported elsewhere to be applied to data.
renku workflow inputs and renku workflow outputs just lists the files without displaying any directional interdependencies, so this does not help detect circularity, unfortunately.

@elisabetc do you have any other ideas how I could resolve this alleged circularity problem? I assume that the circularity checking was introduced more recently, and that it just blindly tests whether any files are listed as outputs and inputs anywhere, without checking for the sequence of actions in the workflow. Could this be a bug?

Hi @schymans, I will check the different steps in the workflow, since it should not happen that the circularity error appears if the outputs of a step are the inputs for a following step. The circularity works checking inputs and outputs of the same step or consecutive steps (e.g. the input of a former step is the output of a latter one). I will look again and let you know.

what this error means is that activity 615a8439d53c8d899f6ae07c6660df421988c52d has notebooks/theory/pyFile_storage/theory_variable.py as output, which is an input of activity bf074d031839a74fce6b2f93ec92fe6e4ba9f2d8 which has notebooks/theory/pyFile_storage/theory_equation.py as an output which in turn is an input to 615a8439d53c8d899f6ae07c6660df421988c52d again, forming a circle.

You could do renku workflow revert /activities/615a8439d53c8d899f6ae07c6660df421988c52d or renku workflow revert /activities/bf074d031839a74fce6b2f93ec92fe6e4ba9f2d8 to undo (delete) either of those activities and that should fix the issue. You can call them with --metadata-only to not undo file changes, just the recorded provenance. You probably also need --force as you can otherwise remove activities that have dependents.

It could be worthwhile to check what those activities are beforehand to better understand the problem. We unfortunately don’t have a command to show the details of an activity by id, but renku log -w shows details of all activities and you can grep for the id’s to see what the activities were, including the command.