Renku run doesn't finish

Renku (CLI)
1

I’m trying to produce some simple plots with a renku command, that finish in seconds without renku, but with renku the command hangs and never finishes. I guess it might have to do with this issue still:

But I can’t do anything with renku anymore in this repository now (Remko Nijzink / VOMcases · GitLab). Is there a way to solve this?

2

Do you get a stacktrace when you interrupt the command with CTRL+C? That might give an indication of where it is stuck.

3

It just said, aborted, nothing more. But when I add strace to the command, it hangs here (I just added the last lines):

clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f6911e29a10) = 22416
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {sa_handler=0x55891a9c6480, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f6911e72210}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=SA_RESTORER, sa_restorer=0x7f6911e72210}, 8) = 0
wait4(-1,
4

Afterwards, the outputfile is there, and there is a new renku-commit. But these files are modified, but not added and commited:

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   .gitattributes
	modified:   .renku/activity_index.yaml

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.renku/workflow/5dac77e9f0044cfd9048a086be7b6f21_python3.yaml
5

We think this might be due to you trying the new, experimental renku graph commands. This creates metadata for the whole project and stores it in 2 files so we don’t need to walk the commit history. But in a project as large as yours, it can take quite a while to load those two files to append the renku run information.

We are still trying different ways of storing the data to improve performance, so this format is not final and hopefully we can get something that even works a on a project of the size of VOMCases and Budyko.

In the meantime, I think if you delete the provenance.json, dependency.json and datasets.json files in .renku, renku run should work again (All this would do is disable the new way of handling metadata again).

6

Okay, thanks! That worked indeed :smiley: