Hello
Regarding the following project Reproducible Data Science | Open Research | Renku
This post is connected to my previous post.
The project of the previous post and this project are forks of the same project. So the error could be connected. Although now the renku workflow visualize command still works but re running a workflow does not.
This project contains only one workflow and updating or executing it again gives the error below.
It seems like renku can not find the mentioned files but they are clearily still in the repository at their respective paths.
Is this a known error or do you have an idea what might cause it ?
base ▶ ~ ▶ work ❯ niod-test ▶ master ▶ $ ▶ renku update --all
Resolved '../../../../tmp/tmp57eg4a00' to 'file:///tmp/tmp57eg4a00'
[job bbbb63ad-da40-4c68-8ad7-ca456f09a9fd] /tmp/xhblx2uq$ python3 \
/tmp/xhblx2uq/main.py \
it_tag_1 \
2022-07-12 \
2022-07-15
m2c_sensors
Traceback (most recent call last):
File "/tmp/xhblx2uq/main.py", line 77, in <module>
main()
File "/tmp/xhblx2uq/main.py", line 26, in main
collector = DataCollecter(testbed='m2c',
File "/home/jovyan/work/niod-test/modules/d01_utils/collection.py", line 17, in __init__
self.read_configuration()
File "/home/jovyan/work/niod-test/modules/d01_utils/collection.py", line 25, in read_configuration
with open(self.configurationfile_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: 'configuration/sensor_configuration.yaml'
[job bbbb63ad-da40-4c68-8ad7-ca456f09a9fd] Max memory used: 118MiB
[job bbbb63ad-da40-4c68-8ad7-ca456f09a9fd] exited with status: 1
[job bbbb63ad-da40-4c68-8ad7-ca456f09a9fd] Job error:
("Error collecting output for parameter '_plans_1e17236b03e54ee1bde45b6d45259c31_outputs_82eb5c8c01b146ab88b896e55f498eb2': ../../../../tmp/tmp57eg4a00:37:5: Did not find output file with glob pattern: '['metrics_it_tag_1.csv']'.", {})
[job bbbb63ad-da40-4c68-8ad7-ca456f09a9fd] completed permanentFail
Error: Unable to finish executing workflow
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/renku/core/workflow/providers/cwltool.py", line 119, in workflow_execute
outputs = process()
File "/opt/conda/lib/python3.9/site-packages/cwltool/factory.py", line 34, in __call__
raise WorkflowStatus(out, status)
cwltool.factory.WorkflowStatus: Completed permanentFail
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/site-packages/renku/ui/cli/exception_handler.py", line 92, in main
return super().main(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/renku/ui/cli/update.py", line 195, in update
update_command(skip_metadata_update=skip_metadata_update)
File "/opt/conda/lib/python3.9/site-packages/renku/command/command_builder/command.py", line 266, in execute
hook(self, context, result, *args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/renku/command/command_builder/command.py", line 202, in _post_hook
raise result.error
File "/opt/conda/lib/python3.9/site-packages/renku/command/command_builder/command.py", line 252, in execute
output = self._operation(*args, **kwargs) # type: ignore
File "/opt/conda/lib/python3.9/site-packages/renku/command/update.py", line 84, in _update
execute_workflow_graph(dag=graph.workflow_graph, provider=provider, config=config)
File "/opt/conda/lib/python3.9/site-packages/inject/__init__.py", line 342, in injection_wrapper
return sync_func(*args, **kwargs)
File "/opt/conda/lib/python3.9/site-packages/renku/core/workflow/execute.py", line 83, in execute_workflow_graph
execute(dag=dag, basedir=project_context.path, provider=provider, config=config)
File "/opt/conda/lib/python3.9/site-packages/renku/core/plugin/provider.py", line 95, in execute
return executor(dag=dag, basedir=basedir, config=config)
File "/opt/conda/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
File "/opt/conda/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
File "/opt/conda/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
return outcome.get_result()
File "/opt/conda/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
raise ex[1].with_traceback(ex[2])
File "/opt/conda/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
res = hook_impl.function(*args)
File "/opt/conda/lib/python3.9/site-packages/renku/core/workflow/providers/cwltool.py", line 121, in workflow_execute
raise WorkflowExecuteError() from e
renku.core.errors.WorkflowExecuteError: Unable to finish executing workflow