# Renku run no workflow recorded

**URL:** <https://renku.discourse.group/t/renku-run-no-workflow-recorded/828>\
**Category:** General\
**Created:** [13 June 2023 13:55 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828 "2023-06-13T13:55:37Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![firat](https://avatars.discourse-cdn.com/v4/letter/f/ecae2f/32.png) [@firat](https://renku.discourse.group/u/firat)\
**Post date:** [13 June 2023 13:55 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/1 "2023-06-13T13:55:37Z")

</div>

Hello,  
In this [public project](https://renkulab.io/projects/firat.ozdemir/oadat-evaluate), I am trying to create a workflow using `renku run` command.  
In particular, I am running the following:

```bash
renku run --name demo -- bash download-oadat-mini.sh; python train_segmentation_demo.py --task_str="seg_swfd_lv128,sc" --datasets_parent_dir="data/OADAT-mini" --logdir="trained_models/seg_swfd_lv128,sc"; python eval_segmentation_demo.py --task_str="seg_swfd_lv128,sc" --datasets_parent_dir="data/OADAT-mini" --fname_out="trained_models/seg_swfd_lv128,sc/eval.p"

```

This properly executes the three lines of scripts that I would like it to do (1. download files, 2. run some NN optimization, 3. compute predictions and save in a pickle file). However, I do not see any output from renku itself. Similarly `renku workflow ls` returns and empty list.

I am not sure what the expected behavior should be, but probably not this. Can you have a look? It takes a few mins to run.  
I am mainly suspecting some renku version mismatch or so from my Dockerfile, but otherwise have no idea.

---

<div class="post-metadata">

**Author:** ![mohammad-sdsc](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/mohammad-sdsc/32/79_2.png) [@mohammad-sdsc](https://renku.discourse.group/u/mohammad-sdsc)\
**Post date:** [13 June 2023 15:14 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/2 "2023-06-13T15:14:01Z")

</div>

Hi Firat,

One issue that I see with your command is that `renku run` doesn’t support multiple commands (i.e. semicolons, pipes, …). So, when you execute this, only the first command is tracked by Renku.

To overcome this limitation, you need to create a script and put multiple commands in it.

I couldn’t run this command to completion due to some missing libraries, but the first part still created a `demo` plan in the project:

```auto
$ renku workflow ls
ID NAME COMMAND
--------------------------------------- ------ ---------------------------
/plans/d36db7098aee4c8893cf0f9373482db9 demo bash download-oadat-mini.sh

```

This should happen in your project because Renku stores the plan after reaching the semicolon and the rest is executed by shell and won’t be tracked by Renku.

Let me know if this changed anything.

Cheers,  
Mohammad

---

<div class="post-metadata">

**Author:** ![firat](https://avatars.discourse-cdn.com/v4/letter/f/ecae2f/32.png) [@firat](https://renku.discourse.group/u/firat)\
**Post date:** [13 June 2023 15:57 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/3 "2023-06-13T15:57:50Z")

</div>

Thanks!  
I put the script into another bash file as follows  
demo.sh:

```auto
#!/bin/bash
bash download-oadat-mini.sh
python train_segmentation_demo.py --task_str="seg_swfd_lv128,sc" --datasets_parent_dir="data/OADAT-mini" --logdir="trained_models/seg_swfd_lv128,sc"
python eval_segmentation_demo.py --task_str="seg_swfd_lv128,sc" --datasets_parent_dir="data/OADAT-mini" --fname_out="trained_models/seg_swfd_lv128,sc/eval.p"

```

When I run  
`renku run --name demo demo.sh`  
or  
`renku run --name demo -- demo.sh`  
I get the error

> Error: Invalid parameter value - Cannot execute command ‘’: This is likely because the executable doesn’t exist or has the wrong permissions set.

can you comment on it?

---

<div class="post-metadata">

**Author:** ![rrrrrok](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/rrrrrok/32/117_2.png) [@rrrrrok](https://renku.discourse.group/u/rrrrrok)\
**Post date:** [13 June 2023 16:01 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/4 "2023-06-13T16:01:47Z")

</div>

Why not do

```auto
$ renku run bash download-oadat-mini.sh
$ renku run python train_segmentation_demo.py --task_str="seg_swfd_lv128,sc" --datasets_parent_dir="data/OADAT-mini" --logdir="trained_models/seg_swfd_lv128,sc"
$ renku run python eval_segmentation_demo.py --task_str="seg_swfd_lv128,sc" --datasets_parent_dir="data/OADAT-mini" --fname_out="trained_models/seg_swfd_lv128,sc/eval.p"

```

?

As for the demo script, it probably doesn’t have execute permission you probably need to do `chmod u+x demo.sh` and run it with `renku run --name demo ./demo.sh`

But personally, I would split this up into three steps.

---

<div class="post-metadata">

**Author:** ![firat](https://avatars.discourse-cdn.com/v4/letter/f/ecae2f/32.png) [@firat](https://renku.discourse.group/u/firat)\
**Post date:** [13 June 2023 16:30 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/5 "2023-06-13T16:30:27Z")

</div>

This worked, for the most part.  
Now I can see IDs for the 2nd and 3rd lines, but nothing for the first line (`renku run bash download-oadat-mini.sh`) under `renku workflow ls`

---

<div class="post-metadata">

**Author:** ![rrrrrok](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/rrrrrok/32/117_2.png) [@rrrrrok](https://renku.discourse.group/u/rrrrrok)\
**Post date:** [13 June 2023 16:38 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/6 "2023-06-13T16:38:20Z")

</div>

How big is this file, is it not easier to just add it to the project? 🙂

---

<div class="post-metadata">

**Author:** ![firat](https://avatars.discourse-cdn.com/v4/letter/f/ecae2f/32.png) [@firat](https://renku.discourse.group/u/firat)\
**Post date:** [13 June 2023 16:59 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/7 "2023-06-13T16:59:37Z")

</div>

Downloaded files for this demo is 14.5GB. However, the download script can be slightly changed to download full dataset for a local renku session, in which case, the data files would be ~800GB. So I would rather not.  
On the other hand, I know the downloaded files will always be the same, because the host will keep the files as is at least for 10 years, so a checksum on downloaded files should remain the same.

---

<div class="post-metadata">

**Author:** ![firat](https://avatars.discourse-cdn.com/v4/letter/f/ecae2f/32.png) [@firat](https://renku.discourse.group/u/firat)\
**Post date:** [13 June 2023 17:00 UTC](https://renku.discourse.group/t/renku-run-no-workflow-recorded/828/8 "2023-06-13T17:00:34Z")

</div>

Can you let me know the exact script you used to get this output? For me, I am not seeing bash download-oadat-mini.sh in workflow ls, but only the other 2 lines
