I have some projects based on different 0.x versions of renku, which I would like to resurrect and start using again. When I open them in renkulab, I am asked to upgrade to the newest version, but the instructions at Upgrade your Renku project — Renku documentation tell me to modify the ARG RENKU_VERSION line in the dockerfile and “if you don’t see this line in your Dockerfile your project was made using an older template”. Well, yes it is, but what can I do in this case? One such project is Reproducible Data Science | Open Research | Renku
Thanks for your help!
1 Like
Hi,
You need to migrate your project and set a template for it. Install the latest version of Renku (or any other version that you want to use) and clone your project. Open a terminal in the project’s root and the run the following command in the command line:
renku migrate
This migrates your project to the Renku version that is used in the CLI. To set a template for the project, run:
renku template set --force python-minimal # Or another template based on your project
You can see a list of available templates by running renku template ls.
Note that this will overwrite some of the project files. You can see what file has been overwritten. If you’ve made modifications to any of those files, you need to re-apply your modifications after this command.
Now, the Dockerfile in your project has a ARG RENKU_VERSION= line. You need to edit this file and put the Renku version that you want:
ARG RENKU_VERSION=1.7.1
Commit changes to the Dockerfile and push your project. Your project should get the intended image.
Let me know if this doesn’t solve your problem.
1 Like
A VERY late thank-you!
This solved now two of my problems. The instructions on how to choose the right version are not very clear, though. https://renku.readthedocs.io/en/stable/how-to-guides/general/upgrading-renku.html points to https://github.com/SwissDataScienceCenter/renkulab-docker#current-images, but the latter does not explain how to identify a release and construct a valid image name. In another repo, your suggestion to use renku template set --force created a new Dockerfile where I changed the line FROM renku/renkulab-py:3.10-0.18.1 to FROM renku/renkulab-py:3.10-b660d07, as I was not able to filter the tags by releases at https://hub.docker.com/r/renku/renkulab-py/tags. In hindsight, I could have left it as it was. Anyway, after these changes, renku migrate finally worked. Actually, only after upgrading my local renku to the newest version, which only worked after creating an environment with the right version of python. The tags on the latter web site implied that I had to use Python 3.10, rather than the current Python version 3.12, with which I was not able to install renku.
1 Like
Hi @schymans thanks for the update - we explain the release image tags here - is there still other information missing to make it clearer?
Thanks for the updated example. Once the example gets outdated, I will be at a loss again finding the updated name at https://hub.docker.com/search?q=renku%2Frenkulab-&type=image as I don’t see how to construct e.g. renku/renkulab-py:3.10-0.15.0 from the information given there. Actually, instead of manually changing that line, could we just replace an exisiting Dockerfile with an up-to-date Dockerfile from a new project? That would make things a lot easier, as it would not require thinking. 
Perhaps it would even be possible to have a repo with up-to-date dockerfiles to choose from when upgrading an old project to work with the current renku.
Hi @schymans that’s what we do already if your project is made with a recent template. When we roll out an updated template (with a new image tag) you will be prompted to update your project.
I see, I think I have experienced this, but didn’t pay attention because it was so smooth. What would be the recipe to achieve the same with older projects? I have a few of them, and would like to avoid editing files by hand if possible.
Hi Stan, when I’ve done this with my projects I’ve usually run renku migrate after having installed a more recent version of the CLI and used the template command that @mohammad-sdsc mentioned above to set a recent template. As Mohammad mentioned, this needs to overwrite some existing files but it stores them in a branch so you can recover whatever you need. Usually this involves slowly updating some of the libraries to work with newer python versions etc.
I just tried creating and pushing two projects using the CLI versions 1.1.0 and 2.5.0 - in the first case, the project settings page instructs you to update twice (first the renku version to bump the metadata and then the template). In the latter case it updates everything in one go. So with relatively recent projects (last ~1.5 years) it should just work in the UI settings page. I believe that projects using earlier versions of the CLI might need a more manual intervention as described above.
1 Like
Thanks @rrrrrok! So editing the Dockerfile after renku template set --force python-minimal is not really needed? Where are the overwritten files stored? I usually create a new branch first, so that I can easily re-import the original requirements.txt etc.
Here are the steps that I do:
- Try to migrate on the project settings page in renkulab.io.
- Update renku on my computer to the same version as on renkulab.io. (What is the easiest way to find out the current version?)
- On the CLI, run
renku migratefrom the base folder of the same project, followed bygit push. If this fails, I run on the CLI:
git checkout -b migrate
renku template set --force python-minimal
git checkout main requirements.txt
git commit -m 'Recreate requirements'
renku migrate
git push --set-upstream origin migrate
Then I test the new branch on renkulab.io and if it runs fine, merge into main.
Perhaps the recommended migrate procedure for different cases could be included in the documentation?
I just checked, they are not actually stored on a branch. But you can easily recover them from git, they are just one commit back. I think your procedure sounds great, with the caveat that I believe you will need to run renku migrate before you do renku template set in some cases, but the CLI should tell you that. Would you like to contribute this to the docs?
You can see the current versions of the different components on this page. “Core” is the same version as the CLI - we should make that clearer.
Wow, Reproducible Data Science | Open Research | Renku is what I was looking for, thank you, @rrrrrok !
I can try to contribute this to the docs myself, but I think you could do this more efficiently, as you probably know exactly where and how. 
Hmmmm, I tried the procedure on another project, but renku migrate fails and I cannot run any other renku command because I’m told I need to run renku migrate first. This is what I get:
$ renku migrate
Applying migration m_0010__metadata_fixes...
Migrating old namespaces
Error: Couldn't execute migration
Traceback (most recent call last):
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/core/migration/migrate.py", line 171, in migrate_project
module.migrate(migration_context)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/core/migration/m_0010__metadata_fixes.py", line 52, in migrate
migrate_old_metadata_namespaces()
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/core/migration/m_0010__metadata_fixes.py", line 87, in migrate_old_metadata_namespaces
data = json.load(file)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 27 column 1 (char 1145)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/ui/cli/exception_handler.py", line 91, in main
return super().main(*args, **kwargs)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/ui/cli/migrate.py", line 135, in migrate
result = command.build().execute(
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/command/command_builder/command.py", line 264, in execute
hook(self, context, result, *args, **kwargs)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/command/command_builder/command.py", line 200, in _post_hook
raise result.error
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/command/command_builder/command.py", line 250, in execute
output = self._operation(*args, **kwargs) # type: ignore
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/inject/__init__.py", line 342, in injection_wrapper
return sync_func(*args, **kwargs)
File "/home/stan/miniconda3/envs/renku_py310/lib/python3.10/site-packages/renku/core/migration/migrate.py", line 173, in migrate_project
raise MigrationError("Couldn't execute migration") from e
renku.core.errors.MigrationError: Couldn't execute migration