I am trying to salvage my 100+ existing projects before gitlab.renkulab.io gets shut down. To preserve the gitlab issues, branches, etc., I am using the export function in gitlab and after re-importing a project on a different instance, I am archiving the original project on gitlab.renkulab.io, to prevent that anybody still pushes to the old place. However, after archiving, I don’t seem to be able to migrate the project from renkulab v1 to v2 any more, as I get a 404 whenever I click on it in the old renkulab v1 interface from where I would normally migrate. Even un-archiving does not make it discoverable again. Before I create more havoc, could anyone tell me whether I should try to migrate the projects first and only then export and archive, or if creating a new project in Renku 2.0 while using the old repo and requirements.txt is easily possible? Thanks for your help!
Actually, the archiving alone was not the problem, as most of my projects in v1 lead to a 404 when I click on them. I guess there is not much point in migrating, I may as well start from scratch. 
What does the migration do at all? Just create the project slug in renku 2.0 and copy across the project description if there is one?
Hi @schymans you should first migrate in Renku from v1 → v2 and then arhive in gitlab.renkulab.io. I am sure that is the source of the issues.
Renku v1 is very closely tied to gitlab so if you archive your project in gitlab then it will essentially not exist any more in v1. So for the rest of the projects try what I suggest please.
For the project that you already archived can you share a link so I take a look and see how I can help it show back up? I will send you a direct message so you dont share the link publicly here.
This should not happen. But if you have archived all your projects in Gitlab then it may. How are you accessing your projects in v1 for which you get 404?
What does the migration do at all?
It will create a new project in Renku v2, copy the description, also add the gitlab repository that held your v1 code as a repository in your v2 project. But this is of limited use since the gitlab will be also decomissioned.
The really useful thing about actually doing the migration is that we have a record that the migration was done. And we are currently working on implementing redirects from the v1 renku project pages to the corresponding v2 project pages. So that if you have links to your old projects they will still work after we decommission renku v1.
If you are getting 404 pages from the v1 search page, try to use this page instead: https://renkulab.io/v1/projects
This should list all your projects from GitLab.
Oh, thanks, I didn’t realise this would give different results! Now, it seems to work. Thanks a lot for the description, too! I think I can just create new projects in renku 2.0 for those that I already archived but still want to keep.
That’s interesting, but what sort of links do you mean?
I am not sure if it makes sense to migrate every project, as with the loss of the whole lineage and workflow functionality, I will probably only keep a few projects on renku 2.0. Essentially, only those that I want collaborators to be able to execute without installing the requirements on their own machine.
By links I mean the link to your project. For example a v1 project can have a link like this: https://renkulab.io/projects/tasko.olevski/test-37
When we decommission v1 then that link will stop working. But if you migrated that project to v2 we will redirect if anyone visits https://renkulab.io/projects/tasko.olevski/test-37 to the correct link on the v2 side.
This is especially important to some users if they have published papers or documents where the link to the renku project cannot be updated. We will do a similar thing with the gitlab projects after we shut down the gitlab. But the mapping of the redirects will be done manually. There will be a form where users will be able to submit requests for such redirects and we will put them in place.
Hope the explanation makes sense.
Also the 404 problem you encountered is a bug. We already have a fix but we juts have to roll this out. I initially thought we had already rolled out the fix but we have not.
This is stored in the git repository. So if you move the repo over to another place (i.e. github.com or gitlab.com) then all the data about lineage is still there. You will not be able to see it in the web ui though. But you could have an environment that has the renku cli where you should be able to still use the cli to work with the lineage and run workflows. Lineage across different repositories and projects will not work though. And the renku cli is not actively supported anymore. So yeah it is not great. But if you want to still salvage some data or extract it then you could do that with a custom session environment that containts the renku cli. Some subset of the renku cli functionality should still work. Although I have not tested this so there may be some more issues than what I outline here.
Quite a few of my old projects cannot be migrated, either because they need to be updated first or because “Container image not available, it does not exist or is currently building.”
I assume both relate to the same problem, an outdated image in the dockerfile. Is there a standard image I could use to fix that? Not sure why this blocks migration, if all the migration does is update some links.
Here is an example: Reproducible Data Science | Open Research | Renku
Since V2 Renku projects don’t have the same mechanism to build images on every commit (mind that the Renku project is no longer closely bound to a specific repository), the migration also creates a new session launcher based on the latest image available for the V1 project.
Before turning off the V1 sessions, you could rebuild the image from the session tab. Now, the quickest way is to go into the GitLab project’s page and go into the “Jobs” section (E.G: Jobs · BA_MATH_GEN-16 / ET_data · GitLab ) and manually re-run the last job to create the image.
Mind that it might still fail for other reasons – E.G. you recently posted this problem.
As @lorenzo mentioned this blocks migration because we try to create a session launcher with the image from your most recent commit of from your v1 project. Since the image for that project fails to build we cannot create a session launcher and we fail the migration.
If you want to use a standard image you can try this: ghcr.io/swissdatasciencecenter/renku/py-datascience-jupyterlab:2.6.2 It has jupyterlab and commonly used data science python packages.
So to get unstuck I think you should try the following:
- go to your v1 renku project ( Reproducible Data Science | Open Research | Renku )
- pin the image to be
ghcr.io/swissdatasciencecenter/renku/py-datascience-jupyterlab:2.6.2 - trigger a v1 → v2 migration (this should now succeed)
