Renkulab 0.7.0 release

Announcements
1

Last week on Friday we released Renku v0.7.0 and over the weekend we updated https://renkulab.io and https://limited.renku.ch to this version. This release brings a number of big changes to the platform - most noticeable in the web UI are the handling of datasets and project version upgrades. The changes in the backends are substantial so please bear with us as we iron out the kinks over the next few days. Please let us know of any issues you encounter either in #general or make in issue on GitHub.

New features

  • Project templates: you can now create custom templates for your projects and use them on project initialization. Great for groups or courses! See the docs for more details.
  • Datasets: you can now search for datasets and import them into projects directly from the Renkulab web UI. The dataset import and creation flow has also been much improved thanks to changes in the core service backend. Note that importing datasets from other renku projects from the UI currently works only for datasets in public repositories.
  • Data import: files can be imported into datasets directly from a URL.
  • Project migration: the metadata of renku projects occasionally changes. In order for all the components to work well together, the metadata must be kept in sync. Previously you needed to do this manually with the CLI but now it’s as easy as clicking a button.
  • Template simplification: We have decoupled the CLI version from the base image and made it easier to override in your own environments. See the project template README for details.
  • Improved editor: All text editing components are using an enhanced editor that allows seamless switching between WYSIWYG and markdown.
  • renku save: the Renku CLI now features a save command that simplifies the process of committing and pushing your project to the server.

Project migrations

We have built a more systematic way of handling metadata, which means that we must insist on some level of uniformity when serving projects in the web UI. As a result, your project’s metadata may need to be updated from now on when we release new versions. If that is the case, you will see a yellow warning icon on the main project page:

image

Clicking on the warning icon will give you the option to try and migrate the project automatically, and provide instructions for doing it yourself. Note: for large projects we recommend that you do this manually to have better oversight of the process!

Migrating the metadata is only one part of the process - you will also want to update the version of renku-python used in your project’s environment to renku==0.11.6. You can do this by using the new Dockerfile from either the new python or R templates, or you can install the new version of renku-python in your image with

pipx install --force renku==0.11.6

We’ve tried to make sure that upgrading projects will be as simple and painless as possible, so if you experience difficulties or you are not sure how you should update your project, please drop us a line in #general. Note that we are presently implementing further improvements to this process so that it will require very little manual intervention for most projects in the near future.

Other Improvements

  • Dataset deletion: Datasets used to stick around even after getting deleted from the project. This has now been fixed and deleted datasets no longer appear in the dataset listings.
  • git credentials: the interactive sessions now handle your git credentials in a way that allows you to seamlessly access any of your private repositories on renkulab from within an interactive session.
  • git hooks in interactive sessions: git hooks were not previously installed per default in interactive sessions, which meant that some nice features like automatically pushing large files to LFS did not work correctly. This has now been corrected and should hopefully save many repositories from improperly handled data!
  • graph redesign: Under the hood, the renku-python library has a completely redesigned knowledge graph model. This enormous effort doesn’t translate to user-visible improvements yet, but they’re coming in the next release!

Many other improvements and bug fixes across all of the renku components, which have significantly improved the stability of the entire platform!

Changes and improvements in the platform deployment

  • Breaking change: Postgresql chart: we have switched to the Bitnami version of the Postgresql helm chart - this requires a manual intervention when upgrading from renku 0.6.8.
  • Kubernetes: renku is now compatible with kubernetes >= 1.16 (tested on 1.16)
  • Helm3: the renku helm charts are now compatible with helm 3
1 Like