Community-contributed project templates

1

By default, renku projects are initialized using our basic templates collection but many Renku users come up with interesting and useful project configuration that can be very useful also for others. This is why we support custom templates, which allow anyone to turn a publicly-available git repository into a source of templates. In order to pull together a curated collection of such useful “expansion” templates, we have created a contributed project templates repository, where the Renku community can share and exchange useful templates. Our first two examples are a template for boostrapping a project that connects to AiiDA (thanks to Leo Talirz) and a template that adds a basic streamlit configuration to the project (thanks @LiliGasser!), making it immediately dashboard-ready.

Contributing to community templates

We would be thrilled if you wanted to contribute a project template to the contributed template repository! If you are not sure if your idea for a template qualifies or if you don’t know how to turn your project configuration into a template, please reach out either as a comment to this topic, by opening an issue or a pull request in the template repository.

If you simply wish that a template for a certain kind of project existed, we would also like to know! Let us know either here or in the contributed project templates issue page.

How do I use the custom templates?

You can use a template from this community-contributed collection (or from any other source) when you initialize a renku project, whether via the web UI or the command-line tool.

Using the web UI

Using custom templates from the web-UI is quite straightforward. On the new project page, under Template Source select Custom, enter the URL of the template repository (in this case https://github.com/SwissDataScienceCenter/contributed-project-templates) and give a valid repository reference (typically you would use a tag or the master or main branch):

image
image935×298 21.4 KB

Once this is entered, you can click on Fetch templates and select which template you would like to use. Depending on the template, it may ask you for additional information here, e.g. in the case of the “streamlit” template we are asked to enter the project description.

image
image943×330 16.7 KB

Once you are satisfied, click Create project and a new project using the template will be created.

Using the renku CLI

In the CLI you can choose a template when making a new project by specifying the template source using a flag to renku init:

renku init --template-source \
https://github.com/SwissDataScienceCenter/contributed-project-templates \ 
--template-ref main \
test-template

Fetching template from https://github.com/SwissDataScienceCenter/contributed-project-templates@main...
OK
  INDEX  ID                        PARAMETERS
-------  ------------------------  -----------------------
      1  aiida                     description,archive_url
      2  python-minimal-streamlit  description
Please choose a template by typing the index:

After choosing the template you want to use, you will be asked to provide information about any of the additional parameters that the template requires.

That’s it! I hope you’ll find these new templates useful and that you might in the future contribute your own!

2 Likes
2

Hi @rrrrrok,

I am unable to Fetch templates into Renkulb from the contributed-project-templates repository. I get the following message :

image
image908×663 43 KB

1 Like
3

Hi @champost yes, we need better URL handling here… but it’s because of the trailing / - if you remove it, it will work. I’ll make an issue for this so we can fix it.

1 Like
4

I can confirm that It works without the trailing /.

5

Perfect, thanks! I’ve opened up an issue to fix this, will hopefully be done in the next sprint. Thanks for the reminder!

6

I’m teaching myself the Renku templating system and I made a modified version of your simple minimal R template that uses Renv for the package management. Renv is very nice for providing exact package versions for a more completely reproducible R environment and it also conveniently provides a unified interface for installing packages from bioc & github repos. I’m not sure if you you’d be interested in including it in the default templates or if not if you’d be open to a PR for including it in the community contributed repos?

I made it in a fork of the default templates and added it in there as I was following along the with template tutorial but I can move it to community if you think it’d be better placed there.

1 Like
7

Hi @RichardJActon that’s great, thanks a lot! We looked at Renv when setting up the initial R templates but it wasn’t obvious how widely used it was at the time. I myself am not too familiar with the trends in R world - is Renv gaining in acceptance? With our python templates we let users mix and match between conda and pip - is something like that possible here? Could we simply add an Renv configuration to the existing R templates?

8

It’s definitely gaining traction (at leas I think so), still a minority of users at the moment but it has the backing of RStudio and it has a number of advantages so I’d expect to see more of it. I’ve taught it to some people as it’s the best tool for (largely) automating reproducible R environments in my view.

It does not really work that well to try and mix Renv with base package management / Bioconductor / remotes. It kind of sits on top of these existing package management solutions and provides an abstraction layer that lets you use them all. With the added versioning features. I’m less familiar with the python ecosystem but I’m given to understand that conda environments and virtual environments don’t really play nice if you try and use them at the same time this might be a bit similar.

There are a couple of approaches to using Renv with docker:

You can either use it to build your image with the packages pre-installed or you can restore the Renv environment after the image is built with a baseline R install. My current implementation takes the first approach as that is what was most similar to the existing install.R approach implemented in your minimal R template.

However, if you do the post-build install & you have some storage you can share between projects you can take advantage of a nice feature of renv’s architecture. It creates a central library of R package installs on your system then symlinks to the packages within each project so you only have one copy of any given version of a package installed on your system even if you use it in many projects. I was able to get this working in Docker on my system by mounting the central library to my project specific containers. Then within the container letting Renv link the project library to the mounted central library. This make package installs almost instant if the requested version is already in the central library. Any new package versions requested are automatically copied to the central library so they can be re-used in any other projects. A central R package library that could be mounted to containers for rapid package installs would be an interesting feature to consider in the long run.

9

@RichardJActon it would be great if you could go ahead and make a PR from your branch in the main template repo. Lets continue the discussion there as it is already getting a bit too technical for this forum topic :slight_smile: Thanks again for taking the initiative on this!

10

here is the PR: Adding and Renv template by RichardJActon · Pull Request #148 · SwissDataScienceCenter/renku-project-template · GitHub

1 Like