I am considering to deploy renku on a workstation that is used by multiple users. How does renku allocate resources? Is it supported to run multiple renku sessions on a single machine?
At the moment we all run individual jupyter-lab sessions there and share the resources: CPUs, GPUs and RAM.
When running a renku session do the resources get preallocated? Does one define how much resources to allocate? Or does each of the multiple renku sessions have access to all resources, and they all share them like regular processes would?
Hello!
There are two ways one might “deploy” Renku on a workstation:
-
Install the Renku CLI (python package).
-
Deploy the RenkuLab application (requires a Kubernetes cluster)
For (1), Installing the Renku CLI means installing a command-line tool for starting sessions on your own machine. In this mode of using Renku, you use your own hardware rather than the resources provided on RenkuLab.io. Installing the Renku CLI is as simple as installing the Renku python package (pip install renku) and Docker. Then, you can clone your RenkuLab projects from RenkuLab.io on your own machine (renku clone <url>) and start the session on your own machine (renku session start). This uses the same Docker image you were using on RenkuLab on your own machine, so once the session is running, the environment (Jupiter Lab) looks exactly the same. Our full documentation for getting started with the Renku CLI on your own workstation is here: Renku on your Own Machine — Renku documentation
In this mode, each user on the workstation may install the Renku CLI in their account. Resource sharing is the same as running any other kinds of processes on multiple accounts at the same time, such as multiple users running python processes at the same time.
For (2), deploying a RenkuLab instance means you host your own version of RenkuLab on your own hardware (instead of using RenkuLab.io). This is a much more involved undertaking compared to (1)! Our documentation for deploying the RenkuLab application is here: Deploy Renku — Renku documentation If this is an option you would like to pursue, I’d be happy to connect you with our deployment team lead!
In this mode, the resources for each user’s session are determined by the session settings the user selects in the application when they start a session, and these resource requests are passed to Kubernetes.
Let me know if you have any further questions!