Hello,
I have 2 different conda environments setup within a renku environment. I am trying to launch jupyter notebook using the second conda environment, however I did not manage to switch kernel to the other conda environment. When I launch notebook from a terminal that already has the desired conda environment activated, I couldn’t manage to view the started jupyter notebook server.
Is it possible to do this?
Just to make sure I understand the situation: you have a renku project where you have defined two conda environments as part of your Dockerfile; you have jupyter installed in both environments and you want to be have kernels from both the conda environments available to you when you start a notebook from jupyterlab. Is that correct?
It turns out the issue was that the base image has conda installed with root privileges, which is problematic if I see that I need additional python libraries in the environment. So my workaround was that I created a new conda environment where I install all the python libraries I need.
While I can start a jupyter notebook server from terminal with this new conda environment, I don’t know how to view it. I got a suggestion that perhaps one needs to add additional port forwards besides 8888 in Dockerfile.
Anyhow, I have a temporary solution at the moment, so this is certainly not an urgent issue.
The default conda environment should be configured to let you add additional packages - if it doesn’t it’s an error we should fix.
@firat for using multiple conda environments in multiple kernels look at this project I just setup:
https://renkulab.io/projects/tasko.olevski/multiple-kernels
You see here for a diff of what changes I made on top of an empty project to make this happen:
With the above changes you can just pick which kernel you want to use in each notebook.
As for the issue with not being able to install the python packages without root privileges I am looking into it and will give you an update soon.
Thanks, that looks straightforward!
@firat if I understood correctly you only made an extra environment because the default environment didn’t let you install packages, right? Or do you actually need multiple environments? I need to fix the new cuda/TF docker image to allow users to install packages anyway in which case you might not need to juggle multiple environments. Which package were you trying to install?
@rrrrrok yes exactly. I don’t need multiple environments.
Here are some of the packages:
I can confirm that building an image based on this dockerfile causes then subsequent image build to fail here.
I think the problem is in these lines which install the tensorflow python package as root and then switch back to the non-root user. I am testing a fix where the tensorflow package is installed as non-root. Will update soon.
indeed - I just fixed this and can confirm that it works. I pushed a fix to the PR - as soon as the CI is done running there, you can try the new base image @firat.
that’s awesome thanks! So it will still be the image at rrrrrok/renkulab-cuda-tf ?
Awesome! This solved it!
Perfect! Thanks for bringing up the issue in the first place! 