Renkulab-py for python 3.8

RenkuLab
1

Hi everyone,

for a project I am looking for a renkulab docker image with python 3.8 but I end up getting 3.9. My Dockerfile looks like this:

ARG RENKU_BASE_IMAGE=renku/renkulab-py:3.8-1b2f470
#ARG RENKU_BASE_IMAGE=renku/renkulab-py:3.8-3bd4628
FROM ${RENKU_BASE_IMAGE}
########################################################
#        Renku install section - do not edit           #

# RENKU_VERSION determines the version of the renku CLI
# that will be used in this image. To find the latest version,
# visit https://pypi.org/project/renku/#history.
ARG RENKU_VERSION=2.3.2

# Install renku from pypi or from github if a dev version
RUN if [ -n "$RENKU_VERSION" ] ; then \
        source .renku/venv/bin/activate ; \
        currentversion=$(renku --version) ; \
        if [ "$RENKU_VERSION" != "$currentversion" ] ; then \
            pip uninstall renku -y ; \
            gitversion=$(echo "$RENKU_VERSION" | sed -n "s/^[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\(rc[[:digit:]]\+\)*\(\.dev[[:digit:]]\+\)*\(+g\([a-f0-9]\+\)\)*\(+dirty\)*$/\4/p") ; \
            if [ -n "$gitversion" ] ; then \
                pip install --force "git+https://github.com/SwissDataScienceCenter/renku-python.git@$gitversion" ;\
            else \
                pip install --force renku==${RENKU_VERSION} ;\
            fi \
        fi \
    fi
#             End renku install section                #
########################################################

# Uncomment and adapt if code is to be included in the image
# COPY src /code/src

# Uncomment and adapt if your R or python packages require extra linux (ubuntu) software
# e.g. the following installs apt-utils and vim; each pkg on its own line, all lines
# except for the last end with backslash '\' to continue the RUN line
#
USER root
# RUN apt-get update && \
#    apt-get install -y --no-install-recommends \
#    make \
#    apt-utils \
#    latexmk \ 
#    texlive-latex-recommended \
#    texlive-fonts-recommended \
#    texlive-latex-extra \
#    texlive-fonts-extra \
#    texlive-lang-all \
#    tex-gyre
USER ${NB_USER}

# install the python dependencies
# COPY requirements.txt environment.yml /tmp/
# RUN conda create -ny py38 python=3.8
# RUN conda activate py38
# RUN mamba env update -q -f /tmp/environment.yml && \
#     code-server --install-extension ms-python.python
# RUN /opt/conda/bin/pip install -r /tmp/requirements.txt && \
#     mamba clean -y --all && \
#     mamba env export -n "root"

As you can see, I have commented out everything except for what I understand to be the code loading the image itself. With both images I have tried in the first two lines, putting

python --version

gives me 3.9. Is there a trick to getting 3.8?

2

Hi @dennis-beermann - can’t seem to reproduce this - here’s what I tried:

$ docker run --rm -ti renku/renkulab-py:3.8-1b2f470 bash
base ▶ ~ ▶ $ ▶ python --version
Python 3.8.13
base ▶ ~ ▶ $ ▶ which python
/opt/conda/bin/python
3

Are you building the image locally? Or using the one that is built on renkulab? Just trying to eliminate options… and where/how are you checking the python version, inside a running session on renkulab or locally?

4

Thanks for the fast reply @rrrrrok . I am using the above Dockerfile in a Renku project (Reproducible Data Science | Open Research | Renku, you’ve taken a look at that before :slight_smile:). You are right that the image itself has python 3.8, but in the browser terminal it definitely has 3.9. My suspicion is that a) something happens during the Renku build process that changes things up somehow, or (much more likely) b) I am misunderstanding something entirely …

5

So just to clarify things: I changed the Dockerfile, commited the changes to trigger the GitLab pipeline (which went through without errors) and then it still says 3.9

6

I just started a session in that project and got python 3.8. How are you starting the sessions? Could you try going to the “Start with options” page and make sure it’s launching from the correct commit?

image

7

The correct commit was selected, yes. But when I clicked on “Start with Options”, it notified me that there was an autosave detected. I then deleted said autosave and now I finally have 3.8 as well. First of all, thank you very much!!

Although I would like to understand this better so it won’t trouble me again in the future, is there like an “official way” to start the session correctly so changes actually take effect? My process so far:

  1. Make changes to the Dockerfile & stage-commit-push
  2. Wait for the pipeline to go through
  3. Stop and restart the session (so far without any options, as mentioned)

Also, as a sidenote: Your support quality is amazing

8

ok I thought the auto-save may have been an issue. We are actually working on a release right now that will remove that problem forever… Glad it got sorted!!

You certainly didn’t do anything wrong, but Renku doesn’t inform the user properly in all cases. That’s why we are currently rebuilding this functionality from scratch. In general your procedure is fine and we should be able to streamline it further soon.

However, for working on and debugging a docker image, I usually prefer to do things locally until I am sure they work. It makes the feedback loop much shorter.

11

Hey @dennis-beermann, just got around to looking into this again. I’ve gotten this environment.yml to work:

name: base
channels:
  - conda-forge
dependencies:
  - code-server
  - pip:
    - jupyter-vscode-proxy
prefix: "/opt/conda"

You can see an example here.

Note: I had to install the proxy via pip instead of conda. Not really sure why that matters, but the launcher is there and all works as expected.

Note2: the image with python 3.10 has a problem upstream (see [BUG] - nodejs pinned in python-3.10 image · Issue #1990 · jupyter/docker-stacks · GitHub) so you have to first remove the nodejs pin in /var/conda/conda-meta/pinned if you want to use that python version.