Failed pipeline after migration to latest version fro Renku

Dear Renku-developpers,

In one of my Renku-project, the image cannot be built anymore since I updated the project:
renkulab.io: migrate to latest version for renku 2.9.1
(and same now when trying a new update: renkulab.io: migrate to latest version for renku 2.9.2)

Should I try to go back to a previous version of Renku?

My project is private (called joint-inv). Please let me know what other information you need to be able to help with this issue.
Many thanks!
Coline

What exception do you get in the pipeline? To my knowledge, there’s not really anything in 2.9.2 that would cause something like this, but the update might have updated your Dockerfile from the project template as well and that might be causing this.

I find this information linked with the message “failed pipeline” (is it waht you were asking?):

#7 234.3 ERROR: Could not find a version that satisfies the requirement postgresql-wheel (from versions: none)
#7 234.3 ERROR: No matching distribution found for postgresql-wheel
#7 ERROR: executor failed running [/bin/bash -c -o pipefail mamba env update -q -f /tmp/environment.yml &&     /opt/conda/bin/pip install -r /tmp/requirements.txt --no-cache-dir &&     mamba clean -y --all &&     mamba env export -n "root" &&     rm -rf ${HOME}/.renku/venv]: exit code: 1
------
 > [stage-1 3/4] RUN mamba env update -q -f /tmp/environment.yml &&     /opt/conda/bin/pip install -r /tmp/requirements.txt --no-cache-dir &&     mamba clean -y --all &&     mamba env export -n "root" &&     rm -rf /home/jovyan/.renku/venv:
------
executor failed running [/bin/bash -c -o pipefail mamba env update -q -f /tmp/environment.yml &&     /opt/conda/bin/pip install -r /tmp/requirements.txt --no-cache-dir &&     mamba clean -y --all &&     mamba env export -n "root" &&     rm -rf ${HOME}/.renku/venv]: exit code: 1
Cleaning up project directory and file based variables
ERROR: Job failed: exit code 1

It sounds like it can’t install this python package. I don’t think this is installed by renku, so it sounds like something is off with your dependencies in either requirements.txt or environment.yaml.

postgres-wheel is not available for python>=3.10. I suggest you switch your base docker image to one that uses python 3.9. To do that, you have to change the base image in two places in the Dockerfile; you have two FROM lines that say probably something like

FROM renku/renkulab-py:3.10-0.21.0

Simply change 3.10 to 3.9 in both places and you should be good to go!

Ok, thanks @ralf.grubenmann and @rrrrrok,
I just removed the “postgresql-wheel” package, as I actually do not need it. And it solved the problem of the failed pipeline. The pipeline passed successfully now.
But, I now get another issue when I open the project :

Build Recommended
JupyterLab build is suggested:
@plotly/dash-jupyterlab needs to be included in build
jupyterlab-dash needs to be included in build

And when I try to click on the “build” button, then I get the following error:

Build failed with 502.

        If you are experiencing the build failure after installing an extension (or trying to include previously installed extension after updating JupyterLab) please check the extension repository for new installation instructions as many extensions migrated to the prebuilt extensions system which no longer requires rebuilding JupyterLab (but uses a different installation procedure, typically involving a package manager such as 'pip' or 'conda').

        If you specifically intended to install a source extension, please run 'jupyter lab build' on the server for full output.

I then searched on the forum and found this Dash app with jupyter-server-proxy - #17 by tolevski but in my case werkzeug is already downgraded to 2.2.1.
Concerning the rest I read through on the post Dash app with jupyter-server-proxy - #23 by rrrrrok, I am not sure what I should try to solve it.
A consequence is that I don’t see anymore the git user-friendly interface (I can only commit through the terminal).

Hi @colinem I’m glad you fixed (one of) the problems! :slight_smile:

For the second one, have you added this line to your Dockerfile? For most JupyterLab extensions it’s not needed anymore, but apparently in this case it is.

Thanks! No I have not this line included yet.

The end of my Docker file is:

# install the python dependencies
COPY requirements.txt environment.yml /tmp/
RUN mamba env update -q -f /tmp/environment.yml && \
    /opt/conda/bin/pip install -r /tmp/requirements.txt --no-cache-dir && \
    mamba clean -y --all && \
    mamba env export -n "root" && \
    rm -rf ${HOME}/.renku/venv 
 
COPY --from=builder ${HOME}/.renku/venv ${HOME}/.renku/venv

Should I try to remove the 2 last lines and add jupyter lab build instead or should I try to add it in addition?

you can have it like this:

# install the python dependencies
COPY requirements.txt environment.yml /tmp/
RUN mamba env update -q -f /tmp/environment.yml && \
    /opt/conda/bin/pip install -r /tmp/requirements.txt --no-cache-dir && \
    mamba clean -y --all && \
    mamba env export -n "root" && \
    rm -rf ${HOME}/.renku/venv  && \
    jupyter lab build

COPY --from=builder ${HOME}/.renku/venv ${HOME}/.renku/venv

(a side note): when copy/pasting code into a message here, use the three back-ticks to create a fenced code block to make it easier to read. :raised_hands:

I tried it, and now at the session opening, I get a window saying:

Error Registering Plugins

Plugin 'jupyterlab_dash' is already registered

jupyter-dash is one of the package in the requirements.txt file.
Any further idea to solve it? (the git button is still missing from the jupsterlab interface)

Hmm… if you can point me to your project that would be a big help, I’m not really sure what to suggest otherwise. If it’s a private project and you can add me, my username is rok.roskar.