Failing image_build

Somehow my renku repositories all report errors in the builds, for example here:

I tried updating renku, but that didn’t help. Do you know what goes wrong? It is something general, as I have it in multiple repositories.

Hi Remko,

We have encountered this problem over the last few weeks - it is due to an external dependency on rdfjson-lib. There is a quick fix here: Error during Environment creation - #10 by ralf.grubenmann where you can modify the Dockerfile slightly. It should solve the problem for that repository. Hopefully the fix is small enough to propagate to the other repos where it is erroring.

Let us know if that works!

Gavin

hi @rcnijzink ,

we have made a series of updates to our templates (including a fix for that failing external dependency). If you could edit your Dockerfile to match our newest dockerfile for python it would be great.
You can apply the same fix to your other repositories as needed, taking into account the type of project (R, python, bioc).

Best,
Pamela

Thanks! But the Docker image still fails unfortunately, but at a different point now:

Or with the other new Dockerfile:

So this issue was originally introduced by a breaking change in setuptools 58 that broke quite a few old packages. We fixed it in the renku 0.16.1.post1 hotfix and also collaborated with some downstream dependencies so they could fix it, which they did.

Only pyshacl fixed it in version 0.17.0 but then they broke it again in 0.17.0.post2 where they reintroduced the obsolete rdflib-jsonld library. Which in turn unfixed our hotfix release.

There is another issue where the templates have been updates for a new feature that will be included in the next version of renku, but the fallback that we provided for the templating engine didn’t work, that’s why you’re getting the new error.

Your Dockerfile currently looks like

[...]
ARG RENKU_VERSION={{ __renku_version__ | default("0.16.1.post1") }}

########################################################
# Do not edit this section and do not add anything below

# Install renku from pypi or from github if it's a dev version
RUN if [ -n "$RENKU_VERSION" ] ; then \
        source .renku/venv/bin/activate ; \
        currentversion=$(renku --version) ; \
        if [ "$RENKU_VERSION" != "$currentversion" ] ; then \
            pip uninstall renku ; \
            gitversion=$(echo "$RENKU_VERSION" | sed -n "s/^[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\(\.dev[[:digit:]]\+\)*\(+g\([a-f0-9]\+\)\)*\(+dirty\)*$/\3/p") ; \
            if [ -n "$gitversion" ] ; then \
                pip install --force "git+https://github.com/SwissDataScienceCenter/renku-python.git@$gitversion" ;\
            else \
                pip install --force renku==${RENKU_VERSION} ;\
            fi \
        fi \
    fi

The {{ __renku_version__ | default("0.16.1.post1") }} is a mistake, the templating engine should have replace that with 0.16.1.post1 in old versions of renku, but clearly it didn’t. We’ll look into that.

if you change the end of your Dockerfile to

ARG RENKU_VERSION=0.16.1.post1

########################################################
# Do not edit this section and do not add anything below

# Install renku from pypi or from github if it's a dev version
RUN if [ -n "$RENKU_VERSION" ] ; then \
        source .renku/venv/bin/activate ; \
        currentversion=$(renku --version) ; \
        if [ "$RENKU_VERSION" != "$currentversion" ] ; then \
            pip uninstall renku ; \
            gitversion=$(echo "$RENKU_VERSION" | sed -n "s/^[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\(\.dev[[:digit:]]\+\)*\(+g\([a-f0-9]\+\)\)*\(+dirty\)*$/\3/p") ; \
            if [ -n "$gitversion" ] ; then \
                pip install --force "git+https://github.com/SwissDataScienceCenter/renku-python.git@$gitversion" ;\
            else \
                pip install --force renku==${RENKU_VERSION} pyshacl==0.17.0.post1 ;\
            fi \
        fi \
    fi

it should work

I just updated the Dockerfile again with the last correction, but I still get an error, related now to the packages you mentioned:

ERROR: Cannot install pyshacl==0.17.0.post1 and renku==0.16.1.post1 because these package versions have conflicting dependencies. ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

That’s unfortunate behavior on the part of pip/pipx. When I tried it locally it complained but still installed.

We’re working on a hotfix renku version, 0.16.2 that fixes this (again…), which should be released relatively soon.

We have a 0.16.2 release that should fix it. so you can just change the Dockerfile to read

ARG RENKU_VERSION=0.16.2

Let me know if that works for you.

Yes, that worked indeed! Thanks!