Last step in Dockerfile failing

RenkuLab
1

Upon pushing to renkulab.io, the last step of building docker image fails, complaining about pipx not found: link to ci/cd job

To be more precise the following line

RUN if [ -n "$RENKU_VERSION" ] ; then \

        currentversion=$(pipx list | sed -n "s/^\s*package\srenku\s\([^,]\+\),.*$/\1/p") ; \

        if [ "$RENKU_VERSION" != "$currentversion" ] ; then \

            pipx uninstall renku ; \

            gitversion=$(echo "$RENKU_VERSION" | sed -n "s/^[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\(\.dev[[:digit:]]\+\)*\(+g\([a-f0-9]\+\)\)*\(+dirty\)*$/\3/p"); \

            if [ -n "$gitversion" ] ; then \

                pipx install --force "git+https://github.com/SwissDataScienceCenter/renku-python.git@$gitversion" ;\

            else \

                pipx install --force renku==${RENKU_VERSION} ;\

            fi \

        fi \

    fi

is causing error

Traceback (most recent call last):
  File "/opt/conda/bin/pipx", line 5, in <module>
    from pipx.main import cli
ModuleNotFoundError: No module named 'pipx'
Traceback (most recent call last):
  File "/opt/conda/bin/pipx", line 5, in <module>
    from pipx.main import cli
ModuleNotFoundError: No module named 'pipx'
Traceback (most recent call last):
  File "/opt/conda/bin/pipx", line 5, in <module>
    from pipx.main import cli
ModuleNotFoundError: No module named 'pipx'

on renkulab.io CI/CD jobs. Could I be doing something to cause this issue?

2

hi @firat,
what base image are you using for this project? We’ve recently changed the way we install the renku-cli into our base image. See this commit and this. Therefore, newer images do not have pipx installed anymore.

In this case, replacing the bottom section of your projects Dockerfile with

########################################################
# Do not edit this section and do not add anything below

# Install renku from pypi or from github if it's a dev version
RUN if [ -n "$RENKU_VERSION" ] ; then \
        source .renku/venv/bin/activate ; \
        currentversion=$(renku --version) ; \
        if [ "$RENKU_VERSION" != "$currentversion" ] ; then \
            pip uninstall renku -y ; \
            gitversion=$(echo "$RENKU_VERSION" | sed -n "s/^[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\(\.dev[[:digit:]]\+\)*\(+g\([a-f0-9]\+\)\)*\(+dirty\)*$/\3/p") ; \
            if [ -n "$gitversion" ] ; then \
                pip install --force "git+https://github.com/SwissDataScienceCenter/renku-python.git@$gitversion" ;\
            else \
                pip install --force renku==${RENKU_VERSION} ;\
            fi \
        fi \
    fi

########################################################

should do it.

3

Hi @andreas,
I’m using RENKU_BASE_IMAGE=renku/renkulab-py:3.8-0.8.0, I want to stick with python 3.8 mainly due to the workstation I’m running the project on supporting that at most.
The project is still using RENKU_VERSION=0.16.0, shouldn’t the recent changes not affect this project?
I regardless tried the new block you suggested, but it fails failing-job since there is no virtual env venv. I’m a bit confused about that one though, I’m not trying to containerize my project, I’m only referring to the default CI/CD that runs on renkulab.io when I push changes to the remote.

4

A quick update, removing

  - python==3.8
  - tensorflow==2.4

from environment.yml fixes it, but I think it only fixes it because then CI/CD just uses cache.

5

Ah ok. Your base image still contains pipx and the renku cli installed through pipx, so adding the snippet which I’ve pasted above doesn’t make sense at all.

Does building the image locally work?

6

When trying locally, make sure to pull the base image (we don’t retag those, but just to be sure) and use the --no-cache option.

7

I am not sure how to build the image locally. Also, the workstation does not have docker. I can clone the repo to my laptop and try that maybe next week sometime… thanks!

8

I can clone the repo to my laptop and try that maybe next week sometime

Yeah, I’d strongly recommend that. Debugging build pipelines is usually much quicker when you can build images locally than through CI jobs.

1 Like