Building renku docker images locally

1

Hello,
For the sake of time, I wanted to build renku v1 docker images locally so that I can quickly link to them on renku v2 sessions.
However, while Dockerfile locally, I get the following error:

[stage-1 7/8] RUN mamba env update -q -f /tmp/environment.yml && /opt/conda/bin/pip install -r /tmp/requirements.txt --no-cache-dir && mamba clean -y --all && mamba env export -n “root” && rm -rf /home/jovyan/.renku/venv:
0.483
0.483 CondaError: Error encountered while attempting to create cache directory.
0.483 Directory: /home/jovyan/.cache/conda/notices
0.483 Exception: [Errno 13] Permission denied: ‘/home/jovyan/.cache/conda’

This does not happen when images are built on renkulab CI/CD runners.

What do I need to do to fix this?

In case it’s useful, below is the full dockerfile I am using:

########################################################
#        Renku install section - do not edit           #

FROM renku/renkulab-cuda:12.2-0.24.0 AS builder

# RENKU_VERSION determines the version of the renku CLI
# that will be used in this image. To find the latest version,
# visit https://pypi.org/project/renku/#history.
ARG RENKU_VERSION=2.9.2

# Install renku from pypi or from github if a dev version
RUN if [ -n "$RENKU_VERSION" ] ; then \
        source .renku/venv/bin/activate ; \
        currentversion=$(renku --version) ; \
        if [ "$RENKU_VERSION" != "$currentversion" ] ; then \
            pip uninstall renku -y ; \
            gitversion=$(echo "$RENKU_VERSION" | sed -n "s/^[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\(rc[[:digit:]]\+\)*\(\.dev[[:digit:]]\+\)*\(+g\([a-f0-9]\+\)\)*\(+dirty\)*$/\4/p") ; \
            if [ -n "$gitversion" ] ; then \
                pip install --no-cache-dir --force "git+https://github.com/SwissDataScienceCenter/renku-python.git@$gitversion" ;\
            else \
                pip install --no-cache-dir --force renku==${RENKU_VERSION} ;\
            fi \
        fi \
    fi
#             End Renku install section                #
########################################################

FROM renku/renkulab-cuda:12.2-0.24.0

# Uncomment and adapt if code is to be included in the image
# COPY src /code/src

# Uncomment and adapt if your R or python packages require extra linux (ubuntu) software
# e.g. the following installs apt-utils and vim; each pkg on its own line, all lines
# except for the last end with backslash '\' to continue the RUN line
#
USER root
RUN apt-get update --fix-missing
RUN apt-get install -yq --no-install-recommends \
    bzip2 \
    ca-certificates \
    curl \
    git \
    gpg-agent \
    rclone \
    vim && \
    apt-get purge && \
    apt-get clean && \
    apt-get autoremove --yes 

RUN apt-get update && \
    apt-get install -y build-essential htop ncdu screen vim wget git-lfs rsync \
    apt-utils binutils 
RUN apt-get install -y unzip zip curl wget 
USER ${NB_USER}

# install the python dependencies
COPY requirements.txt environment.yml /tmp/
RUN mamba env update -q -f /tmp/environment.yml && \
    /opt/conda/bin/pip install -r /tmp/requirements.txt --no-cache-dir && \
    mamba clean -y --all && \
    mamba env export -n "root" && \
    rm -rf ${HOME}/.renku/venv

COPY --from=builder ${HOME}/.renku/venv ${HOME}/.renku/venv
2

Hi Firat,

Are you on a machine with Apple silicon?

If yes, then that might be the issue. It seems that a ~/.cache/rosetta directory is created whenever a RUN command is executed. In your Dockerfile, the first RUN command is executed under root user and this directory will be created and owned by root. Later, when running mamba env update -q -f /tmp/environment.yml under the jovyan user, it doesn’t have permission to create ~/.cache/conda because ~/.cache is owned by root.

To fix this issue, you can either delete ~/.cache or change its owner, right before switching to ${NB_USER}:

...
RUN apt-get install -y unzip zip curl wget 
RUN rm -rf /home/jovyan/.cache
USER ${NB_USER}
...

or

...
RUN apt-get install -y unzip zip curl wget 
RUN chown -R jovyan /home/jovyan/.cache || true
USER ${NB_USER}
...

This change shouldn’t affect your CI/CD build and it should work as before.

Please let me know if the issue still exists.

Cheers,
Mohammad

3

Thanks for the suggestions, I tried the latter and it works!

Yes, I build on apple silicon, but I pass --platform linux/amd64 parameter to docker build. So that’s not good enough.

A request:
Perhaps you can add this line
RUN chown -R jovyan /home/jovyan/.cache || true
into the template Dockerfile created by each new Renku project as well. I assume many people will have the same problem.

1 Like