Hello,
I am trying to install the pyGIMLi library in a Renku project which is a MnWE (Minimal not-Working Example) of my original problem.
I first tried installing pyGIMLi by modifying the environment.yml so that the CI/CD pipeline automatically initiates a Docker build upon pushing changes. After I failed I went over several iterations of the environment.yml but didn’t succeed.
I also tried simply executing the following from the command line :
None of the above worked for me. Thanks for any inputs.
Hi @champost - do you have an example of a conda environment.yml file that works on your local machine?
Hi @rrrrrok,
I did quite a bit of testing since posting my issue. My trials consisted of various build scenarios and making use of distinct Renku projects (all based on the Basic Python template).
I have come to the conclusion that it is the defaults channel specified in the default environment.yml file provided by the Basic Python template which is the source of the problem.
I am not sure whether I am running into some sort of channel collision issue here. In any case, I don’t understand the reason for specifying a defaults channel when all the packages listed in a Basic Python template either come from the conda-forge channel or via pypi.
- For starters, here is what the
environment.yml file looks like in a Basic Python template
name: "base"
channels:
- defaults
dependencies:
# - add packages here
# - one per line
prefix: "/opt/conda"
- Here is an example where the build fails (e.g.
ERROR: Job failed: execution took longer than 2h0m0s seconds after increasing the default build timeout)
name: "base"
channels:
- defaults
- gimli
dependencies:
# - add packages here
# - one per line
- pygimli=1.3.0
prefix: "/opt/conda"
- Here is an example where the build succeeds !
name: "base"
channels:
- gimli
dependencies:
# - add packages here
# - one per line
- pygimli=1.3.0
prefix: "/opt/conda"
Note that the whatever combination I tried, the removal of the defaults channel from the environment.yml file always resulted in a successful build.
Does the above make any sense to you ?
It only makes sense insofar as I know that conda can be fickle 
I tried with this last option and it seems to build fine and works on renkulab afaict.
@rrrrrok
A follow-up on your offline suggestion to use mamba instead of conda for a more efficient CI/CD build on Renkulab, I did try and it works like a charm !
For all others, mamba is already part of the default Python template. Output of conda list | grep mamba :
libmamba 0.23.1 hd8a31e3_1 conda-forge
libmambapy 0.23.1 py39hd55135b_1 conda-forge
mamba 0.23.1 py39hfa8f2c8_1 conda-forge
I transformed only the conda env update part in my Dockerfile into a mamba env update (keeping all other conda’s intact) and that was it :
RUN mamba env update -q -f /tmp/environment.yml && \
/opt/conda/bin/pip install -r /tmp/requirements.txt && \
conda clean -y --all && \
conda env export -n "root"
1 Like
@rrrrrok (and all those interested)
Since the Renku Dockerfile now makes use of mamba instead of conda for more efficient builds, note that the Mamba documentation warns against mixing the Ananconda defaults channel and the popular conda-forge channels" !
I am also beginning to wonder whether this incompatibility between the defaults and conda-forge channels extends to other channels as well (e.g. gimli, see above).
In any case, as the Ananconda defaults channel is proposed by default in the Renku environment.yml, it may be useful to warn Renku users of incompatibility issues with a comment…
1 Like
Thank you @champost for the additional details.
Hi @champost that’s a good point, thanks! I hesitate to disable the defaults channel completely, but a warning there seems reasonable. Extra points if you’re able to send a PR!
Thanks @champost!! Merged, will be part of the next templates release
1 Like