CI/CD Runners - something has changed?

RenkuLab
21

Hi all,

I’ve been in touch with some of you around this so you somehow know the story.

We’re still looking into this; it’s a non trivial issue and we don’t yet have a clear viable longer term solution.

For now, we advise building locally and pushing to renku docker registries as a workaround; while we understand this is suboptimal from a user experience pov, we need to dig a little deeper into the mechanics of conda before formulating a solution. We believe that conda builds when operating on a network of an educational institution should be fine so local builds should work - please let us know if this is not the case (it has been working for us).

We’re also starting to lean towards greater use of conda-forge as we believe this is less likely to give rise to future issues of this type; however, we’re not clear yet on whether things might break if folks switch from the default channel to conda-forge.

We will continue to update you as this evolves and we apologize for any inconvenience.

BR,
Sean.

22

@seanrmurphy @rrrrrok Thanks a lot for looking into this issue. An update: I switched to conda-forge, and the CI/CD runner works now (done within 15 min).

23

@Lin - thanks for your update here. Can you provide a little more detail? What specific steps did you take and can you provide a pointer to the project?

Best,
Sean.

24

@seanrmurphy In the environment.yaml, I changed the defaults channel to conda-forge. That’s the only thing I did. The project is private: git repo.

25

Great - thanks for that @Lin - I did the same on a fork of @mitch 's project and the build worked - he said he will test that everything is in order next week.

@firat remains an outlier - @tolevski was gonna reach out to him to see if we can find some solution which works in his case.

Current status: increasing evidence that removing the default channel and switching to conda-forge addresses this problem.

26

Thanks @seanrmurphy, can confirm that removing defaults in favour of only conda-forge fixed it. Now the image is building as expected. Will test if everything will run as expected, but the original problem is now solved. Thanks everyone!

1 Like
27

In case anyone comes to this thread in the future, here is a snippet that can help from an internal discussion.
If you are experiencing similar behavior in your local environment builds, try the following:

Remove defaults channel from both environment.yml and conda settings

  1. make sure defaults is not in environment.yml
  2. remove defaults from conda config
    conda config --remove channels defaults
  3. add conda-forge to conda config
    conda config --add channels conda-forge
  4. conda config --set channel_priority strict
1 Like