Using renku Docker image to execute tests in gitlab CI pipeline

RenkuLab
1

Hi all,

We want to add a test stage in our gitlab-ci pipeline and would like to run these within the same renku image that is built during the build stage.

Our gitlab-ci.yaml has the following additions:

  • We add an additional tag to the built image with --tag CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA`.
  • We add a test step which uses that particular image. The test script simply echoes a message for now.

We are able to build the image and pull it for the test stage with the added tag, however, it fails before being able to enter the test script with the following error message. I suppose this has to do with the entrypoint of the container, can you give us a hint on how to proceed further?

Many thanks!

renku_gitlab_ci
renku_gitlab_ci1356×599 42.8 KB

2

Hi @Damien which base image are you using? And can you share the full test step from your gitlab-ci file?

3

Hi,
Thanks for the quick reply.

Our base image is renku/renkulab-r:4.2.0-eb9419b and the test step is as simple as that for now:

test:
  image: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
  script:
    - echo 'Hello World'
4

When I run this image locally it works fine:

docker run --rm -ti renku/renkulab-r:4.2.0-eb9419b echo "hello world"

So at least we know it’s not a problem with the image :relieved:

It seems to be a bug in gitlab: helper command breaks build container with entrypoint script containing exec $(@} (#4140) · Issues · GitLab.org / gitlab-runner · GitLab

Luckily there is an easy fix - try this as your stage definition

test:
  image: 
    name: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    entrypoint: [""]
  script:
    - echo 'Hello World'
5

That indeed fixed the issue, thanks a lot!

1 Like
6

perfect, glad to hear it!

Just keep in mind that the entrypoint is setting up git and renku functionality - so if any of that is needed in your CI test job you might have to do it in an additional preceding step.

7

We are indeed using git lfs in our tests, did not have a chance to try it out yet though.

Is there a way to have a look at the script we are missing the execution with this hack?

(Sadly we don’t have access to docker locally nor to a remote kubernetes cluster so our only option is to try things out by updating the .gitlab-ci.yml file ^^)

8

Sure - this is the entrypoint script that runs by default in all images unless you override it (like we did above in your test step): renkulab-docker/entrypoint.sh at master · SwissDataScienceCenter/renkulab-docker · GitHub

This one runs after as a post-init script for the R images: renkulab-docker/post-init.sh at master · SwissDataScienceCenter/renkulab-docker · GitHub

In terms of the git setup, the CI environment in gitlab should already take care of it for you so it’s probably fine. It would only matter if you tried to push changes, for pulling it should not matter at all.

9

Thanks again, that should all be quite helpful!