The Docker image build failed: unable to clone project

Hi,

I am unable to build the docker image when trying to create a new environnement.

Pulling docker image docker:stable …
Using docker image sha256:61b2e482e9de9ca3939dce4c90810c89fa7e7450f774590967c2908cba857ddd for docker:stable …
Running on runner-m6yQ1wRz-project-1985-concurrent-0 via b1401e20dcd2…
Fetching changes…
Initialized empty Git repository in /builds/gitlab/shin.koseki/dyad/.git/
Created fresh repository.
error: RPC failed; curl 52 Empty reply from server
fatal: the remote end hung up unexpectedly
ERROR: Job failed: exit code 1

Hmm that looks a bit strange - are you able to clone the repository locally? Have you checked in a lot of large files into this repository? The error looks like something is going awry during the git clone.

Hi,

I am unable to clore locally. I get :slight_smile:

error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)
fatal: L’hôte distant a fermé la connexion de manière inattendue
fatal: erreur de protocole : mauvais entête de paquet
(base) MacBook-Air-de-admin-4:dyam shinkoseki$

I did output many larger files.

Large files committed to git can cause issues on many levels. I can try to clone your project locally and transfer large files to git-LFS - if this works, I can set it up in a fork of your project and you can check to make sure everything is as it should be. Then you can update your project using my fork. Would that work for you?

@sjinko ok I’ve done this now in https://renkulab.io/gitlab/rok.roskar/dyad – you’ll see that all the csv files in data/ are all in LFS, which means they are not checked in directly to the repository. If you’re not familiar with LFS, I find this tutorial particularly good: https://www.atlassian.com/git/tutorials/git-lfs

You should be able to clone my repository easily, but make sure you have git-lfs installed locally first. It would be great if you can verify that everything is there as expected. Once you’ve convinced yourself that it is indeed the case, you need to rewrite the history in your original repository to make it usable again.

To do this, you need to first unprotect the master branch by clicking the “view in gitlab” button on the upper right of the renku project view:

image

Then, you have to go to Settings --> Repository:

Expand the “Protected Branches” section and unprotect master.

Now, go to the directory where you cloned my repository and add your repository as a remote like this:

git remote add upstream git@renkulab.io:shin.koseki/dyad.git

Now you are ready to push to overwrite your master branch with the version that has the large files stored separately from git:

git push --force upstream master

Go to the gitlab page for your project and make sure that you see “LFS” next to the csv files:

image

If not, you may need to do the push again (I had to do it twice for some reason… don’t know why).

Your repository also has an autosave branch - you can also update that one from my fork if you want, by doing

git checkout renku/autosave/shin.koseki/master/f96da7d/38baafa
git push --force upstream renku/autosave/shin.koseki/master/f96da7d/38baafa

Let me know if you’d like help with this process! We can chat on gitter, it might be easier.

Hi thanks so much for your help!

I can get to the point where I

git push —force upstream master

but, I get the following message:

ref master:: Error in git rev-list --stdin --objects --not --remotes=upstream --: exit status 128 fatal: bad object 9480a1ab36004852171f99f8499f87964f17e10e error: impossible de pousser des références vers 'git@renkulab.io:shin.koseki/dyad.git'

@emmjab I think you encountered something like this recently - do you remember the solution?

@sjinko can you try running git fetch upstream before pushing?

btw, it’s been some time since I did this in your repo - if there are changes that you have in your copy which are not in my fork, do not force push yet!

It works! Thanks a lot for your help.

1 Like

glad to hear it! :sweat_smile:

From this point on, all .csv files in that repository will go to LFS so the repository won’t balloon in size. However, if you add large files with a different suffix you should run git lfs track <wildcard or path> before committing the files. The tutorial from Atlassian I linked to above is really good if you need a reference.

Thanks! Yes, the tutorial is very clear and straightforward. I know all about LFS now

One more note of caution: by default, when you start an environment on Renkulab the LFS files aren’t pulled automatically. To do this, you have to run

git lfs pull

in the terminal when your session starts.

Alternatiely, this can be done for you if you check the box “Automatically fetch LFS data” when starting an environment:

image

If you want to set it to be the default, you can run this command in your project:

renku config lfs_auto_fetch true