Problems with maximum disk space

1

Hi all,

I have running an environment for a few days, generating model. Despite I have been saving only (in LFS) the outputs I really needed to save, it seems I have top up the maximum disk space requested (64GB). Now, I am getting the following error:

 work ❯ democrasci-machine-tr ▶ $ ▶ git status
Error cleaning LFS object: write /work/democrasci-machine-tr/.git/lfs/tmp/201179852: no space left on device
error: packet write failed
error: external filter 'git-lfs filter-process --skip' failed
fatal: models/checkpoints/transformer_wmt_en_de_fr-de_all_sept2021/checkpoint_best.pt: clean filter 'lfs' failed

This does not allow me to do anything: git add, etc. Still, if I check the current disk occupation, I am not exceeding that maximum.

 work ❯ democrasci-machine-tr ▶ $ ▶ du -h -d 1
80K     ./notebooks
44G     ./.git
13G     ./data
2.6G    ./models
8.0K    ./.ipynb_checkpoints
48K     ./src
20K     ./.renku
59G     .

I am trying to remove all files and change them back to the LFS placeholders with
git read-tree HEAD && GIT_LFS_SKIP_SMUDGE=1 git checkout -f HEAD && rm -rf .git/lfs/*
but it is not working either.

What could I do?

Thanks again for all your support!
Cheers
Luis

2

Maybe renku storage clean could help? That will turn checked out LFS files back into pointers and clear them out of the cache.

1 Like
3

Wow, it helped!! I had completely forgotten about this mighty command! I just specified a path for renku storage clean with files I had already pushed, and that it is, I have again enough space! Thanks @rrrrrok

1 Like