After I added datasets to my project Reproducible Data Science | Open Research | Renku it’s become unusable.
In Datasets I see “Checking project version and RenkuLab compatibility…”, which does not finish.
In the box “Linked datasets” I see “Loading links…”
In Settings I see “Refreshing project data…”
How can I diagnose it?
Hi @mrawlik ,
Is the project public? I cannot access link to the project. In case it is private or internal, it would be great that you could include the Renku team to support you on debugging. Best, Elisabet
Hi @mrawlik I am one of the Renku developers. Can you please add me to your project (my email is tasko.olevski@sdsc.ethz.ch) so I can help you troubleshoot. My guess is that you may have committed large files outside of git LFS. And that is now causing trouble with the git operations the website does in the background when you browse the project.
Hi @tolevski I have just added you.
I have rather large notebooks, but those, I understand, shouldn’t be stored in LFS. The datasets (I used renku dataset add) are in LFS.
BTW is there a group that aggregates the Renku support team on the renkulab gitlab? Then I could add it instead of specific people.
@mrawlik it is taking really long time to simply clone your repo (locally outside of Renku). I have been waiting for the clone operation to finish for more than 30 minutes. This occurs if you have data that should be in LFS but is not.
I see that you have some autosave branches in the repo. If you do not need those can you delete them? Sometimes large files can creep in through the autosaves.
If deleting the autosaves does not help we can try to migrate the notebooks (and other large files) to LFS.
As for what should or should not be in LFS I would say anything that is large in size and that you do not wish to see a diff for should be in LFS. This includes jupyter notebooks because their size can get large quickly and the underlying code that they contain does not produce meaningful diffs.
Ok I did a few other checks and I do not think the problem is LFS. Because when I clone your repo without pulling the LFS objects it is quick.
Let me do some more digging @mrawlik.
Thanks!
I also checked if I have large files with renku doctor. I found it quite useful for that.
(base) piazzal@magnifico:/mnt/ssddata/piazzal/improvement-of-spatial-resolution-in-low-dose-phase-contrast-x-ray-imaging$ renku --version
2.6.0
(base) piazzal@magnifico:/mnt/ssddata/piazzal/improvement-of-spatial-resolution-in-low-dose-phase-contrast-x-ray-imaging$ renku doctor
Please note that the diagnosis report is used to help Renku
maintainers with debugging if you file an issue. Use all proposed
solutions with maximal care and if in doubt ask an expert around or
file an issue. Thanks!
caught a TypeError, retrying call to <class 'pyshacl.monkey.memory2.Memory2'>.bind without override, see https://github.com/RDFLib/rdflib/issues/1880 for more info
Traceback (most recent call last):
File "/home/piazzal/.local/pipx/venvs/renku/lib/python3.8/site-packages/rdflib/namespace/__init__.py", line 636, in _store_bind
return self.store.bind(prefix, namespace, override=override)
TypeError: bind() got an unexpected keyword argument 'override'
caught a TypeError, retrying call to <class 'pyshacl.monkey.memory2.Memory2'>.bind without override, see https://github.com/RDFLib/rdflib/issues/1880 for more info
Warning: Git history contains large files - consider moving them to external storage like git LFS using 'renku storage migrate'
*.png 1.5 MB
*.tif 2.4 KB
*.yaml 1.7 KB
*.tiff 792 B
*.h5 543 B
*.gitmodules 75 B
I don’t know if the error I get when running renku doctor is related.
The error seems to just be a warning. But it is a new one we haven’t seen before.
Is there anything I can do then?
We found a bug that we’re currently working on a hotfix for. We recently updated one of our services to enable scaling to improve performance, but this service works with an internal cache and there is a bug where, if a user deletes their cookies or uses a new browser window, they get sent to the wrong service instance which results in an error.
The hotfix is almost ready to deploy and in the meantime we scaled down the service to a single instance, which should prevent this issue from happening.
Does the problem still persist for you?
Unfortunately the problem still persists. I tried a different browser: same.
The error message changed a bit, though. Now when I go to “Datasets” I see
There was an error verifying support for this project.
Fatal error occurred while working on the repository.
Hi Michał,
it took a while but we narrowed down the problem to you using git submodules in your project.
In general, we don’t recommend that users use git submodules, as the gitpython library that Renku uses does not properly support submodules.
They can work for public repositories and when added using a https:// URL (instead of a git+ssh:// URL), but if the repository is private or using an SSH URL, our service cannot clone the project as it fails when trying to pull submodules.
There is a subsequent issue where the resulting error is not properly returned to inform the user that we’ll address, together with not trying to clone submodules when cloning projects. I’ve created an issue for that on git clone should not try to clone submodules and surface errors properly · Issue #3581 · SwissDataScienceCenter/renku-python · GitHub
Until that is fixed, there’s unfortunately no workaround other than removing the submodule.