Running renku commands, even in a freshly cloned repository (with git, as renku clone fails) gives me an error:
Unable to acquire lock.
Hint: Please wait for another renku process to finish and then try again.
I have no other renku processes running.
Where is the lock file? Can I remove it manually? Or could there be another reason for this error?
I have renku 2.7.0.
The lock file is in the root of the project .renku.lock and can be manually deleted.This is usually an issue with running two renku processes in parallel, running renku within renku (renku run renku ...) or lacking permissions to write the lockfile
Maybe I do something wrong then, but even for a fresh repository I have this problem:
(base) mrawlik@hari:~/myterminus/code$ renku --version
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
2.7.0
(base) mrawlik@hari:~/myterminus/code$ renku init test
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Initializing Git repository...
NUMBER ID
-------- --------------
1 python-minimal
2 R-minimal
3 bioc-minimal
4 julia-minimal
5 minimal
Please choose a template by typing its number: 1
Info: These parameters are not used by the template and were ignored:
name
Initializing new Renku repository...
Initializing .dockerignore ...
Initializing .gitignore ...
Initializing .gitlab-ci.yml ...
Initializing .renku/renku.ini ...
Initializing .renkulfsignore ...
Initializing Dockerfile ...
Initializing README.md ...
Initializing data/.gitkeep ...
Initializing environment.yml ...
Initializing notebooks/.gitkeep ...
Initializing requirements.txt ...
Initializing workflows/my-workflow.yaml ...
Project initialized.
OK
(base) mrawlik@hari:~/myterminus/code$ cd test/
(base) mrawlik@hari:~/myterminus/code/test$ renku status
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Unable to acquire lock.
Hint: Please wait for another renku process to finish and then try again.
(base) mrawlik@hari:~/myterminus/code/test$ ls -hal
total 40K
drwxrws--- 7 mrawlik itet-isg-stamplab 4.0K Dec 5 14:12 .
drwxr-sr-x 7 mrawlik itet-isg-stamplab 184 Dec 5 14:12 ..
-rw-rw---- 1 mrawlik itet-isg-stamplab 27 Dec 5 14:12 .dockerignore
drwxrws--- 8 mrawlik itet-isg-stamplab 209 Dec 5 14:12 .git
-rw-rw---- 1 mrawlik itet-isg-stamplab 39 Dec 5 14:12 .gitattributes
-rw-rw---- 1 mrawlik itet-isg-stamplab 5.3K Dec 5 14:12 .gitignore
-rw-rw---- 1 mrawlik itet-isg-stamplab 450 Dec 5 14:12 .gitlab-ci.yml
drwxrws--- 3 mrawlik itet-isg-stamplab 110 Dec 5 14:12 .renku
-rw-rw---- 1 mrawlik itet-isg-stamplab 0 Dec 5 14:12 .renku.lock
-rw-rw---- 1 mrawlik itet-isg-stamplab 976 Dec 5 14:12 .renkulfsignore
-rw-rw---- 1 mrawlik itet-isg-stamplab 2.1K Dec 5 14:12 Dockerfile
-rw-rw---- 1 mrawlik itet-isg-stamplab 1.5K Dec 5 14:12 README.md
drwxrws--- 2 mrawlik itet-isg-stamplab 30 Dec 5 14:12 data
-rw-rw---- 1 mrawlik itet-isg-stamplab 741 Dec 5 14:12 environment.yml
drwxrws--- 2 mrawlik itet-isg-stamplab 30 Dec 5 14:12 notebooks
-rw-rw---- 1 mrawlik itet-isg-stamplab 0 Dec 5 14:12 requirements.txt
drwxrws--- 2 mrawlik itet-isg-stamplab 38 Dec 5 14:12 workflows
(base) mrawlik@hari:~/myterminus/code/test$ rm -rf .renku.lock
(base) mrawlik@hari:~/myterminus/code/test$ renku status
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/home/mrawlik/.local/pipx/shared/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Unable to acquire lock.
Hint: Please wait for another renku process to finish and then try again.
It actually happens to all users on this machine. It is managed by ISG of the D-ITET department, so the setup is not straightforward. Most directories are mounted externally.
I hope I could find a solution, eventually, but I don’t know how to begin tracing where the problem might be.
Simon Alder from the ISG support discovered that it is actually due to a bug in a library that renku uses: lock on NFS mount not working · Issue #92 · wolph/portalocker · GitHub
In the environment that we use at ETH the home directory is always mounted over NFS.
that’s a good catch! I wouldn’t have even considered that.
I’m not really sure how we could address that, we could create the lock somewhere else (named after the project in /tmp/), but that carries the risk of not locking when it should, if two executions of renku end up with different lock paths. Or we could consider some other locking mechanism (we used filelock in the past and that had other issues). I’ll see with the team if we can come up with a solid solution.
ideally the portalocker package could fix this, but it’d be nice to have a fix/workaround in the meantime.
All credit goes to the ISG group of D-ITET at ETH 
Funny enough renku doesn’t seem to mind the project being on an LFS. Only the home directory.
I filed a bug report: Unable to acquire lock when home directory is mounted with NFS · Issue #3669 · SwissDataScienceCenter/renku-python · GitHub
I also found a workaround:
$ HOME=/scratch/mrawlik renku