Permission denied in a local Renku (Docker) session

champost · 21 April 2023 16:42

Hello,

I am trying to start a local Renku session with the instructions provided in Use the Docker Images built on RenkuLab.

I run into the following issue. However, I haven’t opened a Github issue as I am not sure if it is truly a bug or whether I am doing something wrong…

$ docker run --rm -ti -v ${PWD}:/work/${repoName} --workdir /work/${repoName} -p 8888:8888 ${imageName} jupyter lab --ip=0.0.0.0

Ahhhhhhhh! You have found a bug. 🐞

1. Open an issue by typing "open";
2. Print human-readable information by typing "print";
3. See the full traceback without submitting details (default: "ignore").

Please select an action by typing its name (open, print, ignore) [ignore]: print
## Describe the bug
A clear and concise description.

## Details
*Please verify and redact the details.*

**Renku version:** 2.3.2
**OS:** Linux (#1 SMP PREEMPT_DYNAMIC Thu Mar 23 20:08:28 EDT 2023)
**Python:** 3.9.12

### Traceback


Traceback (most recent call last):
  File "[...]/renku/ui/cli/exception_handler.py", line 133, in main
    return super().main(*args, **kwargs)
  File "[...]/renku/ui/cli/exception_handler.py", line 92, in main
    return super().main(*args, **kwargs)
  File "[...]/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "[...]/click/core.py", line 1654, in invoke
    super().invoke(ctx)
  File "[...]/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "[...]/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "[...]/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "[...]/renku/ui/cli/__init__.py", line 238, in cli
    is_renku_project = _is_renku_project(path)
  File "[...]/renku/ui/cli/__init__.py", line 163, in _is_renku_project
    return old_metadata.exists() or new_metadata.exists()
  File "[...]/pathlib.py", line 1424, in exists
    self.stat()
  File "[...]/pathlib.py", line 1232, in stat
    return self._accessor.stat(self)
PermissionError: [Errno 13] Permission denied: '/work/dash-example/.renku/metadata.yml'


## Additional context
Add any other context about the problem.
[I 2023-04-21 16:20:06.718 ServerApp] jupyter_resource_usage | extension was successfully linked.
[I 2023-04-21 16:20:06.721 ServerApp] jupyter_server_mathjax | extension was successfully linked.
[I 2023-04-21 16:20:06.722 ServerApp] jupyter_server_proxy | extension was successfully linked.
[I 2023-04-21 16:20:06.725 ServerApp] jupyterlab | extension was successfully linked.
[I 2023-04-21 16:20:06.725 ServerApp] jupyterlab_git | extension was successfully linked.
[W 2023-04-21 16:20:06.727 NotebookApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2023-04-21 16:20:06.727 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2023-04-21 16:20:06.727 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2023-04-21 16:20:06.730 ServerApp] nbclassic | extension was successfully linked.
[I 2023-04-21 16:20:06.730 ServerApp] nbdime | extension was successfully linked.
[I 2023-04-21 16:20:06.731 ServerApp] Writing Jupyter server cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2023-04-21 16:20:06.931 ServerApp] notebook_shim | extension was successfully linked.
[I 2023-04-21 16:20:06.951 ServerApp] notebook_shim | extension was successfully loaded.
[I 2023-04-21 16:20:06.952 ServerApp] jupyter_resource_usage | extension was successfully loaded.
[I 2023-04-21 16:20:06.953 ServerApp] jupyter_server_mathjax | extension was successfully loaded.
[I 2023-04-21 16:20:06.963 ServerApp] jupyter_server_proxy | extension was successfully loaded.
[I 2023-04-21 16:20:06.963 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.9/site-packages/jupyterlab
[I 2023-04-21 16:20:06.963 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 2023-04-21 16:20:06.966 ServerApp] jupyterlab | extension was successfully loaded.
[I 2023-04-21 16:20:06.970 ServerApp] jupyterlab_git | extension was successfully loaded.
[I 2023-04-21 16:20:06.973 ServerApp] nbclassic | extension was successfully loaded.
[I 2023-04-21 16:20:07.025 ServerApp] nbdime | extension was successfully loaded.
[I 2023-04-21 16:20:07.026 ServerApp] Serving notebooks from local directory: /work/dash-example
[I 2023-04-21 16:20:07.026 ServerApp] Jupyter Server 1.23.5 is running at:
[I 2023-04-21 16:20:07.026 ServerApp] http://221977fcdf22:8888/lab?token=62ad3b7b73d3d1b3c55a60fd4f0fd4eefdd99f34995331f0
[I 2023-04-21 16:20:07.026 ServerApp]  or http://127.0.0.1:8888/lab?token=62ad3b7b73d3d1b3c55a60fd4f0fd4eefdd99f34995331f0
[I 2023-04-21 16:20:07.026 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2023-04-21 16:20:07.029 ServerApp]

The Jupyter server starts and I can connect to it on port 8888 but I do not see any directories mounted

Instead I get the following error which keeps repeating itself on the command line

[E 2023-04-21 16:22:42.224 ServerApp] Uncaught exception GET /api/contents?content=1&1682094162152 (10.0.2.100)
    HTTPServerRequest(protocol='http', host='svx-uo7635resibase.unifr.ch:8888', method='GET', uri='/api/contents?content=1&1682094162152', version='HTTP/1.1', remote_ip='10.0.2.100')
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.9/site-packages/tornado/web.py", line 1713, in _execute
        result = await result
      File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/contents/handlers.py", line 116, in get
        self.contents_manager.get(
      File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/contents/filemanager.py", line 422, in get
        model = self._dir_model(path, content=content)
      File "/opt/conda/lib/python3.9/site-packages/jupyter_server/services/contents/filemanager.py", line 286, in _dir_model
        for name in os.listdir(os_dir):
    PermissionError: [Errno 13] Permission denied: '/work/dash-example'

Finally :

the Renku project that I am trying to reproduce on my local machine - Dash Example
the “imageName” is registry.renkulab.io/champak.beeravolureddy/dash-example:e420c6e
the “repoName” is dash-example (which corresponds to a freshly git cloned folder from which I am running the docker run command)
is this related to Wrong user permissions in local renku sessions · Issue #2873 · SwissDataScienceCenter/renku-python · GitHub ?

Thanks !

champost · 25 April 2023 11:22

– UPDATE –

With a cleaner MWE (dash example clean) and the associated Docker image (registry.renkulab.io/champak.beeravolureddy/dash-example-clean:646f938) I do not run into the Ahhhhhhhh! You have found a bug. 🐞 message.

However, I still have no mounted folders and a repeated loop of Permission denied: '/work/dash-example-clean' errors :

[E 2023-04-25 11:08:15.219 ServerApp] 500 GET /api/contents?content=1&1682420895143 (1ee386f012144d60a385489659c7da75@10.0.2.100) 2.62ms referer=http://svx-uo7635resibase.unifr.ch:8888/lab
[E 2023-04-25 11:08:25.341 ServerApp] Uncaught exception GET /api/contents?content=1&1682420905273 (10.0.2.100)
    HTTPServerRequest(protocol='http', host='svx-uo7635resibase.unifr.ch:8888', method='GET', uri='/api/contents?content=1&1682420905273', version='HTTP/1.1', remote_ip='10.0.2.100')
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.10/site-packages/tornado/web.py", line 1713, in _execute
        result = await result
      File "/opt/conda/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 121, in get
        model = await ensure_async(
      File "/opt/conda/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 182, in ensure_async
        result = await obj
      File "/opt/conda/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 887, in get
        model = await self._dir_model(path, content=content)
      File "/opt/conda/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 754, in _dir_model
        dir_contents = await run_sync(os.listdir, os_dir)
      File "/opt/conda/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
        return await get_asynclib().run_sync_in_worker_thread(
      File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
        return await future
      File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
        result = context.run(func, *args)
    PermissionError: [Errno 13] Permission denied: '/work/dash-example-clean'
[W 2023-04-25 11:08:25.342 ServerApp] wrote error: 'Unhandled error'
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.10/site-packages/tornado/web.py", line 1713, in _execute
        result = await result
      File "/opt/conda/lib/python3.10/site-packages/jupyter_server/services/contents/handlers.py", line 121, in get
        model = await ensure_async(
      File "/opt/conda/lib/python3.10/site-packages/jupyter_core/utils/__init__.py", line 182, in ensure_async
        result = await obj
      File "/opt/conda/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 887, in get
        model = await self._dir_model(path, content=content)
      File "/opt/conda/lib/python3.10/site-packages/jupyter_server/services/contents/filemanager.py", line 754, in _dir_model
        dir_contents = await run_sync(os.listdir, os_dir)
      File "/opt/conda/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
        return await get_asynclib().run_sync_in_worker_thread(
      File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
        return await future
      File "/opt/conda/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
        result = context.run(func, *args)
    PermissionError: [Errno 13] Permission denied: '/work/dash-example-clean'
[E 2023-04-25 11:08:25.343 ServerApp] {
      "Host": "svx-uo7635resibase.unifr.ch:8888",
      "Accept": "*/*",
      "Referer": "http://svx-uo7635resibase.unifr.ch:8888/lab",
      "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/112.0"
    }

champost · 26 April 2023 14:02

– UPDATE –

Ok so (in my case) this issue arose due to the use of the podman client (officially packaged by RHEL) which was aliasing docker on the machine that I was working on ! On an independent machine using true Docker this is not an issue.

I am now checking on how to run Renku images using the “rootless” Podman CLI as suggested here : Running a Container — Docker Stacks documentation

Topic		Replies	Views
Problem running local renku session Renku (CLI)	4	332	23 June 2023
Launching a local session from an image built on Renkulab	1	221	2 December 2022
Renku session on own machine with GPUs Renku (CLI)	7	231	11 April 2023
Error: Cannot start session via the notebook service RenkuLab	12	304	22 August 2023
Forward local renku session over ssh Renku (CLI)	1	170	4 April 2023

Permission denied in a local Renku (Docker) session

Related topics