In my Renku project, I’m using Ollama (a solution to run LLMs locally) to get responses from my code. While I successfully added Ollama to my Docker image during the build process, I haven’t been able to configure it to start automatically in the background.
Currently, every time I start or resume a session, I need to open a terminal and manually run ollama serve. This keeps a terminal window open for the entire session duration.
Ideally: I’d like Ollama to be running in the background automatically whenever I start or resume a Renku session.
The Problem: I’ve tried adding the ollama serve command to both the CMD and ENTRYPOINT instructions in my Dockerfile, but neither approach worked as expected:
Using CMD: Ollama didn’t start automatically, and I still had to run the command manually.
Using ENTRYPOINT: The Docker image built successfully, but it crashed when I tried to start it.
Given my limited experience with Docker, I’d appreciate any suggestions on how to configure my Dockerfile to launch Ollama automatically in the background when a Renku session starts.
Docker images only allow starting a single process, which is the ENTRYPOINT, and CMD are the arguments to that process.
The entrypoint in our images is the jupyter server that runs. So if you set CMD to something, this will be just ignored by the jupyter server entrypoint. If you set ENTRYPOINT to ollama, it should execute ollama, but wouldn’t start jupyter anymore, making the image essentially useless. Plus since jupyter doesn’t run, we detect that the session didn’t start properly and report it as failed.
While docker can only start one process in an image, that process is free to spawn as many subprocesses as it wants, so that’s how you’d solve this issue.
Specifically for our images, we support including a post-init.sh bash script in the image and that will get executed before jupyter gets started. In this bash script, you can start a background process as you normally would ollama serve & (the & executes the command in the background) and that should give you the result you want.
If you have a generic image, you’d have to write your own bash script, set that as the entrypoint and in the script execute any background tasks you want followed by the main process you want to start.