How to deploy Streamlit in renku

How to deploy Streamlit in renku

Streamlit is a great package to create dashboards from python scripts. It is straight-forward to implement and fast to learn. Just import streamlit in your python script and add the streamlit code snippets you need to interact with your project. The resulting dashboards look clean and simple, displaying only what you want to show in a well-designed manner.

The main concepts of streamlit are nicely explained here: https://docs.streamlit.io/en/stable/main_concepts.html
And the API reference provides a great overview of streamlit’s functionality: https://docs.streamlit.io/en/stable/api.html

Streamlit is open-source and can be easily used with your renkulab project in a few steps.

In your renku repository

To deploy a streamlit app with renku, carry out the following steps. If you don’t have a renku project yet, see how to get started here.

  1. add these libraries to requirements.txt

    • jupyter-server-proxy
    • streamlit
  2. tell jupyter how to start the streamlit app

    • add jupyter_notebook_config.py to the main path of your repo (see below)
    • add this line to the Dockerfile: COPY jupyter_notebook_config.py ~/.jupyter/
  3. set the defaults for the interactive environment in .renku/renku.ini

jupyter_notebook_config.py

The configuration to start the app is set in this file. The name of the streamlit environment will be used by renku to reference it.

# Configuration file for jupyter-notebook.

c.ServerProxy.servers = {
    'streamlit': {           # the name of the environment
        'command': [
            # the command to start the streamlit app
            'streamlit',
            'run',
            'app.py',
            '--server.port', '8501',
            '--browser.serverAddress', '0.0.0.0',
        ],
        'port': 8501,
    }
}

Make sure to match the streamlit server port and the jupyter port. This is especially important if you want to open several apps in one session (see below).

.renku/renku.ini

Add the following few lines to your renku.ini file. You might need to create the file, if it doesn’t exist yet.
Make sure to use the same name for the streamlit environment as in the jupyter notebook configuration file, preceded by a slash.

[renku "interactive"]
# again the name of the environment for renku
default_url = /streamlit
lfs_auto_fetch = true
cpu_request = 1
mem_request = 2G

If the app works with data in git lfs, set lfs_auto_fetch to true. The values for the number of CPUs and amount of memory should be set depending on the needs of your app.

Note: In this file, comments need to be on separate lines. Otherwise, they are interpreted as part of the argument value.

On renkulab

When you now want to start a new interactive renku environment, you get a third default environment named /streamlit.

Also, the defaults for number of CPUs, amount of memory and the flag to automatically fetch all git lfs data are adjusted.

And now, you can start an environment and when you connect to it, it automatically opens the streamlit dashboard.

Enhancement

Add several apps in one interactive environment

Currently, you can only add one additional default environment to the renku UI.
A simple work-around is to have a main app, which is accessed from the renku UI and from which other apps can be opened.

Adjust jupyter_notebook_config.py such that each app gets started on a separate port, e.g.

# Configuration file for jupyter-notebook.

c.ServerProxy.servers = {
    'streamlit': {           # the name of the main environment
        'command': [
            'streamlit',
            'run',
            'app.py',
            '--server.port', '8501',
            '--browser.serverAddress', '0.0.0.0',
        ],
        'port': 8501,
    },
    'streamlit2': {           # the name of the second environment
        'command': [
            'streamlit',
            'run',
            'app2.py',
            '--server.port', '8502',
            '--browser.serverAddress', '0.0.0.0',
        ],
        'port': 8502,
    }
}

HTML links can be to added your app.py file using st.markdown(). You need to tell streamlit to allow html by setting the flag unsafe_allow_html to True.

st.markdown(
    "<a href='../streamlit2'>the second streamlit app</a>",
    unsafe_allow_html = True
)

If you want to add the additional app in a new tab, use the following:

st.markdown(
    "<a target='_blank' rel='noopener noreferrer' href='../streamlit2'>the second streamlit app</a>",
    unsafe_allow_html = True
)
4 Likes

Very interesting Lilian. Thanks. Would be great to see an R Shiny equivalent of that (if not already around)

2 Likes

Hi @marounbs, this is in fact already possible, with a little bit of additional tweaking! I made a small example project to demonstrate how to set it up.

I don’t have any simple shiny apps to add to this demo, would be great if you could contribute one! Do you think it would make sense for us to include this in the default R/RStudio setup?

1 Like

Thanks @rrrrrok . I will try to look into it when I can and see if I can add a simple app. Also, had a discussion with @cchoirat, who has a lot of better insights than me, so I guess the Shiny topic has been already visited at SDSC and it’s only a matter of finding the solution that makes sense to the most users. From my perspective, having this as an “environment” by default would be interesting, and I don’t think this would add much complexity to the existing template.

@marounbs if you could add just a super simple demo app I would be very grateful!

Regarding adding this per default - I’ll have to check how much bigger it makes the resulting image. If the difference is small then I agree that it’s not a problem to add it. Otherwise I’d prefer to leave it out by default but with a simple path for users to add it with minimal effort.

Speaking of templates - @LiliGasser we should turn the Streamlit setup into a template! Maybe we could have a collection of “unofficial” community-built templates for special use-cases like this one?

@rrrrrok I like your idea! Having a streamlit template for renku would be very convenient.

Ok great, we can work on preparing a repo that would be a place for such templates. What I have in mind is to set up something where it’s easy for others to add new useful project templates and we can help them along to make sure they are fully functional and well documented etc. In the new renkulab (not yet on renkulab.io but you can see it on dev.renku.ch) we allow users to specify a different source of templates, so that would be the natural thing to do IMO.