Dash app with jupyter-server-proxy

[manuelknott]

I want to deploy a Dash/Plotly application on Renkulab using jupyter-server-proxy similar to the streamlit example described here: How to deploy Streamlit in renku

This is my approach so far:

1) Create a new environment in .renku/renku-ini:

[renku "interactive"]
default_url = /dash
lfs_auto_fetch = true

2) Define the jupyter_notebook_config.py:

c.ServerProxy.servers = {
    'dash': {
        'command': [
            'python',
            'app.py',
        ],
        'port': 5000,
    }
}

3. Add the following requirements:

dash
jupyter-server-proxy
jupyter-dash

4. The actual app is defined in appy.py like this:

from jupyter_dash import JupyterDash
JupyterDash.infer_jupyter_proxy_config()
app = JupyterDash(__name__)

(details are defined here...)

if __name__ == "__main__":
    app.run_server(mode="jupyterlab", debug=False, port=5000, host='0.0.0.0')

If I call the python script locally it works. However, in Renkulab I receive the following error when starting the session:

Any ideas what could be missing? I also tried a version using gunicorn instead of calling the python script directly (this is for example required when deploying the app on Heroku) with the same result.

Thanks in advance!

[rrrrrok]

Hey @manuelknott can you share a link to the project? If the one where this has been tried contains information you don’t want to share, you can make another one just for testing this.

[manuelknott]

I created a public minimal example here:

Manuel Knott / dash_jupyter_server_proxy_example

[rrrrrok]

Hi Manuel, thanks for setting up the example project! Looking at the logs, it seems that the port configuration is wrong?

2022-06-02T13:07:00.648104731Z Dash is running on http://127.0.0.1:8050/
2022-06-02T13:07:00.648150454Z 
2022-06-02T13:07:00.648713674Z  * Serving Flask app 'app' (lazy loading)

In your jupyter config you have

c.ServerProxy.servers = {
    'dash': {           # the name of the environment
        'command': [
            'python',
            'app.py'
        ],
        'port': 5000
    }
}

so if you change the port to 8050 it should work. I’ve tried that in my fork and the app starts but all I see is “Loading…”:

[manuelknott]

Hi Rok,

thanks for pointing out the mismatching ports. I fixed it in the current commit of the minimal example.

However, the “Loading…” issue still remains, and I dug deeper into it:

The main issue is that jupyter_dash works like a charm when being called from a jupyter notebook environment, as this is the library’s main purpose. It does not work when being called from a python file since it is dependent on the jupyter_dash frontend extension.

I tried a couple of things:

• Calling a notebook with papermill or nbconvert does not work as the jupyter extensions are not used.
• Calling the python script with “jupyter run …” does not work (same issue)
• I also tried to not use the above mentioned dependencies but to serve the plain flask app (see the “no_jupyterdash” branch in the example’s repo) → no success so far

There is also a related github issue here.
I tried adding requests_pathname_prefix='/proxy/8050/' to the JupyterDash constructor.
The logs show that the app is running at {host}:8888/proxy/8050, but it is still stuck in the “loading…” phase.

I ran out of ideas at this point.

[manuelknott]

When opening the “lab” environment with my image, I receive a popup that the lab extension “jupyterlab-dash” is missing. Not sure if that would solve anything. Would maybe be worth a try to add it to the Renku Image.

[rdcomineor]

Hi Manuel. Did you manage to solve it? After many hours of trial and error I also got stuck in the same place and I am also running out of ideas. I did not get the popup of lab extension “jupyterlab-dash” missing, but I arrived to the “loading…” phase. Any new suggestions would be highly appreciated. Many thanks in advance!

[tolevski]

Hi @rdcomineor here is a simple working example: Reproducible Data Science | Open Research | Renku

Let me know if you have any problems. I worked with Manuel to resolve the problems he had earlier.

[rdcomineor]

Thanks so much @tolevski! This is perfect.

[champost]

Hi @tolevski,

The JupyterLab (/lab) view of your dash-example results in the message below.

Any idea how to go about it ?

[tolevski]

If you simply click the Build button everything should work just fine @champost.

Did you have any issues with it? I tested on my end and the dashboard works just fine and does not show that message. And from what I can tell the jupyterlab environment is also functional.

I will take a look if I can get that message to go away.

[champost]

@tolevski

The dashboard works perfectly as you mention and clicking on Build button makes the message go away but another comes up if you wait a little :

[tolevski]

I just changed that example to do the build step when the docker image is built.

With that change the message does not appear at all. Give it a try and let me know if you have any other issues.

[champost]

Works like a charm now @tolevski !

[champost]

@tolevski

How does one update a Dash app on renkulab.io while a session is running, for example if I want to replace the app in your example with the below (taken from Part 2. Layout | Dash for Python Documentation | Plotly) :

import pandas as pd

df = pd.read_csv('https://gist.githubusercontent.com/chriddyp/5d1ea79569ed194d432e56108a04d188/raw/a9f9e8076b837d541398e999dcbac2b2826a81f8/gdp-life-exp-2007.csv')

fig = px.scatter(df, x="gdp per capita", y="life expectancy",
                 size="population", color="continent", hover_name="country",
                 log_x=True, size_max=60)

app.layout = html.Div([
    dcc.Graph(
        id='life-exp-vs-gdp',
        figure=fig
    )
])

Do I have to commit these changes, push to Gitlab, wait for the CI/CD build and then restart a session with the new image (in this order) to preview my update ?

[champost]

@tolevski

I found a hack to do the above but I am sure there is more elegant way to go about it…

From within a /lab interface (and with the /dash endpoint open in another tab),

• I simply updated my app.py file with the code above and saved it (without committing any changes)

• I open a terminal and looked up the PID of the python process running the dash app using

ps -fC python

and then killed it (a new process gets automatically spawned)

• I refreshed the /dash tab and my app updated in a running session !

[tolevski]

@champost

Do I have to commit these changes, push to Gitlab, wait for the CI/CD build and then restart a session with the new image (in this order) to preview my update ?

Yes currently that is the only “proper” way to get your changes.

What you suggested with looking up the PID, killing that process and then waiting for the new version to start is really smart and it is great that it works. Thanks for sharing that.

[rrrrrok]

@champost you can use the debug=True flag when you start the server - see here.

[champost]

Thanks @rrrrrok !

Just wanted to mention the following if somebody else runs into a related problem (independent though of Renku).

When using the same port to display/update by running/re-running some Dash app code, there appears an Address already in use... error due to werkzeug==2.2.2 and downgrading werkzeug to 2.2.1 solves the problem as of now.

[champost]

Hello !

The Dash app MWE (proposed by @tolevski further ) doesn’t start automatically on renkulab.io with the latest Renku minimal Python template as it should even though it has been specified in /home/jovyan/work/<project-name>/jupyter_notebook_config.py with the following server process configuration :

# Configuration file for jupyter-notebook.
c.ServerProxy.servers = {
    'dash': {
        'command': [
            'python',
            'app.py',
            '--port',
            '{port}',
        ],
        'absolute_url': False,
    }
}

Here is a clean MnWE (Minimum not-Working Example) : dash example clean