How to deploy Streamlit in renku

RenkuLab
21

Hi @gavin-k-lee,
So the goal is to have the repository be an installable R package that could be installed with remotes::install_local("path/to/repo") so that the app can then be run by anyone who’s installed it from an R session with exampleApp::run_app(). But also have the app run when you launch the shiny environment in Renku.

It is a bit complicated as the app.R file ends up shuffled around between a few locations when you make the project an installable R package.

When you create a shiny app as an R package you place an app.R file in /inst/shinyApp/app.R, when this package is installed the contents of the inst directory are placed in the base directory of the installed package in the library where that package is installed (library path varies by OS).

To run the shiny app with a shiny server on which the R package is installed you then copy the app.R file from /shinyApp/app.R to wherever your shiny server is expecting to serve apps from e.g. /srv/shiny-server/. This is achieved with something like: file.copy(system.file("shinyApp/app.R", package = "exampleApp"), "/srv/shiny-server/app.R") as the system.file function is aware of library paths and able to locate files within installed package directories if given the package’s name.

The app.R file need only contain exampleApp::run_app() as the exampleApp package is installed on the server and exports the main shinyApp function from the application as run_app thus all the app’s actual code lives in the package library rather than in the shiny server app folder /srv/shiny-server/

When I did this before I installed an app from another repo in a renku project copied it’s app.R to the project home directory and that worked. However I seem to be having trouble getting the same thing to work when the renku project itself is an R package. If you clone project repo of my example app and use remotes::local_install() to install it then run exampleApp::run_app() the example app runs. But when I include the install and copy commands in the post-init.sh this does not seem to be installing the package and/or copying the app.R file from it’s installed location to the project home.

One difference is that I’m installing the app in the post-init.sh rather than during the Docker image build but this should not stop the file copy from working.

Does that clarify things at all?

22

Update of debugging this. So I tried to go back to basics and just place a simple app.R file with a minimal app in it in a fork of @rrrrrok’s original example repo (Reproducible Data Science | Open Research | Renku). Leave out the packaging issue entirely for now and get a minimal version working. Unfortunately it’s still not starting the app correctly when I run the session.

I get a generic " An error has occurred! check your logs" message if I include the basic shinyApp() function call in the app.

If I leave out the call to shinyApp() I get a 500 error saying it took too long for the shiny server to start. Shiny Server does not need this call to be explicitly present if there are app.R or ui.R/server.R files in the directory from which it serves app, so I tried it without just ti see if it worked.

I’ve also tried tweaking the host and port. setting host to 0.0.0.0 and port to 3838 as well as only setting the host but I get the same error

I’ll take a look into how jupyter-shiny-proxy works to see if I can understand better to diagnose further.

2022-09-05T11:11:11.381115845Z /home/rstudio//.Rproj
2022-09-05T11:11:11.387902776Z chown: changing ownership of '/home/rstudio/.rstudio/projects_settings/next-session-project': Operation not permitted
2022-09-05T11:11:11.387985082Z chown: changing ownership of '/home/rstudio/.rstudio/projects_settings': Operation not permitted
2022-09-05T11:11:11.403350181Z fatal: repository '@//renku-env.git' does not exist
2022-09-05T11:11:11.407478801Z find: ‘/tmp/renku-env’: No such file or directory
2022-09-05T11:11:12.204643174Z [W 11:11:12.204 NotebookApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
2022-09-05T11:11:13.427092991Z [I 11:11:13.426 NotebookApp] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab
2022-09-05T11:11:13.427145748Z [I 11:11:13.426 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
2022-09-05T11:11:13.549857635Z [I 11:11:13.549 NotebookApp] Serving notebooks from local directory: /home/rstudio/work/minimal-shiny-example
2022-09-05T11:11:13.549912480Z [I 11:11:13.549 NotebookApp] The Jupyter Notebook is running at:
2022-09-05T11:11:13.549923150Z [I 11:11:13.549 NotebookApp] http://racton-40u-minimal-2dshiny-2dexampl-fa490fa9-0:8888/sessions/racton-40u-minimal-2dshiny-2dexampl-fa490fa9/
2022-09-05T11:11:13.549932122Z [I 11:11:13.549 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
2022-09-05T11:11:13.555874107Z [W 11:11:13.555 NotebookApp] No web browser found: could not locate runnable browser.
2022-09-05T11:11:59.400995245Z [I 11:11:59.400 NotebookApp] 302 GET /sessions/racton-40u-minimal-2dshiny-2dexampl-fa490fa9 (127.0.0.1) 1.28ms
2022-09-05T11:11:59.445708515Z [I 11:11:59.445 NotebookApp] 302 GET /sessions/racton-40u-minimal-2dshiny-2dexampl-fa490fa9/shiny? (127.0.0.1) 1.67ms
2022-09-05T11:12:00.053466813Z [2022-09-05T11:12:00.051] [INFO] shiny-server - Shiny Server v1.5.18.979 (Node.js v12.22.6)
2022-09-05T11:12:00.053890625Z [2022-09-05T11:12:00.053] [INFO] shiny-server - Using config file "/tmp/tmpsjae3fk9"
2022-09-05T11:12:00.105989211Z [2022-09-05T11:12:00.105] [INFO] shiny-server - Starting listener on http://[::]:39739
2022-09-05T11:12:00.304953610Z [2022-09-05T11:12:00.304] [ERROR] shiny-server - Error attempting to change ownership of log file at /home/rstudio/work/minimal-shiny-example/logs/minimal-shiny-example-rstudio-20220905-111200-38459.log: EPERM: operation not permitted, fchown
2022-09-05T11:12:00.306854119Z [2022-09-05T11:12:00.306] [INFO] shiny-server - Created bookmark state directory: /home/rstudio/work/minimal-shiny-example/bookmarks
2022-09-05T11:12:00.307429299Z [2022-09-05T11:12:00.307] [INFO] shiny-server - Created user bookmark state directory: /home/rstudio/work/minimal-shiny-example/bookmarks/rstudio
2022-09-05T11:12:02.195449332Z [E 11:12:02.194 NotebookApp] {
2022-09-05T11:12:02.195505893Z "Host": "renkulab.io",
2022-09-05T11:12:02.195518314Z "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36",
2022-09-05T11:12:02.195528677Z "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8",
2022-09-05T11:12:02.195585960Z "Accept-Encoding": "gzip, deflate, br",
2022-09-05T11:12:02.195602638Z "Accept-Language": "en-GB,en;q=0.9",
2022-09-05T11:12:02.195612134Z "Cookie": "_oauth2_proxy=zPVz4uc-F5lwqAxTd-NQD71XOHdte9TGhIa3pgl-PHAPXJfUSHbJwGnciz7Jx0LRQZpEPkvxLVys4aCnLu6WhYYS3Inj88IyBw8Enx730o5_etwCb_Tm9JZbvEu0u96HdP2CNTiq7C3UUViYwhYmmm58Aip5MVkiy4vzi4WIZL7vJC-rGToKIqpwPxZhhgc=|1662376319|l4C4473OCZbqD-lLQLWoVOMTDGbHZDnwgDjiXKZvhT0=; anon-id=anon-b8fkuimhSZBzsLIr_mI-mgHRKUhMzM_ltoz25KNtqZ8; event_filter=all; sidebar_collapsed=false; session=11ef3fcabf0fa16_62fb9be2.14cuwj20ZKq_xImAID_HadAQ9zI; csrf-token=c38c2cb1-db38-4c8a-81c8-9930e65f5b63; user-id=rstudio|Thu%2C%2025%20Aug%202022%2009%3A21%3A25%20GMT|YysWjK2YCj63oErBM6z6JGOaQJa1%2BJrFRj92WCc0j8g%3D; _xsrf=2|05b43edf|7e3c2483d2eeb8c88edc66482e716144|1662112978",
2022-09-05T11:12:02.195627383Z "Sec-Fetch-Dest": "iframe",
2022-09-05T11:12:02.195635512Z "Sec-Fetch-Mode": "navigate",
2022-09-05T11:12:02.195643423Z "Sec-Fetch-Site": "same-origin",
2022-09-05T11:12:02.195651194Z "Sec-Gpc": "1",
2022-09-05T11:12:02.195659159Z "Upgrade-Insecure-Requests": "1",
2022-09-05T11:12:02.195667041Z "X-Forwarded-Email": "racton@uni-koeln.de",
2022-09-05T11:12:02.195674973Z "X-Forwarded-For": "192.168.0.205, 10.42.10.191",
2022-09-05T11:12:02.195703091Z "X-Forwarded-Host": "renkulab.io",
2022-09-05T11:12:02.195711838Z "X-Forwarded-Port": "443",
2022-09-05T11:12:02.195719779Z "X-Forwarded-Preferred-Username": "racton@uni-koeln.de",
2022-09-05T11:12:02.195727925Z "X-Forwarded-Proto": "https",
2022-09-05T11:12:02.195737918Z "X-Forwarded-Scheme": "https",
2022-09-05T11:12:02.195751165Z "X-Real-Ip": "192.168.0.205",
2022-09-05T11:12:02.195769792Z "X-Request-Id": "af0b071d3a71e4ca190c83fe48f7a175",
2022-09-05T11:12:02.195784145Z "X-Scheme": "https"
2022-09-05T11:12:02.195797389Z }
2022-09-05T11:12:02.195820248Z [E 11:12:02.195 NotebookApp] 500 GET /sessions/racton-40u-minimal-2dshiny-2dexampl-fa490fa9/shiny/ (127.0.0.1) 2598.94ms referer=None
23

Webbrowser is part of the python standard library, you don’t have to install a separate package to use it because it comes bundled with your python installation. If you want to get recognized browsers on your system:

import webbrowser
print webbrowser._browsers

If you directly use webbrowser.open() - it will always open the link in the default browser. What you can do is to register the any other browser and then launch a new tab. Something like this:

webbrowser.register(name, constructor, instance=None)

Once a python browser type is registered, the get() function can return a controller for that browser type. You can run open, open_new and open_new_tab on the controller object. This will ensure the commands are executed on the same browser instance you opened.

import webbrowser    
url='https://www.google.com'
chrome_path="C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe"
webbrowser.register('chrome', None,webbrowser.BackgroundBrowser(chrome_path),1)
webbrowser.get('chrome').open_new_tab(url)
24

For anyone following the instructions a few notes.

  1. The line added to the Dockerfile COPY jupyter_notebook_config.py ~/.jupyter/ should be COPY jupyter_notebook_config.py ${HOME}/.jupyter/ because Docker does not understand ~ the way a regular shell does
  2. In some cases the startup time for streamlit can be long so it is a good idea to add a longer timeout in the jupyter_notebook_config.py file like this:
c.ServerProxy.servers = {
    'streamlit': {
        'command': [
            'streamlit',
            'run',
            'app.py',
            '--server.port', '8501',
            '--browser.serverAddress', '0.0.0.0',
        ],
        'port': 8501,
        'timeout': 60
    }
}

Without the increased timeout - I am not sure what the default is - it will often happen that everything is ok but things timeout because the app is taking a while to startup fully.

@LiliGasser if you have some time to make these edits in your original post it would be really nice. I am not allowed to touch other people’s posts.

25

Hi Tasko,

thanks for these changes. Unfortunately, I am not allowed to change the initial post.
Since also other parts could be updated (such as the screenshot to start a session), does it make sense to do a new post “How to deploy Streamlit in renku 2.0”?

26

Hi Lili. Thank you for checking. If the original post cannot be updated do not worry - me or someone from Renku can add another post. Thank you for checking though!

27

I edited it :face_with_hand_over_mouth:

Thanks @tolevski and @LiliGasser ! We should make sure this is reflected in the template as well…

1 Like