Interactively visualize data

RenkuLab
1

Issue

I am working in electron microscopy and it is important for me (and other microscopist) to inspect data and pick up areas of the dataset interactively. To explore the dataset I usually use the PyQt5 backend of matplotlib.
For exemple : I want to be able to pick up and display an area of an image. Then If I find this area suitable for processing I would like to be able to use it as an input for a python script.

I installed everything so that PyQt5 should be working but when I try to use the Ipython magic command line : %matplotlib qt an ImportError shows up (see code output section).

Searching on StackExchange I found this question : StackExchange issue. If I understood well it is normal to get such errors in the renkulab environment.

Can something be done to solve this issue regardless?
More generally about interactivity: Would it be possible in renku to have “hybrid” workflows were the user could have manual input during the processing of the data ?

Docker files

requirements.txt
cython==0.29.15
hyperspy==1.5.2
hyperspy_gui_ipywidgets==1.2
hyperspy_gui_traitsui==1.2
lmfit==1.0.0
PyQt5==5.14.1

DockerFile
FROM renku/singleuser:0.4.3-renku0.8.2

#Uncomment and adapt if code is to be included in the image
#COPY src /code/src

#Uncomment and adapt if your R or python packages require extra linux (ubuntu) software
#e.g. the following installs apt-utils and vim; each pkg on its own line, all lines
#except for the last end with backslash '\' to continue the RUN line

USER root
RUN apt-get update && \
apt-get install -y --no-install-recommends \
libgl1-mesa-glx
USER ${NB_USER}

#install the python dependencies
COPY requirements.txt environment.yml /tmp/
RUN conda env update -q -f /tmp/environment.yml && \
/opt/conda/bin/pip install -r /tmp/requirements.txt && \
conda clean -y --all && \
conda env export -n "root"

code ouput

import matplotlib
print(matplotlib.get_backend())
--> Qt5Agg

%matplotlib qt
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-7-325c2f98aa22> in <module>
----> 1 get_ipython().run_line_magic('matplotlib', 'qt')

/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
2312 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2313 with self.builtin_trap:
-> 2314 result = fn(*args, **kwargs)
2315 return result
2316

</opt/conda/lib/python3.7/site-packages/decorator.py:decorator-gen-108> in matplotlib(self, line)

/opt/conda/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):

/opt/conda/lib/python3.7/site-packages/IPython/core/magics/pylab.py in matplotlib(self, line)
97 print("Available matplotlib backends: %s" % backends_list)
98 else:
---> 99 gui, backend = self.shell.enable_matplotlib(args.gui.lower() if isinstance(args.gui, str) else args.gui)
100 self._show_matplotlib_backend(args.gui, backend)
101

/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py in enable_matplotlib(self, gui)
3412 gui, backend = pt.find_gui_and_backend(self.pylab_gui_select)
3413
-> 3414 pt.activate_matplotlib(backend)
3415 pt.configure_inline_support(self, backend)
3416

/opt/conda/lib/python3.7/site-packages/IPython/core/pylabtools.py in activate_matplotlib(backend)
312
313 import matplotlib.pyplot
--> 314 matplotlib.pyplot.switch_backend(backend)
315
316 # This must be imported last in the matplotlib series, after

/opt/conda/lib/python3.7/site-packages/matplotlib/pyplot.py in switch_backend(newbackend)
235 "Cannot load backend {!r} which requires the {!r} interactive "
236 "framework, as {!r} is currently running".format(
--> 237 newbackend, required_framework, current_framework))
238
239 rcParams['backend'] = rcParamsDefault['backend'] = newbackend

ImportError: Cannot load backend 'Qt5Agg' which requires the 'qt5' interactive framework, as 'headless' is currently running
2 Likes
2

Hi @adriente thanks for raising this question! I think the qt backend is trying to open a window on your machine and because it’s running in a remote container it’s not possible. I’m not sure if there is some inline qt mode for jupyter?

Regarding your second question, can you be a bit more specific about how you imagine it working?

3

Hi @rrrrrok, thanks for this quick reply.

I forgot to mention that I tried %matplotlib nbagg with the configuration described above.

import matplotlib.pyplot as plt
plt.plot([1,2,3])
Javascript Error: IPython is not defined
[<matplotlib.lines.Line2D at 0x7fb054c575c0>]

I think that the notebook interface is particularly adapted for data exploration. I mainly use the hyperspy library to interactively explore microscopy dataset ( https://hyperspy.org/ ). With this library it is possible to create Region of Interest widgets notably to manullay pick up data.

How I would imagine it would work :
–> I pick up a rectangular area of my data and then my notebook would gather (top,left,bottom,right) of the selected areas (maybe in a file???).
–> Still in my notebook I might also want to perform additional processing (such as denoising or background removal) and visualize the result to check it was correctly done.
–> Then my data treatment script would use the outputs of the notebook and process the data originating from the selected regions.
–> Renku would then help to keep track of which regions were used for a given dataset and which data processing was used.

I am not exactly sure how it would be possible in Renku to properly keep track of everything while still having a rather handy process. Maybe I am asking for too much ?

4

Hi @adriente sorry I let this drop a bit - the use-case you describe is very close to what @hluetck has been doing for another imaging project. Maybe you can share the experience @hluetck? I’m not an expert on widgets in notebooks, but the notebooks I saw were pretty convincing :smile:

I’m not sure how flexible hyperspy is in the choice of a backend? Does it need qt specifically?

5

Hi @adriente sorry for the delayed reply, we are a bit ‘under water’ at the moment. I am using a combination of ipywidgets and matplotlib widgets to implement the interactive part of an image processing pipeline. This works well as long as I use the ‘classic’ Jupyter notebook interface (not JupyterLab). For example, to select a rectangular ROI:

from matplotlib import pyplot as plt
from matplotlib.widgets import RectangleSelector

%matplotlib notebook

fig = plt.figure()
ax1 = fig.add_subplot(1,1,1)
ax1.imshow(img, cmap='gray', interpolation='none')
ax1.set_title('Please select ROI for analysis\n(Click & drag)')
rect = RectangleSelector(ax1, ui.line_select_callback, drawtype='box', useblit=False, 
                         button=[1, 3],  # don't use middle button
                         minspanx=5, minspany=5, spancoords='pixels', interactive=True)
plt.show()

roi_coordinates = [int(round(x)) for x in rect.extents]
print('Selected ROI coordinates:')
print(*roi_coordinates)

Hope this helps. Let me know if you need more information.
Cheers, Henry

6

Hi,

Thanks for the answers. I have not been really quick either.
The code you sent doesn’t work and I still get Javascript Error: IPython is not defined.

Could you send your “requirements.txt” file and DockerFile because it seems that I am missing something here.

@rrrrrok hyperspy does not need qt specifically. It should work fine with the nbagg backend.

Best,

Adrien

7

Here is my environment.yml:

name: "base"
channels:
  - conda-forge
dependencies:
  - python=3.7
  - dask=2.11.0
  - dask-labextension>=1.0.3
  - distributed=2.11.0
  - graphviz=2.42.3
  - h5py=2.10.0
  - ipykernel=5.1.3
  - ipywidgets>=7.5
  - matplotlib=3.1.2
  - nodejs=13.9.0
  - numpy=1.17.3
  - papermill=2.0.0
  - pip=19.3.1
  - python-graphviz=0.13.2
  - scikit-image=0.16.2
  - scipy=1.4.1
  - toolz=0.10.0
  - pip:
    - joblib==0.14.1
    - opencv-python==4.1.2.30
    - pystackreg==0.2.2
    - tqdm==4.40.2
    - wget==3.2
prefix: "/opt/conda"

and the Dockerfile:

FROM renku/singleuser:latest

# install the python dependencies
COPY requirements.txt environment.yml /tmp/
RUN conda env update -q -f /tmp/environment.yml && \
    /opt/conda/bin/pip install -r /tmp/requirements.txt && \
    conda clean -y --all && \
    conda env export -n "root"
1 Like
8

@hluetck I don’t know what I am doing wrong but I tried to use the same DockerFile and environment.yml and I still get Javascript Error: IPython is not defined.

I will try to work without interactivity for the moment then.

9

are you using JupyterLab or Jupyter notebook? When you start your environment, does the URL end with /lab or /tree? You can start the ‘classic’ Jupyter Notebook interface by changing the /lab part to /tree.

10

I tried to change from \lab to \tree and it actually works I can use the nbagg backend without any problem.

I guess there is no way to use the qt or nbagg environment in jupyter lab. That should not be a big deal for me now that i know it.

Thanks for your help.

1 Like
11

Great that it works. You can set default_url = /tree in your renku.ini file (in the .renku folder) and it will start your environment in Jupyter notebook mode automatically.

1 Like