I am trying to make nix available/executable in renku. Specifically, I am looking for a way to make it possible to deploy nix tailored to R projects via {rix}, that a colleague from Luxembourg and me are currently developing.
Main reason of going through nix is the ultimate goal that Win and Mac users can easily go from cloud to local, without hassle of different IDEs, manually downloading quarto from Posit etc.
I used the Determinate Systems installer in the Dockerfile, and used --no-init since systemd is not enabled in the renku containerization engine. The build looks green:
corresponding commit that led to that build is:
d12560f116a07bba78efe2b3b8f3397185cdf7af , which is the Dockerfile in the current master:
When I connect to the renku instance, in the browser it is RStudio that is the entry point, I cannot execute it (guess because root owned).
base nix ❯ var ❯ … ❯ profiles ❯ default ❯ bin $ ll
total 3752
dr-xr-xr-x 2 root root 4096 Aug 16 08:42 ./
dr-xr-xr-x 7 root root 4096 Aug 16 08:42 …/
-r-xr-xr-x 1 root root 3831976 Jan 1 1970 nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-build → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-channel → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-collect-garbage → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-copy-closure → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-daemon → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-env → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-hash → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-instantiate → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-prefetch-url → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-shell → nix*
lrwxrwxrwx 1 root root 3 Jan 1 1970 nix-store → nix*
However, the above shows the nix store is there. Would you see any way to make this available to users, given there is no systemd in the running container instance?
Hi @philipp-baumann this is a great initiative, thanks for the question! I spent some time a while back investigating nix as a way to build and distribute our images, but decided that it was too much complexity to expose end-users to it. Recently I’ve been looking at dev containers as a simpler alternative.
For dev containers, rocker-org recently released the rstudio feature which makes installing RStudio on top of any (ubuntu-based) docker image quite easy.
The Renku base images that use RStudio use the rsession-proxy - maybe there are some assumptions made in there that don’t hold for your nix-based build? Have you tried running your image locally so you can debug more easily?
actually I just tried running your project and it put me into the RStudio session fine - but it’s using the one installed in our base image, not from the nix store afaict.
Hi @rrrrrok thanks a lot for providing the links to dev containers, I will definitively check this out.
Currently #nix is getting a bit more attention in the R community. I agree there is still quite a bit of complexity behind. Nevertheless, I think there is great potential, especially when new tooling bridges to established ones, lowering the burden to enter such setups.
Since the Nix R and R agnostic development environments means quite some heavy lifting from a few folks, the versions are lagging behind, which is not a bad thing I think. There are still some stuff issues in combination with reticulate python e.g.
In my Dockerfile example using {rix} I wasn’t planning to replace anything in the Docker templated Rstudio online IDE, because I think the setup you provide in the cloud is really solid
Fair point, I will definitively debug locally, that makes it way easier.
Regarding using nix for data science, I also think it’s good to have some tools that are partly complementary but partly unique in features. The challenging part is always that the devil lays in the details. For this, my plan is currently:
extend the {renv} workflow so that one can in one shot convert R version + system libraries + package source from renv.lock to a default.nix equivalent. Another thing to come soon is support for flakes in {rix}
I’m a fan of nix (use it on my mac) but I think it’s too much to expose it to users. Docker is already too high of a hurdle for most people. But, if there is a slice of the R community behind these efforts, that’s very cool! Once you get something that works we can add a new contributed project template if you think it would be useful.
In my Dockerfile example using {rix} I wasn’t planning to replace anything in the Docker templated Rstudio online IDE, because I think the setup you provide in the cloud is really solid
Oh I misunderstood then - what exactly are you trying to do? What wasn’t working?
I’m a fan of nix (use it on my mac) but I think it’s too much to expose it to users. Docker is already too high of a hurdle for most people. But, if there is a slice of the R community behind these efforts, that’s very cool! Once you get something that works we can add a new contributed project template if you think it would be useful.
Great! Once working and having it tested in the course, I’d definitively upload as a contributed project template
Oh I misunderstood then - what exactly are you trying to do? What wasn’t working?
nix is installed on the cloud instance of my project mentioned. I could not run the nix CLI it when I tried to do so in the terminal. Now I changed to bash (guess was in zsh before in RStudio terminal. However, now I get an error
base ▶ nix ❯ var ❯ … ❯ profiles ❯ default ❯ bin ▶ $ ▶ nix-shell -p nix-info --run "nix-info -m"
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by nix-shell)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by nix-shell)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by nix-shell)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by nix-shell)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/il7dydwf1wfn9b4p4cii8q99b93i106n-editline-1.17.1/lib/libeditline.so.1)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixexpr.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixexpr.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixexpr.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixmain.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixmain.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixmain.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixfetchers.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixfetchers.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixfetchers.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixstore.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixstore.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixstore.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixstore.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixstore.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixutil.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixutil.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixutil.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixutil.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixutil.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixcmd.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixcmd.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixcmd.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /nix/store/3wqasl97rjiza3vd7fxjnvli2w9l30mk-nix-2.17.0/lib/libnixcmd.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/4mxnw95jcm5a27qk60z7yc0gvxp42b9a-openssl-3.0.7/lib/libcrypto.so.3)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/4mxnw95jcm5a27qk60z7yc0gvxp42b9a-openssl-3.0.7/lib/libcrypto.so.3)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-transfer.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-transfer.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-s3.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-s3.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-s3.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-s3.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-core.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-core.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-core.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /nix/store/5glq0q102jva5p0h9ap0f1j9ps24gcc0-aws-sdk-cpp-1.9.294/lib/libaws-cpp-sdk-core.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/5q73izqsdasz81gma1nz6870b6220vkh-aws-crt-cpp-0.18.9/lib/libaws-crt-cpp.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/5q73izqsdasz81gma1nz6870b6220vkh-aws-crt-cpp-0.18.9/lib/libaws-crt-cpp.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by /nix/store/5q73izqsdasz81gma1nz6870b6220vkh-aws-crt-cpp-0.18.9/lib/libaws-crt-cpp.so)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/f116ly4fl72zc5ynb03qrwdk2bp8y61p-aws-c-io-0.13.11/lib/libaws-c-io.so.1.0.0)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/4rkhsf7sig2lh303bygqr3ph5mfwz0ah-s2n-tls-1.3.28/lib/libs2n.so.1)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/4rkhsf7sig2lh303bygqr3ph5mfwz0ah-s2n-tls-1.3.28/lib/libs2n.so.1)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/6qqybxxz6636jymz8x7l2sxj4np9yzsy-aws-c-cal-0.5.20/lib/libaws-c-cal.so.1.0.0)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/3j1h6psl4pzn6b3yck6rk33bpwrmihb1-aws-c-common-0.8.5/lib/libaws-c-common.so.1)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /nix/store/3j1h6psl4pzn6b3yck6rk33bpwrmihb1-aws-c-common-0.8.5/lib/libaws-c-common.so.1)
nix-shell: /usr/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /nix/store/3j1h6psl4pzn6b3yck6rk33bpwrmihb1-aws-c-common-0.8.5/lib/libaws-c-common.so.1)
I might be related to the compiler suite (also installed rust), so maybe LD_LIBRARY_PATH stuff. If this error sounds familiar to you, would be great if you have a solution. Otherwise I won’t bother you too much and try a bit more
nix ❯ var ❯ … ❯ profiles ❯ default ❯ bin 
thanks so much