# Job's log exceeded limit of 4194304 bytes

**URL:** <https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183>\
**Category:** RenkuLab\
**Created:** [29 October 2020 16:50 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183 "2020-10-29T16:50:15Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![lgatto](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/lgatto/32/64_2.png) [@lgatto](https://renku.discourse.group/u/lgatto)\
**Post date:** [29 October 2020 16:50 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/1 "2020-10-29T16:50:15Z")

</div>

The following [job](https://renkulab.io/gitlab/laurent.gatto/wsbim2122-20201029/-/jobs/109429) failed with message

```auto
Job's log exceeded limit of 4194304 bytes.

```

I couldn’t immediately find a parameter to increase that limit - does anyone have a suggestion?

---

<div class="post-metadata">

**Author:** ![lgatto](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/lgatto/32/64_2.png) [@lgatto](https://renku.discourse.group/u/lgatto)\
**Post date:** [29 October 2020 16:57 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/2 "2020-10-29T16:57:15Z")

</div>

> [@lgatto](#):
>
> `Job's log exceeded limit of 4194304 bytes.`

It looks like [setting `output_limit`](https://gitlab.com/gitlab-org/gitlab-runner/-/blob/master/docs/configuration/advanced-configuration.md#the-runners-section) to a greater value is the solution. I am however not sure where to set it.

**Edit** I am trying with `~/gitlab-runner/config.toml`.

---

<div class="post-metadata">

**Author:** ![cchoirat](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/cchoirat/32/115_2.png) [@cchoirat](https://renku.discourse.group/u/cchoirat)\
**Post date:** [29 October 2020 17:08 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/3 "2020-10-29T17:08:11Z")

</div>

Maybe try to use `quietly = TRUE` when installing packages.

---

<div class="post-metadata">

**Author:** ![rrrrrok](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/rrrrrok/32/117_2.png) [@rrrrrok](https://renku.discourse.group/u/rrrrrok)\
**Post date:** [29 October 2020 17:20 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/4 "2020-10-29T17:20:40Z")

</div>

Those are settings in the gitlab runner itself, which you don’t have access to. The best way to fix this is to do what @cchoirat suggested and reduce the verbosity of the package compilation. If you need to debug some step of the build, I suggest you try to build it locally where you have more control and once you’re happy that it works you can move it to renkulab. Happy to assist with the local part if you need!

---

<div class="post-metadata">

**Author:** ![lgatto](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/lgatto/32/64_2.png) [@lgatto](https://renku.discourse.group/u/lgatto)\
**Post date:** [29 October 2020 18:33 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/5 "2020-10-29T18:33:36Z")

</div>

Thanks @cchoirat - that is very significantly reducing the output and should sort my problem.

---

<div class="post-metadata">

**Author:** ![cchoirat](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/cchoirat/32/115_2.png) [@cchoirat](https://renku.discourse.group/u/cchoirat)\
**Post date:** [29 October 2020 19:03 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/6 "2020-10-29T19:03:07Z")

</div>

`Error: Failed to install 'Spectra' from GitHub`

Would we need to update the BioC image to `3.12`?

---

<div class="post-metadata">

**Author:** ![lgatto](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/lgatto/32/64_2.png) [@lgatto](https://renku.discourse.group/u/lgatto)\
**Post date:** [29 October 2020 19:21 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/7 "2020-10-29T19:21:19Z")

</div>

That’s what I’m doing at the moment, manually (with `BiocManager::install(version = "3.12")`) but it would be great to have the latest Bioconductor 3.12 docker ready.

(Note that the `Spectra` error is probably related to something else).

---

<div class="post-metadata">

**Author:** ![rrrrrok](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/rrrrrok/32/117_2.png) [@rrrrrok](https://renku.discourse.group/u/rrrrrok)\
**Post date:** [29 October 2020 22:54 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/8 "2020-10-29T22:54:55Z")

</div>

oh I didn’t realize there was 3.12 - I can add it to the build.

I’m wondering though if it would make sense to set this `quietly = TRUE` option per default in R projects - where would this normally be set in order to apply globally?

---

<div class="post-metadata">

**Author:** ![lgatto](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/lgatto/32/64_2.png) [@lgatto](https://renku.discourse.group/u/lgatto)\
**Post date:** [30 October 2020 07:40 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/9 "2020-10-30T07:40:38Z")

</div>

I’m not sure where to set this globally - I didn’t find it in the `options`. It’s a parameter to the package installation functions:

```auto
install.packages("one_package", quiet = TRUE)
BiocManager::install("other_package", quiet = TRUE)

```

If the installation goes wrong, there no message to debug.

Note also that in my case, a lot of extra output was repeat unnecessarily due to the new Bioconductor release:

> Bioconductor version ‘3.11’ is out-of-date; the current release version ‘3.12’  
> is available with R version ‘4.0’; see [https://bioconductor.org/install`](https://bioconductor.org/install%60)

---

<div class="post-metadata">

**Author:** ![rrrrrok](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/rrrrrok/32/117_2.png) [@rrrrrok](https://renku.discourse.group/u/rrrrrok)\
**Post date:** [30 October 2020 13:47 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/10 "2020-10-30T13:47:31Z")

</div>

Right - I tried to add 3.12 yesterday and they haven’t released their docker images yet. As soon as they do we’ll have a base image based on those.

Regarding the logs: is there no message in the case of a compilation error when you set `quiet=TRUE`?

---

<div class="post-metadata">

**Author:** ![lgatto](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/lgatto/32/64_2.png) [@lgatto](https://renku.discourse.group/u/lgatto)\
**Post date:** [30 October 2020 14:15 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/11 "2020-10-30T14:15:28Z")

</div>

I am not sure if all or only some messages are silenced in case of error. I had an R-level error yesterday that didn’t show up.

---

<div class="post-metadata">

**Author:** ![rrrrrok](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/rrrrrok/32/117_2.png) [@rrrrrok](https://renku.discourse.group/u/rrrrrok)\
**Post date:** [30 October 2020 14:41 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/12 "2020-10-30T14:41:25Z")

</div>

Is there a `quiet = sometimes` option? 🙂

Would be great if we can have some default here that would work for most people - it’s pretty frequent that really long-running builds fail due to the max log restriction and it’s very frustrating for everyone. If you land on some configuration that you think would work well please let me know and we’ll build it into the templates.

---

<div class="post-metadata">

**Author:** ![cchoirat](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/cchoirat/32/115_2.png) [@cchoirat](https://renku.discourse.group/u/cchoirat)\
**Post date:** [30 October 2020 14:58 UTC](https://renku.discourse.group/t/jobs-log-exceeded-limit-of-4194304-bytes/183/13 "2020-10-30T14:58:48Z")

</div>

Maybe pre-build and/or cache packages along the lines of [https://github.com/r-builder/cran2deb](https://github.com/r-builder/cran2deb)?
