Importing a dataset from sandbox.Zenodo failed

Renku (CLI)
1

Hi,

When trying to import a dataset from Zenodo [sandbox] (https://sandbox.zenodo.org/record/830043#.YKZIH6gzZaQ), I get:

> ## Describe the bug
> A clear and concise description.
> 
> ## Details
> *Please verify and redact the details.*
> 
> **Renku version:** 0.14.2
> **OS:** Linux (#119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020)
> **Python:** 3.8.6

### Traceback

```
Traceback (most recent call last):
  File "[...]/renku/cli/exception_handler.py", line 123, in main
    result = super().main(*args, **kwargs)
  File "[...]/renku/cli/exception_handler.py", line 89, in main
    return super().main(*args, **kwargs)
  File "[...]/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "[...]/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[...]/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "[...]/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "[...]/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "[...]/renku/cli/dataset.py", line 683, in import_
    import_dataset().with_communicator(communicator).build().execute(uri=uri, name=name, extract=extract, yes=yes)
  File "[...]/renku/core/incubation/command.py", line 131, in execute
    output = context["click_context"].invoke(self._operation, context["client"], *args, **kwargs)
  File "[...]/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "[...]/renku/core/commands/dataset.py", line 455, in _import_dataset
    record = provider.find_record(uri, client)
  File "[...]/renku/core/commands/providers/zenodo.py", line 519, in find_record
    return self.find_record_by_doi(uri)
  File "[...]/renku/core/commands/providers/zenodo.py", line 525, in find_record_by_doi
    doi = DOIProvider().find_record(doi)
  File "[...]/renku/core/commands/providers/doi.py", line 111, in find_record
    response = self._query(uri).json()
  File "[...]/requests/models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "[...]/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "[...]/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "[...]/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

## Additional context
Add any other context about the problem.

Any help is appreciated.

2

It looks like you called the command with the DOI of the dataset, but sandbox DOI’s aren’t real, e.g. https://doi.org/10.5072/zenodo.830043 this is the DOI lookup for this dataset, which doesn’t exist.

If you instead run renku dataset import https://sandbox.zenodo.org/record/830043 that should work. Though when I just tested this, it fails because the metadata sent by Zenodo is formatted badly. I just tried with a non-sandbox dataset and that works, so I guess there’s some discrepancy there.

We don’t officially support importing from Zenodo Sandbox, but I can have a quick look what exactly the issue is here

3

Hi Amir,
As Ralf mentioned, we don’t support importing dataset from Zenodo Sandbox since it’s ephemeral and gets purged from time to time. This means that dataset import https://sandbox.zenodo.org/record/830043 won’t work either (it actually tries to import https://zenodo.org/record/830043 which is a different dataset than yours).

4

Thank you so much @ralf.grubenmann and @mohammad-sdsc for your answers. If there would be a way to also import data from sandbox Zenodo, that would be great. As people may want to first check whether everything works smoothly, then publish it on the (main) Zenodo.

5

We discussed it internally and it is unlikely that we will support importing from the Zenodo Sandbox. The Zenodo Sandbox environment can be deleted at any time and as such, a new dataset might get the same id as an old one, which could lead to confusion or code issues.

If you export a dataset from renku using renku dataset export without the -p flag specified, it will export as a draft, so you can use that to test things. And you should be able to import a draft dataset as well. So I think that’s a better approach to testing things.

1 Like