# Renku dataset import is extremely slow

**URL:** <https://renku.discourse.group/t/renku-dataset-import-is-extremely-slow/759>\
**Category:** Renku (CLI)\
**Created:** [20 February 2023 08:37 UTC](https://renku.discourse.group/t/renku-dataset-import-is-extremely-slow/759 "2023-02-20T08:37:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mattstark](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@mattstark](https://renku.discourse.group/u/mattstark)\
**Post date:** [20 February 2023 08:37 UTC](https://renku.discourse.group/t/renku-dataset-import-is-extremely-slow/759/1 "2023-02-20T08:37:42Z")

</div>

When importing a dataset from one repository to use in a session from another repository, it takes an extremely long time:

In repository A, I made a dataset using the command:

```auto
wget http://images.cocodataset.org/zips/val2017.zip
unzip val2017.zip
renku dataset add --create --move coco val2017
rm val2017.zip
renku save -m "added coco dataset"

```

This already takes quite a while ~1 hour for 5000 files, 77 MB

In a session for repository B, I try importing this dataset:

```auto
renku dataset import -y https://limited.renku.ch/datasets/c0700196bc954037994d1c201e5b34c3

```

This took about 10 hours. The session doesn’t seem to be resource limited, it was started with 2 cores and 16 gigs of ram.

Repeating the process for a 500MB repository has been running for 60 hours now.

These are repositories of standard image datasets that aren’t all that big. What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![mohammad-sdsc](https://yyz2.discourse-cdn.com/free1/user_avatar/renku.discourse.group/mohammad-sdsc/32/79_2.png) [@mohammad-sdsc](https://renku.discourse.group/u/mohammad-sdsc)\
**Post date:** [22 February 2023 13:30 UTC](https://renku.discourse.group/t/renku-dataset-import-is-extremely-slow/759/2 "2023-02-22T13:30:13Z")

</div>

Hello @mattstark,

Thanks for reporting this issue to us!

You are doing everything right. Unfortunately, it’s Renku CLI that is slow when adding a large number of files (regardless of their size). We’ve created a story to address this issue (normally will be done in 2-3 weeks). I’ll keep you posted once it’s done.

Kind regards,  
Mohammad