Cannot upload many images - remo removes duplicates

Im experiencing a weird bug. When uploading 84 images to my dataset, named Image(1)-…-image(84) remo displays warning that there is a duplicate e.g image(64) and after 54 images it stops uploading. How is it possible if every image has a different number in its name?

Hi @Maciej_Szuchta -

when uploading pictures to a dataset, remo checks the Image Hashing to see if two pictures have actually the same content (regardless of the filenames).

If some picture has the same content as another existing picture within the dataset, remo issues a warning but still adds the picture to the dataset.

If instead you try to upload a picture with the same name as an existing picture, then remo skips that image and shows an error (but still continues with the uploading)

Now, this is the expected behaviour at least.

From what you are saying, it sounds like:

  1. you are uploading some pictures that have all different filenames
  2. some of these pictures actually have the same content
  3. remo recognizes that some pictures have duplicated the content, and it issues a warning
  4. it doesn’t upload all the pictures

Is this correct?

If yes, then it sounds like a bug.

Would you be able to share the dataset and the log from the console?

Ok I found an issue. Im using Nginx to serve my remo app in local network. Nginx returned 413 - Request Body too large for some files. Increasing the limit fixed it. Sorry to bother you.

1 Like

Ok awesome, and no problem at all.

Out of curiosity, can I ask you if there is a specific reason why you want duplicated pictures inside a dataset?

We were tempted to outright skip duplicate pictures by file content because we didn’t see a use case for it, ended up with a warning as a middle ground.