Uploading images failed on MacOS

Hello,

I am a new user and Remo really looks great!

I created a new data set using local data. I selected my current file with label and I inserted the local path to the fold containing images on an external hard disk (10k images).

Remo began to upload data but it is very very slow. It displays “Processing images : 1 of 10000” since 20 miunutes …

Did I do something wrong ?

Thank you

1 Like

Hello, @pytlab, thanks for using Remo :slight_smile:

Could you provide more details, that help to identify the issue.

  • How did you upload data, from UI or from code?
    ** if from UI, did you use tab “Use local data” or “Upload data”?
  • What do you mean that your current file has label?
  • How big are the images?
  • Where is your external hard disk? is it local or in the cloud?
  • Could you provide output from console

Hi thank you for your answer !

  1. I upload data via the remo app and I use “Use local data”
  2. I charged also a csv file containing the list of images to label; the goal is to add a new labelisation via remo.
  3. Each image is around 500x500 and 500KB
  4. I use an external hard disk connected to my computer via a usb port
  5. The terminal just says “Remo app running on http://localhost:8123. Press Control-C to stop it.”

Thank you

I tried to do the same think but using a fold containing 4 images and located on my desktop. Same think happens … it can not upload images.

Below the console messages

objc[35290]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fff8e1c03d8) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x112039f50). One of the two will be used. Which one is undefined.

[12:35:39 RemoApp] INFO - Adding Images and Annotations to Dataset test (ID 5), in Annotation set Label2 (ID 5)

[12:35:39 RemoApp] INFO - Collecting annotation files - found 1 files

[12:35:39 RemoApp] INFO - Collecting image files - found 4 files

(process:46033): GLib- CRITICAL **: 12:35:39.347: g_datalist_id_set_data_full: assertion ‘key_id > 0’ failed

(process:46033): GLib-GObject- CRITICAL **: 12:35:39.347: g_param_spec_pool_lookup: assertion ‘pool != NULL’ failed

(process:46033): GLib-GObject- WARNING **: 12:35:39.347: g_object_set_is_valid_property: object class ‘(null)’ has no property named ‘filename’

(process:46033): GLib- CRITICAL **: 12:35:39.347: g_datalist_id_set_data_full: assertion ‘key_id > 0’ failed

(process:46033): GLib-GObject- CRITICAL **: 12:35:39.347: g_param_spec_pool_lookup: assertion ‘pool != NULL’ failed

(process:46033): GLib-GObject- WARNING **: 12:35:39.347: g_object_set_is_valid_property: object class ‘(null)’ has no property named ‘access’

@pytlab you can check this thread https://github.com/libvips/pyvips/issues/24
here described similar issues that you have

you can try to install remo in new fresh venv or conda environment

I am already using conda.

@pytlab thanks for the feedback!

to expand on this:
It doesn’t sound like you did anything wrong :slight_smile:
We couldn’t manage to replicate the behaviour, but it looks like the error comes from libvips library, which Remo uses to process images.

In the linked git issue, somebody was experiencing a similar issue in MacOSX and it seems that installing it with a fresh python environment fixed it.

Let us know if try it and it works or you still have issues!

Andrea

thank you for your answer, I just tried.

I created a new environment and I installed remo and remo-python library. Then I launched the remo-app from the new environment and I created a new dataset where I uploaded 23 images as local data using the path for data basis location on my Mac.

Same thing happens:

[10:46:08 RemoApp] INFO - Adding Images to Dataset test (ID 15)

[10:46:08 RemoApp] INFO - Collecting image files - found 23 files

(process:4101): GLib- CRITICAL **: 10:46:08.382: g_datalist_id_set_data_full: assertion ‘key_id > 0’ failed

(process:4101): GLib-GObject- CRITICAL **: 10:46:08.382: g_param_spec_pool_lookup: assertion ‘pool != NULL’ failed

(process:4101): GLib-GObject- WARNING **: 10:46:08.382: g_object_set_is_valid_property: object class ‘(null)’ has no property named ‘filename’

(process:4101): GLib- CRITICAL **: 10:46:08.382: g_datalist_id_set_data_full: assertion ‘key_id > 0’ failed

(process:4101): GLib-GObject- CRITICAL **: 10:46:08.382: g_param_spec_pool_lookup: assertion ‘pool != NULL’ failed

(process:4101): GLib-GObject- WARNING **: 10:46:08.382: g_object_set_is_valid_property: object class ‘(null)’ has no property named ‘access’

Thank you for your help

Ok I see. I can suggest two options:

1. Try installing using Docker.

Even if you haven’t used Docker before, it should be fairly straightforward.

You can see instructions here: https://remo.ai/docs/docker-installation/

2. Try other options to install using pip

I would try first to make sure Vips it’s working. You can run the following, and if it fails it’s surely a problem related to that library

import pyvips
image = pyvips.Image.new_from_file("IMG_0015_guides.jpg")

Once we know for certain it’s a problem with Vips, we can try and debug it.
Looking at the issues on their repo, some other similar issues appeared:

Solution seems to be something along these lines:

  • surely try to install vips via homebrew
  • possibly try using a venv, instead of a conda env

Hope this helps, keep us posted!

Hi Andrea,

I have actually a problem with the vips library (parameter filename not set).

I tried to re-install vips via homebrew and install remo in a venv. However I get the same problem.

I never used Docker I am not sure to handle it.

That’a a pity ! Is this issue specific to MacOs ?

Thank you

The issue should be Mac OS specific, but also possibly down to some specific settings/versions…we tested it on different versions Mac OS and it worked fine.

I’d suggest to have a crack at Docker!
I followed the instructions without knowing much myself and installation went through smoothly :slight_smile:
(it was on Ubunt, but it should work without problems on Mac too)
And we are here if something doesn’t work

@pytlab you mentioned problem with vips was parameter

I have actually a problem with the vips library (parameter filename not set).

maybe in your lib version you need to specify this parameter explicitly

import pyvips
image = pyvips.Image.new_from_file(vips_filename="IMG_0015_guides.jpg")

the issue that library was not build correctly maybe related to python version that system uses when you install vips library in brew
on Mac you have some pre-installed python, you can install python from brew and download from python website

but then it’s important which python is set in your PATH
can you check in new terminal window two commands: which python and python

here is how it looks on my system:

➜  ~ which python
/Users/username/miniconda3/bin/python
➜  ~ python
Python 3.7.1 (default, Dec 14 2018, 13:28:58)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

which python gives
/opt/anaconda3/bin/python
and I have python 3.7.6

Hi Andrea,
finally I got it via Docker! And is works fine … at least to upload images :slight_smile:
What about data security via the browser ? I should actually use it in a professional context.

Grazie!

1 Like

Great stuff!

Data is kept locally on your laptop, so no problem using it in a professional context - there are a few companies using it already like this.

You access Remo from the browser, but you will notice that the address is localhost - i.e. the server (and the database) is running locally on your machine.

You maybe saw the section about sharing access with other users in the documentation - that’s something to worry about only if you are using a Virtual Machine and you open the machine itself to the web

Great! Thank you for your help!

1 Like

Hi Andrea!

Every works fine now! That’s great!

Just two questions:

  1. I am trying to upload some images and an annotation file .csv that should be modified with a new column annotation : remo tells me that the images names in the .csv file and in the image folder do not correspond. This is of course false… do you know why ? Anyway it is not really important because I can upload images only and define a new label set from scratch.

  2. If during uploading images remo detects some duplicates, is it possible to get the list of the them and delete the redondant images ?

Thank you again!

Hi @pytlab, great to hear!

I am trying to upload some images and an annotation file .csv that should be modified with a new column annotation : remo tells me that the images names in the .csv file and in the image folder do not correspond. This is of course false… do you know why ? Anyway it is not really important because I can upload images only and define a new label set from scratch.

I’m not sure I follow the issue here. Could you maybe attach an example csv file?
You can also checkout our documentation and the corresponding examples on the formats: https://remo.ai/docs/annotation-formats/

If during uploading images remo detects some duplicates, is it possible to get the list of the them and delete the redondant images ?

That’s already the default behaviour :slight_smile: you’d get a notification on the bell component, showing which images have duplicates and that hence are not being added again.

Hi Andrea!

I used REMO to annotate my dataset, it works really well. Juste 2 questions:

  1. During the annotation each image is shown alone in order to chose the label. Is it possible to show all images in a mosaic-style display in order to select several images together and affect them to the same class ?

  2. I need now to refine my annotation. I would like to do that class by class. Now, when I get into a class, I can see all the images but I can not correct their classes. Do I do something wrong ?

Thanks

Hi @pytlab

Glad you are finding Remo useful! No biggie but it’d be good if you could open a different thread for different questions, just to keep things a bit more tidy :slight_smile:

Unfortunately both options are not supported at present. We could do some custom development for you if you are interested. Let me know in case and we can have chat

Best
Andrea