We consider the problem of object segmentation in cultural sites. Since collecting and labeling large datasets of real i mages is challenging, we investigate whether the use of synthetic i mages can be useful to achieve good segmentation performance on real data. To perform the study, we collected a new dataset comprising both real and synthetic images of 24 artworks in a cultural site. The synthetic images have been automatically generated from the 3D model of the considered cultural site using a tool developed for that purpose. Real and synthetic images have been labeled for the task of semantic segmentation of artworks. We compare three different approaches to perform object segmentation exploiting real and synthetic data. The experimental results point out that the use of synthetic data helps to improve the performances of segmentation algorithms when tested on real i mages. Satisfactory performance is achieved exploiting semantic segmentation together with image-to-image translation and including a small amount of real data during training. To encourage research on the topic, we publicly release the proposed dataset at the following url: https://iplab.dmi.unict.it/EGO-CH-OBJ-SEG/.

Semantic Object Segmentation in Cultural Sites using Real and Synthetic Data

Ragusa, F
Primo
;
Furnari, A;Farinella, GM
2021-01-01

Abstract

We consider the problem of object segmentation in cultural sites. Since collecting and labeling large datasets of real i mages is challenging, we investigate whether the use of synthetic i mages can be useful to achieve good segmentation performance on real data. To perform the study, we collected a new dataset comprising both real and synthetic images of 24 artworks in a cultural site. The synthetic images have been automatically generated from the 3D model of the considered cultural site using a tool developed for that purpose. Real and synthetic images have been labeled for the task of semantic segmentation of artworks. We compare three different approaches to perform object segmentation exploiting real and synthetic data. The experimental results point out that the use of synthetic data helps to improve the performances of segmentation algorithms when tested on real i mages. Satisfactory performance is achieved exploiting semantic segmentation together with image-to-image translation and including a small amount of real data during training. To encourage research on the topic, we publicly release the proposed dataset at the following url: https://iplab.dmi.unict.it/EGO-CH-OBJ-SEG/.
2021
978-1-7281-8808-9
File in questo prodotto:
File Dimensione Formato  
Semantic_Object_Segmentation_in_Cultural_Sites_using_Real_and_Synthetic_Data.pdf

solo gestori archivio

Tipologia: Versione Editoriale (PDF)
Licenza: NON PUBBLICO - Accesso privato/ristretto
Dimensione 1.57 MB
Formato Adobe PDF
1.57 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.11769/541769
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 1
social impact