Egocentric point of interest recognition in cultural sites

We consider the problem of the detection and recognition of points of interest in cultural sites. We observe that a “point of interest” in a cultural site may be either an object or an environment and highlight that the use of an object detector is beneficial to recognize points of interest which occupy a small part of the frame. To study the role of objects in the recognition of points of interest, we augment the labelling of the UNICT-VEDI dataset to include bounding box annotations for 57 points of interest. We hence compare two approaches to perform the recognition of points of interest. The first method is based on the processing of the whole frame during recognition. The second method employs a YOLO object detector and a selection procedure to determine the currently observed point of interest. Our experiments suggest that further improvements on point of interest recognition can be achieved fusing the two methodologies. Indeed, the results show the complementarity of the two approaches on the UNICT-VEDI dataset.