Face analysis and body language understanding from egocentric cameras

The objective of the project described in this position paper is to develop and evaluate algorithms that enable a mobile agent, e.g., a robot, to observe a user during his/her day to day activities and infer relevant information which could help improve human-machine interaction. To achieve this goal we will first explore intelligent navigation strategies. The overall focus will be on visual data, analysing the user's action, face and body language. Once the algorithms run on the robot, they can be used to log user activity/emotional states and support them during daily activities. The collected information of the users will be useful for further analysis by healthcare professionals or assistive applications. In addition to the mentioned domains, attention will also be paid to speech analysis and synthesis to ensure natural interaction with the user. The algorithms will be able to infer age, gender, emotions, activity and body language of the user. Lastly, information obtained by First Person Vision Systems worn by a user will be considered as an external source of data to make more accurate inferences and explore possible correlations.