ISR-UoL 3D Social Activity Dataset

This is a social interaction dataset between two subjects. This dataset consists of RGB and depth images, and tracked skeleton data (i.e. joints 3D coordinates and rotations) acquired by an RGB-D sensor. It includes 8 social activities: {handshake, greeting hug, help walk, help stand-up, fight, push, conversation, call attention}. Each activity was recorded in a period around 40 to 60 seconds of repetitions within the same session at a frame rate of 30 frames per second. The only exceptions are help walking (at a short distance) and help stand-up, which were recorded 4 times to the same session, regardless of the time spent on it.

The activities were selected to address the assisted living scenario (e.g. happening in a health care environment: help walking, help stand-up and call attention), with potential harm situations, such as aggression (e.g. fighting, pushing), and casual activities of social interactions (e.g. handshake, greeting hug and conversation). The activities were performed by 6 persons, 4 males and  2 females with an average age of  29 years old, from different nationalities (Italian, Brazilian and Portuguese). A total of 10 different combinations of individuals (or sessions) were performed, with variation of the roles (active or passive person) between the subjects. Each subject has participated at least with 3 combinations, acting each role at least once. Half of the recorded sessions have been performed by a pair of persons whom never met before the interaction, This was done in order to increase the generalization of the study regarding individual behavior.

Dataset structure, download and conditions of use

The dataset is composed of 10 sessions. Each session provides RGB-D images and skeleton tracks of 8 different activities performed by two people. Each session is zipped in a separate file, which contains a folder that has skeleton tracks in a text format and RGB (24 bits) and depth (both 8 and 16 bit resolution) images. Each row of the skeleton text file contains information about positions (6 DoF) of 15 joints. The work is currently in review of IROS 2016. Source code to extract skeleton data and ROS-compatible files will be provided after the IROS review process.

Individual sessions for download
Session-01 Session-02 Session-03 Session-04 Session-05
Session-06 Session-07 Session-08 Session-09 Session-10

If you use the dataset for your research, please cite our paper that describes the data collection in detail. The paper is under review for IROS 2016 and we will provide the full text on request, contact the paper corresponding author. We attached a bibtex record for your convenience.



Institute of Systems and Robotics (ISR-UC), University of Coimbra, Portugal  and  L-CAS, University of Lincoln, UK.

Contact: Claudio Coppola, PhD Candidate ( and Dr Diego R. Faria ( )
Dataset pages maintained by Tom Krajnik as part of the STRANDS project dissemination activities.