OpenRoboCare Dataset

Naming Conventions

Session Identifiers

Sessions use the format: {OT}-{Manikin Type}-{Task}

• 10-1-5 → OT 10, Type 1, Task 5
• 4-0-2 → OT 4, Type 0, Task 2

Tasks

Tasks are numbered 1-15 representing different caregiving activities (see overview page for task list).

File Formats by Modality

RGB Images

• Format: JPEG files named {frame}_anonymized.jpg
• Content: Anonymized RGB images with faces blurred
• Location: RGB/OT{N}/Task{N}/Cam{N}/RGB/

Depth Maps

• Format: PNG files named {frame}.png
• Content: 16-bit depth maps corresponding to RGB frames
• Location: Depth/OT{N}/Task{N}/Cam{N}/Depth/

Tactile Data

• Format: JSON files named p_{ot}-{type}-{task}.json
• Content: 44 pressure sensor readings per frame
• Location: Skin_tared/p_ot_{N}/

Gaze Data

• Format: CSV files named {ot}-{type}-{task}_pupil.csv
• Content: 2D gaze coordinates and 3D gaze vectors with timestamps
• Location: Gaze/OT_{N}/

2D Pose Annotations

• Format: JSON files named pred.json
• Content: Keypoint coordinates for each frame
• Location: 2D_Annotations/OT{N}/Task{N}/Cam{N}/

3D Pose Tracking

• Format: JSON files named {ot}-{type}-{task}_pose3d.json
• Content: 3D pose tracking data for both manikin and caregiver
• Location: 3D_Pose/OT{N}/

Action Labels

• Format: CSV files with temporal action annotations
• Content: Hierarchical labels (Task → Subtask → Component)
• Location: Action_Labeling/OT{N}/

Third-Person Videos (GoPro)

• Format: MP4 video files
• Content: Third-person perspective camera footage
• Location: GoPro/OT{N}/

Egocentric Videos (Pupil Labs)

• Format: MP4 video files
• Content: First-person perspective from head-mounted camera
• Location: pupil/OT{N}/

Data Synchronization

Each modality has accompanying timestamp data to enable synchronization across sensors. These timestamps provide frame-to-time mappings for all modalities.

Data Coverage