RCareWorld is a simulation world for physical and social robot caregiving designed using inputs from all the stakeholders — expert occupational therapists, care-recipients, caregivers and roboticists.
RCareWorld brings together realistic human modeling (CareAvatars), home environments with multiple levels of accessibility and assistive devices (CareHomes), and robot caregivers within a Unity-based simulation platform to to provide a comprehensive simulation of each component in a real-life caregiving scenario.
CareAvatars are human-like avatars of care-recipients, built using measures of functional abilities taken from community-dwelling adults with mobility limitations.
CareHomes are virtual homes which accurately model caregiving homes with modifications for varying levels of accessibility and containing various assistive devices such as wheelchair, Hoyer-lift, etc. that are commonly used in caregiving scenarios.
RCareWorld provides a set of realistic robotic caregiving tasks which can provide meaningful assistance to the modeled care-recipients with activities of daily living and presents baseline control policies for each of these tasks.
It supports photo-realistic visual rendering, interfaces with various physics engines to accurately simulate various object types such as rigid, articulated, deformable, fluid and tearable objects, and supports various sensing modalities. Furthermore, it supports ROS and interfaces with OMPL, MoveIt and RFMove for planning and Stable Baselines3 for reinforcement learning.
RCareWorld supports robots that are commonly used in physical and social caregiving, including Pepper, NAO, Stretch RE1, Kinova Gen3, Kinova Gen2, PR2, UR5, and Franka. For haptics sensing, users can get force and torque on the joints of these robots. For visual sensing, users can mount cameras on the wrist or head of these robots.
We use SMPL-X models to represent a human avatar’s shape and expression, and integrate it with an actuatable musculoskeletal backbone to model kinematics and dynamics. We use a behavior tree to model social behavior and individual preferences.
Musculoskeletal SMPL-X consists of a human skeleton, muscles, skin and texture
Musculoskeletal SMPL-X adapts to different body shape automatically
Actuated Jaw
Actuated Tongue
Actuated Eyes
Facial Expressions
Musculoskeletal SMPL-X actuated by left Popliteus muscle
Musculoskeletal SMPL-X actualed by right Platysma muscle
Musculoskeletal SMPL-X actualed by left Gastrocnemius Medial muscle
Musculoskeletal SMPL-X deforms due to contact
Using clinical data collected from real users with varying levels of spinal cord injury - both partial and complete, cerebral palsy, and hemiparetic stroke, we construct CareAvatars for six care-recipients.
Morgan
Jose
Natalia
Daniel
Kim
Karan
CareHomes are virtual homes which accurately model caregiving homes with modifications for varying levels of accessibility and contain various assistive devices.
By consulting stakeholders, including occupational therapists, caregivers, and care-recipients, and following the guidance of Universal Design Manual, we present we present three levels of modifications:
CareHome Level-1
CareHome Level-2
CareHome Level-3
CareHomes contains assistive devices commonly used in caregiving scenarios such as wheelchair, Hoyer-lift, etc. (shown below).
Shower Chair
Hospital Bed
Hoyer's Lift
We provide a set of six realistic robotic caregiving task scenes which can provide meaningful assistance to the modeled care-recipients with activities of daily living. Researchers can use these scenes to perform experiments for robotic caregiving, such as benchmarking reinforcement learning algorithms or testing planning and control algorithms.
Task: Care-recipient U6 is fed with a Kinova Gen 3 mounted on a wheelchair. The robot is required to move a soft and deformable food-item on the spoon near the care-recipient’s mouth and insert it into the mouth.
Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)
Demonstration
Task: Care-recipient U2 is lying on a hospital bed. A Kinova Gen 2 robot mounted on the bed holds a soft sponge in its gripper. The robot wipes the care- recipient’s arm with the sponge.
Baseline: Proximal Policy Optimization (PPO)
Demonstration
Task: We consider the scene where a human care- giver and a robot caregiver collaborate with each other to help care-recipient A4 put on shorts. The care- recipient is sitting on a wheelchair. The human caregiver lifts the care-recipient’s leg. The robot caregiver, Kinova Gen 3, mounted on the wheelchair pulls up the shorts from ankle to hip.
Baseline: Proximal Policy Optimization (PPO)
Demonstration
Task: When care-recipient U3 is transferred from their bed to a wheelchair using a hoyer sling, the limbs are usually not in the correct position (e.g. resting on the wheelchair handle) after the care-recipient is transferred from the hoyer sling. In this task, our human avatar sits in a wheelchair, and their limb is hanging down. The robot, Kinova Gen 3, mounted on the wheelchair holds a C-shaped tool to lift their upper limb, and put their arm on the wheelchair handle.
Baseline: Hindsight Experience Replay (HER) with Truncated Quantile Critics (TQC)
Demonstration
In this task, we want to demonstrate that our CareHomes assets can affect the skills that a robot learns in an environment. The Kinova Gen 3 arm mounted on the wheelchair adapts to different door mechanisms and opens them. We show a door knob for a level 1 home, a lever for a level 2 home, and a sliding door for a level 3 home.
Task i: Door with Knob in or Level 1 CareHome
Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)
Demonstration
Task ii: Door with Lever Handle in Level 2 CareHome
Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)
Demonstration
Task iii: Sliding Door in Level 3 CareHome
Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)
Demonstration
Task: We consider a Stretch RE1 robot helping open a toilet cover in a bathroom. .
Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)
Demonstration
RCareWorld uses Unity as rendering front-end, which enables the simulator to povide near-photorealistic rendering effect.
RGB
Depth
Surface Normals
Optical Flow
Domain Randomization
Instance Segmentation Masks
RCareWorld interfaces with various physics engines to accurately simulate various object types which are common in household environments such as rigid, articulated, cloth, deformable, fluid and tearable objects.
Rigid
Articulated
Deformable
Cloth
Fluid
Tearable
@article{ye----RCareWorld,
author = {Ruolin Ye, Wenqiang Xu, Haoyuan Fu, Rajat Kumar Jenamani, Vy Nguyen, Cewu Lu, Katherine Dimitropoulou and Tapomayukh Bhattacharjee},
title = {RCareWorld: A Human-centric Simulation World for Caregiving Robots},
journal = {---},
year = {---},
}