RCareWorld

A Human-centric Simulation World for Caregiving Robots



RCareWorld is a simulation world for physical and social robot caregiving designed using inputs from all the stakeholders — expert occupational therapists, care-recipients, caregivers and roboticists.

RCareWorld brings together realistic human modeling (CareAvatars), home environments with multiple levels of accessibility and assistive devices (CareHomes), and robot caregivers within a Unity-based simulation platform to to provide a comprehensive simulation of each component in a real-life caregiving scenario.

CareAvatars are human-like avatars of care-recipients, built using measures of functional abilities taken from community-dwelling adults with mobility limitations.

CareHomes are virtual homes which accurately model caregiving homes with modifications for varying levels of accessibility and containing various assistive devices such as wheelchair, Hoyer-lift, etc. that are commonly used in caregiving scenarios.

RCareWorld provides a set of realistic robotic caregiving tasks which can provide meaningful assistance to the modeled care-recipients with activities of daily living and presents baseline control policies for each of these tasks.

It supports photo-realistic visual rendering, interfaces with various physics engines to accurately simulate various object types such as rigid, articulated, deformable, fluid and tearable objects, and supports various sensing modalities. Furthermore, it supports ROS and interfaces with OMPL, MoveIt and RFMove for planning and Stable Baselines3 for reinforcement learning.


Interpolate start reference image.

Robots

RCareWorld supports robots that are commonly used in physical and social caregiving, including Pepper, NAO, Stretch RE1, Kinova Gen3, Kinova Gen2, PR2, UR5, and Franka. For haptics sensing, users can get force and torque on the joints of these robots. For visual sensing, users can mount cameras on the wrist or head of these robots.

Interpolate start reference image.

Interpolate start reference image.

CareAvatars

We use SMPL-X models to represent a human avatar’s shape and expression, and integrate it with an actuatable musculoskeletal backbone to model kinematics and dynamics. We use a behavior tree to model social behavior and individual preferences.


Interpolate start reference image.

Musculoskeletal SMPL-X consists of a human skeleton, muscles, skin and texture

Interpolate start reference image.

Musculoskeletal SMPL-X adapts to different body shape automatically

Interpolate start reference image.

Actuated Jaw

Interpolate start reference image.

Actuated Tongue

Interpolate start reference image.

Actuated Eyes

Interpolate start reference image.

Facial Expressions

Interpolate start reference image.

Musculoskeletal SMPL-X actuated by left Popliteus muscle

Interpolate start reference image.

Musculoskeletal SMPL-X actualed by right Platysma muscle

Interpolate start reference image.

Musculoskeletal SMPL-X actualed by left Gastrocnemius Medial muscle

Interpolate start reference image.

Musculoskeletal SMPL-X deforms due to contact

Using clinical data collected from real users with varying levels of spinal cord injury - both partial and complete, cerebral palsy, and hemiparetic stroke, we construct CareAvatars for six care-recipients.

Interpolate start reference image.
Interpolate start reference image.

Morgan

Interpolate start reference image.

Jose

Interpolate start reference image.

Natalia

Interpolate start reference image.

Daniel

Interpolate start reference image.

Kim

Interpolate start reference image.

Karan


Interpolate start reference image.

CareHomes

CareHomes are virtual homes which accurately model caregiving homes with modifications for varying levels of accessibility and contain various assistive devices.

By consulting stakeholders, including occupational therapists, caregivers, and care-recipients, and following the guidance of Universal Design Manual, we present we present three levels of modifications:

  • Level 1 - Usual: No special modifications
  • Level 2 - Partially Barrier-Free: We modify some objects that are frequently used
  • Level 3 - Completely Barrier-Free

Interpolate start reference image.

CareHome Level-1

Interpolate start reference image.

CareHome Level-2

Interpolate start reference image.

CareHome Level-3


CareHomes contains assistive devices commonly used in caregiving scenarios such as wheelchair, Hoyer-lift, etc. (shown below).

Interpolate start reference image.
Interpolate start reference image.

Shower Chair

Interpolate start reference image.

Hospital Bed

Interpolate start reference image.

Hoyer's Lift


Interpolate start reference image.

Benchmark Robot Caregiving Tasks

We provide a set of six realistic robotic caregiving task scenes which can provide meaningful assistance to the modeled care-recipients with activities of daily living. Researchers can use these scenes to perform experiments for robotic caregiving, such as benchmarking reinforcement learning algorithms or testing planning and control algorithms.

Feeding

Task: Care-recipient U6 is fed with a Kinova Gen 3 mounted on a wheelchair. The robot is required to move a soft and deformable food-item on the spoon near the care-recipient’s mouth and insert it into the mouth.

Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)

Interpolate start reference image.

Demonstration

Bathing

Task: Care-recipient U2 is lying on a hospital bed. A Kinova Gen 2 robot mounted on the bed holds a soft sponge in its gripper. The robot wipes the care- recipient’s arm with the sponge.

Baseline: Proximal Policy Optimization (PPO)

Interpolate start reference image.

Demonstration

Dressing

Task: We consider the scene where a human care- giver and a robot caregiver collaborate with each other to help care-recipient A4 put on shorts. The care- recipient is sitting on a wheelchair. The human caregiver lifts the care-recipient’s leg. The robot caregiver, Kinova Gen 3, mounted on the wheelchair pulls up the shorts from ankle to hip.

Baseline: Proximal Policy Optimization (PPO)

Interpolate start reference image.

Demonstration

Limb Repositioning

Task: When care-recipient U3 is transferred from their bed to a wheelchair using a hoyer sling, the limbs are usually not in the correct position (e.g. resting on the wheelchair handle) after the care-recipient is transferred from the hoyer sling. In this task, our human avatar sits in a wheelchair, and their limb is hanging down. The robot, Kinova Gen 3, mounted on the wheelchair holds a C-shaped tool to lift their upper limb, and put their arm on the wheelchair handle.

Baseline: Hindsight Experience Replay (HER) with Truncated Quantile Critics (TQC)

Interpolate start reference image.

Demonstration

Ambulating

In this task, we want to demonstrate that our CareHomes assets can affect the skills that a robot learns in an environment. The Kinova Gen 3 arm mounted on the wheelchair adapts to different door mechanisms and opens them. We show a door knob for a level 1 home, a lever for a level 2 home, and a sliding door for a level 3 home.

Task i: Door with Knob in or Level 1 CareHome

Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)

Interpolate start reference image.

Demonstration

Task ii: Door with Lever Handle in Level 2 CareHome

Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)

Interpolate start reference image.

Demonstration

Task iii: Sliding Door in Level 3 CareHome

Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)

Interpolate start reference image.

Demonstration

Toileting

Task: We consider a Stretch RE1 robot helping open a toilet cover in a bathroom. .

Baseline: Hindsight Experience Replay (HER) with Soft Actor-Critic (SAC)

Interpolate start reference image.

Demonstration


Interpolate start reference image.

Comparison with other Simulation Environments

Interpolate start reference image.

Interpolate start reference image.


Rendering

Visual Rendering

RCareWorld uses Unity as rendering front-end, which enables the simulator to povide near-photorealistic rendering effect.

Interpolate start reference image.

RGB

Interpolate start reference image.

Depth

Interpolate start reference image.

Surface Normals

Interpolate start reference image.

Optical Flow

Interpolate start reference image.

Domain Randomization

Interpolate start reference image.

Instance Segmentation Masks

Physics Rendering

RCareWorld interfaces with various physics engines to accurately simulate various object types which are common in household environments such as rigid, articulated, cloth, deformable, fluid and tearable objects.

Interpolate start reference image.

Rigid

Interpolate start reference image.

Articulated

Interpolate start reference image.

Deformable

Interpolate start reference image.

Cloth

Interpolate start reference image.

Fluid

Interpolate start reference image.

Tearable


Interpolate start reference image.

BibTeX

@article{ye----RCareWorld,
  author    = {Ruolin Ye, Wenqiang Xu, Haoyuan Fu, Rajat Kumar Jenamani, Vy Nguyen, Cewu Lu, Katherine Dimitropoulou and Tapomayukh Bhattacharjee},
  title     = {RCareWorld: A Human-centric Simulation World for Caregiving Robots},
  journal   = {---},
  year      = {---},
}