Background
Legged and wheeled-legged robots that operate in different environments often require physical, privileged information to help determine their dynamic movements (friction, payload, motor strength, etc). However, these values are constantly changing during operation, especially for a robot in changing environments, requiring them to adapt in real time.
This research project builds off of Rapid Motor Adaptation (RMA), a two-phase reinforcement learning framework, applied to a Transformable Wheel Mobile Robot (TWMR) capable of radially extending its legs to switch between wheeled and legged locomotion.
Phase 1: A policy is trained in simulation with privileged information on environmental parameters (COM, friction, terrain)
Phase 2: An adaptation module is trained to infer those same parameters in real time through the use of the robot's proprioceptive ability.
The deployed policy is aimed to adjust almost immediately to any change in terrain.
I joined this project as I wanted to explore more RL and robotic simulations. Currently, I've developed the ROS2 hardware control stack including encoder sensing, actuator calibration, and motor control while studying the RMA and TWMR and have begun implementing a proper training curriculum.
ROS 2 Architecture
The current ROS 2 architecture uses a simple node-topic interface, with an intermediate Initialization node to sequentially initialize each component on the robot. This allows us to prevent any values from being published before any component is ready to run, giving time to check that each sensor has been initialized and/or calibrated.
Policy Training
Currently, I am getting a reward function tuned and a proper base policy trained. I have synced the XML file with the current robot's physical properties (COM, component moment of inertia, size) in order to get a more accurate simulation. I am including domain randomization through varying environmental parameters to improve robustness to sensor noise, and applying a training curriculum for a better policy.
