← Back to the universe 01 / 13

01 / RESEARCH · CLeAR LAB, UT AUSTIN

Robot learning at CLeAR Lab

Reinforcement learning for humanoid robots. Working on a paper targeting RSS 2027.

Lunar Lander PPO on GitHub ↗
PyTorchIsaacLabPPO · GRPO · REINFORCEMambaFlow matching

Humanoid task training

I designed a task in Isaac Lab that trains a Unitree G1 humanoid to insert a plug into an outlet. I built the simulation environment around the plug and outlet, with alignment and contact as part of the task. I designed the reward function and iterated on the training hyperparameters to get the expert policy to complete the insertion.

Simulated Unitree G1 policy inserting a plug into an outlet.

I implemented OmniReset with a reset-state generator I wrote for this task. The generator gives the policy a range of initial states, so training can include more attempts at the later stages of insertion without waiting for the policy to reach those stages on its own. OmniReset paper and project ↗

This task is part of a sim-to-real transfer project. The video shows the simulation-trained expert, which serves as a starting point for studying how the task can transfer to the physical humanoid.

Reinforcement learning from scratch

I implemented REINFORCE, GRPO, and PPO in PyTorch. I tested my PPO implementation on Gymnasium’s Lunar Lander, where the trained policy reached a 92% success rate.

Two Lunar Lander episodes using my from-scratch PPO implementation.

Experiments

Investigating flow-matching world action models — using flow matching in place of MLPs — and comparing ways to fine-tune pretrained models: DSRL, residual policies, and direct fine-tuning.

I’ve been experimenting with fine-tuning flow-matching policies and exploring flow matching in world action models (WAMs).
Visit CLeAR Lab ↗