Adaptation without knowing the physics

Train with the true parameters, deploy with half a second of history.

Train a privileged policy with PPO across thousands of randomized arms, then distill what it knew into an adaptation module that reads only the 50-step history. Compare it with Variant B and the Module 3 EKF, and reuse the same trainer to throw a ball into a cup.

Written lesson + build labEvidence-based gate

The goal for this build

What you’ll be able to do.

Train a privileged teacher with PPO in vmap-parallel envs, with randomization resampled within episodes.
Distill the teacher’s latent into an adaptation module trained on the policy’s own rollouts.
Compare RMA and Variant B with the Module 3 EKF inside 10k parameters and 2 hours of laptop CPU.
Reuse the same trainer with a new reward to throw a ping-pong ball into a cup.

What to have ready

Module 3 EKF benchmark rows, the baseline to beat
Module 4 encoder, split manifest and probe
A Module 1 step() that runs under vmap with randomized Params
Gradients and expectations; no reinforcement learning background is assumed

Download the course starter ↓

Free account

Read the lesson, run the lab, pass the gate.

The whole course is free. Create an account to unlock the lesson, the build lab, the task and the gate command for this module, and to download the starter kit.

Create a free account Sign in

03 / Prove it

Keep the evidence.
Then run the gate.

What you ship

The teacher policy, privileged encoder and adaptation module, with configs and checkpoints
A Variant B policy on the frozen Module 4 encoder
A bench.py rerun against Modules 2 to 4, with commit, config and seed on every row
A training record: deployed parameter count, CPU model and wall-clock time including compilation
The throw policy with its sim hit rate, and the ball-in-a-cup success rate if attempted

Module 05 gateEvidence required

The policy with its adaptation module has no overload trip or safety abort in any held-out cell.
In every held-out cell, its tracking error stays within the non-inferiority margin of the Module 3 EKF: at most 1.10× the EKF’s plus 1 mm.
Summed across the adaptation conditions, every sim perturbation except sensor noise and latency, its tracking error is no higher than the EKF’s.
Recovery within 1 s after the +100 g payload step, measured to the last entry into the 3 mm band around the target after which the error stays inside for at least 0.5 s.
Variant B and RMA are reported side by side.

With a free account: the gate command and how to record your evidence.

The story behind the build

Episode companions.

EP 06RMA in my own JAX sim, and a throw

Written companions are available now. Recordings and customer access are forthcoming.

Made it through the lesson?

Save your reading progress here. Record your gate result in your project.