Mehmet Turan Yardimci

A flow matching policy produces an action chunk by integrating a learned velocity field from noise. Integrating it deterministically gives no action likelihood, so a policy gradient has nothing to act on. The repository takes the standard route around that, following the approach of pi_RL: one step of the integration is treated as stochastic, so the sampled chunk has a tractable log probability. The pretrained action head is left bit identical and driven through a callback. What is built around that is the part that makes such a run trustworthy rather than merely runnable:

• Stochastic sampler for flow matching policies, leaving the pretrained action head untouched.
• Policy gradient update that grades the action the environment actually executed, over the executed part of the chunk only.
• Critic head with adaptive target rescaling, kept outside the action head.
• Parameter manifest and learning rate proof printed before the first update, so what is training and at what rate is a logged fact rather than an assumption.
• Process isolated simulator harness with a sparse terminal reward and no shaping.
• Deterministic evaluation with a completeness gate that refuses to produce a number from an unfinished evaluation.
• Verification protocol whose mechanical subset runs from a single gate entry point.

Multi stage curriculum training pipeline:

• Triple actor critic architecture: separate policies for locomotion (legs and waist), arm reaching (shoulders, elbows, wrists), and hand control (DEX3 three finger hands)
• Dual actor critic loco manipulation: decoupled locomotion and arm policies trained sequentially with curriculum learning, for coordinated walking and reaching
• Anti gaming mechanisms: absolute target sampling, three condition reach validation, movement centric rewards, so the curriculum cannot be exploited by a policy that stands still
• 29 DoF whole body model: 12 leg, 3 waist, 8 arm and 6 wrist joints, with 14 DEX3 finger joints for grasping
• VLM planning layer: Qwen3-VL running locally through Ollama turns a natural language instruction into a skill sequence, with replanning from updated world state
• Goal task: squat, pick an object off the ground, stand, walk to a table, place it in a box