Train for your built robot

The robot supplies measurements; simulation supplies practice

Your completed BDX-R is now a source of evidence. Weigh its actual assemblies, record zero offsets and sensor mounting, and compare timestamped commands with measured joint motion. System identification means fitting a model to that response. A lagging joint might involve communication delay, friction, inertia or a gain mismatch; adjusting every parameter until a graph looks nice cannot identify the cause.

Reinforcement learning then practices in simulation, where failed episodes can reset. The physical robot runs a selected, fixed inference policy during evaluation. This course does not send random training actions to the real robot or let an unreviewed update silently replace its controller. Controlled physical measurements improve the next simulated experiment.

A checkpoint is more than an exported policy

An ONNX file is useful for choosing actions from observations. Continuing a training run usually needs additional state: the training model, optimizer, normalization, iteration and compatible configuration. Kayden’s public Deploy policy.pt is a TorchScript export, confirmed by its archive structure and torch.jit.load call. Its filename alone does not make it a resumable training checkpoint.

The inspected MjLab tree supplies training code, local/W&B checkpoint loading and ONNX export, but no checked-in training checkpoint was found; the v0.1.0 release has no attached assets. Either obtain a compatible full checkpoint with provenance or train a new baseline. Accessible source code does not establish access to a creator’s private experiment artifacts.

An optimizer is the part of training that adjusts the network’s learned numbers. It can keep its own running state about those updates. An inference export contains what is needed to calculate actions, while a full compatible training checkpoint preserves the additional state needed to resume training. The filename extension cannot tell you which contents are present.

Use a separate, pinned training environment

The BDX-R lockfile pins mjlab to a particular Git revision, while current mjlab documentation lists Linux with an NVIDIA GPU for training. Your Mac remains useful for analysis and the course’s CPU inference exercises. Do not replace the working simulation environment with the training dependencies. Record the operating system, GPU, driver, source commit and lockfile together.

The current code includes a standing task and velocity tasks. Start with one clear behavior before imitation or rough terrain. A smoke test checks that resets, observations, learning, saving and reloading work; it does not produce a good walker. Measure its memory use and iteration speed before choosing a longer run or estimating compute cost.

Choose a policy using tests it did not optimize

Reward is the score used during learning. Release criteria should separately measure falls, tracking error, joint saturation, timing and unwanted motion. Evaluate several seeds and held-out combinations of measured mass, delay and floor conditions. Randomization ranges should express uncertainty you can defend, rather than extremes selected to make the policy seem robust.

Compare the exported network with the training actor on recorded input vectors, then use the deployment simulator with identical ordering, units, normalization, gains and timing. The new physical policy starts again at the appropriate supported tests. Keep the previous successful revision available; training success is not permission to skip commissioning.