Sim-to-real: why simulated robots stumble in reality

TL;DR:

• The reality gap is the difference between a simulator's model and the real robot and world.

• Domain randomization trains across many made-up variations so the policy does not depend on one wrong number.

• System identification measures the real values, such as friction, and puts them back into the simulator.

A small four-legged robot learns to walk in simulation. After thousands of virtual trials it walks perfectly. You carry it to a smooth kitchen floor, power it on, and its feet slide out from under it. The code is the same. The floor is not.

What the reality gap is

A simulator is a set of equations with numbers filled in. The reality gap is every way those equations and numbers differ from the real machine: friction, motor strength, delay between a command and the motor moving, sensor noise, and the exact weight of each part. The gap gets large where the physics is hard to model, such as contact between a foot and the ground.

A policy trained in simulation learns to exploit whatever the simulator says is true. If the simulator says the floor grips well, the policy will push hard sideways with its feet, because in that world it works.

Worked example: a walking policy that slips on a real floor

Friction limits how hard a foot can push sideways. The rule is F_friction ≤ μ × N, where N is the force pressing the foot down and μ is the friction coefficient between foot and floor. Say the robot has a mass of 5 kg and, at one moment, one foot carries all of its weight. With g = 9.8 m/s², N = 5 × 9.8 = 49 N.

Suppose the learned gait needs the foot to push sideways with a force equal to 0.6 of the downward force at that moment. That is 0.6 × 49 = 29.4 N of horizontal force. Now compare two floors.

1. Simulated floor, μ = 0.8: the most friction available is 0.8 × 49 = 39.2 N. Since 29.4 < 39.2, the foot holds.

2. Real floor, μ = 0.4: the most friction available is 0.4 × 49 = 19.6 N. Since 29.4 > 19.6, the foot slips.

3. The ratio the gait needs is 0.6, which is above the real μ of 0.4, so no amount of motor strength can fix it. The gait has to push less sideways.

The two ways to repair this are the two named ideas below. Both attack the same weakness: the policy trusted one number that was wrong.

System identification: measure the real numbers

System identification means measuring the real robot and world and putting the values back into the simulator. For friction there is a classic test: put a foot pad on a board and tilt the board slowly until the pad starts to slide. At that angle θ, μ = tan θ. If the pad slides at 22°, μ = tan 22° ≈ 0.40, which matches the real floor above.

You can identify other values the same way. Weigh each part for its mass. Command a motor to a fixed target and record how long it takes to get there, to estimate its delay and strength. Record a real walk and adjust simulated numbers until the simulated motion matches it.

Domain randomization: train across the unknown

You can never measure everything exactly, and values change with wear, temperature and battery charge. Domain randomization handles this by changing the simulation's numbers on every trial, within a believable range. For example, friction μ between 0.3 and 1.0, body mass a little heavier or lighter, sensor noise on, motor delay of a few milliseconds.

A policy that has to walk on all of those worlds cannot rely on any single one. In our example, a policy trained with μ as low as 0.3 learns that pushing sideways with a ratio above 0.3 sometimes fails, so it learns smaller, safer pushes. It may be slower on grippy floors but it keeps its footing on slippery ones.

The two methods work together. System identification narrows the range to realistic values. Randomization covers what is left.

Where it goes wrong

• Ranges that miss reality. If real friction is 0.2 and you only randomized 0.3 to 1.0, the policy still fails.

• Ranges too wide. If you randomize too much, the policy becomes timid because it must handle absurd worlds.

• Missing effects. Randomizing numbers does not help if the simulator lacks a behaviour altogether, like a cable dragging or a wheel compressing carpet.

• Fast dynamics. Contacts and impacts are the hardest part for any physics engine.

• Trusting one success. A single good run on one floor says little. Test on several surfaces.

First steps on hardware: Hold a legged or wheeled robot in a harness or on blocks for its first runs, keep hands and hair clear of legs and gears, and keep the power switch or emergency stop within reach. A robot that slips can fall onto its battery pack, so check the pack and wiring for damage after any fall and stop using a LiPo that is swollen or dented.

How real robots use it

Teams combine all of the above: measure what they can, randomize the rest, add a safety layer that limits torque and speed, and test on hardware in stages. The same thinking applies to any learned controller, such as those in reinforcement learning for robots, and to any simulated test, as in why roboticists test in a simulator first.

Try it yourself

Measure the friction between a rubber eraser and three surfaces: a wooden desk, a tile and a book cover. Tape a protractor or use a phone's angle reading on a board, tilt slowly until the eraser slides, and compute tan θ for each. You will find the numbers differ a lot, which is the whole reason simulation needs to be told which floor it is.

Frequently asked questions

Why not just train on the real robot?

Real trials are slow, wear out parts and can break things. Simulation does the bulk of the learning, and the real robot confirms and fine-tunes it.

Is a better simulator the only fix?

It helps, but a more detailed simulator is still wrong somewhere. Randomization and real measurements make a policy tolerate that.