TL;DR:
Imitation learning trains a policy from examples of the task done correctly, instead of from a reward.
The simplest version, behavior cloning, is supervised learning: given what the robot sees, predict what the demonstrator did.
It generalizes only near what it was shown, and small errors can pile up when the robot drifts into situations the demonstrations never covered.
Writing code to say exactly how to grasp a mug with a handle, lift it and set it on a coaster is fiddly. Showing the robot is easier: take the arm by the wrist, move it through the task a few times, and let the software work out the pattern.
What imitation learning does on a robot
In imitation learning a person, or sometimes another program, demonstrates the task and the robot records it. The record is a sequence of pairs: what the robot sensed at a moment (joint angles, camera image, object position) and what the demonstrator did next (the next joint position or command).
Behavior cloning, the simplest form, treats that record as a table of examples. A model is trained so that, given a sensed state, it outputs the action the demonstrator took. After training, the model is the robot's policy.
Demonstrations as data
Ways to give demonstrations include guiding the arm by hand (kinesthetic teaching), moving a leader arm while the follower copies it, or using a handheld controller. Each has the same goal: to produce clean pairs of state and action.
Quality matters more than most people expect. Ten consistent demonstrations usually teach more than thirty messy ones where the person hesitates, backs up or does it differently each time.
Worked example: teaching a pick-and-place by guiding the arm
The task: pick a block from a marked spot and put it in a bin. The arm has the motors set to a gentle, low-force mode so you can move it by hand, and a recorder saves the joint angles 10 times per second.
Place the block at the marked spot. Press record.
Guide the arm down to the block, close the gripper, lift, swing over the bin, open the gripper, and return home. Say that takes 6 seconds.
Samples per demonstration:
6 s × 10 samples/s = 60.Repeat 20 times with the block shifted by a couple of centimeters each time. Total:
20 × 60 = 1200samples.Train a model to predict the next arm target from the current joint angles, the gripper state and the block's position in the image.
Run it with the arm at low speed and the block in a new, nearby position.
If it works, the arm reaches near the block's new position, because the model learned how the reach changes with block position inside the range you showed it. If the block is in a spot far outside that range, the model has no examples there and its guess can be poor. Generalizing means working in new situations that are similar to the training ones, not in all situations.
Now the drift problem. Suppose, as a hypothetical, the model's prediction is 0.5 cm off in the same direction each step. After 60 steps the arm could be 30 cm from the demonstrated path, because 60 × 0.5 = 30. In reality the error is not always in one direction, but the idea is real: the arm lands in a state no demonstration visited, and the model has never been shown what to do there. This is called compounding error, and it is why people add corrections, such as having the demonstrator fix the robot's mistakes while it runs.
Limits to keep in mind
It copies the demonstrator. If the demonstrations are slow or clumsy, so is the robot. It will not discover a better way by itself.
Narrow coverage. Different lighting, a different bin position or a different block shape can break it.
Several valid ways. If some demonstrations go left of an obstacle and others right, averaging them can send the arm straight into it.
No understanding of why. The model has no concept of the block being fragile or the bin being full.
Data effort. Collecting good demonstrations takes time, and people get tired and inconsistent.
Teaching an arm by hand: Use a hand-guiding mode only if the arm's manual describes one, and keep its force and speed low. Keep fingers out of joints and between links, which are pinch points. Have the emergency stop within reach for both teaching and playback, and never put your hands in the workspace while the learned policy is running.
How real robots use it
Imitation learning is used for manipulation tasks that are hard to program but easy to show, such as folding or placing parts. It is often combined with other methods: demonstrations give a starting policy, and then practice, as in reinforcement learning for robots, can improve it. Policies are also usually trained or checked in simulation first, and the gap between simulation and the real machine is a recurring topic in sim to real.
Try it yourself
You can try a miniature version without a robot. On graph paper, draw a start, a goal and an obstacle. Draw five paths a person would take from start to goal and mark the direction at ten points on each. Then, from only those marks, try to guess the direction at a new point near the paths, and then at one far away. The first guess is easy and the second is hard, and that is generalization in one picture.
Frequently asked questions
How many demonstrations do I need?
It depends on the task and the model. Simple, repeatable tasks need few. Wide variation in objects and positions needs many more, and there is no single number that fits every case.
Is this the same as programming by recording a path?
Not quite. Replaying a recorded path repeats the same motion every time. Learning from several demonstrations lets the robot adjust the motion to what it senses.
Get started
