TL;DR:
Stereo cameras find depth from disparity: the shift of an object between left and right images. Z = fB/d.
Structured light and time-of-flight cameras measure depth with their own infrared light.
Depth error grows with distance, and bright sunlight can blind infrared types.
Hold a finger in front of your face and close one eye, then the other. The finger jumps sideways against the background. Now do the same with a far-off tree: it hardly moves. Your brain uses that difference to judge distance, and a stereo camera does the same with arithmetic.
Three ways to get depth
Stereo. Two cameras a known distance apart. Depth comes from how much objects shift between the two images.
Structured light. The camera projects a known pattern of infrared dots onto the scene and watches how the pattern bends.
Time of flight. The sensor sends pulses or modulated light and measures how long it takes to return, like a LiDAR for the whole image at once.
Each device on the market uses one of these ideas, or a mixture. The rest of this article works through stereo in detail, because the maths is short and shows the main trade-offs.
Worked example: depth from stereo disparity, Z = fB/d
Two cameras are mounted side by side. The distance between them is the baseline B. Each has a focal length f measured in pixels. A point in the scene appears at slightly different horizontal positions in the left and right images. The difference is the disparity d, in pixels. Then the depth is:
Z = f × B ÷ d
Take f = 700 px, B = 0.06 m (6 cm), and a measured disparity of d = 21 px.
Multiply:
f × B = 700 × 0.06 = 42.Divide:
Z = 42 ÷ 21 = 2.0 m.Check the units: pixels times metres divided by pixels leaves metres.
Close things have large disparity, far things small. A point at 1 m has d = 42 ÷ 1 = 42 px. A point at 10 m has d = 42 ÷ 10 = 4.2 px. Depth is the reciprocal of disparity, which has an important effect on accuracy.
Why depth gets less precise with distance
Matching points between the images is accurate to about a pixel or so, depending on the algorithm. Suppose the disparity at 2 m could be off by one pixel:
If
d = 20,Z = 42 ÷ 20 = 2.1 m, which is 0.1 m too far.If
d = 22,Z = 42 ÷ 22 = 1.91 m, which is 0.09 m too near.
So at 2 m, one pixel of error gives about 0.1 m of depth error. Now repeat at 10 m, where d = 4.2. If the disparity is one pixel smaller, d = 3.2, and Z = 42 ÷ 3.2 = 13.1 m, an error of more than 3 m. The same one-pixel mistake is much worse far away. To improve distance accuracy you can widen the baseline or use a longer focal length, with trade-offs: a wide baseline struggles close up, because the two views stop overlapping.
What stereo needs
Stereo works by finding the same point in both images, so it needs texture. A plain white wall has nothing to match, and the depth there is guesswork. Some stereo cameras add an infrared projector that paints a pattern on the wall to give them something to match.
Structured light and time of flight
Structured-light cameras compare the projected pattern with a stored reference, and the shift of each dot gives depth by the same geometric idea. Time-of-flight cameras measure the travel time of light for each pixel. In both cases, the sensor supplies its own infrared light.
That helps indoors, in the dark or on blank walls. It causes trouble in bright sunshine, because sunlight also contains infrared and can drown out the sensor's own signal. Many such cameras work best indoors, so check the data sheet for range and sunlight limits before you buy one for an outdoor robot.
Other things that confuse depth cameras
Shiny or transparent surfaces such as glass and mirrors.
Very dark surfaces that absorb infrared.
The edges of objects, where the depth may be blurred between near and far.
Several cameras of the same type pointing at one another.
Common mistakes
Mixing units: use pixels for
fanddand metres forB.Forgetting to calibrate and rectify the images first (see the article on camera calibration).
Trusting depth at the edge of the range as much as depth nearby.
Your plan for this week
Recompute the example with B = 0.12 m. Depth at d = 21 px becomes 700 × 0.12 ÷ 21 = 4.0 m, and the one-pixel error at that distance is larger. Notice what changed. If you want to experiment with the numbers on a diagram, DuckyHelper can draw the two-camera geometry on a blackboard with you.
Frequently asked questions
Which type is best for a small robot?
It depends on the setting. Indoors, structured light or time of flight is often convenient. Outdoors, stereo is usually more robust to sunlight. Check each sensor's range and data sheet.
Is a depth camera the same as a LiDAR?
They both give distance, but a depth camera captures a whole image at once, typically over a shorter range, while a LiDAR scans with a beam.
Get started
