TL;DR:
A robot sees the new object as a depth image or point cloud, proposes many possible grasps, scores them and tries the best one.
A stable two-finger grasp needs the line between the contacts to stay inside both friction cones, whose half-angle is
atan(μ).Feedback after contact, such as finger spacing and motor current, tells the robot whether the grasp worked or needs another try.
A factory robot that picks the same bolt from the same fixture all day can be programmed with one fixed grasp. A robot asked to pick up whatever is on a kitchen table has no such luxury. It has never seen this mug, this toy or this sponge. It still has to decide where to put its fingers.
Grasp candidates
The robot starts with a depth camera and builds a 3D picture of the table, much as in point clouds: filtering and segmenting what a robot sees. It removes the table plane, separates the object, and then generates grasp candidates: possible positions and orientations of the gripper, each described by where the fingers would touch and how the hand would approach.
Candidates can come from geometry, such as pairs of surface points whose normals face each other, or from a learned model that looks at the depth image and proposes promising grasps. Either way, the output is a list that may hold hundreds of options, most of them poor.
Scoring
Each candidate is scored on questions like these. Will friction hold it? Is the object narrower than the gripper opening? Will the hand or arm collide with the table or other objects? Is the approach reachable for the arm? Is the grasp near the object's center of mass, so it does not twist?
Worked example: picking a new object from a table
A parallel gripper opens up to 8 cm. The object is a rounded block that is 5 cm wide at the grasp location. The fingertip friction coefficient is estimated at μ = 0.5.
Friction cone. A contact can transmit force within a cone around the surface normal, with half-angle
atan(μ) = atan(0.5) ≈ 26.6°. A contact outside the cone slips.Antipodal test. For a two-finger grasp, the line joining the two contact points must lie inside both friction cones. That means the angle between the line and each surface normal must be below 26.6°.
Width test. 5 cm is less than 8 cm, so the gripper fits.
Candidate A: angles of 10° and 15° from the normals. Both are below 26.6°. Score by margin:
1 − 15 / 26.6 ≈ 0.44.Candidate B: angles of 35° and 5°. 35° is above 26.6°, so it fails the test and is rejected.
Candidate C: angles of 5° and 8°. Score:
1 − 8 / 26.6 ≈ 0.70. This is the best of the three, so the robot picks C, after a collision check says the fingers will not hit the table.
Check the geometry of the cone with an example: a 100 g block weighs 0.1 × 9.8 = 0.98 N. For a grasp with the fingers on opposite vertical sides, friction at the two contacts must carry the weight: 2 × μ × N ≥ 0.98, so N ≥ 0.98 / (2 × 0.5) = 0.98 N per finger. A safety factor of two suggests squeezing near 2 N. If the contact normal is tilted away from the squeeze line, as in candidate A, the usable friction shrinks, which is what the lower score reflects.
The numbers here are a teaching version. Real systems use more complicated quality measures, and the estimate of μ is rough because the robot does not know the object's material before touching it.
Feedback after contact
The plan is made from a single view, so the robot checks as it goes. It moves to a point above the object and approaches slowly. When it closes the gripper, it watches the finger spacing, the motor current and any touch sensors.
If the gripper closes fully to its minimum width, it grabbed nothing. Retry.
If it stops at about 5 cm with the current rising to a threshold, it has the block.
After lifting a few centimeters, if the fingers slip or the current drops, put it down and try the next candidate.
If the object moved while the hand approached, take a new picture and re-plan.
Touch sensing, as described in how robotic hands sense touch, makes this check much richer. Without it, the robot relies on indirect signals such as motor current.
Where it goes wrong
Bad depth data. Shiny, transparent or black objects give holes or false readings in the depth image.
Hidden sides. The camera sees only the front, so the shape behind is a guess.
Unknown weight and softness. A heavy object slips, and a soft one squashes.
Clutter. Neighbouring objects block fingers.
Calibration error. If the camera-to-robot transform is off by a centimeter, the grasp lands wrong. See coordinate frames and transforms.
Over-trust. A learned model can confidently propose a bad grasp on something unlike its training data.
Safety: Gripper fingers can pinch hard. Keep hands out of the workspace while the arm is powered, limit gripper force and speed in the controller, test with soft objects first and keep the emergency stop within reach.
How real robots use it
Warehouse picking, kitchen and lab robots use this propose, score, try and check pattern. Some systems combine it with a camera mounted on the hand that keeps correcting the approach, as in visual servoing.
Try it yourself
Draw the outline of a mug from above on paper. Mark 12 points around the edge and draw the surface normal at each. Find pairs where the line between them is close to both normals, and measure the angles with a protractor. Test which pairs would pass for μ = 0.3 and for μ = 0.8. The slippery case has far fewer good grasps.
Frequently asked questions
Do robots need to recognize an object to grasp it?
Not necessarily. Many methods work from shape alone, treating the object as unknown geometry.
What is the difference between a two-finger and a multi-finger grasp?
Two fingers are simpler to plan and cover many objects. Multi-finger hands can hold more shapes and re-orient objects, but they have many more joints to plan.
Get started
