30% off your first 3 months

Get the deal

Checking an AI's answer before you rely on it

by DuckyHelperNov 12, 20264 min read
AI on your computer

TL;DR:

  • An AI answer is a draft to check, not a verdict. Ask for the steps so there is something to check.

  • Recompute any number yourself and substitute the answer back into the problem.

  • For facts you cannot recompute, confirm with a textbook or another source you trust.

An AI explanation can be clear, friendly and wrong, all at once. The tone gives no clue. That is why the most useful habit with any assistant is a short check before you rely on what it said, especially if the answer is going into homework, a report or a decision.

Here is a chemistry problem to show the habit in action. The checks work on almost any subject.

The problem

You dissolve 5.85 g of sodium chloride in enough water to make 500 mL of solution. What is the concentration in moles per litre? The molar mass of NaCl is about 58.44 g/mol.

Suppose an assistant replies: "The concentration is 0.0200 mol/L." It sounds sure of itself. Is it right?

Check 1: ask for the steps

A bare number cannot be checked. Steps can. Ask: "Show how you got that, one line per step." A good answer looks like this.

  1. Moles of NaCl: 5.85 g ÷ 58.44 g/mol = 0.100 mol (to three figures).

  2. Volume in litres: 500 mL = 0.500 L.

  3. Concentration: 0.100 mol ÷ 0.500 L = 0.200 mol/L.

Already there is a conflict. The first answer said 0.0200, and these steps give 0.200. One of them is wrong, and now you know exactly where to look.

Check 2: recompute the number

Do the arithmetic yourself, on paper or a calculator. 5.85 ÷ 58.44 is close to 0.1001. Dividing 0.100 by 0.500 gives 0.200. The step list is right. The first answer was off by a factor of ten, the kind of slip that appears when 500 mL is turned into 5 L instead of 0.5 L.

This is the lesson of the recompute step. You do not need to understand everything the assistant said. You only need to check the part you can check.

Check 3: substitute back

Test the answer by running the problem in reverse. A solution of 0.200 mol/L and a volume of 0.500 L holds 0.200 × 0.500 = 0.100 mol. That mass is 0.100 × 58.44 = 5.844 g, which rounds to 5.84 g, matching the 5.85 g you started with to within rounding.

If you had substituted 0.0200 instead, you would get 0.0200 × 0.500 = 0.0100 mol, which is only 0.584 g. That does not match, so 0.0200 fails the test.

Check 4: consult a source

Some things you cannot recompute: a molar mass, a date, a definition. For those, look in a textbook, your class notes, or the data page of a reputable reference. Here, the 58.44 could be checked against a periodic table: sodium is about 22.99 and chlorine about 35.45, and 22.99 + 35.45 = 58.44.

Tip: Match the check to the claim. Numbers get recomputed, facts get looked up, and reasoning gets read slowly, one line at a time.

A quick unit check as a bonus

Units catch many slips on their own. Moles divided by litres gives moles per litre, which is what the question asked for. Had the working produced grams per litre or moles times litres, you would know something went wrong before touching a calculator. Write the units beside every number, and let them cancel the way fractions do.

When it is worth the effort

You do not need to run all four checks on every answer. A casual question about what a word means deserves a glance. A number going into a lab report deserves all four. Scale the checking to what depends on the answer.

What this does not catch

These habits find errors you can test. They do not catch an assistant that is wrong in the same way you are. If you misread the problem, and it misread it the same way, the check passes. For high-stakes work, a person such as a teacher or tutor is still the final check.

Privacy and control

When you ask about something on your screen, keep in mind what is visible. DuckyHelper looks only while the pill is open and for that answer. Its blackboard works a problem out with you step by step, which gives you steps to check.

Try it yourself

Take a problem you already know the answer to and ask an assistant to solve it. Run the four checks on its reply. You will quickly find which of them you reach for most, and how often a confident answer deserves a second look.

Frequently asked questions

Should I never trust an AI answer?

Trust it in proportion to how easy it is to check and how much depends on it. Easy to check and low stakes means a glance. Hard to check and high stakes means a source.

What if I cannot do the math myself?

Ask for the steps and read them slowly. A step that does not follow from the one before is where to look.

Is asking the same question twice a check?

It can reveal inconsistency, but two matching answers can both be wrong. Recomputing is stronger.

Get started

DuckyHelper can
do more

It sees your screen and works in any app you use. Try it for 10 minutes, no card.

Get it for Mac, Windows or Linux

Share

Questions and jobs

Do anything

Ask it a question or hand it a job, in any app.

Doing jobs for you is in early access on Mac. Asking works on Mac, Windows and Linux.