30% off your first 3 months

Get the deal

How an assistant reads code on your screen

by DuckyHelperNov 12, 20265 min read
AI on your computer

TL;DR:

  • An assistant reads the code text on your screen, explains it line by line and can point at the line that looks wrong.

  • It works best when you tell it what the code should do, because it cannot read your mind.

  • Always run the code yourself. An explanation that sounds right can still be wrong.

You are staring at five lines of code that print the wrong number, and the editor shows no error. Nothing is crashing, which makes it harder: the program does exactly what you wrote, just not what you meant. An assistant that can see your editor is useful here because it reads the same lines you do and compares them with what you say you wanted.

The short answer

When you ask about code on screen, three things happen. The assistant gets a picture of your display. It turns that picture into text, which is reading code text. Then it reasons about that text the way a tutor would: what each line does, what value each variable holds as the code runs, and where that differs from your goal.

What happens step by step

  1. Seeing. The screenshot includes your code, line numbers, and sometimes the output panel or an error. All of it is evidence.

  2. Reading. Text recognition pulls the characters out. Indentation matters in some languages, so a good tool keeps it.

  3. Explaining lines. It walks through the code in order, saying what each line does in plain words.

  4. Tracing. For a loop, it keeps a small table of variable values per step. This is how people debug on paper too.

  5. Pointing at the bug. It names the line that breaks your expectation and says why.

Walk-through: explaining a loop that adds up 1 to 5

Here is the code in your editor, written to add the whole numbers from 1 to 5.

total = 0
for n in range(1, 5):
    total = total + n
print(total)

You tell the assistant: "This should print 15 but it prints 10." That sentence matters. It gives the intent, so the assistant has something to compare against.

A walk through the lines goes like this. total = 0 starts a running sum. range(1, 5) makes a sequence of numbers that starts at 1 and stops before 5. The loop adds each of those to total. A trace makes it concrete:

Pass

n

total after the pass

1

1

1

2

2

3

3

3

6

4

4

10

The loop ends after n = 4

The assistant points at line 2. The program prints 10 because 1 + 2 + 3 + 4 = 10. The fix is range(1, 6), which includes 5, and then the total is 1 + 2 + 3 + 4 + 5 = 15.

Verify by running it: Change the line, run the code and confirm it prints 15. A second check: the sum of 1 to n is n × (n + 1) ÷ 2, and 5 × 6 ÷ 2 = 15.

Why saying what you wanted matters

Code is only wrong relative to a goal. A loop that stops at 4 is perfectly fine if you wanted to add four numbers. The assistant sees what the code does, but only you know what it should do. "It should print 15" turns a guess into a comparison.

Pointing helps as well. If your tool can mark the exact line, you can check straight away whether it means the same line you were worried about.

Reading error messages as well as code

The output panel is part of the evidence. A red error message names a file, a line number and a kind of error, and an assistant that sees both the code and the message can connect them. If the message says a name is not defined, it looks for the line where that name is used and for the place it should have been created.

Not every bug shows an error, as our loop showed. In those cases the clue is the gap between the output and your expectation, which is why "it prints 10, I expected 15" is a better report than "it does not work". Give the assistant both numbers and it can often name the cause in one reply.

If you are learning, ask it to explain without fixing: "Tell me which line is wrong, but let me change it." You keep the practice, and it still saves you the hunt.

What it cannot do

  • It sees only what is visible. A bug in a file that is not open, or in a function scrolled off the screen, is invisible to it.

  • Text recognition can swap look-alike characters, such as a zero and a capital O, or miss a faint dot or comma.

  • It does not run your code unless a tool says it does. A traced explanation is reasoning, not a test result.

  • It can state a wrong explanation confidently. If the explanation does not match what you see when you run it, trust the run.

Privacy and control

Code on your screen can contain secrets such as keys, passwords or private data in a config file. Before you ask about code, check what else is visible and hide any file that holds secrets. Ask your tool whether it looks only when you call it.

DuckyHelper sees your screen and can draw on it to show a line, and it only listens or looks while the pill is open and for that answer. It's built not to type your passwords or card numbers.

Try it yourself

Take a short loop you wrote, tell the assistant what it should do, and ask it to trace the first three passes. Compare its table with your own on paper. If they differ, the first row where they split shows you where somebody is wrong.

Frequently asked questions

Can an assistant find bugs in a big project?

It can help with the part it can see. For a large project, open the relevant file and tell it what the code should do.

Should I paste code or let it read the screen?

Either works. Reading the screen is quicker for a few lines, and it keeps the output panel and error message in view too.

Get started

DuckyHelper can
do more

It sees your screen and works in any app you use. Try it for 10 minutes, no card.

Get it for Mac, Windows or Linux

Share

Questions and jobs

Do anything

Ask it a question or hand it a job, in any app.

Doing jobs for you is in early access on Mac. Asking works on Mac, Windows and Linux.