Download
White paper, not peer reviewed

Showing, Not Telling

Drawing on the learner's screen as tutoring feedback: a design note. Human tutors point, circle and sketch on the learner's own work. This note explains how DuckyHelper's Mac app gives its tutor a pen and a pointer on the real screen, how it decides where a mark goes, what Eddy does instead in the app's Eddy tab, and the research behind each choice.

White paper · The DuckyHelper Team · October 9, 2026

Abstract

Human tutors point. They circle the wrong step, underline the word that matters and sketch the idea in the margin. Most AI tutors cannot: they answer in a chat window, away from the learner's work. DuckyHelper's Mac app gives its voice tutor a pointer and a pen that work on the learner's real screen, over any app. This design note describes what the screen pen does, how it decides where a mark goes, and how each design choice follows research on multimedia learning, gesture and feedback. We also describe Eddy, the tutor in the app's Eddy tab, which cannot see the learner's screen and draws on its own blackboard instead, and the privacy limits on both. DuckyHelper has not been evaluated in a controlled study. This paper reports no learning outcomes, no usage numbers and no benchmark scores of our own. Every number in it belongs to the study that reported it.

First page of Showing, Not Telling

White paper · DH-WP-2026-02

Showing, Not Telling: Drawing on the learner's screen as tutoring feedback: a design note

The DuckyHelper Team, MingLLM, Inc. October 9, 2026. 6 pages.

Download the PDFOpen in a new tab

Cite this paper

The DuckyHelper Team. (2026). Showing, not telling: Drawing on the learner's screen as tutoring feedback: A design note (White paper No. DH-WP-2026-02). MingLLM, Inc. https://duckyhelper.com/research/papers/showing-not-telling/

@techreport{duckyhelper2026showingnottelling,
  author = {{The DuckyHelper Team}},
  title = {Showing, Not Telling: Drawing on the Learner's Screen as Tutoring Feedback: A Design Note},
  institution = {MingLLM, Inc.},
  type = {White paper},
  number = {DH-WP-2026-02},
  year = {2026},
  month = oct,
  url = {https://duckyhelper.com/research/papers/showing-not-telling/}
}

1. The problem with telling

A learner stuck on a geometry worksheet asks for help. A chat assistant replies in a separate window: "In line 3, you squared 3 and wrote 6." The learner now has to find line 3, find the 3, and connect the sentence to the place. A tutor sitting next to them would just put a finger on it.

That small act is backed by research. Learning improves when words are tied to what they describe (Mayer & Moreno, 2003), when the learner can see an explanation being drawn rather than finished (Fiorella & Mayer, 2016), and when gesture carries part of the message (Goldin-Meadow & Alibali, 2013). Feedback works best when it is specific and about the task (Shute, 2008). A mark on the learner's own page does all four.

2. What the screen pen does

In DuckyHelper's Mac app, the learner presses fn and talks. While the pill is open the tutor keeps a small current view of the screen. It can then act on that screen in five ways, all drawn in a layer above the learner's apps:

  • Point. A round blue pointer flies to a spot and shows a 2 to 5 word label.
  • Draw. Circles, arrows, boxes, highlights, underlines, labels and freehand paths, drawn one mark at a time like a pen, and numbered for walkthroughs.
  • Write. Handwriting in the margin beside the learner's line: what comes next in their working, or a correction beside a line that went wrong.
  • Spotlight. On a busy screen, everything but one region dims, with a short callout. The learner can still click inside the lit region, and clicking anywhere brings the screen back.
  • Walk through. Step-by-step tutorials in the learner's own apps, where the pointer shows where to click and the step completes when the learner really does it.
A worksheet with a right triangle; the line 6 + 8 = c² is circled in blue with a label saying 3 squared is 9, not 6.
Figure 1. The pen circles a slip on the learner's own page ("3² is 9, not 6"). Frame from our tutorial film, real app interface.

The learner can point too. Holding fn and cmd while moving the mouse leaves a glowing blue trail and shows their words in a capsule; on release the tutor receives a picture with the path drawn on it, so "what is this?" and "merge those" refer to exactly what was traced.

The square root of 14 is ringed by a blue trail, with a capsule reading: what is this?
Figure 2. The learner holds fn + cmd and traces the square root they are asking about. Frame from our tutorial film, real app interface.

3. Where a mark goes

A mark in the wrong place is worse than no mark. Three rules keep marks where they belong. First, the tutor may not describe what it has not seen:

In DuckyHelper

The tutor does not describe what is on the learner's screen until it has actually looked at it.

Second, when the target has words on it, the mark is anchored to those words rather than to coordinates. The Mac reads the text on the screen, and the tutor names the words:

In DuckyHelper

When the target has words on it, the mark goes on those words as the Mac reads them from the screen, not on an estimated position.

Third, marks stay attached to what they were drawn on when the learner scrolls, and clear when the lesson moves on. We have not measured how often a mark lands on the intended spot; like a human tutor's finger, it is only as good as the tutor's reading of the page.

4. Design choices and the research behind them

Table 1. Each design choice and the finding it follows.
Design choiceFindingSource
One mark at a time, in the order of the explanationWatching a drawing appear beat a finished diagram for beginners(Fiorella & Mayer, 2016)
Point whenever the tutor says "this"Gesture is part of how speakers communicate and how learners learn(Goldin-Meadow & Alibali, 2013; Cook et al., 2008)
Labels beside the part, joined by a linePut words next to the picture they describe(Mayer & Moreno, 2003)
The spoken sentence starts with its markConnect pictures and words in time(Mayer & Moreno, 2003; Clark & Paivio, 1991)
Spotlight on busy screensReduce what the learner has to search through(Mayer & Moreno, 2003; Hegarty, 2011)
Circle the wrong step, not the whole pageFeedback about the task, specific and timely(Shute, 2008; Kluger & DeNisi, 1996)
The learner can point backLearners' own gestures and drawings help them learn(Goldin-Meadow et al., 2009; Bobek & Tversky, 2016)

Two choices deserve a note. Order. The pen draws its marks in the order they are explained, because the rules tell the tutor to order them that way:

In DuckyHelper

Marks appear one at a time, in the order the explanation needs them.

This follows Fiorella and Mayer (2016), where learners with low prior knowledge transferred more after watching the instructor draw:

resulted in significantly better transfer test performance than viewing already-drawn diagrams for learners with low prior knowledge
(Fiorella & Mayer, 2016)

Pointing. The rules ask for a pointer whenever the tutor refers to something on screen, the way a speaker gestures. Goldin-Meadow and Alibali (2013) review evidence that

the gestures speakers produce when they talk are integral to communication
(Goldin-Meadow & Alibali, 2013)

5. Feedback as a mark

The most common use of the pen in tutoring is feedback. A circle around the exact slip, with a few words, answers Hattie and Timperley's question of where the learner is (Hattie & Timperley, 2007) without turning attention to the learner as a person, which Kluger and DeNisi (1996) found can make feedback backfire. On the shared whiteboard, the rules make the same move with the learner's own lines:

In DuckyHelper

On the whiteboard, the tutor marks the learner's first wrong step with a short note and leaves the fix to the learner.

The mark says where; the learner still does the fixing. This is the same line the tutor holds everywhere: hints before answers (Bastani et al., 2025).

6. Eddy: drawing without seeing

In the Mac app's Eddy tab, the tutor is called Eddy, and it never sees the learner's screen. It teaches on its own desk instead: on a blackboard it draws in time with its voice, building one picture piece by piece, and on pictures it labels one part at a time as it names them. The learner can click a part of a picture to ask about it. The principles are the same; the canvas is the tutor's own.

7. Privacy and control

  • The Mac app looks at the screen only while the pill is open, and watches continuously only when the learner asks it to.
  • Nothing is captured while a password field has focus, and DuckyHelper's own windows are never in the picture.
  • The pen only draws. Real clicks and typing happen only when the learner clearly asks, or as a demonstration. Never in a password field, and never to complete or turn in the learner's work, pay for something or delete anything.
  • Eddy, in the Eddy tab, has no screen capture and no camera.

8. Limits

  • DuckyHelper has not been evaluated in a controlled study. This paper reports no learning outcomes, no usage numbers and no benchmark scores of our own. Every number in it belongs to the study that reported it.
  • Drawing on the real screen is in the Mac app only, on the display with the mouse.
  • The research cited here studied human instructors, recorded lessons and classroom gestures. Applying it to an AI's pointer and pen is a design bet that a study would need to test.

9. Conclusion

Telling is cheap for an AI. Showing is harder, and closer to what good tutors do. DuckyHelper's screen pen puts the tutor's explanation on the learner's own work, one mark at a time, in time with its voice, and its rules keep each mark about the task. Whether this helps learners more than telling them is an open question that we would like to test.

References

More papers

Built to teach: learning science in a voice tutor

White paper: how DuckyHelper's tutor turns ten findings from learning science into teaching rules and tools, and the limits of that work.

Hints before answers: guardrails for an AI tutor

White paper: what the research says about AI help that gives answers, and the guardrails DuckyHelper's tutor uses to guide learners instead.

Ready to learn

DuckyHelper is ready on your Mac. Learn with Eddy on its blackboard, or with Ducky right on your screen.

Download for Mac
See what it does
The Mac app needs Apple silicon and macOS 14 Sonoma or later. There is no Chromebook, Windows or web version right now.
DuckyHelper
The AI tutor that shows you
FREE TO TRY
Get the Mac app