30% off your first 3 months

Get the deal

What teachers should know about screen-aware assistants

by DuckyHelperNov 12, 20264 min read
AI on your computer

TL;DR:

  • A screen-aware assistant sees what a student sees, so it can explain the exact problem in front of them.

  • Write a class policy around learning goals: what help is allowed, how to disclose it, and what stays the student's own work.

  • Good uses are explaining, checking and practising. Handing in generated work is the use to rule out.

A student shares a worksheet with an assistant and asks "what does this mean?" The assistant answers about that exact worksheet, because it can see it. That is more useful than a general chatbot, and it raises a fair question for you as a teacher: what should be allowed in your classroom, and how will you know?

What a screen-aware assistant actually does

Such a tool looks at the contents of a student's screen when asked, then answers using that context. It can read a question, a diagram, an error message or a paragraph and respond to it directly. Some can also mark or draw on the screen to point at the part they mean. It only knows what is visible, so it can be wrong about anything off screen.

What happens step by step

  1. The student invokes the assistant while looking at their work.

  2. It reads what is on the screen and takes in the question.

  3. It responds, by voice or text, and sometimes by pointing at part of the page.

  4. The student continues and decides what to write.

The key point for teachers: the assistant is closer to a tutor sitting beside the student than to a search engine. What matters is what the student then does with the answer.

Walk-through: a class policy for homework help

A policy works best as a short page that students and parents can read. Here is one structure, built from learning goals.

  1. Name the learning goal for each assignment. "This worksheet checks whether you can solve one-step equations." The goal decides what help is fair.

  2. Allowed: asking for an explanation of a step, asking for a hint, asking it to check your finished work, and asking it to make practice questions.

  3. Not allowed: pasting a prompt and handing in the output, or having it complete work meant to show your own understanding.

  4. Disclose. Students add one line to each assignment: what they asked the helper and what they changed afterwards. Keep it casual, so honesty is easy.

  5. Show your work. For key tasks, grade the reasoning in the student's own words, in class if needed.

  6. Review. Look at the policy again after a month with the students and adjust it.

Disclosure is the part that makes the rest workable. A student who writes "I asked for a hint on question four, then solved it myself" is showing exactly the habit you want.

Good uses worth encouraging

  • Explaining a step in different words, or from a different angle.

  • Checking a finished solution and finding where it went wrong.

  • Making practice questions or short quizzes.

  • Reading a hard text aloud and explaining unfamiliar words.

  • Helping a student who is stuck at home with no one to ask.

What it cannot do

  • It cannot know your rubric, your class discussion, or what you taught last week unless the student shows it.

  • It can be confidently wrong, so ask students to compare answers with the textbook or notes.

  • It cannot tell you who did the thinking. Only your assessment design can.

  • It does not replace feedback from you. A comment from a teacher who knows the student still matters most.

Privacy and control

Ask what the tool sees and when. Prefer one that looks only when the student invokes it and not while idle, and tell students to close private windows, such as messages or personal accounts, before asking for help. Check the tool's privacy page and your school's rules before recommending anything, and keep student names and grades out of any chat.

Talking about it with students

Spend ten minutes of class time on the policy. Show an example of a good question to the assistant and a poor one, and let students explain the difference. A good question names where you are stuck. A poor one asks for the finished product.

Invite students to tell you when the assistant got something wrong. Finding a mistake in an explanation is itself a strong learning task, and it reminds everyone that the tool supports their thinking and does not replace it.

Finally, think about students who have no one to ask at home. For them a patient helper can be the difference between finishing a worksheet and giving up. A policy that welcomes honest use, with disclosure, protects those students better than one that treats every use as suspicious.

Try it yourself

DuckyHelper is one tool that works this way: press one key in any app (fn on a Mac, Ctrl+Shift+Space on Windows and Linux), ask out loud, and it answers out loud while you keep working. It sees your screen when you ask and can draw on it to show you, so you can try it on a worksheet you wrote and judge whether the explanations fit your teaching. You can try it for 10 minutes with no card.

Frequently asked questions

Can I detect AI use reliably?

Not with certainty. Design tasks and disclosure so that understanding is visible, instead of relying on detection.

Should I ban these tools?

That is your call and your school's. A clear policy with disclosure usually works better than a ban students quietly ignore.

Get started

DuckyHelper can
do more

It sees your screen and works in any app you use. Try it for 10 minutes, no card.

Get it for Mac, Windows or Linux

Share

Questions and jobs

Do anything

Ask it a question or hand it a job, in any app.

Doing jobs for you is in early access on Mac. Asking works on Mac, Windows and Linux.