30% off your first 3 months

Get the deal

How screen-aware assistants can help with accessibility

by DuckyHelperNov 12, 20264 min read
AI on your computer

TL;DR:

  • A screen-aware assistant can read small text aloud, describe what is on screen and explain unfamiliar controls.

  • It is an extra help, not a replacement for dedicated assistive technology like screen readers and magnifiers.

  • Check anything important, and keep your own accessibility settings as the foundation.

Small grey text on a white page is hard for many people to read, whatever the reason: tired eyes, a small screen, low vision, or a long day. A screen-aware assistant offers a simple kind of help. You ask, it looks at what you are looking at, and it reads or explains it to you out loud.

This article describes where that help is useful, where it falls short, and how it fits alongside the tools that people who rely on accessibility features already use.

What it can do

  • Read a piece of text aloud, such as a label, a paragraph or a notice.

  • Describe what is on the screen in words, such as "a form with three fields and a blue button".

  • Explain a control, icon or message whose meaning is not obvious.

  • Rephrase dense text in simpler words.

  • Point at or mark the part of the screen it is talking about.

Walk-through: reading small text aloud

You are on a settings page. A line of tiny grey text under a switch is too small to read comfortably, and it seems important.

  1. Ask: You press the key and say: "Read the small text under this switch."

  2. It looks: It looks at your screen at that moment and finds the text under the switch.

  3. It reads: It says: "Turning this on shares your usage data with the app's makers. You can change this later in the privacy section."

  4. You decide: Now you know what the switch does. You leave it off.

  5. You check: Because this was a choice about your data, you also zoom in and read the line yourself to confirm. It matches.

The assistant did what a person sitting beside you might do: read out a line that was hard for you to see. The final check is worth the extra moments when the line is about something that matters.

Why it is helpful beyond text size

Dense pages overwhelm many people, not only those with low vision. Being able to say "what is this page asking me to do?" and hear a short answer can lower the effort of unfamiliar software. For people who find reading slow or tiring, hearing the information can help. For someone with a sore hand, speaking a question can be easier than hunting for a small target with the mouse.

What it does not replace

Dedicated assistive technology is built and tested for people who depend on it every day. Screen readers such as VoiceOver on Mac, Narrator on Windows and Orca on Linux read every element in a consistent order and let you navigate by keyboard. Magnifiers and high-contrast modes change how things are displayed everywhere. A general assistant does not do all of that, and it should not be asked to.

Watch out: If you depend on a screen reader or magnifier, keep it on. Use an assistant as an extra layer for particular questions, not as your main way to use the computer.

A few asks that work well

People often start by asking for the whole page and get a long answer they cannot hold in mind. It tends to work better to ask in pieces. Try "what is the heading of this section", then "read the first paragraph", then "what is it asking me to do". Short answers are easier to follow by ear, and you stay in charge of the pace.

For forms, ask for one field at a time: "which box is for my date of birth, and what format does it want". That avoids the common problem of entering the right thing in the wrong place.

Limits to keep in mind

  • It can misread small or low-contrast text, so check anything where a mistake would matter.

  • It may describe a layout in its own way, which can differ from what a screen reader announces.

  • It only knows the moment it was asked about. If the page changes, ask again.

  • It cannot make an inaccessible website accessible. Missing labels and poor contrast remain the site's problem.

Making it work for you

Ask in the way that suits you. Short, specific requests work best: "read this paragraph", "what does this button do", "which field is the email field". If the first answer is too long, say "shorter", and if it is too quick, say "slower". Combine it with your system settings, such as larger text, so the screen is as readable as it can be to begin with.

Privacy and control

Anything you ask it to read is something it has seen. DuckyHelper looks at your screen only while the pill is open and for that answer, and it answers out loud while you keep working. On a Mac the key is fn, and on Windows and Linux it is Ctrl+Shift+Space.

Try it yourself

Open a page with small print, such as terms of service. Ask for one small paragraph to be read aloud and then explained in simpler words. Then read the original yourself. Notice how much of the effort moved from your eyes to your ears.

Frequently asked questions

Is this an accessibility product?

No. It is a general assistant that can help with some accessibility needs. Dedicated tools remain essential for people who rely on them.

Can it describe images?

It can often describe what is in a picture, but descriptions can be incomplete. Check them for anything important.

Should a website rely on assistants instead of being accessible?

No. Websites should be built to work with assistive technology directly.

Get started

DuckyHelper can
do more

It sees your screen and works in any app you use. Try it for 10 minutes, no card.

Get it for Mac, Windows or Linux

Share

Questions and jobs

Do anything

Ask it a question or hand it a job, in any app.

Doing jobs for you is in early access on Mac. Asking works on Mac, Windows and Linux.