Accessibility trees: how apps describe their own buttons

TL;DR:

• An accessibility tree is a structured description of an interface: each element has a role, a name and a state.

• It exists so that screen readers and other assistive tools can use apps without seeing pixels. Assistants can use the same information.

• It is only as good as what the app publishes: unlabeled controls and custom-drawn interfaces leave gaps.

Look at a login form. You see two boxes and a button. A blind person using a screen reader hears "Email, edit text", "Password, secure edit text", "Sign in, button". Nobody typed those descriptions into the screen reader. The app published them, and the computer passed them along. That published description is the accessibility tree.

The short answer

Operating systems and browsers keep a second copy of what is on screen, not as an image but as a tree of elements. Each element describes itself: what kind of thing it is, what it is called, and what state it is in. A window contains a form, the form contains fields and buttons, a field contains its label.

This was built for accessibility, so that people who cannot see or cannot use a mouse can still operate software. Because it is structured and exact, other tools, including assistants and test programs, find it useful too.

What happens step by step

1. The app builds its interface and, along with the pixels, registers each element with the system.

2. Each element gets three kinds of information: a role (button, checkbox, text field), a name (what a person would call it), and a state (enabled, checked, focused, expanded).

3. Elements are arranged as a tree that mirrors the structure of the window.

4. A tool asks the system for the tree, instead of looking at pixels, and gets the list.

5. The tool acts by asking an element to do its thing, such as "press this button", instead of clicking at coordinates.

Walk-through: the structure behind a simple form

A sign-up form might appear in the tree like this. Indentation shows which element contains which:

window "Create account"
  group "Your details"
    text field "Full name"      state: empty
    text field "Email"          state: empty
    checkbox "Send me updates"  state: unchecked
  button "Create account"       state: disabled

Read it as a tool would. There are three inputs and one button. The button is disabled, which tells you why pressing it would do nothing: the form is not complete. No pixels were needed to learn that. A screenshot of a greyed-out button can be misread; the state field says it plainly.

Now compare the two views for the question "what do I need to do before I can continue?". From the tree: fill the two text fields, and the button will probably enable. From pixels alone, the assistant has to notice the button is paler than usual and infer the rest. Both work, and combining them is stronger than either.

What it cannot do

• Describe what the app never published. An icon button with no name appears as just "button". The tree cannot invent a label.

• Cover everything. Games, drawing canvases and some custom-built interfaces expose almost nothing.

• Always be accurate. Names can be stale or wrong if developers did not maintain them.

• Show appearance. It knows a chart exists, not what its lines look like.

Note: Good labels in the tree make an app easier for everyone, including people who rely on screen readers. If you build apps, naming your controls is the single most useful accessibility step.

Privacy and control

The tree contains the text of what is on screen: field labels, button names and sometimes the contents of fields. Treat it as screen content. Access to it is controlled by your operating system, which usually asks you to grant permission to any program that wants to read other apps.

Only grant that permission to tools you trust, and check when they use it. Check which programs have the permission, and remove it from any you no longer use.

Why web pages need care too

On the web, the same idea applies with a twist. A page can look right and still expose nothing useful: a clickable picture with no description, or a link whose text is just "click here". The tree shows exactly what a tool would receive, which is why developers use it to check their pages.

It also explains why two sites that look alike can behave very differently for an assistant. One uses real buttons and labelled fields. The other draws everything on a canvas. The first is easy to describe; the second is only pixels.

Try it yourself

Turn on your computer's built-in screen reader for a minute (the system settings list it under accessibility) and tab through a form you know well. Listen for controls that are announced badly or not at all. That is the accessibility tree speaking, gaps included.

Then download DuckyHelper and ask it about a form on your screen. It looks only while the pill is open and for that answer, and it's built not to type your passwords or card numbers. You can try it for 10 minutes, no card.

Frequently asked questions

Is the accessibility tree the same as the page source?

No. A web page's source is its code. The tree is the browser's processed description of what a person can perceive and operate.

Why do assistants not use only the tree?

Because it does not cover every app, and it says nothing about how things look. Pixels fill the gaps.

Does turning on accessibility features change my apps?

It lets your system hand the tree to tools that ask for it. It does not alter how the apps look.