Computer use

• The AI sees the screen, decides what to do, and acts with a cursor and keyboard.
• It works in apps that were never built for it, because it uses them the way a person does.
• For risky steps, like sending or paying, a careful system asks first.

Computer use means an AI operating a computer through its screen. Instead of calling a program's hidden functions, it looks at what is displayed, works out what each part is, and then moves a cursor, clicks, scrolls and types, the same way you would. Because it only needs the screen, it can work in almost any app.

Example: opening a page and clicking a button

Say you ask an AI to download this month's statement from your bank's website. It follows a loop:

1. It looks at the screen and sees a browser with a home page.
2. It decides the next move: open the statements page. It moves its cursor to the link and clicks.
3. It looks again, finds the button labelled "Download", and clicks it.
4. It checks that a file appeared, and reports back.

Each cycle of look, decide, act is one step. If the page asks you to sign in, a careful system hands that to you. DuckyHelper, for example, is built not to type your passwords or card numbers, and to ask first before it sends a message or pays. Its agent work is in early access.

Why it matters

Most software has no way for another program to control it, but all of it has a screen. Computer use opens that software to automation, at the cost of being slower than calling a function directly. Because the AI can act, it needs limits: asking before steps that cannot be undone, which is the idea of a human in the loop. Read what computer use means for AI assistants for more.