How DuckyHelper learns a website once and reuses it

TL;DR:

• On a website, DuckyHelper can write small JavaScript tools and build a set of actions for that site on the fly.

• It reuses what it learned next time, so tasks run faster, cost less and break less often.

• This is early access. We describe it in plain terms and publish no benchmark numbers.

This is one of the ideas we are proudest of and the one that is easiest to explain badly. It is part of how DuckyHelper does work for you on websites, and it is in early access. Here is the idea in ordinary words.

Click by click is slow and fragile

The obvious way for an assistant to use a website is to imitate a person. Look at the page, find the button, click it, wait, look again, find the next field, type, click. It works, and it is how almost any program can use almost any site. It is also slow, because every step needs a look and a decision, and it is fragile, because a small change to a page can send it clicking in the wrong place.

It is also expensive to repeat. If you ask for the same task next week, a purely click-by-click assistant starts from nothing and does all the looking again.

Turning a site into actions

Websites are not really made of buttons. Behind the buttons are the site's own requests and forms: the things that happen when you press search, add to cart or export. A site's buttons and forms can be turned into callable actions, with plain names such as searchFlights(), addToCart(item) or exportCSV().

When DuckyHelper works on a website, it writes small JavaScript tools for that site and builds a set of actions for it on the fly, using the site's own buttons, forms and requests. With the actions in hand, a task that took dozens of clicks can become a few calls.

Reusing what it learned

The part that makes this pay off is that DuckyHelper keeps what it learned. The first time it meets a site, it builds the tools. The next time, it reuses them. A learned site is an instant starting point, and the task does not need to be explored again.

In plain terms, that is why tasks on a site it already knows are faster, why they cost less to run, and why they break less. The work is done through the site's own actions and not through guesses about where things are on the screen. We make these claims in general terms and do not publish benchmark numbers.

When a site changes

Sites change. A form gains a field, a page is redesigned, an action moves. When the learned tools no longer match, they stop being reliable, and DuckyHelper needs to learn that part of the site again. We do not claim it never fails on a changed site; we claim that learning once and reusing is a better default than starting from scratch every time.

And where a site or app has no web side to learn from, it uses the fallback described in when there is no website: it sees and uses the screen with its own cursor.

What does not change

Speed is not allowed to loosen the rules. Whatever route it takes, DuckyHelper is built to ask first before it sends a message or pays, and not to type your passwords or card numbers. A faster path is still the same careful assistant.

An example in words

Say you ask DuckyHelper to pull the latest orders from an online store's admin page into a spreadsheet. The first time, it works through the page and learns what the site's export does. It turns that into an action it can run again. The following week, when you ask for the same thing, it runs the action directly. There is no exploring, only the task.

From your side the second run simply feels quicker. From ours, it is the point of the design: pay the cost of learning once, then spend it many times.

Why we use code

Small JavaScript tools are a natural fit because that is the language websites already speak. A tool written in the site's own terms does what the page itself does, and it can be run again later. It can also be replaced when the site changes.

What this means for you

If you try the early access agent, expect tasks on sites it has learned to go quicker the second time than the first. Read the follow-up post on click by click versus one call for an illustration of the difference.