Why RPA bots break when screens change

You arrive at work in the morning and find that a bot has stopped. On investigation, a button on a business partner’s web system had moved slightly. Teams that run RPA hear this story often.

This article explains how and why RPA bots stop when screens change, and how to build automation that stops less often and can be fixed quickly when it does. The ideas are not tied to a specific product; they apply to any automation that operates web systems.

How RPA sees the screen

An RPA bot records the operations a person performs on screen as a procedure, then replays the same procedure. To do this, it identifies the target of each operation (a button or an input field) using cues such as these:

Screen coordinates: “Click the point 320 pixels from the left and 180 pixels from the top”
Images: “Find the button that looks like this and click it”
Element information: the ID, name, XPath, or CSS selector in the web page’s HTML

Element information is generally considered more resilient to changes than coordinates or images. But every one of these cues assumes the screen as it was at recording time. When that assumption no longer holds, the bot cannot find its target and stops.

Five main causes of stops

1. Changes to screen design or fields

An update to a web system changes a button’s position, an element’s ID, or the order of fields. Even if the screen looks almost the same to users, the bot stops if the cues it relies on have changed. Web systems provided by business partners or external services often change without advance notice, so the bot stops suddenly one day.

2. Unexpected screens appear

A screen that did not exist at recording time interrupts the flow: a notice pop-up, a prompt to change a password, a maintenance page, or an extra authentication step. A person would close it and move on, but the bot cannot find the target of its next operation and stops.

3. Slow page loads

If a screen takes longer to load than the bot is set to wait, the bot tries to operate an element that has not appeared yet and fails. A bot that stops only when the system is busy, such as at month-end or in the morning, often has this cause.

4. Data exceptions

The bot receives data unlike the data at recording time: a blank field, a value longer than expected, or a business partner code that does not exist. Even though the screen has not changed, the input is rejected or the bot lands on an unexpected screen, and it stops.

5. Changes in the runtime environment

Bots also stop because of changes on the side that runs them, such as a browser update, a change in screen resolution or zoom level, or an expired password on the bot’s account.

Worse than stopping: not being able to fix it

No automation can avoid screen changes. The real trouble for operations teams comes after a bot stops.

Only the builder can fix it: only the person who built the bot understands how it works, and after that person transfers or leaves, no one can fix it
No one notices it stopped: no alert reaches anyone, and the unprocessed work is found days later
What to do with half-finished work: the bot stopped halfway through data entry, and someone must decide whether to finish the rest by hand or start over
No one has the full picture: with bots built separately by each department, no one knows how many there are or which ones have stopped

A system for noticing and fixing stops quickly matters as much as measures that make stops less likely.

Designing automation that stops less and is easier to fix

Prefer official integrations over screen operation

If an API, CSV download, or data integration service is available, it is more resilient to changes, faster, and more reliable than operating the screen. As a rule, limit automated screen operation to tasks with no other option. Also check whether the service’s terms of use permit automated operation.

Identify targets by cues that rarely change

Use element information rather than coordinates or images. Within element information, identifying a target by its on-screen label or role, such as “Login ID”, is less affected by system updates than using something that changes easily, such as an auto-generated ID.

Build “wait” and “verify” into the procedure

Before each operation, wait until the target appears. After each operation, verify that the expected screen has loaded. With verification in the procedure, it becomes clear where and why the bot stopped, and the bot no longer continues with incorrect input.

Route exceptions and judgment calls to people

There is no need to leave all unexpected data or all decisions to the bot. Deciding in advance to send items that do not meet the conditions to a person for review, and to require approval before registration, prevents the bot from forcing its way forward and processing things incorrectly.

Alert someone as soon as it stops

Make sure the person in charge receives a notification on failure, and keep a screenshot of the screen where the bot stopped along with the logs. Being able to see on screen what happened makes a large difference in how long it takes to find the cause.

Split the procedure so it can resume midway

Divide the procedure into several steps so that it can resume from the step where it stopped. If running a completed step again causes no problems, deciding how to rerun becomes simple.

Keep the procedure in a form anyone can read

What it is for, which screen it works on, and what it does there. If the procedure reads in business terms, people other than its builder can fix it. If a change history is kept, you can also trace when its behavior changed.

Another option: AI that reads and operates the screen

Automation is now available in which you write the procedure as text and AI reads the screen and operates it on each run. If you write “Enter it in the Login ID field”, the AI finds that field on the screen and enters the value. Because it does not record coordinates or element names, small changes, such as a moved button or one added field, do not require rewriting the procedure.

It also has weaknesses.

Cost and time: it uses an AI model for every operation, so it incurs usage fees and each operation takes time. Conventional RPA is better suited to tasks that repeat large volumes of routine operations at high speed
Verification is needed: AI decisions are not guaranteed to be the same every time. Combine verification steps and human approval before anything is registered or sent
Major redesigns still need work: a large-scale redesign that changes the screen flow itself requires revising the procedure
Data handling: the content displayed on screen is sent to the AI model provider. Decide in advance which model to use and what information may be sent
Unsupported sites: sites whose terms of use prohibit automated operation, and sites with CAPTCHA (image verification), are out of scope

A practical approach is to use AI screen operation for web tasks that tend to break, and conventional RPA or APIs for high-volume, stable operations.

Building it with Kitewell

Descarty’s Kitewell is business automation software designed on the ideas above.

Write procedures in words: list steps such as “Open the page”, “Enter”, and “Verify”, and AI operates Chrome. Passwords are referenced by name, and their values are never shown to the AI
When it stops, resume from there: a failed step is shown with its screenshot and logs. After you fix it, only the failed step and the steps after it run again, and the results of completed steps are reused as they are
Fix it through conversation: from the screen of the stopped step, you can discuss the cause with an AI assistant. Proposed fixes are applied after you review the diff
People approve decisions: approval can be set on any step, and who made each decision and when is recorded
Notifications: failures and pending approvals are sent to Mac, email, Slack, Microsoft Teams, and more

For how Kitewell differs from RPA and how to migrate, see Moving from RPA. For examples by industry and department, see Use cases.

Summary

RPA runs on the assumption that the screen is as it was at recording time, so it stops when screens change, unexpected screens appear, pages load slowly, data has exceptions, or the environment changes
The bigger problem is not the stop itself but being unable to fix it, not noticing it, and not being able to resume midway
Prefer official integrations, identify targets by cues that rarely change, and design in waiting, verifying, routing to people, alerting, and resuming midway
Writing procedures as text and having AI read and operate the screen holds up well against small screen changes. Choose between approaches based on cost, speed, verification, and data handling