Why AI agents fail at login: what we found fixing ours

AI agents fail at login for three reasons, in roughly this order: the agent’s own driving is sloppier than a script (the under-diagnosed one), bot-protection scores the automation’s reputation and rejects it, and the agent forgets its session between runs so every run replays the riskiest step. We hit all three in one week on one customer app, and the diagnosis surprised us: the failure everyone — including us — blamed on reCAPTCHA turned out to be our LLM clicking a button that wasn’t the submit button. In the failed runs, the login request never even fired.

This is the write-up of that investigation and the three fixes we shipped, with the numbers. One boundary up front: everything below is about logging into apps our customers own and asked us to test, with credentials they supplied. The goal is never to sneak past a bot detector — two of the three fixes exist precisely so the detector fires less, and when it does fire, our job is to say so honestly.

The investigation: was it the bot detector?

Gremlin, our chaos-testing mode, takes credentials the customer provides and explores the authenticated app looking for bugs. On one customer’s production app — behind reCAPTCHA Enterprise — login kept failing, and the runs nearly went out the door saying so.

The obvious suspect was the bot protection. reCAPTCHA Enterprise is score-based: instead of showing a puzzle, it assesses how risky an interaction looks from signals about the browser, the network, and the behavior, and lets the site act on the score. Automated logins are exactly what it exists to flag. The standard advice for testing behind it — disable the CAPTCHA in staging, or use test keys — is the right answer for CI on your own staging environment, and useless here: the entire point of an external QA agent is to verify the production login a real user faces.

So we isolated the variables. Same target, same credentials, same browser stack, one change at a time. The result, from the investigation we recorded directly in the code that fixed it: a real browser on a residential ISP egress, driving the form with a clean scripted recipe, completed 10 out of 10 logins with zero 403s — across different residential IPs. The browser and the network were fine. The bot detector was not, at that point, the problem.

The LLM was. When the model drove the login instead of the script, the login failed 5 out of 5 times — and in every one of those runs, the network log showed the login POST had never fired. The site never got the chance to reject us. Our agent was rejecting itself.

What actually failed: the LLM’s driving

Three failure patterns, all visible in the step logs:

• Fill-loops. The model re-typed the email while the password sat empty, then re-typed it again — filling a form never changes what the page looks like, so nothing told it it was making progress by standing still. We now detect this with plain code (a counter of consecutive fill actions with no submit) and steer the model out of it.
• Actionability timeouts. Twelve-second waits for elements the model had targeted imprecisely, burning the run’s budget on a step a script does in half a second.
• The ambiguous submit. The model asked to click the element matching the text “SIGN IN.” On this app, that text matched a non-submitting element. The click “succeeded,” the page stayed put, and the login request never fired. This was the killer: an action that reports success while doing nothing.

None of this is stupidity — it’s mismatch. An LLM is good at open-ended navigation of an app it has never seen, which is why one navigates our chaos runs at all. But a login form is not open-ended. It’s the single most deterministic interaction on the web: identifier, password, submit. Handing it to a probabilistic navigator adds latency, cost, and — as measured — failure, to the one step where creativity has no value. We’d written the principle down long before this bug: the LLM navigates; plain code verifies. We just hadn’t applied it to the step before navigation.

Fix 1: log in with plain code, not the LLM

The fix is a deterministic login executor that runs once, before the LLM loop starts. It’s deliberately conservative:

• It only acts on an actual login screen — a visible password field. Single-page apps paint the form a beat after the page loads, so it waits up to 8 seconds for the field to render rather than concluding there’s no login.
• It only acts when the credential set is unambiguous — one identifier, one secret. Ambiguity means it does nothing.
• It fills each field exactly once, then submits by pressing Enter in the password field — the native form submit, not a text-matched button click, for the reason above.
• Any shortfall — fields not found, unexpected DOM, submit failed — falls straight through to the LLM loop with no state changed. A form the script can’t read is no worse off than before.

The scripted path is also cheaper (zero LLM calls for the login) and faster — which matters more than it sounds, because anti-bot tokens embedded in login forms go stale. A fill-loop that dawdles for a minute submits an old token; a two-second scripted login submits a fresh one. In a July 2026 investigation of a production login behind reCAPTCHA Enterprise, this plain-code path completed 10 out of 10 logins with zero 403s, on the same runs where LLM-driven login had failed 5 out of 5.

Fix 2: never report a bot-block as a wrong password

Here’s the part we think most agent builders skip. Some login failures are the bot detector — score-based systems also weigh account and cookie reputation, and repeated automated sign-ins from the same account degrade that reputation over time, no matter how clean the browser is. When that happens, a rejection says nothing about the credentials. Reporting “your credentials were rejected” to a customer whose password is correct is a false alarm — and for a QA product, a false alarm about their login is the fastest way to lose the trust every other finding depends on.

So the failure classification is structural now. After any submit, one shared judge inspects the traffic and the page: an explicit 401/403 on the auth call is a rejection; a page still showing the login form (the POST may never have fired — bot protection can block it client-side) is its own case. And if a bot challenge is active in either case, the run reports a bot-block, not a credential failure — a separate finding, at warning severity rather than critical, whose fix-hint tells the customer the durable truth: allowlist the tester in your bot protection, or provide a QA login path that skips it. One function decides which case applies, and both the finding and the run’s failure status read from it — so the report can never say “wrong password” while the logs say “bot challenge.”

That’s the same honesty rule we apply to testing your login flow before launch from the founder’s side: a tool that can’t tell “broken” from “blocked” isn’t measuring what it claims to measure.

Fix 3: log in once, reuse the session

The last fix attacks the frequency. If every run replays the login, every run rolls the reputation dice, and an every-few-hours chaos schedule looks exactly like credential-stuffing. Real users don’t log in forty times a day — their browser keeps a session. So now our agent does what a browser does:

• After a run that ends authenticated, it captures the browser’s storage state — cookies plus localStorage — and persists it encrypted, keyed to exactly one customer workspace and one domain, with a 7-day expiry.
• The next run for that workspace and domain seeds its browser context with the saved session and skips login entirely. The login — and the bot protection behind it — fires once, not every run.
• Two details turned out to be load-bearing. First, IP stability: the egress IP is pinned per workspace-and-domain, because a saved session presented from a different IP each run trips session-hijacking defenses — the app invalidates the session and you’re silently back at the login screen. Second, the hydration race: a valid reused session can briefly look like the login screen while a single-page app hydrates and processes the restored tokens. Judge too early and you “helpfully” submit a login you didn’t need — re-triggering exactly the bot protection the reuse exists to avoid. Settle first, judge second, and only log in if the session is genuinely stale.

We found the follow-on bugs in fixes 2 and 3 the same way we found the original — by running our chaos agent against real apps and reading the logs when something smelled wrong.

What this doesn’t solve

Honest limits, because the SERP for this topic is full of pages that don’t have any:

• MFA, SSO, and magic links are different problems. A deterministic executor handles identifier-password forms; an OTP or a corporate IdP redirect needs its own machinery, and “allowlist the tester” becomes “provision a test identity.”
• Adaptive reputation can still win. Session reuse reduces how often you log in; it doesn’t make the login invisible. A site that won’t allowlist an automated tester retains the right to block one — and when it does, the honest output is a bot-block finding, not a workaround.
• The staging toggle is still correct for CI. If you own the app and the tests run pre-deploy, disabling the CAPTCHA in test environments is simpler than everything above. Our constraints exist because an external agent tests production.
• This is one app’s numbers. 10/10 and 5/5 are from one production target and one investigation, recorded in our code on 2026-07-03. The failure patterns — fill-loops, ambiguous clicks, phantom submits — we’ve seen broadly; the exact rates will vary.

If you’d rather point an agent at your site than build one: a free Prufa audit walks your public pages in 60 seconds, and Gremlin runs the credentialed chaos testing described here on domains you’ve authorized.

FAQ

Can an AI agent get past reCAPTCHA on a login page? Often, yes — though not by evasion. Score-based protection like reCAPTCHA Enterprise weighs the browser, the network, and the behavior; in our July 2026 testing, a real browser on a residential connection, driving the form cleanly, passed 10 of 10 logins. That applies only to apps you’re authorized to test with owner-supplied credentials — and the durable answers remain allowlisting the tester or logging in once and reusing the session, not fighting the detector every run.

Why does reCAPTCHA reject correct credentials? It doesn’t check credentials at all — score-based bot protection evaluates whether the interaction looks automated, using browser, IP, and behavior reputation. A login rejected while a challenge is active says nothing about whether the password was right. That’s why a testing tool must report the two cases separately: telling a customer their correct credentials were “rejected” when the real cause was a bot score is a false alarm that erodes trust in every later report.

Should the LLM or plain code perform an automated login? Plain code. Login is the most deterministic step in any web app — an identifier field, a password field, and a submit. Handing it to an LLM adds cost and, in our measurements, failure: model-driven runs re-filled fields in loops and clicked non-submitting elements, so the login request never fired. A single fast scripted fill-and-Enter passed reliably, partly because a quick submit uses a fresh anti-bot token. Save the LLM for what’s genuinely open-ended: exploring the app after login.

How do you keep an AI agent logged in across runs? Persist the browser’s storage state — cookies plus localStorage — after a successful login, and seed the next run’s browser context with it. Ours is encrypted at rest, scoped to one customer workspace and one domain, and expires after 7 days. One non-obvious requirement: present the reused session from the same egress IP each run. Sites treat a session that hops IPs as hijacked and invalidate it, which silently puts you back at the login screen.