Engineering notes
How we build Prufa, and what we find when we point it at real launches: architecture decisions, trade-offs, and first-party audit data — including the June 2026 audit of 49 Show HN launches where 38 shipped a critical bug on day one.
Start here: We audited 49 Show HN launches
-
AI testing false positives: 4 of our 6 findings were wrong
Our chaos agent flagged 6 findings on a live production app in July 2026. Four were wrong. The guards, the labeled corpus, and the CI precision floor.
-
Why AI agents fail at login: what we found fixing ours
Our QA agent kept failing a customer login behind reCAPTCHA. The browser passed 10/10 — the LLM's driving failed 5/5. The three fixes that shipped.
-
Idempotency keys for AI agents: what shipping ours taught us
A worked example of idempotency keys for AI agents: client-supplied vs server-derived keys, a Redis-down fallback, and a CLI key that only half-works.
-
Our AI QA agent caught our own landing page contradicting itself
We pointed Prufa at prufa.dev. It found our meta tag promising a 60-second audit while the homepage said 2 minutes — a real trust bug on our own site.
-
State of Launch QA 2026: we audited 63 launches. Here's what breaks.
We ran real-browser QA audits on 63 product launches (Show HN + r/SideProject) in June 2026. 78% recorded zero analytics on launch day. The full data.
-
Website monitoring without analytics pollution
Synthetic and uptime monitoring quietly inflate your GA4 pageviews and conversions. Why GA4 won't save you, and how exclusion should be the monitor's job.
-
We audited 14 side-project launches. Zero critical bugs, same quiet flaws.
Real-browser audits on 14 side-project launches from r/SideProject. None had a critical bug — but 11 of 14 shared the same quiet, growth-capping flaws.
-
We pointed our chaos-QA agent at our own site. It found a shipped bug.
We aimed Prufa's chaos tester at prufa.dev. It caught a 103px mobile overflow our green CI had just shipped — plus two of its own false positives.
-
How Prufa verifies a signup flow (and why the LLM never grades the result)
One signup run, end to end: an LLM-backed agent drives the browser, plain code grades the outcome against a public flow-spec. Same input, same verdict.
-
We audited 49 Show HN launches. 38 had a critical bug on day one.
Free QA audits on 49 fresh Show HN launches: 78% had a critical finding. The full breakdown of what actually breaks at launch — analytics, links, cookies.