Why AI-Generated Apps Break in Production
The Review-to-Production Gap
Survey of 200 U.S. technology decision-makers at manager level and above, conducted by Hanover Research for New Relic, published June 2026.
| What leaders reported | Share |
|---|---|
| Rate AI-generated code higher quality than human code at review | 94% |
| Had at least one production failure tied to AI code in six months | 82% |
| Report more incidents once AI code ships | 78% |
| Report more senior-engineer time spent fixing code | 86% |
| Say at least 25% of AI code needs significant rework | 74% |
Those first two rows are the whole problem in one place. The code reads well. It fails anyway.
It is a highly predictable scenario: an application functions flawlessly during private beta testing and easily handles its first fifty users. However, the moment a customer hosts a webinar, the entire system stops responding for eleven minutes.
There were no code deployments that day, nor did any systems crash. The traffic simply breached an unforeseen threshold.
Why do AI-generated apps break in production?
The problem isn't that AI writes bad code. It writes perfectly functional, clean code, sometimes even better than a developer rushing to meet a deadline. The issue is that LLMs are optimized to solve the problem directly in front of them, not to handle the chaos of real-world traffic. They build for the 'happy path' of a single user. Scaling that to a thousand users requires a completely different set of architectural decisions that are invisible during local testing.
Sonar analyzed more than 4,400 Java assignments across six leading models using its SonarQube Enterprise engine. Every model showed the same weakness: they struggle with anything requiring a global, context-aware understanding of the application (Sonar, 2025).
The examples Sonar names are the ones that bite under load. Resource leaks, like failing to close a file stream. API contract violations, like ignoring an error return.
A leaked file handle is harmless when one person clicks a button. It exhausts the operating system's limit when enough people click it at once.
Sonar also found that newer models don't necessarily fix this. Comparing Claude Sonnet 4 to Claude 3.7 Sonnet, the benchmark pass rate improved 6.3% while the share of bugs rated blocker-severity rose 93%. Better at solving the problem, worse at the failure modes that follow.
The efficiency gap has been measured directly too. EffiBench-X, published at NeurIPS 2025, benchmarked leading models against human-expert solutions across six languages and found that models frequently produce functionally correct code that consistently underperforms humans on efficiency (EffiBench-X, 2025).
The best-performing model in that study reached roughly 62% of human efficiency on average. Correct, and meaningfully more expensive to run.
What is the Concurrency Cliff?
The Concurrency Cliff is the point where an AI-built application stops degrading gracefully and starts failing outright.
Traditional performance problems announce themselves. Pages get slower, then slower, then unbearable. AI-built apps tend to hold steady and then fall off, because the underlying failures are threshold-based rather than gradual.
A connection pool doesn't get worse. It has a fixed number of connections, then it has none, and every request after that queues until it times out.
That's why these apps pass the tests most teams actually run before launch. Functional tests confirm the feature works; they never generate the concurrency that trips the threshold.
Vibe coding production readiness comes down to knowing where those thresholds sit before your users find them. Here's how the recurring failure classes map to what you'd actually observe.
| What you see | What's actually happening | Where to look first |
|---|---|---|
| Fine in testing, times out under real traffic | Connection pool exhausted by per-item queries | Query count per page render |
| One page slow, rest fine | Missing index, full table scan on a filter or foreign key | EXPLAIN on that page's slowest query |
| Random 504s under normal traffic | Blocking third-party call inside the request cycle | Any await on an external API in a route handler |
| Slow after weeks, fine at launch | Unindexed query degrading as the table grows | Row counts against query plans |
| Memory climbs until restart | Unreleased handles, streams, or connections | Resource cleanup in error paths |
| One user sees another's data | Authorization checked in the browser, not the database | Row-level security policies |
Work top to bottom. The database rows are cheapest to check and cheapest to fix, so rule them out first.
Recognize a row?
Request a readiness review and we'll tell you what's exposed before your customers find it.
Request a Readiness Review →Why does the database break first?
An N+1 query is one query to fetch a list, plus one more query for every item in that list.
It's a recurring cause when an AI-built app fails under load, and models generate it readily because loop-then-fetch is the most direct way to express the intent and it returns correct data every time.
Correct data. Wildly wrong query count.
Consider a dashboard listing 50 projects, each showing its owner's name. Fetching the list is one query. Fetching each owner separately is 50 more.
The table below is a query-count model, not a benchmark. It counts what the code does, so you can compare shapes rather than trust someone else's hardware.
| Concurrent renders | Loop-then-fetch (1 + 50) | Batched join or eager load |
|---|---|---|
| 1 | 51 queries | 2 queries |
| 10 | 510 queries | 20 queries |
| 100 | 5,100 queries | 200 queries |
| 500 | 25,500 queries | 1,000 queries |
For scale: PostgreSQL ships with a default max_connections of 100 (PostgreSQL Docs). You can see where 25,500 queries goes.
Three fixes, in the order they pay off:
- Batch the reads. Replace per-item lookups with eager loading or an explicit join. It's usually a few lines. In the example above it takes the page from 51 queries to two.
- Index what you filter and join on. This is easy to miss because the database doesn't do it for you. PostgreSQL's documentation is explicit that declaring a foreign key constraint does not automatically create an index on the referencing columns (PostgreSQL Docs). Without that index, every lookup scans the whole table, and the cost grows with your data instead of staying flat.
- Pool your connections. A pooler such as PgBouncer lets many application requests share a smaller set of real database connections, which absorbs spikes instead of hitting a hard wall.
Do them in that order. Indexing a query you shouldn't be running 51 times is optimizing the wrong thing.
What blocking work is hiding in your request path?
The second reason an AI-built app fails under load is work that shouldn't happen while a user waits. Sending email, parsing an upload, calling a third-party API, waiting on a model response. AI tools put all of these directly in the route handler, because that's the shortest correct path from prompt to working feature.
A four-second upstream call is a mild annoyance for one user. Under concurrency it's hundreds of requests holding server capacity open while doing nothing, whether that capacity is threads, workers, or event-loop connections.
The fix is to move that work out of the request cycle. Accept the request, put the job on a queue, return immediately, and let a background worker handle it. Redis with BullMQ or Celery are common choices.
Two things go with it. Timeouts on every outbound call, so a slow dependency can't hold capacity open indefinitely, and retries with exponential backoff, so a brief upstream failure doesn't reach the user as a broken feature.
The tell is easy to find. Search your route handlers for calls to anything you don't control, then ask whether the user needs that result before the page can render. Usually they don't.
What security gaps do you inherit from AI-built code?
Escape.tech scanned 5,600 publicly available vibe-coded applications in October 2025 and found more than 2,000 high-impact vulnerabilities, over 400 exposed secrets, and 175 instances of exposed personal data including medical records and bank account numbers (Escape.tech, 2025).
These weren't lab builds. The scan targeted publicly reachable, already-deployed applications.
Georgia Tech's Systems Software and Security Lab tracks the other end of this through its Vibe Security Radar project, which scans public vulnerability databases and traces each fix back to the commit that introduced it. Across more than 43,000 advisories, the project confirmed roughly 18 AI-linked CVEs across the second half of 2025, then 56 in the first quarter of 2026 alone, with March 2026 producing more than all of 2025 combined (Georgia Tech, 2026).
Two patterns recur: credentials written into client-side code, and authorization checked in the browser instead of enforced at the database. Both look correct in the interface, which is why they survive to launch.
We cover data isolation and authentication in depth in our guide to turning an AI-built app into a business you can scale, and what to require from any engineering partner in our pre-scaling guide for engineering leaders.
Why can't the AI fix its own scaling bugs?
This is the part teams underestimate, and it's why "just ask it to make this faster" stops working as the codebase grows.
AI coding tools work within a context window. As a repository grows, the model can no longer hold the whole application in view at once, so it reasons about the file in front of it and guesses at the rest.
Scaling bugs are precisely the bugs that require seeing the whole application. An N+1 query looks fine in isolation, because it's a loop and a lookup, both correct. It's only wrong in relation to how often that page renders and how large that table has grown, and neither fact lives in the file being edited.
This produces a recognizable loop. You report the symptom, the model patches the file it can see, the symptom moves, you report it again. Each pass adds code, which consumes more context, which makes the next pass worse.
Two things break the cycle:
- Shrink what the model has to read. Exclude build output and dependency directories from indexing. Split oversized files along route or domain boundaries you choose deliberately, because auto-refactoring large files tends to produce circular imports.
- Change the question. Instead of "make this faster," give the model the query plan, the row count, and the specific line. Models are good at applying a fix they've been pointed at, and unreliable at finding one across files.
The structural point stands regardless. Diagnosing a load failure means holding the database, the request path, and the traffic pattern in mind at once, which is an architectural judgment rather than a coding task.
Stuck in the patch loop?
Talk to us about a refactor that keeps your product live while the layers underneath get replaced.
Talk to Us About a Refactor →Frequently asked questions
Preview environments run with permissive settings and hardcoded redirect URLs. A custom domain exposes what was never configured: missing environment variables, OAuth callback URLs still pointing at the preview host, CORS rules that don't include the new origin, and database policies that reject requests from an unrecognized source.
Almost always query volume rather than query speed. A page issuing 51 queries instead of two is fine at one user and exhausts the connection pool at a few hundred. Once the pool is empty, every request queues until it hits the timeout, so the whole app fails rather than slowing down.
Turn on query logging, load one page, and count the queries. If the number scales with the rows displayed rather than staying constant, you've found one. Rails and Laravel both ship a strict-loading mode that raises an error on lazy loads, which stops new ones appearing.
Refactor in most cases. Your front end encodes validated demand that's expensive to recreate, while the database schema, authorization layer, and request handling can be replaced incrementally with the product live. Rebuilding is justified mainly when the platform blocks code export or the core data model is structurally wrong.
It depends on how much production data has accumulated and how many failure classes are open at once. Batching queries and adding indexes usually takes days, while moving blocking work to queues and correcting authorization takes weeks. Data migrations take longest, because you're changing shape underneath live records.
What to check this week
Five things, in order, before you add any feature:
- Count queries on your three busiest pages. Query logging on, one page load, read the number. Anything scaling with row count is your first fix.
- Run
EXPLAINon the slowest one. A sequential scan on a table you filter or join is a missing index. - Grep your route handlers for external calls. Anything the user doesn't need before render belongs on a queue.
- Check whether authorization is enforced at the database. If the only check is in the browser, treat it as unenforced.
- Search your client bundle for keys. Anything shipped to the browser is public, including keys the model added as a placeholder.
None of this requires starting over. AI-generated apps break in production because of what's underneath them, not because of what users see, and the front end you validated is the part worth keeping.
Fix the layer holding it up. The demo was never the hard part.
Ready to find out what breaks before your customers do?
Hireplicity is a U.S.-led engineering team based in Cebu, building secure, scalable software for EdTech and SaaS companies. Book a 30-minute call and we'll walk your app's failure classes in order, then give you a written plan with a budget range.
Book a 30-Minute Readiness Call →- New Relic — 2026 State of AI Coding Report (June 10, 2026; survey conducted by Hanover Research, n=200 U.S. technology decision-makers at manager level and above) — https://newrelic.com/press-release/20260610
- Sonar — The Coding Personalities of Leading LLMs: A State of Code Report (August 2025; 4,400+ Java assignments analyzed with SonarQube Enterprise) — https://www.sonarsource.com/blog/the-coding-personalities-of-leading-llms/
- Escape.tech — The State of Security of Vibe Coded Apps (October 2025; 5,600 publicly available applications scanned) — https://escape.tech/state-of-security-of-vibe-coded-apps
- Georgia Tech Systems Software & Security Lab — Vibe Security Radar / "Bad Vibes: AI-Generated Code is Vulnerable, Researchers Warn" (2026; 43,000+ advisories scanned) — https://research.gatech.edu/bad-vibes-ai-generated-code-vulnerable-researchers-warn
- Huang et al. — EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code (NeurIPS 2025, Datasets & Benchmarks Track) — https://openreview.net/forum?id=oYD6AP03PZ
- PostgreSQL — Documentation: Constraints (foreign keys do not auto-create an index on referencing columns) — https://www.postgresql.org/docs/current/ddl-constraints.html
- PostgreSQL — Documentation: Connections and Authentication (max_connections default) — https://www.postgresql.org/docs/current/runtime-config-connection.html
- Infosecurity Magazine — Researchers Sound the Alarm on Vulnerabilities in AI-Generated Code (March 26, 2026) — https://www.infosecurity-magazine.com/news/ai-generated-code-vulnerabilities/
- Veracode — Spring 2026 GenAI Code Security Update (March 2026) — https://www.veracode.com/blog/spring-2026-genai-code-security/
