Andrej Karpathy told a story about his app MenuGen, and most engineers heard the wrong lesson in it.
Users sign in with Google. They buy credits with Stripe. The agent he was vibe coding with wrote code to connect the two accounts, and it picked the most reasonable-looking join it could find: match them by email address. Clean. Readable. Wrong. People use different emails for Google and Stripe all the time, so the payment got attached to the wrong account, or to no account. He filed this under the part of engineering that’s still human: taste, judgment, knowing what a user “really means.”
What the agent shipped is not a failure of taste. It’s a named, documented, bounty-paying security vulnerability called nOAuth, and the reason the agent walked straight into it tells you precisely which decisions you can never hand off, no matter how good the model gets.
The bug has a name, and it’s not “the agent lacks aesthetics”
Matching accounts by email is one of the most well-worn anti-patterns in authentication. It has a CWE. It has a nickname (nOAuth). It has a standing place on every account-takeover checklist bug bounty hunters run.
Here’s the attack the agent’s code invites, independent of the wrong-funds bug Karpathy hit. If your system links a Google identity to an existing account whenever the email matches, I register an account with your email through a provider that lets me set an unverified email claim, you later sign in with Google, and the system helpfully merges us. Now I’m in your account. The WorkOS and Auth0 writeups both lead with the same warning: an email address is an attribute, not an identity. It can be reassigned, spoofed, or simply differ across providers. The correct key is the (issuer, subject) pair the identity provider mints, with email stored as a mere attribute and never trusted as the anchor.
The agent didn’t know that, because the agent doesn’t carry a threat model. It carries a plausibility model.
An agent optimizes for the join that looks right in the diff. Security and identity are exactly the domains where the right answer looks wrong and the wrong answer looks clean.
Why agents specifically lose here
Karpathy has a sharper frame for this elsewhere in the same talk than the one he used for MenuGen. He says these models are jagged: superhuman in domains the labs trained hard with verifiable rewards (code that runs, math that checks), and bizarrely weak everywhere a verifier was never built. The same model that finds a zero-day will tell you to walk to a car wash 50 meters away because it forgot you need the car at the car wash.
Identity resolution sits in the valley of that jaggedness. There’s no unit test that goes red when you pick the wrong join key. The code compiles. The happy path works. The demo with one email passes. The failure only appears when two real humans show up with mismatched accounts, or when an attacker shows up on purpose, and neither of those is in the reward signal the model was trained against. So the model does what it does in every unverifiable domain: it produces the most statistically reasonable-looking artifact and moves on, confident.
This is the thing to internalize. The agent isn’t bad at security because security is hard. It’s bad at security because security has no cheap verifier, and a model is only ever as good as its verifier. Wherever you can’t write a test that catches the mistake, the agent will confidently ship the mistake. That’s not a list of edge cases. That’s a map of your job.
“Taste” is the wrong word, and the wrong word leads you to the wrong defense
If you accept Karpathy’s framing that what’s left for humans is taste, you’ll defend your codebase by reviewing for taste, by skimming agent output and asking “does this look reasonable?” And the join-by-email bug looks completely reasonable. That’s the trap. Reasonable-looking is the exact failure mode. You can’t out-taste a problem whose entire danger is that it looks fine.
The defense isn’t taste. It’s specifying the load-bearing decision before the agent ever fills the blank.
Karpathy almost says this himself, then undersells it. He notes the real fix was that there should have been a persistent user ID, and the agent couldn’t know that because “the agent doesn’t really know what a user actually means.” Right. So that decision, what is the unique identity of a user in this system, is not a blank you let the agent fill. It’s the blank you fill before you hand the work over. The agent fills blanks beautifully, but only when the blank is sharp. Identity is the blank that’s never sharp by default, because the sharpening requires a threat model and a data model the prompt didn’t contain.
Concretely, the spec the agent needed wasn’t “link the accounts.” It was:
The canonical user identity is a
user_idwe mint on first contact. Nothing else is identity.- Google and Stripe identities are linked records that point at a
user_id, keyed by their own provider subjectsub), never by email.- Linking a second provider to an existing user requires the user to be authenticated as that user at link time. No silent merge on a matching attribute.
- Email is stored, displayed, maybe used for notifications. It is never a lookup key for auth.
Four sentences. Hand the agent those four sentences and it writes the correct code, because now the blank is sharp. The agent didn’t lack taste. It lacked the four sentences, and producing those four sentences is the actual work that didn’t go away.
## The pattern under the pattern
Account linking is one instance. The general shape is this: any time an agent has to resolve identity or establish a relationship between two records, it will reach for the attribute that’s sitting right there in the data, and the attribute sitting right there is almost never the durable key.
I’ve shipped the backend version of this bug enough times to recognize the smell. A worker that “authenticated as the user” by passing the service key, which silently degraded to the anonymous role, so row-level security quietly stopped protecting anything and writes went nowhere. An RPC that trusted a user-supplied ID as identity and handed back another user’s data. Every one of those is the same mistake wearing different clothes: trusting a convenient value as identity when identity should have been a key you control and verify. The agent will make this mistake faster and more fluently than you ever did by hand. That’s the whole problem with raising the floor without raising the ceiling.
So here’s the reframe to carry into your next agent session, the one that actually changes behavior:
> Before you let an agent write any code that joins two records or decides who someone is, write the identity spec yourself. That sentence is the join key, and the join key is the one thing the model cannot infer and cannot test its way into.
Treat the agent like a fast, fluent engineer who has never been burned by a production incident and never read a bug bounty report, because that is exactly what it is. It will give you clean code on the happy path forever. The places it hurts you are the places with no verifier: identity, authorization, money movement, idempotency, anything where “looks right” and “is right” come apart.
Karpathy is correct that the ceiling is rising faster than the floor, and that the engineers pulling away are the ones who direct agents instead of typing for them. But directing isn’t taste. Directing is knowing, before the blank gets filled, which blanks are load-bearing, and filling those yourself in plain, sharp sentences.
The agent will write the code. You still have to decide what a user is. That part didn’t get automated, and from where I’m sitting, it’s not about to.
What’s the join-key bug you’ve caught an agent shipping? I’m collecting the failure modes, the more boring and load-bearing the better.