The Emperor’s New Prompt

Over the last year, several managing and executive directors at a global bank have, with great pride, handed down to their teams the custom prompts they've decided are <the> correct way to address an AI chatbot. Not as drafts or suggestions, almost as a decree. The MD has spoken, and the team is to copy, paste, and ask no follow-up questions. The most memorable, for all the wrong reasons, went something like this: you are a world expert in any domain, and you can treat me as your peer.

I had to go for a walk after I read that.

Peer to the “world expert” they had just summoned into existence. Makes Aladdin and his lamp seem dull.

The sentence doesn't configure the model. It configures the writer's self-image for the model. It's a horoscope dressed as a system prompt.

Same energy, different zip code

A week or so ago, Marc Andreessen, the Netscape co-founder-turned-full-time poster, published his own custom prompt on X for his almost 3 million followers to admire. It opens similarly: "You are a world class expert in all domains. Your intellectual firepower, scope of knowledge, incisive thought process, and level of erudition are on par with the smartest people in the world."

Better grammar. Different tax bracket. Same wrong instinct. The MDs wanted the oracle to sit at their level. Andreessen wants the oracle to sit at his too, except he skipped the "treat me as your peer" line because he'd already implied it by spending eight clauses telling the model how the room should feel. Same vanity, same mirror, mirror on the wall, tell me who is the best prompter of them all.

Persona non functional

The thing both prompts get wrong is also one of the most-studied topics in the LLM literature, which makes the consensus on the floor of every C-suite a little embarrassing. Telling a model "you are a world class expert in all domains" does nothing the model can act on. There's no internal switch that flips. There's no expertise dial. The instruction is a vibe, not a configuration.

A 2023 study by Zheng and collaborators, with a title that could have been a tweet, "When 'A Helpful Assistant' Is Not Really Helpful," tested 162 different personas across four LLM families and 2,410 factual questions. The headline finding: personas in system prompts don't improve model performance compared to no persona at all. A 2024 follow-up by Kim and colleagues went further. Persona prompts "sometimes distract LLMs, degrading their reasoning abilities" in seven of the twelve datasets they tested on Llama 3.

So when Andreessen tells the model that its intellectual firepower is on par with the smartest people in the world, the model does what it was always going to do, sometimes a little worse. The prompt doesn't raise the floor. It raises the writer's mood.

Confidence is a costume

The other shared move, explicit in Andreessen's prompt and implicit in the bank versions, is the demand for "explicit confidence levels (high/moderate/low/unknown)." The published research begs to disagree...

A 2024 paper in the Journal of the American Medical Informatics Association, by Savage and colleagues, evaluated three different ways of getting an LLM to report its own uncertainty. The verdict on the method Andreessen asked for: verbalized confidence "consistently overestimates model confidence." Sample consistency, where you run the same question a few times and check whether the answers agree, worked better. The Nature Machine Intelligence paper that came out around the same time named what its authors call the calibration gap, which is exactly what it sounds like. Humans believe a model is more sure than it is, and the longer its explanation, the wider the gap.

The "moderate" the chatbot just told you it is? It's performing moderate. It doesn't know moderate from any other vibe word. Confidence labels from a language model are theater the user is supposed to be too polite to interrupt.

Schemas beat slogans

The interesting question isn't whether the Andreessen prompt is bad. It is bad. The interesting question is what a prompt with discipline actually looks like, and why it doesn't look anything like that one.

Real discipline lives in the shape of the output, not the language of the instruction. If you require the model to produce structured fields for every claim it makes (the source it came from, the verbatim quote, the things it doesn't know, and the places its sources disagree), empty fields become visible. Prose hides gaps. A schema exposes them. The model can still fabricate, but the fabrication has to put on a costume now, because the "source quote" field is asking for something quotable.

The instructions that actually move the needle look boring. "If you don't know, say so" beats "never hallucinate," because the first is permission and the second is a command the model has no internal way to obey. "Quote the source verbatim or flag the paraphrase" beats "verify your own work," because the first is something a reader can check and the second is something a reader has to trust. "Tell me where sources disagree" beats "be provocative, aggressive, argumentative," because the first targets the actual failure mode and the second prescribes a style that introduces its own.

None of these sounds like a system prompt for a peer-level intellectual companion. They sound like a checklist. That's the point. The Andreessen prompt is a costume. A disciplined prompt is a budget.

Alignment, redefined

The pattern has, in the last six months or so, gotten worse. The same MDs and EDs who decreed the canonical prompt have started sort-of-kind-of refusing to read documents that haven't first been run through it. Pre-chewed by their oracle, in their voice, with their priors baked in. The team's job is to submit. The MD's job is to read what the model has to say about what the team submitted.

This is cognitive offload, but it is also something more particular than that. The model handed the document was conditioned, four paragraphs up, on a prompt that asks it to think like the person doing the reading. So what comes back isn't a second opinion. It's a reflection of the first, smoothed and confident. The MDs are reading themselves reading the document – I'm sure Nolan is jealous he didn't think of that for Inception.

The unsettling part is the word the industry has chosen for this. "Alignment." Originally a term about getting models to behave safely and helpfully (which has always been an illusion anyway). In practice, on the floor of a bank, it has slid into something else: alignment to the prior beliefs of whoever wrote the prompt. The team's professional experience is supposed to defer to the model, and the model has been instructed to defer to the boss. Disagreement has to travel through a filter designed by the person being disagreed with.

That isn't governance. That's a hall of mirrors with a corner office.

The self-portrait problem

The reason both versions of "you are a world expert and we are equals" keep showing up, in bank chat windows and on Twitter alike, is that they're doing exactly what the writers want them to do, which has nothing to do with the model. They produce outputs that feel like the writer has been handed a peer-level brief and confirm the user's standing every time they run.

The literature, if you read it, is a polite tap on the shoulder. The prompt isn't changing the model. The prompt is showing you what the writer thought rigor looked like. That's sometimes a flattering picture. It's rarely an accurate one.

Whatever you wrote at the top of your chatbot last week, go look at it. You're in there.