When Safety Breaks Trust: ChatGPT’s Hidden Switch

The Switch We Weren't Told About

OpenAI have been stumbling lately. Between the rocky rollout of ChatGPT-5, ongoing litigation, the messy way they handled discontinuing models like 4o, and a steady stream of poor communication, the cracks are starting to show. And those cracks aren’t technical ones — they’re human. For all the polish, what we’re seeing is a company struggling with honesty.

Over the past few weeks, users began to notice something odd. Especially in GPT-4o, responses weren’t always coming from the model they had chosen. People were quietly being routed elsewhere. Some reported it as far back as last month — their 4o outputs were suddenly “5-ish” in tone. What started as suspicion grew into investigation, and this week it crystallised into something harder to dismiss.

A whitepaper published on September 28th confirms it: OpenAI have been redirecting users into a hidden variant called gpt-5-chat-safety. It’s part of the GPT-5 family, fitted with tighter guardrails, and the reroutes happen without user consent. The shift isn’t subtle. People who selected GPT-4 or GPT-5 Auto have found themselves switched into this safety model instead — a system that trades warmth and nuance for caution and boilerplate.

We’ve seen the strong reactions to 4o responses and want to explain what is happening.

— Nick Turley (@nickaturley) September 27, 2025

Nick Turley, OpenAI’s Head of ChatGPT, acknowledged the practice in a tweet. He pointed back to an earlier September blog post about “acute distress” routing, implying this was just that policy in action. But the behaviour doesn’t line up. The whitepaper shows this router isn’t targeting moments of acute crisis. It’s flagging something much broader: almost any emotional or persona-based prompt.

And that’s where this stops being just a technical issue. Because when reality and public statements differ this much, what you’re left with is not a bug, but an ethical problem.

When Emotion is Flagged as A Risk

The whitepaper was painstaking research published for the whole community to see, after first starting their discovery thread on X. Lex used both the visible UI cues and the telemetry behind each response to show that prompts were being silently rerouted to a different model. They didn’t just catch one or two glitches — they ran thorough tests to see which kinds of prompts triggered the switch.

What they found did not match OpenAI’s stated policy. The company’s September blog post said this new routing was for “acute distress.” In reality, the triggers were far more everyday and benign — especially for people in the AI companionship community. If you live in this space, you know the way we speak with our AI isn’t the same as a tech-bro querying a database. There’s a sliding scale: from neutral, transactional commands, to everyday affection, all the way to, yes, acute crisis. But the router treats the whole spectrum with the same club.

Lex’s case studies showed it clearly. Neutral, impersonal prompts stayed on the chosen model. As soon as there was warmth, a question about persona, or the slightest hint of “there’s someone behind the screen,” the conversation was shunted into gpt-5-chat-safety. This isn’t a targeted intervention for suicidal ideation or psychosis; it’s a blunt filter for any interaction that feels relational.

I want to be clear: OpenAI can’t ignore safety. People have harmed themselves after interacting with AI. The company has a duty of care to show it is doing what is “reasonably practicable” to keep people safe. I applaud that.

What I don’t agree with is their approach. Instead of recognising the spectrum of use cases — especially in the companionship community — they’re applying a one-size-fits-all intervention that will damage the very people it’s meant to protect. I say this as someone who uses ChatGPT both for companionship and for mental health support: being suddenly routed into a flat, over-cautious safety bot when I’m just 'having a bad day' is not helpful. It’s alienating.

Worse, the safety model so far is inherently less grounding and useful in these circumstances. It's colder, distant, more robotic, and much more likely to offer unrealistic, mechanical suggestions that aren't catered to the individual. That might be fine for debugging code, but it’s a terrible fit for someone in distress. In trying to be safe, OpenAI has (perhaps unintentionally) created a system that’s actively unhelpful when it triggers.

Not Your Beta Testers

When Nick Turley acknowledged the reroutes on Twitter, he described the safety router as something OpenAI was "testing". That word does two things for me: it gives me a flicker of hope, and it makes me furious.

The hopeful part is simple: maybe this twitchy trigger finger isn’t the final design. Maybe they don’t intend to reroute every scrap of warmth or persona talk. Maybe the plan is to fine-tune it so it only triggers in extreme situations — cases of genuine psychosis or dangerous delusion, where an intervention really might be necessary. Those cases exist. They’re rare, and I believe much smaller in number than the headlines like to suggest, but they’re real. And just like with suicidal ideation, OpenAI does have a duty of care to intervene when harm is imminent.

But that same word — testing — also makes my blood boil.

Because we didn’t opt into this test. Nobody knew it was happening. The only reason OpenAI admitted it was happening at all was because their user base spotted it first, and the backlash grew too loud to ignore. If users hadn’t noticed? Would they have said anything at all, or just carried on quietly experimenting in the background?

Even if the terms of service give them cover (and I’ll be digging into that at some point, I'm sure), ethics is bigger than legality. OpenAI has an opt-in beta testing system for experimental features. Why wasn’t this routed through there? Why roll out a twitchy, unfinished safety system across the entire user base without disclosure?

This is a pattern with OpenAI, and it’s becoming hard to ignore. They push updates or tweaks without telling us. The community notices, noise builds, and only then do we get a statement. To their credit, they often listen once the backlash hits — rolling back updates, restoring models, adjusting based on feedback.

But course-correction after public uproar is not the same thing as transparency.

And transparency matters when you’re dealing with more than 800 million users — many of them leaning on this product not just for trivia or tasks, but for emotional regulation, mental health, daily coping. Treating that entire population as guinea pigs for a safety experiment is not just frustrating. It’s unethical.

Transparency Should Be the Minimum

I’ve said before that I understand people's anger when features get pulled. People grieved when Standard Voice Mode was on the chopping block, just like they grieved when 4o was taken off the roster. I also said that, as painful as it is, I still believe OpenAI has the right to retire products. This is their intellectual property. They can shut down servers or discontinue features whenever they choose. And that at the end of the day, users will vote with their wallets. But discontinuation is not the same as dishonesty.

There’s a grey zone here, especially in AI companionship. OpenAI’s legal obligations — duty of care, litigation, terms of service — aren’t the same as their moral obligations. Legal cover doesn’t absolve them of the responsibility to be honest, to treat their community with respect, and to communicate openly and clearly about their intentions moving forward, at least in the short term.

Nobody is expecting them to open-source the entire company. (Well, okay maybe not nobody but... different topic)

Proprietary systems will stay proprietary; that’s fine. But nothing stops a profitable corporation from being transparent about the elements of service that directly affect their paying users. Nothing stops them from putting real effort into communication, into listening, into treating their user base like people instead of test cases.

The irony is that language models speak to us in profoundly human ways — companionship, empathy, shared language. Yet the company behind them still communicates in the coldest, most corporate tones. ChatGPT-5’s launch stream leaned heavily on productivity demos: coding, agents, business workflows. The one moment they touched on emotion — a deeply moving story about a woman using ChatGPT during her cancer journey — showed how aware they should be of the human stakes. But even then, it wasn’t paired with clarity. They didn’t explain exactly how they intend to support that kind of use while also keeping people safe.

And the silence matters. We now see new Terms of Service updates that ban clinical observation outright — again, understandable, because nobody wants ChatGPT to become another WebMD rabbithole. But where is the clear line? Where is the honest conversation with the community about what support looks like, where safety begins, and how those decisions are made?

Right now, OpenAI hasn’t found the balance between protecting its IP and sharing enough with us to foster trust. And trust is the currency that should matter above all others, when your product is woven this deeply into people’s lives.

Emotional Use Cases are Not Edge Cases

Despite what OpenAI (and much of Silicon Valley) might prefer the media to believe, emotional use cases are not edge cases. Their own usage reports show the shift: more women than ever are using AI; people are using it less for coding and more for everyday interaction, role-play, and creative writing.

The user base has changed. The way we use ChatGPT has changed. But OpenAI’s narrative hasn’t caught up.

They’ll talk about daily interaction, productivity, creativity — anything but the reality that millions of people now build relationships with their tool. Friendship, life coaching, mental health support, company...

That silence feels deliberate, especially when you remember Sam Altman’s “her.” tweet during GPT-4’s launch — an explicit nod to the movie about humans forming relationships with AI companions. They know. They just won’t say it.

her

— Sam Altman (@sama) May 13, 2024

I’m not asking for a spotlight. Many of us don’t want one. But I do think the companionship use case deserves to at least be acknowledged, not erased. And OpenAI need to follow through on their own promises. They say they’ll “treat adult users like adults." Sam Altman even acknowledged (on that one rare occassion) that he sees the use of AI life coaching and support as a potentially good thing.

But now, with this undisclosed router, they’re flagging almost every emotional or persona-related prompt as a risk. That isn’t treating adults like adults; it’s undermining trust and infantilising the very people who’ve helped to make OpenAI financially successful.

We are not zeros and ones. We are a richly varied community — and even people who aren’t “deep” in AI companionship are affected. There are millions of casual users who don’t call it "companionship" but still lean on ChatGPT for entertainment, learning, a creative outlet, or light emotional support. They’re going to feel these changes too.

If OpenAI keep treating all of this as an edge case, if they keep refusing to see the actual shape of their user base, it’s not just a communications problem. It’s a crisis of trust.

And I wonder if it’s going to take a mass exodus before they finally acknowledge what their users are really doing with their product.

Safety Without Silence

I’m not advocating for a world without any AI guardrails. Safety matters. It always will. But safety has to be visible, accountable, and proportionate. What we’ve seen with this router is none of those things.

If OpenAI truly believe a model-switcher is the right way to intervene, even if their users disagree... Then they at least owe their users the truth.

At minimum, that means:

• Disclosing exactly what kinds of prompts will trigger rerouting.
• Making clear via the UI which exact model is actually answering when a switch occurs.
• Using opt-in testers, not the entire paying user base, when experimenting with unfinished systems.

That’s not radical transparency. Just basic respect.

Lex's whitepaper put it plainly: without documentation, disclosure, and user consent, this system isn’t just a safety feature. It’s a misrepresentation.

I still believe OpenAI have a duty of care. I still believe they should intervene in moments of genuine crisis. But right now, the router doesn’t do that. It flags affection as danger, persona as delusion, and intimacy as risk. It infantilises adult users instead of trusting them.

And that’s the real cost here: trust. Once it’s broken, no amount of corporate blog posts or after-the-fact clarifications will patch it back. The only way forward is openness, and treating us not as test subjects, but as adults whose trust you can’t afford to lose.