Cite as: Real Problem AI problem “Is Claude getting worse? How do I tell if an AI model was quietly nerfed?”. Opportunity score 8.2 out of 10 (severity 7, AI feasibility 9, market signal 9, competition gap 8). Category AI / Agents. Trend LLM. Source signal: Hacker News thread on Livenerf, a tracker asking "Has Opus 5.5 been nerfed yet?", 29 September 2026, 394 comments.. Canonical URL: https://www.realproblem.ai/idea/is-claude-getting-worse-how-to-tell-if-an-ai-model-was-nerfed.
Is Claude getting worse? How do I tell if an AI model was quietly nerfed?
Every new model launch is followed by weeks of arguments over whether it got dumber, and paying users have no neutral, repeatable way to check if the model behind their plan changed or if they just got used to it.
Who has it: Developers and teams paying for Claude, ChatGPT or Gemini plans who depend on consistent model quality for daily work.
- Severity7/10
- Feasible today9/10
- Market signal9/10
- Competition gap8/10
Heard in public
“Hot take, none of the models are getting "nerfed", people are just getting used to the new level of intelligence.”
Quoted word for word from the public post linked below. Nobody submitted it to Real Problem AI.
Hacker News thread on Livenerf, a tracker asking "Has Opus 5.5 been nerfed yet?", 29 September 2026, 394 comments.What nothing does yet
A personal drift monitor: replay a frozen sample of your own real prompts against the same model name every day, score the answers against your accepted outputs, and alert with evidence when quality, length or refusal rate shifts, including whether the plan route and the API route behave differently.
Tools that solve this
3 named so far, and where each one stops.
- Livenerf and similar community trackers
Run a fixed set of prompts on a schedule, but cover a few models and tasks and are not tied to your own workload.
- Public leaderboards (LMArena, Artificial Analysis)
Rank models at a point in time, not drift of the same model name over weeks.
- Provider status pages
Report outages, never quality changes.
How you might build it
- 01Scheduled prompt replay
- 02Answer scoring against golden outputs
- 03Statistical drift detection
- 04Plan vs API comparison
- 05Shareable evidence reports
#NOV3 · Canonical URL: https://www.realproblem.ai/idea/is-claude-getting-worse-how-to-tell-if-an-ai-model-was-nerfed