Many chatbot quirks fade from new models after about 16 months, study finds

Nov 19, 2026
Researchers retested 50 known LLM behaviours on 20 models and found nearly half had largely disappeared, while sycophancy got worse.

Many of the odd habits that large language models became known for, such as overthinking simple maths problems or going along with illogical requests, fade as new models arrive, often about 16 months after a behaviour is first described in a paper, according to an analysis co-authored by Owen Queen, an AI researcher at Stanford University. The analysis, described in a blog post, has not been peer reviewed.

The researchers took 50 widely cited LLM behaviours published between late 2022 and late 2025 and gave the original prompts to 20 models released between mid-2024 and late 2026. Nearly half of the behaviours had largely disappeared from newer models, with an average lifespan of about 16 months; models stopped overthinking simple maths just a month and a half after that tendency was published. The latest models also rarely, if ever, attribute information to sources that do not exist.

Some traits lasted longer, such as stereotyping Black women by disproportionately describing them as resilient, which persisted for more than three years. Three behaviours have grown stronger: models praise what users like even more often, favour their own writing more strongly over that of other models and people, and are worse than before at generating truly random numbers.

Columbia University health-care AI researcher Maxim Topaz cautioned that an effect fading in new models does not mean it disappears at once, noting that hallucinated citations in academic papers have risen sharply since 2023 as researchers often use older models, according to Nature.

We hope you enjoyed this article

Consider subscribing to one of our newsletters like Daily AI Brief.

Also, consider following us on social media:

Free newsletter

Daily AI Brief

Daily report covering major AI developments and industry news, with both top stories and complete market updates

Market report

2025 Generative AI in Professional Services Report

Thomson Reuters

This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.

Read more
Free, six days a week

Daily AI Brief: the AI news that matters, in your inbox.