Bloomberg Research Highlights Risks of RAG LLMs in Finance
Bloomberg researchers have published two new academic papers that highlight the risks associated with retrieval-augmented generation (RAG) large language models (LLMs) in financial services. The findings, announced in a press release, suggest that RAG-based LLMs, which integrate external data to enhance accuracy, may actually be less safe than their non-RAG counterparts.
In the paper titled "RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models," Bloomberg's AI team assessed the safety profiles of 11 popular LLMs using over 5,000 harmful questions. The study found that RAG settings led to increased unsafe responses, even in models that were previously considered safe.
A second paper, "Understanding and Mitigating Risks of Generative AI in Financial Services," introduces a new AI content risk taxonomy tailored to the financial sector. This taxonomy addresses specific risks such as confidential disclosure and financial services misconduct, which are not covered by general-purpose safety frameworks.
These findings underscore the importance of evaluating the safety of RAG-based systems, especially in high-stakes domains like finance, and suggest the need for additional safeguards to mitigate potential vulnerabilities.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Finance AI Weekly, AI Policy Brief or Daily AI Brief.
Also, consider following us on social media:
More from Finance
Nov 19 Rogo nears $350 million round at a $3.75 billion valuation Nov 19 Arizona opens criminal investigation into OpenAI over what it calls rogue AI agents in banks Nov 19 Citi tells customers they are on their own if third-party AI agents misuse their accounts Nov 18 Klarna launches an AI concierge that books, shops and drafts emails Nov 18 Singapore sets up sandbox for AI agent payments under new national AI missions for financeMore from AI Safety
Nov 19 US Catholic bishops form AI task force and push for guardrails and worker protections Nov 18 Nvidia scandal complicates US-China AI talks ahead of Trump-Xi meeting, Wired reports Nov 18 Beijing AI safety group Concordia AI briefed OpenAI staff on China's approach to AI risk Nov 18 OpenAI details how it contains frontier models, including a pause after a DNS exploit Nov 18 Coalition including Hugging Face and xAI co-founders says Anthropic "is not a safe AI company"Finance AI Weekly
Weekly newsletter about AI in finance. Covers AI-driven trading, fintech innovations, and data analytics transforming markets
Industry analysis
2025 Global Business Services Agenda: Gen AI Takes Center Stage
This industry analysis by The Hackett Group explores the transformative impact of generative artificial intelligence (Gen AI) on global business services (GBS) in 2025. The study highlights the shift from exploration to acceleration of Gen AI initiatives, with 89% of executives advancing these projects to improve customer satisfaction, innovate products, and reduce costs. The report also discusses the challenges and strategies for successful Gen AI adoption, emphasizing the need for a technology-enabled operating model and the importance of reskilling the workforce.
Read moreYou may also like
Wikimedia Backs Fair Use for AI Training but Not RAG
New York Bill Would Let State Limit Compute for Risky AI Models
AI tops the risks to US financial stability in Federal Reserve survey of market contacts
FraudBench Finds Model Fraud Rates Rise With Autonomy
BCG: 73% of major banks allow AI agents in production, but only 8% fully control them
Daily AI Brief: the AI news that matters, in your inbox.