Bloomberg Research Highlights Risks of RAG LLMs in Finance

Bloomberg researchers have published two new academic papers that highlight the risks associated with retrieval-augmented generation (RAG) large language models (LLMs) in financial services. The findings, announced in a press release, suggest that RAG-based LLMs, which integrate external data to enhance accuracy, may actually be less safe than their non-RAG counterparts.

In the paper titled "RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models," Bloomberg's AI team assessed the safety profiles of 11 popular LLMs using over 5,000 harmful questions. The study found that RAG settings led to increased unsafe responses, even in models that were previously considered safe.

A second paper, "Understanding and Mitigating Risks of Generative AI in Financial Services," introduces a new AI content risk taxonomy tailored to the financial sector. This taxonomy addresses specific risks such as confidential disclosure and financial services misconduct, which are not covered by general-purpose safety frameworks.

These findings underscore the importance of evaluating the safety of RAG-based systems, especially in high-stakes domains like finance, and suggest the need for additional safeguards to mitigate potential vulnerabilities.