Researchers examining challenges of incorrect financial advice provided by AI
A Keele Business School lecturer has embarked on a new study with colleagues at the universities of Warwick and Coventry to help the financial sector adapt to the challenges posed by AI and to study the safe and responsible deployment of AI for financial advice.
Dr Geetika Jain, Assistant Professor in Digital Transformation in Keele Business School, is part of a team led by Warwick Management Group’s Dr Anita Khadka and Professor Carsten Maple, investigating how AI systems such as large language models (LLMs) like ChatGPT might influence the financial services sector, and the potentially negative consequences of this influence.
Dr Jain and her colleagues on the “RAISE-Fin” project, funded by UKFin+/EPSRC, are particularly interested in AI “hallucinations”, where LLMs provide incorrect or inaccurate responses to users which could include incorrect financial advice.
This hallucinated information has the potential to mislead consumers, disrupt markets or spread false information, which the researchers say could lead to major economic impacts by damaging financial institutions’ reputation, or trust among their customers.
The researchers aim to change this by identifying the areas where AI systems might fail or potentially could be manipulated, and developing tools to help companies assess how reliable and accurate an AI-generated output might be.
Their project will also look at the broader issue of financial regulations in the UK, EU, and USA to evaluate how effectively they address these emerging AI-driven challenges in finance.
Their ultimate goal is to produce clear guidance, best practices and tailored auditing tools to help financial institutions and regulators, which will help enhance the safe, ethical, and innovative deployment of AI in financial services.
Dr Jain said, “The project provides a crucial framework for navigating the immense opportunities and risks of advanced AI in the FinTech sector. It moves beyond theoretical discussion to offer actionable policy pathways and risk mitigation strategies.
“The project outputs will directly inform both regulators and financial institutions, helping to shape a safer and more trustworthy financial services ecosystem grounded in responsible AI deployment.”
Dr Anita Khadka, Assistant Professor in Trustworthy and Responsible AI at Warwick Management Group (WMG) and Principal Investigator for RAISE-Fin, said: “As generative AI becomes increasingly embedded in financial services, ensuring the reliability, auditability, and regulatory alignment of these systems is critical. This project brings together technical, legal, and business expertise to address these urgent challenges.”
Professor Carsten Maple, Professor of Cyber Systems Engineering, WMG added: "This project is part of our programme of work in trustworthy AI being undertaken by the Secure Cyber Systems Research Group here at WMG. Our aim is to deliver world-class research with direct pathways to impact, that allows the confident and responsible design and deployment of AI."
