XX ValgrAI Morning Research Meetings
The twentieth edition of the ValgrAI Foundation’s ‘Research Mornings’ is here! Next Friday, 7 November, at 9:30 a.m., we will celebrate twenty sessions of the Morning in the best possible way, with new presentations showcasing the latest innovations in Artificial Intelligence from the ValgrAI ecosystem. Come and celebrate with us!
Why can't you miss it?
-
Immerse yourself in the most innovative research in artificial intelligence.
-
Talk directly with experts during a question-and-answer session.
-
Expand your network by connecting with other technology enthusiasts.
-
Be inspired by new ideas and opportunities in the exciting world of AI.
Isabel Benlloch
Title: Retri-Eval, an evaluation method for retrieval-augmented generation modules.
The advent of large language models (LLMs) has driven the development of various proposals for evaluating their results. However, a major challenge in this regard has been the absence of standardised evaluation criteria, often compounded by concerns about their reliability. A practical example is the responses generated by RAG systems, which use a combination of information retrieval techniques and LLMs to improve the quality of the responses generated.
Despite the existence of numerous evaluation frameworks for RAG systems, contemporary RAG practices focus predominantly on evaluating the response based on the language model. Furthermore, the metrics used to evaluate the retriever’s performance have often been limited to quantitative metrics such as accuracy and recall, which may be inadequate in certain cases. This approach raises significant reliability issues, given that RAG systems rely primarily on retrieved documents to substantiate their responses. To address this problem, a new framework for evaluating the retrieval module of RAG systems is proposed: a multi-agent system specialised in evaluating the retrieval module. Specifically, a set of four LLM-based agents is integrated into a cooperative system to analyse. Specifically, a set of four LLMs-based agents is integrated into a cooperative system to analyse the retrieved documents together with the user query.
Olaf Meneses
