The VertexLit group from ValgrAI and VRAIN at the UPV is developing a pioneering project that restores a person’s voice, demonstrating the humanising power of AI and the need to raise awareness of this group.
The project has managed to reconstruct the voice of Fran Vivó, a resident of Benaguasil affected by ALS, with just 20 minutes of audio in Spanish and Valencian, his native language.
When the voice is silenced, the silence weighs heavily upon us. It conceals emotions, memories, and nuances that are integral to each person. Fran Vivó, a resident of Benaguasil affected by amyotrophic lateral sclerosis (ALS), a disease that suddenly robbed him of the ability to speak with his own voice, knows this silence well. Today, thanks to artificial intelligence, he has regained it.
The VertexLit research group, led by Jordi Linares, has presented a project that exemplifies the transformative power of artificial intelligence applied to people’s well-being. This project came to ValgrAI through documentary filmmaker Alex Badia and researcher Gema Piñero from the UPV’s iTEAM Institute. The researcher who volunteered to work on recovering Fran’s voice was Jordi Linares, a member of the Joint Research Unit of the Valencian Graduate School and Research Network of Artificial Intelligence (ValgrAI) and the Valencian University Institute for Research in Artificial Intelligence (VRAIN) at the Polytechnic University of Valencia (UPV).
The team faced a major challenge: reconstructing Fran Vivó’s voice with just 20 minutes of audio in Spanish and Valencian, his native language. Using neural networks trained to work with limited corpora —especially in Valencian, where resources are scarce— the researchers analysed his vocal dynamics, incorporated emotional modulations, and created an adaptive model. All with the aim of offering a language full of human traits and with the tones typical of the Benaguasil area, rather than a robotic voice.
The audio the team had to work with consisted of voice messages extracted from WhatsApp, but this was scarce material, since Fran lost his ability to speak when the audio system in this messaging app was just beginning to be used.
An AI system that has managed to recover Fran Vivó’s tone of voice and identity
The project developed for Fran Vivó transcends technology: the faithful recreation of a person’s voice, using an AI-based cloning system that has managed to recover his tone of voice, his prosody, his identity. And ultimately, his unique way of saying things, thus replacing the robotic voice of an eye-controlled communication application with his own voice.
The family has played a fundamental role in this process, helping to construct speech and enabling the emotional charge, intonation, and intention of the message to be adjusted using an editor that respects Fran’s vocal identity. In addition, the tool is editable so that they can generate the text, given that they understand what he want to say. The technology only provides the voice: the emotion is still his.
This project has not only allowed Fran Vivó to regain his identity, but also opened up a range of possibilities for ALS patients, helping them to recover who they are and have hope in life, as Fran himself acknowledges.
The humanising potential of AI
The VertexLit group, which has carried out this project on a completely altruistic basis, aims to show the humanising potential of AI, as it can restore lost abilities and dignify the lives of people living in extreme situations. This project does not seek to stand out for its technical complexity, but rather for its human impact.
Likewise, Jordi Linares, director of the VertexLit group, affirms that the project seeks to give visibility to people affected by ALS, since ALS continues to be an invisible disease in social and media discourse, despite the immense suffering it causes patients and families, so that ‘this voice is not only for Fran. It is for all of them,’ he assures.
As explained by Vicent Botti, director of VRAIN and general director of ValgrAI at the UPV, who opened the conference where this development was presented, ‘this is not an experiment or a demonstration. It is an ethical promise: science and technology must be put at the service of those who need the most support, and just as Fran has regained his voice, thousands of other voices will also be able to do so.’
The presentation of this pioneering project by the VertexLit research group took place within the framework of the 2nd Conference of the Valencian Institute for Artificial Intelligence Research (VRAIN) at the Higher Polytechnic School of Alcoy. At the conference, researchers from different groups at the institute also presented advances in artificial intelligence in fields such as automatic subtitling, detection and extraction of web reviews, automated detection of microplastics, and security in virtual and hybrid environment interactions.
