Artificial intelligence detects respiratory failure by analyzing audio

Characterized by a low blood oxygenation rate, respiratory failure is a condition that often requires hospitalization and can have a variety of causes, including asthma, heart or lung disease and viral infections. With the aim of assisting in patient screening, Brazilian researchers developed a tool based on artificial intelligence (AI) capable of detecting the problem through the analysis of speech audio. The idea is that, in the future, the methodology can integrate telemedicine systems to continuously monitor hospitalized people.

“Such tools would make it possible to identify the condition in patients with COVID-19, for example, by verbalizing a sentence on a cell phone”, explains the researcher Marcelo Matheus Gauy , scholarship holder FAPESP post-doctorate at the Institute of Mathematics and Statistics of the University of São Paulo (IME-USP).

The research was supervised by Marcelo Finger, professor in the Department of Computer Science at USP and coordinator of the project System for Early Detection of Respiratory Failure through Audio Analysis (Spira ), which uses artificial intelligence to compare people's audio and was initially designed to automatically recognize voice variations in COVID-19 patients during the height of the pandemic.

The results were published as a chapter in the publication Lecture Notes in Computer Scienceresulting from the 21st International Conference on Artificial Intelligence in Medicine (Aime), held in Slovenia in 2023. Researchers from USP and the Universidade Estadual Paulista (Unesp) on the Marília, São José do Rio Preto and São Paulo campuses collaborated.

Data collect

The development of the tool began by collecting data from people diagnosed with COVID-19 during the first phase of the pandemic. The group collected voices from approximately 200 patients from two partner hospitals in the project – the Hospital das Clínicas of the Faculty of Medicine and the Hospital Universitário (HU), both from USP – and trained modern AI models that achieved more than 95% accuracy, showing the viability of the system at that time.

Three types of audio were analyzed: a sentence that naturally induces pauses, a children's song with predetermined pauses and the sustained vowel 'a'. Pauses at unnatural times allowed the identification of a pattern analyzed by AI.

“During the height of the pandemic, we collected audio from patients suffering from respiratory failure in hospitals. We also collect audio from healthy people [controle] through an internet application. Artificial intelligence models achieved an accuracy of around 95% in detecting respiratory failure, while models that only use the differences in pause structures in the audios of patients and controls achieved an accuracy of around 87%”, explains Gauy.

In a second stage, data from people who developed respiratory failure due to various causes were included.

One of the findings of the study sequence was that, when expanding the audio collection beyond COVID-19 cases, it was necessary to change the tool, as the accuracy results were below 50%. This showed that models trained for COVID do not fit for other causes, indicating that in the case of coronavirus there are specific characteristics.

“Artificial intelligence models used for this task need to be carefully trained on a database robust enough so that the different sources of respiratory failure do not end up biasing the result,” says Gauy. On the other hand, these findings give hope that it will be possible to identify, in the future, not only respiratory failure through audio, but also its cause.

The article Discriminant audio properties in deep learning based respiratory insufficiency detection in Brazilian Portuguese can be read at: https://link.springer.com/chapter/10.1007/978-3-031-34344-5_32 .