Blood oxygen saturation estimation from face videos using transformers

Abstract

Blood oxygen saturation (SpO2) is a vital physiological marker for assessing respiratory and cardiovascular health. While traditional pulse oximetry systems are widely used, their reliance on direct skin contact limits their practicality in remote or contactless environments. The remote photoplethysmography (rPPG) offers a compelling non-contact alternative by leveraging facial videos captured with RGB cameras to estimate SpO2. However, most existing rPPG-based methods depend on handcrafted features and conventional machine learning models, which are inadequate for capturing the intricate temporal and global signal patterns required for accurate estimation. Moreover, the lack of dataset diversity, especially the under-representation of individuals with darker skin tones, further impairs the generalizability and fairness of these systems. To address these limitations, we propose SPECTRA, a novel transformer based system for SpO2 estimation from rPPG signals. SPECTRA is the first in this domain to utilize transformers, enabling the modeling of both temporal dynamics and global patterns more effectively. The system enhances feature learning using supervised contrastive learning and incorporates all possible combinations of red, green, and blue channel Ratio of Ratios (RoRs) to better capture variations in light absorption across different skin tones. A quality-weighted consolidation strategy is introduced to prioritize less noisy RoRs, thereby improving the robustness of the SpO2 estimation. In addition, we present a new large-scale rPPG dataset comprising subjects with diverse skin tones, addressing the fairness concerns prevalent in current datasets. SPECTRA demonstrates consistent and superior performance across this dataset as well as on two publicly available benchmarks, PURE and MSPM, validating the strength of its architectural and algorithmic contributions.