SHINE: Synergizing transformers with contrastive learning for thriving rPPG-based SpO2 estimation

Abstract

Blood oxygen saturation (SpO<inf>2</inf>) is a critical physiological indicator for assessing respiratory and cardiovascular health. Conventional pulse oximetry systems, while widely used, require direct skin contact, thereby limiting remote applications. Remote photoplethysmography (rPPG) offers a non-contact alternative by using RGB cameras to estimate SpO<inf>2</inf> from facial videos. Despite its potential, current rPPG-based SpO<inf>2</inf> systems often depend on hand-crafted features and traditional machine learning, limiting their ability to capture complex temporal patterns. These systems also struggle with generalizability due to the lack of diverse datasets, particularly those representing darker skin tones. To address these challenges, we introduce SHINE, a novel transformer-inspired system for non-contact SpO<inf>2</inf> estimation from rPPG signals. SHINE is the first system in this domain to leverage transformers, enabling it to model temporal dynamics and global patterns more effectively. It further enhances feature learning through supervised contrastive learning and incorporates all combinations of red, green, and blue channel ratios of ratios (RoRs), accounting for skin tone differences in light absorption. Additionally, it utilizes a quality-weighted consolidation strategy that prioritizes less noisy RoRs, ensuring more reliable SpO<inf>2</inf> estimation. We also present a new large-scale rPPG dataset, including subjects with diverse skin tones, helping bridge the fairness gap in rPPG-based SpO<inf>2</inf> estimation. SHINE consistently outperforms existing systems on our proposed dataset, as well as on two publicly available datasets, PURE and MSPM, demonstrating the effectiveness and importance of each of its components. Upon acceptance, our dataset will be made publicly available to foster fairness and advance future research in this field. © 2025 Elsevier B.V., All rights reserved.