Back in the 1940s, a Dutch physician called John Poel invented a device that changed the nature of heart monitoring. His device is based on the idea that the color of skin changes according to the amount of blood it contains. That’s easy to see in the way people quickly become red faced when exerting themselves,
But it turns out more subtle color changes occur at rest as the heart pumps blood around the body. The ebb and flow of blood through capillary vessels changes the amount of light reflected from the skin. Poel’s device beamed light into the skin and monitored changes in the amount reflected to determine a person’s heart rate. He called the technique photoplethysmography and it has since become a standard in medical settings the world over, with cheap blood oximeters that clip onto a finger now widely available over the counter.
But in recent years, physicians have begun developing remote techniques to measure heart rate—analyzing video recordings of a person’s skin to detect the tiny color changes as the heart beats. In this way, they have learned to measure heart rate in a wide range of lighting conditions using relatively cheap cameras with images taken from a distance.
But an important question is how far this remote technique can go to providing ubiquitous heart rate monitoring.
Heart Beats
Now we have an answer thanks to the work of Ming-Zher Poh at Google Research and colleagues who have developed a deep-learning system that remotely monitors heart rate using a smartphone’s front-facing camera. Their method is an updated version of Poel’s called remote photoplethysmography, which measures changes in blood volume by analyzing subtle variations in facial skin color captured through video.
Their innovation could democratize heart health tracking by making it available to billions worldwide without requiring additional hardware or manual effort. “To our knowledge, this is the first demonstration that smartphones can be used to passively monitor both HR and RHR during normal phone usage in the real-world,” say Poh and his colleagues.
Resting heart rate is a key biomarker of cardiovascular health and overall well-being. A higher rate has long been linked to increased risks of heart disease and stroke. Various studies have shown that persistent high rates can indicate declining cardiovascular fitness or the onset of underlying health conditions.
Traditionally, measuring resting heart rate has required individuals to remain still in a controlled environment, such as lying down or sitting quietly. While wearable devices have made continuous HR monitoring more accessible, adoption remains limited, particularly among populations at greater risk of heart disease. This is where the ubiquity of smartphones presents an unprecedented opportunity.
Poh and co’s Passive Heart Rate Monitoring system captures facial video during regular smartphone use, such as unlocking the screen. The system processes short, 8-second video clips to estimate heart rate in real time. To improve its accuracy, the system aggregates multiple measurements collected throughout the day. The result is a resting heart rate measurement that is comparable in accuracy to dedicated wearable devices.
One of the major breakthroughs of this study is its comprehensive validation process. Unlike previous remote photoplethysmography studies, which were limited to controlled environments and small datasets, the researchers tested their system on an unprecedented scale, using 225,773 videos from 495 participants and validating it on 185,970 videos from 205 participants. Testing in real-world conditions involved 101 participants using their own smartphones across a diverse range of lighting conditions, skin tones, and motion scenarios.
A critical challenge is to ensure its accuracy across different skin tones. Melanin absorbs more light, which can reduce the signal quality for darker-skinned individuals. Previous studies have shown significant declines in accuracy for those with darker complexions. To address this, the researchers intentionally overrepresented participants with darker skin tones and ensured their model was trained on a balanced dataset. This represents a major step forward in ensuring that smartphone-based health technologies are inclusive and effective for all users.
The team benchmarked the smartphone-based system against measurements from traditional devices such as electrocardiographs and bespoke wearable devices. The results were comparable and higher resting heart rate readings also correlated with higher body mass index and lower cardiovascular fitness. That reinforces the system’s potential as a meaningful health-monitoring tool.
Poh and colleagues also underscore the broader health implications of their system, noting that “this opens up the possibility of automatically collecting resting heart rate data across weeks, months, seasons, and years that may provide valuable health information.”
While this technology is highly promising, it is not without limitations. For example, measurement success rates were lower in real-world conditions compared to controlled laboratory settings, as body motion and extreme lighting conditions sometimes led to unreliable readings.
To improve performance, the researchers suggest implementing motion compensation algorithms, optimizing camera exposure settings to enhance signal quality particularly under incandescent lighting and increasing the number of measurement attempts when the signal is poor. Battery consumption considerations will also be crucial for real-world implementation. Future iterations may focus on optimizing measurement frequency to balance data collection with energy efficiency.
Given the sensitive nature of health data and facial video capture, privacy remains a major concern. The researchers emphasize that their system was designed with privacy in mind. Video processing is performed on-device, ensuring that sensitive biometric data does not leave the user’s smartphone. HR measurements are only initiated when the user unlocks their screen, avoiding unauthorized recordings.
Healthy Futures
Future implementations could integrate secure enclaves to protect data within a smartphone’s trusted execution environment. Users would also need to provide explicit consent before enabling passive HR monitoring, ensuring transparency and control over their personal health data.
That’s interesting work with potential applications that extend beyond individual health tracking. Widespread adoption could enable large-scale heart health studies and population-level monitoring, supporting public health initiatives and remote monitoring programs. Longitudinal heart rate data could help identify early signs of cardiovascular issues, enabling proactive intervention and early disease detection.
From a lifestyle and wellness perspective, passive tracking could help users monitor changes in cardiovascular fitness over time without the need for wearables, offering valuable insights into fitness progress and recovery patterns. “These results highlight the potential of smartphones to enable passive and equitable heart health monitoring,” say the team.
The advent of smartphone-based passive heart rate monitoring could initiate an important change in the way we monitor heart health. How long before remote photoplethysmography comes to a phone near you? John Poel would surely be amazed.
Ref: Passive Heart Rate Monitoring During Smartphone Use in Everyday Life: arxiv.org/abs/2503.03783v2
English