Related manuscript: Plante TB, et al. "User experience of Instant Blood Pressure: Exploring reasons for the popularity of an inaccurate mobile health app." npj Digital Medicine. 2018. https://doi.org/10.1038/s41746-018-0039-z

The Food & Drug Administration (FDA) passively regulates the consumer medical device market and, until recently, this has proven to be an effective approach for keeping unsafe devices out of the hands of consumers. Given the high cost of bringing a device to market, no entrepreneur would risk skirting regulations only to bear the wrath of the FDA and have their product pulled from the market. For the most part, device developers played by the rules and demonstrated efficacy or equivalency before their device appeared on store shelves.

That is, until July 2008 when Apple opened its App Store. Steve Jobs called this opening “[The] biggest launch of my career.” For the first time, small coding teams could leverage advanced smartphone technology to deliver custom-designed software with instantaneous distribution to iPhone owners across the globe. Clearly, iPhones are not medical devices. However, apps that give medical functionality to consumer-grade hardware are medical devices, a designation known as Software as a Medical Device, or SaMD. One theoretical SaMD product might turn a smartwatch into a heart rate monitor and use algorithms to identify atrial fibrillation. Another might give a smartphone held against the chest the functionality of a stethoscope, transmitting heart sounds to the connected headphones. Suddenly, small groups or individual developers could release SaMD medical products at low cost. Because of the low costs for development and distribution of apps, the incentive to follow FDA guidelines and abide by their passive regulation is largely gone.

Consumer-targeting medical smartphone apps began to proliferate and now number in the hundreds of thousands of individual apps. Most of these mobile health apps are low-risk and are unlikely to harm consumers (think blood sugar diaries for diabetes). A small portion are higher-risk apps providing functionality to diagnose, monitor, or treat medical conditions. A subset of these high-risk devices are bad-actor “snake oil” apps with fantastical claims of functionality and a lack of clinical evidence supporting their use. Notorious examples of the functionality of snake oil apps that were once available for download include treating acne with light from an iPhone, identifying if a skin lesion was melanoma with a picture taken by the smartphone’s camera, and bipolar management protocols that recommended treating mania by drinking alcohol.

One of the top-selling apps of 2015 was Instant Blood Pressure, made by AuraLife. The app claimed that it could “measure blood pressure with just your iPhone… no cuff required.” By holding the microphone of the iPhone against the chest and placing your finger over the illuminated light and camera, it would produce a blood pressure reading. Obviously, this technology would revolutionize diagnosis and treatment of hypertension – if it was accurate.

Priced at around $5 per download, Instant Blood Pressure sold like hotcakes. It was downloaded >148,000 times and generated >$600,000 in revenue before it disappeared from the Apple App Store for unclear reasons in 2015. Like the majority of mobile health apps, Instant Blood Pressure had not undergone independent clinical assessment and had not cleared the FDA’s passive regulatory process. In a possible attempt to skirt FDA regulations, it carried disclaimers that it was for recreational purposes only and was not a medical device. (These disclaimers appeared to be ineffective, as user reviews document use of the app in managing serious medical conditions like hypertension, end-stage renal disease, postural orthostatic tachycardia syndrome, and post-heart transplant care.)

I was curious about the quality of an SaMD product that made claims of such functionality without FDA review or independent assessment. I worked with a group of hypertension experts to compare measurements from a validated automated blood pressure cuff to the app in 85 adults. Not surprisingly, the app was highly inaccurate in the measurement of BP when assessed in our clinical validation study, receiving the lowest possible British Hypertension Society accuracy grade. More importantly, it systematically underreported elevated measurements and missed hypertension (with the threshold of 140/90 mm Hg) nearly 80% of the time. Following the publication of our validation study results, the Federal Trade Commission (note: not the FDA) brought litigation against the company and its CEO, Ryan Archdeacon, who eventually settled for nearly $600,000, with payment suspended for lack of funds.

I was struck by how an inaccurate blood pressure-measuring product could be so popular with consumers. Fortunately, we had included some key features in the design of our validation study that gave us the opportunity to explore possible reasons for its popularity. First, we asked participants to guess their blood pressure before we took any measurements. Second, we developed a questionnaire to gauge domains that we thought might be related to user experience. Participants were prompted with statements akin to “I enjoyed this app”, “I think this app was accurate”, and “I’m more likely to monitor my blood pressure after using this app.” Participants indicated their responses to each statement on a traditional Likert scale, from “Strongly Disagree” to “Strongly Agree”. The timing of this survey was key: Participants indicated their responses after they had guessed their own blood pressure and had learned their blood pressure from the app but before they learned the result from our reference automated blood pressure cuff device.

Our validation research group had the opportunity to look into the reasons why users liked one very successful, inaccurate app. In the present analysis, we compared a participant’s self-estimation of their blood pressure to the result from the Instant Blood Pressure app. We classified participants as receiving a lower-than-expected blood pressure result from the app (i.e., the app’s blood pressure result was lower than what they had guessed), similar-to-expected blood pressure results, or higher-than-expected blood pressure results (i.e., the app’s result was higher than what the participants had guessed). From an accuracy perspective, the lower-than-expected and higher-than-expected groups experienced a similar degree of inaccuracy, just in different directions. It’s reasonable to assume that the average adult would classify lower-than-expected blood pressure in a doctor’s office as “good” and higher-than-expected blood pressure as “bad”.

We observed several key findings. First, perceived accuracy of the app was similar in all groups. Second, user enjoyment seemed higher if the app’s blood pressure results were lower-than-expected or similar-to-expected. Flipped around, higher-than-expected results seemed to adversely affect user experience. Third, participants reported that they were more likely to monitor their blood pressure if they received higher-than-expected or similar-to-expected results.

So why was Instant Blood Pressure so wildly popular even though it poorly performed its core functionality? Recall that the fundamental failure of this app is that it systematically underreports elevated blood pressure. While by no means definitive, our findings suggest that this defect may have improved user experience and motivated future blood pressure monitoring. People like being told that their blood pressure is “good”. This app did just that – if you have uncontrolled hypertension, you have only a 1 in 5 chance of having it tell you that it is in that range. It’s like a scale that tells you that you weigh 20 lbs less than you are. What’s not to love?

It turns out that there's a lot not to love. Hypertension is called the “silent killer” because it usually has no symptoms until it leads to a heart attack or stroke, which likely could have been prevented with lifestyle modifications and medications. We internists recommend that patients with hypertension monitor their blood pressure at home. Patients fiddle with their medications all of the time and given bad data, they may opt to stop a highly effective blood pressure regimen. Accurate consumer-grade blood pressure monitors are critical tools in the diagnosis and management of hypertension and in the prevention of cardiovascular disease. 

Our findings support what we have known throughout the existence of the FDA: We need regulation to ensure that medical devices are safe and effective. Apple and the FDA have made claims that they are improving their regulation of app marketplaces in order to improve safety and weed out apps like Instant Blood Pressure that claim to measure blood pressure without any supporting evidence. Yet, apps like the iCare Health Monitor that claim to measure blood pressure with just and iPhone – without any supporting clinical data – are still available for download on the Apple App Store today. Until regulation can catch up with the smartphone revolution, I am concerned that snake oil apps will continue to proliferate and well-meaning smartphone users will be at risk for unintended harm. 

Photo by Jason Howie, attribution 2.0 creative commons license.