Why Is Femtech Software Unreliable? + 3 Strategies For Improvement

By Bethany Corbin, founder and CEO, FemInnovation

The female health technology (femtech) industry is poised for substantial growth. With more than 1,800 startups and a projected market value exceeding $97 billion by 2030,¹ femtech is redefining the landscape of women’s health. The goals of the femtech industry are two-fold: (1) derive meaningful and medically valuable insights about women’s health (which has historically been neglected from modern medicine) and (2) empower women to understand their bodies and advocate for personalized health solutions.

Since the term was coined in 2016, femtech has made significant progress toward creating a common language for the discussion of women’s health and bringing women’s health to the forefront of medical attention. Femtech startups raised more than $1.16 billion in funding last year and captured 13.2% of all digital health funding in 2022.² Femtech applications currently rank as the second most popular software for female adults.³ Given that women comprise 50% of the world’s population and account for 80% of consumer healthcare purchasing decisions,⁴ it is easy to understand the allure of this market to innovators and investors. However, as the femtech market expands, low regulatory hurdles and race-to-market strategies have made it easy for some femtech companies to develop inaccurate and untested products with the potential to harm consumers. This article describes the harmful inaccuracies that exist in some femtech products and proposes strategies for enhancing clinical relevance of femtech data and products.

Regulatory Oversight Of Femtech Devices

The problem of faulty device design for certain femtech products stems, in part, from the relatively relaxed regulatory framework that applies to many femtech inventions. The regulation of femtech devices in the United States is based on risk and, in turn, classification. The FDA classifies devices based on their description, intended use, and patient risk profile, with devices falling into one of three categories: Class I, Class II, or Class III.⁵ When the device’s risk to patients is deemed low, the FDA may exercise enforcement discretion and elect to forgo enforcing certain regulatory requirements on certain medical devices. In particular, the FDA has stated that it only intends to apply its regulatory oversight to software functions and medical devices whose functionality could pose a risk to patient safety if the device were to malfunction.⁶

The majority (52.8%) of femtech companies offer consumer products, devices, applications, and software.¹ Many of these products involve the use of software, including algorithms and predictive analytics, to collect and interpret data and provide health insights. Overwhelmingly, these software products exist in the reproductive health space — e.g., period-tracking apps, ovulation and fertility apps, pregnancy symptom trackers, etc. — and are subject to FDA enforcement discretion due to the low risks they present to consumers. Further, apps and software that are merely intended for (1) general patient education, (2) allowing individuals to log, record, evaluate, or make decisions related to general health and wellness, or (3) facilitating patient access to common reference information, are not medical devices and are not subject to FDA oversight.⁷

This means that a significant number of software-based femtech devices are not reviewed and regulated by the FDA.⁸ As such, they are not subject to safety and efficacy requirements, are not required to undergo clinical testing or trials, and do not have to prove accuracy. The result is that some femtech products on the market have no scientific backing for their claims or don’t deliver as promised.

The Problem For Consumers

Given that many femtech devices are not actively regulated by the FDA and are not required to satisfy any threshold accuracy standards, it is no surprise that these devices vary widely in terms of accuracy and usability. Repeated studies have found that most untested and unverified femtech applications do not live up to their promises and are inaccurate to the point of being medically unreliable. While the goals of the femtech industry are laudable, inaccurate devices have the potential to harm women and may cause consumers to lose trust in an industry that could one day revolutionize their care.

For example, the Organization for the Review of Care and Health Apps (ORCHA) examined hundreds of the femtech industry’s most downloaded apps and found that more than 85% of the apps did not meet ORCHA’s quality threshold and had significant clinical assurance concerns.⁹ A 2016 study by researchers at Columbia University Medical Center similarly found that more than 95% of the 108 free smartphone menstrual cycle apps tested were inaccurate, with most apps having no scientific grounding or consultation with medical professionals.¹⁰ In a separate 2018 study conducted by Current Research and Opinion, the reviewers examined 73 menstrual cycle apps and found that none could correctly predict ovulation.¹¹ The best app was only 21% accurate.¹¹

The prevalence of inaccurate femtech devices is alarming for an industry that purports to promote women’s health, particularly when consumers are generally unable to evaluate an app’s methodology and proprietary algorithms to determine whether the device is accurate and effective. As more women turn to femtech devices to understand and monitor their health, inaccurate devices can result in missed fertility periods, delayed family planning efforts, improper triaging, unnecessary communications and visits with providers, unintended pregnancies due to inaccurate menstrual cycle predictions, and much more.

Inaccurate devices further operate to disempower the female consumer base by providing a false sense of health monitoring, information, and control in exchange for extremely sensitive health data. Once the femtech company has access to a consumer’s data, it may disclose that data to third parties or sell it downstream for profit. Many women willingly exchange access to their most intimate data – particularly reproductive health data – with the understanding that they will receive the benefits of health monitoring and tracking. However, with inaccurate devices, women relinquish control of their data in exchange for clinically meaningless predictions. This serves to further disempower women in modern medicine.

The Problem For Clinicians

In addition to consumer harm, inaccurate femtech devices can have unexpected impacts on clinicians. As wearables and apps drive consumer data collection, clinicians are often presented with unverifiable data from unproven technologies. While many patients would like this data incorporated into their care plan, clinicians struggle to determine the value such data should be afforded, particularly when there is frequently a lack of medical involvement and oversight during the device’s creation.

Indeed, more than seven out of 10 clinicians admitted having access to more data than they can handle and analyze.¹² Clinicians have similarly acknowledged that they are unsure how to handle and incorporate such data into the patient’s medical record and care plan,¹² and that they are intimidated by the amount of data health technology devices generate.¹³ In a separate study, 94% of clinicians indicated they felt overwhelmed by the amount of useless data from apps and wearables and that they questioned the reliability of such data.¹³ The inability to trust femtech data is one key reason why many clinicians do not rely on or use femtech devices in their own practices.

Unless and until femtech devices can demonstrate clinical accuracy, they will have limited use and relevance within the regulated practice of medicine. The devices will continue to serve as consumer tracking tools, but their impact on the future of women’s health will be undeniably limited. Accordingly, it is crucial that the femtech sector produce data and insights that are medically relevant and usable by clinicians to accomplish effective change for women’s health.

3 Proposed Solutions To Enhance Accuracy

Remedying femtech’s inaccuracy record should be a top priority for the industry to accomplish the underlying goals driving women’s health innovation. New and existing femtech companies can take steps to prioritize accuracy and promote clinician involvement in product design and development. Let’s examine three strategies femtech device manufacturers can implement to help enhance product reliability and usability.

1. Use Scientifically Proven and Medically Tested Methodologies to Drive Device Design

It is no secret that femtech devices and applications consist of hundreds to thousands of design decisions, including which data to collect, how to quantify that data, whether certain data inputs should be defined or restricted, and how data should be interpreted. Each decision during the design process can have a significant impact on the device’s functionality, usability, and accuracy. Despite this, few femtech devices cite any medical, scientific, or health literature.³ What’s more, femtech companies commonly use debunked science when determining the data inputs for their devices.

For example, more than half of the period-tracking apps on the market use a simplistic calendar calculating methodology, which assumes that every menstruating person has a 28-day cycle, and that ovulation occurs 14 days before the beginning of the next menstrual cycle.¹⁴ However, scientific research has proven that only 13% of women have period cycles that last 28 days and that calendar calculating methods are not reliable for this purpose.¹⁴ Further, most menstrual apps are not properly configured to handle missed or irregular periods — which can occur frequently for women — further diminishing their accuracy for a large subset of the female population.

Similarly, a 2018 study published by Frontiers in Public Health found that calendar-based and calculo-thermal apps were inadequately reliable in predicting fertility and ovulation.¹⁵ These apps failed to account for month-to-month variations in menstrual cycle and ovulation days. Almost all the apps studied provided predictions that were inaccurate by more than a couple of days. With respect to ovulation, a 2000 study found that only 30% of women had fertile windows falling within the 10^th to 17^th day average, which made ovulation prediction based on tracking and algorithms exceptionally unreliable.¹⁶ Yet, despite this disproven methodology, hundreds of femtech fertility apps on the market today still adopt this framework as the basis for their algorithmic predictions.

To derive meaningful data and provide accurate health predictions, femtech manufacturers must ensure that their devices and algorithms are based on verifiable scientific methodologies and evidence. Before developing a femtech device or app, manufacturers must understand the underlying scientific data for the problem they are trying to remedy. Established medicine should be used to determine which data inputs the app must collect and should be used to train the algorithms to provide more accurate predictions. While this endeavor can be more challenging in women’s health than in other health fields given the dearth of data on some female health conditions, the gender data gap is finally starting to narrow. It is essential for femtech manufacturers to keep abreast of changing medical and scientific knowledge and update their apps and devices to ensure symmetry with clinical standards of care. Relying on outdated and disproven methodologies harms women for the sake of corporate profit.

2. Incorporate Clinician Oversight into the Design Process

To enhance clinical accuracy and usability, femtech companies should consult licensed clinicians and other experts during the device design process. Clinicians and medical experts not only possess the appropriate knowledge base to help manufacturers understand existing medical literature, but they also regularly interact with patients and understand how the consumer base will use and respond to the product.¹⁷ Incorporating clinical expertise can enhance accuracy rates for femtech devices and ensure that the device manufacturers remain aware of any new developments in accepted scientific methodology for the products they create.

In addition, clinician involvement in device design and development can help other medical professionals feel more confident using the device with their patients or incorporating data from the device into the patient’s medical record. Clinician involvement can add legitimacy to the device and help assure providers that the device is based on sound medical and scientific literature. This can help bridge the gap between tech developers and clinicians and allow femtech data to acquire clinical relevance. Clinicians should, at a minimum, be involved during the device’s design and development phases and during the verification and validation phases.

3. Train Algorithms on Representative Test Data

Finally, when vetting new femtech products, devices, and features, it is important to use data representative of the target population. Many device manufacturers fail to prioritize diversity and inclusivity during product testing, which can result in improperly trained algorithms. It is well known that health data can vary by race and ethnicity; for example, the average age of menopause for an Indian woman is 46.2 years compared to 51 years for a Caucasian woman.¹⁸ However, if a diverse test group is not consulted, the device may fail to provide accurate predictions and insights for certain racial and ethnic groups. Further, the design features and interfaces could unknowingly be offensive to the excluded groups. Diverse test data is essential for femtech device accuracy.

Incorporating a wide variety of test users can also help promote user-centric controls and enable testers to identify instances where data inputs are too narrow or responses are inappropriate. For example, when Fitbit first released its menstrual tracking feature in 2018, the company limited the number of days a woman could input for her menstrual cycle to 10.¹⁰ This meant women who bled longer than 10 days could not input the full range of their data. Had this functionality been adequately tested with a diverse audience, Fitbit likely would have discovered the irrational data entry limit before the feature was released. Instead, Fitbit experienced backlash over the limiting design feature. Thus, diverse and inclusive consumer testing is critical to determining design features that may unintentionally impede device accuracy.

Conclusion

As the femtech industry continues to grow, its devices and apps must prioritize accuracy. Meaningful change to women’s health can be achieved, but it will require device manufacturers to hold themselves accountable for accuracy and usability in a way that is not yet regulatorily mandated. Failure to ensure accuracy can diminish women’s trust in the femtech industry and harm women who rely on these devices when making clinical judgments.

References:

1. FemTech Analytics, FemTech Industry Landscape Q4 2022 Teaser, DKV Analytics (Dec. 2022), https://analytics.dkv.global/femtech-industry-landscape-q4-2022/teaser.pdf.
2. Dominic-Madori Davis, Despite 2022’s Headwinds, Women’s Health Startups Did Better than Ever Before, TechCrunch (Jan. 24, 2023), https://techcrunch.com/2023/01/24/despite-2022s-headwinds-womens-health-startups-did-better-than-ever-before/.
3. Naomi Jacobs & Jenneke Evers, Ethical Perspectives on Femtech: Moving from Concerns to Capability-Sensitive Designs, Bioethics (Feb. 20, 2023), https://onlinelibrary.wiley.com/doi/full/10.1111/bioe.13148
4. McKinsey & Company, Unlocking Opportunities in Women’s Healthcare (Feb. 14, 2022), https://www.mckinsey.com/industries/healthcare/our-insights/unlocking-opportunities-in-womens-healthcare#.
5. Food & Drug Administration, Classify Your Medical Device, https://www.fda.gov/medical-devices/overview-device-regulation/classify-your-medical-device#:~:text=Class%20I%20includes%20devices%20with,I%2C%20II%2C%20and%20III (last updated Feb. 7, 2020).
6. Food & Drug Administration, Policy for Device Software Functions and Mobile Medical Applications: Guidance for Industry and Food and Drug Administration Staff (Sept. 28, 2022), https://www.fda.gov/media/80958/download.
7. Food & Drug Administration, Examples of Software Functions that are Not Medical Devices, https://www.fda.gov/medical-devices/device-software-functions-including-mobile-medical-applications/examples-software-functions-are-not-medical-devices#:~:text=This%20list%20provides%20examples%20of,FDA%20does%20not%20regulate
   %20them (last updated Sept. 29, 2022).
8. Genevieve Grabman & Cara Tenenbaum, FDA Regulation Must Uphold Women’s Health, 77 Food & Drug Law J. 318, 333 (2022), https://www.fdli.org/wp-content/uploads/2022/12/Grabman-Tenenbaum-FDLJ-77-3.pdf.
9. Femtech Apps Rating Map, ORCHA (July 24, 2019), https://orchahealth.com/femtech-apps-rating-map/.
10. Does Digital Health Technology Know Women?, Med. Futurist (Feb. 21, 2019), https://medicalfuturist.com/femtech-womens-health/.
11. Zoe LaRock, ‘Femtech’ Companies are Likely Poised for Speedy Growth – Despite Failing to Prove that Their Tools Live up to the Hype, Insider (July 22, 2019), https://www.businessinsider.com/femtech-space-booms-despite-tepid-efficacy-2019-7.
12. Ben Leonard et al., Buried in Data: A Doctor’s Lament, Politico (Feb. 8, 2023), https://www.politico.com/newsletters/future-pulse/2023/02/08/buried-in-data-a-doctors-lament-00081673.
13. Evan Sweeney, Data Overload, Access and Affordability Limit Patient Monitoring Technology, Fierce Healthcare (June 1, 2017), https://www.fiercehealthcare.com/mobile/data-overload-access-and-affordability-limit-home-health-monitoring.
14. Pragya Agarwal, “Femtech” is Booming – But Does it Really Make Healthcare More Equal?, Prospect (Sept. 5, 2021), https://www.prospectmagazine.co.uk/science-and-technology/femtech-is-booming-but-does-it-really-make-healthcare-more-equal.
15. Alexander Freis et al., Plausibility of Menstrual Cycle Apps Claiming to Support Conception, Frontiers Pub. Health (Apr. 2018).
16. Jenny McGrath, With Period-Tracking Apps, the Fate of Your Fertility is Far from Clear, Digital Trends (Sept. 2, 2019), https://www.digitaltrends.com/mobile/the-problems-and-promises-of-period-tracking-apps/.
17. Saba Akbar et al., Safety Concerns with Consumer-Facing Mobile Health Applications and Their Consequences: A Scoping Review, 27 J. of American Med. Informatics Ass’n, 330-340 (Feb. 2020), https://academic.oup.com/jamia/article/27/2/330/5585394
18. Ahuja Maninder, Age of Menopause and Determinants of Menopause Age: A PAN India Survey by IMS, 7 J. of Mid-Life Health 126-131 (2016), https://journals.lww.com/jomh/Fulltext/2016/07030/Age_of_menopause_and_determinants
    _of_menopause.6.aspx#:~:text=Average%20age%20of%20menopause%20of,married%20status%
    2C%20and%20parity%20status.

About the Author:

Bethany Corbin, founder and CEO of FemInnovation and Women’s Health Innovation Consulting, is a healthcare attorney dedicated to driving innovation, enhancing access, and revolutionizing modern healthcare for women across the globe. She supports women’s health through legal counsel, advocacy, clinician collaboration, and educational campaigns. You can learn more about Corbin at www.femtechlawyer.com or connect with Corbin on LinkedIn.