David Chataway

January 2020

Overview

Infertility poses a widespread challenge in society, with one in six couples facing some form of infertility issue [1]. While various technologies have been developed to address infertility, in vitro fertilization (IVF) remains the most effective and commonly utilized method. However, the success rates of IVF cycles have remained disappointingly low, with only 30%-40% resulting in viable pregnancies [2].

As a part of the dual degree MS/MBA program at Harvard, a team of students and I set out to tackle this problem by creating a healthcare-AI venture. Our solution was to use an interpretable machine learning approach researched by Helen Yang, a graduate student of Harvard University, and Professor Daniel Needleman’s lab [1]. Within a two-week timeframe, our team, which included Amanda Durham, Daniel Evangelakos, Sebastian Fischer and Alec Meade, evaluated the business case of the venture and it pitched to investors. However, over the course of the project, our team found a number of barriers to entry, including the vast number of unregulated technologies and misaligned incentives with clinics.

Problem

“The hormones and mood swings from the anxiety of the [IVF] process make you so desperate you’d do anything” - former IVF patient (January 2020).

IVF is a challenging process for all parties. For doctors, accurately predicting the best embryos for IVF is highly complex. Time-lapse microscopy during the initial five days is used to select the most promising one or two embryos [1]. Yet, this manual analysis is subjective, time-consuming, and requires substantial effort. Moreover, the process is hindered by the limited understanding of the cellular, molecular, and genetic aspects of embryo development, leaving a considerable element of chance involved in the success rate [2, 3].

For patients, the IVF process is arduous, expensive, and emotionally taxing. For the woman, there are internal ultrasounds, dozens of self-administered hormone injections into her increasingly bruised abdomen, and uncomfortable swelling of her ovaries. Unfortunately, due to the low success rates, many women undergo multiple cycles, each taking several months and bringing heightened desperation and emotional strain.

At the time of our project, many “black-box” models were being developed for IVF. Most of these models provided an automated embryo grading system with little interpretability. We recognized that this approach introduced fundamental issues for embryo selection, such as imperfect performance metric selection (e.g., fetal heartbeat vs. live birth) and potential biases related to external and confounding variables like patient age or body-mass index [1, 4, 5]. In addition, we found that doctors and embryologists were unlikely to place their trust in such systems.

Solution

Our belief was that an interpretable feature approach had the greatest potential for widespread adoption. Our aim was not to create an ML model that would dictate which embryos doctors should select, but rather to provide them with key features to aid their decision-making process. This approach would maximize the "clinical usefulness" and be most likely to earn the trust of both doctors and patients.

The foundation of the solution rested on the research by Helen Yang and her colleagues. [1] They developed a machine-learning model consisting of five convolutional neural networks (CNNs) trained to automate feature extraction from time-lapse microscopy of embryos. As input, the model took movies consisting of 1400-2450 images per embryo. The individual CNNs extracted the following features: (1) semantic segmentation of the regions of the embryo, (2) regression predictions of fragment severity, (3) classification of the developmental stage, (4) segmentation and detection of cells and (5) segmentation and counting of pronuclei. The outputs of each CNN were then combined when presented to clinicians (refer to the figure below).

When tested, the model demonstrated promising results, achieving classification accuracies of approximately 88% for the fragmentation and stage tasks and precisions around 0.7 mAP for the cell and pronuclei segmentation tasks. Much of the error was thought to be from the human labeling of the test data. Overall, the performance of the model suggested that it would greatly assist with the embryo selection process.

Figure: A demonstration of the model’s pipeline of CNNs and the output for each feature extraction task. First is the segmentation providing the region-of-interest (ROI). Then the fragmentation is scored. If the fragmentation score is low, then the model proceeds with classifying the development stage. Last, cell contours or pronuclei are identified depending on the number of cells present [1].

Go-to-market

Initially, the go-to-market plan appeared straightforward: license the technology to clinics, who would then offer it to patients as an "add-on" for their IVF treatment. This business model already existed, with many clinics providing optional add-ons at varying costs, some as high as $5,000, alongside the standard IVF cycle priced at approximately $20,000 [2].

However, the economics of these add-ons introduced a dilemma for IVF clinics. If these technologies proved too effective, it could lead to a decrease in returning patients. As a consequence, the clinics would experience a reduction in their primary revenue stream: the revenue generated from the IVF treatment itself.

For a clinic, determining the price (or more precisely, the margin) that justifies integrating an AI add-on capable of delivering incremental success rates involves comparing the customer lifetime values (CLV) with and without the add-on. This simplified assessment is shown in the equations below.

Therefore, for every one percentage point increase in the average success rate, clinics would need to charge a minimum of $1140 more for the AI add-on, assuming a baseline IVF margin of $20,000 and a success rate of 35%. Based on our estimations (which lacked randomized controlled trials), Helen's technology could potentially boost success rates by 2%. Consequently, the add-on would need to be priced at a minimum of $2300. Naturally, clinics might aim to set higher prices, considering that some patients are willing to pay more for any improved chances of success.

However, our research revealed that only a few insurers or health plans cover the costs of IVF add-ons, leaving most patients to bear these expenses out of pocket. Furthermore, the market is highly competitive, with many clinics offering loosely regulated menus of "add-ons" that lack demonstrable evidence of increased success rates [3, 6].

Conclusion

After the intense two-week period, we presented our business plan to a panel of investors. The pitch drew expressions of warm interest along the lines of "if you decide to proceed with this, let me know."

I didn’t decide to go forward with the idea. While I developed an appreciation for the pain of IVF and thought the technological approach was sound, I was concerned with go-to-market challenges. There were too many alternative add-ons that were not proven to be effective, and clinics lacked economic incentives to adopt effective add-on treatments. As the Economist recently noted, one problem is that, given scant public funding for research in this area, randomised and blinded studies with large numbers of patients are infrequent. Private clinics have little incentive to spend investors’ money on trials with thousands of patients, especially when there is a risk they will show add-ons that are already being sold to be ineffective [2].

Nevertheless, I remain optimistic that recent findings and recommendations, such as those from the European Society of Human Reproduction and Embryology (ESHRE), will pave the way for increased accountability, regulation, and research in this field. I agree with ESHRE’s principles of responsible innovation as it relates to treatment add-ons, which among other things emphasizes the importance of high-quality studies to prove effectiveness, patient transparency and the charging of patients only if treatment is proven effective [6]. I look forward to a future where IVF is easier and cheaper for patients thanks to technology such as interpretable ML models for embryo selection.

References

Image: https://health.clevelandclinic.org/iui-vs-ivf/

[1] https://arxiv.org/abs/2006.00067

[2] https://www.economist.com/technology-quarterly/2023/07/17/ivf-remains-largely-a-numbers-game

[3] https://www.economist.com/leaders/2023/07/20/making-babymaking-better

[4] https://link.springer.com/article/10.1007/s10815-021-02254-6

[5] https://arxiv.org/abs/2105.00060