Why the Cognitive Clinical Success Machine is so Far Beyond Predictive Analytics
Predictive analytics as a solution category comprises a broad mix of methods including regression analysis, neural networks, and random forests that are applied to predict an event or action. Within healthcare, these solutions are used to stratify patients across bands and identify individuals at high-risk of an adverse medical event.
The problem with predictive analytic solutions is twofold:
Predictive analytic solutions deliver models that only work for one segment at one point in time and are at risk of overfitting
Predictive analytic solutions focus on model performance rather than patient impact
Avoid the Overfitting Trap
Overfitting predictive models is a marketing tool. By selecting the variables that will drive up accuracy numbers, marketers can create the best possible narrative around the potential value of a solution. But the resulting model only works at the stated accuracy numbers for specific variables and at a specific point in time.
While overfitting is hardly a phenomenon isolated to any one industry, it happens to have the biggest potential negative impact within healthcare. By overfitting patient cohorts to drive model performance, we create a solution that misses large portions of the at-risk population. In other words, we compromise care when we base our actions on the accuracy of an overfit solution.
When you are presented accuracy numbers, ask about the cohorts used to drive the model’s performance. Ask if the model has been tested against different demographic areas and points in time. Ask if the model had to be “recalibrated” (i.e. overfit) to meet accuracy thresholds for other, diverse populations. If a model doesn’t work for the entire population over time and demographic differences, it simply won’t work for healthcare.
Value Impact over Accuracy
Predictive analytic modes focus on accuracy. But instead of accuracy, we need to think in terms of patient impact. By changing how we measure a solution’s effectiveness, we move the objective from model accuracy to the number of patients who can be helped.
Predictive analytic solutions are focused on identifying the patients who fall into the high-risk band. But the biggest potential for patient impact lies outside of the high-risk segment to those patients who are not currently high-risk but are on a trajectory to becoming high risk.
When we shift our focus to impact, the approach is transformed. Patient impact forces us to look across the entire population. The solution is therefore aimed at identifying the individuals where we can have a significant impact on outcomes. Moreover, with a focus on impact we are driven to account for the operational and external variables that influence intervention effectiveness.
By putting the patient at the center of the way we think about and evaluate solutions aimed at prevention, we also change the value conversation. Focusing only on high-risk individuals doesn’t do much to improve patient outcomes. By broadening the drive toward patient action across the entire population, we see that value can be realized in the earliest stages of solution adoption. Simply by being better informed and aware of patient risk, we engender more targeted action that leads to more effective patient intervention and improved business impact. This a change that requires no modifications to workflow. By being better informed about who is at risk and the actions that will improve outcomes, we can help the large portion of patients who would have been missed by predictive analytic solutions.