The magic behind the XPRIZE pandemic response challenge – and why it matters (Part 7)

The recently concluded XPRIZE Pandemic Response Challenge reveals the role advanced artificial intelligence (AI) can play in helping to find solutions to humanity’s greatest challenges.  The competition challenged AI practitioners to develop not only better models to predict where the COVID-19 pandemic is headed, but also to recommend what we should do about it: that is, which non-pharmaceutical interventions (NPIs) should be enacted in different countries over time to minimize cases and economic cost.

The teams took many different approaches, which ranged from standard epidemiological modeling to advanced machine learning (ML). The results revealed that there is plenty of room for effective innovation – which will likely translate into better NPI policies even before the pandemic recedes. Two top-performing teams were selected from among 48 finalists in 17 countries. Both teams will share equally the total prize purse of $500,000.

The goal of the Pandemic Response Challenge was to inspire data scientists to apply advanced forms of AI – building on our prior work on the Evolutionary AI^TM platform – to optimize decision making on COVID-19 mitigation. The AI was developed and evaluated using global COVID-19 infection-rate and NPI data provided by Oxford University. The teams combined surrogate modeling (i.e., prediction) with optimization of decision making (i.e., prescription) to unlock optimal COVID-19 decision-making strategies. This same approach is viable for a variety of business strategy and design challenges, from healthcare and manufacturing, to marketing and finance. 

Our previously developed model on NPI optimization served as a starting point for the competition. It was then extended with mechanisms to evaluate new predictors and prescriptors.

Developing the models

The teams were first asked to develop better predictive models. Using a sequence of cases and NPIs as input, they were challenged to estimate the number of cases that would develop in a given country or region over a designated timeframe. We used a real-time leaderboard to compare their predictions with the real world, revealing differences in accuracy among the different predictive approaches across different regions and timeframes. This result suggested that in the future, it may be possible to build a super-predictor by ensembling diverse entries in the competition.

The second, and the main, challenge was to develop models that would make effective NPI prescriptions. Note that it is not possible to evaluate such arbitrary recommendations in the real world. However, the predictive models developed in the first challenge could be used as a surrogate for the ground truth. This is the main principle behind Cognizant’s ESP technology, as well: the ability to discover decision strategies without incurring a large cost in the real world.

Evaluating the prescriptors

An important innovation in developing the prescriptors was to take into account local preferences. Decision makers may prefer different kinds of policies: For instance, closing public transportation has a higher cost for London than for LA. Each prescriptor was thus evaluated with several different weights for the NPIs. Those teams that were able to take such preferences into account in recommending policies ranked higher in the competition. Learning such policies is a challenging problem, and a powerful approach for flexible decision making in general.

Because each prescriptor represents a particular tradeoff between minimizing cases and minimizing economic impact, each team was invited to submit multiple prescriptors. Another innovation in this competition was to evaluate each prescriptor based on how many other prescriptors they dominated in this tradeoff space in terms of beating them both in terms of cases and economic impact. By doing so, we could identify which teams were able to develop not just one but an entire set of effective prescriptors representing the different tradeoffs.

A final, and most innovative, dimension of prescriptor evaluation was to judge the merit of the ideas in them. As it turns out, this evaluation can be done systematically and quantitatively using Evolutionary AI. Each prescriptor was first distilled into an equivalent neural network. The population of such neural networks (i.e., original solutions) was then evolved (i.e., solutions were mutated and recombined with other solutions), thus creating new solutions that improved upon them.

The solutions representing the best tradeoffs were identified in the final improved population, and their evolutionary lineage was traced back to the original solutions. In this manner, it was possible to identify those original solutions whose "DNA" was widespread in the final population – indicating that the ideas in them were useful. This result is particularly exciting because it shows how Evolutionary AI can be used to leverage the creativity of human problem solvers.

Choosing the winners

To decide the winners, an independent judging panel comprising academics, pandemic response specialists, AI experts and epidemiologists, among others considered all these quantitative metrics, as well as qualitative dimensions such as how innovative, collaborative, inclusive, general and consistent the solutions were.  The solutions developed by the winning teams were, indeed, highly effective and elegant, exceeding expectations and strongly achieving the main goal of the competition. (See the judging criteria here.)

In addition, the technology used to evaluate the solutions, described above, broke new ground in several ways as well, and constitutes an exciting result on its own merits. In the future, we will continue to build this technology further, thus advancing AI-based decision making and enabling its applications to business problems.

Learn more about Cognizant’s Pandemic Response Challenge with XPRIZE, a $500K, four-month challenge that focused on the development of data-driven AI systems to predict COVID-19 infection rates and to prescribe intervention plans.

Take a look at the finalists, the winners and the technology behind the Pandemic Response Challenge, and see Parts 1, 2, 3, 4, 5, 6 and 8 of our blog series.