Mathematical Experiments and Modeling Scenarios

Many people feel mathematics is a field in which experimentation doesn’t really come into play. The term "experiment" generally conjures up images of chemistry, biology, physics and so on, i.e. what the average person considers "science" (though I personally would argue that math is absolutely a science). However implementing mathematical modeling to compare multiple scenarios lands squarely in the realm of experimentation - a hypothesis is formed (the question to be addressed by the model), evidence is gathered by variation of inputs (parameter values) and evaluation of outputs (model results), and the hypothesis is either verified or disproven.

We have addressed the importance of developing a good testable hypothesis for modeling in a previous article. Now let’s investigate the second aspect of experimentation - gathering evidence.

When simulating scenarios with a mathematical model, the most important place to start is to run a baseline or control scenario. This is, of course, highly dependent upon your hypothesis. For example, if the hypothesis to be tested is:

(1) Implementation of WHO-recommended levels of measles vaccination in villages in rural Northern India can reduce local measles incidence by 90%

then a suitable baseline scenario would be to run the model with vaccination coverage of 0% to represent the current state against which all other scenarios will be compared.

However, if the hypothesis is:

(2) Increasing measles vaccination from WHO-recommended levels to 100% coverage is a cost-effective means of reducing measles incidence to zero

then an appropriate baseline scenario would be to run the model with WHO-recommended vaccination levels to establish the point of comparison. Running the model with 0% v accine coverage would provide no useful information to answer the specific question to be addressed by this hypothesis.

With the baseline established, variation of inputs can begin - however care must be taken to ensure that these inputs are varied only one at a time in order to generate reproducible and clear results. In the first example above, if the mathematical model is run with closure of schools during the simulation period in addition to implementation of the WHO-recommended vaccine coverage rate, then any reduction in measles cases shown in the model results cannot definitively be associated with vaccination alone and the evidence gathered from this simulation would not be useful for verifying the hypothesis. Running further scenarios with vaccination alone and school closure alone could, in addition to the results from combining interventions, provide a great deal of valuable information for public health officials in regards to reducing measles incidence... but this is gathering evidence to test a different hypothesis than the one at hand.

When evaluating model outputs to verify or disprove a hypothesis, it is essential to determine which values generated by the model to track. For hypothesis one above, the desired outcome is measured in terms of disease incidence, which is a function of measles cases over a period of time; for hypothesis two both disease incidence and costs must be evaluated, adding a layer of complexity to the overall analysis.

Thus gathering evidence to evaluate a hypothesis really comes down to a three-step process - establish a control (run the model with baseline conditions), vary inputs singly (change one parameter value in the model at a time), and measure the appropriate output. With these pieces of information, determining incremental changes becomes a more straightforward process, and the final aspect of experimentation using a mathematical model - confirming or refuting your hypothesis - should flow naturally from these results.

If your organization has a hypothesis that needs testing via scenario modeling, or if you need to set up a mathematical experiment, contact MathEcology today - we’d be happy to help out!

Back to Table of Contents... >>