Mathematical Experiments and Modeling Scenarios
Many people feel mathematics is a field in which experimentation doesn’t really
come into play. The term "experiment" generally conjures up images of chemistry,
biology, physics and so on, i.e. what the average person considers "science" (though I
personally would argue that math is absolutely a science). However implementing
mathematical modeling to compare multiple scenarios lands squarely in the realm of
experimentation - a hypothesis is formed (the question to be addressed by the model),
evidence is gathered by variation of inputs (parameter values) and evaluation of
outputs (model results), and the hypothesis is either verified or disproven.
We have addressed the importance of developing a good testable hypothesis for
modeling in a previous article. Now let’s investigate the second aspect of
experimentation - gathering evidence.
When simulating scenarios with a mathematical model, the most important place to
start is to run a baseline or control scenario. This is, of course, highly dependent
upon your hypothesis. For example, if the hypothesis to be tested is:
||Implementation of WHO-recommended levels of measles vaccination in villages in
rural Northern India can reduce local measles incidence by 90%
then a suitable baseline scenario would be to run the model with vaccination coverage of 0%
to represent the current state against which all other scenarios will be compared.
However, if the hypothesis is:
||Increasing measles vaccination from WHO-recommended levels to 100% coverage is
a cost-effective means of reducing measles incidence to zero
then an appropriate baseline scenario would be to run the model with WHO-recommended
vaccination levels to establish the point of comparison. Running the model with 0% v
accine coverage would provide no useful information to answer the specific question to
be addressed by this hypothesis.
With the baseline established, variation of inputs can begin - however care must
be taken to ensure that these inputs are varied only one at a time in order to generate
reproducible and clear results. In the first example above, if the mathematical model
is run with closure of schools during the simulation period in addition to
implementation of the WHO-recommended vaccine coverage rate, then any reduction in
measles cases shown in the model results cannot definitively be associated with
vaccination alone and the evidence gathered from this simulation would not be useful
for verifying the hypothesis. Running further scenarios with vaccination alone and
school closure alone could, in addition to the results from combining interventions,
provide a great deal of valuable information for public health officials in regards to
reducing measles incidence... but this is gathering evidence to test a different
hypothesis than the one at hand.
When evaluating model outputs to verify or disprove a hypothesis, it is
essential to determine which values generated by the model to track. For hypothesis one
above, the desired outcome is measured in terms of disease incidence, which is a
function of measles cases over a period of time; for hypothesis two both disease
incidence and costs must be evaluated, adding a layer of complexity to the overall
Thus gathering evidence to evaluate a hypothesis really comes down to a
three-step process - establish a control (run the model with baseline conditions), vary
inputs singly (change one parameter value in the model at a time), and measure
the appropriate output. With these pieces of information, determining incremental
changes becomes a more straightforward process, and the final aspect of experimentation
using a mathematical model - confirming or refuting your hypothesis - should flow
naturally from these results.
If your organization has a hypothesis that needs testing via scenario modeling,
or if you need to set up a mathematical experiment, contact MathEcology today - we’d be
happy to help out!
Back to Table of Contents... >>