Prospective evaluation of multiplicative hybrid earthquake forecasting models in California

Toño Bayona and Max Werner (University of Bristol), in collaboration with Bill Savran (Southern California Earthquake Center) and Dave Rhoades (GNS Science New Zealand), prospectively evaluated the abilities of sixteen multiplicative hybrid and six “single” seismicity models to forecast M>5 earthquakes in California over the past decade. This prospective evaluation tested models developed before 2011 against earthquakes that have occurred since then. Among others, single models use past earthquake records, tectonic data, and interseismic strain rates to formalize multiple hypotheses about the seismogenesis, while multiplicative hybrids combine these models / geophysical datasets to potentially gain predictive skill.

This evaluation, now published in Geophysical Journal International, involves a set of traditional and new tests implemented in the pyCSEP toolkit of the Collaboratory for the Study of Earthquake Predictability (CSEP), which rely on a Poisson and a binary likelihood distribution. Until recently, CSEP tests were mainly based on a probability function that approximates earthquakes as independent and Poisson-distributed. However, the Poisson distribution is known to insufficiently capture the spatiotemporal variability of earthquakes, especially in the presence of clusters of seismicity. Therefore, this research introduces a new binary likelihood and associated tests to reduce the sensitivity of traditional CSEP evaluations to clustering.

The consistency test results show that most forecasting models overestimate the number of earthquakes and struggle to explain the spatial distribution of the observed data. Furthermore, the comparative test results show that, contrary to retrospective analyses, none of the hybrid models are significantly more informative than a model (HKJ) that adaptively smooths the epicentral locations of small earthquakes, suggesting that small-magnitude seismicity is statistically useful for mapping the locations of future larger events in California.

This investigation elevates the international standard for transparent, reproducible, and prospective CSEP earthquake forecasting experiments, as it includes a publicly available software reproducibility package to fully replicate its scientific results. This novel open science practice includes a data repository containing forecast files and earthquake catalogue freely accessible in Zenodo and documented code openly available on GitHub. This case study is now shortlisted for the University of Bristol Open Research Prize 2021, which is an effort to support open research in all disciplines across the university and the final winners will be announced in the coming weeks.