Hydrocarbon volume prediction performance in the Dutch subsurface and the impact of selection bias (2020)

Deze MSc scriptie is geschreven door oud-stagiair Vincent van der Kraan.

The E&P industry is frequently characterized by disappointing project outcomes. Specifically, the industry fails to deliver what is promised in term of hydrocarbon volumes due to overly optimistic predictions (NPD Resource Report, 2018). Although this prediction bias problem is well-known amongst specialists involved, literature is scarce. Two suggested causes of the prediction bias are evaluation tool bias (e.g. imprecise seismic interpretation) and cognitive bias (e.g. individual motivational bias). In addition, Hoetz (2016) proposed the idea of a Selection Bias (SB) in the E&P industry. SB is based on the idea that more attractive prospects are assumed to be more matured; when these overly optimistic projects are drilled, they therefore result in disappointing volume delivery. In the Netherlands, the state-owned company EBN participates in virtually all E&P projects and has been reporting disappointing volumes for years (EBN Focus, 2019). This study aims to address this problem by identifying and quantifying key parameters that contribute to the volume prediction bias.

First, using EBN data of the Dutch subsurface, a statistical look-back analysis is performed to check the quality of subsurface parameters used for volumetric assessments. Past volumetric performance is evaluated for both exploration and development wells in the time frame of 2004-2019 (215 cases). This is then broken down in relevant subsurface parameters. Using statistical tools, the prediction bias for recoverable volume, top reservoir depth, GWC, PHIE, Sw, GRV, NRV, N/G and pressure is quantified. Significant prognosis errors on single well scale are observed as well as a significant bias on portfolio scale. Prediction errors often indicate over-optimism. GRV and Sw are found to be major contributors to the observed volumetric bias. 

Secondly, the effects of prediction bias are modelled on portfolio scale using synthetic portfolio modelling. A stochastically generated synthetic drilling portfolio is designed. Prospects are ranked based on attractiveness and drilled on paper. As the industry works with incomplete/noisy data the perception of a prospect often differs from reality. Hence for each prospect a prognosis is synthesized using Monte Carlo simulation. When the synthetic portfolio is drilled on paper the prediction quality is assessed by comparing the generated prognosis and its actual. Findings are that prediction bias can be modelled on portfolio scale based on the concept of SB. A volume bias is unavoidable due to SB as the ranking process prefers large prospects (being truly large or perceived as large). Matured portfolios in which prospects sizes are more clustered, might lead to increased SB. SB is a function of evaluation uncertainty though, so rigorous technical work might reduce SB.

The main outcome of this study is that the look-back analysis of the EBN portfolio shows a volume bias of 42%. This can partly be explained by SB, but other factors contribute. The findings of this study can help prioritize which parameters need more careful attention in reviewing project proposals. Furthermore, including the effect of EB in resource prediction tools might help to improve prognosis quality.