Frequency-based performance measure for hydrologic model evaluation

Abstract

Scale-dependent and -independent error and performance measures are used to evaluate hydrologic forecasting and simulation models. However, these single-valued measures may not always provide a comprehensive assessment of model performance developed for a specific hydrologic process and application. A new frequency-based performance measure (FBPM) that can incorporate application-specific information for model evaluation is proposed to address the limitations of existing measures. FBPM is derived using a data classification scheme to partition the observed data into several classes and evaluate frequencies of each class's chronologically paired observed and forecasted values. A variant of FBPM, composite performance measure (CPM), is also developed to include additional indices that evaluate error, variance, and other statistical characteristics of the observed and forecasted series. Several univariate and multivariate data classification schemes are initially evaluated, and the best one is selected for use in the measures. FBPM and CPM are used to assess the performance of daily streamflow forecasting models developed for the Tapi River, India, using a data-driven model tree (MT) approach. The measures are also evaluated using two synthetic datasets representing different model forecast scenarios. A comprehensive evaluation of streamflow forecasts using FBPM and CPM with the best data classification scheme (i.e., geometric interval scheme in this work) indicates a better and more robust assessment of the forecasting model performance than that from existing error and performance measures. The traditional measures such as coefficient of determination, index of agreement, Nash Sutcliffe, and Kling Gupta efficiency have overestimated model performances compared to FBPM and CPM on an average by at least 100%. The traditional measures have also failed to identify inferior models mainly due to their inflated numerical values compared to those from FBPM and CPM. The measures developed in this study allow the user-specific definition of classes and assignment of weights to these classes based on the intended purpose of the hydrologic modeling effort. The FBPM and CPM are scale-independent, informative, interpretable, and outlier-resistant. These measures can be used for visual and/or statistical evaluation of the performance of single or multiple hydrologic simulation models. © 2022 Elsevier B.V.