You are here

Benchmark Test Distributions

Measurement uncertainty evaluation is an important aspect of industrial and scientific measurement activities. It plays a crucial role in decision making and conformity assessment. Standard uncertainty is a universal parameter used to characterize the quality of a measurement but expanded uncertainty is the better representation of uncertainty in mission-critical applications, especially when health and safety aspects are involved.

To date, evaluation of the expanded uncertainty is handled exclusively by Monte Carlo (MC) simulation method which is robust and easy to implement in a majority of realistic scenarios. However, there are cases where MC cannot be applied and one of such cases is type-A evaluation. An alternative approach to problems where MC cannot be applied is to identify an appropriate distribution based on the information obtained from the high-order moments of the data.

While there are numerous well known fitting methods available in literature, there is no common set of test distributions to measure the actual performance of the fitting technique. In addition, different set of test distributions in literature are not of wide range in terms of skewness and kurtosis. For example, The red shaded region in figure (a) above shows where most test distributions used for performance assessment of the fitting techniques lie on skewness-kurtosis plot. The square and diamond points in the same figure show test distributions used for expanded uncertainty estimation using distribution fitting in literature. Therefore, while the fitting techniques can be reliably used for distributions that fall within the shaded region, there is insufficient information on their performance for distributions outside the shaded region.

Here, we provide a set of analytically derived distributions (blue circles in figure (a) above) to establish a benchmark distribution set and a framework for measuring performance of any parametric distribution fitting method especially for expanded uncertainty estimation. A fitting technique can be assessed easily using the framework shown in figure (b) above.

Download Benchmark Test Distributions

Minor part of the benchmark test distributions has been published in 2015 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) in Pisa, Italy. Therefore, the best citation for using the benchmark test distributions is:

  1. Y. C. Kuang, M. P.-L. Ooi, A. Rajan, and S. Demidenko, "Performance Comparison between Expanded Uncertainty Evaluation Algorithms," in International Instrumentation and Measurement Technology Conference (I2MTC), 2015 IEEE, Pisa, Italy, 2015.

  2. A. Rajan, Y. C. Kuang, M. P. L. Ooi, and S. N. Demidenko, "Benchmark Test Distributions for Expanded Uncertainty Evaluation Algorithms," IEEE Transactions on Instrumentation and Measurement, vol. 65, pp. 1022-1034, 2016.