Loughborough University
Browse
Humphrey et al Updated Final.pdf (1020.69 kB)

Beyond validation: assessing the legitimacy of artificial neural network models

Download (1020.69 kB)
conference contribution
posted on 2023-02-14, 12:06 authored by G.B. Humphrey, H.R. Maier, W. Wu, N.J. Mount, G.C. Dandy, Christian DawsonChristian Dawson

Artificial neural network models have been used extensively for prediction and forecasting over the last 25 years. As the data used to develop ANNs contain important information about the physical processes being modelled, it is generally implied that a model that has been calibrated (trained) and performs well on an independent set of validation data represents the underlying physical processes of the system being modelled.

However, this is not necessarily the case, most likely due to problems with equifinality, where different combinations of model parameters (e.g. connection weights) result in similar predictive performance.

Consequently, there is also a need to check the behaviour of calibrated ANN models as part of the validation process, which is commonly referred to as structural, conceptual or scientific validation (Figure 1). This checks whether the input-output relationship captured by the model is plausible in accordance with a priori system understanding.

In this paper, the importance of considering structural validation is demonstrated. This is achieved by developing ANN models with different numbers of hidden nodes for two environmental modelling case studies from the literature namely, salinity forecasting in the River Murray in South Australia and the prediction of treated water turbidity at a water treatment plant based on raw water quality and the administered alum dose.

The validation errors are then compared with corresponding model behaviours. This was done using the validann R-package, which caters to a range of structural validation approaches. Results show that ANN models producing the best fit to the data do not necessarily result in models that behave in accordance with underlying system understanding.

History

School

  • Science

Department

  • Computer Science

Published in

Proceedings of the 24th International Congress on Modelling and Simulation

Pages

85 - 91

Source

24th International Congress on Modelling and Simulation

Publisher

Modelling and Simulation Society of Australia and New Zealand (MSSANZ)

Version

  • AM (Accepted Manuscript)

Rights holder

© The Authors and Modelling and Simulation Society of Australia and New Zealand Inc. (MSSANZ)

Publisher statement

These proceedings are licensed under the terms of the Creative Commons Attribution 4.0 International CC BY License (http://creativecommons.org/licenses/by/4.0), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you attribute MSSANZ and the original author(s) and source, provide a link to the Creative Commons licence and indicate if changes were made. Images or other third party material are included in this licence, unless otherwise indicated in a credit line to the material.

Publication date

2021-12-07

Copyright date

2021

ISBN

9780987214386

Language

  • en

Editor(s)

R. W. Vervoort; A. A. Voinov; J. P. Evans; L. Marshall

Location

Sydney, Australia

Event dates

5th December - 10th December 2021

Depositor

Dr Christian Dawson. Deposit date: 13 February 2023

Usage metrics

    Loughborough Publications

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC