Measuring system entropy with a deep recurrent neural network model

In this paper, a methodology for assessing the unpredictability of systems with memory was developed. The proposed approach consists in approximating the probability distribution exhibited by the response of a system, understood as a stochastic process, with a deep recurrent neural network; such networks offer increased forecasting capability by exploiting an accumulative register of previous system states. Once the probability distribution is computed, the uncertainty or entropy of the underlying process is measured. This measure determines the degree of regularity in the system, and identifies how atypical the system dynamics are. The proposed model was validated by identifying industrial gas turbine engine faults from recorded sensor data.