RAPIDS: RAdar Pluvial flooding Identification for Drainage System (2010 - 2014)
RAPIDS: is a project to develop and demonstrate a tool to deliver rapid forecasting of urban flooding from manholes and other sewerage nodes. The project focuses on utilising machine-learning modelling tools that can also deliver acceptable levels of accuracy. Simple transferability has been demonstrated through the UKWIR RTM project, together with a number of industrial partners, in which 3-case study cities were modelled and results assessed.
CWS has developed the RAPIDS software (currently in MATLAB), which includes two programs: RAPIDS1, which addresses the need for a faster surrogate for hydrodynamic simulators for early warning of urban flooding from sewers, and RAPIDS2 (under development), which aims to provide nowcasting for rainfall over the catchment containing the modelled Urban Drainage network (UDN). It is hoped to be able to demonstrate the cascading of these two systems to provide the required urban flood predictive model, which can deliver operationally useful forecast times in excess of 2-hours ahead.
RAPIDS overall schema
The program is based on a lagged-input, 2-layer, feedforward Artificial Neural Network (ANN), used to relate incoming rainfall data to the extent of flooding present at each node in the UDN. It has the same number of output neurons as sewerage nodes of interest – i.e. there is no requirement to model nodes that never flood, making an immediate computational saving. The ANN architecture is varied to establish an optimum. The supervised training regime uses either backpropagation of error, quasi-Newton gradient-descent or NSGA-II Evolutionary Algorithm method. A moving time-window approach is implemented whereby lagged time-series signals (e.g. rainfall intensity, cumulative rainfall, soil moisture, pump states, tidal levels etc) are provided in parallel over the time-window as inputs to the ANN. If no direct SCADA observation data is available for the network to be modelled, output target signals for training and evaluation of ANN model performance are provided from the flood-level, volume or flow hydrographs generated by hydrodynamic simulator outputs for each sewerage node to be modelled. This only needs to be done for the training dataset of rainfall events. The trained ANN thus aims to generate the same hydrographs for new rainfall events as would the UDN itself, based on having learned and generalised the (non-linear) relationship between the provided input signals and observed or simulator-generated targets.
Fig. 1 (a) Architecture of RAPIDS 1
Figure 1(a) illustrates the architecture of the RAPIDS1 system to predict sewer network outputs. The target signals selected are the flood levels at each sewerage node at a time-step that corresponds to the desired prediction lead-time i.e. up to network Time of Concentration (ToC). Event profile data arrays of the input-signals are prepared for use as the time-series input to the ANN as illustrated in Fig. 1(b). In line with best practice, all input data are normalised.
Fig. 1 (b) Selected ANN Input signals for a typical rainfall event
A selection of (historic) rainfall events is needed for the training dataset. These need to be representative of the envelope of likely intensities and rainfall totals for the future events to be modelled. If sufficient of these are not available, existing events can be augmented by factorally increasing rainfall intensity and modelling resulting target hydrographs using a hydrodynamic simulator.
Rainfall radar images are sourced from the UK Met Office NIMROD system, which produces a composite 1km resolution Quantitative Precipitation Estimate (QPE) image covering the whole UK, every 5-minutes. A live RSS feed is available on request. Historic data images (from April 2004 to present) are available for download from the CEDA Archive website. Treatment of radar QPE images pixel-by-pixel by an ANN is computationally prohibitive since, for example, for a 3-h prediction there would be 36-images, each with at least 3602-pixels (allowing for a maximum storm advection velocity of 60 km/h). This would potentially require ~5 ´ 106 neurons (at 1-neuron per pixel). Therefore features are extracted from the rain echoes in each time-step and associated with features from previous time-steps. These can then be applied to the inputs of an ANN as time-series signals. The feature extraction approach proposed is similar to Discrete Wavelet Transforms (DWT) using Haar wavelets, but using different sized grids depending on the proximity to the catchment being modelled. The mean rainfall for the whole area is evaluated; then residuals of mean rainfall over each sub-grid square are computed: See figure 2:
Fig. 2 RAPIDS2: Rainfall Event 2007-06-14 – QPE snapshot at 22:30 showing original image (top left) and feature extraction of residuals at finer grid resolutions (128 to 1 km); standard deviations show that information is contained at all spatial scales.
The extracted residuals from multiple images over the duration of each event become time-series signals, which can be applied as input signals to ANNs: see figure 3:
Fig. 3 RAPIDS2: Rainfall Event 2007-06-14 – Time-series ANN input signals over 24-hours at spatial resolutions as shown; x-axis is radar image number; y-axis is Δ rainfall intensity in mm/hr.
A similar time-windowed ANN framework as for RAPIDS1 will then be implemented. Target rainfall for training and evaluating the ANN is derived from the rainfall intensities in grid squares covering the required catchment containing the UDN to be modelled, advected into the future by the required prediction period.
UKWIR – RTM – Case Studies
Three case studies for RAPIDS1 have so far been conducted under the UKWIR-funded joint-industry RTM project, in two-stages to evaluate effectiveness in different sized catchments under different conditions; stage 1 used design rainfall and stage 2 used real rainfall. Overall RTM project coordination was performed by HR Wallingford.
Case Study Cities
- Dorchester: small urban catchment (Partner: RAA) - Evaluation of the significance of use of soil moisture as ANN input
- Portsmouth: medium urban catchment (Partner: Mouchel) - Island location; tidal effects; need for pumping; evaluation of effectiveness of ANN models to provide early starting of pumps – as a flood-mitigation strategy
- Crossness (South London): large urban catchment (Partner: Halcrow) - Evaluation of model effectiveness using spatially varying rainfall as ANN inputs.
HydroMAT – Hydrographic Model Analysis Tool
In order to allow all partners to present results consistently, the HydroMAT model analysis tool was developed to provide automated assessment of ANN output using a number of standard metrics:
- Nash-Sutcliffe Efficiency Coefficient (NSEC)
- RMSE-Observations Standard Deviation Ratio (RSR)
- Percentage Bias (PBIAS)
- Total Volume Error (TVE)
- ANN Normalised Root Mean Square Deviation (NRMSD)
- % Samples in Limits - All Nodes
- Amplitude Error of Hydrograph Peak
- Timing Error of Hydrograph Peak
- R-Squared - All Nodes
- Pearson Correlation Coefficient - All Nodes
- ANN Output vs Target X-Y Plot (ATXY) - Single Node
- ANN Output & Target Hydrographs - Single Node
- Confusion Matrix for Peak Flood Depth Categories
- Confusion Matrix for Flood Positives & Negatives
- Confusion Matrix Accuracy Band summary analysis
UKWIR – RTM – Results and Discussion
In summary, results for UKWIR case studies demonstrated that:
- Dorchester: Use of soil moisture levels (NAPI) as ANN input demonstrated a small improvement in model performance, but this was probably not sufficient to offset additional costs of data gathering, preparation and application to ANN model.
- Portsmouth: Use of ANN models were demonstrated successfully to prevent flooding in the 'Morass' area of Portsmouth, when used as a trigger for early initiation of pumping at the Eastney pumping station.
- Crossness: Although modelling the entire 220km2 catchment using 23 raingauges was not possible, spatial rainfall as ANN-input worked will when applied to smaller areas (4-5 raingauges subcatchments). Further work is needed with spatially varying rainfall and large catchments.
Work is ongoing to develop RAPIDS1 to optimise using MOEA methods and allow a universal set of inputs to be definable at run-time, via input data files (MS Excel workbooks).
Work is at too early a stage to present results; the methodology is still under development.