Congress Resources: Papers, posters and presentations

< Return to abstract list

Integrated Water Network Management: Big Data And Analytics

Congress: 2015
Author(s): David Kenny, Amir Peleg

Keyword(s): Sub-theme 1: Water supply and demand,
AbstractINTRODUCTION: "Big Data" refers to the collection of large and complex data sets that are difficult to process using traditional data management tools. These larger data sets enable organizations to derive trends and patterns across a large set of related data, finding important correlations between previously disparate smaller sets of data and breaking down silos. The big data trend is changing water utilities' approach to network operations and management, challenging them to manipulate and use data to develop actionable insights. TaKaDu is an Integrated Water Network Management (IWNM) solution that empowers utilities to use their data to improve overall network efficiency, making use of big data analysis techniques for day-to-day as well as strategic long-term decision making. METHODS/MATERIALS: TaKaDu's technology uses a number of big data analysis techniques to improve water network efficiency. Network data that is already being collected by the utility is sent to the TaKaDu statistical algorithms engine where it is cleaned, processed, and analysed. Confirmed anomalies are then fed back to the application in the form of "events." TaKaDu's big data capabilities include anomaly detection based on historical and network-based predictions and geolocation of leaks among others. The historical prediction compares current network activity with similar periods in the past. This guarantees that seasonal and periodic patterns are considered and evolving conditions are thoroughly compared. By contrast, network-based analysis compares the behavior of a meter, a metered area, or a network segment to the behaviour of other parts of the network that were previously correlated with these measurement points. This allows for accurate detection, helping to classify an event as an anomaly or as part of the normal state of the network. Both algorithms utilize new data to refine their predictions, learning at a balanced pace that allows for a quick reaction to network changes on the one hand, and validating those results on the other. With geolocation capabilities, TaKaDu's technology helps indicate the location of an event, in addition to the event type and magnitude. The geolocation utilizes flow and pressure measurements, along with a readily- made hydraulic model based upon GIS data, to limit the event to a smaller part of the DMA, pressure zone, or sector. This algorithm utilizes TaKaDu's advanced predictions to automatically create an accurate consumption model, as well as statistical and data cleansing techniques to extract meaningful results out of the noisy real-life data fed into the error-prone hydraulic model algorithm. Similar techniques are used to create the aforementioned hydraulic model, while accounting for GIS errors of all kinds (missing/misplaced assets, missing/invalid attributes, etc.) as well as measurement errors. TaKaDu's technology also automatically detects background leakage called "flow trend events," which represent small evolving leaks that typically turn into larger bursts. When these long-term leaks are fixed, not only is future leakage and damage prevented, but there is also a reduction in overall NRW. Detecting subtle events requires careful examination of long-term data while accounting for distracting effects such as seasonality. RESULTS AND DISCUSSION: The following examples show how TaKaDu's IWNM approach affects utility operations in real life. Holidays and unique periods are considered by some of the predictive algorithms, ensuring that all potential explanations for the deviation from the norm are evaluated prior to alerting. Historical and network-based predictions accurately identified and categorized network anomalies during the viewing of the Netherlands vs. Spain World Cup match on June 13, 2014 in the Netherlands. While the historical prediction for flow predicted regular water usage when compared to the previous same weekday and time, the network-based prediction successfully predicted the actual supply, so no alerts were raised despite significant variation from the norm. In May 2013, TaKaDu alerted a large regional utility in Australia that a 16 l/s burst had occurred, geolocating the event and color-coding the area where the leak was most likely to be. A detection team was sent to the "red" zone, finding the break near a railway cutting that would have been almost impossible to find without geolocation. Within 24 hours of receiving the alert, the leak was located and repaired and TaKaDu automatically verified the repair. Narrowing the detection zone to less than one third of the original area resulted in a shorter detection time, lower costs, immediate repair, and prevention of additional damage. TaKaDu flow trends identified an event in December 2012, just a few weeks after the TaKaDu solution was implemented at a Spanish utility. The leak was identified as having started months earlier, with less than a 1 l/s loss, and grew gradually to a magnitude of 4.8 l/s. The slow flow increase and data spikes throughout that year means the small leak would have been very difficult to detect using standard analysis methods (i.e. nightline analysis and threshold based alerts). The leak eventually reached 6 l/s when it was repaired in April 2013. A repair team was sent to the field and the leak was repaired, reducing the nightline back to its pre-increase level and saving approximately 15,500m3 of water per month. CONCLUSION: IWNM helps utilities improve network efficiency, detecting hidden leakage and reducing losses, better understanding of network performance, avoiding catastrophic events and improving customer service. The TaKaDu solution applies an innovative big data approach and uses the power of cloud computing to help water utilities make smarter decisions through the analysis of multiple types of data. Big data and analysis will play an increasingly large role in addressing water utility challenges such as energy costs and aging infrastructure.
2011 IWRA - International Water Resources Association - - Admin