Dipartimento di Ingegneria dell’Energia, dei Sistemi, del
Territorio e delle Costruzioni
TESI DI LAUREA MAGISTRALE IN INGEGNERIA
ELETTRICA
Development and implementation of an
advanced monitoring and diagnostic
system for Hydro Power Plants based on
Big Data analytics and Computational
Intelligence
Advisors:
Prof. Ing. Mauro Tucci Ing. Antonio Piazzi
Candidate: Gianluca Paolinelli
In this work, the development and the implementation of a real-time monitoring and di-agnostic system for hydro power plants is introduced. The tool was developed, tested, and validated on data coming from a hydro power plant in Veneto. The plant is deeply sensorized, and thousands of signals are acquired requiring a Big Data approach. The operating system is able to detect plant malfunctions and deviations from nominal be-haviour and to identify critical signals applying Computational Intelligence methods. The detection is performed through a key performance index based on Self Organizing Maps. The system is currently in full operation in the Test Plant and the results can be explored through a web dashboard.
Introduction 1
1 Overview on Hydro Power Plants operation and management 4
1.1 Hydropower, world status and trends . . . 4
1.2 Hydropower Plants . . . 6
1.2.1 Hydroelectric energy transformation principles . . . 6
1.2.2 Main plant components . . . 12
1.3 Operation and Maintenance . . . 16
1.3.1 Maintenance management methods . . . 17
1.3.2 Cost Evaluation of different maintenance methods . . . 21
1.4 Fault modes in hydroelectric power plants and related diagnostics approaches 22 1.4.1 Turbine faults . . . 24
1.4.2 Generator faults . . . 26
1.4.3 Transformer faults . . . 29
2 Statistical monitoring techniques and data driven fault detection models 30 2.1 Motivations . . . 30
2.2 Working on data: what is Data Analysis . . . 31
2.3 Monitoring and diagnostic models based on Statistical approaches . . . 33
2.3.1 Statistical Process Control . . . 33
2.3.2 Univariate methods: Control charts . . . 35
2.3.3 Multivariate methods . . . 38
2.3.4 Multivariate methods in energy production systems . . . 40
2.4 Computational Intelligence proposed approach: Self Organizing Maps . . . . 42
2.4.1 Training Phase . . . 46
3.1 Big Data Hydro Project . . . 49
3.1.1 Big Data Hydro pilot plants . . . 50
3.1.2 i-EM approach, components of the system . . . 51
3.2 Big Data Hydro Data Set . . . 60
3.2.1 Data quality assessment . . . 62
3.2.2 Not Regular Data Management . . . 67
4 Case of study: Data Analytics 68 4.1 Univariate Analysis . . . 68
4.2 T2 Control Chart application . . . 73
4.3 SOM based approach . . . 75
4.3.1 Phase I: training . . . 75
4.3.2 Distortion Measure based KPI proposal . . . 75
4.3.3 Phase II: warning generation . . . 77
4.3.4 Results . . . 80
4.4 Dashboard . . . 88
Conclusion 96 A Additional Stopping Rules for control charts 98 A.1 Nelson Stopping Rules . . . 98
A.2 Western Electric Rules . . . 101
B Imputation with KNN Algorithm 102
C Efficiency analysis 105
1.1 Pumped hydroelectric energy storage system integrated with windfarm [4] . 7
1.2 Hydroelectric power plant with storage capability [3]. . . 9
1.3 Run of river hydroelectric power plant [3]. . . 10
1.4 Pumped storage hydroelectric power plant [5]. . . 11
1.5 The Three Gorges Dam (China), the world’s largest power station in terms of installed capacity. . . 11
1.6 Generic layout of a hydro plant. . . 13
1.7 Turbine and generator coupled at the same vertical shaft. . . 13
1.8 Kinds of turbine runner. [9]. . . 15
1.9 Range of application for different turbines [9]. . . 15
1.10 Bathtub failure curve. . . 18
1.11 Cash flow diagram for lost production. . . 22
1.12 Cost of a preventive scheduled maintenance activity. . . 22
1.13 Potential savings using condition based monitoring maintenance. . . 23
1.14 Common faults on hydro power plant. . . 23
1.15 Failure occurrences, damage vs root causes [16]. . . 27
1.16 Influence of condition monitoring methods on insulation root causes [16]. . . 28
2.1 Steps in Data Analysis. . . 32
2.2 Probabilities associated to normal distribution [19]. . . 34
2.3 Univariate and bivariate regions for two variables, from [27] . . . 39
2.4 T2 Control chart example . . . 40
2.5 SOM representation [37] . . . 43
2.6 SOM Neighborhood [38] . . . 44
2.7 SOM Neurons initial position [38] . . . 45
2.8 SOM Architecture . . . 46
2.9 Training data set [38] . . . 47
3.2 Scheme of the analysis chain; from "Descriptive" to "Prescriptive". . . 52
3.3 Data acquisition system architecture. . . 53
3.4 Descriptive analytic tool architecture. . . 54
3.5 Diagnostic tool architecture. . . 56
3.6 Predictive maintenance tool architecture. . . 56
3.7 Interview to submit to plant operators . . . 57
3.8 Prescriptive tool architecture. . . 58
3.9 Pilot plant main info. . . 61
3.10 Example of duplicate data sample. . . 62
3.11 Example of typos. . . 62
3.12 Data Quality definition. . . 64
3.13 Example of labeled time series, signal from Group1. . . 64
3.14 Example of labeled time series, signal from Generatore G1. . . 65
3.15 Vibration signal from FL Gruppo1 and Current signal from Generatore Gruppo1 . . . 66
4.1 Shewhart Control Chart application (a) . . . 69
4.2 Shewhart Control Chart application (b) . . . 70
4.3 Shewhart Control Chart application (c) . . . 71
4.4 Shewhart Control Chart application (d) . . . 72
4.5 Example of T2 Control Chart, training: 2017-05-01 to 2018-02-28. . . 74
4.6 Attribute contribution to distortion measure algorithm. . . 77
4.7 Example of SOM based KPI and signal scores . . . 79
4.8 Flow chart of SOM based analysis implementation. . . 81
4.9 Comparison between warning models, G1 . . . 82
4.10 Comparison between warning models, G2 . . . 83
4.11 Comparison between warning models, G3 and G4 . . . 84
4.12 Test case Gruppo 1, KPI (synthetic values 02/05-03/05) . . . 85
4.13 Test case Gruppo 1, data (synthetic values 02/05-03/05) . . . 86
4.14 Warnings for Trasformatore Principale G1 from dashboard analytics section. 87 4.15 Raw data for fl Trasformatore G1. . . 87
4.16 Phase II KPI trend (training 2017/09/01 - 2018/02/28). . . 88
4.17 Overview Section (map). . . 90
4.18 Overview Section (list). . . 91
4.19 Plant Overview Section. . . 92
4.22 Analytics Section. . . 95
A.1 Nelson rules [23] . . . 100
B.1 Imputation algorithm scheme. . . 104
1.1 New installed capacity by country in 2016 (IHA). . . 5
1.2 Italian Energy production in 2016, TERNA . . . 6
1.3 General issues of condition monitoring . . . 21
1.4 Interview answers [16] . . . 26
2.1 Constants for R Chart control limits. . . 37
2.2 Constants for S Chart control limits. . . 38
3.1 Numeric Data Quality, Archive mod. . . 65
3.2 Flag Data Quality (Not-Regular values are only NaNs), Archive mod. . . . 65
Hydro Power Plants area is currently experiencing both a revolution through digitiza-tion and digitalizadigitiza-tion transformadigitiza-tion. The two concept are related, but they are not synonymous: typically, digitization is referred as the automation of existing manual and paper-based processes enabled by the digitization of information from analog to digital. In contrast, digitalization is the use of digital technologies to change a business model and provide new revenue and value-producing opportunities; it is the process of mov-ing to a digital business. In the sector, enhanced digital systems are part of a growmov-ing trend towards improving the performance of turbines, plants and equipment, by reducing costs, adding flexibility and enhancing asset management. Equipment manufacturers are embracing digitalisation as a way to widen their scope of services and to extend the life of existing hydropower assets. New concepts are exploited: methods and models such as industry 4.0, machine learning, cyber-physical systems, internet of things, and internet of services. Digitalization will also effect maintenance and operation of hydropower plants, and there is a large unexploited potential for improving maintenance and reducing costs. By 2030 over half of the world’s hydropower plants will be due for upgrade and mod-ernisation or will have already been renovated, according to IHA’s database [1]. In this year’s 2018 Hydropower Status Survey, however, 79 of 95 respondents working at hy-dropower companies said they expect to contract major upgrading and modernisation works within the next 10 years. What explains the high percentage of respondents who say major works are just around the corner? Part of the reason is a commitment among industry to adopt best practice in operations and asset management plans: a desire for optimised performance and increased efficiency. Another major driver is the sheer pace of technological innovation in hydro power operations and maintenance, in both developed and developing country contexts.
Until now, condition monitoring and application of monitoring data is mainly restricted to protection systems shutting down the plants when single monitoring signals exceed pre-defined thresholds (e.g. bearings with temperature and vibration protection). However,
more advanced models, in combination with past history (failures and maintenance ac-tions) and domain knowledge (design and function of equipment), can utilize monitoring data in a new and better way than done today, providing:
1. Monitoring technical condition;
2. Diagnosis of technical condition and fault prognosis;
3. Short term and long term prediction of incoming fault (and specific fault detection); 4. Prescriptive maintenance tools;
5. Monitoring of remaining lifetime and risk.
Benefits from the system will be the avoiding of catastrophic faults as well as un-necessary component replacements, the prioritization of the most critical components for maintenance, and the scheduling of new optimized maintenance criteria.
This work was developed to apply these principles in Italian hydropower facilities. The main goal of the candidate was to explore different statistical approaches to condition monitoring and fault diagnostic for hydro power plants. Different solutions have been tested off-line on a real data set coming from a test hydro plant. In a second part, a Computational Intelligence based model has been implemented and tested in on-line modality, producing outputs in a web dashboard. The candidate personally developed and tested analytic models through a MatLab environment.
The models consist on the core of the Big Data Hydro project developed at i-EM S.r.l. (Intelligence in Energy Management company). Since 2005, the company develops energy management solutions for renewable energy projects. Public and private players requires expertise in innovation, reliability and risk minimization in the following areas:
• Distributed generation from renewable sources; • Smart grids and energy storage systems; • Energy efficiency solutions;
• Control and optimization of energy consumption; • Electric vehicles and sustainable mobility.
i-EM role in the BDH (Big Data Hydro) project - approved by one of the bigger company managing generation from renewable sources - consists in developing statistical and machine learning models and in delivering their results in a web application working in real time.
The project aims to maximise the performance of hydroelectric fleet and quickly iden-tify potential malfunctions or anomalies. The programme plans the use of techniques of statistical analysis, new optimisation techniques and predictive maintenance [2].
The first chapter of this document contains a review of hydro power plants opera-tion and maintenance, analysing the plant technologies, their crucial components and the most frequent fault modes with most common techniques in operation and maintenance are presented. In the second chapter, theoretical basis for algorithms used in monitor-ing management are discussed. The third chapter introduces the Big Data Hydro project with the proposed approach. The activity consists in analysing data with univariate and multivariate process control techniques, and with a machine learning approach based on Self-Organizing Maps. In the last part, the application of the models on real data, coming from the pilot power plants, is reported together with first results and an overview on the web dashboard environment.
Overview on Hydro Power Plants
operation and management
1.1
Hydropower, world status and trends
Hydro-power is the generation of power by withdrawing energy of falling water or fast run-ning water, which may be harnessed for useful purposes as electrical energy production. Since ancient times, hydropower from many kinds of watermills has been used as a renew-able energy source for irrigation and the operation of various mechanical devices. Water constantly moves through a vast global cycle, evaporating from lakes and oceans, forming clouds, precipitating as rain or snow, then flowing back down to the ocean. Nowadays the energy of this water cycle, which is driven by the sun, is tapped to produce electricity for the grid or for local usage (micro hydro for isolated realities or small communities). Because the water cycle is an endless, constantly recharging system, hydro-power is considered a renewable energy.
Infrastructure for hydropower projects is also used for freshwater management, and projects with reservoir storage generally provide a variety of value-added services. For example, in addition to providing reliable energy supply, hydropower typically brings a variety of macroeconomic benefits such as water supply, flood protection, drought manage-ment, navigation and irrigation. As water management infrastructure, it is also expected to play an increasing role in climate change adaptation.
In the late 19th century, hydropower became a source for generating electricity. The first commercial hydroelectric power plant was built at Niagara Falls in 1879. In 1881, street lamps in the city of Niagara Falls were powered by hydropower.
exploited for electricity production. It is the leading renewable source for electricity gener-ation globally, supplying 71% of all renewable electricity at the end of 2015. Undeveloped potential is estimated by the WEC (World Energy Council) approximately 10000 TWh/y worldwide. The global hydropower capacity increased by 39% between 2005 and 2015 with an average growth rate of nearly 4% per year, accounting to a total of 1209 GW installed in 2015, of which 145 GW is pumped storage (Data from the World Energy Council).
This rise has been concentrated in emerging markets, as can be seen in table 1.1, where hydropower offers not only clean energy, but also provides water services, energy security and facilitates regional cooperation and economic development.
Rank Country Capacity Added (MW)
1 China 11740 2 Brazil 6365 3 Ecuador 1987 4 Ethiopia 1502 5 South Africa 1332 6 Vietnam 1095 7 Perù 1040 8 Switzerland 1022
Table 1.1: New installed capacity by country in 2016 (IHA).
Top countries per installed capacity, as reported by the IHA (International Hydropower association) in its 2017 report (together with pumped storage) include Norway (31626 MW), France (25405), Italy (21884 MW), Spain (20354 MW), Switzerland (16657 MW), Sweden (16419 MW), Austria (13177 MW), and Germany (11258 MW).
In Italy 15.3% of total energy produced (52.77% of renewable sources) comes from hydroelectric. In 2016, the total hydropower production amounted to 44.257 TWh of electricity (see 1.2). It has been calculated that the hydroelectric potential of the Italian territory could be approximately 200 TWh, of which 47 TWh is economically exploitable. When compared with the amount of energy produced, this indicates that the potential of the hydroelectric resources in Italy is exploited to about 95% and the maximum limit of possible exploitation has been reached.
Relevance for electricity storage
The way towards sustainable energy systems will require a significant increase in power system flexibility. Flexibility in this case refers to the ability of a power system to maintain the continuous service when faced with potentially rapid changes in demand. Augmenting
Total Energy Production 2016 [TWh] 289.768
Renewable sources Fossil sources 84.05 205.718 Hydro Wind PV Thermal 44.257 17.689 22.104 205.718
Table 1.2: Italian Energy production in 2016, TERNA
power system flexibility can be achieved by a variety of options, e.g.: supply-side improve-ments, demand-side management, increased transmission networks, increasing system ef-ficiencies, and the provision of added energy storage. Hydropower storage systems, also in the form of pumped hydropower (HPSS: Hydro Pumped Storage Systems), can be seen as an alternative to battery storage systems (BSS). In traditional fossil powered systems, flexibility has been controlled by the generation side. Base load power is supplied by run-of-river hydro, coal and oil, which are more efficient technologies in continuous operation. Rapidly responding generation is instead supplied by hydropower with storage capacity and gas turbine power plants, capable of follow rapid load variations, at time-scales ranging from seconds, to days, to several months. Increasing the proportion of Variable Renewable Energies (VREs) reduces the flexibility and predictability of a power system. High pene-tration of VREs into an existing energy system introduces more variability on the supply side, while also displacing existing flexible technologies. Electricity storage technologies act as both supply and demand in the system, adding flexibility, and so have the potential to increase the system’s overall efficiency and reduce overall costs. Currently, PHS remains the primary technology used to provide energy storage services on the grid scale. WEC estimates that 99% of the world’s electricity storage capacity is in the form of hydropower.
1.2
Hydropower Plants
1.2.1 Hydroelectric energy transformation principles
To exploit hydraulic energy, water is channeled from the withdrawal point to the generation unit, and then returned to the river. Maximum exploitation is obtained when all the elevation water head can be used. Those principles are usually followed:
• Preserve elevation head and transform elevation energy in pressure energy vertically to the point of return;
Figure 1.1: Pumped hydroelectric energy storage system integrated with windfarm [4]
• Use this energy to generate mechanical energy in a hydraulic turbine.
This is obtained building a pressure channel with a minimum slope to lead water right above the power plant. In this position a surge tank compensates variations in flow, caused by changes in the flow request of the turbine. Position energy is then converted in pressure energy in the penstock, an almost vertical tunnel that bring water at the inlet valve of the turbine.
In order to generate electric power from hydro power, we can identify two key param-eters:
• Flow, or the minimum amount of water that is constantly available throughout the entire year;
• Head, the difference in height between the withdrawal and the turbine discharge.
Choosing the right location and planning requires specific knowledge. Knowing of water flow and height difference the potential water-power exploitable can be estimated. The total available energy in a water flow can be computed applying the conservation of energy principle appropriate for flowing fluids, or the Bernoulli’s equation:
H = h +p γ +
V2
2 g (1.1)
where h represents the elevation head, pγ is the pressure head, and V2 g2 is the velocity head that represent the energy of the fluid due to its bulk motion. Considering a water mass
∆G, the total energy results:
∆W = ∆G H (1.2)
The available power is:
P = ∆W
∆t = ∆G H
∆t (1.3)
If ∆G∆t = Qp is the mass flow rate times g and Qv is the volumetric flow rate:
Pe= η γ QvH (1.4)
To obtain more electrical power both the flow and the head can be increased.
Head H Volumetric Flow Qv
Low < 20 m Low < 10 m3/s
Medium < 250 m Medium < 100 m3/s
High > 250 m High > 100 m3/s
A cubic meter of water falling from a 367 meters head, or 367 cubic meters falling from 1-meter head, produce 1 kWh. At the same time, a water flow of a cubic meter per second, falling from a 102 meters head, or 102 cube meters per second falling from 1-meter head produce a power of 1 MW.
The current commercially available technologies generate electricity through the trans-formation of hydraulic energy into mechanical energy to activate a turbine connected to a generator. It is a versatile energy source, which can respond to different power system requirements while adapting to different physical and environmental constraints as well as stakeholders’ interests. Although hydropower plants are highly site-specific, the local topography and hydrology will define the type of facilities that can be built. They can be broadly categorized into four main typologies:
Storage hydropower
Plant that uses a dam to impound river water, which is then stored to be released when needed. Electricity is produced by releasing water from the reservoir through operable gates into a turbine, which in turn activates a generator fixed to the same shaft. Storage hydropower can be operated to provide base-load power, as well as peak load through its ability to be shut down and started up at short notice according to the demands of the system. It can offer enough storage capacity to operate independently of the hydrological inflow for many weeks, or even up to months or years depending on the capacity of the reservoir. Given their ability to control water flows, storage reservoirs are often built as multi-purpose systems, providing additional benefits. The primary advantage of hydro
facilities with storage capability is their ability to respond to peak load requirements. They are strategical for the system management.
Figure 1.2: Hydroelectric power plant with storage capability [3].
Run-of-river hydropower
Plant that channels flowing water from a river through a canal or penstock to drive a turbine, are called run-of-river plants. Typically, a run-of-river project will have short term water storage and result facility in little or no land inundation relative to its natural state. Run-of-river hydro plants provide a continuous supply of electricity, and are generally installed to provide baseload power to the electrical grid. These facilities include some flexibility of operation for daily/weekly fluctuations in demand through water flow that is regulated by the facility.
Pumped-storage hydropower
Provides peak-load supply, harnessing water which is cycled between a lower and upper reservoir by pumps, which use surplus energy from the system at times of low demand. When electricity demand is high, water is released back to the lower reservoir through turbines to produce electricity. Some pumped-storage projects also have natural inflow into the upper reservoir which augments the generation available. Pumped-storage hydropower is practically a zero-sum electricity producer if does not have natural inflow. Its value is in
Figure 1.3: Run of river hydroelectric power plant [3].
the provision of energy storage, enabling peak demand to be met, assuring a guaranteed supply when in combination with other renewable energies, and other ancillary services to electrical grids, it can also guarantee a night load to keep frequency under control without the duty of turning off base thermal plants. One major advantage of pumped-storage facilities is their synergy with variable renewable energy supply options such as wind and solar power that represent growing and growing non-flexible power supply options. This is because pump-storage installations can provide back-up reserve which is immediately dispatchable during periods when the other variable power sources are not available.
The basic technology is the same irrespective of the size of the development. Large-scale hydropower installations typically require storage reservoirs as mentioned earlier in this section. Smaller-scale hydropower systems can be attached to a reservoir, or they can be installed in small rivers or in the existing water supply networks, such as drinking water or waste-water networks. Small-scale hydropower plants are typically run-of-river schemes or implemented in existing water infrastructure. Another source of smaller-scale hydropower capacity is extracted from modernization of existing hydropower facilities. In cases where it is economically feasible, capacity additions to existing facilities are possible by extending the existing powerhouse to add more units, in addition to the more traditional approach of uprating the existing generator/turbine sets or increasing the efficiency of turbines.
Figure 1.4: Pumped storage hydroelectric power plant [5].
As with all energy technologies, hydropower facilities are reported on in terms of their installed capacity. Hydropower facilities installed today range in size from less than 100 kW to more than 22 GW on the Chinese river Yangtze, with individual turbines reaching 1000 MW in capacity.
Figure 1.5: The Three Gorges Dam (China), the world’s largest power station in terms of installed capacity.
1.2.2 Main plant components
Forebay and Intake Structures The forebay is a reservoir of water in front of intake.
The reservoir acts as forebay when penstock takes water directly from it. When a canal leads water to the turbines the section of the canal in front of turbines is enlarged to create forebay. The forebay temporarily stores water for supplying the same to the turbines. The water cannot be allowed to pass as it comes in the reservoir or the canal. At intake gates are provided with hoist to control the entry of water. In front of the gates trash racks are arranged to prevent debris, trees, etc., from entering into the penstock. Trash racks are cleaned at intervals through rakes.
Head Race or Intake Conduits Carries water to the turbines from the reservoir. The
choice of open channel or a pressure conduit, Penstock, depends on site conditions. The pressure conduit may be in the form of an intake passage in the body of the dam or it may be a long conduit of steel or concrete or sometimes a tunnel extending for few kilometres between the reservoir and the power house. The pressure conduit does not follow the ground contours, potential gradient is given to minimize losses. The velocity of water in the pressure conduit is also higher than in the open channel. Up to about 60 metres head the velocity may range between 2.5 ms to 3.0 ms, and increases as the head increases. Sometimes it is convenient or economical to adopt open channel partly or wholly as the main conduit. The head race canal may lead water to the turbines or to the penstocks and is usually adopted in low-head plants where head losses are relevant. An advantage of an open channel is that it could be used for irrigation or navigation purposes.
Surge Tank A surge tank is a storage reservoir made on a long pipe line or penstock
to receive the rejected flow when the pipe line is closed by the turbine intake valve. The surge tank, therefore, relieves the pipe line of excessive pressure produced due to its closing, eliminating the water hammer effect (when the turbine gates are closed, the moving water has to go back, if the pressure wave is not dampened, it reflects at the begin of the penstock and then returns to the turbine intake valve. That effect must be avoided as can damage the valve). This can be done by accepting in the surge tank the mass of water which would have flown out of the pipe, but returns to the tank due to closure of pipe end. It also constitute a backup supplying suddenly additional flow when required by the turbines. The surge tank is mostly employed in a water power plant or in a large pumping plant to dampen the pressure variations resulting from rapid changes in the flow. The surge tank which is generally located near the turbine will favours the increased demand of water till such time the velocity in the upper portion of the line changes.
Figure 1.6: Generic layout of a hydro plant.
Figure 1.7: Turbine and generator coupled at the same vertical shaft.
Generation Unit The generation unit is the ensemble of a turbine and a generator,
keyed to the same shaft. Turbine is the crucial device that converts hydraulic energy into mechanical energy. The mechanical energy developed by a turbine is used in running an electric generator coupled to the same shaft of the turbine. The generator develops electric power and delivers it to the network through a transformer. The turbine key component is a wheel called runner. The runner is provided with specially designed blades or buckets.
The water possessing large hydraulic energy strikes the blades and the runner rotates. • Impulse Turbine:
In the impulse turbine, all the available potential energy or head is converted into kinetic energy or velocity head by passing the water through a nozzle or by guide vanes before it strikes the buckets. The wheel revolves free in air and water is in contact with only a part of wheel at a time. The pressure of water all along is atmospheric.
To guide the water discharged from the buckets to the tail race, a casing is provided. An impulse turbine is essentially a low-speed wheel and is used for relatively high heads. Pelton wheel, Turgo impulse wheel, and Girard turbine are some types of impulse turbine. In the Pelton wheel water strikes the runner tangentially.
• Reaction Turbine:
In a reaction turbine, only a part of the available potential energy is converted into velocity head, at the entrance to the runner. The pressure at the inlet of the turbine is much higher than the pressure at the outlet.
Pressure varies along the passage of water through the turbine. Mostly the power is developed by the difference in pressure acting on front and back of runner blades. Only small part of power comes from the dynamic action of velocity. Since the water is under pressure, the flow from head race to tail race takes place in a pressured system.
Francis and Kaplan turbines are the two more diffused types of reaction turbines. In Francis turbine there is inward radial flow of water. In modern Francis turbine the flow enters radially inward but leaves in parallel direction to shaft at centre. This is hat is called mixed flow. In Kaplan turbines the flow is axial or parallel to the axis of the turbine shaft. The selection of the type of turbine depends primarily upon the available head.
Turbines may be classified as follows with reference to type of power plant: Low head turbine less than 50 m
Medium head turbine 50 to 200 m High head turbine over 200 m
Low head turbines are Propeller turbine and Kaplan turbine. These turbines use large quantity of water. Medium head turbines are modern Francis turbines. Impulse turbines are high head turbines. These turbines require relatively less quantity of water (fig: 1.9).
Figure 1.8: Kinds of turbine runner. [9].
Figure 1.9: Range of application for different turbines [9].
Hydro generators are often synchronous electrical machines, the number of poles de-pends on the typology and the typical rotation speed of the turbine.
Tail Race and Draft Tube The channel into which the turbine discharges in case of impulse wheel and through draft tube in case of reaction turbine is called a tail race. The suction pipe or draft tube is nothing but an airtight tube fitted to all reaction turbines on the outlet side. It extends from the discharge end of the turbine runner to about 0.5 m below the surface of the tail water level. The straight draft tube is generally given a flare of 4 to 6 degrees to gradually reduce the velocity of water.
The suction action of the water in this tube has same effect on the runner as an equivalent head so that the turbine develops the same power as if it were placed at the surface of the tail water. The tail race of the impulse wheel is commonly an approximately rectangular passage, running from a point under the wheel to a point outside the power house foundations where it enters the exit channel or the river. Because of the small discharge of the impulse wheel, as well as higher allowable velocity, the tail race passage is much smaller than that of the reaction turbine.
In case of the reaction turbine the width of the tail race channel under the power house depends upon the unit spacing and thickness of piers and walls between the unit bays. The depth of the tail race channel depends upon the velocity which is generally taken to be about 1ms. Where the power house is close to the river, the tail race may be the river itself. In other case a tail race channel of some length may be provided to join the turbine pit with the river.
1.3
Operation and Maintenance
Condition monitoring of hydropower facilities is challenging for the owners of ageing fa-cilities facing with strategic asset replacement and/or refurbishment decisions. A great proportion of the existing fleet of hydropower mechanical and electrical equipment is reaching its life expectancy. For this reason, owners are often facing difficult economic decisions between overhaul and replacement. In developed countries with significant hy-dropower assets basic O&M practices such as regular inspections for cavitation damage on turbine blades, stator and rotor windings, bearings and excitation systems, are based on established guidelines and are generally carried out under a scheduled work program. The key challenge is developing an optimised asset management strategy that improves safety and maximises unit availability. Common strategies include implementing remote opera-tion at older facilities, installing real time asset monitoring systems, maintaining key spare components on-site to reduce outage time and other solutions to minimize O&M costs. Asset managers are increasingly turning to digitalization for implementing sophisticated risk-based decision-making tools to optimise their near-term and long-term O&M asset
management plans for maintaining, overhauling or replacing the most critical components of their fleet.
1.3.1 Maintenance management methods
In a digitalised scenario, the way decisions are taken in industry in relevant area as main-tenance management and scheduling is changing. The costs of mainmain-tenance account for 15%–40% of the whole production, and one-third of which is caused by unnecessary and inaccurate maintenance. Machine learning approaches have been shown to provide effec-tive solutions in these areas, thanks to cloud-based solutions and implementation of IoT (Internet of things) systems. The efficient management of maintenance activities is becom-ing essential to decrease the costs associated with components unscheduled unavailability due to forced outages, or with maintenance operation carried out in non optimal periods. Different maintenance procedures have been developed over the past forty years, which can be grouped into three main categories:
• Run to failure:
This type of reactive maintenance is performed only after failures are detected. Fail-ures are defined as a complete degradation of performance of the process. This is the simplest approach dealing with maintenance (when a machine breaks down, than fix it), and is also frequently adopted, but off course results are the least effective one. The cost of interventions and associated downtime after failure are usually more sub-stantial than those associated with planned corrective actions taken in advance. This approach can be otherwise adopted for minor, non-vital components of the plant, whose faults (a fault is defined as a non-permitted deviation of a characteristic property of the process which will cause a certain level of deterioration in the perfor-mance) are not frequent and relevant from the point of view of energy production, and if spare parts are cheap and quickly available. It is actually a "no-maintenance" approach of management. In almost all instances, plants perform basic preventive tasks (i.e., lubrication, machine adjustments, and other adjustments), even in a run-to-failure environment. The major expenses associated with this type of maintenance management are high spare parts inventory cost, high overtime labour costs, high machine downtime, and low production availability. A plant that uses true run-to-failure management must be able to react to all possible run-to-failures within the plant. This requires maintaining extensive spare parts inventories that include at least all the major components for vital equipment in the plant the alternative is to rely on external vendors that can provide immediate delivery of needed components. Also,
maintenance personnel must also be able to react immediately to all machine fail-ures. The final result of this reactive type of maintenance management consists in higher maintenance cost and lower availability of the production plant.
Analysis of maintenance costs indicates that a repair performed in the reactive or run-to-failure mode will average about three times higher than the same repair made within a scheduled or preventive mode. Scheduling the repair minimizes the repair time and associated labour costs. It also reduces the negative impact of expedited shipments and lost production. [10].
• Preventive maintenance:
Maintenance actions are taken following a planned schedule based on time or on hours of operation. This approach is often referred to as scheduled maintenance, failures are usually prevented, but unnecessary corrective actions have to be often performed. As can be seen in fig. 1.10, the mean time to failure (bathtub) curve presents the probability of failure for a machine. A new one has high probability of failure because of installation problems, in the first operating weeks. After that period, the probability of failure is relatively low for a long period, then it increases sharply with elapsed time. There are many programs implemented, some consist of limited and simple operation as lubrification and adjustments, others provide ma-chine rebuilds for critical mama-chinery. The common guideline consists in time schedul-ing. Preventive maintenance management programs assume that machines will
de-Figure 1.10: Bathtub failure curve.
grade within a time frame typical of their classification, without distinguishing the real health status of every particular machine, that can be very different depending on the working conditions. The result is that unnecessary repairs, because the equip-ment that has suffered the intervention is in perfect working order, and catastrophic
failure are both still possible. This lead to an inefficient use of human and technical resources, and increasing operating costs. Because of the blindness of the scheduled maintenance, the breakdown maintenance cannot always be avoided. This approach is however commonly used in the world of industry and results anyhow three times less costly than the simple run to failure maintenance approach.
• Condition Based Monitoring for Predictive maintenance:
Predictive maintenance is the means of improving productivity, product quality, and overall effectiveness of manufacturing and production plants. Maintenance is performed only when indicated by an expert system that monitors actual operating conditions in order to estimate the health status of the equipment and the MTTF (mean time to failure), also called RUL (remaining useful life). The system, based on the analysis of historical data and real-time monitoring, allows early detection of incipient failure, enabling timely interventions. Including predictive maintenance in a comprehensive maintenance management program optimizes the availability of process machinery and greatly reduces the cost of maintenance. It also improves the product quality, productivity, and profitability of manufacturing and production plants. This approach is a condition based (driven) maintenance and it has to be distinguished to the average-life statistic model seen above. The program can also identify machine problems before they become serious. Most mechanical problems can be minimized if they are detected and repaired early. Normal mechanical failure modes degrade at a speed directly proportional to their severity. If the problem is detected early, major repairs can usually be prevented. Studies on equipment reli-ability problems from last 30 years, wrong maintenance can be said responsible for about 17% of production interruptions or quality problems. As well as the remaining 83% is due to human errors and accidental or unpredictable causes. Inappropriate operating practices, poor design, non specification parts, and a myriad of other non maintenance reasons are the primary contributors to production loss and product quality problems. Predictive technologies based on data analysis, should be consid-ered as plants optimization tool and not exclusively a maintenance management or fault and failures prevention issue. This evolution should take place at corporate level and should be extended throughout plants organization. Predictive technolo-gies are capable to detect, isolate, and provide suggested solutions for deviations from the optimal performance, as in lost capacity, abnormal costs, or a threat to employee safety. So that predictive maintenance represents only in a further step following early detection and diagnostic, being different only from a temporal point
of view. On computer based systems, the features or variables that define the op-erating conditions are automatically acquired and then used by the process control system already used for previous steps of monitoring. The type (string, digital or analogue) and number of variables vary from system to system, but are based on the actual design and mode of operation for that specific type of plant. It is a relatively simple matter to acquire these data from the control system and use it as part of the predictive diagnostic phase. In most cases these data, combined with traditional predictive technologies and the operator experience, provide all of the data one needs to fully represent the system’s performance.
Condition based monitoring is the technique of monitoring a process or the op-erating characteristics of a machine, in the way that trends and changes in those characteristics can be used to predict the health status of the machine. The Condi-tion Monitoring (CM) system can monitor the machine in normal operaCondi-tion without electrical interference, indicating the need of maintenance before catastrophic break-downs occur. The system is composed of four parts:
1. Sensors: convert physical quantities in electrical signals, they should be adapted to the operation condition and suitable for on-line measurement.
2. Data acquisition: built to amplify and pre-process the output signals coming from sensors. In this part data are usually transmitted to a storage with remote access.
3. Fault detection: finds out if an incipient fault is occurring, so that an alarm can be triggered to begin further accurate analysis. Fault detection methods can be divided into two classes, depending on the presence or absence of a proper physical model:
– techniques using only measurable signals (input and output signals from the
process, through statistical control charts and machine learning algorithms
– techniques based on deterministic physical model of the machinery
(know-how tipically owned by the manufacturer).
4. Diagnosys: abnormal signals are identified and post-processed, to make a pre-scription or an indication to maintenance. Advanced signal processing and artifi-cial intelligence technologies are the most suitable techniques. Modern condition monitoring can be for this reason named Intelligent Condition Monitoring.
Table 1.3: General issues of condition monitoring
Task Process Main Techniques Feature Output
What’s the
indication? Data acquisition
Sensor, A/D, data communication (cloud, csv, etc...) On-line Raw data, gross or preprocessed Does any anomaly exist? Fault detection (model based or feature etraction) Signal processing, statistical methods, Neural networs Predictive KPI, thresholds, warnings, new informing variables What’s and where’s the anomaly? Pattern recognition,
classification Neural netorks, fuzzy logic, computer techniques
On-line Suggestion for maintenance or diagnostic What should be done? State assessment, assist decisions
1.3.2 Cost Evaluation of different maintenance methods
The need of a condition monitoring based maintenance is where a device is strictly de-pendent on a single component such as a bearing or a coil, and faults or failures of this component would create a prolonged, unexpected outage involving the main areas of the plant. The cost of such an event depends on different components such as extent of dam-age, time of forced outage to repair it and the cost of substitution if needed. Meanwhile the decision about which kind of maintenance can be adopted depends strictly on knowing how often this sort of outage happens and having a precise knowledge of the factors referred to earlier. In the case these outages were likely to happen, the appropriate preventive action could be taken. In conclusion, the monitoring of plant condition and the consequent cash flow are determined by practical observations and analyses of machine performance data, such as the following:
• Frequency of breakdowns
• Randomness of breakdowns
• Need for repetitive repairs
• Seriousness of fault or failures
• Potential dangers
Figure 1.11: Cash flow diagram for lost production.
Figure 1.12: Cost of a preventive scheduled maintenance activity.
1.4
Fault modes in hydroelectric power plants and related
diagnostics approaches
Hydro power plants consist of three sections: turbine, generator and transmission system. Information about faults and failure of turbine, generator and transmission system is generally kept confidential by the plant management and by the machine manufacturer, therefore not all the cases have been reported and analyzed in literature, especially in
Figure 1.13: Potential savings using condition based monitoring maintenance.
the recent years. Catastrophic failure in hydro power plant are rare, but potentially very cost effective[11]. Fault tree analysis results from literature to be applied in many kinds of technical processes to improve operation reliability and safety.
1.4.1 Turbine faults
Classification of turbines is crucial to differentiate the failure mechanism that the turbine may experience. Depending on the turbine type used, the failure modes of cavitation, ero-sion, fatigue and material defect may affect the impulse and reaction turbine differently (material defects refer to defects generated in the turbine components during the installa-tion process and not during the manufacture of turbine. It is assumed that once the hydro turbine left the manufacturing site, it is fully checked). For example, a reaction turbine has major probabilities to fail due to cavitation while an impulse turbine is most probable to fail due to erosion. Moreover, the failure due to material fatigue and material defect may depend on the operating condition of the power plant [12].
As said above, most accountable turbine failure are:
Cavitation
The water enters hydraulic turbines undergoes changes in pressure and velocity. Such vari-ations may cause changes in flow characteristics with consequences on turbine performance and useful life.
Cavitation is due to the formation of vapour bubbles and the burst of such bubbles as a result of changes in the fluid pressure at the vicinity of moving vapour bubbles falling below the vapour pressure of the fluid. This is the case for flowing fluids over the surface of a machine component where the dynamic component of the fluid pressure increases due to fluid velocity suppressing the static component. Repeated formation and collapse of these vapour bubbles during the fluid flow deteriorates the surface of the machine components due to pitting action [14]. It has been reported that cavitation causes surface penetration damage of up to 10 mm per year to critical components such as impellors, turbine blades, and casings [15].
The main causes of cavitation in hydraulic turbine may be due to the design profile of the turbine and to the frequent change in the operating condition of the plant to meet various load requirements.
Turbine parts which are more susceptible to cavitation are:
Turbine classification Type of turbine Parts damaged by cavitation Impulse Pelton The bucket, due to the rough surface
caused by the impignement of erosive material from the river. Reaction Francis, Kaplan The edges of the blades,
Cavitation in case of Francis turbine initiates as a result of the changes in the fluid flow velocity as it encounters the blades. The fluid flow which is usually turbulent in nature becomes even more turbulent as the velocity and pressure change over the turbine blades. The increase in velocity changes the level of dynamic pressure and creates different flow patterns over the surface of the turbine blades with respected cavitation damage.
Based on the analysis of vibrations, acoustic noise and hydrodynamic pressures mea-sured, the likelihood of cavitation may be evaluated and monitored.
Online vibration monitoring devices which detects abnormal vibration during plant op-eration is the most effective and common method of controlling cavitation in hydropower plants. The turbine section which experiences cavitation as a result of explosion–collapse of vapour bubbles induces abnormal vibration to the machine. The on-line vibration mea-surements are compared to the standard permissible vibration limit of the turbine section. Any abnormal vibration beyond the permissible limit is internally linked to the machine control system that commands machine shut down to prevent further damage and also for the safety of the operational and maintenance officials in the power house.
Erosion
Erosion is the process of gradual removal of material from the surface of a component as a result of repeated deformation and cutting action. The erosive wear of turbine and its components in hydropower plants occurs as a result of the flow of high velocity and impingement of abrasive sediments on the surface of the turbines – for example the sedi-ment that breaks down the oxide coating layer on the flow guiding surface. Instantaneous breakage of the oxide layers leads to the formation of surface irregularities in the flow guiding surfaces initiating cavitation type effects on the turbine unit.
There are different methods of slowing sediment erosion in hydropower plants. Dam-age due to sand erosion portrays serious issues in hydropower plants due to increased shut down time during maintenance and subsequent revenue losses as a result of the damage. However, the impact of sediment on turbine blades and its components can be minimised to an acceptable limit by: Constructing civil structure such as dam and de-silting cham-bers, monitoring the concentration of sediment flow to power house, coating to improve resistance against erosion. Another method of preventing expensive turbines from damage is to monitor the concentration of the sediment particles. An on-line silt measuring device (part per million – PPM) is installed to measure silt content in the river. The on-line measured data is compared to the permissible silt limit of the plant. If the silt limit is greater than the acceptable PPM value, the sensor directs power house to shut down the machine automatically [13].
Fatigue
Material fatigue is another form of turbine failure mode. The turbine components which are subjected to repeated alternating or cyclic stress below the normal yield strength fail progressively by cracking. The turbine assembly constitute various interconnected compo-nents and as a result, the vibration in one of the members is transferred to others, ensuing deformation in all the components. Furthermore, an additional stress in the affected parts may result in abrupt failure of the component. Analysis of water flow over the turbine surface showed formation of eddy current which initiates vibration and stresses on tur-bine blades and to other components. Turtur-bine materials which are subjected to repeated hydraulic vibration may result into material failure due to fatigue. Similar fatigue initia-tion is also highly noticeable at the corners of the runner of Francis turbines and Pelton buckets. To combat fatigue failure, turbine parts which experience fatigue may be manu-factured by nickel alloy steel with 13% Cr and 4% Ni for Pelton buckets and with 16% Cr and 5% Ni for Francis runners. Fatigue failure in turbines may also be minimised if the proper material selection. Material with good fatigue strength and endurance limits and with sufficient factor of safety may be considered during the design stage. Furthermore, fatigue failure can be avoided by monitoring the vibration level of the turbine unit [12].
1.4.2 Generator faults
Regarding hydro generators, a relevant survey made by Cigre shows statistical information to help evaluate the risks of the main hydro generator failures. The survey was limited to failures that produced forced outages of more than 10 days on hydro generators rated higher than 10 MVA operating for more than 10 years. Table 1.4 summarizes information from sixteen utilities and one manufacturer.
Country No. of No. of Total Power No. of Involved
companies hydrogen. (MVA) incidents Power (MVA)
Australia 1 31 3756 1 145 Germany 5 111 3785 5 597 Norway 2 410 20000 11 1343 Spain 4 409 16384 37 2700 Sweden 5 238 13317 15 804 1199 48692 69 5589
Table 1.4: Interview answers [16]
Failures have been considered from different point of views as the damage, defined as harm or injury to property (a hydrogenerator in our case), resulting in loss of value or the
impairment of usefulness and failure root cause, with the purpose to find root causes to prevent more serious problem on key components.
Four different categories have been established: insulation, thermal, mechanical and bearing, results are showed in fig. 1.15.
Figure 1.15: Failure occurrences, damage vs root causes [16].
The results of the Cigre survey [16] show that insulation damages are the most frequent causes of failure (57%) and those that produce the greater extent of damage (in some cases above 30% of possible causes). The failure root causes distribution is similar to the damage one, but an insulation failure is not always due to an insulation root cause. The Cigre survey shows that sometimes insulation failures originate in mechanical root causes, so during the ongoing mechanism of failure a root cause can derive in a different failure.
Risk reduction methods can be adopted, in order to minimize these root causes. Some root causes (bearing and thermal ones) results to be almast insensitive to the risk reduction methods. But insulation and mechanical root causes are very sensitive to some of the reduction methods (fig. 1.16) such as:
• Major refurbishment
• Improved maintenance techniques
Condition monitoring is the tool used for utilities to slow down processes related to ageing and delay refurbishment. From that research it is surprising that vibration monitor-ing has a great influence on insulation root causes. Partial discharge monitormonitor-ing is another possible method. Monitoring methods diffused in modern plants are:
• Air-gap monitoring;
• Detection of hot spots by cooling air analysis;
• Shaft line / bearing vibration monitoring;
• On line partial discharge monitoring.
Figure 1.16: Influence of condition monitoring methods on insulation root causes [16].
Diffused monitoring methods are:
• for Stator-Winding Faults:
These type of fault includes insulation faults, winding sub-conductor faults and wind-ing faults. Failures due to insulation damages do occur frequently as they are manu-facturing defects, they lead to unbalance in stator currents and changes in harmonic content of air-gap flux.
The indicator of stator-winding insulation faults is an increased partial discharge activity in the machine, so Partial Discharge monitoring results the main tool to implement in condition base monitoring. Interpreting PD records, the health of the stator winding insulation, the PD source location and the root causes of the defect can be discovered, and the right measures can be taken.
• for Rotor-Body Faults:
Caused by centrifugal stress, large negative sequence current transients end eccen-tricity. Eddy current losses due to the negative sequence current can overheat the body and begin a fatigue crack, eccentricity can unbalance magnetic pull and this can
lead to vibration. This type of faults can be revealed through vibration monitoring and air-gap magnetic flux monitoring.
• for Rotor-Winding Faults:
Older generators may suffer from insulation migration that leads to intern-turn shorts, while recent built generators are manufactured properly to prevent insu-lation migration. The consequences of this fault are local overheating and rarely rotor-earth faults. Through the monitoring of air-gap magnetic flux, the number and the location of shorted turns can be revealed.
Vibration monitoring, PD monitoring and Air-gap flux monitoring, are the most used methods in condition monitoring for hydro generators.
1.4.3 Transformer faults
Dealing with transformers, the main components are windings, core, main tank, oil, cooler and On Load Tap Changer (OLTC is a mechanism which allows for variable turn ratios to be selected in discrete steps). From failure statistics [17] the top failure occurrences are: OLTC failure and winding failure of mechanical and electrical nature. Crucial parameters to be monitored during operation are the ageing of oil-paper insulation and the health status of the on load tap changer.
The monitoring for OLTC, whose failures are mostly caused by mechanical faults, fol-lowed by electrical faults as coking of contact, burning of resistors and insulation damages, is set on torque measurement of the motor drive, and on a contact wear model in combina-tion with measurement of load current. Vibracombina-tion monitoring is also an effective method for on-line detection at a moderate cost.
Insulation problems instead are revealed by monitoring temperature, gas in oil, partial discharge and moisture analysis. In particular dissolved gas in oil can give an early detec-tion of an incipient fault. Normally Hydrogen, Oxigen, Carbon dioxide, Carbon monoxide, methane, ethylene and acetylene are analysed. The analyses are conducted over concen-trations and ratios of component gases.
Other monitoring worth quantities are voltages and currents as basic information of the operating condition.
Statistical monitoring techniques
and data driven fault detection
models
2.1
Motivations
The behaviour of a complex system such as a hydro power plant can be analysed using statistical process control (SPC) approaches. In this context, SPC is considered as a syn-onymous of a system able to monitor and define a nominal behaviour of a subsystem, and then able to detect or predict a not-nominal behaviour. Following this very general defini-tion, we can also include new data analysis tools such as Neural networks (especially, Self Organising Maps). From a generic point of view, two main approaches can be identified: univariate or multivariate analysis.
Univariate analysis: only one variable (one signal from a sensor) at a time is
anal-ysed. This is a simple approach but guarantees immediate interpretation of the analysis results, that is a significant change in the signal behaviour.
Multivariate analysis: respect to univariate method, this considers correlation
be-tween signals. Both the univariate and multivariate approaches have typically a two steps procedure. In the first step, a nominal behaviour for the variable (where variable in this context can be a single signal or a KPI defined using a combination of several signals) is identified and tagged as "regular" or "not-regular". To tag the behaviour, a priori domain knowledge from experts is most beneficial, in particular identifying "not-regular" sub-classes that correspond to a specific fault/issue/problem; if no specific domain knowledge is available, a less reliable but still useful approach is to use just statistical information (for
example, the median a confidence interval extracted from a cleaned sample can be used as nominal behaviour estimator, and clustering methods can be used to classify different behaviours. In the second step, the variable behaviour is monitored and if the values shows anomalies with respect to the "regular" tagging, the system is said to be "out-of-control", and a warning is created. If the variable behaviour is identified as one specific sub-class of "not-regular" behaviour, the warning will be created according to the specific problem. Different severity levels, corresponding to different problems, can be added in the warning information, and finally also recommended action to be taken can be provided, both as simple information or also as automatic task in an automatic management system. If no identification is possible, for example because a certain behaviour condition was never occurred before and/or there is no a priori knowledge of the corresponding phase-space, a generic warning is triggered.
In the BDH activity, the candidate used two main approaches:
• Univariate analysis: when the a priori knowledge is available, confidence intervals are assessed for each of the plant-components signal. The confidence intervals are typically conservative, and are used to detect large anomalies with respect to data-sheet-like behaviour. Also, automatically tagging not-regular data such as holes, missing, NaN, None, frozen, and spikes are used to classify the data sample and create a cleanliness-data report, useful to identify those sensors with low performance.
• multivariate analysis: Hotelling’s T2 variable and Self Organizing Map (SOM) approaches were used to extract out-of-control data. Two tools are used to compare the results, coming from different approaches: T2 is based on correlation analysis, while SOM are focused on distance metrics. An artificial intelligence approach based on Self Organizing Maps was chosen because of its wide and effective use found in literature; when dealing with unsupervised classification problem and if the feature (variable) space is heterogeneous and has a great dimensionality.
2.2
Working on data: what is Data Analysis
Data Analysis consists of a set of methods with the goal of extracting meaningful infor-mation from a data set, supporting decision making. Typical procedures are:
Figure 2.1: Steps in Data Analysis.
1. Data collection: Raw data are acquired from the acquisition systems;
2. Exploratory analysis: Useful to detect basic problems, consists in checking time ex-tension, kind of data representation (if averaged, instantaneous or interpolated data), checking time zones. It is also appropriate to check and report missing periods that can affect the data set and report basic statistics as mean, variance, and percentiles. Then a visual analysis can be conducted through plots, histograms and other visu-alization tools.
3. Pre-processing: This phase can be composed by
• Data cleaning: Removing or replacing null values, non-useful or empty variables or sub-samples.
• Data munging: Consists in transforming data type, eventual filtering, creation of indexes.
• Data transformation: A frequent example is dimensionality reduction, it can be performed as feature selection or feature extraction. Raw data can even be normalized ( normalization means adjusting values measured on different scales to a notionally common scale) or new attributes can be computed from available variables to add information.
• Non-regularities detection: Detection and tagging of not-regular data as frozen signals, spikes or outliers.
4. Modelling: Is the crucial step where to apply statistical, AI (Artificial Intelligence) or deterministic physical model if available, in order to extract information according to the initial question.
5. Test and Validation: The model has to be tested on new unseen raw data and compared to other established approaches to be validated.
6. Explanatory activity: Results from the model have to be interpreted and explained studying raw data.
7. Results reporting and storytelling: Through BI (Business Intelligence) interfaces, web applications or report documents.
2.3
Monitoring and diagnostic models based on Statistical
approaches
2.3.1 Statistical Process Control
Statistical Process Control (SPC) is a method of quality control which employs a set of statistical approaches to monitor the characteristics of a process. Key tools like Control Charts (described in section 2.3.2) are commonly used. The general objectives of process monitoring are:
• Routine monitoring, ensuring that process variables are within specified limits.
• Detection of abnormal process operation and diagnose the root cause.
• Preventive monitoring, detecting abnormal situations early enough that corrective action can be taken before the process is seriously upset.
The SPC methodology is based on the fundamental assumption that normal process op-eration can be characterized by random variations about a mean value. If this situation exists, the process is said to be in a state of statistical control (or in control), and the control chart measurements tend to be normally distributed about the mean value. By contrast, frequent control chart violations would indicate abnormal process behaviour or an out-of-control situation. Then, a search would be initiated to attempt to identify the root cause of the abnormal behaviour. The root cause is referred to as the assignable cause or the special cause in the SPC literature, while the normal process variability is referred to as common cause or chance cause. SPC is more of a monitoring technique than a control technique because no automatic corrective action is taken after an abnormal situation is detected.
Confidence interval in Normal and Non-Normal Distribution The normal
dis-tribution, also known as the Gaussian disdis-tribution, plays a central role in SPC. The prob-ability that a random variable x has a value between two arbitrary constants, a and b, is
given by:
P (a < x < b) =
Z b
a
f (x) dx. (2.1) where P () denotes the probability that x lies within the indicated range and f (x) is the probability density function. If the variable x has a normal distribution with a mean µ and a variance σ, the function is written as:
f (x) = 1 σ√2πexp −(x − µ) 2 2σ2 (2.2)
The following probability statements are valid for the normal distribution [19]:
Figure 2.2: Probabilities associated to normal distribution [19].
If the variable has a Normal distribution 99.7% of observations fall within µ ± 3 σ (3 sigma limits). Thus, if observations start overcome these limits then we would suspect that the process is no longer in control. The distribution of the observed variable is not the expected one or something in the system caused the changing of µ or σ.
At the same time, if the observed variable has not a Normal distribution, which is the case of the data set we will work on, a famous result called the Chebyshev’s inequality tells that, irrespective of the distribution, at least 89% of observations fall within the 3-sigma limits. The probability of falling within the 3-sigma limits that increases towards 99.7% as the distribution becomes more and more Normal [21].
Variability in processes: Common and special-cause variation in variables
When trying to understand the variation observed in a run chart, it is useful to begin with the idea of stable, background variation which appears random and is called common-cause variation. Common-common-cause variation is present to some extent in all processes. It is an inherent characteristic of the process which stems from the natural variability in inputs to the process and its operating conditions. When only common-cause variation is present,
adjusting the process in response to each deviation from target increases the variability. This variability is an inherent characteristic of the way the system operates and can only be reduced by changing the system itself in a deep way. On the other hand, for variation which shows up as outliers or specific identifiable patterns in the data, very often turns up a real cause. This latter type of variation is classified as special-cause variation. A proper investigation should take place as soon as possible after the signal has been given by the chart so that memories of surrounding circumstances are still fresh. If the cause can be located and prevented from recurring, a real improvement to the process has been made. Control charts tell us when we have a problem that is caused by a special, external cause. The basic Statistical Process Control concepts and control chart methodology were introduced by Shewhart (1931). The current widespread interest in SPC techniques began in the 1950’s when they were successfully applied first in Japan and then in North America, Europe, and the rest of the world. While the run chart provides a picture of the history of the performance of the process without an analysis, control charts place additional information onto the run chart. Those information are aimed at helping us to decide how to react, in real time, in response to the most recent information about the process shown in the charts.
2.3.2 Univariate methods: Control charts
In a control chart, the behaviour of a certain variable is described as time goes on. Control chart construction is based on the idea that when we observe an attribute with a stable pattern of variation, and control charts signal has a shift from that pattern when things have started to go wrong, or however differently than they used to go. The aim is to generate warning signal early enough to avoid failures, but reactions to common-cause variation are not desired. It is necessary to determine if what is happening is background variation or if the process is starting to go out of control.
Many physical measurements are approximately Normally distributed and we shall improve the approximation by using means of groups of observations. Thus, 3-sigma limits give a simple and appropriate way of deciding whether or not a process is in control. When 3-sigma limits are used, special causes variations can be detected. There are two different uses of control charts:
Phase I: A set of data is collected and analysed (initial study);
