Review on Energy Resilience

DataDrivenBayesianNetworksforReliabilityof.pdf

Home >Computer Science homework help >Review on Energy Resilience

Data – Driven Bayesian Networks for Reliability of Supply from Renewable Sources

Alexandra Ciobanu, Florin Munteanu, Ciprian Nemes, Dragos Astanei “Gheorghe Asachi” Technical University of Iasi, Romania

[email protected]

Abstract– Based on a previously detailed data mining about primary renewable sources concerning solar radiation and wind speed, the authors present some results related to reliability of supply using Bayesian networks (BNs). The paper focuses on how the structure and parameters of BN can be obtained from the data. A short review of learning methods, scoring function types and specific algorithms is included. The case study includes details about Necessary Path Condition (NPC) algorithm as well the reliability of supply (LOLP) with respect of a given load node of the power network considered. The input data are the result of the systematic measurements of meteorological parameters and their conversion to generated power while the load profile is according to that attached to RTS system. The analysis and calculations were performed using Hugin Expert software.

Keywords– conditional probability, Bayesian networks, learning methods, reliability systems, power networks, LOLP.

I. INTRODUCTION

The interest about BNs has grown in recent decades and they became a suitable method of analysis for different areas such as medical diagnosis, availability and reliability, risk analysis, genetics, agriculture, geotechnical, psychology, decision-making procedures, classification, data extraction, etc.. Judea Pearl [1] and F. V. Jensen [2] work about the time- series modeling for Dynamic BNs (DBN) and J. B. Dugan papers about dynamic fault tree models for fault tolerant computer systems [3] and fault-trees and Markov models for reliability analysis of fault tolerant digital systems [4], were just the beginning of a great development of many papers about reliability of power systems, sub-systems and components.

The first functional applications on BNs [5] tended to focus on problems of classical diagnosis in the medical field for the diagnosis of neuromuscular diseases, which led to the creation of the MUNIN system (a probabilistic network for interpretation of the EMG results), by specialists from the Department of Intelligent Machines at the Aalborg University. This subsequently led to the creation of the first instrument to use BNs, namely Hugin Expert software.

In a publication about risk management [6], Norman Fenton and Neil Martin are showing how BNs have been developed in the modern world.

BNs have been used in fields other than medicine by companies like Microsoft and Hewlett-Packard for fault diagnosis, and in particular, widely to diagnose faults printers.

Paper [7] is divided into two parts: I. Bayesian Networks; II. Decision trees, and presents the usual method used for graphical learning models and, at the same time, develops approaches to graphical language for modeling decision problems.

The learning BNs techniques are actually used to generate their structure and calculate the corresponding parameters. This could be extremely useful especially in the case of the large BNs, when manually methods can be difficult at least for some domains. There are some excellent published books dedicated to this aspect [8], [9], [10], [11].

II. BUILDING AND LEARNING BAYESIAN NETWORKS

There are different techniques for manually construction of BNs. In principle, the necessary steps are:

- establishing a causal model of the domain to study; - constructing a conditional independence graphical

model; - decomposition of the distributions; - bringing out the corresponding evidence propagation.

For reliability analysis, the practical techniques are based on experts knowledge, technical design model [12], [13], reliability equivalent block diagrams, Markov chain models.

In the case of large networks automatically methods are suitable for learning BNs.

1. Learning Bayesian networks Learning a Bayesian network means to infer both its

structure and parameters using a data set of cases as well the domain expert prior knowledge. This is not a simple task due to the fact that an enormous number of structures can approximate a given reach data set.

Fortunately, there are some learning techniques and algorithms capable to simplify this process while balancing the accuracy of results and computing efforts.

Fig. 1 shows the modeling (learning) process. Unfortunately, whole specific literature dedicated to

Bayesian networks as an artificial intelligence method is presenting the learning Bayesian networks process only based on examples (Asia, Apple Jack, etc.) proving it is a very complex, difficult to synthetically explain.

)|(P)|(P),|(P)(P)(P

),,,,(P

BCDEDABDA

EDCBA

 

))(|(P),...,,..,,(P 1

21 i

i ini XXXXXX 





Fig. 1. The essence of Bayesian networks learning process

An ideal situation can be when a strictly correlation

between a Bayesian network structure like that in Fig. 2 and (1) encoding it. Equation (1) gives the joint probability distribution related to network in Fig. 2, where variables B, E, C are conditionally depending on their parent variables.

Fig. 2. Structure of the Bayesian network implicitly encoded by (1)

(1)

Generalizing, (1) becomes

(2)

where П are parents of (Xi).

This example is not useful for learning BNs excepting the case when the data are proving the marginal probabilities, conditional probabilities and dependencies between random variables are encoded in (1).

Usually, a data set allows for different network structure. Selecting the optimal one is based on suitable classifiers. Adding the fact that precise probabilistic inference is NP-hard (nondeterministic polynomial-time hard) which means there is no general efficient algorithm [14], the BNs inference and learning are relative difficult imposing compromising solutions.

In principle, there are two categories of techniques for learning structure of BNs:

- constrained based techniques which means to establish a structure having independence constraints matching those detected in the data;

- scoring techniques meaning to detect a structure that can represent probability distributions matching the data, or generated by data; scoring functions are usually classified in Bayesian and information-theoretic.

The problem to find the best graph is very difficult because the number of possible BN structures is super exponential in the number of variables.

Inference tasks are: - to calculate the posterior probabilities of an unobserved

variable given evidence(s); - to calculate the most likely joint state of all unobserved

variables ≡ most probable explanation (MPE); - to compute the most likely joint state of some

unobserved variables ≡ maximum a posteriori assignment (MAP);

- to calculate the most relevant partial explanation for the evidence ≡ most relevant explanation (MRE).

Generally, given a data T = {y1, . . . , yN} and a scoring function ϕ, learning a Bayesian network means to find a Bayesian network nBB  that maximizes the value ϕ(B, T) Optimization of ϕ involves heuristic or metaheuristic search over a some restricted space defined by network structures, equivalence classes of these structures and allotment over the network variables.

Concerning the algorithms for space searching, well known are greedy hill-climbing, simulated annealing, genetic algorithms or tabu search, etc.

2. Scoring functions The scoring functions, offering ranking criteria to select the

right network fitting the data set, are: - Bayesian: pure Bayesian (B), Bayesian Dirichlet (BD)

[15], K2 type [16], Bayesian Dirichlet considering likelihood equivalence and possible structure (BDe) [15], BDeu scoring function, a variant of BDe [17];

- information-theoretic scoring functions: log-likelihood (LL), minimum description length/Bayesian information criterion MDL/BIC, Akaike information criterion (AIC), normalized minimum likelihood (NML), mutual information tests (MIT), etc.

To better understand and compare the scoring functions some details they are presented as it follows.

a) B scoring function means to calculate P(B|T) for all possible networks and select the best network having maximum posterior probability; to take the advantage of working in logarithmic space, log(P(B,T)) will be used and not P(B,T).

b) BD scoring functions are defied on some important assumptions presented in [15] and it is given by:

(3)

where Г is Gamma function, P(B) is the prior probability of network B, ri number of states of the finite random variable Xi, i=1, ...., n; qi, Nij, given by (4), are the number of possible

configurations of the parent set iX of Xi, and number

Inferred Bayesian model

Data base (set of data cases)

Prior (experts) knowledge

The process to model







 





k ijkijkijk

j ijijij

NNN

NNNBTB

1 1

))](/)((

))(/)([()(P),(P

)|,(P'' GwXxXNN ijiikiijk 

)/(1)|,(P iiijiiki qrGwXxX 

appearances in the data T where the variables in iX take their j-th configuration wij (1 ≤ j ≤ qi) respectively while Nijk is the number of appearances in the data T where the variable Xi

takes its k-th value xik and the variables in iX take their j-th configuration wij.

(4)

A new BD scoring function was presented in [5]:

(5)

c) K2 scoring function [16] is a specific case of the BD:

(6)

considering corresponding to zero pseudo counts. d) BDe scoring [15] function has the same form like (3).

The difference is made through some new assumptions compared to BD: likelihood equivalence and structure possibility. The first one establishes that two directed acyclic graphs are equivalent if they encode the joint probability distributions. Given a Bayesian network B, data T can be seen as a multinomial sample of the joint space D with parameters

nirx iixxD n ,...,1,,....,1}θ{ ......1 

where

   n

i xxx ixin 1 |,.... θθ

This is based on an assumption of BD scoring function: given a directed acyclic graph G such that P(G)>0 then Θij is Dirichlet distributed for all Θij in ΘG. Given two directed DAGs, G and G' such that P(G)>0 and P(G')>0, if G and G' are equivalent, then ρ(ΘD|G) = ρ(ΘD|G').

The second assumption assumes for any complete DAG G we have P(G)>0 (a DAG is complete if its skeleton in complete).

Finally, the BDe score function is also given by (3) supposing that ρ(ΘD|G) is Dirichlet with equivalent sample size N' for some complete DAG G in D. This time, in (3),

The equivalent sample size N' shows the belief degree in the prior distribution.

e) BDeu score function [17] is a particular case of BDe score and is given by

(7)

available if . BDeu score is depending on one parameter, the equivalent sample size N' which needs to be carefully selected and repeated.

f) Concerning the information theoretic type of scoring functions, these are useful and complex and this paper is not covering them. Only mutual information tests (MIT) method is briefly introduced here because it seems to be of a great interest [18], [19], [20].

Basically, the mutual information between two variables Xi and Xj are

(8)

The MIT score [19] is given by

(9)

where I(Xi; ПXi) is the mutual information between Xi and ПXi in the BN measuring the relationship between each variable and its parents. The second term means the penalization related to the Pearson χ2 test of independence, α is a free parameter showing the confidence level related to statistical test, ))(σ),...,1(σ(σ *** iiii s denotes any permutation of the index set (1, ...., si) of the variables in ПXi = (Xi1, ...., Xisi) with where rij is the number of possible configurations when the parent set of Xi is limited only to Xj. The freedom degree number

corresponds to

3. Learning algorithms Correlated with the practical case study presented in this

paper and with the Hugin Expert software used for BN construction and learning, the PC and NPC constrained algorithms are shortly introduced here as they are clearly explained in [21].

The PC algorithm is based on the next steps: - testing the (conditional) independence between each pair

of variables represented in D (database of N cases D = {c1, . . . , cN} over n variables X = {X1, . . . , Xn}) to derive MD

 

 i

iXj

k ijkij

X ji NNrq







 







k ijk

j iiji

rNr

BTBK

1 1

)}!log(

])!1/()!1{log[(

))(Plog(),(2







 







k ijkijkijk

j ijijij

NNN

BTB

1 1

)]}(/)(log[

)](/()({log[

))(Plog(),(BD







 







k iiiiijk

j iiji

qrNqrNN

qNNqN

BTB

1 1

)]}/(/)/(log[

)]/(/)/({log[

))(Plog(),(BDeu

 i jx x

jiijiji xxxxxXX )](P)(P/((Plog[),(P),(MIT

  

    

   

 

j jiXi

ii lXNITB

1 1 )(σα *

,χ);(2)|(MIT

)(σ* ji i l



  



 

 



1)1)(1(

,...,2)1)(1(

)(σ

1 )(σ)(σ

)(σ *

jrr

sjrrr l

jii

k ikijii

1' ijkN

)(σ)2(σ)1(σ *** ...

iiii siii rrr 

(the the set generated by conditional independence tests) of CIDRs (conditional independence and dependence relations);

- identification of the skeleton of the graph induced by MD; - identification of colliders in the graph (edges to the same

node); - identification of derived directions of edges. The PC algorithm generate a partially directed acyclic

graph (PDAG) representing an equivalence class. The necessary path condition (NPC) algorithm includes an

additional feature in for the case of absence of an edge. It says that in “order for two variables X and Y to be independent (in a DAG faithful data set) conditional on a minimal set SXY, there must exist a path between X and every ZSXY (not crossing Y) and between Y and every ZSXY (not crossing X). Thus, in order for an independence relation to be valid, a number of edges (or paths) are required to be present in the graph. This is the Necessary Path Condition”.

III. CASE STUDY: BAYESIAN NETWORK FOR CALCULATION OF THE RELIABILITY INDEX LOLP

USING DATA – DRIVEN MODELING

In paper [22] the importance of Bayesian networks modeling in nodal power quality is presented, with a view to detect the availability of the renewable sources: wind turbines and solar panels.

The authors have been analyzed a part of the power network shown in Fig. 3, supplied from renewable sources with respect of load considering the up-stream components failures, short-circuits as well as correlated sources reliability: S-solar and W – wind.

Fig. 3. Reference power network considered for interruption risk analysis

with respect of load point L: S-solar source; W-wind source; R-circuit- breaker (recloser) reliability equivalent

In a research stage at Aalborg University, Denmark, has been developed a Bayesian network model for analyzing the reliability to supply the consumers using only renewable energy sources (solar and wind generators) and not taking into account the existence of the grid.

For this implementation, the authors have used information from experts in the field, information from the specific literature about BN’s and a database comprising values of wind speed and solar radiation.

The database comes from a professional meteorological station Vantage Pro 2 Plus [23], located near the laboratory

LACARP – Laboratory for Applied Research and Prototype Design of the Power Engineering Department at "Gheorghe Asachi" Technical University of Iasi.

The station can record daily, monthly or yearly data concerning simple weather parameters like wind speed, solar radiation, temperature, humidity, barometric pressure, rain fall, etc. or composed indices: dew point, UV radiation dose, heat index. Suitable graphs or tables are indicated on the console or on the connected computer screen.

The authors have used a database with minute by minute recorded values for five months of 2013.

Hugin Expert software was used as an instrument for data processing, generating models and probability calculations.

Fig. 4 represents the starting point of the developed model.

Fig. 4. Structure of the initial Bayesian network

Starting from the initial Bayesian network structure (Fig. 4) has been built using the Hugin Expert software package a model where:

 W (Wind source) was introduced as probability distribution function for wind speed: type and parameters resulting from data  S (Solar source) was introduced as probability

distribution function for solar radiation: type and parameters resulting from data  PW (Wind Power) represents the power generated by a

wind turbine having the rated power 10 kW. The power generated as a function of wind speed is given by (10).

 



 





 



 







  

 

 



elseother0

forP

for )(P

)(PW offcutratedrated

ratedincut incutrated

incutrated

www

www ww

w (10)

 PS (Solar Power) represents the power generated by a solar panel with a rated power of 10 kW. The power generated as a function of solar radiation is given by (11), where: Am is the array area, Nm is the module reference efficiency (assumed to be 0.11), Pf is the packing factor (assumed to be 0.9), Npc is the power conditioning efficiency (assumed to be 0.86), and S is the solar radiation contained in the data.

SPS  pcfmm NPNA (11)

 PR (Renewable Power) represents the power generated by renewable sources calculated by summing the upper variables.  PL (Load Power) represents the requested power by

the consumer and it is expressed in p.u. with respect to rated power.  LOLP (Loss of Load Probability) is calculated as the

probability that PR<PL. The network was built by inserting the available data about

wind speed and solar radiation, data belonging to 5 months of the year 2013 as shown in Fig. 5 and Fig. 6.

Fig. 5. Inserting database in Hugin Expert software

The data file was processed and the variables discretized in

classes of variables to minimize the number of states from the corresponding CPT (Conditional Probability Table).

After processing the data it was necessary to establish the constraints between variables as shown in Fig. 7.

Fig. 6. Inserting 5 months of data file

The Hugin Expert software is able to learn from the data

that we have inserted using learning algorithms to identify the DAG structure that best encodes a set of conditional dependence and independence relations (CDIR’s) that are derived from the data source by statistical tests.

In this model the NPC algorithm (Necessary Path Condition) was used for structure learning, algorithm that introduces the concept of an ambiguous edge.

When using the constraint-based NPC algorithm for learning the Bayesian network structure, is important to decide on a significance level α that will be used in the statistical tests.

The authors have used a significance level that is not uncommon, α = 0,05. The NPC algorithm performed a sequence of statistical tests for pairwise (conditional) independence.

Fig. 7. Establishing constraints in Bayesian network

This algorithm says that in order for two variables

“WindSpeed” and “SolarRad” to be independent conditional on a minimal set “Date, Time, DayType”, there must exist a path between “WindSpeed” and every “Date, Time, DayType” (not crossing “SolarRad”) and between “SolarRad” and every “Date, Time, DayType” (not crossing “WindSpeed”), as shown in Fig. 8.

More specific, the algorithm ensures that some inconsistencies in the test results are reported and the user has the option to make decisions on the edges to include in the network in order to resolve those ambiguous regions.

Fig. 8. NPC algorithm in Hugin Expert The generated Bayesian network contains variables that are

described in the data. To calculate the LOLP index, other auxiliary nodes were defined, nodes with constant states and values like: Vr, Pr, Am, Nm, Pf, Npc and Load Power.

The values for “LoadPower” node were taken from the RTS and inserted as p.u values in the corresponding CPT.

The variables “WindPower”, “SolarPower”, “RenewablesPower” and “LOLP” are conditionally depending on other nodes and they were calculated by inserting in the program (10), (11).

For calculating „WindPower”, (10) was translated as: - if (and (wcut-in < WindSpeed, WindSpeed < Vr), Pr *

((WindSpeed - wcut-in) / (Vr – wcut-in)), if (and (WindSpeed > Vr, WindSpeed < wcut-off), Pr, 0))

where: Vr =wrated = 7 m/s; WindSpeed = w (database); wcut-in = 4 m/s; wcut-off = 20 m/s and Pr = Prated = 10 kW.

For calculating “SolarPower” we have translated (11) as: - if (SolarRad > 0, Am * Nm * Pf * Npc * SolarRad, 0)

For calculating “RenewablesPower” we have summed the values from the above mentioned power sources and the Loss of Load Probability “LOLP” was calculated comparing the generated power renewable sources to the requested load power.

The corresponding learned structure generated is the Bayesian network depicted in Fig. 9.

Fig. 9. Bayesian network structure generated by the Hugin Expert

software

The case study results are presented as it follows based on the idea to see if and when two cumulated renewable power sources can meet the power demand.

The maximum of energy production from solar panels corresponds to a power between 1, 5 – 2 kW with a probability of 36% and is obtained when solar radiation exceeds 244 W/m2.

Wind turbine is generating a maximum power between 9 – 10 kW with a probability of 32% when wind speed reaches its rated value of 7 m/s.

Higher wind speed and solar radiation are generating the same amount of energy but with higher probabilities.

Combined these two power sources can produce enough power to maintain the demand only when the wind turbine or both of them are reaching their maximum energy production.

IV. CONCLUSIONS

Bayesian networks are a useful tool for monitoring and diagnosing the state of a supply network. Being able to use data – driven models we can predict future availability conditions for reaching energy demand.

The developed model is the result of a four months research stage within an Erasmus+ Program at the Aalborg University, Denmark.

The current paper is one of the results of the research on availability supply from renewable sources of a standard consumer. The results showed that a 10 kW wind turbine and a 10 kW solar panel can maintain the necessary energy request with a corresponding probability.

The maximum of energy production from solar panels is achieved when solar radiation exceeds 244 W/m2.

The maximum of energy production from wind turbine can be achieved with a probability of 32% when wind speed reaches its rated value of 7 m/s.

Future work will be dedicated to more complex modeling using data – driven models involving more variables that are conditional depending on others. Using Bayesian networks for fault diagnosis on electricity distribution networks, power transformers monitoring or for modeling complex nodal architectures is an important task also.

REFERENCES

[1] Pearl, J. (1993), “Graphical models, causality, and intervention”, Statistical Science, Vol. 8, no. 3, pg. 266 – 273.

[2] Jensen F., (2001), “Bayesian Networks and Decision Graph”, Springer.

[3] Dugan J.B., Bavuso S.J. and Boyd M.A. (1992), “Dynamic Fault Tree models for Fault Tolerant Computer Systems”, IEEE Trans. Reliability, vol. 41, 363-377, DOI: 10.1109/24.159800.

[4] Dugan J.B., Bavuso S.J. and Boyd M.A. (1993), “Fault-trees and Markov models for reliability analysis of fault-tolerant digital systems”, Reliability Engineering and System Safety 39, 291-307.

[5] Needham J.C., Bradford R.J., Bulpitt J.A., Westhead R.D. (2007), ”A Primer on Learning in Bayesian Networks for Computational Biology”, PLoS Computational Biology, Volume 3, Issue 8.

[6] Fenton N. and Neil M. ( 2007), “Managing risk in the modern world. Applications of Bayesian Networks”, A Knowledge Transfer Report from the London Mathematical Society and the Knowledge Transfer Network for Industrial Mathematics London Mathematical Society De Morgan House, 57/58 Russell Square London WC1B 4HS.

[7] Jensen V.F., Nielsen D.T. (2007), “Bayesian Networks and Decision Graphs”, Springer Science, 445 pgs. ISBN 978-0-387-68281-5.

[8] Neapolitan E.R. (2003), “Learning Bayesian Networks”, Pearson Publisher, April 6, 674 pgs. ISBN 978-0130125347.

[9] Barber D. (2012), “Bayesian Reasoning and Machine Learning”, Cambridge University Press, 1st edition March 12, 735 pgs., ISBN 978- 0521518147.

[10] Chickering D.M., Heckerman D., Meek C., & Madigan D. (1994), “Learning Bayesian networks is NP-hard”, Technical Report MSR-TR- 94-17, Microsoft Research, Microsoft Corporation, Redmond, Washington.

[11] Darwiche A. (2009), “Modeling and Reasoning with Bayesian Networks”, Cambridge University Press, 548 pgs. ISBN 978-0-521- 88438-9.

[12] Munteanu F., Ciobanu A. and Nemes C. (2016), "From technical design structures to Bayesian networks in power engineering", International Conference on Applied and Theoretical Electricity (ICATE), Craiova, Romania, 2016, pp. 1-6. doi:10.1109/ ICATE.2016.7754625.

[13] Ciobanu A., Munteanu F. and Nemes C. (2016), “Bayesian networks utilization for reliability evaluation of power systems”, International Conference and Exposition on Electrical and Power Engineering (EPE), Iasi, Romania, 2016, pp. 837-841.doi: 10.1109/ICEPE.2016.7781454.

[14] Cooper G. F. (1990), “The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks”, Artificial Intelligence, Vol.42, no. 2-3.

[15] Heckerman D., Geiger D., Chickering D.M. (1995), “Machine Learning”, Vol.20, Sept, pg. 197-243. doi:10.1023/A:1022623210503.

[16] Cooper G.F., Herskovits E. (1992), “Machine Learning”, pg. 309-347. doi:10.1007/BF00994110.

[17] Buntine W. (1991), “Theory refinement on Bayesian networks”, Proceedings of the 7th Conference on Uncertainty in Artificial Intelligence, Los Angeles, CA, July 13-15, pg. 52-60.

[18] Vinh NX., Chetty M., Coppel R., Wangikar P.P. (2011), “Global MIT: learning globally optimal dynamic Bayesian network with the mutual information test criterion”, Bioinformatics, Oct. 1, 27(19), 2765-6. doi 10.1093/bioinformatics/btr457.

[19] L.M. de Campos (2006), “A scoring function for learning Bayesian networks based on mutual information and conditional independence tests”, Journal of machine learning research, pgs. 2149-2187. ISSN 1533-7928.

[20] Silva J.; Narayanan S.S. (2010), “Nonproduct Data-Dependent Partitions for Mutual Information Estimation: Strong Consistency and Applications”, IEEE Trans. Signal Process, 58, 3497–3511.

[21] Kjærulff U.B., Madsen A.L. (2008), “Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis”, Springer-Verlag, New York, ISBN 978-1-4419-2546-6.

[22] Munteanu F., Nemes C. (2012), “Belief networks utilization for nodal power quality and availability assessment”, U.P.B. Sci. Bull., Series C, Vol. 74, Iss. 1, ISSN 1454-234x.

[23] Ciobanu A., Munteanu F., Nemes C. and Astanei D. (2016), "Availability model of wind and solution to immunize the generators against short time perturbations", 2016 International Symposium on Fundamentals of Electrical Engineering (ISFEE), Bucharest, pp. 1-6. doi: 10.1109/ISFEE.2016.7803166.