A formal approach to automatically assess and manage ICT risk

(1)

UNIVERSITY OF PISA

DEPARTMENT OF COMPUTER SCIENCE

Ph.D. Thesis

A formal approach

to automatically

assess and manage ICT risk

Committees: Prof. Emil Lupu

Prof. Salvatore Ruggieri

Candidate: Tonelli Federico

(2)

(3)

Acknowledgements

•

Ringraziamenti

(rigorously and only in Italian)•(rigorosamente e solo in italiano)

Sin da quando ero piccolo ho sempre voluto "spippolare" con i computer, ma mai avrei pensato di arrivare fino a questo punto. Ottenere un dottorato è un per-corso faticoso fatto di sacrifici, di momenti bassi e qualche soddisfazione. Ci sono molte persone che ruotano intorno a te, qualcuna che viene, qualcuna che va, e qualcuna che, forse, dopo essersene andata, tornerà. La soddisfazione di essere ar-rivato a questo punto è tantissima così come sono tantissime le persone che vorrei ringraziare o a cui vorrei porgere almeno un saluto. Mi perdonerete, quindi, se utilizzerò queste due paginette per ricordare ogni persona che, in questi anni, ha significato qualcosa per me.

Strano come i ringraziamenti siano, almeno per me, una delle parti della tesi più difficile ed emozionante da scrivere. Ripenso ad ogni singola persona che ha passato un po’ di tempo con me, le risate, i pianti, le gioie ed i dolori condivisi assieme e devo capire se voglio citarla, ma sopratutto cosa di quella persona voglio far ricordare alle altre che leggeranno queste poche righe, magari curiose di vedere se mi sono ricordato di loro.

Chiaramente, la parte dei ringraziamenti più facile da scrivere è quella verso la persona che, sin dalla tesi triennale, mi ha seguito, ha creduto in me, mi ha istruito e mi ha messo a disposizione tutta la sua esperienza e conoscenza. Parlo del Prof. F. Baiardi, al quale va tutta la mia stima e la mia gratitudine. Prima di tutto perché mi ha insegnato il pensiero critico, che non è una cosa facile da avere in questi tempi, e poi perché mi ha trasmesso la passione che oggi ho per la sicurezza informatica.

Alla mia famiglia dedico questo lavoro, i miei genitori ed i miei nonni, poiché è sicuramente grazie a loro che sono potuto arrivare dove sono oggi. Ringrazio mamma Serena e babbo Claudio che mi sono stati vicini e mi hanno supportato, sopportato e viziato, amorevolmente, in ogni momento della mia vita, soprattutto negli ultimi periodi più difficili. Grazie per avermi insegnato a sognare, a credere e a non mollare. Mai.

(4)

contorto dei miei pensieri, vorrei ringraziare tutte le altre persone. Ognuno dei qui presenti citati è stato per me importante in maniera diversa quindi non ci sono né classifiche né graduatorie.

Un grande grazie lo dedico a Marco non solo perché non c’è stato un giorno in cui lui non abbia creduto in me, ma anche perché c’è sempre stato in ogni momento in cui io ne abbia avuto la necessità. Gli devo molto.

Ringrazio Rita, una parte molto importante della mia vita; lei è stata quella che si è sorbita tutto il peggio di me e mi ha ricordato, ogni giorno, l’importanza ed il valore delle mie scelte.

Ringrazio Fabio, il compagno delle avventure universitarie e post-universitarie più belle, quelle che lasciano il segno e che non si possono dimenticare. Ed in fondo parte di questa ricerca è merito anche suo. Il suo abbandono ha segnato comple-tamente la mia vita accademica. Lo ringrazio anche perché mi ha insegnato come il rispetto, per qualcuno, sia importante.

Ringrazio Eva, la quale, nell’ultimo periodo, mi ha saputo con pazienza ascoltare, capire e consigliare. Devo a lei la mia dedizione ed il rispetto di alcune regole, in particolare la 28, la 39 e la 45...

Ringrazio Alessandro, Lorenzo e Jacopo, elencati qua in ordine di come ho avuto il privilegio di conoscere. Colleghi, nonché amici e compagni di ricerca, grazie alle loro capacità ed ai loro sforzi questa ricerca è potuta diventare quella che adesso è descritta in questa tesi. E forse anche di più...

Un caro saluto va al mio compagno di stanza, perché nei pochi momenti in cui abbiamo avuto la fortuna di incontrarci mi ha sempre saputo dare una spinta in avanti e mi ha fatto capire quanto fosse importante quello che stavo facendo. A lui auguro ogni bene.

Dedico affettuosi saluti a tutti gli amici dottorandi che, chi più chi meno, hanno condiviso con me questo percorso ed auguro un grosso in bocca al lupo a chi ancora vede la fine di questo percorso come un lontano puntino bianco.

Infine, saluto con gioia anche "i ragazzi del lab" in particolare Andrea e Michael. La loro allegria e compagnia è sempre stata piacevole, e sempre ben accetta, in questi anni. A loro auguro di avere tutto il successo e la fortuna che si meritano. Tutti, uno ad uno, giorno dopo giorno, mi avete fatto diventare quello che oggi sono.

Grazie.

(5)

List of Tables

2.1 SCAP components enumeration . . . 13

3.1 List of parameters that characterize a system . . . 39

3.2 List of parameters that characterize vulnerabilities and attacks 43

3.3 List of parameters that characterize an agent . . . 45

5.1 Subclass properties in vulnerability classification . . . 94

6.1 Locked Shield 2014: assessment results according to standard vulnerability scanners . . . 152

6.2 Locked Shield 2014: Manager’s results with maximum priority153

6.3 Locked Shield 2014: Manager’s results with low priority . . . 154

7.1 Hydroelectric ICS: percentage of vulnerability classes accord-ing to Vulnerability Builder . . . 180

7.2 Hydroelectric ICS: assessment results according to λ(ag) = 1 181

7.3 Hydroelectric ICS: assessment results according to λ(ag) = 2 182

7.4 SmartEnergy ICS: assessment results according to standard vulnerability scanners . . . 187

7.5 SmartEnergy ICS: assessment results according to Haruspex results . . . 188

7.6 GasDistribution ICS: results of Haruspex experiments includ-ing web-environments . . . 193

(11)

7.7 MailProvider ICT: assessment results according to standard vulnerability scanners . . . 198

(12)

List of Figures

2.1 The metric groups of CVSS . . . 14

2.2 A simple IDS architecture . . . 23

2.3 The general SIEM architecture . . . 29

3.1 Transition from physical to logical topology . . . 41

3.2 The automa of the agent . . . 47

3.3 An example of two attack sequences . . . 49

3.4 An example of how λ(ag) works. . . 51

4.1 The SIEM framework architecture . . . 68

5.1 An overview of the Haruspex suite . . . 87

5.2 Reducing misclassification through CVE Details. . . 95

5.3 Relations among components . . . 99

5.4 A model of application environments to describe web vulner-abilities . . . 102

5.5 First version of the ICS . . . 113

5.6 Second version of the ICS . . . 114

5.7 Third version of the ICS . . . 115

5.8 First ICS version: most dangerous agents’ stress curves . . . 116

5.9 Second ICS version: most dangerous agents’ stress curves . . 117

5.10 Third ICS version: most dangerous agents’ stress curves . . . 117

5.11 Robustness of the three ICS versions . . . 118

5.12 Labyrinth: network topology . . . 125

5.13 Labyrinth: stress curves of the four agents in terms of time . 126 5.14 Labyrinth: stress curves of the four agents in terms of attacks127 5.15 Labyrinth: stress curves at distinct iterations, maxP rob agent128 6.1 Locked Shield 2014: network topology . . . 151

6.2 Locked Shield 2015: network topology . . . 156

(13)

6.4 Locked Shield 2016: a detail of the SCADA system . . . 163

6.5 Locked Shield 2016: a detail of the server room handled by the PLC . . . 164

7.1 Thermoelectric ICS: network topology for thermoelectric pro-duction . . . 171

7.2 Thermoelectric ICS: most dangerous AS agents’ stress curves according to the steps . . . 173

7.3 Thermoelectric ICS: most dangerous AS agents’ stress curves according to the time. . . 173

7.4 Thermoelectric ICS: most dangerous AP agents’ stress curves according to the steps . . . 174

7.5 Thermoelectric ICS: most dangerous AP agents’ stress curves according to the time. . . 175

7.6 Hydroelectric ICS: network topology for hydroelectric pro-duction . . . 178

7.7 Hydroelectric ICS: stress curves of an insider agent . . . 183

7.8 Hydroelectric ICS: stress curves of an external agent . . . 183

7.9 SmartEnergy ICS: network topology for an experimental smart energy production . . . 185

7.10 SmartEnergy ICS: stress curves of the agents according to the time . . . 189

7.11 GasDistribution ICS: network topology for gas distribution . 191

7.12 GasDistribution ICS: a detail of the network topology for gas distribution . . . 192

7.13 GasDistribution ICS: some agents’ stress curves according to the time . . . 194

7.14 GasDistribution ICS: some agents’ stress curves according to the time . . . 195

(14)

Chapter 1

Introduction

This section informally describes the main problems this thesis addresses, namely the automation of ICT risk assessment and management, and the various issues this problem poses.

To assess and manage the risk posed by a complex ICT system, first of all we consider that this system consists of several nodes. Some of these nodes interface the system users while other ones compute and store the information of interest. To implements its functions, each node runs several software modules at distinct abstraction level. The whole system is inho-mogeneous and there is no a direct physical connection from the nodes that interface the user to the most critical ones. An intelligent attacker targets this system to achieve some predefined goals, each consisting of a set of privileges, e.g. access rights, on some components. These privileges enable the attacker to read, modify and steal some information the system man-ages or to drive the behavior of some physical devices the system controls. To acquire all privileges in a goal, one attack rarely suffices. Hence, the attacker needs to implement a sequence of attacks that escalate its privi-leges on the various components till it acquires all those in its goals. An attacker implements most of these attacks only to acquire the rights to ex-ecute further attacks till it can implement those that grant the privileges in its goal. Obviously, several distinct sequences can grant the same set of access rights and these sequences may share some attacks. Furthermore, distinct attackers may implement the same attack to reach some distinct goals. Each attacker selects the sequence to implement according to its own

(15)

1 – Introduction

priorities and its risk attitude. All these factors strongly increase the com-plexity of computing the probability that an attacker reaches some or all its goals. It is also worth noticing that an approximation of this probability may be of little use if it does not enable the system owner to estimate in a realistic way the expected impact of an attacker. For the same reason, even an estimate of the average success probability may be of little utility.

The work described in this thesis aims to define an automatic quanti-tative approach to assess and manage the risk due to some attackers that target a complex ICT system to reach their goals. To manage the risk, the proposed approach computes a set of countermeasures to minimize the attacker success probabilities in a cost effective way. We are interested in defining a model based approach to assess and manage the risk without dis-turbing the target system. Such an approach defines and applies a model of the system and of the attackers. The system model focuses on the de-scription of the component vulnerabilities, the attacks they enable and the countermeasures that may be adopted and deployed. For each attack, the model describes the privileges it grants and those that an agent needs to implement it. Instead, an agent describes the attacker goals and how it se-lects the attack to implement among those it may select according to both the component vulnerabilities and its current privileges.

The interactive execution of the models of the system and those of the agents of interest implements a simulation of the attackers that describes how they select and implement their attacks against the system in the time frame of interest. However, the output of a single simulation is seldom significant due to the various source of randomness that affect the agent behavior such as the success probability of attacks, the sequence each agent selects, the effectiveness of the deployed countermeasures. We solve these problems by adopting a Monte Carlo approach that runs several model executions and that returns a statistical sample on the agent attacks. The sample is built by collecting, in each simulation, information on the attacks the agents have selected and implemented, the goals they have reached, and the time this takes. An assessment uses this sample to compute the statistics of interest to evaluate and manage the risk. Obviously, both the accuracy and the completeness of these statistics strongly depend upon those of the models of the system and of the attackers. This favors the design and the implementation of tools to automate the building of these models according to the critical features of the target system and of the attackers.

(16)

1 – Introduction

One of the key advantages of the proposed approach is that it replaces the historical data that is usually adopted to evaluate risk with synthetic data generated through the modeling of the system to be assessed and of its attackers. This implies that it can evaluate and manage the risk before deploying the system provided that it can build accurate models. In this way, the owner of a system can adopt a proactive attitude that assesses the risk as a step of the system design rather than after a successful intrusion. This thesis also considers the problem of real-time protection. While current Intrusion Detection and Prevention Systems only considers each at-tack in isolation, they have to face a much more complex scenario where each attack is a step of an escalation. The thesis advocates an alternative approach that uses the output of the Monte Carlo method to forecast the attacker behaviors and the attacks they will select and execute. The pro-posed approach assumes that the system to be protected includes a Security Information and Event Management (SIEM) that consists of a network of sensors and a central entity that collects the events signaled by each sensor. The entity uses the output of the simulations to correlate the events from the sensor network. By merging information on the current attacks from the sensors with the one on the attacker behaviors from the simulations, this SIEM can produce accurate results in a very short time. In particular, it can correlate, attribute and predict the attacks in the scenario of interest. The following sections give some more details the proposed approach, the model parameters, and the final results. Here and in the following the adopted definition of risk is: "Risk is the result of a threat with adverse

effects to a vulnerable system" [1]. It can be quantified as a monotone

increasing function of the loss for the system owner and of the probability that the loss occurs.

1.1 Vulnerabilities, Agents, and Components

As previously said, to apply the Monte Carlo method we build a model that describes an ICT system as a set of components, each defining operations invoked by users of the system or by other components. The abstraction level of the description determines the number of components and that of operations. The system security policy defines the access rights of each user, i.e. the operations it is entitled to invoke. A threat agent, or simply

(17)

1 – Introduction

agent, models an attacker with the intent and the capability to violate the security policy to achieve some goals. Each goal is a set of rights to invoke some operations. It may be paired with a benefit for the attacker and a corresponding impact, i.e. loss, for the system owner. This impact occurs as soon as an agent owns all the rights in a goal. Agents are rational and adaptive and they minimize their efforts to reach the goal. They select the sequence of attacks to implement according to the goal to be reached and to the effort each sequence requires. Agents can model, among others, crackers, terrorist, users and insiders. As an example, an agent may model a terrorist that aims to acquire the right to shut down some critical components or to change their configuration to damage a whole nation.

Besides components and operations, the system model describes in more details the system features related to the attacks the agents implement. Hence, the component vulnerabilities and the attacks they enable are critical elements of the model. While the general definition of vulnerability is "the

manifestation of the inherent states of the system that can be exploited to adversely affect a system or its users", in an ICT context a vulnerability is a

defect in a hardware/software component or an erroneous or malicious user behavior [2, 3]. Vulnerabilities enable the elementary attacks the agents implement to illegally escalate their rights. Distinct components may be affected by the same or by distinct vulnerabilities. Each vulnerability may be effective or suspected. The proposed approach pairs each suspected vulnerability is paired with a probability distribution of being discovered at time t. This distribution is deduced from information on the component and on the supplier/developer of the component.

1.2 Elementary Attacks and Attack Sequences

An elementary, or atomic [4] attack, or simply attack, is a set of actions de-pending upon the abstraction level of the assessment and it is characterized by some attributes such as the set of enabling vulnerabilities. An attack may be successful only if all these vulnerabilities affect the corresponding components and they have been discovered. Each agent can implement a subset of all possible attacks and for each of these attacks we define the probability that the agent successfully executes it. An attack may fail due

(18)

1 – Introduction

to factors outside the agent control that influence its success. As an exam-ple, some actions of an attack may have to be executed in the time window in-between two commands of two distinct processes. If the agent cannot control these processes, it can only execute the action at distinct times try-ing to guess the correct one. Here, the success probability of the attack increases with the window size. In other cases, the value of a parameter has to be guessed or the success may depend upon the current machine or network load [5]. An agent can execute of an attack only after acquiring the rights in the attack precondition. As an example, an agent can transmit a dangerous parameter to an operation only if it can invoke this operation. A successful attack grants some further rights to an agent, those in the attack post condition. In the following, we assume that an agent can collect a reliable and correct information on vulnerabilities and attacks.

To exemplify all the information of interest for an attack, we consider a cross-site scripting attack. In this attack, the agent injects some code into a page of a web site that, in general, is unrelated to the target one. The browser of some users of the target system may retrieve and download this code when surfing the Internet. The target component is the web browser and the affected operation is the one to download some content on the site. The enabling vulnerability is the access to the web server hosting the page and the attack precondition is the right of downloading some content. The success probability of the attack is the one that the user visits the page that downloads the code. This probability can be increased through phishing messages. The attack post condition depends upon the code that is injected and those of the user browsing the page.

The attacks each agent can implement at a given time depend upon its rights and the security status of the system. This status includes any effective vulnerability in the system components and the suspected ones that have previously been discovered.

An agent reaches its goals through a sequence of attacks by sequentially composing some elementary attacks. In this way, it implements a privilege escalation that exploits the privileges acquired through the first i attacks in a sequence to implement the i + 1th attack. A sequence of attacks always

starts from the system attack surface, and it ends when the agent reaches

(19)

1 – Introduction

1.3 Enumeration vs Simulation

Nowadays, an assessment is interested in evaluating the return of investment (ROI) resulting from the deployment of a countermeasure in the target ICT system. The complexity of retrieving the information of interest and of eval-uating the cost effectiveness of a countermeasure favors the adoption of an approach based upon multiple simulations of the agent behavior rather than on an analytic model of this behavior. In other words, given the unman-ageable complexity of any formal model to compute the probability that a sequence of attacks is selected and implemented, the only way to compute statistics of interest is to produce a proper statistical sample through a Monte Carlo method. The method simulates how agents select and imple-ment attack sequences in a scenario of interest. By collecting information on events of interest in each simulation, we build a statistical sample on the attacks each agent implements, their success probability, the impact due to an agent or to an attack.

The proposed approach generates the same statistical data that is cur-rently used to assess and manage the risk without collecting historical data. Instead, it builds the models of the agents and of the system and uses them to predict the behavior of the target systems and of its attackers. This can assess the risk posed by an ICT system even before its deployment because this only requires the description of a scenario with the system, the agents, and their predefined goals. However, it should be stressed that the samples will result in an accurate assessment and management of the risk if and only if the models of the target system and of the agents are accurate and reliable. This assumes we discover all the system vulnerabilities and the attacks they enable. Furthermore, we have to define in an accurate way the attributes of the attacks and of the agents as they determine the scenario evolution. This is a rather challenging task but we believe it the only feasi-ble solution given the complexity of building and validating a mathematical model that describes in some details the interactions among the system, the agents and the attack sequences they may implement. As an example of the problems posed by this model, consider its definition requires the discovery of all the global vulnerabilities that affect a system. In turn, this requires an analysis of all the possible sequences an attacker can compose through the elementary attacks. The complexity of this analysis is exponential in the number of attacks. Instead, to apply the Monte Carlo method we only have

(20)

1 – Introduction

to model only how an agent selects each attack at each time step given the access rights it has acquired and the system vulnerabilities. This strongly reduces the overall complexity because we can drop alternative attacks as soon as a selection occurs.

1.4 Validating the Approach

An important problem the proposed approach has to face is the validation of the overall approach and of the solutions we propose to build the model of interest. While important information on the overall accuracy of the models is achieved by several assessments of real systems, we have had the opportunity of applying the proposed approach to a system before it has been attacked in a cyber war exercise. This has offered us the unique opportunity of knowing in all the details how a system has been attacked and to compare this information with the output of our models.

1.5 Thesis structure

Beside this one, the thesis presents seven sections. Sect. 2reviews the cur-rent state-of-the-art on several aspects of information security. At first, this section considers the aspects related to the assessment of vulnerabilities and the analyses of physical and logical topology. Then, the section discusses attack graphs Monte Carlo methodologies and security metrics. Then, the section recalls some theorems about set cover problems. At the end, Sect. 2

discusses Intrusion Detection Systems and Security Information and Event Management.

Sect. 3presents the methodology we propose to address the problems previ-ously outlined on risk assessment and management. The section is organized into two parts. The first one concerns the modeling of the scenario and it includes three subsections. The second part discusses the simulation of the entities in a scenario. Most of this section is a re-elaboration of [6,7,8,9,10] Sect. 4describes how to exploit the output of Monte Carlo simulations. It discusses the statistic about both the attacker behavior and the countermea-sures to deploy to manage the risk. Most of this section is a re-elaboration of [11,12,13].

(21)

1 – Introduction

Sect. 5 discusses the implementation of a suite of tools to support the application of the methodology. We detail the architecture of the suite we named Haruspex and the implementation of the tools of this suite. Most of this section is a re-elaboration of [14,15,16,17,18,19,20].

Sect. 6 discusses the validation of the methodology and of the tools that have been implemented by adopting it in three NATO defense exercises, the Locked Shield exercises. The participation in these exercises has been fundamental to have some assurance on the output of the suite tools. We describe the scenarios in the exercises, the output of the tools and evaluate the accuracy of the suite outputs. Most of this section is a re-elaboration of [21].

Sect. 7 describes several assessments where we have experimentally evalu-ated the Haruspex suite. The large number of ICT infrastructures that have been assessment and the accuracy of the outputs further confirms the ability of the suite of predicting how a system can resist to intelligent attackers. The adoption of the tools in real assessments has enabled us to improve both the suite and the methodology. Most of this section is a re-elaboration of [9,22,23,24].

Sect. 8 resumes the works described in this thesis, recalls its main results and outlines future developments.

My research group has always believed that the need for credible and accu-rate information for the public at large is very important. For this reason, it is always willing to communicate its activities not only throughout peer reviewed papers and conferences but also through science journalism and partecipation to security summits [25,26,27,28,29,30,31].

(22)

Chapter 2

State-Of-The-Art

This section presents the state-of-the-art in eight main research fields that related to the topics we discuss in the following:

• Vulnerability Assessment; • Topology Analysis; • Attack Graph;

• Monte Carlo Simulation; • Metric Analysis;

• Set Cover Problems;

• Intrusion Detection Systems;

• Security Information and Event Management.

Each of the eight following subsections introduces and describes some recent results in each field.

2.1 Vulnerability Assessment

As described in the following, the first step of the proposed methodology builds the model of the target system. To this purpose, it executes a vulner-ability assessment to discover all known vulnerabilities in any component.

(23)

2 – State-Of-The-Art

Vulnerability assessments are a longstanding aspect of information assur-ance. In fact, several organizations already have some form of vulnerability assessment process in place, ranging from the identification of the patches to be applied to the execution of active vulnerability scans on a regular basis. In this framework, the outputs of a vulnerability assessment can form an important aspect of a computer security improvement process. A vulnerability assessment may discover some vulnerabilities by scanning the components. The scanning sends some predefined inputs to the tools and analyzes the corresponding outputs. A more general assessment can be im-plemented if the source code of each component is available because some static tools can be applied to the program of a component to discover its vulnerabilities. In the following, we assume that a vulnerability assessment includes both run time analysis, the scanning, and a static analysis. Infor-mally, we say that a proper tool to discover the component vulnerabilities scans each component.

We split vulnerabilities in two sets: standard vulnerabilities and

non-standardones. The first set includes all vulnerabilities that refer to a specific

flaw in a specific component with a specific version includes vulnerabilities where the root cause is not a programming error but the methodology or the strategy used to design the component.

2.1.1 Standard Vulnerabilities

One of the most popular tools to implement a vulnerability scan is Nessus [32]. Born in 1998 thanks to Renaud Deraison, first as open source software and now as closed source, Nessus is used by computer security experts to audit their networks by scanning the proper ranges of Internet Protocol (IP) addresses to identify vulnerabilities. Currently, the scan is enriched by applying several plugins. These plugins are written in the Nessus Attack

Scripting Language (NASL) and they implement attacks at distinct levels

of the ISO/OSI model [33].

NASL plugins identify specific vulnerabilities and flaws in network re-sources and they can compromise the standard protocols using the libraries that the tool offers. One of the most interesting features of Nessus is that anyone can write NASL plugins and run them as part of a scanning. Custom plugins can detect vulnerabilities specific to the organization that developed the plugin. The plugins can also be shared with the Nessus development

(24)

team and may be included in updates to the Nessus platform.

The component that supports the communication among plugins is the

Knowledge Base. When a plugin is executed, it may store information about

open ports, account lists, active services, and so on. Then, this information is shared with the other plugins.

To analyze a system, the analyst has to configure the Nessus server and the client application. After setting up these tools and determining the scope of the vulnerability scan, it is possible to configure the scanner to scan a single IP address or entire blocks of IP addresses. The time to complete a scan depends on the number of plugins, on the network bandwidth, scan speed settings, and the number of IP addresses to be scanned. Another parameter that influences the scan time is the "scan mode", because the user can select if the scan has to be "safe" or "aggressive". The former mode forces a plugin to check a vulnerability only by identifying the components (or services) on a scanned node. In the latter mode, a plugin can also exchange several messages with the scanned host.

The Nessus server reports the vulnerabilities it discovers to the Nessus application that presents this information to the user in a variety of formats. Launched in summer 2005, NIST’s SAMATE project aims to improve the state of the art of existing software assurance methodologies and tools. The project’s main objectives include:

1. Developing metrics to gauge the effectiveness of existing software as-surance tools;

2. Assessing current software assurance methodologies and tools to iden-tify deficiencies that may introduce software vulnerabilities or con-tribute to software failures.

The activities of SAMATE in the software assurance realm are outlined in the 2007 IATAC Software Security Assurance: A State-of-the-Art Report

(SOAR)[34].

One of the main goals of the project is to define a metric to evalu-ate individual software assurance tools. Currently, this metric evaluevalu-ates a benchmark approach by evaluating the performance of a tool with respect to the SAMATE Reference Dataset (SRD), a collection of source code with

(25)

known security flaws. In the long run, SAMATE plans to support labora-tories that can assess software assurance tools.

A future SAMATE goal is to identify effective metrics to evaluate soft-ware security [35]. Most software assurance tools provide their own propri-etary measurement that represents the security of assessed software against "other" software. Eventually, these metrics could be incorporated into soft-ware assurance tools and verified by SAMATE laboratories. With these metrics in place, organizations could deploy tools such as source code scan-ners, binary or web application scanners and produce robust, well-understood measures of the software security.

The Security Content Automation Protocol (SCAP) [36] is "a suite of vul-nerability management standards aim to standardize and automate vulner-ability management, measurement, and technical policy compliance check-ing along with enhanced product and database integration capabilities with machine-readable reporting."

While SCAP is not directly associated with generating security measures, several standards within the SCAP suite provide well-documented measures than can be accessed through SCAP-enabled products. SCAP components are as described in Tab. 2.1.

The Common Vulnerabilities and Exposures (CVE), Common Configu-ration EnumeConfigu-ration (CCE), and Common Platform EnumeConfigu-ration (CPE) are essential to produce machine-readable information that can, in turn, pro-duce security metrics. The Common Vulnerability Scoring System (CVSS), Extensible Configuration Checklist Description Format (XCCDF), and Open Vulnerability and Assessment Language (OVAL) can be used to produce useful security metrics within an organization. The CVSS provides explicit metrics while the XCCDF and OVAL do not provide such metrics, they provide a framework against which organizations can measure their sys-tems compliance to organizationally defined configurations or information, based on the assessment results of the system.

The OSD Computer Network Defense (CND) Pilot [43] aims to leverage the SCAP standards to produce a better understanding of Department of Defense (DoD) networks. The expected benefits include:

• An architecture that leverages the SCAP standards to correlate asset data, event data, policy, and vulnerability data;

(26)

SCAP Components Description

Common Vulnerabilities and

Expo-sures (CVE) [37] A standard name and identifier for in-dividual vulnerabilities and exposures that have been publicly identified Common Configuration Enumeration

(CCE) [38] A standard name and identifier forindividual configuration issues associ-ated with software components

Common Platform Enumeration

(CPE) [39] A standard name and identifier forspecific systems, platforms and pack-ages

Common Vulnerability Scoring

Sys-tem (CVSS) [40] A metric for quantitatively communi-cating the impact of a specific vulner-ability

Extensible Configuration Checklist

Description Format (XCCDF) [41] A language for writing security check-lists, benchmarks, and related docu-ments

Open Vulnerability and Assessment

Language (OVAL) [42] A language for describing system in-formation, including its current state as well as the results of a vulnerability assessment

Table 2.1: SCAP components enumeration • The ability to generate metrics based on these values. The CND Pilot interface answers the following metrics questions:

• What vulnerabilities affect my assets?

• How many assets are affected by each vulnerability? • What patches are available?

• What patches have been applied?

Lastly, it is worth to detail the CVE database and the CVSS score. The CVE vulnerability naming scheme is a dictionary of common names for pub-licly known IT system vulnerabilities. It is an emerging industry standard

(27)

that has achieved wide acceptance by the security industry and a number of government organizations. Technical vulnerability experts from 31 indus-try, academia, and government organizations vote on the common names. CVE provides the computer security community with:

• A comprehensive list of publicly known vulnerabilities;

• An analysis of the authenticity of newly published vulnerabilities; • A unique name to be used for each vulnerability.

General CVE information is available at http://cve.mitre.org and the vul-nerabilities it lists can be viewed using the NIST ICAT vulnerability index at http://icat.nist.gov.

Each CVE vulnerability is paired with a CVSS score. CVSS is composed of three metric groups: Base, Temporal, and Environmental, each consisting of a set of metrics, as shown in Fig. 2.1.

Figure 2.1: The metric groups of CVSS These metric groups are described as follows:

• Base: represents the intrinsic and fundamental characteristics of a vulnerability that are constant over time and user environments; • Temporal: represents the characteristics of a vulnerability that change

over time but not among user environments;

• Environmental: represents the characteristics of a vulnerability that are relevant and unique to a particular user environment.

(28)

The purpose of the CVSS base group is to define and communicate the fundamental characteristics of a vulnerability. This objective approach to characterizing vulnerabilities provides users with a clear and intuitive rep-resentation of a vulnerability. Users can then invoke the temporal and environmental groups to provide contextual information that more accu-rately reflects the risk to their unique environment. Contextual information has been introduced to enable the users to make more informed decisions when trying to mitigate risks posed by the vulnerabilities. However, this points out the main problems of CVSS that is it does not consider how each vulnerability relates to the other ones that affect the target system. This results in a system independent evaluation that returns little information for an assessment.

2.1.2 Non-Standard Vulnerabilities

Currently,no official standard has been defined to classify or to support the discovery of web related vulnerabilities. Two projects in this field are Open Web Application Security Project (OWASP) [44] and Open Vulnerability and Assessment Language (OVAL) [42].

OWASP is an open community that aims to support organizations to con-ceive, develop, acquire, operate, and maintain applications that can be trusted. All the OWASP tools and documents are free and open. One of the most interesting tools this project has developed is Zed Attack Proxy (ZAP) [45], an integrated penetration testing tool to discover vulnerabili-ties in web applications. It is designed to be used by people with a wide range of security experience. ZAP includes automated scanners as well as tools to support a manual vulnerability analysis. ZAP returns a list of vulnerabilities, each with a CVE or CWE ID.

OVAL is an "information security community" effort to standardize how to assess and report upon the machine state of computer systems. The repositories are collections of publicly available and open content that uti-lize a language provided to encode system details. The OVAL definitions are machine-readable, standard tests that determine if the specified soft-ware vulnerability, configuration issue, program, or patch is present on a system. OVAL Definitions include metadata, a high-level summary, and

(29)

the detailed definition. Definition metadata provides the OVAL-ID, status of the definition (Draft, Interim, or Accepted), the CVE name or other ref-erence on which the definition (or definitions) is based, the version of the official OVAL Definition Schema the definition works with, a brief descrip-tion of the security issue the definidescrip-tion covers, the main author, and a list of the significant contributors to the definition.

2.2 Topology Analysis

Any assessment has to discover and analyze the topology of the target system. The fundamental tool that supports this task is Network Mapper (Nmap) [46]. Among the huge number of works that use the Nmap method-ology to discover and draw a network topmethod-ology we only recall [47], [48], [49], [50,51], [52], and [53]. In particular, Zenmap is the official tool that offers a graphical front-end to the Nmap methodology [54]. Zenmap is a multi-platform, free and open-source application developed to simplify the usage of Nmap. The GUI of Zenmap allows a user to draw a topology map of the subnetworks that are discovered and to use distinct colors for distinct type of nodes (i.e. routers, hosts, firewall, etc.). Furthermore, Zenmap can show the difference between two scans to simplify the tracking of new hosts or services on a system. Nmap uses different techniques to scan a network host:

1. TCP connect() scanning. The tool uses the connect() system call to open a connection to each interesting port on the machine. If the port is listening, connect() will succeed otherwise the port isn’t reachable.

2. TCP SYN scanning. This technique is often referred to as "half-open" scanning because it doesn’t open a full TCP connection. The tool sends a SYN packet, as when opening a real connection and waits for a response. A SYN|ACK indicates the port is listening. A RST signals a non-listener.

3. TCP FIN scanning. This technique discovers the state of the ports. It assumes that closed ports usually reply to a FIN packet with the proper RST while open ports usually ignore the packet.

(30)

4. Fragmentation scanning. The tool simply splits the TCP header into two tiny packets and sends them to the scanned host. In this way, it bypasses firewall and packet filter protection.

5. TCP reverse ident scanning. The ident protocol allows the disclo-sure of the username of the owner of any process connected via TCP. Thus, it is possible to discover active services and whether the server is running as root.

6. UDP ICMP port unreachable scanning. This scanning method uses the UDP protocol instead of TCP. Since open UDP ports do not send an acknowledgement in response to the probe, it is possible to find ports that are NOT open, and to determine by exclusion the open ones.

7. ICMP echo scanning. Since ICMP does not have a port abstraction this is not a real scanning. However, it can determine the hosts in a network by ping messages.

2.3 Attack Graph

Attack graphs are very important security tools because they represent the alternative attacks an agent can implement and how they can be composed into sequences. There are several alternative definitions of these graphs, the one we adopt uses the nodes of the graph to represent distinct sets of privileges. Furthermore, each arc is labeled by an attack that is enabled by the component vulnerabilities. The attacker may be executed when the attacker owns the set of rights paired with the attack source node. By executing the attack labeling an arc, the attacker acquires the rights in the arc destination node. Each path on this graph represents a sequence of attacks the agent can implement.

Since each attack is enabled by some vulnerabilities, a path also represents the vulnerabilities the agent exploits to implement an attack sequence. Sys-tem administrators analyze attack graphs to understand their sysSys-tem weak-nesses and to select effective security measures to deploy. As an example, we can compute the success probability of a plan by mapping a graph into a Markov chain where the transition probabilities are a function of the success

(31)

probabilities of attacks [55]. This mapping assumes that any vulnerability has been discovered and it returns a chain with a large number of states. [56] takes into account trust among network nodes to build an attack graph and to compute the success probability of a plan when both an agent has selected it and all the vulnerabilities have been discovered.

Defense graphs extend attack graphs with countermeasures [57, 58], and they may be transformed into Bayesian networks to compute the probabil-ity that a plan is successful. Since attack graphs are usually very large and complex, most of the work on these graph has focused on their automatic generation rather than on their analysis. A first analysis is described in [59] that uses attack graphs to define a tool that supports a security assessment and a vulnerability analysis of computer networks. The algorithm generates the graph by exploiting information on attack requirements, network con-figuration, and attacker profiles. A configuration file describes the system topology and the attacker profile includes information on its capabilities. This information is matched against attack attributes to discover those the agent can implement. [60] describes a framework to manage the attack graph complexity through an interactive visualization and including a hier-archical aggregation of graph elements. In particular, the rules to drive this aggregation are based either on common attributes or on the same connec-tions between nodes. This work uses Nessus to discover the vulnerabilities and the available exploits. [61] presents an approach to automatically ana-lyze the attack graphs through the use of logical expressions and conditional preference networks. The output assists the network administrator to select and apply the best countermeasures based on some preference criteria. In principle, the model is applicable to any topology and results in a proactive attitude. However, even if attack graph are based upon a nice mathematical theory, they have been seldom adopted due to their huge complexity even when the system to be assessed is a toy example. New interest in these graphs has been recently generated because of the availability of tools and databases for big data [62].

(32)

2.4 Monte-Carlo Method

The Monte Carlo method, we propose, solves, through multiple simulations, the uncertainty in the modeling of parameters such as attack selection, suc-cess of intrusions, and impact estimation. In each simulation, the assessor quantifies the uncertainty in the estimation of a parameter by pairing such a parameter with a probability distribution rather than with a single value. Most recent works in this field have focused on securing critical systems like energy, telecommunications, health-care, finance, and transportation. [63] shows an approach to assess and to harden a power generation plan by estimating the risks and the corresponding impacts through several attack scenarios. Further, [64] applies the Monte Carlo method and the CVSS [40] to estimate the frequency of successful attacks. This information supports the discovery of the most effective countermeasure to security vulnerabil-ities. Lastly, [65] describes a tool that integrates a Monte Carlo method with sophisticated optimization techniques to find the best combination of factors that lead to the desired result under uncertain conditions. It can be applied to resource allocation, scheduling, investment, route planning, and other problems to determine the best combination of inputs to maximize a return, minimize a cost, or achieve a specific optimization result.

2.5 Metric Analysis

Attack graph security metrics are important because an assessment can apply these metrics to compute important information about the target system. A security metric has to evaluate identifiable entity attributes that affect physical, personnel, IT, or operational security. Most metrics proposed in the literature belong to one of five classes:

• Architectural-Based; • Attack Graph-Based; • Performance-Based; • Time-Based;

(33)

[66] defines a metric in the first class to evaluate the resistance of an ap-plication to a malware attack. [66] proposes a classification scheme that defines a partial ordering on security requirements. Then, it ranks these requirements along three dimensions: Data Integrity, Code Integrity, and Data Confidentiality. Hence, a system is more secure than another if the security requirement of the former if larger than the other one. [66] also assumes that weaker security requirements result in a weaker system. The Shortest Path metric [67,68] is an example of a metric in the second class. It considers the length of attack paths in the attack graph of interest. This length determines the distance from an agent initial state to the agent goal and it assumes that the agent is interested only in minimizing its effort to reach the goal. Here, the effort represents the assigned and estimated amount of time or resources to exploit vulnerabilities. Resources may be the tenacity, the skill, and/or the money of the attacker [69]. Hence, given two ICT systems, the one with the shortest attack path is the less secure one.

The False Positive Rate Metric and False Negative Rate Metric belong to the third class and they are used to assess intrusion detection systems (IDS) and firewalls. An IDS produces a false positive anytime it wrongly classifies legal activities as an attack. On the other hand, a false negative occurs when an IDS fails to block an unauthorized access. The rates are calculated with the formulas:

FalsePositiveRate = _{all_elements − elements_should_be_detected}number_of _elements_detected

and

FalseNegativeRate = 1 −number_of _elements_detected_{elements_should_be_detected}

Smaller values of these measures denotes a more accurate behavior of the tool under evaluation.

[70] presents the metric Mean-Time-To-Breach based on time units, hours in the paper, to breach a system. The measure of interest is the total amount of time to breach the system divided by the number of success-ful breaches. Obviously, this metric favors the number of breaches over their quality. In fact, it considers as more secure a system where a single

(34)

attack achieves root privileges, than a system where ten breaches can be successfully implemented but do not achieve these privileges.

A metric in the last class is presented in [71]. This metric, the System

Vulnerability Index(SVI) pairs each vulnerability with a value ranging from

0 to 1 where 1 represents the most critical vulnerability. SVI is computed through a set of rules that represent the system characteristics, potentially neglectful acts, and potentially malicious acts. A rule has the form "if

antecedent then consequent". The antecedent defines the conditions to be

satisfied to produce something in the consequent. Once all rules have been defined, the SVI value is calculated through a formula that takes the rules two by two, by combining more than two rules and by using values already computed as a single value. The final output evaluates the overall system security. A value between 0 and 0.15 represents a secure system. A value between 0.15 and 0.3 represents a moderate level of vulnerability. A value greater than 0.6 is classified as extremely vulnerable.

2.6 Set Cover Problems

We consider set cover problems because they have a critical role when se-lecting countermeasures to deploy. This section classifies and sums up three basic methods to resolve these problems. The set cover problem is a NP-Complete problem [72,73] that gives a universe U and a family F of subsets of U, it answers the question: “Which is the minimum sum of subsets of F that equals the universe U?”.

The methods that we consider are: • combinatorial tree search;

• a specific type of integer programming; • a variant of the boolean algebra approach.

[74] proposes a tree search approach to solve the set partitioning problem that is the task of deciding how a set can be partitioned into two or more subsets. The proposed approach creates for each element of the universe U a block of columns. Each block represents one or more of the subset of the

(35)

family F that contain the element that corresponds to the number of the block. So, the block k-th exactly contains those sets that include xkbut no

element x1, ..., xk−1. The execution of algorithm sequentially searches the

block but where the block k-th is not searched till every element x1, ..., xk−1

is already been covered. Hence, if any set in the block k inlcudes elements with an index lower than k, it has to be discarded. Then, the sets are arranged heuristically in the ascending order of their cost and they are renumbered because the j-th set should correspond to the j-th column. [75] and [76] suggest a similar algorithm that incorporates a lower bound on the cost.

[77] describes a method in the second class to solve the problem by solving a sequence of problems. Consider a set cover problem with a binary matrix

Ap and a vector of solution vp with cp cost. Then, it can exist a further i-th

constraint, such that all solutions of cost less than cp _{satisfy it while the old}

vector vp _{does not. We can generate a new matrix A}p+1 _{by adding the new}

constraint to the row of the matrix Ap_{. If at any stage p∗, no such constraint}

can be generated, the procedure terminates with an optimal solution s of cost c, the minimum between 0 and p∗

The last method are suggested by [78] that describes a method to solve the set cover problem by initially considering a unicost set cover problem and after it generalizes the problem. Given a unicost set cover problem, which has an optimal solution of cost c, we generate a further unicost problem that has the cost equals c − 1. This process terminates when the optimal cover has a cost equal to zero. Then, c, the optimal cost of the given problem, is the number of intermediate generated problems and the optimal cover can be computed through a backtracking method.

2.7 Intrusion Detection Systems

An Intrusion Detection System (IDS) is a device, or software module, that monitors a system to detect attacks or policy violations. The monitor-ing phase is implemented through a network of sensors, entities that check events to discover violations as described in the following. When the sensors detect a violation, they raise an alert with information relative to the event. These concepts were firstly introduced in [79] and then again formalized in

(36)

[80]. A working group created by DARPA in 1998 defined a common frame-work for the IDS field [81] and a common intrusion specification language about attacks and events to enable cooperation among the components of an intrusion detection and response (ID&R) system.

Figure 2.2: A simple IDS architecture

Fig. 2.2shows a very simple IDS architecture [82]. Basically, an IDS is a detector that processes information coming from the system to be protected. Three kinds of information are used. The first one is long-term information related to the technique to detect intrusions, i.e. a knowledge base of at-tacks. The second kind of information describes the current configuration of the system to be protected. The third and last kind of information is audit information that describes the events that may occur in the system. The role of the IDS is to eliminate useless information from the audit trail and present a synthetic view of security-related actions by the users. A decision is then made to evaluate the probability that these actions can be considered symptoms of an intrusion.

According to [83], several reasons support the adoption of an IDS such as preventing attacks, increasing the perceived risk of discovery and pun-ishment of attackers, documenting the existing threat, and detecting the

(37)

preambles to attacks.

The IDS has to perform its task in real-time to be beneficial from a security perspective. In principle, an IDS has to deal with any kind of in-trusion, as an example network attacks against vulnerable services, data driven attacks on applications, host based attacks such as privilege escala-tion, unauthorized logins and access to sensitive files, and malware (denial of service attack, viruses, trojan horses, and worms). Alternative kinds of sensors analyze different data and apply distinct detection methods to deal with the distinct features of an intrusion [84,85,86].

We can distinguish between network IDS (NIDS) and host IDS (HIDS). NIDS can be further partitioned into two subtypes, the wireless IDS, focusing on wireless network, and the network behavior analysis (NBA) IDS, examining traffic flow on a network in an attempt to recognize abnormal patterns like Distributed Denial of Service (DDoS), malware, and policy violations.

Another kind of IDS of interest is a distributed IDS (DIDS) that com-bines HIDS and NIDS to build an efficient and cooperative security environ-ment to acquire a broader view of the whole security status of the system. One of the first examples is [87].

2.7.1 Network IDS

Since a NIDS monitors the network traffic, it is rather obvious it should be able to inspect most of inbound and outbound traffic. However, while on one side all the network traffic should be analyzed, on the other this could affect the overall network performance. To evaluate the overall complexity of a NIDS, consider that the packets captured by a sniffer, both hardware and software sniffers are available, is analyzed to detect possible attacks. This exploits a special implementation of the TCP/IP stack that reassem-bles the packets and applies protocol stack verification, application protocol verification, or other verification techniques.

The protocol stack verification looks for malformed data packets that vi-olate the TCP/IP protocol. This process mainly helps to detect DoS or DDoS attacks, because they create improperly formed packets to exploit any weaknesses in the protocol stack.

(38)

like HTTP, to discover unexpected packet behavior or improper use. As an example, it can discover DNS cache poisoning. This process is more computation intensive than the previous one, and it could affect the NIDS performance.

Among the several advantages a NIDS offers, we underline:

• if the network is well designed, few NIDSs can monitor a very large network;

• a NIDS used as passive devices can be deployed with little or without disruption to the normal network operations;

• a NIDS is usually not susceptible to direct attacks and it is not de-tectable by attackers.

However, NIDSs also have some disadvantages, the major ones:

• since network bandwidth is steadily increasing, a NIDS can be over-whelmed by network traffic and this compromises its detection capa-bilities;

• an encrypted communications and/or fragmented packets may affect the NIDS effectiveness;

• NIDS discover whether an attack was successful.

One of the most known NIDS tools is Snort [88]. Snort is a free and open source network intrusion detection (and prevention) system that runs on the most modern operating systems and it is largely supported by several communities.

Snort is able to perform traffic analysis in real-time and packet logging on Internet Protocol (IP) networks. In particular, it implements proto-col analysis, content searching, and content matching. Since Snort has to both detect attacks and generate output in a required format, it integrates distinct components that cooperate among themselves.

Its main five main components are:

• Packet Decoder: it receives packet from different types of network interfaces;

(39)

• Preprocessors: it normalizes protocol headers, detect anomalies, reassemble packets and TCP stream for the next detection phase; • Detection Engine: it detects intrusion activity by looking into the

packets;

• Logging and Alerting System: it raises alerts and logs;

• Output Modules: they generate the final output from the previous results.

2.7.2 Host IDS

A HIDS monitors a specific host or device. Usually, the HIDS is installed as a dedicated software or hardware that analyzes the events generate locally. Since the HIDS monitors the status of key system files, it is also known as

system integrity verifier. A HIDS can detect when an intruder creates,

mod-ifies, or deletes a monitored file. In particular, it examines the files and the system logs to determine if an attack is underway or has occurred and if the attack was failed or successful. To distinguish the levels of priority paired with distinct resources, the most common method to categorize folders and files is by color coding. Red coded resources, like OS kernel, are the most critical one. The yellow coded, like device drivers, are less critical, while the green coded resources, like user data, are less urgent, because they are frequently modified.

With respect a NIDS, a HIDS offers some advantages:

• it can detect events generate locally on the host system and attacks that may elude a NIDS;

• it can process encrypted traffic because it is decrypted by the host; • it can detect misuse in applications and system programs. As an

example, this enables the detection of Trojan horse;

• it can detect success or failure of attacks with respect to NIDS. As a counterpart, there are some disadvantages:

• the management of HIDSs is more complex because they have to be configured and managed on each monitored host;

(40)

• it is vulnerable both to direct attacks and to those against the host operating system;

• it needs large amounts of disk space to store the host OS audit logs; • it affects the performance of its host systems.

OSSEC [89] is one of the most widely used HIDS. It is free and open, and provides intrusion detection for many operating systems. OSSEC executes log analysis, integrity checking, Windows registry monitoring, rootkit de-tection, time-based alerting, and active response. The architecture of this HIDS is centralized and cross-platform. This simplifies the management and the monitoring of multiple systems, for example databases (like MySQL), web servers (like Apache HTTP Server), firewalls (like Iptables), NIDSs (also Snort), and many others.

OSSEC results from the composition of three main components:

• Main Application: it supports the distribution on the network sys-tem and/or the stand-alone installations;

• Windows Agent: it monitors Windows environments; • Web Interface: it provides a graphical user interface.

2.7.3 Detection methods

An IDS can adopt three strategies to analyze network traffic or local events and detect malicious activities:

• Signature Based Detection; • Anomaly Based Detection;

• Stateful Packet Inspection Detection.

2.7.3.1 Signature Based Detection

The Signature based approaches detect attacks by matching their input against a database of signatures of known intrusions. Hence, the attacks are signaled in a fairly accurate way.

(41)

Furthermore, this approach is widely used because several known attacks have unambiguous and distinct signatures. As an example, fingerprinting activities and worms implement one of a small set of attack sequences exploit some vulnerabilities and control a system.

The advantages of this approach are the ease of writing a new signature and of understanding those others have developed. Obviously, this assumes that enough information on possible attacks is available. Further advantages in-clude a very precise notification of the events that caused the alerts and the ability of simplifying the signature database by disabling rules not needed. As an example, rules for SMTP traffic are not enabled if an administrator knows that this traffic is not used.

On the contrary, the approach can be adopted provided that a fairly large set of information on an attack is available to extract an accurate signature. Another problem is the frequency of updates to the signature database. In the end, this method cannot detect a completely new vulnerability and its exploit, or, in other terms, it cannot discover 0-day attacks.

2.7.3.2 Anomaly Based Detection

Anomaly based detection works completely different from the signature based one because it adopts a statistical approach. In particular, it as-sumes that the normal behavior of a node or of the whole system is known. Under this assumption, detection is implemented through statistic indica-tors such as the average volume of network traffic or CPU usage. For each indicator a threshold is defined when one or more indicators exceed the threshold, the IDS raises an alarm.

One of the main advantages of this approach is the detection of 0-day attacks that result in a behavior that does not match the normal one. Another advantage is the ease of adaption as it only requires the update of the thresholds rather than the definition of new signatures.

However this approach presents some drawbacks such as the high number of false positives; events signaled as malicious while they are not; the com-plexity to collect data to statistically define the normal behavior.

A formal approach to automatically assess and manage ICT risk

UNIVERSITY OF PISA

DEPARTMENT OF COMPUTER SCIENCE

Ph.D. Thesis

A formal approach

to automatically

assess and manage ICT risk

Acknowledgements

Ringraziamenti

Contents

Acknowledgements

Ringraziamenti

List of Tables

List of Figures

Chapter 1

Introduction

1.1

Vulnerabilities, Agents, and Components

1.2

Elementary Attacks and Attack Sequences

1.3

Enumeration vs Simulation

1.4

Validating the Approach

1.5

Thesis structure

Chapter 2

State-Of-The-Art

2.1

Vulnerability Assessment

2.2

Topology Analysis

2.3

Attack Graph

2.4

Monte-Carlo Method

2.5

Metric Analysis

2.6

Set Cover Problems

2.7

Intrusion Detection Systems