Brain network analysis and deep learning models for studying neurological disorders based on EEG signal processing

(1)

DOCTORAL SCHOOL

UNIVERSITY MEDITERRANEA OF REGGIO CALABRIA

DIPARTIMENTO DI INGEGNERIA CIVILE, ENERGIA, AMBIENTE E MATERIALI (DICEAM)

PHD IN CIVIL, ENVIRONMENTAL AND SAFETY ENGINEERING with the additional label of “Doctor Europaeus” PARTNERSHIP BETWEEN UNIVERSITY MEDITERRANEA OF REGGIO CALABRIA UNIVERSITY OF MESSINA

S.S.D. ING-IND/31 XXXI Cycle

BRAIN NETWORK ANALYSIS AND

DEEP LEARNING MODELS FOR

STUDYING NEUROLOGICAL DISORDERS

BASED ON EEG SIGNAL PROCESSING

CANDIDATE

C

OSIMO

IERACITANO

ADVISOR

Prof. F

RANCESCO

C

ARLO

MORABITO

CO-ADVISOR

Dr. N

ADIA

MAMMONE

COORDINATOR

P

rof. F

ELICE

ARENA

(2)

C

OSIMO

I

ERACITANO

BRAIN NETWORK ANALYSIS AND

DEEP LEARNING MODELS FOR

STUDYING NEUROLOGICAL DISORDERS

(3)

Abstract

This Ph.D. thesis addresses dierent aspects related to neuroscience. Specif-ically, new methodologies are proposed to monitor and investigate the evolution of the brain-electrical network connectivity in subjects with neural decits. Brain network analyses are carried out to estimate the evolution of cortical connectivity in patients with Alzheimer's disease (AD), Mild Cognitive Impairment (MCI) and Childhood Absence Epilepsy (CAE) in order to nd possible correlations with the disorders onset. Since Electroencephalogram (EEG) is the most popular tool for investigating the cerebral electrical activity, the proposed methods are based on the analysis of EEG recordings. Indeed, abnormal patterns in the electrical potentials detected on the scalp may reect abnormalities in the communication between neu-rons and can be used as diagnostic and prognostic markers. Moreover, advanced machine learning techniques are employed to build up intelligent systems to aid in diagnosis of neurological disorders. Specically, Deep learning (DL) techniques are employed to discriminate subjects with neuropathologies by analyzing scalp EEG recordings. To this end, a data-driven customized Convolutional Neural Network (CNN) is proposed for dierentiating subjects with Alzheimer's Disease (AD), Mild Cognitive Impairment (MCI) and Healthy Control (HC); and, a Stacked Autoen-coders (SAE) architecture is proposed to learn latent features able to dierentiate patients aected by psychogenic non-epileptic seizures (PNES) from HC. Exper-imental results show the eectiveness of DL in clinical applications, indeed, the designed DL-based systems achieved better classication performance as compared to the conventional shallow machine learning and existing state-of-the-art methods. The thesis ends showing the results of other DL-systems also in other real-world applications.

(4)

Acknowledgements

This thesis includes the main research results achieved during my Ph.D., but before starting with technical details, I would like to express my sincere thanks to the people who contributed to the work here presented.

First and foremost, I would like to express my deepest gratitude to Prof. Francesco Carlo Morabito who has been my accademic advisor since my Bachelor's degree. I am greatly indebted to him for his continuing condence in me as well as for giving me constant support throughout these years, from the beginning, when I decided to undertake this route. I thank him for suggesting me this challenging but current research topic and for making my Ph.D. productive and stimulating through national and international collaborations/experiences. He led me to the world of Deep Learning and Biomedical Engineering and from then on, I dedicated myself to the related research. His precious guidance, his hunches and advices have profoundly contributed to my professional and also personal growth.

Besides my advisor, I would like to express my heartfelt appreciation to my co-advisor Dr. Nadia Mammone. I am extremely thankful to her for the immense help, insightful comments, continuous scientic (and moral) support and encouragement. Her valuable guidance helped me in all the time of my Ph.D., from the rst day I set foot in the laboratory, also with some wise advice. The possibility of collaborating with her daily, has represented an invaluable opportunity for improving my scientic knowledge and growing as a researcher.

I would like to express my gratitude and my warmest wishes also to Prof. Amir Hussain who was my placement Supervisor at University of Stirling (UK) where I spent six months through the ERASMUS+ programme. I thank him most sincerely for giving me the great opportunity to join his laboratory and especially for always making me feel a full member of his team. His valuable supervision, his continuous involving myself in several challenging research topics have contributed to raise my interests also in other research domains.

(5)

II

I wish to thank Prof. Fabio La Foresta for his collaboration and Eng. Maurizio Cam-polo for his valuable technical and moral support provided during my Ph.D.

I thank also all the doctors and researchers of IRCCS Centro Neurolesi Bonino Pulejo Messina (Italy) for providing the EEG dataset on which this work is mainly based. Last but not least, I would like to extend my thanks to the reviewers and members of my Ph.D. doctoral committee for reading and reviewing this thesis.

(6)

5.3 A deep learning approach for automatic classication of SEM images of nanomaterials . . . 92 5.3.1 Methodology. . . 93 Electrospinning Process . . . 93 Experimental Setup . . . 94 CNN architecture proposed . . . 96 5.3.2 Results. . . 98 5.3.3 Conclusions . . . 98 6 Conclusions . . . 101 6.1 Future works. . . 104 References . . . 105

(10)

List of Tables

4.1 Total number of learnable parameters of CNN1, for the binary

classication. . . 65 4.2 Epoch-based classication performance of the proposed CNN1

evaluated on test sets. . . 68 4.3 PSD epoch-based classication performance of the proposed CNN2

evaluated on test sets. . . 68 4.4 F-measure and accuracy performance of the proposed CNN1 and

conventional machine learning techniques (MLP1,MLP2, MLP3,

l-SVM, LDA), evaluated on test sets, when PSD-images are used as input. . . 69 4.5 F-measure and accuracy performance of the proposed CNN1 and

conventional machine learning techniques (MLP1, l-SVM, LDA),

evaluated on test sets. In this case the inputs of MLP, SVM and LDA classiers are handcrafted features manually extracted from spectral proles. . . 71 4.6 Patient-based classication performances of the proposed CNN1

classier. . . 72 4.7 Performance of the proposed system compared to other classication

systems. . . 81 5.1 Attack types of DoS, R2L, U2R, Probe categories. . . 87 5.2 NSL-KDD dataset composition without additional test attacks types. . 87 5.3 NSL-KDD∗ _{dataset composition after removing outliers. . . 88}

5.4 NSL-KDD∗ _{performance (Precision, Recall, F_measure) for the deep}

AE classier and the shallow MLP classier. . . 91 5.5 Accuracy performance and comparison with state of the art models. . . 91 5.6 Electrospinning setup of the 16 experiments. . . 95

(11)

VIII List of Tables

5.7 Layers conguration of the deep CNN proposed . . . 97 5.8 Accuracy performance of deep CNN with dierent numbers of hidden

layers. . . 99 5.9 Precision, Recall, F_score of the deep CNN∗

(12)

List of Figures

1.1 Lobes of the brain. . . 8 1.2 Standard EEG recording cap . . . 9 1.3 The 10-20 International System seen from left (A) and above (B) the

scalp. A ear lobe, C central, Pg nasopharyngeal, P parietal, F -frontal, Fp - frontopolar, O - occipital. (19 electrodes: Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T 3, T 4, T 5, T 6, Fz, Cz, Pz; whereas, the linked earlobe (A1-A2) are typically used as reference.) . . 10 1.4 EEG sub-bands. . . 11 1.5 Clinical course of Alzheimer's disease (from Sperling et al. [1].. . . 14 1.6 EEG recorded at location O1 (occipital) from an AD patient (top)

and a healthy subject (bottom). . . 15 1.7 Power Spectral Densities (PSD) of the EEG signals (shown in Figure

1.6) recorded from an AD patient (top) and a healthy subject, HC (bottom). The abscissa represents the frequency and the ranges of the brain rhythms (δ, θ, α, β EEG rhythms are emphasized). . . 16 2.1 The owchart of the procedure. (1) EEG recording and storing on a

computer; (2) partitioning of the n-channel EEG recording into m non-overlapping windows; (3) for each window, the wavelet coherence (Cxy) is evaluated for every pairs of electrodes and decomposed in the

six EEG sub-bands (delta, theta, alpha1, alpha2, beta1 and beta2) (4);

the m n X n coherence matrices are averaged over the time, obtaining the average coherence matrix C. . . 22 2.2 2D representation of the average coherence of the six sub-bands at

time T0 and time T1 for the patient MCI-41. . . 24 2.3 2D representation of the average coherence of the six sub-bands at

(13)

X List of Figures

2.4 2D representation of the average coherence of the six sub-bands at

time T0 and time T1 for the patient MCI-41. . . 25 2.5 Boxplot of Coherence at time T0 and T1. For each patient, the

boxplot on the left represents the values assumed at time T0 whereas the boxplot on the right represents the values assumed at time T1. The blue boxplots are associated to patients who were diagnosed MCI at time T0, whereas the red ones are associated to AD patients. Each box consists of the median (central mark), the 25th and 75th percentiles (the edges of the box); the whiskers extend to most

extreme data points not considered outliers. . . 26 2.6 Given two time series x and y (i.e., two EEG signals), a time point

t, an embedding dimension m, and a lag L, the two time series can be projected into the vectors Xt and Yt. When m = 3, Xtand Ytare

vectors with three elements and six possible ordinal patterns (Motifs) can occur. Given a time point t, the algorithm checks which motifs occurred in x and which one in y. In the example illustrated in gure, at time t, motif π4 occurred in x and motif π1 occurred in y. The

procedure is reiterated for every time point t so that we can end up with a nal occurrence rate of every motif πi in x (pX (πi )) and in y

(pY (πi )). We also end up with a joint occurrence rate pX,Y(πi, πj )

for every possible couple of motifs πi and πj , which accounts for the

probability that πi occurs in x and πj occurs in y. . . 29

2.7 P JD as a function of coupling strength c and embedding dimension m for the identical Henon Maps (top) and the non-identical Henon

maps (bottom). . . 31 2.8 P JD as a function of coupling strength c and lag L for the identical

Henon Maps (top) and the non-identical Henon maps (bottom). . . 31 2.9 Dendrograms of patient 51 (who was diagnosed MCI at T0 and AD

at T1), in every EEG sub-band, at T0 and T1. The vertical axis of the dendrogram represents the distance or dissimilarity between clusters (fusion level). The horizontal axis represents the electrodes. Each joining (fusion) of two clusters is represented by a vertical line splitting into two vertical lines. The horizontal position of the split, shown by the short vertical bar, provides the distance (dissimilarity) between the two clusters. An example of how the electrodes are

(14)

List of Figures XI

2.10 ND evolution in subject Pt 30 (stable MCI). ND is depicted as a function of the fusion level threshold, in every sub-band, at T0

(continuous line) and at T1 (dashed line). . . 35 2.11 ND evolution in subject Pt 51 (converted to AD at T1). ND is

depicted as a function of the fusion level threshold, in every sub-band, at T0 (continuous line) and at T1 (dashed line). . . 35 2.12 Time-frequency WC representation. (Top subplots) EEG of two

selected channels during the epileptic seizure (enclosed between the red lines); (Bottom subplot) Evolution of wavelet coherence between the two electrodes before, during and after the epileptic seizure with a coloration going from blue (0, low coherence) to yellow (1, high

coherence). . . 43 2.13 Patient 17, third seizure: evolution, over the time, of the WC-based

dendrograms, starting from the EEG window corresponding to 24 sec before seizure onset, towards the EEG window corresponding to 12

sec after seizure onset (the seizure lasted for 12 seconds). . . 44 2.14 Patient 17: temporal evolution of ND with respect to dierent fusion

level threshold values th (from 0.55 to 0.58). Vertical dashed lines

mark the seizure onset. . . 44 2.15 Patient 18: temporal evolution of ND with respect to dierent fusion

mark the seizure onset. . . 45 3.1 (a) A shallow (one hidden layer) and (b) a deep (multiple hidden

(15)

XII List of Figures

3.2 Deep Belief Network (DBN) architecture composed by stacked Restricted Boltzmann Machines (RBMs). Each RBM consists of a visible layer v and a single hidden layer hn. RBM1 is trained using the

input data as visible units. The hidden layer h2 of RBM2 is trained

using the output of the previous trained layer h1 of the RBM1. The

output of h2 is the input of the next RBM3 and so on. The trained

layers h1, h2... hn form the stacked architecture. Finally, the whole

DBN is ne-tuned with standard back propagation algorithm. . . 53 3.3 Autoencoder standard conguration. . . 54 3.4 Stacked Autoencoders architecture. The rst Autoencoder (AE1)

maps the input instance x into a compressed representation h1(coding

operation) which is used to reconstruct the input data (decoding operation). After training AE1, the code h1 is used as input to train

AE2, providing the code vector h2 and so on. The procedure is

repeated for all AEn autoencoders. The compressed representations

h1, h2... hn form the stacked architecture (SAE) which is typically

ne-tuned using conventional back propagation algorithm. . . 55 3.5 CNN architecture. It includes a sequence of convolution (CONV) and

pooling (POOL) layers followed by a standard fully connected neural network. In the convolutional layer the input map convolves with K lters (or kernels), providing K feature maps. After applying a non-linear activation function (Sigmoidal or ReLu) to each feature map, the pooling layer is performed. The features learned are the input of a fully connected neural network followed by a softmax layer, which performs the classication tasks. . . 56 4.1 Flowchart of the proposed method. . . 61 4.2 PSD-image conversion of an EEG epoch of an healthy control

individual. (a) Power spectral proles of each EEG channel. (b) 2-d gray scale PSD-image. Dark pixels correspond to low power values; brighter pixels correspond to high power values. Since a typical EEG epoch of a healthy subject is characterized by a dominant alpha activity, the PSD-image presents brighter pixels between 8-12 Hz.

(16)

List of Figures XIII

4.3 Convolutional Neural Network Architecture. It consists of an automatic feature extraction process and a standard MLP. First, the PSD-image convolves with 16 kernels sized 3 x 3, producing 16 features maps sized 19 x 159. Then, the ReLu function is applied to each feature map. The max pooling operation reduces the input image resolution, providing 16 features maps sized 9 x 79. At this stage, the feature maps are reshaped into a vector sized 16 x 9 x 79 = 11376 and processed by a standard Multi Layer Perceptron (MLP). It consists of a fully connected layer with 300 hidden neurons and a softmax layer which performs the binary or 3-way classication. In the gure, an

example of the proposed architecture for binary classication is shown. 65 4.4 ROC curves of CNN1, MLP1, MLP2, MLP3, l-SVM, LDA classiers

for AD vs HC (a), AD vs MCI (b), MCI vs HC (c) and AD vs MCI vs HC (d) classication, when PSD-images are used as input. . . 70 4.5 ROC curves of CNN1, MLP1, l-SVM, LDA classiers for AD vs HC

(a), AD vs MCI (b), MCI vs HC (c) and AD vs MCI vs HC (d) classication, when handcrafted features manually extracted from

spectral proles are used as input of MLP1, l-SVM, LDA. . . 72

4.6 Flowchart of the method proposed. . . 76 4.7 Time frequency representation of the psychogenic non-epileptic

seizures (PNES) and healthy control (HC). Each epoch of the 19-channels electroencephalography (EEG) is transformed in a time frequency map (TFM); then, the mean over the 19 channels, over the subjects and over the epochs is evaluated coming up with a single TFM per class. (a) TFM averaged over the 19 channels, the 20 epochs, and the six PNES subjects; (b) TFM averaged over 19 channels, the 20 epochs, and the 10 HC subjects. . . 77 4.8 The two Autoencodes (AE) implemented: AE [228:50:228] and AE

(17)

XIV List of Figures

4.9 Entropy representation of PNES (red dots) and HC (blue dots) evaluated at the outputs of the hidden nodes of the two compressed representations. (a) Entropy values related to PNES and HC features extracted from the rst AE (50 x 1). At this stage, the entropies of the two classes are comparable; (b) Entropy values related to PNES and HC features extracted from the second AE (20 x 1). At this stage, the entropies decrease and they are dierent for the two classes and generally greater for PNES than HC. . . 79 4.10 Softmax output representation of PNES (a) and HC (b) for the 20

leave-one-out testing sessions carried out for every subject. Each bin represents the output estimated by the softmax layer ranged between 0 and 1 (1 correct classication; 0 misclassication). The red dotted line is the average output level of the network, evaluated over the 20 sessions. . . 82 5.1 Flowchart of the proposed method . . . 87 5.2 Distribution of the number of zeros in each numeric features of the

training set. Features with null values greater than 80% are depicted in red and are discarded from the analysis. . . 88 5.3 (a) Proposed Deep AE Classier: The AE [102:50:102] compresses the

102 features (x) into 50 most signicant variables (h) used as input for a nal softmax output layer (o) to perform the multi-class detection. (b) Shallow MLP Classier: standard single layer feed-forward neural network with 50 hidden neurons (˜h) followed by a softmax output

layer (o) . . . 90 5.4 Flowchart of the method proposed. . . 93 5.5 Electrospinning setup. . . 94 5.6 Eect of the parameters variation on the morphology. (a-b) SEM

images of not-homogeneous nanobers (NHNF) due to the presence of beads. (c) SEM image of homogeneous nanobers (HNF). . . 96 5.7 Features maps learned by conv1 and conv3 on a SEM image input of

homogeneous (HNF) and nonhomogeneous nanobers (NHNF). (a-c) 16 feature maps sized 128 x 128 learned by conv1of NHNF and HNF,

respectively. (b-d) 64 feature maps sized 32 x 32 learned by conv3 of

(18)

I

Introduction

Our brain is a complex network that contains more than 1010 _{nerve cells. It consists}

of dierent cerebral regions, each with specic functions, but that share information and communicate each other. The result is a functionally interconnected network be-tween neuronal activation patterns of anatomically separated brain regions. However, brain functional connectivity is sensitive to neurodegenerative processes. The main diagnostic tool to investigate neural alterations caused by a wide variety of nervous system diseases (such as Alzheimer's disease) is Electroencephalography (EEG). EEG is a quick, cheap, noninvasive and well tolerated neurophysiological measurement. It measures the oscillations of brain electric potential recorded from electrodes on the human scalp. EEG analysis can provide valuable information as abnormalities in in-teraction between neurons, depending on the neurophysiological decits caused by the disease under analysis. The most common way to assess functional connectiv-ity and consequently investigate changes in human cortex is EEG coherence, that is a measure of synchronization between two EEG signals, recorded simultaneously at dierent locations of the scalp. Such measurement was used here for studying brain network connectivity of people with neurological disorders: Alzheimer's disease (AD), Childhood Absence Epilepsy (CAE).

This item represents the rst objective of the present research. Specically, as regards AD, longitudinal studies based on coherence and on a novel metric (called permutation joint distance) were carried out to investigate how the brain connectivity changed along the dementia. AD represents nearly 60% of the total dementia cases and is typically diagnosed after the age of 65. AD is still an intractable disease and patients survive on average only few years after AD diagnosis. The National Institute on Aging and the Alzheimer's Association workgroup of experts postulated that AD is a stage of a long and complex degenerating process. Indeed, they have hypothesized three stages of AD progression: Preclinical-AD, Mild cognitive impairment (MCI) and dementia due to AD. Nowadays, we still do not know how the EEG abnormalities

(19)

2 I Introduction

evolve together with the disease progression. Hence, longitudinal (follow-up) studies on MCI patients and normal aging healthy controls (HC) may help in the development of biomarkers for monitoring the progression of dementia. The methods here proposed are based on EEG signal processing as it has provided encouraging results in MCI and AD diagnosis and could be the basis of future systems for the early diagnosis of AD. As regards CAE, coherence based studies were carried out to estimate the evolution of the connectivity of EEG signals of patients aected by absence seizures and to nd possible correlations with seizure onset. Epilepsy is one of the most common neurological disorders as it aects about 1% of the worldwide population. Antiepileptic drugs can control seizures in about 66% of patients whereas about 8% of patients successfully undergo surgery. However, anti-epileptic drugs have signicant side eects and surgery is highly invasive and not always eective. At the moment, there is no available treatment for the remaining 25% of patients, whose quality of life is severily undermined, as seizures are still unpredictable. Hence, the extraction of distinctive features of epileptic activity in the EEG signals may help the automatic detection of seizures and also the extraction of other diagnostic information, not detectable through a visual EEG inspection.

The second objective of the present research is applying advanced machine learn-ing (ML) techniques (known as deep learnlearn-ing, DL) for studylearn-ing neurological disorders from EEG signal processing. DL was introduced by Hinton and Salakhutdinov in 2006. It has then become a hot topic in ML as deep neural networks (DNN) have achieved impressive results in many classication and regression problems. DL techniques al-low to learn discriminating features from raw input data, avoiding the hand-crafted feature extraction typical of a standard ML approach. In this research, DL models have been developed to extract the most signicant features from EEG recording of patients with neurological disorders. Specically, two classication problems were addressed with a "deep-oriented" approach: 1) classifying EEG recordings of people with AD, MCI and HC; 2) classifying EEG recordings of people with Psychogenic Non-Epileptic Seizures (PNES) and HC. As regards the rst item, a Convolutional Neural Network (CNN) approach for classication of dementia stages (HC/MCI/AD) based on 2D-spectral representation (power spectral density, PSD) of EEG record-ings was proposed. Indeed, PSD estimation is commonly used in clinical practice to analyze spectral changes in EEG recordings of HC subjects and MCI/AD patients. As regards the second item, an information theoretic based interpretation of a DNN approach in diagnosing PNES was proposed. PNES are short and sudden attacks that resemble epileptic seizures. Frequently, people with PNES are erroneously diagnosed

(20)

I.1 Thesis structure 3

with epilepsy, receiving consequently inappropriate treatments. However, in this re-search, a preliminary studied was carried out to classify people with PNES and HC. Finally, the potential of DL was evaluated in other real-word application elds. In this regards, two preliminary case studies are reported: cybersecurity and material informatics.

I.1 Thesis structure

The organization of this thesis can be summarized as follows:

• Chapter 1: this chapter introduces the principals concepts of the brain structure and Electroencephalography. Then, the brain electrical connectivity is presented and the mathematical denition of coherence is given. Finally, a brief introduction of the neurological disorders here investigated (Alzheimer's disease (AD), Epilepsy, Psychogenic Non-Epileptic Seizures (PNES)) is presented.

• Chapter 2: this chapter introduces new methodologies for studying the brain electrical connectivity of people with AD and Epilepsy. Specically, changes in cortical connectivity due to the progression of AD are studied through wavelet co-herence and permutation joint distance based approaches; whereas, brain network connectivity in absence epileptic patients are studies through a wavelet coherence and hierarchical clustering based approach.

• Chapter 3: this chapter introduces the principals concepts of deep learning theory. Moreover, the most known DL architectures are presented: Deep Belief Network (DBN), Stacked Autoencoders (SAEs) and Deep Convolution Neural Networks (CNN).

• Chapter 4: this chapter introduces new deep learning based frameworks for clas-sifying EEG patters of people with neuropathologies. Specically, a CNN based approach to discriminate AD, MCI and HC subjects using 2d-spectral represen-tation of EEG recordings is proposed; and a SAE based approach to discriminate PNES from HC individuals is discussed.

• Chapter 5: this chapter addresses the potential of DL in other application elds. Specically, two case studies are presented: cybersecurity and material informatics. As regards cybersecurity research item, a deep AE based framework is proposed to detect normal and abnormal trac patterns; whereas, as regards material infor-matics research item, a deep CNN based framework is proposed to classify Scan-ning Electron Microscope (SEM) images of homogeneous and nonhomogeneous nanobers produced by electrospinning process.

(21)

4 I Introduction

• Conclusions: this chapter addresses conclusions and future works.

I.2 List of publications

The work presented in this thesis has been published in (or to be publish in) dierent journals or conference proceedings. The list of all publications delivered during the Ph.D. is provided below.

Journals

• Ieracitano, C., Mammone, N., Bramanti, A., Hussain, A., & Morabito, F. C. (2018). A Convolutional Neural Network approach for Classication of Dementia Stages based on 2D-Spectral Representation of EEG recordings. Neurocomputing. • Mammone, N., Ieracitano, C., Adeli, H., Bramanti, A., & Morabito, F. C. (2018). Permutation Jaccard Distance-Based Hierarchical Clustering to Estimate EEG Network Density Modications in MCI Subjects. IEEE Transactions on Neural Networks and Learning Systems.

• Gasparini, S., Campolo, M., Ieracitano, C., Mammone, N., Ferlazzo, E., Sueri, C., ... & Morabito, F. C. (2018). Information Theoretic-Based Interpretation of a Deep Neural Network Approach in Diagnosing Psychogenic Non-Epileptic Seizures. En-tropy, 20(2), 43.

• Mammone, N., De Salvo, S., Bonanno, L., Ieracitano, C., Marino, S., Marra, A., ... & Morabito, F. (2018). Brain Network Analysis of Compressive Sensed High-Density EEG signals in AD and MCI subjects. IEEE Transactions on Industrial Informatics.

• Mammone, N., Salvo, S. D., Ieracitano, C., Marino, S., Marra, A., Corallo, F., & Morabito, F. C. (2017). A permutation disalignment index-based complex network approach to evaluate longitudinal changes in brain-electrical connectivity. Entropy, 19(10), 548.

Book chapters

• Ieracitano, C., Mammone, N., La Foresta, F., & Morabito, F. C. (2018). Investigat-ing the Brain Connectivity Evolution in AD and MCI Patients Through the EEG Signals' Wavelet Coherence. In Multidisciplinary Approaches to Neural Comput-ing (pp. 259-269). SprComput-inger, Cham.

• Ieracitano, C., Adeel, A., Gogate, M., Dashtipour, K., Morabito, F. C., Larijani, H., ... & Hussain, A. (2018, July). Statistical Analysis Driven Optimized Deep Learning System for Intrusion Detection. In International Conference on Brain Inspired Cognitive Systems (pp. 759-769). Springer, Cham

(22)

I.2 List of publications 5

• Ieracitano, C., Pant`o, F., Paviglianiti A., Mammone N., Frontera, P., & Morabito, F. C. - Towards an Automatic Classication of SEM Images of Nanomaterials Via a Deep Learning Approach. In Italian Workshop on Neural Nets. Springer, Cham. In press

• Morabito, F. C., Campolo, M., Ieracitano, C., & Mammone, N. (2019). Deep Learning Approaches to Electrophysiological Multivariate Time-Series Analysis. In Articial Intelligence in the Age of Neural Networks and Brain Computing (pp. 219-243). Academic Press.

• Adeel, A., Gogate, M., Farooq, S., Ieracitano, C., Dashtipour, K., Larijani, H., & Hussain, A. (2019). A survey on the role of wireless sensor networks and IoT in dis-aster management. In Geological Disdis-aster Monitoring Based on Sensor Networks (pp. 57-66). Springer, Singapore.

• Dashtipour, K., Gogate, M., Adeel, A., Ieracitano, C., Larijani, H., & Hussain, A. (2018, July). Exploiting deep learning for persian sentiment analysis. In Interna-tional Conference on Brain Inspired Cognitive Systems (pp. 597-604). Springer, Cham.

• Ieracitano, C., Pant`o, F., Frontera, P., & Morabito, F. C. (2017, August). A Neural Network Approach for Predicting the Diameters of Electrospun Polyvinylacetate (PVAc) Nanobers. In International Conference on Engineering Applications of Neural Networks (pp. 27-38). Springer, Cham.

• Mammone, N., Ieracitano, C., Duun-Henriksen, J., Kjaer, T. W., & Morabito, F. C. (2017, June). Coherence-Based Complex Network Analysis of Absence Seizure EEG Signals. In Italian Workshop on Neural Nets (pp. 143-153). Springer, Cham. Proceedings of peer-reviewed international conferences

• Ieracitano, C., Duun-Henriksen, J., Mammone, N., La Foresta, F., & Morabito, F. C. (2017, May). Wavelet coherence-based clustering of EEG signals to estimate the brain connectivity in absence epileptic patients. In Neural Networks (IJCNN), 2017 International Joint Conference on (pp. 1297-1304). IEEE.

• Morabito, F. C., Campolo, M., Ieracitano, C., Ebadi, J. M., Bonanno, L., Bra-manti, A., ... & BraBra-manti, P. (2016, September). Deep convolutional neural net-works for classication of mild cognitive impaired and Alzheimer's disease patients from scalp EEG recordings. In Research and Technologies for Society and Indus-try Leveraging a better tomorrow (RTSI), 2016 IEEE 2nd International Forum on (pp. 1-6). IEEE.

• Mammone, N., Bonanno, L., De Salvo, S., Bramanti, A., Bramanti, P., Adeli, H., ... & Morabito, F. C. (2016, July). Hierarchical clustering of the

(23)

electroencephalo-6 I Introduction

gram spectral coherence to study the changes in brain connectivity in Alzheimer's disease. In Evolutionary Computation (CEC), 2016 IEEE Congress on (pp. 1241-1248). IEEE.

(24)

1 Electroencephalogram and Brain Electrical

Connectivity

1.1 Introduction

Electroencephalogram (EEG) is a reliable and powerful tool in dementia research and diagnosis. Indeed, EEG contributes to study and monitor the functional connectivity of the human brain which may be damaged by neurological disorders. In order to provide the reader further understanding of the research results reported later, this Chapter introduces rst of all the basic concepts of the brain (Section 1.2) and Elec-troencephalogram (EEG, Section 1.3); then, the importance of the brain electrical connectivity is presented (Section 1.4). Finally, the neurological disorders taken into account in this research are briey introduced (Section 1.5).

1.2 The brain: an overview

The brain is a very complex and closely interconnected network of nerve cells. It is estimated that the number of of neurons in the brain is about 1010_-1011_{: one single}

neuron may receive stimuli via synapses from as many as 103 _{to 10}5 _{other neurons.}

The brain consists of ve main parts: • The cerebrum;

• The interbrain (diencephalon); • The midbrain;

• The pons Varolii and cerebellum; • The medulla oblongata.

The largest part of the brain is the cerebrum. It is splitted into helves: right and left hemispheres (separated by a deep ssure and connected by the corpus callosum). Each hemisphere controls the opposite side of the body: for example, if a stroke occurs on the left side of the brain, the right arm or leg may be weak or paralyzed. The cerebral

(25)

8 1 Electroencephalogram and Brain Electrical Connectivity

hemispheres are organized into four lobes: frontal, temporal, parietal, and occipital (Figure 1.1). Each lobe is divided into areas that controls specic dierent functions. Specically:

• Frontal lobe: solves complex problems, processes most voluntary movements, con-trols intellectual activities, such as organizational skills, personality, behavior and emotional control.

• Parietal lobe: includes sensory areas, responsible for feel- ings of temperature, touch, pressure, and pain from the skin. It controls mainly the ability to read, write and understand spatial relationships.

• Occipital lobe: is the area of the brain that receives images and consequently controls the vision.

• Temporal lobe: controls language, speech, memory and comprehension.

The interbrain or diencephalon includes the thalamus, that connects the sensory paths. The lower part of interbrain is the hypothalamus, which controls both the autonomic (involuntary) functions and hormonal secretions (together with the hy-pophysis). Dierently from cerebrum, the midbrain is a small part of the brain. The pons Varolii, instead, is an interconnection of neural paths; the cerebellum controls ne movement. The medulla oblongata includes many reex centers (i.e. vasomotor and breathing center).

Fig. 1.1. Lobes of the brain.

1.3 Electroencephalogram (EEG)

The Electroencephalogram (EEG) measures the electrical activity of the brain. It was discovered in 1924 by Berger and it has become the most used routine examination

(26)

1.3 Electroencephalogram (EEG) 9

in neurology [2] and the basic neurophysiological measurement of many brain com-puter interface (BCI) applications [3]. The EEG records and displays, over the time, the voltage dierence between two scalp sites: the location of interest and the "ref-erence" location. A network of EEG electrodes is located at the surface of the scalp (as shown in Figure 1.2) and collects the electrical elds generated in the brain by groups of pyramidal neurons that produce ionic current ows. Extracellular current ow is generated because of the excitatory and inhibitory post-synaptic potentials produced by cell bodies and dendrites of pyramidal neurons. The EEG waveforms are mainly produced by layers of pyramidal neurons whose synchronous activity produce a bio-electromagnetic eld, that propagates from the sources to the recording scalp electrodes. Fields propagate through tissues that have dierent conduction properties and overlap to the elds generated by other neuronal populations. As a result, the potentials recorded at a specic electrode site will reect the combination of the con-tributions of dierent cortical sources (volume conduction eect). The issue of volume conduction is not trivial in EEG analysis as it can result in apparently high functional connectivity between channels thus leading to a wrong neurophysiological interpreta-tion of the results. It has long been discussed in the literature and all of the proposed approaches (Laplacian ltering, source estimation, the use of connectivity measures not sensitive to phase interactions, among others) introduce various limitations and may cancel relevant source activity at low spatial frequencies [4]. EEG waveforms are characterized by amplitude, shape, morphology and frequency.

Fig. 1.2. Standard EEG recording cap

Nowadays, EEG is largely employed in clinical practice mainly because it is quick, cheap, noninvasive, and well tolerated neurophysiological measurement which has

(27)

pro-10 1 Electroencephalogram and Brain Electrical Connectivity

vided encouraging results in the analysis of several neurological disorders. Among them, this study focuses mainly on the analysis of EEG of people with the Alzheimer's disease.

1.3.1 EEG recording

EEG is recorded by placing a network of sensors (or electrodes) on the scalp. The most common electrode location method used is the 10-20 International System. In such system the reference points to determinate the electrode location are nasion and inion. From these points, the skull perimeters are measured in the transverse and median planes. The "10" and "20" represent the distance between adjacent electrodes that is either 10% or 20% of the total front-back or-right left distance of the head. Each electrode is labeled with a letter that indicates the six areas of the brain (Pre-frontal (Fp), Frontal (F), Temporal (T), Parietal (P), Occipital (O), and Central (C)) and a number (even numbers indicate electrodes placed on the right side of the head; odd numbers indicate those on the left side of the head). Moreover, electrodes with subscript letter "z" idicates zero or midline placement, electrodes with subscript letter "a" idicates lateral placement. Figure 1.3 reports the standard set of the 10-20 system.

Fig. 1.3. The 10-20 International System seen from left (A) and above (B) the scalp. A ear lobe, C central, Pg nasopharyngeal, P parietal, F frontal, Fp frontopolar, O -occipital. (19 electrodes: Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T 3, T 4, T 5, T 6, Fz, Cz, Pz; whereas, the linked earlobe (A1-A2) are typically used as reference.)

In the EEG measurement two type of electrodes can be used: bipolar or unipolar. In the bipolar conguration the potential dierence between a pair of electrodes is measured; whereas, in the unipolar conguration the potential of each electrode is compared either to a neutral electrode or to the average of all electrodes. In this study, the unipolar electodes have been used in the recording apparatus.

(28)

1.4 Brain electrical connectivity 11

1.3.2 EEG sub-bands

EEG signal is usually decomposed in its frequency rhythms and respective frequency sub-bands (delta (δ), theta (θ), alpha (α), beta (β)) which are associated with specic physiological and mental processes [5]:

• δband [0.5 - 4 Hz]: delta rhythms are the slowest rhythms visible when the subject is sleeping. The highest value appears at deep-sleep stage.

• θ band [4 - 8 Hz]: theta rhythms appear at the early stages of sleep (in healthy subjects) .

• α band [8 - 13 Hz]: alpha rhythms are the main resting rhythms of the brain, commonly observed in awake adults, especially in the occipital electrodes; • β band [13 - 32 Hz]: beta rhythms appear because of anxiety or intense mental

activity.

The brain waves components of EEG signals can be investigated by frequency analysis or, when keeping track of the temporal evolution of EEG frequencies necessary, by time-frequency analysis [6].

Fig. 1.4. EEG sub-bands.

1.4 Brain electrical connectivity

Brain electrical connectivity describes the network of functional connections of dier-ent area of the brain. Neurological disorders (such as Alzheimer's disease, epilepsy) may change the integrity of functional connections in the human brain. The most common way to assess the synchronization among brain areas is measuring the coher-ence between EEG recordings. Conventional cohercoher-ence measures the synchronization

(29)

between two signals, observing only the spectral components and losing time informa-tion. The magnitude squared coherence between two EEG channels x and y is dened as:

Cxy(f ) =

|Pxy(f )|2

Pxx(f )Pyy(f ) (1.1)

where Pxy(f ) is the cross power spectral density of x and y, Pxx(f )and Pyy(f )are

the power spectral densities of x and y, respectively. All of them are functions of the frequency f. Cxy(f ) is a function of frequency as well and it is a measure of

synchronization between x and y. Cxy can range from 0 to 1, which indicates how well

x corresponds to y at each frequency. Cxy is usually calculated through the Welch's

averaged, modied periodogram method. 1.4.1 Wavelet Coherence (WC)

Wavelet Coherence (W C) takes into account the temporal and frequency information, thus, it measures the coherence between two signals in the time-frequency space. As noted, W C outperforms conventional coherence in the analysis of AD EEGs [7], [8]. W C is based on the denition of Continuous Wavelet Transform (CW T ), used in time-frequency analysis. The CW T of a time series x is dened as:

CW T (a, b) = Z +∞

−∞

x(t)Ψ_a,b∗ (t)dt (1.2)

where Ψ is the mother wavelet, a the scaling parameter and b the shifting parameter. In this study, the Morlet wavelet was chosen as mother wavelet because it is reasonably localized in both time and frequency [9]. As each scale is inversely related to a specic frequency, the CW T is a time-frequency representation. The wavelet coherence W Cxy

between two time series is dened as: W Cxy(a, b) =

|S(C∗

x(a, b)Cy(a, b))|2

S(|Cx(a, b)|2)S(|Cy(a, b)|2)

(1.3) where Cx(a, b)and Cy(a, b)are the wavelet functions of x and y at scales a and shift

b; the asterisk sign * indicates the complex conjugate operator. S denotes a smoothing operator in time and scale. For the Morlet wavelet, a proper smoothing function S was provided by Torrence and Webster [10]. The numerator is the wavelet cross spectrum and it displays the areas with high common power between two signals.

1.5 Neurological disorders

In this research, EEG recordings of people with Alzheimer's disease (AD), Epilepsy and Psychogenic Non-Epileptic Seizures (PNES) are taken into account for

(30)

monitor-1.5 Neurological disorders 13

ing the brain electrical connectivity modications (Chapter 2) or for classication purposes (Chapter 4). Therefore, this Section briey introduce the neuropathologies previously mentioned (AD, Epilepsy, PNES)

1.5.1 Alzheimer's disease

Alzheimer'disease (AD) is a neurodegenerative disorder with a subtle, asymptomatic onset and a gradual progression towards the full-blown stage of the disease, when the clinical symptoms become noticeable [11]. Today, 47 million people live with dementia worldwide. This number is expected to increase to more than 131 million by 2050, together with the average age of the population. Alzheimer's disease represents nearly 60% of the total dementia cases [12]. AD is typically diagnosed after the age of 65. AD patients survive on average only 4 to 8 years after diagnosis, because AD is nowadays still intractable [13], [14], [15]. AD causes damage and consequently the likely death of neurons in the brain. The irreversible loss of neurons aects the brain functionality such as loss of intellectual abilities (including memory, reasoning), intellectual decits and behavior disturbance. Currently, the clinical data available support the idea that the development of AD is associated with abnormal processing of β-amyloid (Aβ) peptide, that leads to the formation of Aβ plaques in the brain, interfering with the communication between neurons. This process occurs while individuals are still cognitively normal. Nowadays, there is no cure for AD, but taking certain medicines in the early stages of AD may delay the onset of the dementia [16].

Brain evolution in Alzheimer's disease

The National Institute on Aging and the Alzheimer's Association workgroup of ex-perts postulated that what is commonly considered Alzheimer's should rather be considered a stage of a long and complex degenerating process [1]. They have hypoth-esized three stages of AD progression.

1. Preclinical-AD: when the disease has already triggered the brain degeneration but no clinical symptoms are visible yet.

2. Mild cognitive impairment (MCI) due to AD (MCI-AD or Prodromal-AD): an intermediate stage where symptoms related to the thinking ability may start to be noticeable, but they do not aect the daily life of the subject;

3. Dementia due to AD (Dementia-AD): In the last stage, impairments in memory, thinking and behavior undermine a person's ability to live and act independently. AD progression is not the same in all the individuals. It may depend not only on the age of the pathology declaration but also on the health conditions of the person. MCI

(31)

could remain stable or return to be normal over time but a lot of them progress to dementia of Alzheimer's disease within 5 years (the annual rate of MCI progression to AD is between 8% and 15% [17]). Figure 1.5 shows the clinical course of Alzheimer's disease. It is to be noted that the denitive diagnosis of AD can only be made by postmortem analysis of the brain of a patient with dementia. Therefore, longitudinal (follow-up) studies on MCI patients and normal aging subjects can be of great help in the development of biomarkers for monitoring the progression of dementia and early detect the Alzheimer's disease. Before discussing the state-of-the-art of longitudinal MCI/AD studies, the main eects caused by AD in the EEG recording are introduced in the next subsection.

Fig. 1.5. Clinical course of Alzheimer's disease (from Sperling et al. [1].

Eects of Alzheimer's disease in EEG recording

AD aects the neuronal metabolic processes and leads to a loss of connections be-tween nerve cells. Three main characteristics can be commonly observed in AD pa-tients' EEG signals, compared to healthy controls are: slowing eect (the power at low frequencies increases whereas the power at high frequencies decreases); reduction of complexity and reduction of synchrony between pairs of EEG signals [18]. Such ef-fects come with the functional disconnection caused by the death of neurons [19], [20]. Although these eects have been largely discussed in the literature, they are not eas-ily detectable because of the existing high variability among AD patients. Previous studies have proved that EEG data of AD patients show a decreasing of power in higher frequency bands (α and β) and consequently a power increasing in lower fre-quency bands (δ and θ) when the behavior of AD patients is compared with healthy subjects (or healthy controls, HC) [21], [19]. It is widely assumed that, during the AD

(32)

1.5 Neurological disorders 15

progression, the power decrease and increase in β and θ band, respectively, are the earliest changes. Then, the decrease in the α activity is observed. Finally, the power increase in the δ band is produced as last eect. Figure 1.6 shows the EEG traces of a patient aected by AD and a HC, recorded by the occipital channel O1. The EEG was recorded in a comfortable eye-closed resting state. The signals were band-pass ltered at 0.5-32 Hz and sampled at 256 Hz. The AD EEG (Figure1.6, top) shows the slowing eect, peculiar of cerebral degeneration due to AD. The AD slowing eect is evident in Figure1.6 (top) and Figure 1.7 (top). The dominant peak in α band (8-13 Hz), peculiar of healthy subjects (HC) in eye-closed resting state, is indeed present in the Power Spectral Densities (PSD) of the HC whereas it slowed and reduced in the AD patient. δ band (0-4 Hz) looks prominent in AD patients, as rather expected from clinical considerations. As it will be discussed in Chapters 3 and 4, in Deep Learning approaches these clinical behaviors can be automatically extracted and represented in suitable features then used for classication tasks.

Fig. 1.6. EEG recorded at location O1 (occipital) from an AD patient (top) and a healthy subject (bottom).

The hallmarks of EEG abnormalities in AD patients include also a decline in func-tional signal connectivity between dierent brain regions is [19] and a decline of EEG complexity. As regards the loss of syncronization, the most common way to compute functional connectivity is quantifying the coherence between EEG signals [22]. Co-herence measures the relationship between two time series (i.e. EEG recording) in the frequency domain. Spectral coherence was proposed in the past to spot dierences between the EEG of AD patients and Healthy Controls (HC) [23], [24], [22], show-ing that EEG connectivity seems to be lower in AD patients rather than in healthy individuals. As regards the loss of complexity, the most common way to compute EEG complexity is the entropy, that is a measure of the uncertainty of a random

(33)

Fig. 1.7. Power Spectral Densities (PSD) of the EEG signals (shown in Figure 1.6) recorded from an AD patient (top) and a healthy subject, HC (bottom). The abscissa represents the frequency and the ranges of the brain rhythms (δ, θ, α, β EEG rhythms are emphasized).

variable [25]. It has been shown that EEGs of AD patients display lower values of entropy than EEGs of HC subjects.

1.5.2 Epilepsy

Epilepsy is a neurological condition also known as a seizure disorder because seizure is the primary symptom. However, it is to be noted that one seizure is not necessarily a sign of epilepsy, indeed, it is usually diagnosed when an individual has had at least two seizures that were not caused by some known medical condition (such as alcohol withdrawal or extremely low blood sugar). A seizure may last for a few seconds or minutes causing dierent symptoms (rapid blinking, staring at nothing, loss of consciousness, falling, muscle jerks), depending on the type of seizure (there are more than 40 syndromes).

Doctors generally classify seizures into two major categories: generalized and partial (or focal).

Focal seizures. Focal seizures appear with an electrical discharge in just one area of the brain. These seizures include:

• Simple Focal seizures: these seizures do not cause a loss of consciousness. They start in the motor cortex of the frontal lobe and may include involuntary

(34)

move-1.5 Neurological disorders 17

ments. Moreover, these seizures may alter emotions or change the way things look, smell, feel, taste or sound.

• Complex Focal seizures: these seizures cause a change or loss of consciousness. They start in a small area of the temporal lobe or frontal lobe, but they quickly involve areas that aect alertness and awareness. When the seizure occurs, the person may stare blankly and perform meaningless repetitive movements like lip smacking or picking at his clothing.

Generalized seizures. Generalized Seizures involve all areas of the brain. These seizures include:

• Tonic-clonic seizures or grand mal seizures. Each seizure consists of a tonic and clonic stage. During the tonic phase, the muscle tone increases, causing stiening; during the clonic phase muscles start rapidly alternating tightening and relaxation. • Tonic seizures. This seizure causes the sudden stiening of muscles, and conse-quently of movements. It usually aects muscles in the back, arms and legs and last less than 20 seconds.

• Atonic seizures. This seizure causes a loss of muscle control. The person remains conscious but may fall to the ground without warning.

• _{Myoclonic seizures. This seizure causes a brief, shock-like jerk of a group of} muscles (arms, shoulders, neck, body, legs). It is to be noted that people who do not have epilepsy may experience this kind of jerk while falling asleep (this is considered normal).

• Absence seizures or petit mal seizures. Absence seizures often occur in childhood and consist of staring into space, blinking eyes, making slight movements of mouth or hands. There is no warning before the seizure, and the person is completely alert as soon as it's over.

1.5.3 Psychogenic Non-Epileptic Seizures

Psychogenic Non-Epileptic Seizures (PNES) are short and sudden behavioral changes that resemble epileptic seizures, but do not exhibit EEG ictal patterns. In these sub-jects, there is no evidence of other possible somatic causes of the seizures, whereas there is strong evidence that such seizures are caused by psychogenic factors [26]. Many patients with PNES are erroneously diagnosed with epilepsy, and thus they may receive inappropriate treatment leading to inecacy and relevant side eect [27]. This may be due to diculties in getting a correct medical history of such patients and to the fact that interictal EEG is often normal in epilepsy patients. Thus, ac-cording to the International League against Epilepsy, PNES diagnosis is based on a

(35)

stepwise approach, involving several clinical and neurophysiological examinations, to formulate a diagnosis with a growing level of certainty [28]. In particular, the diagno-sis of denite PNES is based on the visual examination of seizures captured during video-EEG recording. PNES may occur spontaneously during long-term EEG record-ing or may be evoked by means of suggestion techniques. While the former is costly for the national health system, the latter is not free from ethical concerns [29]. The challenge for neurologists is to achieve an early and accurate diagnosis of PNES, based on clinical data and standard EEG. Hence, the availability of an alternative method to diagnose PNES from interictal EEGs, with no need to capture the ictal events, would be of great benet for both physicians and patients.

(36)

2 Brain Network Analysis based on EEG Signal

Processing for Studying Neurological Disorders

2.1 Introduction

This Chapter focuses on new methodologies for studying the brain network connec-tivity of people with neurological disorders, in particular: Alzheimer's disease (AD) and epilepsy (Childhood Absence Epilepsy, CAE). In Section 2.2 two approaches for investigating the changes of the cortical connectivity due to the progression of AD (logitudinal studies), are presented. In Section 2.3, instead, the network connectivity density in absence epileptic patients is studied by using wavelet coherence between EEG signals and hierarchical clustering technique.

2.2 Brain network evolution in Alzheimer's disease patients

Today, 47 million people live with dementia worldwide. This number is expected to increase to more than 131 million by 2050, together with the average age of the population. AD represents nearly 60% of the total dementia cases [30]. AD is typically diagnosed after the age of 65 and AD patients survive on average only 4 to 8 years after diagnosis, because AD is nowadays still intractable [3136]. As discussed in Section 1.5.1, the NIA/AA hypothesized three stages of AD progression: Preclinical-AD, Mild Cognitive Impairment (MCI) due to AD (MCI-AD or Prodromal-AD) and Dementia due to AD (Dementia-AD). Hence, longitudinal (follow-up) studies on MCI patients and normal aging subjects can be of great help in the development of biomarkers for monitoring the progression of dementias. To this end, this Section aims to evaluate changes in the brain-electrical connectivity of patients enrolled within a follow-up program that consisted of two successive evaluations, at time T0 and at time T1, three months later. This Section is organized as follow. In Subsection 2.2.1 a review of recent longitudinal studies is presented. Then, two approaches to study the brain electrical evolution in MCI patients are discussed: a wavelet coherence based approach

(37)

20 2 Brain Network Analysis, EEG Signal Processing, Neurological Disorders

(Subsection 2.2.2) and a Permutation Jaccard Distance based hierarchical clustering approach (Subsection 2.2.3).

2.2.1 Longitudinal studies: a review

Early diagnosis of AD could help improve life quality, but unfortunately there is no valid criteria or symptom able to make a diagnosis before the manifestation of illness. Currently there is not a single clinical test for the exact diagnosis of AD but, as discussed previously, EEG signal analysis has been becoming a promising and powerful tool for early detect the dementia of AD. In this context, researches have been investigating how the brain connectivity of MCI and AD patients changes over time, at T0 and after a predetermined range of time called "follow-up" (T1,T2 etc..), in order to determine possible biomarkers able to early detect the AD. Howerver, keeping the patients and their caregivers loyal to a follow-up program is challenging, resulting in a lack of longitudinal studies in the literature. Indeed, Moretti et al. [37] observed a link between MCI prodromal to AD and a decit of the temporoparietal cortex as well as a link between shrinkage of the temporoparietal zone and increased alpha3/alpha2 EEG power ratio. Babiloni et al. [38] , by applying LORETA to resting eyes-closed EEG recorded from MCI and AD patients, found that the progressive atrophy of hippocampus looks correlated with decreased cortical alpha power. Ahmadlou et al. [39] investigate the complexity of functional connectivity networks in MCI patients during a working memory task using magnetoencephalography (MEG) signals. They introduce two measures for brain networks complexities: graph index complexity and eciency complexity. Houmani et al. [40] use epoch-based entropy for early diagnosis of the AD. Stam et al. [41] observed that synchronization likelihood (SL) signicantly decreased in the 14-18 and 18-22 Hz bands in AD patients compared with both MCI patients and subjects with subjective memory complaints (SC). Martinez-Murcia et al. [42] study the structural parametrization of the brain using hidden Markov models-based paths in AD.

The forementioned studies are cross section. There are just a few longitudinal studies on MCI [43], resulting in a lack of diagnostic tools to allows the neurologist to objec-tively monitor the progression of the disease. Morabito et al. [44] present a longitudinal EEG study of AD progression based on a complex network approach. Romero-Garcia et al. [45] note dierent scales of cortical organization are selectively targeted during the progression to AD. Buscema et al. [43] proposed a system which exploits special types of articial neural networks assembled in a novel methodology named implicit function as squashing time (IFAST). They reported that IFAST method was able to

(38)

2.2 Brain network evolution in Alzheimer's disease patients 21

predict the conversion from MCI to AD with high accuracy (85.98%) in a one-year follow-up study. This methodology was later improved but it has been validated only in an MCI versus AD versus Controls cross-sectional study so far [46].

Therefore, motivated by the preliminary results achieved in the abovementioned works, one objective of the present research thesis is to study the evolution of the cortical connectivity in MCI patients through the EEG analysis (within a follow-up program). To this end, two novel methods baed on Wavelet Coherence and Permuta-tion Jaccard Distance (PJD) respectively are proposed and discussed as follow. 2.2.2 Brain network evolution through EEG signals' Wavelet Coherence Here, a preliminary study on the brain connectivity analysis of MCI patients is pre-sented. Specically, a wavelet coherence approach is proposed for investigating how the brain connectivity evolves among cortical regions with the disease progression. Indeed, EEG recordings acquired from eight patients were studied at time T0 and at time T1 (follow up, three months later). At T0 all patients were aected by MCI, at T1 three of them converted to AD and ve remained MCI. The EEGs were analyzed over the main EEG sub-bands: delta, theta, alpha1, alpha2, beta1 and beta2.

Dier-ently from MCI stable subjects, MCI patients who converted to AD, showed a strong reduction of cortical connectivity in theta, alpha(s) and beta(s) sub-bands. Delta band showed high coherence values in each pair of electrodes in every patient. This Subsec-tion is organized as follows: in paragraph 2.2.2 the available EEG recordings and the proposed methodology are described. In paragraph 2.2.2 the achieved experimental results are presented and discussed. Discussion and conclusion are addressed in 2.2.2. Methodology

The steps of the proposed methodology are schematized in Figure 2.1 and can be summarized as follow: (1) recording and storing on a computer the n-channels EEG (n=19 electrodes); (2) partitioning the EEG into m non-overlapping windows and processing it epoch by epoch (3) for each window, estimation of the wavelet coherence (WCxy) for all pairs of electrodes. WCxy is a k X p matrix with k samples and

p frequency values; (4) after choosing the frequency range from 0.5 to 30 Hz, the WCxymatrix is decomposed in six sub-matrices corresponding to delta, theta, alpha1,

alpha2, beta1 and beta2. In this way the wavelet coherence has been evaluated in the

six EEG sub-bands: delta (0-4 Hz), theta (4-8 Hz), alpha1 (8-10 Hz), alpha2 (8-12

(39)

Fig. 2.1. The owchart of the procedure. (1) EEG recording and storing on a computer; (2) partitioning of the n-channel EEG recording into m non-overlapping windows; (3) for each window, the wavelet coherence (Cxy) is evaluated for every pairs of electrodes and

decomposed in the six EEG sub-bands (delta, theta, alpha1, alpha2, beta1 and beta2) (4);

the m n X n coherence matrices are averaged over the time, obtaining the average coherence matrix C.

EEG recording and preprocessing

Study popoulation. Eight MCI patients (4 males and 4 females), at dierent clini-cal stages, were recruited at the IRCCS Centro Neurolesi Bonino-Pulejo of Messina (Italy) and enrolled within an ongoing cooperation agreement that included a clinical protocol approved by the local Ethical Committee of IRCCS Centro Neurolesi Bonino-Pulejo of Messina (Italy). An informed consent form was signed by each patient. The MMSE (Mini-Mental State Examination) score was used as inclusion criteria for the enrollment of the patients. After the conrmation of the diagnosis, all patients were evaluated for gender, age, schooling, estimated age of onset of the disorder, marital status and on the MMSE scores. The use of any medications and drugs was esti-mated: all patients had been receiving them for at least three months before the assessment. All cognitive and clinical valuations were carried out by the same ex-aminers. The Alzheimer Diseas's was diagnosed according to the National Institute on Aging Alzheimer's Association criteria. Each subject was evaluated at a baseline time (time T0) and 3 months later (time T1). At the rst evaluation, time T0, the neuropsychological as-sessment of patients exhibited MMSE scores of 23.4_6.69 for the MCI group. At T1, 3 MCI patients exhibited a conversion from MCI to AD. EEG registration. The EEGs were recorded during the resting state of the patient, in accordance with the 10-20 International System (19 electrodes: Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T 3, T 4, T 5, T 6, Fz, Cz, Pz). The linked earlobe (A1-A2) was used as reference. The sampling rate was 1024 Hz and a 50 Hz notch

(40)

2.2 Brain network evolution in Alzheimer's disease patients 23

lter was applied. Before the recordings, patients and caregivers were interviewed about the duration and quality of the sleep of the last night and about the last meal timing and content. The EEGs were recorded in the morning. During the acquisition stage, the patients kept their eyes closed but remained awake. In order to prevent the sleepiness of the patient, the technician kept the subject alert, by calling her/him name. In fact, observing the EEGs, the recordings did not show any sleep patterns. The EEG duration was about 5 minutes. EEG preprocessing: each n-channel EEG was processed through a sliding tem-poral window (5 second width). After the down sampling operation, the sampling rate was 256 Hz, so one window includes N = 5 X 256=1280 samples. This window was stored on a computer as a n X N (channels X samples) matrix. The EEG was splitted into m non-overlapping windows, where m is the number of windows and is function of the length of the EEG recording. Once the wavelet coherence was calculated, it was partitioned into the six EEG sub-bands: delta (0-4 Hz), theta (4-8 Hz), alpha1 (8 to 10 Hz), alpha2 (8 to 12 Hz), beta1 (12 to 18 Hz) and beta2 (18 to 30 Hz). Every sub-band refers to a dierent functional and physiological area of the brain.

Estimating WC from the EEG signals

The wavelet coecients of each W Cxy(t, f )are then averaged, over the frequency, in

order to come up with a single average value of wavelet coherence W Cxy(t), for a

given EEG epoch EEG(t) (where t = T 0 or t = T 1), in every specic sub-band under consideration f1− f2: W Cxy(t) = 1 f2− f1 Z f2 f1 W Cxy(t, f )df (2.1)

where f2 and f1 are the upper and lower frequency bounds of the sub-band.

and over time:

˜ W Cxy= 1 N N X 1 W Cxy(t) (2.2)

For each EEG window, a n X n matrix of average values of coherence has been evaluated, coming up with m coherence matrices. Finally, the m coherence matrices are averaged also over the time, in order to calculate, for each sub-band, the average coherence matrix C.

Results

Analysis of the coherence matrices. Figures 2.3, 2.4 show the average coherence evolution from time T0 to T1 in the six sub-bands (delta, theta, alpha1, alpha2,

(41)

beta1 and beta2). It is to be noted that each sub-band is characterized by a 2D

representation (gure 2.2) where the pixel i,j correspond to the average coherence value of the pairs of electrodes i,j and is represented with a coloration from dark blue (0) to dark red (1). As regards MCI converted to AD: in Patient AD-03, at time T0, parietal-occipital zone was characterized by high coherence values in all sub-bands and in particular in the alpha 2 sub-band; at time T1, the coherence moved in the left frontal temporal area identify by Fp1, F7, F3, T3 electrodes, in all sub-bands. In Patient AD-71 (gure 2.3) and Patient AD-51 at time T0, coherence was uniformly distributed in all areas and all sub-bands; at time T1 we detected a signicantly reduction of coherence and concentrated in the frontal zone especially in alpha 2 and beta sub-bands. As regards MCI not converted to AD: Patient MCI-23 and Patient MCI-30 show a similar behavior to the Patient AD-03 described above: coherence values moved from parietal-occipital zone to the frontal temporal area in all sub-bands. Patient MCI-41, Patient MCI-57 do not have signicant changings from time T0 to T1. Co-herence remained between the frontal temporal electrodes. Figure 2.4 shows the behavior of Patient MCI-41. As regards Patient MCI-72 we observed an increase of coherence from time T0 to time T1 in all areas and sub-bands.

Fig. 2.2. 2D representation of the average coherence of the six sub-bands at time T0 and time T1 for the patient MCI-41.

Boxplots Analysis. Figure 2.5 reports the bloxplot representation of the estimated parameters coherence values of MCI (blue box) and AD (red box) patients at time T0 and T1. As regards MCI converted to AD: the boxplots showed a reduction of the median of the coherence in all sub-bands, except delta band in Patient AD-51 and beta2 band in Patient AD-03. Moreover, a strong reduction, not only of medians but also of the range of coherence can be observed in Patient AD-03 (delta band), Patient AD-51 (theta, alpha(s), beta(s) bands), Patient AD-71 (every bands). As regards MCI not converted to AD: the median of coherence was mostly constant even if Patient MCI-72 showed an increase of the median in all bands, Patients MCI-57 a decrease in

Brain network analysis and deep learning models for studying neurological disorders based on EEG signal processing

BRAIN NETWORK ANALYSIS AND

DEEP LEARNING MODELS FOR

STUDYING NEUROLOGICAL DISORDERS

BASED ON EEG SIGNAL PROCESSING

C

IERACITANO

Prof. F

C

MORABITO

Dr. N

MAMMONE

P

rof. F

ARENA

C

I

BRAIN NETWORK ANALYSIS AND

DEEP LEARNING MODELS FOR

STUDYING NEUROLOGICAL DISORDERS

Acknowledgements

Contents

List of Tables

List of Figures

I

Introduction

I.1 Thesis structure

I.2 List of publications

1

Electroencephalogram and Brain Electrical

Connectivity

1.1 Introduction

1.2 The brain: an overview

1.3 Electroencephalogram (EEG)

1.4 Brain electrical connectivity

1.5 Neurological disorders

2

Brain Network Analysis based on EEG Signal

Processing for Studying Neurological Disorders

2.1 Introduction

2.2 Brain network evolution in Alzheimer's disease patients