Improving generalization of LSTM NNs with data augmentation using the capacitive sensor data

Politecnico di Torino

Master Degree Course in Mechatronic Engineering

Master Degree Thesis A.A. 2021/2022

July 2022

Improving generalization of LSTM NNs with data augmentation using the capacitive

sensor data

Supervisors: Candidate:

Prof. Luciano Lavagno Prof. Mihai Lazarescu

Lei Yuping

S mmar

Machine learning is a general term for a class of algorithms that attempt to e tract the implied la s from a large amo nt of historical data and se them for prediction or classification. More specificall , machine learning can be seen as finding a f nction here the inp t is sample data and the o tp t is the desired res lt, e cept that this f nction is so comple that it is less con enient to e press formall . It is important to note that the goal of machine learning is to make the learned f nction ork ell for the "ne sample", not j st to perform ell on the training sample. The abilit to appl the learned f nction to ne samples is called generali ation. Ne ral net ork is a machine learning model, in this thesis ork, it ill be sed to predict the trajector of a person in a 3m * 3m room.

We sed fo r capaciti e sensors operating in loading mode for indoor h man locali ation, each one ith a sensing plate installed at chest le el in the center of a all of the room. These sensors pro ide reading fi e times per second. The gro nd tr th as obtained sing fo r ltraso nd anchors that can locali e a mobile tag ith 2 cm acc rac at 15H . When a person ears the tag and alks aro nd the room, the ltraso nd anchors record the reference trajector . At the same time, the capaciti e sensors also records position-related data.

We cond cted fo r e periments at different times to obtain fo r data sets. Different en ironmental conditions (temperat re, h midit and so on) affected differentl each data set meas rements. Each e periment has a different alking elocit and a different trajector , b t the all lasted 30 min tes.

To impro e the reliabilit of the tracking s stem, e process the capaciti e sensor data sing a ne ral net ork (NN). The NN I sed is a 2 la ers LSTM ith 8 cells per hidden la er. Ho e er, the generali ation abilit of the ne ral net ork is not so good beca se e don't ha e s fficient e perimental data. To sol e this problem, I sed data a gmentation techniq e. Data a gmentation is a techniq e that artificiall e tends the training data set b allo ing limited data to prod ce more eq i alent data. It is an effecti e means to o ercome the shortage of training data. Common data

II

a gmentation methods based on image processing techniq es incl de flipping, cropping, rotation, translation, noise injection, etc. In m case, I sed the method of noise injection.

In this thesis, I chose t o t pes of noise to add to the training set: hite Ga ssian noise and pink noise. This is beca se hite Ga ssian noise is commonl fo nd in the en ironment, hile pink noise sim lates the drift of a capaciti e sensor. In addition, since noise is s all a mi t re of these t o t pes of noise in dail life, this thesis also considers combining these t o t pes of noise together in different a s to form mi ed noise. The parameter to be determined for hite noise is the ariance,

hile for pink noise it is amplit de.

After data pre-processing and determined the most performing data set and the best train- alidation-test seq ence perm tation of that set (more details in Chapter 3), I began the e ploration of data a gmentation. In order to determine hether a training res lt has impro ed and b ho m ch, I created a baseline for comparison. Using the original training set, repeat the training 30 times.

Beca se the initial eights are randomi ed for each training, the res lts of the training ill be different. Taking the best among 30 trainings as the baseline. To select the best, the first criterion is that the s m of fo r test MSEs is lo , the second criterion is the standard de iation of them is lo .

After b ild the baseline, I started to impro e the NN training sing data a gmentation. I added noise to the original training set ia MATLAB. Before adding noise, I rescale the data, and after adding the noise, I rescale again so that the nois data is bet een 0 and 1, to impro e the learning abilit of the ne ral net ork. After e ploration for optimi ation (Section 3.4), I decided to create 6 training sets (5 nois sets + 1 original set), each training set becomes one batch for batch training, ith one batch at a time, to preser e the proper temporal seq ence of the e perimental data. In hite and pink noise part, I started from a parameter ( ariance for hite Ga ssian noise, amplit de for pink noise) al e lo er than the smallest signal, aro nd 10 ppm or belo , then grad all increase. For each noise parameter, the NN is trained 30 times ith the same data set. I pick the best among these 30 independent trainings for comparison.

After that, combining hite and pink noise together in different a s to form mi ed noise. Here are the steps:

- First, selecting the t o best-performing ariance al es in the hite noise e periment, fi ing them, and then ar ing the amplit de of the pink noise (aro nd the best t o amplit des of the

III

pink noise). Each combination of parameters is trained 30 times and pick the best.

- Second, selecting the t o best-performing amplit de al es in the pink noise e periment, fi ing them, and then ar ing the ariance of the hite noise (aro nd the best t o ariances al e of the hite noise). Each combination of parameters is trained 30 times and pick the best.

After all these e periments, I ha e come to this concl sion: the most significant impro ement as achie ed hen sing mi ed noise. The impro ement of s m of test MSEs respect to the baseline is 9.91%, the impro ement of standard de iation is 12.55% (Table 1). I achie ed the goal of impro ing the generali ation of NN.

Figure Virtual room used for movement tracking experiments

Table Comparison of the best results for white noise (variance . E- , pink noise (amplitude . E- and mixed noise (variance . E- , amplitude . E-

Test MSE A [𝑚²]

Test MSE B [𝑚²]

Test MSE C [𝑚²]

Test MSE D [𝑚²]

Improvement of Standard Deviation of test

MSEs [𝑚²]

Improvement of Sum of test

MSEs [𝑚²]

Baseline 0.0526 0.3098 0.2486 0.1798 0% 0%

White Noise 0.0509 0.2917 0.2412 0.1362 2.36% 8.95%

Pink Noise 0.0502 0.2997 0.2635 0.1358 -4.52% 5.25%

Mixed Noise 0.0612 0.2692 0.2441 0.1379 12.55% 9.91%

IV

Con en

Li of Table VI

Li of Fig re VII

1. Indoor per on locali a ion 1

1.1 Re ie of locali ation techniq es . . . 1

1.2 Capaciti e sensors for indoor locali ation . . . . 2

1.3 Capaciti e sensor mod le and data acq isition s stem . . . . 4

1.4 Ne ral net orks for person locali ation . . . 5

1.4.1 Main NN architect res . . . . 5

2. Impro ing generali a ion of ne ral ne ork i h da a a gmen a ion 9

2.1 Problems ith small data sets . . . . 9

2.2 Introd ction of data a gmentation . . . . 10

3. E perimen al re l 11

3.1 Methodolog . . . 11

3.2 Optimi er AdaMa . . . 15

3.3 Establish baseline . . . . 18

3.4 Data a gmentation . . . 20

3.4.1 Data a gmentation ith hite Ga ssian noise . . . . 21

3.4.2 Data a gmentation ith pink noise . . . 25

3.4.3 Data a gmentation ith mi ed noise . . . . 29

3.4.4 Concl sion . . . . 40

Bibliograph 41

V

Li of Table

Table 1 Comparison of the best res lts for hite noise ( ariance = 5.00E-06), pink noise (amplit de = 5.00E-05) and mi noise ( ariance = 5.00E-05, amplit de = 1.00E-05) Table 3.1 Baseline res lt

Table 3.2 Comparison of different n mber of training sets Table 3.3 White Ga ssian noise res lt

Table 3.4 Pink noise res lt

Table 3.5 Res lts hen the ariance of hite noise is fi ed at 5.00E-06 Table 3.6 Res lts hen the ariance of hite noise is fi ed at 7.50E-06 Table 3.7 Res lts hen the amplit de of pink noise is fi ed at 1.00E-05 Table 3.8 Res lts hen the amplit de of pink noise is fi ed at 5.00E-05 Table 3.9 The best res lts for different noise combinations

Table 4.1 Comparison of the best res lts for hite noise ( ariance = 5.00E-06), pink noise (amplit de = 5.00E-05) and mi ed noise ( ariance = 5.00E-05, amplit de = 1.00E-05)

VI

Li of Fig re

Fig re 1 Virt al room sed for mo ement tracking e periments

Fig re 1.1 Main sensor capacitances in load mode: plate-bod (𝐶 ), plate-gro nd (𝐶 ), and bod -gro nd (𝐶 )

Fig re 1.2 Fo r capaciti e sensors centered on the alls of a 3 m 3 m irt al room in the lab trace the position of a person mo ing in the space

Fig re 1.3 Main b ilding blocks of Sensor Node and Base Station. Fo r Sensor Nodes ere connected to a single Base Station

Fig re 1.4 (a) A simple 2-la er F ll Connected Ne ral Net ork; (b) Ne rons process the inp t data

Fig re 1.5 Acti ation f nctions graph

Fig re 1.6 Con ol tional Ne ral Net orks (CNN) Fig re 1.7 Rec rrent Ne ral Net orks (RNN) Fig re 1.8 Long short-term memor (LSTM)

Fig re 3.1 Net ork str ct re and data access for the Long-Short Term Memor (LSTM) net ork Fig re 3.2 Flo diagram of processing steps

Fig re 3.3 (a) SGD itho t Moment m; (b) SGD ith Moment m Fig re 3.4 Algorithm of Adam

Fig re 3.5 Algorithm of AdaMa

Fig re 3.6 Gro nd tr th s prediction for baseline

Fig re 3.7 Dependenc of test MSEs on ariance of the hite Ga ssian noise sed for data a gmentation

Fig re 3.8 Gro nd tr th s prediction for the best hite Ga ssian noise ( ariance = 5.00E-06) Fig re 3.9 Dependenc of test MSEs on amplit de of the pink noise sed for data a gmentation

VII

Fig re 3.10 Gro nd tr th s prediction for the best pink noise (amplit de = 5.00E-05)

Fig re 3.11 Dependenc of test MSEs hen changing the amo nt of pink noise for fi ed hite noise ariance 5.00E-06

Fig re 3.12 Dependenc of test MSEs hen changing the amo nt of pink noise for fi ed hite noise ariance 7.50E-06

Fig re 3.13 Dependenc of test MSEs hen changing the amo nt of hite noise for fi ed pink noise amplit de 1.00E-05

Fig re 3.14 Dependenc of test MSEs hen changing the amo nt of hite noise for fi ed pink noise amplit de 5.00E-05

Fig re 3.15 Gro nd tr th s prediction for the best mi noise ( ariance = 5.00E-05, amplit de = 1.00E-05)

VIII

1

Cha e 1

Indoo e on locali a ion

1.1 Re ie of locali a ion echni e

L -c , - ai e a ce, acc a e i d de ec i a d ca i a i f e ha a i c ea i g i a e a da . N ca i ide e ice f e e' c e ie ce, b i ca a a a e i he fie d f hea h ca e. F e a e, i he a ea f hea h ca e, die ha e h ha b 2050, he be f e de e e d ide i e i a ed be c e 2.1 bi i , e ha d b e he be i 2015 [2]. Wi h a e de e e, i i i a i he a ec f he e de , e ecia h e i i g a e. Th gh he i i g f a ec ie , he e ca de e i e he he he e de ha e ab a c di i , ha i e a e ca be i ed a d he e de ca ecei e i e he .

O e he ea , a i d i i i g ech i e ha e bee ed. F e a e, ide i agi g ca e a , a ic e , i f a ed e , e c. The ha e diffe e d a bac .

Ca e a: i ca e he i age ide , a d he e ce e he i age ide ca c a e he ca i f he e . I e i e high c a i a , e i g a d e e g e ce , a di ec i e f igh , a d ade a e igh i g, hich i c ea e he i a a i c e i a d e c . I a ai e ig ifica i ac c ce .

U a ic e : c i f a , a a i e a d a ecei e . U a , he a e aced ide b ide a ch a ib e. The a i e e i a ic i a ce ai di ec i a d a i i g a he a e i e a he e f e i i . The a ic a e h gh he ai a d e i edia e he i hi a b ac e he a , a d he ecei e i i g a a i ecei e

he ef ec ed a ic. The di a ce f he b ec ca be ca c a ed b ea i g he i e i a e f he a ic be ef ec ed b he b ec . U a ic e e i e he e ca a ag a d

g- e e e a ic i e ca ca e ha f hea h effec .

2

I f a ed e : The h a b d e e a e i ge e a a 36.5 deg ee , i e i i f a ed a i h a a e e g h f ab 10 . The i f a ed e b de ec i g i f a ed a f ab 10 e i ed b he h a b d . H e e , he a e e i i e a -h a hea ce , hich i c ea e he cha ce f fa e de ec i .

Ca aci i e e i g i a ed f h a de ec i , ca i a i a d ide ifica i . I i e ac ha e ed i e e i e . I de c ibe i i e de ai i Sec i 1.2.

1.2 Ca aci i e en o fo indoo locali a ion

Ca aci i e e i g i a ech g ha ca be ed de ec a d ac c d c i e a d - c d c i e b ec . Ca aci i e e e ca aci i e a d ce ha ca e a e i h ee diffe e

de ( c fig a i ): a i , h , a d ad. The fi de a e i d ced b Zi e a e a . [3] i 1995. The ca be ed f i d h a de ec i . H e e , he a d ce

e a i g f b h de e i e a ea ga a ica c ed a e , hich ig ifica i c ea e he c e i a d c f he i a a i . Tha f , he ad e a i de e he h a b d a a c a - e ia a e (Fig e 1.1) a d e i e e- a e a d ce , hich i a g d i he e i e i ed b e . S e f hei ad a age , ch a c , ea i a a i , a i ac c ce , e c., gi e a ea ch e he .

The ca aci a ce de e d i a i he ge e f he a e, die ec ic e ie f he e , di a ce be ee h a b d a d he a e. We i di ec ea e he cha ge i he ca aci a ce f he e b ea i g he f ee i g f e e c f a a ab e i ib a , hich e ea ed cha ge a d di cha ge he ca aci f he e .

O i d ca i a i e e i e a e c d c ed i a 3 * 3 . We ed f ca aci i e e e a i g i adi g de f i d h a ca i a i , each e i h a e i g a e i a ed a che e e i he ce e f a a f he (Fig e 1.2). The e e ide eadi g fi e i e e ec d. The g d h a b ai ed i g f a d a ch ha ca ca i e a bi e ag i h 2 c acc ac a 15H . Whe a e ea he ag a d a a d he ,

he a d a ch ec d he efe e ce a ec . A he a e i e, he ca aci i e e a ec d i i - e a ed da a.

T i e iga e he ea e e f ca aci i e e de diffe e c di i , e c d c ed f

3

e e i e a diffe e i e b ai f da a e . Diffe e e i e a c di i ( e e a e, h idi a d ) affec ed diffe e each da a e ea e e . Each e e i e ha a diffe e a i g eed a d a diffe e a ec , b he a a ed 30 i e .

Fig re Main en or capaci ance in load mode pla e bod 𝐶 pla e gro nd 𝐶 and bod gro nd 𝐶

Fig re Fo r capaci i e en or cen ered on he all of a m m ir al room in he lab race he po i ion of a per on mo ing in he pace

4

1.3 Ca aci i e en o mod le and da a ac i i ion em

Each e ha a c e c ad a e a ached a he e e a ca aci a 555 i eg a ed ci c i i a ab e i ib a c fig a i , f hich he ci a i f e e c i :

𝐹𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 1

0.7 𝑅 2𝑅 𝐶

he e 𝑅 200𝑘𝛺 a d 𝑅 560𝑘𝛺.

We ed a A d i U b a d ea e he f e e c , a d a XBee 802.15.4 de a i he ea e e a ce a de f - ce i g a d e ca i a i . The b c diag a

f ca i a i e i h i Fig. 1.3.

Fig re Main b ilding block of Sen or Node and Ba e S a ion Fo r Sen or Node ere connec ed o a ingle Ba e S a ion

5

1.4 Ne al ne o k fo e on locali a ion

T i e he e iabi i f he ac i g e , e ce he ca aci i e e da a i g a e a e . M e e , e ca a fi e he da a i g e a e , i ead f i g a digi a

edia fi e a i [4].

Ne a e ca ce a ge a f da a ic a d acc a e , a d he e c e ea - i e b e . Ne a e ca he e a i e a d de c e a d i ea a cia i be ee i e a iab e d a i fe e ce . A e a e ai i e f b c a ad i g

he c ec i be ee i e . Thi a he e a e be c i i ed.

1.4.1 Main NN a chi ec e

I ge e a , he ba ic c e f e a e de ca be di ided i e acc di g he he he i f a i i i fed bac : Feedf a d Ne a Ne a d Feedbac Ne a

Ne .

I Feedf a d Ne a Ne , he i f a i e e ed f he i a e , a d he e i each a e ecei e he i f he e i a e a d he e a e i he a e . The e i feedbac i he a i i f i f a i h gh he e .

C Feedf a d Ne a Ne i c de C i a Ne a Ne (CNN), F C ec ed Ne a Ne (FCN), Ge e a i e Ad e a ia Ne (GAN), e c.

I Feedbac Ne a Ne , e ca ecei e ig a f he e , b a f he e e . C a ed i h he Feedf a d Ne a Ne , he e i he Feedbac Ne a Ne ha e a e f c i a d ha e diffe e a e a diffe e e . The i f a i aga i f Feedbac Ne a Ne ca be i g e bidi ec i a .

C Feedbac Ne a Ne i c de Rec e Ne a Ne (RNN), L g Sh - Te Me e (LSTM), B a Machi e , e c.

Thi ec i b ief i d ce he f i g c NN a chi ec e ：FCN、CNN、RNN、

LSTM.

F C ec ed Ne a Ne (FCN): a e he c e c e f dee ea i g. I ha h ee ba ic e f a e : i a e , hidde a e , a d a e . Each e i

6

he c e a e i c ec ed each e i he e i a e . D i g each c ec i , he ig a f he e i a e i i ied b a eigh , a d a bia i added, he a b a ac i a i f c i (Fig e 1.4, Fig e 1.5), ch a ig id, a h, Re , e c. C e a i g f i

ace ace i achie ed b i e c i i f i e i ea f c i .

(a) (b)

Fig re a A imple la er F ll Connec ed Ne ral Ne ork b Ne ron proce he inp da a

Fig re Ac i a ion f nc ion graph

7

C i a Ne a Ne (CNN): I age ha e e high di e i a i , ai i g a a da d feedf a d e ec g i e i age d e i e a h ge a f e . I i

e c a i a i e i e, b ca a ead a b e a cia ed i h di e i a

di a e i e a e . C i a e a e ide a i ha e c i

a d i g a e ed ce he di e i a i f a i age. Si ce he c i a a e i ai ab e b ha ig ifica fe e a a e e ha a a da d hidde a e , i i ab e high igh he i a

a f he i age a d aga e each i a a f a d.

Fig re Con ol ional Ne ral Ne ork CNN

Rec e Ne a Ne (RNN): a e a ecia e f e ha c ai a d e f- e e i i . B a i g i f a i be ed i he e , RNN e i fe e ce f e i

ai i g a e be e i fe e ce ab c i g e e . D e hei a e, RNN a e c

ed f e ge e e ia a ch a ge e a i g e d b d edic i g i e e ie da a.

8

F gire Rec rren Ne ral Ne ork RNN

L g h - e e (LSTM): i ecifica de ig ed e he b e f g adie a i hi g a d e i ha cc he RNN ea g c e a i f a i , i h e b c i c a ed i he c e. Each b c c ai e e a c c ica c ec ed e ce a d h ee ga e (i , a d f ge ). The i f i f a i ca i e ac i h e

h gh each ga e, he e ga e ea e a d c e i e ige e e g adie f e di g a i hi g.

Fig re Long hor erm memor LSTM

9

Cha e 2

Im o ing gene ali a ion of ne al ne o k i h da a a gmen a ion

Whe e e e ai e a e , e eed ee a e e he ge e a i a i f he e a e . E e ia , hi efe h e he de ea f he gi e da a a d a ie ha i ea e e he e. A e a e i h g d ge e a i a i i be ab e d ce he c ec i -

a i g e e he he i da a i igh diffe e f he ai i g a e .

2.1 P oblem i h mall da a e

Whe ai i g a e a e , i i ea ca e e fi i g if he da a e i e a i e a . Tha i , he high-di e i a i ace c e di g he a a e i e i a e, a d i i diffic f he e a e ea he a i g e a i hi f i .

S a da a e e a b e he ai i g a ge e a e .

The fi b e i ha he e ca effec i e e i e he ai i g da a e . The de ca ea ecific i e a e a d hei a cia ed a he ha ea i g a ge e a a i g f i . Thi i e i a de ha e f e he ai i g da a e a d

e da a, i.e., ge e a i a i .

The ec d b e i ha a da a e ide e i de c ibe he c e f he i ace a d i e a i hi he . M e ai i g da a ide a iche de c i i f he b e ha he de ca ea . Le da a ea ci a a d di c e e i ace a he ha

h i ace , hich a a e i e diffic f he de ea fea e a i g .

10

2.2 In od c ion of da a a gmen a ion

O e e h d h he i ace i da a a g e a i , hich faci i a e e a e ea i g. Da a A g e a i i a ech i e ha a ificia e e d he ai i g da a e b a i g i i ed da a d ce e e i a e da a. I i a effec i e ea e c e he h age f ai i g da a. C da a a g e a i e h d ba ed i age ce i g ech i e i c de f i i g, c i g, a i , a a i , i e i ec i , e c.

I hi he i , I e he e h d f i e i ec i . B addi g i e d i g ai i g, he abi i f he de ea a i g e f he i ace ca be i ed, a d he ge e a i a i abi i a d e e a ce f he de ca be i ed.

Thi e h d a ha he a e effec a eg a i a i , hich i i e he b e f he de . A i h he eigh eg a i a i e h d, i ha bee h ha addi g i e ha a i i a effec he f c i a he e a e .

11

Cha e 3

E e imen al e l

3.1 Me hodolog

I de gi e he e a e g d e e i e e hi e a i g e e ce , I ed 2 a e LSTM i h 8 ce e hidde a e . The g a f he he i i de a d h diffe e e f i e affec he 2 a e -LSTM e a e a d i e he ge e a i a i abi i f he e a e b addi g i e.

F each i e a a e e , he e a e i ai ed 30 i e i h he a e da a e . Beca e he i i ia eigh a e a d i ed f each ai i g, he e f he ai i g i be diffe e . I ic he be a g he e 30 i de e de ai i g f c a i . The be ai i g ea b h g d edic i acc ac a d g d ge e a i a i abi i . I ca be e e e ed b f e Mea S a e E (MSE) a d a da d de ia i f f e e , e ec i e . T e ec he be , he fi c i e i i ha he f f e MSE i , he ec d c i e i i he a da d de ia i f he i .

The Mea S a e E i defi ed a :

𝑀𝑆𝐸 𝑦 𝑦

𝑁

he e i he abe , 𝑦 i he , N i he be f a e . The i f ea e e f MSE i 𝑚 .

I he i , I ch e e f i e add he ai i g e : hi e Ga ia i e a d i i e. Thi i beca e hi e Ga ia i e i c f d i he e i e , hi e i i e i a e he d if f a ca aci i e e . I addi i , i ce i e i a a i e f he e e f i e i dai ife, hi he i a c ide c bi i g he e e f i e ge he i diffe e a f i ed i e.

12 I ca ied he f i g e :

S e 1: P e- ce i g f da a. T e e cca i a ga a d i e i he e e i e a da a, he ca aci i e e da a e e e a ed a 5 H . T i e he ea i g abi i f he e a e , I e ca ed he da a i he i e a [0, 1] f each e e a a e . Se he i i d i e 5 , beca e i [1] i ha bee ed be he i a c fig a i .

S e 2: B i d he e a e . Fig e 3.1 h a ica LSTM c e. A a ead e i ed, he e a e I e i a 2- a e LSTM i h 8 ce e hidde a e . Each a e i f ed b a 50% d ( a id e -fi i g). The he a e . The i i e I ed i AdaMa . Sec i 3.2 ha a i d c i i i e AdaMa . I he Ke a API f c i fi (), he e a e ba ch ge e a f c i , e i ai i g ba ch ge e a , a he i a ida i ba ch ge e a . I da a a g e a i a , he e a e i ai i g e , he fi e i he igi a ai i g e , he a e a ied i e i h he a e a a e e he igi a a e . E e ba ch ge e a a e he NN i h e da a e i each ba ch f a a f 6 ba che e e ch, i g he h ff e de f he da a e . Si ce i ca e, e ce a " ie" f he e e e , he de f he a e i e i a (e c de he d a ic e e f he e ), e h d e e h ff e he a e , e ca h ff e he de f he da a e .

Fig re Ne ork r c re and da a acce for he Long Shor Term Memor LSTM ne ork in hich he LSTM cell in he inp la er proce he da a ple from he inp

indo Each indo i labelled for raining i h he per on coordina e corre ponding o he middle ple in he indo no ho n for readabili

13

S e 3: A g 4 da a e f diffe e e e i e , he ai i g e i h he be ai i g e (g d edic i acc ac a d g d ge e a i a i abi i ) i ide ified. I hi ce , I ai ed,

a ida ed a d e ed each f he e 4 da a e a d a c ide ed a e a i f he ai i g- a ida i - e e e ce. I he e d, I e ec ed he fi 20% a e f e A a he a ida i e , he a 60% a he ai i g e , a d he e a he e i g e .

S e 4: Ob ai a c a i ba e i e b a i g he be ai i g e f a ea 30 ai i g . A b e e da a a g e a i e e i e i be c a ed i de e i e if a d h ch i e e he e i . I i a e ab hi i Sec i 3.3.

S e 5: Add a i e f i e（ hi e, i , i ）, e a a i e, he a e f he ai i g e e ce f e A a d chec he effec he a ida i a d e i g e de he a e c di i a ba e i e. M e de ai i Sec i 3.4.

The f diag a f ce i g e i h i Fig e 3.2.

14

Fig re Flo diagram of proce ing ep

15

3.2 O imi e AdaMa

Bef e i d ci g AdaMa , I b ief i d ce he a g i h : M e a d RMS . M e i a i i a i f he S cha ic G adie De ce (SGD) a g i h . I i d ce

e he ba i f SGD. F a h ica i f ie , he i d c i f e ca add ai ad a age e he SGD. Fi , he e c e i g a ca i , i i ib e ch he ca i de he ac i f e . Sec d, he SGD a g i h i c e e de e i ed b he g adie , hich a ead e i h c i he ce f fi di g he i a i a d d he eed, h e e i he ca e f e , he di ec i f i i de e i ed b b h e a d g adie , hich ca a e he ci a i ea e a d

e fa e he i a i (Fig e 3.3).

(a) (b) Fig re a SGD i ho Momen m b SGD i h Momen m

He e i he i e e a i de ai f M e . O i e a i :

C e dW, db he c e i i-ba ch

𝑣 𝛽 𝑣 1 𝛽 𝑑𝑊

𝑣 𝛽 𝑣 1 𝛽 𝑑𝑏

𝑊 𝑊 𝛼𝑣

𝑏 𝑏 𝛼𝑣

he e W i eigh , dW i g adie f eigh , b i bia , db i g adie f bia , 𝛼 i a

h e a a e e ( ea i g a e), 𝛽 i a h e a a e e ( e e e he effec f a g adie he c e g adie ).

16

RMS , he f a e i ea a e . I ca a e i i a e ci a i i g adie de ce , a d a he e f a a ge ea i g a e, h eedi g a g i h ea i g.

He e i he i e e a i de ai f RMS . O i e a i :

C e dW, db he c e i i-ba ch

𝑆 𝛽 𝑆 1 𝛽 𝑑𝑊

𝑆 𝛽 𝑆 1 𝛽 𝑑𝑏

𝑊 𝑊 𝛼

𝑏 𝑏 𝛼

he e W i eigh , dW i g adie f eigh , b i bia , db i g adie f bia , 𝛼 i a

h e a a e e ( ea i g a e), 𝛽 i a h e a a e e ( e e e he effec f a g adie he c e g adie ), 𝜀 i a a be a id a de i a f 0.

The Ada i i a i a g i h i ba ica a c bi a i f M e a d RMS . I i a c ea i g a g i h ha ha bee h be effec i e f diffe e e a e . AdaMa i a a ia f Ada ba ed i fi i . S e i e AdaMa i be e ha Ada , e ecia i de i h e beddi g. I ca e, I ed AdaMa . The e a g i h e e fi

ed b Diede i P. Ki g a a d Ji Lei Ba i [12]. The a g i h a e h i hei a e (Fig e 3.4, Fig e 3.5).

17

Fig re Algori hm of Adam

Fig re Algori hm of AdaMa

18

3.3 E abli h ba eline

I de de e i e he he a ai i g e ha i ed a d b h ch, e eed c ea e a ba e i e f c a i . The a 60% f he a e f e A a e ed a he ai i g e , he fi 20% a e a he a ida i e , a d he e a he e i g e . Re ea he ai i g 30 i e . Ta i g he be a g 30 ai i g . The be e i h i Tab e 3.1. The MSE a e h ha he e e B, C a d D c ib e a he f MSE, e ecia B a d C, hi e he c ib i f A i e a . I ca a be ee f Fig e 3.6 ha he edic i acc ac f A i e a i e high, he edic i c e f efe e ce c e e , hi e B, C a d D a e . E ecia e e B a d D, e i e he g i he i e di ec i f he g d h.

Table Ba eline re l

Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ]

Baseline 0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

19

Fig re Gro nd r h predic ion for ba eline

20

3.4 Da a a gmen a ion

Whe ai i g a e a e , i i ea ca e e fi i g if he da a e i e a i e a . The NN ca effec i e e i e he ai i g da a e . I ca ea ecific i e a e a d hei a cia ed a he ha ea i g a ge e a a i g f i . Thi i e i a de ha e f e he ai i g da a e a d e da a, i.e., ge e a i a i . T

e hi b e , I ed da a a g e a i ech i e.

Da a a g e a i i a ech i e ha a ificia e e d he ai i g da a e b a i g i i ed da a d ce e e i a e da a. I i a effec i e ea e c e he h age f ai i g da a. N i e i ec i i a e h d f da a a g e a i .

I added he i e he igi a ai i g e ia MATLAB. Bef e addi g he i e, I e ca e he da a, a d af e addi g he i e, I e ca e agai ha he i da a i be ee 0 a d 1, i e he ea i g abi i f he e a e . B fi f a , I h d decide h a be f ai i g e h d be c ea ed. Ta i g hi e Ga ia i e i h a ia ce a e 5.00E-06 (I i ed be he be a ia ce a e f hi e Ga ia i e i Sec i 3.4.1) a a e a e, I did a e e i e c a e he ai i g e f diffe e be f ai i g e . Tab e 3.2 h he e . Si ce he fi c i e i i he f f e MSE , he ec d c i e i i he a da d de ia i f he , i he e d, I decided c ea e 6 ai i g e (5 i e + 1 igi a e ), each ai i g e bec e e ba ch f ba ch ai i g. I hi a , he ai i g da a bec e 6 i e e ha bef e.

Table Compari on of differen n mber of raining e Number of

training sets

Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ] 1(Baseline) 0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

6 0.0509 0.2917 0.2412 0.1362 0.1077 0.7200

10 0.0566 0.2924 0.2392 0.1507 0.1035 0.7389

15 0.0534 0.2902 0.2701 0.1394 0.1121 0.7531

20 0.0535 0.3103 0.2370 0.1342 0.1129 0.7350

21

3.4.1 Da a a gmen a ion i h hi e Ga ian noi e

Whi e Ga ia i e (WGN) i a ba ic i e de ed i i f a i he i ic he effec f a a d ce e ha cc i a e. Whi e efe he idea ha i ha if e ac he f e e c ba d f he i f a i e . I i a a a g he c hi e hich ha if e i i a a f e e cie i he i ib e ec . Ga ia beca e i ha a a di ib i i he i e d ai i h a a e age i e d ai a e f e [13].

F hi e i e, ea a e i 0 a d ea e e a a ia ce. The highe he a ia ce, he highe he e f he i e, i.e. he ge he i e. S he a a e e be decided i he a ia ce.

Va ia ce e a 0 ea i e i added. S a i g f a a ia ce a e (1.00E-07) e ha he a e ig a , a d 10 be a d he g ad a i c ea e. A i h ba e i e, he ai i g

a e ea ed 30 i e f each a ia ce a e, a d he he be ai i g a e ec ed.

A he begi i g, e e he e d f ai i g e i h a ia ce a e , I ch e he f i g a ia ce: 1.00E-07, 5.00E-07, 1.00E-06, 5.00E-06, 1.00E-05, 5.00E-05. B c a i g he e (Tab e 3.3) I f d ha 5.00E-06 a he be . S e I c i ed e e a d 5.00E-06. The

a ia ce a e I ch e a : 2.5E-06, 4.00E-06, 6.00E-06, 7.50E-06. Fi a I f d ha 5.00E-06 a i he be a d 7.50E-06 a a g d. The a e he be a ia ce a e .

T a e he da a e i i i e, I e e he da a i he f f (Fig e 3.7). I ca be ee f he , he he a ia ce i a , he i e e f he e a e i i e i i ed.

Whe he a ia ce eache 5.00E-05, he ai i g e a e e i ead, b h a da d de ia i a d f e MSE i c ea e.

F he ab e a d , e ca ha a a ia ce a e e ce 1.00E-06 a d 5.00E-05 ca a e he f MSE dec ea e. H e e , f a da d de ia i , 1.00E-07, 4.00E-06, 5.00E- 06 a d 7.50E-06 dec ea e, hi e he he i c ea e i ead, hich i dica e ha he ge e a i a i abi i d e i e. Whe he a ia ce i e a 5.00E-06, b h f MSE a d a da d de ia i a e he e , hich i dica e ha b h edic i acc ac a d ge e a i a i abi i ha e i ed. I deed, a ca be ee f he ab e, he he a ia ce i 5.00E-06, he MSE f a f

e e dec ea e , e ecia e D dec ea ed a .

Fig e 3.8 h he g d h a d edic i f he be hi e Ga ia i e f d. F he X a i f e e B, he e i a ig ifica i e e i he i e a f ab he 4000 h a e

22

he 6000 h a e. A he ig ifica i e e a ea i he Y a i f e e D, a i g f ab he 6000 h a e he e d. I deed, i ca be ee f Tab e 3.3, B a d D h e i e e ha A a d C.

Table Whi e Ga ian noi e re l

Variance Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ] 0(Baseline) 0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

1.00E-07 0.0492 0.2777 0.2755 0.1759 0.1079 0.7782

5.00E-07 0.0513 0.2796 0.2855 0.1643 0.1110 0.7807

1.00E-06 0.0459 0.2942 0.2864 0.1696 0.1169 0.7961

2.50E-06 0.0474 0.3176 0.2457 0.1333 0.1195 0.7440

4.00E-06 0.0580 0.2527 0.3040 0.1451 0.1101 0.7598

5.00E-06 0.0509 0.2917 0.2412 0.1362 0.1077 0.7200

6.00E-06 0.0537 0.3036 0.2877 0.1370 0.1207 0.7820

7.50E-06 0.0531 0.2842 0.2583 0.1352 0.1081 0.7308

1.00E-05 0.0544 0.2885 0.2663 0.1363 0.1107 0.7455

5.00E-05 0.0454 0.3355 0.2780 0.1468 0.1306 0.8058

23

Fig re Dependenc of e MSE on ariance of he hi e Ga ian noi e ed for da a a gmen a ion

24

Fig re Gro nd r h predic ion for he be hi e Ga ian noi e ariance E

25

3.4.2 Da a a gmen a ion i h ink noi e

Pi i e i a ig a ce i h a f e e c ec ch ha he e ec a de i ( e e f e e c i e a ) i i e e i a he f e e c f he ig a [14].

Pi i e ha highe a i de f e f e e cie , i e a e he d if f he e . I hi e e i e , he a a e e be decided i he a i de f i i e. Re ea ai i g 30 i e a d

ic he be .

S a i g f a a i de a e e ha he a e ig a , a d 10 be , he g ad a i c ea e. A he begi i g, e e he e d f ai i g e i h a i de a e , I ch e he f i g a i de : 1.00E-06, 5.00E-06, 1.00E-05, 5.00E-05, 1.00E-04, 5.00E-04, 0.001.

F Tab e 3.4 i ca be ee , a h gh he e f e MSE (0.7375 𝑚 ) a f d he a i de = 1.00E-05, he e MSE dec ea ed f e e (C a d D). H e e e e h gh he f MSE i he e he a i de = 5.00E-05, he e a e h ee e MSE ha dec ea e.

The ef e, a g a he e a i de a e , i ca be c ide ed ha 5.00E-05 e f he be . Ne I c i ed e e a d 5.00E-05. The a i de a e I ch e a : 2.5E-05, 7.50E-05.

B he c c i e ai he a e.

Agai , I e e he da a i he f f (Fig e 3.9). I ca be ee f he , Whe he a i de i a big, he i e e f he e a e i g d. F a a i de

a e , he a da d de ia i i c ea ed, f MSE dec ea ed (e ce f 2.50E-05), b ig ifica .

Fig e 3.10 h he g d h a d edic i f he be i i e (a i de = 5.00E-05) f d. C i e i h he da a i Tab e 3.4, e e D ha a ig ifica i e e , a i g f ab he 6000 h a e he e d.

26

Table Pink noi e re l

Amplitude Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ] 0(Baseline) 0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

1.00E-06 0.0476 0.2736 0.3108 0.1554 0.1195 0.7874

5.00E-06 0.0450 0.3472 0.2440 0.1510 0.1291 0.7871

1.00E-05 0.0532 0.3173 0.2300 0.1370 0.1143 0.7375

2.50E-05 0.0467 0.2903 0.2951 0.1714 0.1176 0.8034

5.00E-05 0.0502 0.2997 0.2635 0.1358 0.1153 0.7493

7.50E-05 0.0444 0.2678 0.3038 0.1603 0.1169 0.7763

1.00E-04 0.0515 0.3293 0.2283 0.1383 0.1193 0.7473

5.00E-04 0.0464 0.2909 0.2829 0.1445 0.1176 0.7647

0.001 0.0523 0.3048 0.2745 0.1588 0.1155 0.7904

27

Fig re Dependenc of e MSE on ampli de of he pink noi e ed for da a a gmen a ion

28

Fig re Gro nd r h predic ion for he be pink noi e ampli de E

29

3.4.3 Da a a gmen a ion i h mi ed noi e

I ea ife, i i f e he ca e ha diffe e i d f i e a ea a he a e i e. T be e i a e hi i a i , I i ed hi e i e a d i i e ee h i d affec he e . I he e i e e i e , I ha e f d he be - e f i g hi e a d i i e, a d I i e he e c d c e e i e i h i ed i e. He e a e e :

- Fi , e ec i g he be - e f i g a ia ce a e (5.00E-06 a d 7.50E-06) i he hi e i e e e i e , fi i g he , a d he a i g he a i de f he i i e (a d he be a i de f he i i e 1.00E-05 a d 5.00E-05). Each c bi a i f a a e e i ai ed 30 i e a d ic he be .

- Sec d, e ec i g he be - e f i g a i de a e (1.00E-05 a d 5.00E-05) i he i i e e e i e , fi i g he , a d he a i g he a ia ce f he hi e i e (a d he be a ia ce f he hi e i e 5.00E-06 a d 7.50E-06). Each c bi a i f

a a e e i ai ed 30 i e a d ic he be .

30

Tab e 3.5 h he e f i i g diffe e i i e he he a ia ce a e f hi e i e i 5.00E-06. A ca be ee f Fig e 3.11, he a ia i f a da d de ia i i ab e, hi e

he f e MSE c i e i e i he i e a [0.000001, 0.0001]. The e a e e f be he he a i de f i i e i e a 1.00E-05, h i g a da d de ia i (0.1078 𝑚 ) a d f e MSE (0.7309 𝑚 ).

Table Re l hen he ariance of hi e noi e i fi ed a E Pink Noise

Amplitude

Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ]

Baseline 0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

1.00E-06 0.0518 0.2892 0.2853 0.1353 0.1170 0.7615

2.50E-06 0.0489 0.2998 0.2926 0.1384 0.1226 0.7797

5.00E-06 0.0578 0.3022 0.2521 0.1495 0.1089 0.7615

7.50E-06 0.0574 0.3222 0.2626 0.1427 0.1189 0.7849

1.00E-05 0.0491 0.2807 0.2586 0.1425 0.1078 0.7309

2.50E-05 0.0511 0.3116 0.2370 0.1540 0.1119 0.7537

5.00E-05 0.0492 0.2975 0.2599 0.1405 0.1136 0.7471

7.50E-05 0.0539 0.2951 0.2495 0.1459 0.1080 0.7444

0.0001 0.0478 0.2894 0.2786 0.1548 0.1143 0.7706

31

Fig re Dependenc of e MSE hen changing he amo n of pink noi e for fi ed hi e noi e ariance E

32

Tab e 3.6 h he e f i i g diffe e i i e he he a ia ce a e f hi e i e i 7.50E-06. Si i a he ca e he e he hi e i e a ia ce i 5.00E-06, he a ia i f a da d de ia i i ab e, hi e he f e MSE c i e i e i he i e a (Fig e 3.12).

The e a e e f e he he a i de i e a 1.00E-05 a d 7.50E-05. I ch e he e i h a a i de f 7.50E-05 a he be . Beca e c a ed i h 1.00E-05, hi e he

f e MSE f 7.50E-05 i 0.075% highe , he a da d de ia i i 3% e .

Table Re l hen he ariance of hi e noi e i fi ed a E Pink Noise

Amplitude

Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ]

Baseline 0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

1.00E-06 0.0588 0.3258 0.2350 0.1344 0.1166 0.7540

2.50E-06 0.0570 0.2882 0.2894 0.1437 0.1144 0.7783

5.00E-06 0.0523 0.2919 0.2706 0.1456 0.1123 0.7605

7.50E-06 0.0653 0.3106 0.2518 0.1481 0.1090 0.7758

1.00E-05 0.0552 0.2831 0.2391 0.1414 0.1020 0.7188

2.50E-05 0.0530 0.2941 0.2725 0.1433 0.1134 0.7629

5.00E-05 0.0584 0.2960 0.2850 0.1402 0.1154 0.7796

7.50E-05 0.0549 0.2728 0.2430 0.1488 0.0987 0.7194

0.0001 0.0549 0.3187 0.2570 0.1519 0.1164 0.7826

0.0005 0.0481 0.2959 0.2650 0.1418 0.1144 0.7508

33

Fig re Dependenc of e MSE hen changing he amo n of pink noi e for fi ed hi e noi e ariance E

34

Tab e 3.7 h he e f i i g diffe e hi e i e he he a i de f i i e i 1.00E-05. Fig e 3.13 h he . We ca ee ha b h a da d de ia i a d f e MSE

a i e ha a i g f a a ia ce f 7.50E-05. I ca be a ed ha he high i e i c d ci e he ai i g f he e a e . The e a e e f be he he a ia ce

f hi e i e i e a 5.00E-05, h i g e a da d de ia i (0.0965 𝑚 ) a d e f e MSE (0.7124 𝑚 ).

Table Re l hen he ampli de of pink noi e i fi ed a E White Noise

Variance

Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ]

0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

2.50E-07 0.0553 0.2986 0.2545 0.1483 0.1093 0.7567

5.00E-07 0.0487 0.2789 0.2864 0.1604 0.1125 0.7744

7.50E-07 0.0538 0.3110 0.2509 0.1422 0.1143 0.7580

1.00E-06 0.0467 0.2867 0.2767 0.1324 0.1164 0.7426

2.50E-06 0.0526 0.2918 0.2668 0.1440 0.1114 0.7552

5.00E-06 0.0491 0.2807 0.2586 0.1425 0.1078 0.7309

7.50E-06 0.0486 0.2887 0.2637 0.1466 0.1111 0.7476

1.00E-05 0.0491 0.3055 0.2754 0.1470 0.1187 0.7769

2.50E-05 0.0689 0.3022 0.2532 0.1466 0.1050 0.7709

5.00E-05 0.0612 0.2692 0.2441 0.1379 0.0965 0.7124

7.50E-05 0.0537 0.3371 0.2826 0.1377 0.1302 0.8110

1.00E-04 0.0607 0.3051 0.2906 0.1400 0.1187 0.7965

2.50E-04 0.0567 0.3270 0.3311 0.1316 0.1390 0.8463

35

Fig re Dependenc of e MSE hen changing he amo n of hi e noi e for fi ed pink noi e ampli de E

36

Tab e 3.8 h he e f i i g diffe e hi e i e he he a i de a e f i i e i 5.00E-05. F he ab e a d (Fig e 3.14), I ca be ee ha he a da d de ia i f e MSE i highe ha he ba e i e i he h e i e a , a d he f e MSE i i ed i he i e a [2.50E-07, 1.00E-05] (e ce f 1.00E-06), h e e he i e e i ch. I

hi ca e, he be e i he he hi e i e a ia ce i e a 1.00E-05.

Table Re l hen he ampli de of pink noi e i fi ed a E White Noise

Variance

Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ]

Baseline 0.0526 0.3098 0.2486 0.1798 0.1103 0.7908

2.50E-07 0.0500 0.2830 0.2865 0.1663 0.1125 0.7857

5.00E-07 0.0478 0.3027 0.2683 0.1585 0.1154 0.7773

7.50E-07 0.0605 0.3353 0.2378 0.1412 0.1190 0.7747

1.00E-06 0.0542 0.3131 0.2744 0.1622 0.1169 0.8038

2.50E-06 0.0482 0.3421 0.2537 0.1350 0.1294 0.7791

5.00E-06 0.0580 0.3022 0.2710 0.1422 0.1137 0.7734

7.50E-06 0.0522 0.3141 0.2853 0.1394 0.1236 0.7909

1.00E-05 0.0521 0.3120 0.2492 0.1582 0.1131 0.7716

2.50E-05 0.0556 0.3109 0.2950 0.1299 0.1253 0.7914

5.00E-05 0.0491 0.3064 0.2862 0.1517 0.1209 0.7934

37

Fig re Dependenc of e MSE hen changing he amo n of hi e noi e for fi ed pink noi e ampli de E

38

A a a f hi ec i , he be e f i g c bi a i f hi e i e a d i i e i he e e i e i a ia ce 5.00E-05 a i de 1.00E-05 (Tab e 3.9)

Fig e 3.15 h he g d h a d edic i f he be i i e f d. A h gh f e e A, NN i ha acc a e edic i , he agged f b h X a d Y i e ced c a ed ba e i e. F e e B, e e h gh he ca e f g i g i he i e di ec i g d h i e i , e ica i i be e ha ba e i e. F e e D, a e a bef e, he e i a ig ifica i e e f ab he 6000 h a e he e d.

Table The be re l for differen noi e combina ion

Test MSE A

[𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Standard Deviation of

test MSEs [𝑚 ]

Sum of test MSEs

[𝑚 ] var5e-6 + ampl1e-5 0.0491 0.2807 0.2586 0.1425 0.1078 0.7309 var7.5e-6 + ampl7.5e-5 0.0549 0.2728 0.2430 0.1488 0.0987 0.7194 var5e-5 + ampl1e-5 0.0612 0.2692 0.2441 0.1379 0.0965 0.7124 var1e-5 + ampl5e-5 0.0521 0.3120 0.2492 0.1582 0.1131 0.7716

39

Fig re Gro nd r h predic ion for he be mi noi e ariance E ampli de E

40

3.4.4 Concl ion

Tab e 4.1 h he i e e f he h ee i e e a i e he ba e i e i e f a da d de ia i a d f f e MSE , e e ed a a e ce age. I h ha , he i i e d e e f e , e ecia he a da d de ia i i a ega i e a e, i dica i g ha he edic i acc ac f he e a e a e ide f diffe e da a e . Whi e i e a d i ed i e ade e i e e he e a e . E ecia he i ed i e, b h f e MSE a d a da d de ia i a e he i ed a g he h ee e f i e. Thi i dica e ha he i ed i e i e b h he edic i acc ac a d ge e a i a i abi i f he

e a e .

I he i , he e f a ce f he e a e i i ed b a i g da a a g e a i ech i e. B addi g i e he ai i g e , he e a e ' abi i a he

ea ed f c i e a e i i ed

Table Compari on of he be re l for hi e noi e ariance E pink noi e ampli de E and mi ed noi e ariance E ampli de E

Test MSE A [𝑚 ]

Test MSE B [𝑚 ]

Test MSE C [𝑚 ]

Test MSE D [𝑚 ]

Improvement of Standard Deviation of test

MSEs [𝑚 ]

Improvement of Sum of test

MSEs [𝑚 ]

Baseline 0.0526 0.3098 0.2486 0.1798 0% 0%

White Noise 0.0509 0.2917 0.2412 0.1362 2.36% 8.95%

Pink Noise 0.0502 0.2997 0.2635 0.1358 -4.52% 5.25%

Mixed Noise 0.0612 0.2692 0.2441 0.1379 12.55% 9.91%

41

Bibliog a h

[1] O a a Bi Ta i , Mihai Te d La a e c , a d L cia La ag , Ne a Ne f I d H a Ac i i Rec c i , IEEE Se J a , . 20, . 22, N e be 15, 2020.

[2] W d P a i Agei g 2019 High igh , U i ed Na i , Ne Y , NY , USA, 2019.

[3] Zi e a , T.G.; S i h, J.R.; Pa adi , J.A.; A , D.; Ge he fe d, N. A i g e ec ic fie d e i g h a -c e i e face . I P ceedi g f he SIGCHI C fe e ce H a Fac i C i g S e , De e , CO, USA, 7 11 Ma 1995; ACM P e /Addi -We e P b i hi g C .: Ne Y , NY , USA, 1995; . 280 287.

[4] A i e a Ra e a i A h a eh, Mihai Te d La a e c , O a a Bi Ta i a d L cia La ag , A Tag e I d L ca i a i S e Ba ed Ca aci i e Se i g Tech g , Se ,

16(9):1448, 2016.

[5] O a a Bi Ta i , Mihai Te d La a e c , Ja ed I ba , a d L cia La ag , Pe f a ce f Machi e Lea i g C a ifie f I d Pe L ca i a i Wi h Ca aci i e Se , IEEE Acce , . 5, . 12913 12926, 2017.

[6] A d e Ng, "Ne a e a d dee ea i g," h :// .c e a. g/ ea / e a - e -dee - ea i g.

[7] Wha i a ac i a i f c i ? Wha a e c ed ac i a i f c i ?

h :// .da a cie ce e a a i .c /b g/a ic e / ha -i -a -ac i a i -f c i - ha -a e- c - ed-ac i a i -f c i /

[8] S i Saha, "A C ehe i e G ide C i a Ne a Ne ,"

h :// a d da a cie ce.c /a-c ehe i e-g ide- -c i a - e a - e - he-e i5- a -3bd2b1164a53

[9] Abid A i A a , "Rec e Ne a Ne T ia (RNN),"

h :// .da aca .c / ia / ia -f - ec e - e a - e

42

[10] S e Da aScie ce Tea , "Rec e Ne a Ne (RNN) - L g Sh Te Me

(LSTM)," h :// . e da a cie ce.c /b g / ec e - e a - e - - g- h - e -

e -

[11] Seba ia R de , "A e ie f g adie de ce i i a i a g i h ,"

h :// de .i / i i i g-g adie -de ce /i de .h #f 5

[12] Diede i P. Ki g a a d Ji Lei Ba, "Ada : A Me h d f S cha ic O i i a i ,"

a Xi :1412.6980 9 [c .LG],30 Ja , 2017.

[13] Wi i edia, "Addi i e hi e Ga ia i e,"

h ://e . i i edia. g/ i i/Addi i e_ hi e_Ga ia _ i e

[14] Wi i edia, "Pi i e," h ://e . i i edia. g/ i i/Pi _ i e