A convolutional neural network approach to detect congestive heart failure

Contents lists available atScienceDirect

Biomedical

Signal

Processing

and

Control

j o u r n a l h o m e p a g e :w w w . e l s e v i e r . c o m / l o c a t e / b s p c

A

convolutional

neural

network

approach

to

detect

congestive

heart

failure

Mihaela

Porumb

_,

_Ernesto

_Iadanza

_,

_Sebastiano

_Massaro

_,

_Leandro

_Pecchia

a_{UniversityofWarwick,SchoolofEngineering,CoventryCV47AL,UK} b_{UniversityofFlorence,v.S.Marta,3,Florence,Italy}

c_{TheOrganizationalNeuroscienceLaboratory,LondonWC1N3AX,UK} d_{UniversityofSurrey,GuildfordGU27XH,UK}

a

r

t

i

c

l

e

i

n

f

o

Articlehistory: Received28March2019

Receivedinrevisedform23May2019 Accepted7June2019

Keywords:

Convolutionalneuralnetworks Congestiveheartfailure Machinelearning

a

b

s

t

r

a

c

t

CongestiveHeartFailure(CHF)isaseverepathophysiologicalconditionassociatedwithhigh preva-lence,highmortalityrates,andsustainedhealthcarecosts,thereforedemandingefficientmethodsfor itsdetection.Despiterecentresearchhasprovidedmethodsfocusedonadvancedsignalprocessingand machinelearning,thepotentialofapplyingConvolutionalNeuralNetwork(CNN)approachestothe automaticdetectionofCHFhasbeenlargelyoverlookedthusfar.Thisstudyaddressesthisimportant gapbypresentingaCNNmodelthataccuratelyidentifiesCHFonthebasisofonerawelectrocardiogram (ECG)heartbeatonly,alsojuxtaposingexistingmethodstypicallygroundedonHeartRateVariability. WetrainedandtestedthemodelonpubliclyavailableECGdatasets,comprisingatotalof490,505 heart-beats,toachieve100%CHFdetectionaccuracy.Importantly,themodelalsoidentifiesthoseheartbeat sequencesandECG’smorphologicalcharacteristicswhichareclass-discriminativeandthusprominent forCHFdetection.Overall,ourcontributionsubstantiallyadvancesthecurrentmethodologyfor detect-ingCHFandcaterstoclinicalpractitioners’needsbyprovidinganaccurateandfullytransparenttoolto supportdecisionsconcerningCHFdetection.

©2019ElsevierLtd.Allrightsreserved.

1. Introduction

CongestiveHeartFailure(CHF)isapathophysiologicalcondition responsibleforthefailureoftheheartinpumpingbloodinthebody [1]whichhasencounteredwidespreadresearchandsocietal atten-tion[2].AccordingtotheEuropeanSocietyofCardiology,around 26millionpeopleworldwideareaffectedbyaformofheart fail-ure[3].CHFisastronglydegenerativecondition,anditsprevalence increasesquicklywithage[4,5].Themortalityrateisclosely associ-atedwiththedegreeofseverity,reachingpeaksof40%inthemost seriousevents[e.g.,NewYorkHeartAssociation(NYHA)classes III-IV][6].CHFisalsooneoftheforemostreasonsforhospitalizationin theelderly,anditischaracterizedbyaresilientrelapserate,with halfoftheoutpatientsreadmittedwithinafewmonthsfrom hos-pitaldischarge[7].Moreover,justamongthemostindustrialized countries,thehealthcareexpenditureforCHFconsumes2–3%of

∗ Correspondingauthor.

E-mailaddresses:Ernesto.Iadanza@unifi.it(E.Iadanza),

sebastiano.massaro@theonelab.org(S.Massaro),L.Pecchia@warwick.ac.uk (L.Pecchia).

thehealthcarebudgets,withthecostofhospitalizationbeingthe greatestproportionofthespending[8–10].Thus,withaworldwide agingpopulationandsustainedpressuresonhealthcaresystems andresources,thereisthecompellingdemand—amongpatients, healthcareproviders,policymakers,andthesocietyasawhole—to addressthis scenariobyidentifyinghighlyaccuratemethods to improvedetectionofheartfailures[11]andinturnenableearly andmoreefficientdiagnoses.

Recently,researchhasmadesignificantprogressintheseareas. Inparticular,giventhequantityandcomplexityofdatainvolved, machinelearning techniquesand classsifiers(e.g.,SVM, MLP, k-NN,CART,RandomForest)[12–18]havebeensuccessfullyapplied toanalyze,detect,andclassifyheartfailures,aswediscussedand showedinpreviousworks[12,13,19–21].Theseapproachesableto distinguishbetweenheartfailureandhealthysubjectsaremostly basedonHeartRateVariability(HRV)—thevariationovertimeof theperiodbetweenconsecutiveheartbeatsextractedfrom elec-trocardiographic(ECG)signals[22]—,showingthatdepressedHRV patternsrepresentaccuratemarkersfordetectingthecondition. However,buildingaccurateHRV-basedmodelsistime-consuming andpronetoerrorsteps,duetothepreprocessingandtheiterative processofmanuallyselectingappropriatefeatures.Moreover,the https://doi.org/10.1016/j.bspc.2019.101597

2 M.Porumb,E.Iadanza,S.Massaroetal./BiomedicalSignalProcessingandControl55(2020)101597 bestperformingHRV-basedmodelsgenerallyrequireeither

long-termsignals(i.e.,24h)oratleastthecombinationofshort-term HRVwithnon-standardlong-termHRVfeatures,asshownin[23]. Totackletheseissues,wepresenta novelframeworkofCHF detectionthatdoesnotrelyonHRVfeatures,ratheritusesraw ECGsignals only.Thismethodis basedona 1-D Convolutional NeuralNetwork(CNN).CNNsarehierarchicalneuralnetworksthat mimicthehumanvisualsystemandhaveproventobeeffective inrecognizingpatternsandstructuresofinputdatainimage clas-sification,localization,anddetectiontasks,amongothers[24–26]. Moreover,theyhavebeenextensivelyusedfortimeseriesanalysis inclassificationtasks.Somesuccessfulexamplesinclude,among others:generaltime seriesclassification [27–29], speech recog-nitiontasks[30],arrhythmiadetection[31,32],andmultivariate diagnosticmeasurementsmodeling[33,34].

Inspiredbythis growingbody ofresearch,we aimtodetect CHFthrougha1-DCNNapproachontheECGsignals.Addingto thesignificantbenefit of buildingupon raw physiological data, thismethodenablesvisualizationoftheinputtimeseries subse-quencesthatareclassdiscriminative(i.e.,CHFvs.healthysubjects). ThisfeatureconsiderablyimprovestheinterpretabilityoftheCNN modelandrepresentsacrucialaspecttoensurethe‘transparency’ ofthemethod[35,36].Suchatransparencyisfundamentaltohelp researchersexplainhowconclusionsarereached,andtoaid profes-sionalsinbetterunderstandingcorrelationsofpathophysiological behaviorsandpropertiesrevealedbythemodel.

Asweshallexplain,weusedaformofclassactivationmapping (Grad-CAM)[37]thathighlightstheclassdiscriminativeregionsin theinputdata.Inotherwords,Grad-CAMusesthemodel’slast acti-vationmapstooriginateaheatmapthatcanbeoverlappedwith theinput,thusshowingwhatregionsintheinputdatacontribute mosttoCHFdetection.Overall,theproposedframeworkputs for-wardseveraldevelopmentsforCHFdetection,suchasrefraining fromusingheftypreprocessingandfeaturesselectionstepsof HRV-basedmodels,aswellasenablingvisualizationofthesubsequences intheinputtimeserieswhichareusedtoreachcertain clinically-relevantconclusions.

2. Methods

2.1. Data

Weperformedaretrospectiveanalysisontwopublicly avail-abledatasets.Thedataforthenormalsubjects(i.e.,controlgroup) wereretrievedfromtheMIT-BIHNormalSinusRhythmDatabase [38] includedin PhysioNet [39]. Thisdatasetincludes 18 long-termECG recordingsofnormal healthynot-arrhythmicsubjects (Females=13;agerange:20to50).ThedatafortheCHFgroup wereretrievedfromtheBIDMCCongestiveHeartFailureDatabase [40] from PhysioNet [39]. Thisdataset includes long-term ECG recordingsof15subjectswithsevereCHF(i.e.,NYHAclasses III-IV)(Females=4;agerange:22to63).Altogether,thepoolofdata availableforthisstudyconsistsofabout20hofECGrecordingsper eachsubject;itcontainstwo-channelECGsignalssampledat250 samplespersecondfortheBIDMCdataset,and128samplesper secondfortheMIT-BIHdataset.

Thetwodatasetsusedinthisstudywerepublishedbythesame laboratory,theBeth IsraelDeaconess MedicalCenter,and were digitizedusingthesameproceduresaccordingtothesignal specifi-cationlineintheheaderfile.Thesedatasetshavebeenwidelyused inseveralstudiesconcerningCHFdetection,asexplainedin[11]. Indeed,itisanacknowledgedpracticetobuildandvalidateCHF classificationmodelsonthesesourcestoensureopportunitiesfor reproducibility.

Table1

Totalnumberofextractedbeatsfrombothdatasets:MIT-BIHandBIDMC.

Controlgroupbeats CHFgroupbeats

275,974 214,531

2.2. ECGpreprocessing

The ECG recordings from the BIDMC dataset were

down-sampledin order tomatch thesampling frequency of theECG

signalsfromtheMIT-BIHdataset(i.e.,128Hz).Recordsfromboth

databaseswerealreadymadeavailablewithcomputedbeat

anno-tations,whichweusedtoisolateandextractindividualheartbeats.

Weconsideredawindowof235msbeforetheannotatedRpeak

(equivalentto30samples=128samples/s*0.235s),andawindow

of390msaftertheRpeak (equivalentto50samples). Onlythe

beatsthatwereannotatedasnormal(N)wereretainedforfurther

analysis.

Eachheartbeatwasz-normalizedtoobtainanapproximately

zeromeanvalue(i.e.,amplitude)andastandarddeviationclose

to1.Becausethenumberofheartbeatsextractedforeachsubject

wasverylarge(∼70,000beats),andbecausetemporallyclosebeats

holdsimilarpatterns,werandomlyselectedonesinglebeatevery

5softheECGsignal.Thetotalnumberofbeatsusedinthisworkis

showninTable1.

2.3. Beatsclassification

Eachheartbeatwaslabeledwithabinaryvalueof1or0 (here-after“class”nottobeconfusedwiththeNYHAclasses)according tothestatusofthesubject:healthyorsufferingfromCHF, respec-tively.Ascustomaryinmachinelearning,thedatasetwasrandomly splitintothreesmallersubsetsfortraining,validation,andtesting (correspondingto50%,25%,and25%ofthetotaldata,respectively). Becauseoneperson’sheartbeatsarehighlyintercorrelated,these wereincludedonlyinoneofthesubsets(i.e.,training,testing,or validationset)atatime.Inordertoreducethevarianceofthe classi-ficationresultswerepeatedtherandomsplittingprocess10times. Inthisway,theCNNwastrainedandevaluatedover10runs;the reportedresultsrepresenttheaverageperformanceontheseruns, unlessotherwisestated.Performing10differentrandomsplitsof thedataset,andthenaveragingtheresultsofthemodels,isa simi-larproceduretothatofcross-validation(i.e.,averagingthevariance andbiasoftheestimator).Thesoledifferencebetweenthetwo methodsisthattheadditionalvalidationdatasetisusedforearly stoppingofthetrainingprocessandtuningthehyperparameters oftheCNN.

Two different performance evaluation strategies were employed: a) individual beat classification, and b) classifica-tionon5minofECGexcerptsthroughwhatwecalleda“majority votingscheme.”Specifically,bydenotingthenumberofindividual heartbeatsclassifiedasnormalwithN1,andthenumberof

heart-beatsthatwereclassifiedasCHFwithN0 within5-minutesECG

excerpts,thefinal classassociated witha given5-minutes ECG excerptwascalculatedasmax (N0,N1) .

2.4. Performancemeasures

Wecomputedthefollowingmeasuresforbinaryclassification toassesstheperformanceoftheCNNclassifier:Accuracy(Acc), Pre-cision(P),Sensitivity(Sens),Specificity(Spec)andAreaunderthe Curve(AUC)[23].Theseperformanceindicatorscorrespondtothe goldenstandardintheliterature[17,41],therebyenablingongoing models’comparison.

Fig.1.The1-DCNNarchitectureforbeatclassificationandvisualizationoftheclassdiscriminativesequencesintheinputheartbeats.

2.5. 1-DCNNclassifier

ThecompletearchitectureofourmodelisshowninFig.1. A1-DCNNclassifierwasusedforfeatureextraction,andthe MLPmodulefortheclassificationoftherawECGheartbeats.For thebeatclassificationproblem,theCNNactsasafeatureextractor block.Thefinalactivationsobtainedfromthelastlayerof convolu-tionareusedasinputsinaMultilayerPerceptron(MLP)network. Thefinaloutputisobtainedfromasoftmaxlayer[42].Thebasic convolutionallayerisfollowedbyabatchnormalizationlayer[43] andaReLUactivationfunction[44].Theconvolutionoperationis performedineachlayerby201-Dfilters(i.e.,kernels)withsizes {10,15,and20_}usingastrideof1,hyperparameters,asinprevious studies[28,32].Thus,theconvolutionalblockcanbeformalizedas asetofthree operations—convolution,batchnormalization,and non-linearactivation—definedbytheequationsbelow:

y=Wx+b (1)

bn=BN (y) (2)

act=ReLU(bn) (3)

whereistheconvolutionoperator,Waretheconvolutionallayer parameters(i.e.,thelearnablefilters),xrepresentstheinputtime series,bthebias,BNthebatchnormalizationfunction,andReLU theactivationfunction.TheresultingCNNisbuiltbystackingthree convolutionalblocks,eachcomprising20filtersofsizes{10,15, and20}.

ThefinalactivationsareflattenedandpassedtoanMLPwith ahiddenlayerof30neuronsthatarethenfullyconnectedtothe outputneurons.Thefinallabelisobtainedfromthesoftmax func-tion.Poolingoperationisexcludedfromtheconvolutionalblocks accordingtorecentresearchshowingthatitdoesnotaffectthe classifierperformanceandmightaffectoverfitting(see,e.g.,ResNet [45]).

2.6. Visualizationofclassspecificsequences

We employed a Grad-CAM technique to visualize the sub-sequences in theinput ECG beats discern a certain class [37]. Grad-CAMrepresentsageneralizationofCAM[46]applicableto abroadrangeofCNNs,includingthosecontainingfully-connected

layers.InGrad-CAMtheweightsusedinthelinearcombinationof theforwardactivationmapsarecomputedastheglobalaverage poolingofthegradientsofthescoreforclasscwithrespecttothe lastfeaturemaps.Specifically,themethodimpliesthecomputation ofthegradientofthescoreforclassc(yc_{)withrespecttofeature}

mapsA(i.e.,inourcasethefeaturemapsofthelastconvlayer)that areglobal-average-pooledtoobtaintheweights˛c

k.Therefore,the

weights˛c

karecomputedasfollows:

˛c_k= _m1





∂

∂

kcapturetheimportanceoffeaturemapkforatargetclass

c.TheGrad-CAMheatmapforaclassc,isobtainedasaweighted combinationoffeaturemaps,followedbyReLU[37].

GradCAMmapc=ReLU



k˛ c kA

k



AsshowninFig.1,weusedthistooltoderivetheclassactivation mapforclassc,whichindicatestheimportanceoftheactivationat atemporallocationxi;thisleadstotheclassificationoftheinput

timeseriesasaclassc. 2.7. Experimentalsettings

WeusedAdamOptimizer[47]astheoptimizationfunctionwith aninitiallearningrateof10−3.Thebatchsizewassetto200.The maximumnumberoftrainingstepswassetto3000which repre-sentsapproximately2.5epochswhenconsideringthebatchsize aboveandtheapproximatetrainingdatasetsizeof250,000beats. We usedthevalidationsettomonitorthetraining processand accountfortheoccurrenceofoverfitting.Insuchacase,anearly stoppingcriterionwasimplemented:Thiscriteriondoesstopthe trainingwhentheAUConthevalidationdatasetdoesnotimprove over ak=30optimizationsteps.Weusedthecategorical cross-entropyaslossfunction.Theweightswereinitialized usingthe Xavierinitializer[48];thebiaseswereinitializedtozero.The net-workwasimplemented inPythonusingTensorFlow framework [49].

Due to the high number of hyper-parameters in the CNN, weassessedacombinationofarchitectureandhyper-parameters

4 M.Porumb,E.Iadanza,S.Massaroetal./BiomedicalSignalProcessingandControl55(2020)101597

Table2

IndividualHeartbeatsClassification.

Training Validation Test

Accuracy 0.999±1.1*10−16 _{0.982±2.0*10}−3 _{0.978±2.0*10}−3 AUC 0.999±1.1*10−16 _{0.990±1.1*10}−16 _{0.980±1.1*10}−16 Sensitivity 0.999±1.1*10−16 _{0.973±2.0*10}−3 _{0.963±4.0*10}−3 Specificity 0.999±1.1*10−16 _{0.978±2.0*10}−3 _{0.986±7.0*10}−4 Precision 0.999±1.1*10−16 _{0.970±3.0*10}−3 _{0.985±7.0*10}−4 Table3

MeanConfusionMatrix.

TrueCHF Truenormal TotalACC

ClassifiedCHF 67181 694

Classifiednormal 2071 61512

Classifiedcorrectlyfromeachclass ∼97% ∼98.8% ∼97.9%

Error ∼3% ∼1.1% ∼2.1%

throughaniterativeprocess,usinggrid-searchandmanualtuning.

Asconcernsthearchitecturestructure,wesearchedoverthe

num-berofconvolutionallayers(3–9),differentfiltersizes(from5to

20),andthenumberoffiltersineachconvolutionallayer(5to

max-imum30).Thelearningratewasinsteadmanuallytunedtoachieve

fasterconvergence;theconsideredvalueswere{10−1_to10−5_}.The

resultswepresentbelowwereobtainedwiththeCNNarchitecture

thatachievedthehighestperformanceonthevalidationdataset

withtheminimumnumberofparameters.

3. Results

3.1. Individualbeatsclassification

Table2reportstheperformanceforindividualbeatclassification astheaverageofthe10differentrunsforthe10randomsplitsof theinputdatafortraining,validation,andtesting.

Table3showsaconfusionmatrixforthemeanofthe10runs, withoutconsideringthevotingstrategy,correspondingtothe test-ingdataset.Thesefindingsshowthatonaveragethenumberof falsepositivesisaround1%ofthemeannumberofnormal heart-beats,whilethefalsenegativesarearound3%ofthetotalnumber ofheartbeatsinCHFclass.

Acloseranalysisofoneoftherunsrevealedthatoutofthetotal of854misclassifiedCHFbeats,614belongedtothesamesubject (id:chf01).Thesameappliedtothecontrolgroup,where478beats thatweremisclassifiedbelongedtothesamesubject(id:16272). Nonetheless,thenumberofmisclassifiedbeatsforthesetwo sub-jectswasnegligible,representingaround2%–4%ofthetotalnumber ofbeatsthatwereextractedforthetwosubjects.Inotherwords, subjects16,272andchf01couldstillbecorrectlycategorizedas healthyandCHFsubjectsrespectively,becausemorethan95%of theirheartbeatswereappropriatelyclassified.The5consecutive heartbeatswereacceptedtobeverysimilar,thusaveragingthem shouldnotresultinsignificantchangesintheheartbeat

morphol-ogy.We confirmedthisoccurrencebyperforminganadditional simulation inwhich weaveragedtheheartbeatsin 5sinterval. Asexpected,weobservednochangeintheperformanceofthe classifier.

3.2. Classificationusingmajorityvotinginaspecifictimeframe Weemployed a“majority votingscheme”for theheartbeats withina5-minutesECG timeframetoenabledirect comparison withpreviousstudiesusingshort-termHRVfeaturescomputedon 5-minutesECGsegments(seeSection2.3fordetails).Insum,the classassociatedwitha5-minutesECGsegmentwasthemajority classoftheindividualheartbeatsin thatsegment.Usingsucha votingstrategy,wecouldevaluatetheaccuracyofourmethodin discriminatingbetweenhealthyandCHFpatientswhenusing 5-minutesECGrecordings.Alltheperformancemeasuresconsidered (i.e.,ACC,Sens,Spec,andAUC)improvedby99%when employ-ingthe5-minutesvotingincomparisontotheindividualheartbeat predictions.Moreover,themeannumber ofthemisclassified 5-minutesECGexcerptsusingthevotingschemeonthe10runswas 23outofatotalof2,227,whichcorrespondstoabout0.1%.

Theseresultssuggestthat,differentlyfromexistingevidence [23], a highly accurate model for CHF diagnosis can be built by using relatively short (∼5-minutes) raw ECG recordings.To furthercorroboratethisargument,weinvestigatedsubjectsand specifictimeframeswhereanymisclassified5-minutesECG seg-mentoccurred,asshown inTable4.Asamatterofillustration, herewepresentarandomlyselectedrun.Wecanobservethatout ofthetotalnumberof16subjectsincludedinthetestandvalidation datasets,5ofthempresentmisclassified5-minutesECGsegments. Fig.2presentsacloserinspectionoftheseresults.Itillustrates theheartbeatclassificationononeofthe10modelsweusedonthe training,validation,andtestingdatasetsincludingallthesubjectsof thisstudy.Onceagain,wecanreadilyappreciatethattheextracted heartbeatswerecorrectlyclassified(i.e.,themisclassified heart-beatsfor somesubjectsareconsiderablylessthan thecorrectly classifiedones).Moreover,themisclassifiedbeatsshowa consec-utivetrend,whichismostlikelyindicatinganoisyECGsegmentin therawrecording.Itmightalsobepossiblethatthesespecific heart-beatswereintrinsicallynot-discriminative,anissuethatcouldbe tackledbyextendingthetrainingdataset.Wefurthersupportthese propositionsbyillustratingthemisclassifiedECGexcerptsinFig.3, showingthatourmodel failedtoclassifyeithernoisysegments (e.g.,subjects16,272,19,830)orECGexcerptsaffectedby move-mentartifacts,asalsoacknowledgedbyindependentcardiological assessments.Thiscircumstancecaneasilybeconfirmedby inspect-ingthedatawiththeonlineviewerLightWAVE[50]fromPhysionet. 3.3. Classspecificsequences

Movingtothevisualizationfeatureofthemodel,Fig.4 repre-sentsasummaryofkeyregionsintheinputbeats,obtainedthrough

Table4

TestSubjectsWithMisclassified5-MinutesECGSegments.

SubjectID ECGsegmentstarttime→endtime SubjectID ECGsegmentstarttime→endtime

16795 13:10→13:15 17453 13:10→13:15 16795 13:30→13:40 16795 13:55→14:20 19093 4:35→4:40 16795 14:30→14:40 19093 21:55→22:25 16795 14:55→15:00 16795 15:30→15:40 19830 10:40→10:45 16795 15:50→15:55 16795 16:05→16:15 16272 16:05→16:10 16795 16:50→16:55 16272 16:15→17:05 16795 14:20→17:35 16272 17:30→17:35

Fig.2. Heartbeatsclassificationforallthesubjectsincludedinthetraining,validationandtestdatasets,evaluatedononeofthe10runs.Panelsb)andd)presentthe predictionsforthevotingina5-minuteswindowoftheindividualheartbeats.TheredandgreenbarsrepresenttheextractedheartbeatsinchronologicalorderforCHF patientsandhealthysubjects,respectively.Thegreygapsinthebarsaremissingdata.

Fig.3. ExampleofmisclassifiedECGsegmentsevaluatedonthevalidationandtestingdatasets.

Grad-CAM,thatareclassdiscriminative.ThetwoECGbeatsshown

inFig.4representtheaverageofalltheheartbeatsfromthe test-ingsetcorrespondingtothenormal(i.e.,green)andCHF(i.e.,red) classeswiththerespectiveerrorbands.Thehistogramsfeaturethe datapointsintheinputECGbeatsabove0.8inthenormalizedheat

mapsobtainedthroughGrad-CAM.Thesamplepointsthatwere foundtobesignificant(i.e.,wherethehistogrampointsreacheda thresholdof0.25)arepresentedinFig.4.Inotherwords,these are thepointsthat contributethemostin identifyingacertain class.

6 M.Porumb,E.Iadanza,S.Massaroetal./BiomedicalSignalProcessingandControl55(2020)101597

Fig.4.HistogramsobtainedfromtheGRAD-CAMheatmap,indicatingthemostimportantsamplepointsintheinputbeatsforacertainclass.

4. Discussion

WepresentedanovelandinnovativeframeworkforCHF detec-tionbasedonCNN.WhileCHFdetectionusingmachinelearning onECGhastraditionallyreliedontheextractionofHRVfeatures tobuildclassifiersabletodiscriminatebetweenhealthyandCHF subjects, in this study we substantially advanced this body of knowledgebyusingasinputdatarawECGheartbeats.

ThedevelopmentofsuchamodelallowstobypassHRV fea-turesextractionandselectionsteps,yet,andimportantly,without compromisingtheclassificationperformanceofthemodel.Indeed, asamatterofcomparison,themodelproposedbyIsleretal.[14] basedonshort-termHRVfeaturesextractedfromarandomly cho-sen5-minutesECGexcerptoutofthetotal24hECGrecordings, reportsaclassificationperformanceof∼96%sensitivityand speci-ficity.Thus,notonlyweimprovedtheseclassificationresults,but alsoweshowedthat99%ofthe5-minutesECGexcerptscouldbe usedforprovidingacorrectCHFdiagnosis.Wearguethatthese benefitsareduetotheabilityofCNNs’toautomaticallyextractand learnpatternsintheinputdata,ratherthanrelyingonspecialized featuresthatmightnotfullycapturethetruecharacteristicsofthe underlyingphysiologicalsignals.

Inthisthisstudywepurposivelychoseashallowarchitecture (i.e.,with3convolutionallayers),yetusinglargerfiltersizes. Exper-imentsperformed withdifferent CNN architectures reveal that similarclassification resultscanalsobeobtainedwithadeeper architectureand smallerfiltersizes.In ourcase,thenumber of neuronsintheMLPlayerdidhavegreatinfluenceonthemodel’s performanceand we observeda much faster convergencewith batchnormalizationthanwithout.

Anotherrelevant note concerns the difference betweenour workandthatbyAcharyaetal.[51]whoemployedaCNN-based modelforCHFdetection.Thecurrentstudyputsforward impor-tant benefits.First, as regardsthe evaluation methodology,the authorsin [51] used therecordings correspondingto a certain subjectforbothtrainingandtestingthemodel;here,instead,we consideredtheextractedheartbeatsonlyineitherthetrainingor testingdatasets for arun. Thatis,we dividedthesubjectsinto trainingandtestingsubsets,ratherthanusingtheir correspond-ingECG heartbeats.Moreover,differently fromthearchitecture proposedin[51],ourmodelissimplerbecauseitcontains3 con-volutionallayersonlyand nopoolingoperations,thusreducing

thecomputational complexity to thebenefit of possible future mobileapplications.TheCNNinputsaredifferenttoo:Inthe cur-rentstudy,we extractedindividual heartbeats,whereas in [51] ECGexcerptsof2swereusedasinputs.Thisrepresentsacritical strategythatourmodeladvancesbecauseitallowsusto investi-gatetheheartbeatmorphologyinconnectiontotheCHFdiagnosis through theGrad-CAM method(see Fig. 4).To thebest of our knowledge,thisisalsotheopeningevidenceinwhich discrimi-nativesegmentsintheinputheartbeatarerevealedfortheCHF diagnosis.

Finally, mean differences in the heartbeat morphology are clearlyshowninFig.4andthehistogramshighlighttheregionsin theinputheartbeatsthatweremostlyusedbytheCNNinmaking thepredictions.Thus,ourmodelallowstovisualizethosefeatures thatare automaticallylearnedduring theclassificationprocess. AccordingtoarecentstudybyTaylorandHobbs[52],heart fail-ureisassociatedwithabnormalECGin89%ofthecases.Moreover, Jamesetal.[53]showedthatheartfailurepatientshavesignificant prolongedQRSduration,prolongedQTduration,prolongedQTcand morerightwardTwaveaxis.Also,deLunaetal.[54]reportedlong QTintervalandvisiblevariationsoftheshape,amplitudeorpolarity oftheTwaveonanalternate-beatbasisasECGchangesassociated withheartfailure.InFig.4,wepresentedtwoheartbeats,obtained asaverageonallnormalandCHFheartbeatsincludedinthe avail-abledatasets.Wecanappreciatethatsomeofthemorphological changesreportedintheliteraturecanbeeasilyidentifiedinFig.4. Theseinclude,amongothers,flattened,prolongedTwave,visible changesintheamplitudeoftheTwave.Thehighlightedpartsin thenormalandCHFheartbeatsrepresenttheECGheartbeat subse-quenceusedbytheCNNnetworktogenerateaprediction.Whilst notenteringintoclinicalinterpretations,wecananywayobserve thattheidentifiedimportantpartsintheECGaremainlylocated nearthefiducialpoints(Q,RandTwaves).

AsshowninTable5,theclassificationresultssuggestthatthis methodishighlycompetitivewith,andinsomerespectsuperior to,previousmodelsusingHRVfeatures.Itisworthnoticingthat, whiletheclassificationprocesspresentedhereisperformedatthe heartbeatlevel,theobtainedresultscanbeeasilycomparedwith previousstudiesbyperformingaper-subjectclassification,using thepresentedvotingscheme,eitherfor5-minutesECGexcerptsor ontheentireECGrecording(i.e.,byquantifyingthemajorityclass predictedforalltheheartbeatscorrespondingtoeachsubject).

Table5

Classificationperformanceofthecurrentstudyascomparedtotheexistingliterature.

Author(s)(Year) Methods Data Results

Asyalietal.(2013)[17] Lineardiscriminantanalysis Bayesianclassifier BasedonHRVfeatures

1.NormalSinusRhythmRRIntervalDatabase 2.CongestiveHeartFailureRRIntervalDatabase 54normalsubjects 29CHFsubjects Sensitivity:0.82 Specificity:0.98 Accuracy:0.93 Isleretal.(2003)[14] k-NN BasedonHRVfeatures

1.NormalSinusRhythmRRIntervalDatabase 2.CongestiveHeartFailureRRIntervalDatabase 54normalsubjects

29CHFsubjects

Sensitivity:0.96 Specificity:0.96 Accuracy:0.96 Melilloetal.(2014)[12] CART

BasedonHRVfeatures

1.NormalSinusRhythmRRIntervalDatabase 2.MIT-BIHNormalSinusRhythmDatabase 3.CongestiveHeartFailureRRIntervalDatabase 4.BIDMBCongestiveHeartFailure

72normalsubjects 44CHFsubjects

Sensitivity:0.9 Specificity:1.0 Accuracy:0.96

Chenetal.(2015)[18] Non-equilibriumdecision-tree Nasedonsupportvector machineclassifier

Basedondynamicsof5-minute short-termHRVmeasures

1.NormalSinusRhythmRRIntervalDatabase 2.MIT-BIHNormalSinusRhythmDatabase 3.CongestiveHeartFailureRRIntervalDatabase 4.BIDMBCongestiveHeartFailure

72normalsubjects 44CHFsubjects

Accuracy:0.96

Liuetal.(2016)[15] SVNandk-NN

Non-standardHRVfeatures

1.NormalSinusRhythmRRIntervalDatabase 2.CongestiveHeartFailureRRIntervalDatabase 30normalsubjects

17CHFsubjects

Sensitivity:1.0 Specificity:1.0 Accuracy:1.0 Acharyaetal [51] Raw,2secondsECG

CNN

1.MIT-BIHNormalSinusRhythmDatabase 2.BIDMBCongestiveHeartFailure 3.Fantasia

Sensitivity:0.98 Specificity:0.99 Accuracy:0.98

Thisstudy RawECGheartbeats

CNNbasedmethod

1.MIT-BIHNormalSinusRhythmDatabase

2.BIDMBCongestiveHeartFailure

18normalsubjects

15CHFsubjects

Sensitivity:1.0

Specificity:1.0

Accuracy:1.0

Inourmodel,theerrorinthesubjects’classificationisnil:Each subjectiscorrectlyclassifiedasbeingeitherhealthyoraffectedby CHF.Previously,Liuetal.[15]presentedwhattothebestofour knowledgeistheonlyothersystemachievingasimilaraccuracy inCHFdetection.Yet,ourworkfocusesonapurposivelyselected sampleofsubjectsratherthanontheentirepoolavailableinthe originaldatasets.Themisclassifiedheartbeatsaretrivialin com-parisontothecorrectlyclassifiedones,indicating100%accuracy fortheclassificationatthesubjectlevel,asevidentinFig.2.

Addedtotheirsuperiorperformance,ourresultsbringforward theintuitionthatshortECGrecordings,ofjustabout5min,canbe sufficienttocorrectlydiagnosesevereCHF.Thisisanimportant resultbecause,withtheincreasingavailabilityofwearabledevices capturinginterimECGrecordings(e.g.,smart-watches),accurate CHFdetectionandpredictionmightbesoonperformedthrough devicespeoplecarryineverydaysituations.Inthisrespect,current approacheshavelookedatHRV,whichisconventionallyanalyzed using5-minexcerpts,ifnotlonger(e.g.,nominally24h).However, physiologicalphenomenarunningovermuch shortertimespans mightnotbefullycapturedwhenconsideringsuchlongexcerpts. Thus,byusingindividuallyextractedheartbeatstoprovidean accu-ratediagnosticprediction,theapproachproposedherefacilitates futureimplementationintoreal-timesystemstowardsthe real-ization of a Clinical Decision Support System (CDDS). In CDDS envisionedend-usersarenotonlyexperiencedcardiologists,but alsopatients,caregivers,generalpractitioners,nurses,trainees,and soforth.Moreover,oncetrained,thenetworkrespondsinavery shorttimeandcanbeembeddedintomobilephonesordeployed tocloudarchitectures,aswealreadydiscussedelsewhere[13,55]. Ourstudymustalsobeseeninlightofitslimitations,which anywayrepresentvaluablecallsforfutureresearchavenues.First, theCHFsubjectsusedinthisstudysufferfromsevereCHFonly(i.e., NYHAclassesIII-IV);thediscriminationcouldyieldlessaccurate resultsformilderCHF(i.e.,NYHAclassesI-II).Second,theuseof

acknowledgedpublicdatasetsincludingrawECGsignalswasaimed atimprovingtheopportunityforcomparisonandtransparencyof science.Yet,thischoicenecessarilyresultedinadatasetwhich,at thesubjectlevel,isonaveragesmallerthanthoseusedinother models.Futurestudieswillneedtoextendtheresultspresented hereinlargerdatasets.

Finally, we would like toencourage future research as well asclinicalpracticetoapplythisframeworktothepre-diagnostic assessment of CHF. This strategy will not only allow further model’s validation, but it will also enable moving from retro-spectivedetectiontoprospectivediagnosis,potentiallyimproving humanwell-being,reducinghealthcarecosts,andhopefullyeven savinglives.

5. Conclusions

Inthisarticleweproposedanovelandhighly-effectivemethod forCHFdetectionbasedonCNNs.Comparedtoexistentmodels theaddedvalueofourmodelisthatitusesrawECGsignals,rather thanHRVfeatures,andyieldsprominentaccuracyindetectingCHF. Indeed,weshowedthattheperformanceoftheCNNmodelholds considerablyhighscoreswhentheclassificationisperformedatthe heartbeatlevel,anditiserror-freeaspertainsthesubject classifi-cationcomponent.Moreover,thisisthefirststudyusingadvanced machinelearningapproachestorevealwhichmorphological char-acteristicsoftheECGbeatsarethemostimportanttoefficiently detectCHF.Webelievethatthisisacrucialcontributionforclinical practicebecauseclinicians,whoareultimatelyresponsibleforCHF patients,aregenerallyrefractorytotheadoptionofmethodsthat arenotfullytransparentinshowinghowcertaindecisionswere reached.Ourmethodsuccessfullyrespondstothisissue.

Finally,wearehopefulthatthemodelprovidedherewillbeused asa highly-effectiveand generalizableclassificationmethodfor othertimeseriesclassificationtasksinmedicineandbeyond(e.g.,

8 M.Porumb,E.Iadanza,S.Massaroetal./BiomedicalSignalProcessingandControl55(2020)101597 inneuro-behavioralresearch[56]).Wethuscallforfutureworks

toexpandtheresultspresentedinthisstudytowardsimproving humanwellbeing andreducing currenthealthcarefinancial and societalburdens.

Acknowledgement

WewouldliketothankDr.DindealAlinandDr.MoldovanEmil forassessingtheECGexcerptsthatresultedasmisclassifiedbyour model.

DeclarationofCompetingInterest

Theauthorsdeclarethattheyhavenoknowncompeting finan-cialinterestsorpersonalrelationshipsthatcouldhaveappearedto influencetheworkreportedinthispaper.

References

[1]J.J.McMurray,S.Stewart,Epidemiology,aetiology,andprognosisofheart failure,Heart83(May(5))(2000)596–602.

[2]D.M.Lloyd-Jones,etal.,Lifetimeriskfordevelopingcongestiveheartfailure: theFraminghamHeartStudy,Circulation106(December(24))(2002) 3068–3072.

[3]P.Ponikowski,etal.,Heartfailure:preventingdiseaseanddeathworldwide, ESCHeartFail.1(September(1))(2014)4–25.

[4]S.Stewart,K.MacIntyre,S.Capewell,J.J.V.McMurray,Heartfailureandthe agingpopulation:anincreasingburdeninthe21stcentury?Heart89 (January(1))(2003)49–53.

[5]M.W.Rich,Epidemiology,pathophysiology,andetiologyofcongestiveheart failureinolderadults,J.Am.Geriatr.Soc.45(August(8))(1997)968–974. [6]B.M.Massie,N.B.Shah,Evolvingtrendsintheepidemiologicfactorsofheart

failure:rationaleforpreventivestrategiesandcomprehensivedisease management,Am.HeartJ.133(June(6))(1997)703–712.

[7]J.M.Vinson,M.W.Rich,J.C.Sperry,A.S.Shah,T.McNamara,Earlyreadmission ofelderlypatientswithcongestiveheartfailure,J.Am.Geriatr.Soc.38 (December(12))(1990)1290–1295.

[8]J.B.O’Connell,Theeconomicburdenofheartfailure,Clin.Cardiol.23(March (3Suppl.))(2000)III6–10.

[9]A.P.Ambrosy,etal.,Theglobalhealthandeconomicburdenof hospitalizationsforheartfailure:lessonslearnedfromhospitalizedheart failureregistries,J.Am.Coll.Cardiol.63(April(12))(2014)1123–1133. [10]C.Berry,D.R.Murdoch,J.J.V.McMurray,Economicsofchronicheartfailure,

Eur.J.HeartFail.3(June(3))(2001)283–291.

[11]E.E.Tripoliti,T.G.Papadopoulos,G.S.Karanasiou,K.K.Naka,D.I.Fotiadis,Heart failure:diagnosis,severityestimationandpredictionofadverseevents throughmachinelearningtechniques,Comput.Struct.Biotechnol.J.15 (January)(2017)26–47.

[12]P.Melillo,R.Fusco,M.Sansone,M.Bracale,L.Pecchia,Discriminationpower oflong-termheartratevariabilitymeasuresforchronicheartfailure detection,Med.Biol.Eng.Comput.49(January(1))(2011)67–74. [13]L.Pecchia,P.Melillo,M.Bracale,Remotehealthmonitoringofheartfailure

withdataminingviaCARTmethodonHRVfeatures,IEEETrans.Biomed.Eng. 58(March(3))(2011)800–804.

[14]Y. ˙Is¸ler,M.Kuntalp,CombiningclassicalHRVindiceswithwaveletentropy measuresimprovestoperformanceindiagnosingcongestiveheartfailure, Comput.Biol.Med.37(October(10))(2007)1502–1510.

[15]G.Liu,L.Wang,Q.Wang,G.Zhou,Y.Wang,Q.Jiang,Anewapproachtodetect congestiveheartfailureusingshort-termheartratevariabilitymeasures, PLoSOne9(April(4))(2014),e93399.

[16]Z.Masetic,A.Subasi,Congestiveheartfailuredetectionusingrandomforest classifier,Comput.MethodsProgramsBiomed.130(July)(2016)54–64. [17]M.H.Asyali,Discriminationpoweroflong-termheartratevariability

measures,Proceedingsofthe25thAnnualInternationalConferenceofthe IEEEEngineeringinMedicineandBiologySocietyvol.1(2003)200–203. [18]W.Chen,L.Zheng,K.Li,Q.Wang,G.Liu,Q.Jiang,Anovelandeffectivemethod

forcongestiveheartfailuredetectionandquantificationusingdynamicheart ratevariabilitymeasurement,PLoSOne11(November(11))(2016). [19]G.Guidi,M.C.Pettenati,R.Miniati,E.Iadanza,RandomForestforautomatic

assessmentofheartfailureseverityinatelemonitoringscenario,Annual InternationalConferenceoftheIEEEEngineeringinMedicineandBiology Societyvol.2013(2013)3230–3233.

[20]G.Guidi,L.Pollonini,C.C.Dacso,E.Iadanza,Amulti-layermonitoringsystem forclinicalmanagementofCongestiveHeartFailure,BMCMed.Inform.Decis. Mak.15(Suppl.3)(2015)S5.

[21]G.Guidi,M.C.Pettenati,R.Miniati,E.Iadanza,Heartfailureanalysis dashboardforpatient’sremotemonitoringcombiningmultipleartificial intelligencetechnologies,AnnualInternationalConferenceoftheIEEE EngineeringinMedicineandBiologySocietyvol.2012(2012) 2210–2213.

[22]U.RajendraAcharya,K.PaulJoseph,N.Kannathal,C.M.Lim,J.S.Suri,Heart ratevariability:areview,Med.Biol.Eng.Comput.44(December(12))(2006) 1031–1051.

[23]L.Pecchia,P.Melillo,M.Sansone,M.Bracale,Discriminationpowerof short-termheartratevariabilitymeasuresforCHFassessment,IEEETrans.Inf. Technol.Biomed.15(January(1))(2011)40–46.

[24]A.Krizhevsky,I.Sutskever,G.E.Hinton,ImageNetclassificationwithdeep convolutionalneuralnetworks,USA,in:Proceedingsofthe25thInternational ConferenceonNeuralInformationProcessingSystems,vol.1,2012,pp. 1097–1105.

[25]StanfordUniversityCS231n:ConvolutionalNeuralNetworksforVisual Recognition.[Online].Available:http://cs231n.stanford.edu/.

[26]Y.LeCun,Y.Bengio,G.Hinton,Deeplearning,Nature521(May(7553))(2015) 436–444.

[27]Y.Zheng,Q.Liu,E.Chen,Y.Ge,J.L.Zhao,Exploitingmulti-channelsdeep convolutionalneuralnetworksformultivariatetimeseriesclassification, Front.Comput.Sci.China10(February(1))(2016)96–112.

[28]Z.Wang,W.Yan,T.Oates,Timeseriesclassificationfromscratchwithdeep neuralnetworks:astrongbaseline,2017InternationalJointConferenceon NeuralNetworks(IJCNN)(2017)1578–1585.

[29]Z.Cui,W.Chen,Y.Chen,Multi-ScaleConvolutionalNeuralNetworksforTime SeriesClassification,ArXiv160306995Cs,Mar.,2016.

[30]A.Graves,A.Mohamed,G.Hinton,Speechrecognitionwithdeeprecurrent neuralnetworks,in:2013IEEEInternationalConferenceonAcoustics,Speech andSignalProcessing,2013,pp.6645–6649.

[31]A.Y.Hannun,etal.,Cardiologist-levelarrhythmiadetectionandclassification inambulatoryelectrocardiogramsusingadeepneuralnetwork,Nat.Med.25 (January(1))(2019)65.

[32]S.Kiranyaz,T.Ince,M.Gabbouj,Real-timepatient-specificECGclassification by1-Dconvolutionalneuralnetworks,IEEETrans.Biomed.Eng.63(March (3))(2016)664–675.

[33]Z.C.Lipton,D.C.Kale,C.Elkan,R.Wetzel,LearningtoDiagnoseWithLSTM RecurrentNeuralNetworks,ArXiv151103677Cs,Nov.,2015.

[34]M.Fiterau,J.Fries,E.Halilaj,N.Siranart,C.Ré,Similarity-basedLSTMsfor timeseriesrepresentationlearninginthepresenceofstructuredcovariates, ConferenceonNeuralInformationProcessingSystems(2016)7.

[35]F.Doshi-Velez,B.Kim,TowardsARigorousScienceofInterpretableMachine Learning,ArXiv170208608CsStat,Feb.,2017.

[36]S.Chakraborty,etal.,Interpretabilityofdeeplearningmodels:asurveyof results,in:2017IEEESmartWorld,UbiquitousIntelligenceComputing, AdvancedTrustedComputed,ScalableComputingCommunications,CloudBig DataComputing,InternetofPeopleandSmartCityInnovation,2017,pp.1–6. [37]R.R.Selvaraju,M.Cogswell,A.Das,R.Vedantam,D.Parikh,D.Batra,

Grad-CAM:visualexplanationsfromdeepnetworksviagradient-based localization,in:2017IEEEInternationalConferenceonComputerVision (ICCV),2017,pp.618–626.

[38]TheArrhythmiaLaboratoryAtBoston’sBethIsraelHospital(NowTheBeth IsraelDeaconessMedicalCenter),TheMIT-BIHNormalSinusRhythm Database,1990physionet.org.

[39]A.L.Goldberger,etal.,PhysioBank,PhysioToolkit,andPhysioNet:components ofanewresearchresourceforcomplexphysiologicsignals,Circulation101 (June(23))(2000)e215–e220.

[40]D.S.Baim,etal.,TheBIDMCCongestiveHeartFailureDatabase,2000 physionet.org.

[41]P.Melillo,E.Pacifici,A.Orrico,E.Iadanza,L.Pecchia,heartratevariabilityfor automaticassessmentofcongestiveheartfailureseverity,XIIIMediterranean ConferenceonMedicalandBiologicalEngineeringandComputing(2013) 1342–1345.

[42]J.S.Bridle,Probabilisticinterpretationoffeedforwardclassificationnetwork outputs,withrelationshipstostatisticalpatternrecognition,Neurocomputing (1990)227–236.

[43]S.Ioffe,C.Szegedy,Batchnormalization:acceleratingdeepnetworktraining byreducinginternalcovariateshift,InternationalConferenceonMachine Learning(2015)448–456.

[44]V.Nair,G.E.Hinton,RectifiedlinearunitsimproverestrictedBoltzmann machines,Proceedingsofthe27thInternationalConferenceonInternational ConferenceonMachineLearning(2010)807–814.

[45]K.He,X.Zhang,S.Ren,J.Sun,Deepresiduallearningforimagerecognition, 2016IEEEConferenceonComputerVisionandPatternRecognition(CVPR) (2016)770–778.

[46]B.Zhou,A.Khosla,A.Lapedriza,A.Oliva,A.Torralba,Learningdeepfeatures fordiscriminativelocalization,2016IEEEConferenceonComputerVisionand PatternRecognition(CVPR)(2016)2921–2929.

[47]D.P.Kingma,J.Ba,Adam:amethodforstochasticoptimization,Proceedings ofthe3rdInternationalConferenceonLearningRepresentations(2015). [48]X.Glorot,Y.Bengio,Understandingthedifficultyoftrainingdeepfeedforward

neuralnetworks,ProceedingsoftheThirteenthInternationalConferenceon ArtificialIntelligenceandStatistics(2010)249–256.

[49]M.Abadi,etal.,TensorFlow:Large-ScaleMachineLearningonHeterogeneous DistributedSystems,2015.

[50]LightWAVE0.64.[Online].Available:https://www.physionet.org/lightwave/. (Accessed28January2019).

[51]U.R.Acharya,etal.,Deepconvolutionalneuralnetworkfortheautomated diagnosisofcongestiveheartfailureusingECGsignals,Appl.Intell49 (January(1))(2019)16–27.

[52]C.Taylor,R.Hobbs,Diagnosingheartfailure–experienceand‘Bestpathways’, Eur.Cardiol.Rev.6(3)(2010)10–12.

[53]S.James,etal.,22roleof12-leadelectrocardiographyinpredictingheart failureinthecommunity,Heart101(September(Suppl.5))(2015)46–49. [54]A.B.deLuna,BasicElectrocardiography:NormalandAbnormalECGPatterns,

1edition,Wiley-Blackwell,Malden,MA,2007.

[55]P.Melillo,A.Orrico,P.Scala,F.Crispino,L.Pecchia,Cloud-basedsmarthealth monitoringsystemforautomaticcardiovascularandfallriskassessmentin hypertensivepatients,J.Med.Syst.39(October(10))(2015)109.

[56]S.Massaro,L.Pecchia,Heartratevariability(HRV)analysis:amethodologyfor organizationalneuroscience,Organ.Res.Methods22(January(1))(2019) 354–393.