Effects of Poisson noise in a IF model with STDP and spontaneous replay of periodic spatiotemporal patterns, in absence of cue stimulation

(1)

ContentslistsavailableatSciVerseScienceDirect

BioSystems

j ou rna l h o me p ag e :w w w . e l s e v i e r . c o m / l o c a t e / b i o s y s t e m s

Effects

of

Poisson

noise

in

a

IF

model

with

STDP

and

spontaneous

replay

of

periodic

spatiotemporal

patterns,

in

absence

of

cue

stimulation

Silvia

Scarpetta

a,b,∗

_,

_Ferdinando

_Giacco

c

_,

_Fabrizio

_Lombardi

d

_,

_Antonio

_de

_Candia

e

a_Department_of_Physics_‘E._R._{Caianiello’,}_University_of_Salerno,₈₄₀₈₄_Fisciano_(SA),_Italy b_INFN_Unita’_di_Napoli_Gruppo_coll._di_Salerno,_Italy

c_Department_of_Mathematics_and_Physics,_Second_University_of_Naples,₈₁₁₀₀_Caserta,_Italy d_Institute_of_{Computational}_Physics_for_Engineering_Materials,_ETH,_Zurich,_Switzerland e_Department_of_Physics,_University_Federico_II,_CNR-SPIN_and_INFN,_Napoli,_Italy

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received16October2012

Receivedinrevisedform28February2013 Accepted19March2013 Keywords: Neuralcriticality STDP Associativememory

a

b

s

t

r

a

c

t

Weconsideranetworkofleakyintegrateandﬁreneurons,whoselearningmechanismisbasedonthe Spike-Timing-DependentPlasticity.Thespontaneoustemporaldynamicofthesystemisstudied, includ-ingitsstorageandreplayproperties,whenaPoissoniannoiseisaddedtothepost-synapticpotentialof theunits.Thetemporalpatternsstoredinthenetworkareperiodicspatiotemporalpatternsofspikes. Weobservethat,eveninabsenceofacuestimulation,thespontaneousdynamicsinducedbythenoise isasortofintermittentreplayofthepatternsstoredintheconnectivityandaphasetransitionbetween areplayandnon-replayregimeexistsatacriticalvalueofthespikingthreshold.Wecharacterizethis transitionbymeasuringtheorderparameteranditsﬂuctuations.

1. Introduction

We study theeffects of noise in the spontaneous collective dynamicsof anetwork of Nleaky Integrateand Fire(LIF) neu-rons, whose learning mechanism is based onthe Spike-Timing DependentPlasticity(STDP).Thetemporalpatternsweconsiderare periodicspike-timingsequences,whosefeaturesareencodedinthe relativephaseshiftsbetweenneurons’spikes.Wereviewthe stor-ageandreplaypropertiesofsuchLIFnetworkswhoseconnections comefromalearningstrategybasedonSTDPwithPphase-coded patternswithrandomphases.

ThenwefocusontheeffectsofanindependentPoissoniannoise addedtothepost-synapticpotentialsduringspontaneous activ-ity.Inpreviousworks(ScarpettaandGiacco,2012;Scarpettaetal., 2010)wefocusedonthedynamicsemergingasaconsequenceof ashortcuestimulation,andweshowedthatinordertotrigger spontaneouspatternsofactivityreminiscentofthosestored dur-ingthelearningstage,afewspikeswiththerightphaserelationship aresufﬁcient.Herewefocusonthespontaneousdynamicswithout anyexternalstimulus,andweshowthatanoisysignaladdedtothe potentialsmayinduceanintermittentreplayofthestoredpatterns.

∗ Correspondingauthorat:DepartmentofPhysics‘E.R.Caianiello’,Universityof Salerno,84084Fisciano(SA),Italy.Tel.:+39089969418;fax:+39089969658.

E-mailaddress:[email protected](S.Scarpetta).

Thisspontaneousreactivationinducedonlybynoise,is particu-larlyrelevantconsideringthatrelationshipbetweenspontaneous andevokedcorticalactivityduringdevelopmentistheobjectof manyrecentstudies(Berkesetal.,2011;Hanetal.,2008).Notably westudythephase-transitionwhichappearsinthenoisy sponta-neousdynamicsofourmodel,inabsenceofexternalstimulation. Wecharacterizethecriticalregime,atthetransitionbetweenthe regionofspontaneouspersistentreplayandtheregionofno-replay. Inthiscriticalregimeashorttransientintermittentreplayis initi-atedbynoise.Thereplaylastsforashorttimeandafterawhile anewpatternmaybetransientlyreactivatedspontaneously.To characterizethespontaneousactivityofthenetwork,weevaluate theorderparameteranditsﬂuctuationsasafunctionofspiking thresholdofthecoupledneurons.Atalowspikingthresholdthe orderparameterishighandﬂuctuationsarelow,whileatthe criti-calspikingthreshold,wherethereisatransitionbetweenno-replay (highth)andpersistentreplay(lowth), theorderparameter’s

ﬂuctuationsaremaximized,asexpectedfora continuousphase transition.Interestinglymanyrecentexperimentalresults,suchas

Tagliazucchietal.(2012),suggestthatrestingbrainspendsmost ofthetimeneartothecriticalpointofasecondorderphase transi-tion.Animportantresultofthispaperisthestudyofthedynamical criticalpointandofthedifferentdynamicalregimesobservedby changingtheexcitabilityparametersofthenetwork.

(2)

theconnections,withabriefillustrationoftheevokedbehavior (Section2.1);inSection3westudytheabilityofthenoisetoinduce thespontaneousrecallofthepatterninabsenceofexternal stimula-tion,weintroduceanorderparameter,andwefocusonthecritical region;summaryanddiscussionareoutlinedinSection4.

2. Themodel

ThespikingmodelhasbeenintroducedinScarpettaandGiacco (2012),Scarpettaetal.(2010).Webrieﬂyreviewthemodelhere. ThesingleneuronmodelisaLIF.WeusetheSpikeResponseModel (SRM) formulation (Gerstnerand Kistler, 2002; Gerstner et al., 1993)oftheIFmodel.Inthispicture,thepost-synapticmembrane potentialofaneuroniisgivenby:

hi(t)=i(t)+

j Jij

ˆtj>ˆti

(t− ˆtj), (1)

wherei=1,...,N,Nbeingthenumbersofunitsinthenetwork,i(t)

isaPoissonnoise,Jijarethesynapticconnections,

(t)describesthe

responsekerneltoincomingspikesonneuroni,andthesumover ˆtjrunsoverallpre-synapticﬁringtimesfollowingthelastspike

ofneuroni.Namely,eachpre-synapticspikej,witharrivaltime ˆtj,issupposedtoaddtothemembranepotentialapost-synaptic

potentialoftheformJij

(t− ˆtj),where

(t− ˆtj)=K

exp

−t− ˆtj m

−exp

−t− ˆtj s

(t− ˆtj) (2)

wherem isthemembranetimeconstant(here10ms),sisthe

synapsetimeconstant(here5ms),istheHeavisidestep func-tion,andKisamultiplicativeconstantchosensothatthemaximum valueof thekernelis 1. Thesignof thesynapticconnectionJij

setsthe signofthe post-synapticpotential’s change,sothere’s inhibitionfornegativeJijandexcitationforpositiveJij.Whenthe

membranepotentialhi(t)exceedsthespikingthresholdi_th,aspike

isscheduled,andthemembranepotentialisresettotheresting valuezero.

Wewanttomimicthespontaneousactivityofcorticalnetworks invitro,intheabsenceofexternalstimulation.Therearemany differentpossiblecausesofnoiseinthesystem.Here,wefocuson theeffectsofPoissonnoise,thereforei(t)ismodelledas

i(t)=Jnoise

ˆtnoise>ˆti

(t− ˆtnoise). (3)

Thetimes ˆtnoisearerandomlyextractedforeachneuroni,andJnoise

arerandomstrengths,extractedindependentlyforeachneuroni andtime ˆtnoise.Theintervalsbetweentimes ˆtnoiseareextractedfrom

a Poissonian distribution P(ıt)∝e−ıt/(Nnoise)_, _while _the _strength

JnoiseisextractedfromaGaussiandistributionwithmeanJnoiseand

standarddeviation(Jnoise).Thisdescribesthenoisyenvironment

inwhichournetworkisembedded,intheabsenceofanyexternal stimulation.

Asshowninmanyrasterplotofinvitrospontaneous dynam-icswithneuronalavalanches,thereisoftenasmallsubsetofunits whichhasahigherspikingratethantheotherunits.Theseunits withspontaneoushigherspikingratearemodelledhereasa sub-setofunitswithlowerspikingthreshold.Theseunitswithlower spikingthresholdarethereforemoresensitivetothePoissonian noisewhichactsonthemembranepotentialsofalltheunitsof thenetwork.Ifsomeoftheselow-thresholdunitsbelongstoone ofthestoredpatternandhaveconsecutivephasesinthispattern, thenitmayhappenthattheseunitsareabletoinitiateacollective replayofthepattern.Tocheckthishypothesis,therefore,we imag-inethatforeachstoredpatternthereisasmallsubsetofZ=N/30 units,withconsecutivephasesinthepattern,thathavelowspiking

Fig.1. PlotofthelearningwindowA()usedinthelearningruletomodelSTDP effects.TheparametersofthefunctionA()aredeterminedbyﬁttingthe experi-mentaldatareportedinBiandPoo(1998).

thresholdth1.Theselowthresholdunitswillbemoresensitiveto

noise,andtheiractivationmayeventuallytriggerthereplayofthe storedpatterns.WhiletheothersN₋PZunitswillhavethreshold th2.

Numericalsimulations of thedynamics are performed for a networkwithPstoredpatterns,whereconnectionsJijare

deter-minedviaalearningrulepreviouslyintroducedinScarpettaand Giacco(2012),Scarpettaetal.(2010,2011,2001,2002),Yoshioka etal.(2007).Theconnectionsvaluesarefrozen,andthecollective dynamicsisstudied.

Therefore,followingScarpettaandGiacco(2012),the connec-tionsJijduetothelearningoftheperiodicpatternissimplygiven

by J_ij = ∞

n=−∞ A(t_j−t_i+nT)=

n A

_j 2− _i 2+ n

, (4) wheret_j (t_i)isthespiketimeofunitj(i)inthepattern.The learn-ingwindowA(=t−t’)isthemeasureofthestrengthofsynaptic changewhenatimedelayoccursbetweenpreandpost-synaptic activity(Fig.1).Thestoredpatternsaredeﬁnedasprecise peri-odicsequence of spikes,i.e. spike-phasecoded patterns,asthe onesshown inFig.2,madeupofonespikepercycle,and each spikehasaphase _j randomlychosenfromauniform distribu-tionin[0,2).Thesetoftimingofspikesofunitjinpatternis t_j+nT₌₍

j )/(2)+n/,where=1/Tistheoscillation

frequencyoftheneuronsinthestoredpattern.Thus,eachpattern isrepresentedthroughthefrequency_and_the_speciﬁc_phases

ofspike _j oftheneuronsj=1,...,N.

Whenmultiplephase coded patternsarestored,thelearned connectionsaresimplythesumofthecontributionsfrom individ-ualpatterns,namely

Jij= P

=1

J_ij. (5)

AsdiscussedinScarpettaandGiacco(2012),thefunctionA()used satisﬁesthebalancecondition

∞

−∞A()d=0.Thisassuresthatin

theconnectionmatrixthesummedexcitation(1/N)

_i,J

(3)

Fig.2.Examplesoftwoperiodicspatiotemporal(phase-coded)patternsofspikes thatarestoredinourIFnetwork.Only10units(randomlychosen)areshown.

thesummedinhibition(1/N)

_i,J

ij<0Jijareequalinthe

thermody-namiclimit.

2.1. Emergingofcollectivepatterns

TherecurrentnetworkwithNLIFunits,withconnectionsﬁxed tothevaluescalculatedinEq.(4)fordifferentvaluesofPisstudied. FollowingScarpettaandGiacco(2012),theresponseofthesystem toacueexternalstimulation,inabsenceofnoise,isillustratedin

Fig.3.AshortcuewithM=300spikeswiththeproperphase rela-tionshipisabletoinducethepersistentreplayofthestoredpattern inaproperregionofparameters,i.e.forlowspikingthresholds. Whileathighthreshold,suchasinFig.3c,thecuestimulationonly triggerashorttransientreplay,andatevenhigherthresholdno activityistriggeredbythestimulation.

Intheregimeofcorrectreply,showninFig.3aandb,wecheck thatifthepartialcueistakenfrompattern=2,theneuronsphase relationshipsrecallthephaseofpattern=2(uncorrelatedwith pattern=1)(notshown).

Notethat,aftertheshortcuepresentation,theretrieval dynam-ics hasthesame phase relationship amongunits as thestored pattern,butthereplaymayhappenonatimescaledifferentfrom thescaleusedtostorethepattern,andthecollectivespontaneous dynamicsisatimecompressed(ordilated) replayofthestored pattern. The time scale of replay changes slightly with spiking threshold.IntheexampleofFig.3aandb,thetimescaleofthe retrievaldynamicsisfasteratlowerthreshold,andisdifferentthan thetimescaleusedtolearnthepatterns.Thetimescaleofreplay activityalsodependson_,_as_studied_in _Scarpetta_and_Giacco (2012).Herewefocusonthecase₌₃_Hz_because_the

correspond-ingtimescaleoftheretrievaldynamicsissuchthattheoscillations frequencyisintherangeofthetaoscillations.Thestorage capac-ity,deﬁnedasthemaximumnumberofencodedandsuccessfully retrievedpatterns,hasbeenstudiedinScarpettaandGiacco(2012)

whenthenetworkrespondtoashortcuestimulation.

Thestoragecapacity,asdeﬁnedinScarpettaandGiacco(2012), isshownasafunctionofthespikingthresholdth,inFig.4a,when

M=N/10spikesareusedascue,and₌₃_Hz._For_the_proper

val-uesofthresholdandforPlowerthanPmax,theevokedretrieval

issuccessful(i.e.cuewithafewspikes(M=N/10)isableto selec-tivelyactivatetheself-sustainedreplayofthestoredpattern).We alsoshow,fortheregionofcorrect evokedreplay,thevalueof

theoscillationfrequencyofcollectiveactivityduringthereplayin

Fig.4b.Atspikingthresholdhigherthanacriticalvaluecrit th no

per-sistentreplaywaspossibleforanyvalueofPinresponsetothecue stimulation.FollowingresultsofScarpettaandGiacco(2012),and

Fig.4,weestimatethiscriticalpointwhenN=3000,=3Hztobe aboutcrit

th 95.Atthresholdsclosetothiscriticalvaluethe

net-workrespondstocuewithatransientactivitythatisashortreplay ofthestoredpattern,butnotwithapersistentreplay.

Thecriticalregime,nearthephasetransition,isinvestigated here,butintheabsenceofanyexternalcuestimulation(M=0). Whilein previousworks(ScarpettaandGiacco,2012;Scarpetta etal.,2010;Yoshiokaetal.,2007)westudiedthedynamics trig-geredbyashortstimulationcue,inthefollowingweinvestigate thespontaneous dynamics withoutanycue stimulation, in the presenceofPoissoniannoiseonly.

Whileintheregionofpersistentreplaytheevokeddynamicsis robustwrtnoise,intheregionnearthecriticalpointthesystemis moresensitivetonoise,thereforeitisimportanttofocusoneffects ofnoiseinthisregionintheabsenceofcuestimulation.Notablyin thiscriticalregionitisthenoiseitselfwhichmayinitiateashort replay,withoutanyexternaltrigger.

InthefollowingthereforewestudyifaPoissoniannoise,ina networkwiththeproperconnectivityasours,isabletoinduce tran-sientpatternreplay,intheabsenceonanycuestimulation.Thisis motivatedalsobyrecentliteratureontheintriguingrelationship betweenevokedandspontaneouscorticalactivity,andonthe pat-ternreplayobservedinthepost-tasksleep.Indeedduringsleep thereactivationofpatternsstoredin theconnectivityisclearly nottriggeredbysensorystimulationbutinducedbynoiseitself spontaneously.

3. Effectsofnoiseinthespontaneouscriticalregime,in absenceofanycuestimulation

Weinvestigatehereif,eveninabsenceofexternalcue stimula-tion,collectivepatternsofactivity,reminiscentofstoredpatterns, emergespontaneously,inducedbynoise.Wethereforestudythe networkspontaneousdynamicsinthecriticalregime,inabsence ofanycuestimulation,inthepresenceofaPoissoniannoise.We focushereonthecriticalbehaviorobservednearthedynamical phase-transition,betweentheregionofpermanentreplayandthe regionofno-replay.Figs.5–7showthespontaneousdynamicsofa networkofN=3000units,P=2patternsstoredintheconnectivity, andnoisegivenbyEq.(3)withnoise=1ms,Jnoise=0,(Jnoise)=5,

intheabsenceofexternalstimulation.AsubsetofN1=200units

havealowspikingthresholdth1=26(tomodeltheheterogeneity

observedininvitrodynamics),andtheotherN−N1=2800units

haveth2=80,90,105,respectively,inFigs.5–7.Differentvaluesof

th1=20,22,24,26,28,30havebeeninvestigated,aswelldifferent

valuesofnoisevariance,andthephenomenaobservedaresimilar butlesspronounced.Soweﬁxthesevaluesofth1=26,(Jnoise)=5

andweshowthespontaneousbehaviorasafunctionofthreshold ofthemajorityofunitsth2.

Atlowerthreshold,th2=80 inFig.5thecollectivereplayof

thepattern=1,inabsenceofanycuestimulation,istriggered spontaneouslyandlastsforaverylongtime.Therasterplotofthe networkdynamicsisshownwithadifferentsortingofunitson theverticalaxes,showingthatpattern=1isreplayed.Thereplay isinitiatedsincethenoisyunitshavetriggeredit,andthereplay ispermanentandveryrobustinthisregime.Fig.6showsthatat th2=_th2c =90,fromtimetotime,thereisashorttransientreplay

ofthepatterns.Eachreplayeventlastsforalimitedintervaloftime andthenfadesaway.InFig.6a,theunitsaresortedaccordingto increasingvaluesofstoredphasesinpattern=1,whileinFig.6b unitsaresortedaccordingto=2.Whenpattern=1isrecalled,

(4)

a

b

c

Fig.3. Networkdynamicsatdifferentvaluesofspikingthresholdth.Inthesesimulationsacueexternalstimulation(Mspikes,showningreen)isused,andasinglevalue ofspikingthresholdisusedforallunitsofthenetwork,andthenoiseisabsent.Atpropervalueofthespikingthresholdthecueinitiatesapersistentretrievalofthe spatiotemporalpattern,whileattoohighthresholdthenetworkissilentorwithonlyashorttransientretrieval.Exampleofselectivesuccessfulevokedretrieval(a,b)and exampleoffailure(c)areshown.Therasterplotof50units(randomlychosen)isshownsortedontheverticalaxisaccordingtoincreasingvaluesofphase 1

i oftheﬁrst storedpattern=1.ThenetworkhasN=3000IFneurons,th=75,90,105,respectively,ina,bandc.ConnectionsaregivenbyEq.(5)withP=2storedpatternsat₌₃_Hz.

Fig.4. (a)Storagecapacity˛c₌_Pmax/N,_deﬁned_as_in_Scarpetta_and_Giacco₍₂₀₁₂₎_,_is_shown_at₌₃_Hz_as_a_function_of_spiking_threshold,_when_M₌_N/10_spikes_are_used_as cuestimulation.AsalwaysinthispaperthenumberofunitsisN=3000.Theﬁgureshowsthatnearcrit

th 95thereisatransitionfromaregionofevokedpersistentreplay toaregionofno-replay.(b)Frequencyofthecollectiveoscillationsofthenetworkdynamicsduringevokedreplay,asafunctionofspikingthreshold,forP=Pmax.

Fig.5.Spontaneousdynamics,withoutanycuestimulation,inanoisyenvironment(noise=1ms,Jnoise=0,(Jnoise)=5)withth2=80,th1=26,N=3000,=3Hz.Spikesare shownwithunitssortedontheverticalaxesaccordingtoorderofunitsintheﬁrst(a)orsecond(b)storedpattern.Unitswiththresholdth1=26areshowningreen,the otherinblack.Permanentreplayofﬁrstpatternisobserved,initiatedbythenoisyunits(lowthreshold,moresensitivetonoise).

(5)

Fig.6. Spontaneousdynamicswithoutanycuestimulationinanoisyenvironment(noise=1ms,Jnoise=0,(Jnoise)=5)withth2=90,th1=26,N=3000,₌₃_Hz._Spikes_are shownwithunitssortedontheverticalaxesaccordingtoorderofunitsintheﬁrst(a)orsecond(b)storedpattern.Unitswiththresholdth1=26areshowningreen,the otherinblack.Shorttransientreplaysofthetwopatternsareinitiated,fromtimetotime.

Fig.7.Spontaneousdynamicswithoutanycuestimulationinanoisyenvironment(noise=1ms,Jnoise=0,(Jnoise)=5)withth2=105,th1=26,N=3000,₌₃_Hz._Spikes_are shownwithunitssortedontheverticalaxesaccordingtoorderofunitsintheﬁrst(a)orsecond(b)storedpattern.Unitswiththresholdth1=26areshowningreen,the otherinblack.Noreplayisobserved.

ashortsortedsequenceofspikesappearsinFig.6a,whilewhen pattern=2isretrieved,ashortsortedsequenceofspikesappears inFig.6b.Athigherthreshold,suchasth2=105inFig.7noreplay

isobserved,andonlynoisyactivityisobserved.Itisnotablythat, atintermediatevalueth2=90thisintermittentreplayisobserved

here,inabsenceofanyexternalcuestimulation.Weobserveasort ofspontaneousreactivationofallstoredpatterns,astheonethat seemstohappenduringsleep,usefulformemoryconsolidation.

3.1. Thecriticalpoint

In orderto characterizethis transition betweena regime of spontaneouspermanentretrieval(intheabsenceofcue stimula-tion)toaregimeofno-retrieval,wemeasurethevarianceandthe meanvaluem_(Tw₎_of_the_order_parameter.

In analogywith theHopﬁeld model, we introduce an order parametertoestimatetheoverlapbetweenthenetworkcollective activityduringthespontaneousdynamicsandthestored phase-codedpattern.Thisquantityis1whenthephases jofneuronsj

coincideswiththestoredphases _j,andisclosetozerowhenthe phasesareuncorrelatedwiththestoredones.

Therefore, we consider the following time-dependent dot product|M_(t,_Tw₎_|_=<_(t)|_>_where _is_the_vector_having

componentsei j ,namely: |M_(t,_Tw₎_|₌

1 N

j=1,...,N t<t∗_j <t+Tw e−i2tj∗/Tw_ei j

(6)

wheret_j∗isthespiketimingofneuronjduringthespontaneous dynamics,andTw_is_an_estimation_of_the_period_of_the_collective

spontaneousperiodicdynamics.

Then,weconsiderthemeanvalueoftheorderparameter: m(Tw)=_N1

s|M(t,T

(6)

Fig.8. Meanvaluem(TW₎_of_the_order_parameter_for_pattern₌₁_(circles)_and_its ﬂuctuations(stars)asafunctionofthechosenwindowTw_,for_different_th2._N₌_3000, th1=26and₌₃_Hz_as_in_Figs._5–7_.

wheretheaverage···isdoneonthestartingtimetofthewindow, and_NsistheaveragenumberofspikesonawindowoftimeTw.

Theﬂuctuationsoftheorderparameteraredeﬁnedby 2(|M_(t,_Tw₎_|)₌ 1

Ns2

[|M(t,Tw)|2₋_|M(t,

Tw)|2

]. (8)

Theorderparameter|M_(Tw₎_|,_at_the_optimal_time-window_Tw_,

i.e.withTw _which_maximizes_|M_(Tw₎_|,_measures_the_similarity

inthesequenceofspikingneuronsandinthephaselagbetween thespikes.Thisisasuitablechoiceespeciallywhenthereplayofa spatiotemporalpatternhastobedetectedindependentlyfromthe compressionofthetimescale.Notethatifwehaveaspiketrain thatisnotperiodic,wecannotdeﬁnetheperiod,howeverwecan deﬁnetheorderparameterlookingatthetime-windowTw_which

maximizes|M_(Tw₎_|.

InFig.8starsareﬂuctuationswhilecirclesarethemeanvalue m_(Tw_),_for_three_values_of

th2=80,90,100,anddifferentvalues

ofthetime-windowTw_._We_simulate

th1=20,22,24,26,28,30,

andweplotresultsforth1=26(whichisthevaluewhichgives

higherorderparameter)inFig.8.Soweevaluatethemeanvalue andtheﬂuctuationsoftheorderparameterforthevalueofTw

whichmaximizesit.

InFig.9themeanvaluem_of_order_parameter_and_its

ﬂuctu-ations2_(|M_|)_are_shown_as_a_function_of_spiking_threshold th2,

whenth1=26andfortheoptimalTw.Atlow spikingthreshold

(th2=80)mishighand2(|M|)islow,indicatingthat,asshown

inFig.5thenoiseisabletoinitiateasuccessfulretrieval(persistent replay)ofthestoredpattern.Athighthreshold(th2=100)either

themeanvalueandtheﬂuctuationsoftheorderparameterarelow. Atthecriticalpoint(th2=90)betweenthetworegimes,the

ﬂuc-tuationsoftheorderparameteraremaximized,asexpectedina continuousphasetransition.

The transition is then characterized by a dynamical order parameter,intermsofadynamicalphasetransition,betweentwo differentdynamicalbehaviors(allintheabsenceofanyexternalcue stimulation).Acriticalbehavior,withmaximizationofﬂuctuation oforderparameteratthecriticalthreshold,isobservedhereasin continuousphasetransitions.Notablyrecentresults(Tagliazucchi etal.,2012)show,usinglargescalefMRImeasures,thatthebrain spentmosttimenearacriticalpoint.Futureworkswillfocuson

Fig.9.Meanvaluem(TW₎_of_the_order_parameter,_at_optimal_Tw_,_and_its_{ﬂuctuations} asafunctionofth2,forN=3000,th1=26and₌₃_Hz_as_in_Figs._5–7_.

thespontaneousactivityatthecriticalthresholdfromthepointof viewofneuralavalanches.

4. Discussion

Here, using an STDP-based learning process (Scarpetta and Giacco,2012;Scarpettaetal.,2011,2011,2009;STDP,2008),we storeintheconnectivityofaLIFnetwork,severalspatiotemporal spikepatterns,andweﬁndthat,dependingontheexcitabilityof thenetwork,differentworkingregimesarepossible,with collec-tive,transientorpersistent,replayactivityinducedsimplybynoise. Inourmodelnotonlytheorderofactivationofthesequenceis pre-served,butalsotheprecisephaserelationshipamongunitsofthe periodicspatiotemporalpattern.

Whileintheregionofpersistentreplaythesystemisrobust w.r.t.noise, asdiscussed inScarpettaand Giacco(2012),inthe regionnearthecriticalpointthesystemismoresensitivetonoise, asshownhere.Inthecriticalregimeindeed,thedifferentsequences storedintheconnectivitymaybereactivatedtransiently,fromtime totime,duetonoise,andinabsenceofanycuestimulation.

Weshowthatatthecriticalpointtheﬂuctuationsoftheorder parameteraremaximized,sincethecollectiveactivityismadeof differentsequencesofdifferentlengths,whichgiveriseto corre-latedactivityonthetopofanoisyuncorrelatedactivity.

Recentlythereisrenewedinterestinreverberatoryactivity(Lau andBi,2005)andincorticalspontaneousactivity(Ringach,2009; Pastalkovaetal.,2008)whosespatiotemporalstructureseemsto reﬂecttheunderlyingconnectivity,whichinturnmaybetheresult ofthepastexperiencestoredintheconnectivity.

Similaritybetweenspontaneousandevokedcorticalactivities hasbeenshowntoincreasewithage(Berkesetal.,2011),andwith repetitivepresentationofthestimulus(Hanetal.,2008). Interest-ingly, inourIFmodel, inorder toinducespontaneouspatterns ofactivityreminiscentofthosestoredduringthelearningstage, alimitednumberofspikeswiththerightphaserelationshipare sufﬁcient,andmoreimportantly,eveninabsenceofsensory stim-ulus,anoisewiththerightphaserelationshipsisabletoinducea patternofactivityreminiscentofastoredpattern.Therefore,by adapting thenetwork connectivitytothephase-codedpatterns observedduringthelearningmode,thenetworkdynamicsbuildsa

(7)

representation of the environment and then, during proper conditions, suchas sleepor rest, thedynamical attractors cor-responding to the stored patterns are transiently activated by noise.

Regardingplacecellsforexample,apossiblescenarioisthat,the patternactivatedrepeatedlyduringexperienceisstoredinthe con-nectivity,andthenactivatedduringsleepwhenthenetworkisnear criticalpointandnoiseisabletoinitiateshortreplaysequences,in absenceofsensorystimulation.Duringexploration,whenthe ani-malvisitadjacentplaceﬁelds,theevokedactivityoftheplacecells isa sequenceofspikeswiththeconsecutiveactivationofplace cells,thenalsoinsideasinglethetacyclethecellsareactivatedin therightsequence.Duringsleep,intheabsenceofexternalinputs, therole ofrecurrentconnections increases,probablydue toan increaseofexcitabilityvianeuromodulatorsorothermechanisms, andthespontaneousactivityofthenetworkshowstemporarily shortreplayofstoredpatterns,probablyinitiatedsimplybythe noise,asproposedinthiswork.

References

Berkes,P.,Orban,G.,Lengyel,M.,Fiser,J.,2011. Spontaneouscorticalactivity revealshallmarksofanoptimalinternalmodeloftheenvironment.Science331, 83–87.

Bi,G.Q.,Poo,M.M.,1998. Precisespiketimingdeterminesthedirectionandextent ofsynapticmodiﬁcationsinculturedhippocampalneurons.J.Neurosci.18, 10464–10472.

Gerstner,W.,Kistler,W.,2002.SpikingNeuronModels:SingleNeurons,Populations, Plasticity.CambridgeUniversityPress,NY.

Gerstner,W.,Ritz,R.,vanHemmen,J.L.,1993. Whyspikes?Hebbianlearningand retrievaloftime-resolvedexcitationpatterns.Biol.Cybern.69,503–515.

Han,F.,Caporale,N.,Dan,Y.,2008. Reverberationofrecentvisualexperiencein spontaneouscorticalwaves.Neuron60,321–327.

Lau,P.-M.,Bi,G.-Q.,2005.Synapticmechanismsofpersistentreverberatoryactivity inneuronalnetworks.Proc.Nat.Acad.Sci.U.S.A.102,10333–10338. Pastalkova,E.,Itskov,V.,Amarasingham,A.,Buzsaki,G.,2008.Internallygenerated

cellassemblysequencesintherathippocampus.Science321,1322–1327. Ringach,D.L.,2009.Spontaneousanddrivencorticalactivity:implicationsfor

com-putation.Curr.Opin.Neurobiol.19,439–444.

Scarpetta, S.,Giacco,F.,2012. Associative memoryofphase-coded spatiotem-poralpatterns inleaky IntegrateandFire networks.J. Comput.Neurosci., http://dx.doi.org/10.1007/s10827-012-0423-7.

Scarpetta,S.,Zhaoping,L.,Hertz,J.,2001. Spike-Timing-DependentLearningfor OscillatoryNetworks.AdvancesinNeuralInformationProcessingSystems,vol. 13.MITPress.

Scarpetta,S.,Zhaoping,L.,Hertz,J.,2002.Hebbianimprintingandretrievalin oscil-latoryneuralnetworks.NeuralComput.14,2371–2396.

Scarpetta,S.,Giacco,F.,Marinaro,M.,2009.Roleoftemporallyasymmetricsynaptic plasticitytomemorizegroup-synchronouspatternsofneuralactivity.Front. Artif.Intell.Appl.204,247–260.

Scarpetta,S.,DeCandia,A.,Giacco,F.,2010. Storageofphase-codedpatternsvia STDPinfully-connectedandsparsenetwork:astudyofthenetworkcapacity. Front.Synapt.Neurosci.2,1–12.

Scarpetta,S.,Giacco,F.,DeCandia,A.,2011.Storagecapacityofphase-codedpatterns insparseneuralnetworks.EPL95,28006.

Scarpetta,S.,DeCandia,A.,Giacco,F.,2011.Dynamicsandstoragecapacityofneural networkswithsmall-worldtopology.ProceedingsoftheConferenceonNeural NetsWIRN10.Front.Artif.Intell.Appl.226,218–226.

Encodingandreplayofdynamicattractorswithmultiplefrequencies:analysisofa STDPbasedlearningrule,2008.LectureNotesinComputerScience(including subseriesLectureNotesinArtiﬁcialIntelligenceandLectureNotesin Bioinfor-matics),LNCS5286,pp.38–60.

Tagliazucchi,E.,Balenzuela,P.,Fraiman,D.,Chialvo,D.R.,2012.Criticalityin large-scalebrainfMRIdynamicsunveiledbyanovelpointprocessanalysis.Front. Fract.Physiol.3,15,http://dx.doi.org/10.3389/fphys.2012.00015.

Yoshioka,M.,Scarpetta,S.,Marinaro,M.,2007. Spatiotemporallearninginanalog neuralnetworksusingspike-timing-dependentsynapticplasticity.Phys.Rev.E 75,051917.