Chaos Characterization of Pulse-Coupled Neural Networks in Balanced State

Fa oltà di S ienze Matemati he,

Fisi he e Naturali

Corso di Laurea Spe ialisti a in S ienze Fisi he

Anno a ademi o 2010/2011

Tesi di Laurea Spe ialisti a

Chaos Characterization of Pulse-Coupled Neural

Networks in Balanced State

Candidato

Relatore

Esterno

Relatore

Interno

grazieaVoi,so he sidà

Introdu tion 7 ComplexNetworks . . . 7 NeuralNetworks . . . 8 Ergodi theory . . . 10 ThesisStru ture . . . 12 1 Mathemati al Framework 15 1.1 DierentiableDynami alSystemsandErgodi ity . . . 15

1.2 Chaoti Dynami sand LyapunovExponents. . . 18

1.3 Cal ulatingLyapunovExponents . . . 21

1.4 Cal ulatingthe(lo al)Lyapunovve tors. . . 24

1.5 MatrixCal ulation . . . 25

1.6 InformationandEntropy . . . 27

1.7 InformationDimensionoftheAttra tor . . . 28

2 Model 31 2.1 SingleNeuronDynami s . . . 31

2.2 NetworkDynami s . . . 37

2.3 TheBalan edState . . . 38

2.4 ModelAr hite ture. . . 40

2.5 EquationsforaQIFNetwork . . . 42

2.6 Cal ulatingtheJa obian. . . 43

3 Computation and Results 47 3.1 NetworkComputation . . . 47

3.2 OutputData . . . 49

3.3 SpikeTrain . . . 50

3.4 Colle tiveDynami s . . . 53

3.5 Attra torDimensionandEntropy . . . 55

3.6 LyapunovVe torsConvergen e . . . 56

3.7 AnglesandHyperboli ity . . . 61

3.8 Parti ipationRatioandChaosIndex . . . 64

4 Dis ussion 71

4.1 Con lusions . . . 71

4.2 Extensions . . . 72

A Temporal Flu tuationsin Balan ed State 75

Fromthersttheoriesofmatteras onstitutedbyelementaryparti les[

Dem-o ritus( a. 460BC a. 370BC)℄uptothetwentieth entury,thedominating

trendinphysi al,and,moregenerally,ins ienti resear hhasbeen

redu tion-ism: the prin iplethat any omplexsystem anbede omposedin elementary

parts,whi harethenstudiedsingularly,sothatthefullknowledgeoftheglobal

phenomenon is expressed in terms of the elementary intera tions between its

simpleentities.

Su h approa h hasprovided, and still provides, marvellous insights, ru ial

fortherealizationofmostofthes ien eand thete hnology weseearoundus.

Nevertheless,itprovesnotsu ientwhenit omestodealingwiththestudyof

theemergent phenomena.

Emergen e anbedened,innaturalandso ials ien e,asthearisingofnovel

and oherent stru tures,patterns andpropertiesin omplexsystems. In

appar-ent ontradi tion tothethirdprin ipleof thermodynami s, wearesurrounded

byemergent,highlyorderedstru tures,lifeitselfbeingthemostastonishing

ex-ampleofallofthem. Unfortunately,aredu tionistapproa hdoesnotproveto

besu ient,orevenapt,forthestudy ofsu hphenomena. Thenew hallenge

fors ien eandresear histondmodelsandrulesforemergentphenomena,and

applythem toreality. Complexnetworks aretypi almodelsshowingthearise

of omplexbahviourand olle tivephenomenafromanaggregationofrelatively

simple onstituents.

Complex Networks

Althoughnetworks anreveal ari h variety ofbehaviours, andemergent

phe-nomena,theirdenitionisquitesimple:anetworkisasetofitems,usually alled

nodes,with onne tionsbetweenthem,referredtoasedges [1℄. Thetopologi al

studyofnetworksstartedwithEuler,andhis solutionto theKönisberg Bridge

Problem, whi h laiddownthefoundationsof themathemati alGraph Theory.

Sin ethen,extensivestudiesonahighvarietyofnetworksandonthedynami s

taking pla e on them have been ondu ted, and now many s ienti models

relyon network stru tures; just to name afew: food webs, geneti networks,

powergrids, modelsfor the spreadingof omputer virusesorhumandiseases,

et . [2℄. A surprising number of examples on the importan e of networks in

s ien e,te hnology, so ietyandevery-daylife anbefound in apopularbook

Neuralnetwork onstitutea lassof omplexnetworkinspiredbythestru ture

of the nervous system. The nodes are identied with neurons, the edges

rep-resentthelinks betweenthem (axon andsynapses)andthe ex hangedsignals

aretheele tri alpeaks,orspikes,(a tionpotentials)triggeredwhenthe

inter-nalpotentialofthe neuronrea hesa ertain threshold. Theywere introdu ed

mainlytosolvearti ialintelligen eproblem,su hasautomatedlearning,visual

pattern re ognition,adaptivity, fault toleran e,et . orasmodels fornetworks

ofrealnervous ells[4℄.

Inthepresentwork,we onsiderthelatter ategoryofneuralnetworks,

aim-ingat abetterunderstandingand modelling ofreal biologi alstru tures. The

rst exampleof su h approa h datesba kin 1975, when Peskin modelled the

pea emaker ellsof the heart asa fully onne ted network of identi al leaky

integrate-and-reneurons(LIF),andveriedthattheyspontaneously

syn hro-nisetheirsignals,asrealpea emaker ellsdo[5℄. TheLIFmodelisthesimplest

onedimensionalmodelfortheele tri al a tivityof aneuron. Theuseofmore

omplex and rened models would add more biophysi al value to the

inves-tigation, but therequired omputations would in reasedramati ally with the

networksize;moreoverphenomenaemergingfromthe olle tivedynami softhe

systemshouldnotdepend toomu honthepre ise andrealisti re onstru tion

ofitssingle omponents.

Thespe i systemweareinterestedinisnotthehearth,butthepyramidal

neuronsoftheneo ortexintheirstationarystate. Manyexperimentsonliving

ells,bothinvitroandin vitro,showedthat,althoughverydense,anddensely

onne ted(

∼ 10

6

neuronsonamm

3

,with

∼ 10

4

onne tionsea h),theyexhibit

verylowaverageringrates(

∼ 1

Hz),withaspikepatternveryvariableintime, andveryweakly orrelated,evenwhen sampledfromneurons loseea hother;

atthesametime,theresponseintimeseemssurprisinglyfastwhen omparedto

rea tiontimes forisolated,individualneurons. Thesystemisthereforedoubly

e ient: thelowspikingrateredu estheenergy onsumption,but,atthesame

time, itisableto rea teven fasterthananyofitsisolated omponentswould.

Admittedly,brainsstilldetainthere ordofmoste ientand omplexparallel

al ulatorsintheknownuniverse.

Goingba ktothetheneo ortex,themodelusedtoexplaintheobserved

a -tivity is the balan ed statemodel. Namely ea h neuron, for most of thetime,

is kept lose to its ring threshold by a balan e between ex itatory and

in-hibitoryinputsfromotherneurons,sothatevenverysmalldeviationsresultin

aqui kresponsebytheinterestedneuron. Dierentexperimentsandmeasures

onrmed this theory (see se tion 2.3 ). Our aim isnow to onstru ta

neu-ralnetworkwhi hrepli ates,atleast qualitatively,this kindof behaviour,and

studyit.

The reation of a neural network in stationary balan ed state requires just

themixingoftherightingredients,namely:

a) apropermodelforsingleneurondynami s, assimpleaspossible;

b) averylargenumberofneurons,andnumberof onne tionsbetweenthem

still verylargebut smallwhen omparedto thesize ofthesystem, soto

graph. Theimageshows

∼

60neuronswithanaverageof

1.5

onne tionsea h. The simulatednetworksrangefrom

200

to

1000

neuronswith

20 − 50

average onne tions. Theorientationofthe onne tionsisnotshown.

) bothex itation andinhibitiona tingonneurons.

The rst hoi e for point a) would be the LIF neuron, however they have

been ex luded for an intrinsi aw in their dynami s: when the membrane

potentialofarealneuron rossesa ertainthreshold,apositive-feedba kpro ess

furtherly in reases it, produ ing a peaked signal; LIF neurons, on the other

end, are arti ially reset in the instant they tou h theirthreshold. With this

qualitativedieren e, LIF neural networks have thetenden y to syn hronize.

Innetworksmadeof inhibitoryneuronssubje tedtopositiveexternal urrents

thisphenomenon hasbeenprovedanalyti ally [6℄. Forthat reasonthe hosen

model is thesimples anoni al 1D neuron model: thequadrati integrateand

redes ribedin hapter2.

Conditionb) is obtained by onne tinga large numberof neurons with an

Erd®s-Rény random graph stru ture, as represented in gure 1. If

N

is the totalnumberof neuronsand

K

theaverage onne tionsper neuron,our ideal onditionswould be

N ≫ K ≫ 1

. In thea tual model

N

rangesfrom

200

to

1000

, whileK from

20

to

100

. Finally all onne tionsareassumed to be one-dire tional,asforrealneurons. Forgraphi alreasonstheguredoesnotshow

onne tionorientations,andtheparametersareredu edto

N ≈ 60

,

K ≈ 1.5

. Asregardsthepoint ), it hasbeenfound thatanetworkof onlyinhibitory

neurons, subje ted to a onstant, ex itatory external urrent, su es to full

it. This simpliesthe model andthe al ulations,sothat we an on entrate

onmoreessentialparameters.

The system dened with those requisites shows deterministi haos. For a

un-expansionis odedbytherstLyapunovve tor.

derstanding, we rely onthe mathemati al framework typi ally used for large,

omplex,dynami alsystems: theErgodi Theory.

Ergodi theory

Sin e hapter 1 ontainsmostoftheformalism,the orre tdenitions andthe

mathemati alrigourne essarytodeneanddes ribetheergodi propertiesand

quantitiesweareinterestedin,inthisse tionwegiveamoreintuitive(andless

formally orre t)des riptionofwhat ergodi ityisabout.

Givenanite-dimensional,deterministi dynami alsystem,itsphasespa eis

dened as then-dimensional manifoldin whi h everysingle pointfully

repre-sentstheexa tstate of thesystemat agiventime. As the systemevolves, it

drawsatraje torythephasespa ewhi h annotinterse titself, ortraje tories

asso iatedto othertime evolutions.

Thebasi ideaof ergodi theoryis that, for su ientextents oftime,

aver-agingovertimeanyquantityasso iatedtothedynami alsystemis ompletely

equivalentto performinganaverageonthewhole phasespa e,aslongassu h

manifoldisweightedwithanergodi measure. Intuitively,su hameasureshould

negle ttheareasofphasespa enever(oralmostnever)tou hedintheevolution

of oursystem,and givemoreimportan e to thosein whi h thesystemlingers

mu hmorein time(theattra tingmanifold).

Withthisapproa hwe anndquantitiesthatareglobalandinvariantforthe

system just followinga single time evolutionstarting at arandom point: the

information gained from su h traje tory would beequivalent to studying the

systemasawhole,ortowhatwewouldndusinganyothertraje torystarting

fromdierentpoints;aslongas,of ourse,thesysteminquestionisergodi . One

ofthemost ru ialquantitiesforthe hara terisationofadynami alsystemis

therst ovariantLyapunovexponent,asitisstri tlyboundwiththedenition

ofdeterministi haos.

The ideaof a dynami al systemboth deterministi and haoti , - i.e. with

a time evolution ompletely and uniquely determined by its variables on one

hand, but omplexand not predi table in the longterm on the other hand

-is only an apparent paradox, easily resolved stating that deterministi haos

d(t

0

) = δ ≪ 1

, evolve in time on paths diverging with an exponential rate

d(t) ≃ δe

λ

1

t

,

λ

1

> 0

(see gure 2) . The result is that, sin e we an know thestate of asystemonly with nite pre ision, any longterm fore astwould

eventuallybeoutgrownbytheexponentiallyin reasingerrorasso iatedto our

initial un ertainty. This is basi ally what Lorentz originally intended as the

butteryee t.

The rst Lyapunov exponent is, roughly speaking, the

δ → 0

limit of the exponent asso iated tothat divergen e, averagedfortime

t → ∞

. If positive, thesystemis learly haoti . Ifnegative,we ansaythat separatetraje tories

wouldeventually onverge,althoughforaparti ular lassofdynami alsystems

thetimeneededgrowssofastwiththesystemdimensionalitythatthedynami s

appearpra ti ally haoti . Su h propertyis alledstable haos [7℄.

Going ba k to gure 2, wenoti e that the maximum divergen e denes, in

time,alsoadire tion. We allthedire tionasso iatedtothemaximum

expan-siontherstLyapunovve tor

v

(1)

_(t)

. Itislo al,sin eitdepends onthepoint

inphasespa eweare onsidering.

Apartfrompoints,we an onsidern-dimensionalphasespa evolumes. This

introdu estheideaofdierentordersofLyapunovexponents,aswellas

dier-entasso iateddire tions: a

k

dimensionalhyper ubewouldin rease(orshrink) itssizeas

V (t) ≃ V (t

0

) exp (t (λ

1

+ . . . + λ

k

))

,andthedeformationofitsshape wouldfollow,intime,thedire tionsoftheasso iatedLyapunovve tors. Figure

3representsanexampleofthis: therstdire tionisexpanding,these ond

or-respondstoa0exponent,nallythethirdis ontra ting. Thevolumeexpands

asthesumof thethree exponents,and itsshape hanges: weexpe t that the

dire tion asso iatedto the 0exponentkeepsbeing parallelto the motion,but

nothing anbesaid,ingeneral,abouttheexpandingand ontra tingdire tions,

andtheanglebetweenthem. Thesystemsweare interestedin aredissipative:

thesumofallLyapunovexponentsisnegative,sothatavolumeinphasespa e

shrinksduringthedynami alevolution,progressivelyfalling onasubsetofthe

totalphasespa emanifold alledattra tor.

The attra tor has, in general, a very omplex shape, possibly fra tal. A

usefulpropertyoftheexponentsisthatthey angiveanesteemoftheattra tor

dimension. Ifthe rst exponent is positive, but the totalsum isnegative,for

a ertaininteger

k

we have

λ

1

+ . . . + λ

k

≃ 0

(if thesumisnot exa tlyzero, we anaddasmallnon-integer orre tionto

k

). Then, forwhat statedabove, avolume ofdimension

k

will neitherexpand nor ontra tin time, thus giving anupper bound totheattra tordimension.

Anotheresteem derived from the exponents is the entropy produ tionrate.

Assuming we know the initial state of the system with nite pre ision, the

exponentialspreadingoftraje toriesfrompointsinitiallyindistinguishableadds

informationregardingtheirinitialstate,thus haoti dynami alsystems anbe

seenasprodu ersofinformation, andtheprodu tionrate anbeestimatedas

thesumofthepositiveLyapunovexponents.

The al ulationsofentropyandattra tordimensionarea tuallyesteemsthat,

in general, annot be taken as exa t equalities. There is, however, a lass of

dynami alsystems, alledaxiom-A,forwiththeentropy al ulationisprovedto

be orre t,whiletheattra tordimensionesteemis onje turedtobe. Estimating

ifa omplex, haoti dynami alsystemisaxiom-Aornotis mathemati allya

another, and shrinksinthe third:

λ

> 0

,

λ

= 0

,

λ

< 0

. Thedire tions are givenbytheLyapunovve torsasso iatedtotheexponents.

thepresentworkisinvestigateifthesystemweareinterestedinsatisesoneof

thekeyrequisitesforaxiom-A:hyperboli ity.

Basi ally,inhyperboli alsystemsexpandingand ontra tingdire tionsdono

mixup(seetheendof hapter1forfurtherdetails. Inthepresentworkwe he k

if this requisite is respe ted by studying the angles between ontra tingand

expandingdire tions,givenbytheLyapunovve torsasso iatedto,respe tively,

positiveornegativeglobalexponents.

Thesis Stru ture

The aim of the present work is to simulate and, using the ergodi formalism,

fully hara terisealarge-s aleneuralnetworkofinhibitoryquadrati

integrate-and-reneurons.

Chapter1introdu esandexplainstheergodi theorymoreformally.

Quanti-tiessu hasattra tordimension,Lyapunovexponents,entropyprodu tionrate

andLyapunovve torsaredened,explainingalsohowthey anbenumeri ally

omputed.

Thefollowing hapterdealswiththe onstru tionoftheneuralnetwork.First

wemodel theone-neurondynami s, taking thegeneral Hodgkin-Huxley

equa-tionsasastartingpoint. Troughdierentlevelsofapproximationwerea hthe

simplest anoni al1Ddes riptionforaneuron: thequadrati integrate-and-re

ortheta neuron. Afterwards,starting from the resultsof in-vivomeasures on

largepopulationsofpyramidalneurons,wedenethebalan edstate,andbuild

a model for the omputation of a large-s alenetwork with similar qualitative

behaviour.

Chapter 3 deals with the omputation and the ndings. It starts with a

step-by-stepdes ription of the omputer simulation we used. Then we assess

them theentropy produ tion rateand the attra tor dimension. The ndings,

sofar, aresubstantiallya onrmation ofwhat has alreadybeenpresentedin

re ent works on the subje t [8, 9℄. We onsequently fo us on the Lyapunov

ve tors: to onrm their invarian eand the robustnessof the algorithm used

for their al ulation, dierent onvergen e tests are performed. The minimal

anglesbetweenve tors, orrespondingtoexpandingand ontra tingdire tions

are usedto estimate thehyperboli ity ofthe dynami alsystem. Theresultis

that,asthesystembe omeslarger,theangledistributionismorepeakedfora

nonzerovalue,showingmoretransversalitybetween ontra tingandexpanding

dire tions.

Finally, for ea h ve tor, we measure the average parti ipation ratio, that

ounts the ee tive number of neurons ontributing to the ve tor dynami s,

andthe haos index (dened asthetime averageofthe squareve tor

ompo-nents),that revealswhi h neuronshaveapredominantroleforasingleve tor.

From the interplay between these two parameters (and some others derived

from them)we an hara terizethe network dynami sbothglobally andfrom

the perspe tive of the single neurons. In parti ular, we nd that the

ontri-bution to the expanding dire tions omes from a group with an average size

thats aleswith

K

,andthesingleneuronsthat takeparttoit hange intime, overinguniformlymostofthenetwork;ontheotherhand,strongly ontra ting

dire tionstendtobelo alizedonfewneurons,xedin time.

The nal part summarises the novel results, namely the use of Lyapunov

ve tors to assess the hiperboli ity of the neural network and to hara terise

the role of individual neurons in the olle tive dynami s. A list of possible

Mathemati al Framework

Westatedthatasetofinter onne tedneurons anbemodeledasanetworkand

studied asa

n

-dimensional dynami alsystem. Inthis hapter wewill des ribe thegeneralmathemati alframeworkusedtostudysystemsofknowndynami s,

hara terizedby haoti behaviorandhigh-dimensionality.

We start with an introdu tion about the statisti al study of dierentiable

haoti dynami alsystems,throughthedenitionofanatural orphysi al

prob-abilisti measureonphase-spa eandtheappli ation ofBirkho'sErgodi

the-orem.

Afterwardswefo usonthe hara terizationof haos,deningbothintuitively

andinamorerigorouswaytheLyapunovexponents.WeshowhowtheOselede

Ergodi theorem impliesthatsu h exponentsexist and arenite, andhowthe

presen eofaOselede splitting ofphase-spa eemergesfromthat.

Usingthe givendenitions, weillustrate the lassi alalgorithmused to

al- ulatethe ompleteset ofLyapunovexponents,whentheequationsregulating

thedynami sarefully known;then weintrodu eaveryre entmethod forthe

al ulationofave torbase orrespondingtotheexpandingdire tionofthe

dy-nami s,i.e. thelo al Lyapunovve tors. Themeaningofsu h ve torsandtheir

onne tionwith thelo al Lyapunovexponents isbrieyexplained.

Inthelastpartwedes ribetwo onje turesofErgodi theorywhi hinvestigate

the onne tion between the Lyapunov spe trum the attra tordimension and

entropy.

Allquantities des ribedinthis hapterwill beexpli itly al ulatedand

ana-lyzedin thespe i dynami almodeldes ribedinthenext hapter.

1.1 Dierentiable Dynami al Systems

and Ergodi ity

A dierentiable dynami alsystem is a time evolutionon a ompa t,

dieren-tiablemanifold

M ⊆ R

m

(thephasespa e),denedbyadierentiablemapping

orow[10℄

f

t

_{: M → M ,}

Figure1.1: Exponentiallydivergingtraje toriesinphasespa e

withtheproperties:

f

0

=

identity and

f

s

_f

t

_{= f}

s+t

_.

(1.1)

Ifthesystemisdissipative,agivenportionofphasespa eisusually ontra ted

bythetimeevolutiononasmallervolume. Theportionofphasespa ewherethe

motionis  on entrated,possiblyafter aninitial transient,is alled attra tor.

We andenetheattra torasaset

A

withthefollowingproperties[11,12℄:

1. is invariantinthedynami alevolution:

∀t

,

f

t

_{A = A}

;

2. attra ts an open set:

∃

openset

U ⊃ A

su hthat

∀x ∈ U

thedistan e between

A

and

f

t

_x

redu es tozeroin thelimit

t → ∞

.

3. is minimal:there isnosubsetof

A

whi hsatisesproperties1and2.

Thelargest

U

satisfyingproperty2is alled basinofattra tion ofA,if

U = M

theattra toris alleduniversal.

Adynami alsystemissaidtobe haoti whenitpresentshighsensitivityto

initial onditions,i.e. whenthetraje toriesoftwodistin t phasespa e points,

arbitrarilynear at

t = 0

, divergeexponentiallyduring time evolution,as illus-trated in gure 1.1. In the next se tion we elaborate the on ept in greater

detailandquantitatively;fornowwe annoti ethat,foragivenstartingpoint,

traje tories al ulated on ma hines with slightly dierent pre isionsor

dier-ent round-omethods would ompletely diverge after a relatively short time.

Unlessweusenumbersofinnitepre ision,there willalwaysbesomeintrinsi

randomnoisethat ompromisesanylong-termfore astofsu hsystems.

Never-thelessglobalstatisti alproperties,su hasthepresen eorthestru tureofthe

attra tor,stay un hanged, regardlessthestarting point and thelevelof noise

(assuming, of ourse, that the latter is reasonablysmall). In haoti systems

theattra toristherefore hara terizedbytheadditionalproperty:

4. is stableunder smallrandom perturbations.

Su h aproperty is essential to guarantee that, in experiments and numeri al

simulations, the motion falls asymptoti ally on the attra tor despite the fa t

The attra tors of haoti dynami al system are alled strange. The name

omes fromthe fa t that theyoftenpresentavery omplex, fra talstru ture,

i.e. withnon-integerHausdor dimension. The denitionof fra talsand their

properties is beyond the s ope of the present work (see for example hap 11

in [12℄ or hapter 3 in [13℄) here we just mention that similar attra tors are

quitehard to model andto study dire tly. This, along with the impossibility

of al ulatingthe true evolutionof apointin phasespa e, suggeststhat the

onlypossibleapproa htota klethoseproblemsisstatisti al andprobabilisti .

ThetoolweuseforouranalysisistheErgodi theory. Itbasi allysaysthat

av-eragesonasingletraje toryintimeequalaveragesoverthewholephase-spa e,

wherethephase-spa eisweightedbyanappropriatemeasure

µ

,withthe funda-mentalrequisitesofbeinginvariantundertimeevolutionandergodi . Abstra t

ergodi theory dealsalot with thestudy and denition of measures that an

satisfythoserequisites. Asphysi ists,hopefullyinterestedin realdierentiable

dynami alsystem,we anlu kilybypassthisproblembyoperationallydene

auniquenatural orphysi al measurein thewaydes ribedbelow.

Westartbytakinginto onsiderationtheprobabilitydensityfun tions(PDF)

on

M

, dened so that, for

x

∈ M

,

ρ(x) dx

is the probability of nding the system in the small volume of phase spa e

dx

around the point

x

. Imagine to have, for

t = 0

a given ensemble of phase-spa e points, orresponding to a density

ρ

0

(x)

, then evolve ea h point in time with

f

, and study the series of pdfs

ρ

1

(x), ρ

2

(x), . . . ρ

t

(x)

. In the

t → ∞

limit, we may expe t that they onvergetoadensitywhi hisinvariantunderthea tionofdynami s:

ρ

inv

(x)

.

This is not always true, but in the systems of our on ern, i.e. haoti and

nite-dimensional, the appli ation of Perron-Frobenius theorem ensures that

su haPDFexists,isunique,andisapproa hedexponentiallyfastintime[14℄:

ρ

t

(x) = ρ

inv

(x) + O(e

−α t

₎

Intuitively, if webegin with a homogeneousdistribution of points in phase

spa e,andevolvethemforsometime,allpointswouldfallovertheattra torand

stay onnedonit. Underapra ti alpointof view,unstable xedpointsand

y lesdonotplayanyrole,sin epointsonthemaredrivenawaybytherandom

noiseintrinsi tonumeri al omputation. Tobemathemati allymorerigorous,

we ouldfollow theidea expressed byE kmann and Ruelle [10℄ (said to have

been rst formulated by Kolmogorov), and dene ourdensity asymptoti ally,

asthe

ǫ → 0

limitofdensities

ρ

ǫ,

inv

hara terizedbyadynami s perturbed by

randomnoiseofmagnitude

ǫ

.

Thephysi al measure

µ

anbedenedastheprobabilityofndingthemotion inagivenphase-spa earea:

µ(B) =

Z

B

ρ

inv

(x) dx .

Fromtheway

ρ

inv

(x)

hasbeen onstru ted,thismeasurehasthe ru ial

prop-ertyofbeinginvariant underdynami alevolution:

µ(B) = µ(f

−t

B) .

Afterndinganatural andidatefortheinvariantmeasure,we andenethe

Theinvariantmeasure

µ

issaidtobeergodi ifthephasespa e

M

ismetri allyinde omposablewithrespe ttoit;i.e. there annot betwodistin tandinvariantsubsets,

A

and

B

,bothwithpositive

µ

measure. Inotherterms,if

A

isinvariant(

f

t

_{A = A}

),then

µ(A) = 1

or

0

.

We an imagine that in systems with distin t attra tors, the dynami

evolu-tion would sele t one of them, a ording to the spe i starting point, and

moveeternallyoverit. Asa onsequen e,thestatisti alpropertiesderivedfrom

su htraje torywouldbeneitherglobalnorinvariantofthestarting ondition,

invalidatingthenotionofergodi ity.

Ifsu hasimpleandintuitiverequisiteisalreadyenoughtostatethepresen e

of ergodi ity, in pra ti e the ergodi hypothesis is impossible to demonstrate

for the great majority of systems. Howeverwe anmake the reasonable

as-sumptionthatthesystemweare onsideringhasonlyoneattra tor,andgoon

illustratingBirkho'sErgodi theorem:

Foraintegrablefun tion

φ : M → R

,thelimit

lim

T →∞

1

T

Z

T

0

dt φ f

t

x

0

=: hφ(x

0

)i

exists. If

µ

isanergodi measure,then

hφ(x

0

)i

isalmosteverywhere onstant(doesnotdepend ontheinitialpoint

x

0

)andequalto:

hφ(x

0

)i =

Z

M

φ(x) µ(dx) =: [φ(x

0

)] .

Foraproofsee, forexample,theoriginalarti lebyBirkho[15℄.

Thepowerfulresultobtainedisthatanyobservableofoursystem,

orrespond-ing to an integrablefun tion

φ

hasa denite global average valueoverphase spa e. This value an be omputed by integrating the fun tion on a random

traje toryreasonablylongintime,andtheresultdoesnotdependontheinitial

point

x

0

ofthespe i traje tory. Thisfundamentalprin iplegivesmeaningto basi allyalltheanalysisperformedin thepresentwork.

1.2 Chaoti Dynami s and Lyapunov Exponents

As mentioned before, a dynami alsystem is said to be haoti when its time

evolution is highly sensitive to the initial onditions. Namely, if we taketwo

points atdistan e

ǫ

arbitrarilysmall, the divergen eof thetwotraje toriesin timewillbe

∼ ǫ e

λt

,

λ > 0

. For

λ < 0

thedieren ewouldqui klyde ay,making thedynami sstable;if

λ = 0

nothing anbesaid. Ifthemotionis onnedonan attra tor,thedistan e annotgrowindenitely. Whathappensisthat thetwo

traje tories,aftertheinitialstrongdivergen e,be ome ompletelyindependent

oneanother.

Ifweapplythis on epttoapointinphasespa e

x

,withasmallerror asso i-ated

δx

,werea hthe on lusionthateventhougha haoti systemisregulated bypre iseequations,theexponentialgrowoferrorsmakesany al ulatedtime

evolutionself-independent of itslong-term pasthistory, i.e. non-deterministi

inanypra ti alsense[11℄.

The parameter

λ

, alled the Lyapunov exponent, is of primary importan e in studying quantitatively the haoti behavior and estimating its long-term

predi tabilityinsu hsystems. Forapre isedenitionand al ulation,weneed

asatisfyingmathemati aldes riptionoftheentities inquestion.

Let

M

beasmooth, ompa tmanifoldand

f

amappingorowover

M

with theproperties(1.1);let

µ

beameasureinvariantundertimeevolution;

x

isour starting point on

M

. We anidentify the small dieren e between

x

and an arbitrarily losepointasave tor

u

belongingtothetangentve torspa eto

M

inpoint

x

,

T

x

M

. Pairsexpressedas

(x, u)

,with

x

∈ M

and

u

∈ T

x

M

anbe interpretedaselementsofthetangentbundle of

M

,thatwe all

T M

.

We andeneaniterationofagivenamountoftime

t

asamapoftheform:

T

t

: T M → T M

(x, u) → f

t

(x), T

x

t

u
.

Thelinearoperator

T

t

x

mapstheve tor

u

∈ T

x

M

in

u

˜

∈ T

f

t

(x)

. Thisoperation an be seen asthe push-forward of

u

by

f

: this implies that the operator is simply thelinearized versionof

f

t

_(x)

, i.e. its total derivative,represented by

theJa obian matrix:

T

t

x

:= Df

t

(x)

.

Fromthepropertieson

f

and the hainderivationrulewehavethat:

T

x

t+s

= T

s

f

t

_(x)

T

_x

t

(1.2)

in other words, along time iteration is equivalent to a produ t of short time

iterations, onsistingofelementsdenedfollowingthetraje toryofthestarting

point

x

.

From now onwewill onsider

f

asa dis retemap, with

t ∈ N

. Howeverit ispossibleto generalizeallthefollowingdenitions andresultsfor ows,with

little hangeinthenotation. If

|| · ||

x

istheve tornormoverthespa e

T

x

,we anexpresstherateof hangeoftheve tor

u

foronestep

(t = 1)

as:

r(x, u) =

||T

1

x

u

||

f

1

_(x)

||u||

x

forthenexttimestep,using(1.2),wehave:

r(f

1

_{(x), T}

1

x

u) =

||T

1

f

t

_(x)

T

x

1

u

||

f

2

_(x)

||T

1

x

u

||

f

1

_(x)

=

||T

2

x

u

||

f

2

_(x)

||T

1

x

u

||

f

1

_(x)

.

Wenwe omputethegeometri mean of the hange rateforthe

n

elementsof thetimeseries

x

,

f

1

_(x)

,

f

2

_(x)

...

f

(n−1)

_(x)

,theresultis:

h

r



f

(n−1)

(x), T

x

(n−1)

u



· r



f

(n−2)

(x), T

x

(n−2)

u



· . . . · r(x, u)

i

1

n

=

=

"

||T

n

x

u

|| ||T

(n−1)

x

u

|| . . . ||T

x

1

u

||

||T

x

(n−1)

u

|| . . . ||T

x

1

u

|| ||u||

#

n

1

=

 ||T

n

x

u

||

||u||



n

1

.

The logarithm of this quantity in the large

n

limit is alled the Lyapunov Chara teristi Exponent (LCE)of

(x, u)

λ(x, u) := lim

n→∞

log

 ||T

n

x

u

||

||u||



1

n

=

lim

n→∞

1

n

log ||T

n

x

u

|| − lim

n→∞

1

n

log ||u|| = lim

n→∞

1

n

log ||T

n

x

u

||

(1.3) Thisdenition satisestheproperties:

∀k ∈ R\{0} λ(x, k u) = λ(x, u) ;

∀u, v ∈ T

x

M

λ(x, u + v) ≤ max {λ(x, u), λ(x, v)} .

(1.4)

TheLCEdepend onlyonthedire tion oftheve tor,andnotonitsmagnitude

(adire t onsequen eofthelinearityoftheoperator

T

t

x

).

The existen e of su h limits is granted by Oselede 's multipli ative

er-godi theorem[16℄. It anbeexpressedasfollows:

Givena ompa t,dierentiablemanifold

M

,amapping

f : M →

M

andameasure

µ

invariantover

M

; let

T

1

beamap from

M

to thespa eallthe

m×m

realmatri es,withthenotation

T

1

_{(x) = T}

1

x

, su hthat

Z

M

µ(dx) log

+

||T

x

1

|| < ∞ ;

where

log

+

(k) = max{0, log(k)}

and

|| · ||

isamatrixnorm; let

T

n

x

bedened astheprodu t

T

n

x

= T

f

1

(n−1)

_(x)

...

T

1

f (x)

T

x

1

. Thenthere is a

f

-invariantsubspa e

N ⊆ M

su hthat

µ(N ) = 1

and

∀x ∈ N

(indi atingwith

A

∗

theadjointof

A

)thematrix

Λ

x

:= lim

n→∞

((T

n

x

)

∗

T

x

n

)

1

2n

exists.

It has

s ≤ m

distin t, real eigenvalues, that an be ordered as

exp λ

(1)

x

> . . . > exp λ

(s)

x

, orresponding to the eigenspa es

U

(r)

x

;

r = 1, . . . , s

. The

λ

(r)

x

exponentsassumerealvaluesor anbe

−∞

ifthe orrespondingeigenvalueis

0

.

Ifwedene

L

(r)

x

= U

x

(r)

⊕ U

x

(r+1)

⊕ . . . ⊕U

x

(s)

,and

L

(s+1)

x

={0}, wehavethatfor

u

∈ L

(r)

x

\ ∈ L

(r+1)

x

lim

n→∞

1

n

log ||T

n

x

u

|| = λ

(r)

x

,

i.e. thelogarithms of theeigenvalues of

Λ

x

are theset ofall LCE we anndfrom

x

. Finally,dening

d

(r)

x

:=

dim

U

(r)

x

,wehavethat thefun tions

x

→ λ

(r)

x

and

x

→ d

(r)

x

are

f

-invariant(

λ

(r)

x

= λ

(r)

_f

t

_(x)

, et .) and, ifthe systemisergodi , arealmost everywhere onstant

(withthepossibleex eptionofasetof

0

measure).

Thematrix

Λ

x

is alled Oselede Matrix. If wewrite the ve tornorm asa s alarprodu t

||u|| =

phu, ui

;theexpansionrateafterasingleiterationis:

||T

1

x

u

||

||u||

=

s

hT

1

x

u, T

x

1

u

i

hu, ui

=

s

h(T

1

x

)

∗

T

x

1

u

, ui

hu, ui

;

thelastequality omesfromthedenitionofadjointmatrix. Now,assuming

u

isaneigenve torof

(T

1

x

)

∗

_T

1

x

witheigenvalue

k

,theresultis:

s

h(T

1

x

)

∗

T

x

1

u

, ui

hu, ui

=

s

hu, ui k

hu, ui

=

√

k .

(1.5)

This explanation hasthe only purpose of des ribing theidea behindthe

on-stru tionof theOselede matrix. Forafulldes riptionofthe theoremand its

relationwith theLyapunovexponents,see [11,17, 18℄. An exhaustive

mathe-mati aldemonstration anbefoundin[19℄. TheOselede multipli ativeergodi

theorem isnot simplyan alternativewayto express theLyapunovexponents,

butmorethanthat,itstatestheexisten eofsu halimitasaninvariant

prop-ertyofthedynami alsystem,independentfromtheinitialpoint hosenfortime

evolution.

Thefolding ve torspa es

L

(1)

x

⊇ L

(2)

x

...

⊇ L

(s)

x

indu e anaturalsplitting on thetangentspa e,knownasOselede splitting. Theimportan eofsu hsplitting

anbeillustratedasfollows: ifwetakearandomve tor

u

belongingto

T

x

M

, itsmeangrowratewillbe

exp λ

(1)

x

,i.e. theexponentialofthehighestLyapunov exponent. This omesfromthefa t that

u

∈ L

(1)

x

: thesubspa es

L

(2)

x

, L

(3)

x

. . .

havezeromeasurewith respe tto thetotalspa e, sotheprobability ofa

ran-domve tortobelimitedtotheminsubstantiallyzero. However,ifwe onsider

arandomve tor

u

˜

fromwhi hwesystemati allyremovethe omponentin the dire tion(s)ofhighestexpansion,i.e. itsproje tionon

U

(1)

x

, then

u

˜

∈ L

(2)

x

and its LCE will bethe se ond highest exponent,

λ

(2)

x

, and so on. The numeri al al ulationofthe ompletespe trumofLCE isessentiallybasedonthis

me h-anism. We an alsonoti e thatthe orderingof thespe trumindu esdierent

ordersofexpansion(andstability,fornegativeLCEs)indierentsubspa es.

Moreover,the basisof ea h

U

(r)

x

subspa e(i.e. the

Λ

x

normalized eigenve -tors), representsthe (average) dire tion asso iated to the expanding (or

on-tra ting) average rate

exp λ

(r)

x

. Su h dire tions are alled Lyapunov ve tors and,asdes ribedin se tion1.4, anbe omputedandstudied lo ally, inorder

togainusefulinformationonthelo aldynami sandglobaldynami s.

1.3 Cal ulating Lyapunov Exponents

The lassi alalgorithmforthe al ulationoftheLyapunovexponentsdatesba k

to 1980 [20℄. It onsiders only dynami al systemswhose governing equations

arefully known and omputable. If thedynami s ishidden, it isstill possible

to estimate some of the exponents by using an empiri al time series of some

observableof the system. In[21℄, forexample, both ases are onsidered (see

also [22℄). However, for the s ope of the present work, we will sti k to the

hypothesisofthe lassi alalgorithm.

Following the reasoning of the original arti le, westart with the pro edure

forthe rst, highestexponent, and then wegeneralize theresult to thewhole

spe trum. Firstofallweremindthatforastartingpointinourmanifold

x

∈ M

andave torinitstangentspa e

u

∈ T

x

M

, thelinearoperatorinvolvedin the push-forward

(x, u) → (f

t

_{(u), T}

t

x

u)

is the Ja obianof

f

t

x

. This valueis basi ally the ompletederivative of

f

t

: in se tion 2.6of next

hapterwewillshowthe al ulationinourspe i setting. Fornowweassume

the

T

t

x

matri esasgivenforevery

x

in

M

.

As said in previousse tion, for the rst exponent is su ient to follow the

evolutionofarandomve tor

u

0

. Tohaveanintuitiverepresentationofthe pro- ess,we animaginetode ompose

u

0

inthebasisofthedire tionsofexpansion asso iated to ea h exponent (i.e. the Lyapunovve tors). Being random, our

starting ve torwill haveanonzero omponentfor ea h dire tion. Due to the

linearityofthepro essea h omponentwillin rease,onaverage,exponentially,

a ordingto therespe tiveLyapunov exponent. The exponentialdieren e of

thegrowingrateswill ausethe omponentasso iatedto thehighestexponent

to dominateoveralltheothersafter ashort amountofiterations,so thatit is

theonlyonesele tedinthelarge

n

limit.

Theonlydi ultyisthatanexponentiallygrowingve torwouldsoongoout

oftheboundariesofour omputational apabilities(that'sbasi allythereason

whywe annon al ulate

Λ

x

dire tly). It issolvedasfollows. Givenaninteger

k & 1

su h that

T

k

x

lies safely in our numeri limits, westart with a random ve tor

u

0

∈ T

x

M

and al ulate iterativelytheseries:

u

1

= T

x

k

u

0

||u

0

||

,

u

2

= T

f

k

k

_(x)

u

1

||u

1

||

,

u

3

= T

f

k

2

_k(x)

u

2

||u

2

||

, . . .

u

i

= T

_f

k

(i−1)k

(x)

u

i−1

||u

i−1

||

=

T

k

f

(i−1)k

(x) T

k

f

(i−2)k

(x) . . . T

k

f

k

_(x)

u

0

||u

i−1

|| ||u

i−2

|| . . . ||u

0

||

;

||T

x

ik

u

0

|| = ||u

i

|| ||u

i−1

|| . . . ||u

0

|| ;

nally,from

u

0

∈ L

(1)

x

and(1.3),wehave:

λ

(1)

x

= lim

n→∞

1

n

n

X

i=1

log ||u

i

|| .

(1.6)

Both thefa t that

T

k

x

isapplied onlytonormalizedve torsandthat the loga-rithmis omputedateverysinglestephavethepositiveee tof ontainingthe

sizeofthenumbersinvolved,makingthe omputationpossible.

To al ulatethewholespe trum,wemustthinkintermsofvolumevariations.

Weassumethe

Λ

x

matrixto be

m

dimensional and,forsimpli ity, that allits eigenvalueshavemultipli ityone(i.e. wehave

m

distin tLCE).Let

U ⊂ T

x

M

beaopenset ofvolumeVol

(U )

; usingthe denitionof LCEand theOselede theorem,weinferthatitsaveragegrowingratein timeis

∝ exp

P

m

i=1

λ

(i)

x

. We start from

U

0

, dened for onvenien e as the

m

-dimensional hyper ube en losed in a random orthonormal basis of

T

x

M

:

{u

(1)

0

,

u

(2)

0

, ...

u

(m)

0

}

; Let

A

be the linear operator orresponding to a single time iteration of the set. Followingthesamereasoningasbefore,wedeneiterativelythesets

U

i

as:

U

i

=

A(U

i−1

)

Vol

(U

i−1

)

;

sothat, fromthelinearityof

A

,itfollows:

Thequantityontherightisthevolumeofourinitialhyper ubeafter

i

timesteps, namely: Vol

(AA . . . AU

0

) =

Vol

(T

ik

x

u

(1)

0

, T

x

ik

u

(2)

0

, . . . T

x

ik

u

(m)

0

) ;

thisexpressionleadsto thesumofallLCE:

lim

n→∞

1

n

log

Vol

(A

n

_U

0

)

Vol

(U

0

)

= lim

n→∞

1

n

log (

Vol

(A

n

_U

0

)) =

m

X

j=1

λ

(j)

x

.

Thequantitiesontheleftassumetheform:

Vol

(U

i

) =

Vol

(A(U

i−1

))

Vol

(U

i−1

)

=

(1.7)

=

Vol

(T

k

f

(i−1)k

_(x)

u

(1)

i−1

, T

f

k

(i−1)k

_(x)

u

(2)

i−1

, . . . T

f

k

(i−1)k

_(x)

u

(m)

i−1

)

Vol

(U

i−1

)

.

Byiteratingtheve torsasbefore(

u

(j)

i

= T

f

k

(i−1)

(x)

u

(j)

i−1

/||u

(j)

i−1

||

)and omputing thevolumeforea hstep,we an,inprin iple,ndtherightresult;howeverthis

method is not feasible, due to the fa t that all ve tors would soon onverge

onthe dire tion of maximumexpansion, sothat theangles betweenthem are

beyondthenumeri alresolutionandthevolume annotbe al ulated.

To ir umventthisproblem,forea htimestepwere omputetheve tors

den-ingthevolume

U

i

performingaGram-S hmidtorthogonalizationpro edure:

U

i

doesnot hange,anditsvolumesimplybe omestheprodu toftheorthogonal

ve tor norms. In short, if we assume

h·, ·i

asthe s alar produ t in the spa e

T

f

ik

_(x)

M

,webuildtheseriesof

u

(j)

i

as:

u

(1)

_i

= T

_f

k

(i−1)k

_(x)

u

(1)

_i−1

||u

(1)

i−1

||

;

for

j > 1

:

u

(j)

_i

= T

k

f

(i−1)k

_(x)

u

(j)

_i−1

||u

(j)

i−1

||

−

j−1

X

r=1

*

T

k

f

(i−1)k

_(x)

u

(j)

_i−1

||u

(j)

i−1

||

, u

(r)

_i

+

u

(r)

_i

||u

(r)

i

||

2

.

(1.8)

With this denition Vol

p

_(T

k

f

(i−1)k

_(x)

u

(1)

i−1

, T

f

k

(i−1)k

_(x)

u

(2)

i−1

, . . . T

f

k

(i−1)k

_(x)

u

(m)

i−1

)

isequivalentto:





m

Y

j=1

||u

(j)

i−1

||





·

Vol

p

_(u

(1)

i

, u

(2)

i

, . . . u

(m)

i

) =





m

Y

j=1

||u

(j)

i−1

||





·





m

Y

j=1

||u

(j)

i

||





.

Thelast equalityisduetotheorthogonalityofthe

u

(j)

i

ve tors. From(1.7)we obtainVol

p

_(U

i

) =

Q

m

j=1

||u

(j)

i

||

,whi hleadsto theresult:

m

X

j=1

λ

(j)

x

= lim

n→∞

1

n

log





m

Y

j=1

||u

(j)

0

|| ·

m

Y

j=1

||u

(j)

1

|| · . . .

m

Y

j=1

||u

(j)

n

||





.

Rearranging the produ t indexes and de omposing the logarithm, we nally

obtain,forthe

s

th exponent:

λ

(j)

x

= lim

n→∞

1

n

n

X

i=1

log ||u

(j)

i

|| .

(1.9)

Equation(1.9)isanextendedversionof(1.6),withthesigni antdieren ethat

the

u

(j)

i

ve tors,for

j > 1

, are al ulateda ordingto theorthonormalization pro edure (1.8) . As we will see in se tion 1.5 this pro ess anbe simplyand

straightforwardlytranslatedinbasi operationonmatri es.

Before moving to the al ulation of Lyapunov ve tors, some onsiderations

about dimensions should to be done. The result of (1.9) is dimensionless, as

it represents the logarithm of average expansion per step. To obtain a more

generalquantity,independentof thesteplengthsused fortheevolutionof the

dynami alsystem(aslongastheintervalsaresmallenoughtofollowproperly

thedynami alevolution),weneedtores aletheexponentsusing theduration

ofatimestep

∆t

:

λ

(j)

i

←

λ

(j)

_i

∆t

(1.10)

The oe ientsarethusexpressedin se onds

−1

.

Thesystemanalyzedthepresentwork hasthepe uliarityofhavingstepsof

dierent time lengths. Sin e the LCE is dened as an average quantity, we

divide itby theintervallengthaveragedonallperformedsteps:

[∆t]

s

(see for examplepag. 120of[14℄).

1.4 Cal ulating the (lo al) Lyapunov ve tors

As said, the Lyapunov ve tors, dened as a base for the eigenspa es of

Λ

x

, representthedire tionoftheglobalaverageexpansionasso iatedtoea h

expo-nent. It is then possible to dene thelo al Lyapunovve torsasthepreferred

expanding dire tions forthe (linearized)dynami s of ea h point in time. The

lo alLyapunovexponentsarethenthepun tualexpansionratesinea hofthose

dire tions.

Theidenti ationat anystep

s

ofthe maximumlo alexponent

λ

(1)

s

and its asso iated dire tion

v

(1)

s

is straightforward: as said, a random ve tor, freely evolvingin time,wouldrapidlyalignwithit. Sothat,usingthe

u

(1)

s

ve torsas dened above,weobtain,fortheve torandtheexponent:

v

(1)

s

=

u

(1)

s

||u

(1)

s

||

;

λ

(1)

s

= log





||T

k

f

ik

_(x)

u

(1)

s

||

||u

(1)

s

||





= log ||u

(1)

(s+1)

|| ,

(1.11)

for values of

s

reasonably distant from

0

, so that

u

(1)

has su ient time to

align,andassumingthatthemappingissmoothenough,sotheve torproperly

followsthedominatingdire tionateverypointintime.

Movingto

v

(2)

s

and

λ

(2)

s

,theideaintheseries(1.8)isthat,afterweapplythe linearoperator,weproje ttheresultingve toronthespa eorthogonalto

v

(1)

s

,

t

t

v

1

v

2

Figure 1.2: In a haoti dynami al system any perturbation will onverge onthe

dire tion of the rst Lyapunov ve tor. Onthe other hand, a timeinversion inthe

dynami sresultsinthemarkeddominationoftheleastforwardexpandingdire tion,

i.e. these ondve tor.

thusthegrowthinthatdire tiondire tion(expe tedtobedominating)is

om-pletelysuppressed, whilethenexthighestgrowingdire tion,regulatedby

λ

(2)

s

, be omes visible. As aresult,

u

(2)

s

/||u

(2)

s

||

hasthesamedire tionasthe proje -tion of

v

(2)

s

onthespa eperpendi ularto

v

(1)

s

. Ingeneral

u

(2)

s

/||u

(2)

s

|| 6= v

(2)

s

: noreasonfor estheLyapunovve tors,eitherglobalorlo al,tobe

perpendi u-larea hother. La kingtheknowledge ofthe orre tdire tion andofthelo al

exponentdenedasabove(it annotbefounditthe orrespondingdire tionis

notknown),we annot al ulatethetrue

v

(2)

s

.

Asolutionto ir umventthisproblemhasbeenproposedonlyin2007[23℄. It

isbasedonthequiteintuitiveandwellknowprin iplethatatime-inversionin

thedynami sofoursysteminvertstheLyapunovspe trum. Avolumenormally

expanding in some dire tions, in a ba kward motion would ontra t in those

samedire tionswithinvertedratios,whilearandomve tortravelingba kwards

in timewouldsoonfollowthedire tion oflessexpansionin forwardtime. See

gure1.2to haveagraphi alideaofthepro ess.

Fortheargumentsabove,weknowthat Span

(v

(1)

s

, v

(2)

s

) =

Span

(u

(1)

s

, u

(2)

s

)

. Ifwetakearandom ombinationof

u

(1)

s

and

u

(2)

s

andevolveitba kwards,after someiteration(let's say

h

)it will bealignedwith the lessforward expanding dire tionofoursubspa e,namely

v

(2)

s−h

. Forve tor

v

(s)

s−h

wesimplystartwitha linear ombinationofve tors

v

(1)

s

, u

(2)

s

,...

u

(s)

s

,keepinginmindthat

h

should bebigenoughtolettheve torsalignproperly.

1.5 Matrix Cal ulation

Assaid,thereisasimpleandelegantwaytotranslatethealgorithmsdes ribed

westartdeningarandom,orthonormal,

m×m

matrixwhose olumnsrepresent abasisfor

T

x

M

:

Q

0

= (q

(1)

0

| . . . |q

(m)

0

)

. Duetotheirstru ture,the

q

(j)

ve tors,

i.e. the olumnsofthe

Q

matrix,are alledthe Gram-S hmidt basis.

T

k

x

istheJa obianmatrixasso iatedtoa

k

-stepsevolution,with

k

hosenso thattheexponentialgrowthsarekeptinthenumeri allimitsofour al ulator.

Wenowperformthemultipli ation:

T

_f

1

k

₋₁

x

. . . T

1

f

1

_x

· Q

0

= (T

x

k

q

(1)

0

| . . . |T

x

k

q

(m)

0

) = (˜

q

(1)

1

| . . . |˜q

(m)

1

) = ˜

Q

1

;

(1.12) where

T

1

x

...

T

1

f

k−1

_x

are the Ja obianmatri es fora singletimestep, omputed forthespe i dynami alsystemweareanalyzing.

Theve tors

q

˜

(j)

1

arenotorthogonal. Toobtainthe orre titerated

Q

1

matrix, we apply a

QR

de omposition on

Q

˜

1

. The

QR

de omposition on a generi square matrix

Q

˜

onsists in writingit astheprodu t

Q = Q · R

˜

,su h that

Q

isorthonormal and

R

isuppertriangular. Calling

˜

q

(j)

and

q

(j)

the olumnsof,

respe tively,

Q

˜

and

Q

,thenewmatri esaredened by:

q

(1)

=

q

˜

(1)

||˜q

(1)

||

;

q

(j)

₌

˜

q

(j)

₋

j−1

P

r=1

D

˜

q

(j)

, q

(r)

E

_q

(r)

















˜

q

(j)

₋

j−1

P

r=1

D

˜

q

(j)

, q

(r)

E

_q

(r)

















{R}

ij

=

D

q

(i)

, ˜

q

(j)

E

,

for

i ≤ j .

(1.13)

Itisstraightforwardtoseethatthesevalues anbeexpressedin termsofthe

u

(j)

ve tors(1.8)as

q

(1)

1

= u

(1)

1

/||u

(1)

1

||

...

q

(m)

1

= u

(m)

1

/||u

(m)

1

||

Moreover,from thedenitionof the

QR

algorithmdes endsthat thediagonalof the

R

matrix orrespondstothe

u

(j)

1

norms:

{R

1

}

jj

= ||u

(j)

1

||

.

Theorthonormalized

Q

1

isnowreadyforanotheriteration:

T

_f

1

2k−1

_x

. . . T

1

f

k

x

· Q

1

= ˜

Q

2

= Q

2

· R

2

...andsoon. The

u

(j)

i

ve tor modules, given by the diagonal of

R

i

, are stored in ea h iterationforthe al ulationoftheLyapunovspe trum.

Thissamesetting,alongwiththeresultsalready omputed, anbealsoused

tondtheLyapunovve tors. Theendingpointofour omputation

T ≫ 1

will beassumedasthestartingpointofourba kwarditeration. Thestarting

ran-domve tors

v

(1)

T

∈

Span

{u

(1)

T

}

,

v

(2)

T

∈

Span

{u

(1)

T

, u

(2)

T

}

...,

v

(m)

T

∈

Span

{u

(1)

T

, ...

u

(m)

T

}

anbedenedastheprodu tof

Q

T

witharandomtriangularmatrix

C

T

. Consideringthetimeiteration,weobtain:

Q

T

· C

T

= T

_f

k

(1−T )k

_(x)

· Q

T −1

· C

T −1

= ˜

Q

T

· C

T −1

= Q

T

· R

T

· C

T −1

;

sothat

C

T −1

= (R

T

)

−1

· C

T

.

(1.14)

The previously al ulated triangular

R

s

matri es must thus be fully stored, invertedandusedtoiterate ba kwardin timethe

C

i

matrix:

C

i

= (R

i−1

)

−1

· (R

i−2

)

−1

· . . . · (R

T

)

−1

· C

T

.

Sin e weare interestedonly in the ve tordire tions, we an freelynormalize

the olumnsofthe

C

i

matri es. Ifwedothatateverystep,we anidentifythe normalizationfa torsatpoint

i

withthelo al Lyapunovexponents

λ

(j)

s

, while theLyapunovlo alve torsaresimplythe(normalized) olumns of

Q

s

· C

s

.

1.6 Information and Entropy

Considertwopointsvery loseoneanotherinthephase-spa eofa haoti

sys-tem: for any observerwhose instruments have a pre ision oarser than their

distan e,theywillappear ompletely indistinguishable. Eventually,asthe

sys-temevolvesintime,thetraje toryseparationdueto haoti dynami smakesthe

distan esigni ant,sothatthepointsareper eivedasseparate. Thus,systems

verysensitivetoinitial onditions anbeseenasprodu ersofinformation.

Let

{A

1

, A

2

, . . . , A

α

}

beanite,

µ

-measurablepartitionofthephasespa e. We anassumeit orrespondstotheresolutionofourinstruments,sothattwo

pointsinthesamesetofthepartition

A

annotbeseenasdistin t. We anthen dene

f

−

k

_(A

i

)

astheset of points

x

su h that

f

k

_{(x) ∈ A}

i

, and all

f

−

k

_{(A )}

thepartition

{f

−k

(A

1

),

...

f

−k

(A

α

)}

. Wenally onsider thepartition given bytheleast ommonrenement:

A

(n)

_{= A ∨ f}

−

1

_{(A ) ∨ f}

−

2

_{(A ) ∨ . . . ∨ f}

1−n

_{(A ) ;}

itisdenedsothatageneri setin ithastheform:

A

i

1

∩ f

−1

(A

i

2

) ∩ . . . ∩ f

1−n

(A

i

n

)

for

i

j

∈ {1, 2, . . . , α}

It is lear that the latter, dynami s-related, partition has a resolution mu h

nerthanthestartingone,sin eanyelementin itisdis riminated byitspast

history,upto

n − 1

steps.

Now,we andenetheinformation ontentofthepartition

A

(n)

withrespe t tomeasure

µ

as:

H(A

(n)

) =

X

i

1

, ...,i

n

µ(A

i

1

∩ . . . ∩ f

1−n

A

i

n

) log µ(A

i

1

∩ . . . ∩ f

1−n

A

i

n

) ;

where we sum overevery element of

A

(n)

. The rate of information reation,

withrespe ttotheinitialpartition

A

isthen givenbythelimit:

h

_{(µ, A) = lim}

n→∞



H(A

(n+1)

) − H(A

(n)

)



= lim

n→∞

 1

n

H(A

(n)

₎



.

TheShannon-Ma Millantheoremguaranteestheexisten eofthis limit.

The Kolgomorov-Sinai entropy

h

(µ)

is then dened as the further limit of

h

_{(µ, A)}

fornerandnerstartingpartitions

A

.

As stated at the beginning of this se tion, the information reation rate is

originatedbytheexpandingmotionofthesysteminthephasespa e, onne ted

withits haoti behavior. In1978Ruelledemonstratedthatitsvalue annotbe

greaterthanthetotalpositiveexpansionrateofthesystem,givenbythesum

overallpositiveLyapunovglobalexponents:

h

_{(µ) ≤}

X

λ

(i)

_>0

λ

(i)

.

(1.15)

Pesin extended this theorem, proving that (1.15) is an identity if (and only

if) the measure

µ

is aSRB measure. Weshall briey dene and dis uss SRB measures at the end of the next se tion, sin e they result ru ial also in the

1.7 Information Dimension of the Attra tor

As mentioned at the beginning of this hapter, (see pag 17) the attra torin

the phase spa e

A ⊆ M

has, in general, afra tal stru ture. We assume the notionofHausdordimensionasgiven,withthenotationdim

H

,anddenethe informationdimension ofthemeasure

µ

as:

dim

H

(µ) = inf{

dim

H

(S)|µ(S) = 1} .

Young's theorem (1982)shows that, if

µ

is an ergodi measure, this value is equivalentto: dim H

(µ) = lim

r→0

log µ(B

x

(r))

log r

.

Here

B

x

(r)

representsthe ball enteredin

x

of diameter

r

. The expression is validand onstantforevery

x

∈ A

ex ept,possibly,forasetof

0 µ

-measure.

Now, if

λ

(1)

_{, . . . , λ}

(m)

are the global Lyapunov exponents asso iated to

µ

, and

k = max{ i | λ

(1)

_{+ . . . + λ}

(i)

_{≥ 0}}

,we andenetheLyapunovdimension

as:

dim

Λ

(µ) = k +

λ

(1)

_{+ . . . + λ}

(k)

|λ

(k+1)

_|

;

(1.16)

these ondtermisasmall,noninteger orre tionforthe ase:

P

k

i=1

λ

(i)

> 0

and

P

K+1

i=1

λ

(i)

< 0

. Inoursystem,duetogreatdimensionality,itresultsnegligible. The onne tionbetween thetwo quantities dened hereis given by the

fol-lowing onje ture byKaplanandYorke: if

µ

isanergodi ,SRBmeasure,then dim

H

(µ) =

dim

Λ

(µ) .

(1.17)

It is analyti ally proved that this equality holds in some spe i ases, but

ex eptionsarefound.

SRB measures and hyperboli ity

Intheprevioustwose tionswestatedthata ru ialpropertyofthemeasurewe

use,bothfortheexa t al ulationofentropyprodu tionandareasonableesteem

oftheattra tordimension,isbeingaSRBmeasure(fromSinai,Ruelle,Bowen);

namely ameasure whi h is absolutely ontinuous along unstablemanifolds. A

rigorousdenition anbefoundin[10,11℄.

Itis provedthat fora lassof dynami alsystems,namelythe Axiom-A

sys-tems,existsauniqueSRBmeasure,whi h anbeexpressedphysi ally asthe

ergodi average:

ρ = lim

n→∞

1

n

n−1

X

k=0

δ

f

k

_x

.

The problemis then transferred ondemonstrating that the dynami alsystem

isAxiom-A:in that asetheSRBmeasurenaturally orrespondstoanaverage

overlongdynami al traje tories, asthe ergodi prin iplestates; onsequently

PesinidentityholdsandKaplan-Yorke onje tureisonsolidground.

The ru ialpropertyofAxiom-Asystemishyperboli ity. Aset

A

ishyperboli foradieomorphism

f

(mappingof ow), if

∀x ∈ A

thereexists adire t sum

de omposition of thetangentspa e betweenstable (expandingovertime)and

unstabledire tions(expandingoninverted time). If

A

isahyperboli attra ting setandtheperiodi pointsof

f

aredensein

A

,the onditionsforanAxiom-A dieomorphism are satised. In ase thewhole manifold

M

is hyperboli ,we havethestronger onditionsofAnisovdieomorphism andstru tural stability.

ProvingthatasystemisAxiom-Ais, in general,averyhard task. However,

thepresentworkrepresentsanexampleofhowLyapunovVe tors anbeused

toidentify theexpandingand ontra tingdire tionsofthetangentspa e. The

measure of their transversality represents then a quantitative esteem of the

Model

Inthis hapterabasi modelforastati ,larges alenetworkof orti alneurons

is built. Starting form the Hodgkin-Huxley lassi equations, simpler models

areinferred. Itisthenexplainedwhythe anoni alquadrati integrateandre

model(QIF)isasuitabletodes ribethedynami sofverylarges alenetworks.

Inthenext se tionamoregeneralmodelfor pulse- oupled neuronalnetworks

ispresented. Expli it equationsareobtainedfromthe QIFmodeland usedto

al ulateanalyti allydierentglobalparametersofthenetwork;thenaformula

fortheJa obianatanygivenspiketimeispresented. Su hanalyti alexpressions

will be the basis of the a tual network simulation and of all the subsequent

resultspresentedin next hapters.

2.1 Single Neuron Dynami s

Hodgkin and Huxley model

Thebestknownandmostwidelya eptedequationsusedtodes ribethe

poten-tialofaneural ellsoma asafun tionofexternal urrentandinternal

ondu -tan e parameters,dates ba kto the pionieristi work of Hodgkin and Huxley

[24℄. Theequationsintheirstandardformare:

C

dV

dt

= I − g

L

(V − E

L

) − g

Na

m

3

_{h(V − E}

Na

) − g

K

n

4

_{(V − E}

K

) ;

dm

dt

=

1

τ

m

(V )

(m

∞

(V ) − m) ;

dh

dt

=

1

τ

h

(V )

(h

∞

(V ) − h) ;

dn

dt

=

1

τ

n

(V )

(n

∞

(V ) − n) ;

(2.1) with

m

∞

(V ) = α

m

(V )/(α

m

(V ) + β

m

(V )) , τ

m

(V ) = 1/(α

m

(V ) + β

m

(V )) ,

h

∞

(V ) = α

h

(V )/(α

h

(V ) + β

h

(V )) ,

τ

h

(V ) = 1/(α

h

(V ) + β

h

(V )) ,

n

∞

(V ) = α

n

(V )/(α

n

(V ) + β

n

(V )) ,

τ

n

(V ) = 1/(α

n

(V ) + β

n

(V )) .