from rats of the ﬁctive respiratory output in a brainstem–spinal cordpreparation Error signals as powerful stimuli for the operant conditioning-likeprocess Behavioural Brain Research

(1)

ContentslistsavailableatScienceDirect

Behavioural Brain Research

j o ur na l h o me p a g e :w w w . e l s e v i e r . c o m / l o c a t e / b b r

Research report

Error signals as powerful stimuli for the operant conditioning-like process of the ﬁctive respiratory output in a brainstem–spinal cord preparation from rats

Alessandro Formenti

^∗

, Luciano Zocchi

DEPT—DepartmentofPathophysiologyandTransplantation,UniversityofMilan,ViaMangiagalli,32,20133Milano,Italy

h i g h l i g h t s

•Proprioceptiveinputinducedoperantconditioning-likeprocessonrespiratoryactivityinvitro.

•Afferencescontingentonrespiratoryburstsincreasetheirfrequencyandamplitude.

•Longandshorttermeffectsaredescribed.

•Diaphragmaticproprioceptiveinputsrepresentunconditionedstimuliwithhedonicfeatures.

•Aforwardmodelisassumedtointerprettheresults.

a r t i c l e i n f o

Articlehistory:

Received10March2014

Receivedinrevisedform19June2014 Accepted23June2014

Availableonline28June2014

Keywords:

Operantconditioning-likeprocess Fictiverespiratoryoutput Corollarydischarge Sensoryfeedback Learning

a b s t r a c t

Respiratoryneuromuscularactivityneedstoadapttophysiologicandpathologicconditions.Westudied theconditioningeffectsofsensoryfiber(putativeIaandIItypefromneuromuscularspindles)stim- ulationonthefictiverespiratoryoutputtothediaphragm,recordedfromC4phrenicventralroot,of in-vitrobrainstem–spinalcordpreparationsfromrats.Therespiratoryburstfrequencyintheseprepara- tionsdecreasedgradually(from0.26±0.02to0.09±0.003bursts⁻¹±SEM)astheageofthedonorrats increasedfromzeroto4days.ThefrequencygreatlyincreasedwhenthepHofthebathwaslowered, andwassignificantlyreducedbyamiloride.

C4lowthreshold,sensoryﬁberstimulation,mimickingastretchedmuscle,inducedashort-termfacil- itationofthephrenicoutputincreasingburstamplitudeandfrequency.Whenthesamestimuluswas appliedcontingentlyonthemotorbursts,inanoperantconditioningparadigm(a500mspulsetrain withadelayof700msfromthebeginningoftheburst)astrongandpersistent(>1h)increaseinburst frequencywasobserved(from0.10±0.007to0.20±0.018bursts⁻¹).Conversely,withrandomstimu- lationburstfrequencyincreasedonlyslightlyanddeclinedagainwithinminutestocontrollevelsafter stoppingstimulation.

A forward modelis assumed tointerpretthe data,and the notionof errorsignal, i.e. thesensory ﬁberactivation indicating anunexpected stretched muscle, is re-considered in terms ofthe reward/punishmentvalue.Thesignal,gaininghedonicvalue,isreviewedasapowerfulunconditioned stimulussuitableinestablishingalong-termoperantconditioning-likeprocess.

∗ Correspondingauthor.Tel.:+390250315464;fax:+390250315455.

E-mailaddress:Alessandro.Formenti@unimi.it(A.Formenti).

1. Introduction

1.1. Learningisaprocessthatproducesadaptivechangesinthe organism

Operantorinstrumentalconditioning[1]hasbeenproposedas afundamentaltypeofassociativelearning¹inwhichitisassumed

1Herethe term“associative learning”refers to theempirical phenomenon thatanimalsadapttheirbehaviortothepresenceofsigniﬁcanteventsinthe http://dx.doi.org/10.1016/j.bbr.2014.06.038

(2)

laughingorcrying[4].Whilesomeefforthasbeenmadetoclarify themechanismsthatunderlieoperantconditioningoftherespi- ratoryrhythminsimpleanimalmodels[5]littleisknownabout thisfunctioninmammals.Intheratarhythmicelectricalactivity recordedfromtherootofthefourthcervicalmotornerve(abranch ofthephrenic)hasalwaysbeendescribedcomprehensively,asthe motoroutputtothediaphragm[6].Thefunctionofthismotorout- putistocontrolrespiratorymovements.Changesinmotoroutput directlyaffectventilation.Therespiratory-relatedburstsrecorded fromthe4thcervicalmotorspinal nerve(C4)appeartobecor- related withan increase ofactivity in thepreBötCnucleus [7].

Thisnucleusishypothesizedtogeneratetheinspiratoryrhythm [8,9]drivenbyasubsetofneuronsexhibitingburstingorpace- makerproperties[10,11].Therespiratoryrhythmismodulatedby descendingandascendingsignals[4].Extracellularrecordingsof neuronalpopulationswithinthepre-BotCofperinatalmedullary slice preparations from rats demonstrated that the rhythmical respiratorydischargecommencesapproximatelyatembryonicday E17[12].

1.3. Thebrainstem–spinalcordpreparationinvitro

Thebrainstem–spinalcordpreparationfromtheratisawell- knownmodelandhasrecentlybeenreviewed[13].Itseemsvery promisinginstudyingtheoperantconditioning-likeprocessinthe mammaliannervous system.In fact, in addition tothe general advantagesofferedbyaninvitropreparation,suchaseasyaccess toelectrophysiologicaltechniques,andalmostfullbiochemicaland pharmacologicalcontroloftheextracellularmediumbypassingthe blood-brainbarrier,thispreparationalsoretainsarhythmicburst dischargeidentiﬁedasﬁctiverespiratoryactivityonthephrenic motorroots.Thisputativerespiratorybehaviorisquitewellchar- acterizedandrepresentsthemotorcommandtothediaphragm.

In addition, sensory input from the periphery can be simu- latedbyactivatingthedorsalrootstumpsbymeansofelectrical stimulation.

1.4. Maythespontaneousﬁctiverespiratoryactivitybechanged byoperantconditioninginvitro?

We addressed the question if the plasticity necessary for operant conditioning of respiratory circuits was retained in brainstem–spinal cordpreparations. This in vitro model seems particularly interesting since it permits the exclusion of the unavoidableeffects derivedfromthestimulationoftheperiph- eralstructures(e.g.musclefatigue,receptorhabituationetc.)that

environment,andconsidersbehavioralandphysiologicalchangesastheultimate criteriontodeterminewhetherlearninghastakenplace.

latedthattheerrorsignal,i.e.thedifferencebetweentheexpected andtheactualsituation,willcauseanadjustmentofmovement.

Assumingthattheafferencesfromtheproprioceptorsrepresent anerrorsignaltotheCNS,indicatinganunexpectedlengthand tension of the muscles, we can hypothesize that the operant conditioningparadigmandtheforwardmodelmaycoexistinone coherentandmoregeneraltheoryhelpfulinthisspeciﬁccontext tobetter understandthepathophysiology ofrespiratory neural control.

2. Methods 2.1. Ethicalapproval

All experiments were carried out in accordance with the acceptedstandardsofhumaneanimalcareundertheguidelines oftheethiccommitteeoftheUniversityofMilan.

2.2. Invitroisolatedbrainstem–spinalcordpreparation

The brainstem–spinal cord was prepared, with someminor changes, using themethods already described in theliterature [6]. Sprague-Dawley rats 0–4 days (total: n=43) were deeply anaesthetizedbymeansofequitensine(4ml/kg)intraperitoneum, dissected,andimmediatelyplacedintoanartiﬁcialcerebrospinal ﬂuid(aCSF)at4^◦Cwithhigh[Mg²⁺]todecreaseneuronalexcitabil- ityduringpooroxygenatingconditions.ThecompositioninmM was:MgSO42,NaHCO326,NaH2PO41.25,KCl3,Glucose30,and Sucrose210.Therostralend oftheenblocpreparationwascut atthelevelofthecaudalcerebellarartery.Oncemeningeswere removed,thebrainstem–spinalcordpreparation(Fig.1A)wasput intoarecordingchambermadeofsiliconeelastomer,andperfused attherateof6ml/minwithasecondaCSFrecordingsolution.The compositioninmMoftheaCSFrecordingsolutionwas:NaCl130, KCl8,CaCl21.3,MgSO41.3,NaH2PO40.58,NaHCO325,andGlucose 30.Theperfusionsolutionwascontinuouslybubbledwithacarbo- genmixture95%O₂and5%CO₂.Temperaturewasmaintainedat 27^◦C±1andpHat7.4unlessotherwiseindicated.Allsubstances weresuppliedfromSigma.

2.3. Invitroelectrophysiology

Electricalrecordingsandstimulationswereperformedutilizing ﬁrepolishedborosilicatemicropipettes(tipdiameter100–150␮m) placedonhomolateralventralanddorsalroots,asindicatedindia- gramsinFigs.1and2.Signalswereampliﬁed,digitized,recorded onapcusingaDigidata1200(AxonInstruments)andanalyzeduti- lizingthesoftwarepClamp(AxonInstruments).Stimulationwas deliveredbymeansofaSD9stimulator(Grass)andtimedutilizing aPulsmasterA300(WPI).

(3)

Fig.1. Invitronewborn-ratbrainstem–spinalcordpreparationspresentedafictiverespiratoryactivitythatwasmodulatedvaryingphysiologicalandpharmacological conditions.(A)Schematicrepresentationofthebrainstem–spinalcordpreparationfromaventralviewindicatingthesiteofplacementoftheventralrootelectrode.Onthe rightatypicalrecordingfromtheIVcervicalanteriorrootisshown.Intheuppertraceaburstisexpandedonthetimescale.Themeanfrequencyofburstdischargewasa functionoftheageoftherat.Thebarchartin(B)showsthemeanfrequencyofburstdischargeat27±1^◦Cfrom43brainstem–spinalcordpreparationsdissectedbetween postnatal(P)days0to4andrecordedincontrolconditions.(C)ApHvariationfrom7.4to7.1modulatedthemotoroutputrecordedfromrootC4nearlydoublingtheratein P3ratpreparations(n=4)asexpected.(D)Amiloride(0.5mM)reducedtheburstrate.Theinhibitoryeffectofamiloridewascompletelyremoveduponwashing.Thisgraph referstothedatarecordedfromrepeatedperfusionsandwashingsfrom3P0-onlyratpreparations.Inallpanels,Cstandsforcontrol,andRfortherecoveryafterremovingthe drugfromthebath.Eachbarrepresentsthemean(verticallines=SEM).Asterisksindicateasignificantdifference:^***P<0.0001;nsmeansnosignificantdifference:P>0.05.

A recording electrode on the phrenic motor root was uti- lizedtomonitorthespontaneous fictiverespiratory activity. In theexperimentsinwhichelectricalstimulationwasutilized,the stimulusvoltagewasadjustedtoelicitonlytheactivationoflow threshold,largediameter,sensoryfibers(see also[14–16]).The effectivenessofstimuliwasevaluatedbyrecordingboththefiber dischargefromthesameactivatedsensorynervestumpandthe relatedHreflexonthehomologousventralroot(Fig.2,diagram above). The threshold of the H reflex (T) was determined and thepulsevoltagewasdefinedas3timesthethresholdpotential.

Theexperimentswerecarriedoutexclusivelyonbrainstem–spinal cordpreparationsthatpresentedastableﬁctiverespiratoryactiv- ity and H reﬂex from the fourth cervical (C4) ventral root (Figs.1Aand2A).

Dataisexpressedasmean±SEM.Statisticalanalysiswasper- formedbymeansof theGraphPadPrism4 program(GraphPad Software,Inc.)usingone-wayanalysisofvariance(ANOVA)with Bonferroni’s post-test. Fitting analysis with exponential equa- tions was conducted with the Clampﬁt 10 (MDS Analytical Technologies).

3. Results

3.1. Brainstem–spinalcordviabilityandcharacterization

Some preliminary experiments were carried out in order to check the functional integrity of the model. At the begin- ningof each experimental session thefrequency of the ﬁctive

Fig.2. Lowthreshold,largediameter,fiberactivationinducedanincreaseinphrenicburstfrequency.(A)ontheleftadiagramofatransversesectionofthespinalcordatthe fourthcervicalneuromere(C4)isshown,indicatingthesitesofplacementofthedorsalrootstimulatingandrecordingelectrodes,andtherecordingelectrodeonthemotor rootofthephrenicnerve.Ontheright,theuppertraceindicatesatypicaldorsalrootrecordingoftheafferentvolleyinducedbymeansofalowvoltagepulseonthesame rootstump(blacktriangle);theamplitudeoftheafferentvolleywasconstantduringthewholeexperiment.ThelowertraceindicatestheHreflexfromtheventralroot.The shadedareaofthevoltageoscillationswithin20msafterthedorsalstimuluswasdeterminedasameasureoftheamountofreflexvs.thestimulusintensity.(B)Histogram barchartindicatingthattheburstfrequencyincontrolconditions(C)increasedduringelectricalstimulationofthedorsalrootatlevelC4,utilizing500mstrainsofvoltage pulses50␮seach,repeatedevery50ms.Thepulsetrainswererepeated,onepulsetrainevery3minfor5repetitions.Theburstfrequencydroppedtocontrollevelsduring therecovery,attheendofstimulation(R)(n=6).Eachbarrepresentsthemean(verticallines=SEM).Asterisksindicateasignificantdifference:^***P<0.0001;nsmeansno significantdifference:P>0.05.

(4)

generationoftherhythmicdischargeinotherpacemakercellsin theCNS,wasused.Amiloride(0.5mM)whenaddedtotheaCSF recording solutionreduced the burstfrequency (6 rat preparations; P 0 to 4).All the effects were completely reversed in a fewminutesafterwashingoutthedrug.Fig.1Dshowsthedata recorded during repeated perfusions and washings, from 3 P0 preparations.

3.2. Fourthcervicaldorsalrootstimulation

Inaseriesoftenexperimentsthepossibilitytomodulatethe motoroutputfromtheC4nervebymeansofstimulationoflow thresholdsensoryfibersofthehomologousandhomolateralposte- riorroot(putativesensoryfibersfromthephrenicproprioceptors) wasevaluated.Trainsofstimuliof500msdurationweredelivered ontothedorsalroot.Withinthetrain,50␮svoltagepulseswere repeatedevery50ms.Thepulsevoltagewaspreviouslydetermined asnomorethan3timesthethresholdvoltagenecessarytoevoke theH-reflexonthehomologousventralroot.Atthisvoltage,the monosynapticventralreflexachievedabout95%ofitsmaximum intensity(whichwasevaluatedmeasuringtheareaofthevolt- ageoscillationwithin20msafterthedorsalstimulus;Fig.2A)and, onthedorsalroot,onlythepeakindicatingtheactivationoflow threshold,largediameter,sensoryfibers,putativeIandpossiblyII fibers,waspresent(Fig.2A).Utilizingthisstimulationparadigm,a shorttermfacilitationofthephrenicmotorneuronswasobserved.

Since,itisreasonabletoassumethattheintegratedburstampli- tudeis proportional tothephrenicnerve activityand thetidal volume[17],insomeexperimentstheareaofthebursts(i.e.the areadefinedbythepotentialprofileand thebaseline;Vs)was measuredincontrolledconditions,andafterlowthreshold,fiber activation.Thestimulationofthefibersoflargediameter,inside thedorsalroot,increasedthephrenicburstdischarge.Thefacili- tationlastedonlyforafewsecondswithasignificanteffectonly evidentonthefirstburst.Then,thedischargedroppedtocontrol levelsasshowninFig.3.Inanotherseriesofexperiments,pulse trainswererepeated,onepulseevery3minforatotaloffiverepe- titions.Thelargediameter,lowthreshold,fiberactivationinduceda strongincreaseinburstfrequencyimmediatelyafterthefirstpulse train,comparedtothatofcontrol.However,attimes,itincreased slightlyfollowingthesecondandthirdpulsetrains,butthentended tostabilize.Thehighburstfrequencypersistedforabout10–15min aftertheendofstimulation,thenreturnedtocontrollevels(Fig.2B).

3.3. Operantconditioning

Inordertoexaminehowsensoryinputs,eithercontingentor non-contingentontheburstsinC4ventralrootsmayaffecttheir rhythmofdischarge,anoperantconditioningprotocolwasdeﬁned fortheﬁctiverespiratoryactivity.Toallowabettercomparison

Fig.3.Lowthreshold,largediameter,ﬁberactivationinducedashorttermfacilita- tionofthephrenicmotorneurons.TheupperdiagramshowsarecordingfromtheIV cervicalanteriorrootbeforeandafterstimulationofthehomologousdorsalroot.

FortheX-axisrefertothelowergraph,thestimulationisindicatedbyblacktri- angle.Thestimulusartifacthasbeenblankedoutforclarity.Thestimulusprotocol consistedofa500mstrainofpulses.Theareaofthebursts[Vs]i.e.theareadefined bythepositiveandnegativepotentialoscillationsandthebaseline,comprisedina arbitrarilyfixedtimeintervalof1shasbeenmeasuredincontrolconditionsand aftersensoryfiberactivation.Thenormalizedvalues(verticalbars)areplottedon thegraphbelow.Onthesamegraph,thebasaldischargerecordedduringtheinter- burstintervalshasbeenplottedforcomparison(filledsquares).Thestimulation inducedatransientincreaseintheareaoftheburststhatreturnedtocontrolvalues withinafewseconds.Thedecayoffacilitationhasbeenfittedwithamonotonic functionwithatimeconstant=3.3±0.62sSE(thinline).

between the effects of contingent and non-contingent operant stimulationitwasnecessarytominimizethedifferencesinspon- taneousfictiverespirationfrequency.Therefore,theexperiments werecarriedoutonpreparationsfromratsdividedintotwosetsfir- ingalmostatthesamefrequency.Tomeetthesecriteria,theywere chosenfromthesamepostnatalday(P3),andhadbeenselected afteraperiodofcontrolduringwhichthefrequencyofburstdis- chargewasassessed.Afterthat,thetrainingperiodfollowedonthe firstsetofpreparationsandthecontingentprotocolwasapplied.

A500mspulsetrainactivatinglowthresholdﬁbersinC4dorsal rootwasseteverytimethataburstwasdetectedinthehomolo- gousventralroot.Thetrainwassetwithanintervalof700msfrom thebeginningofeachburst.Theprotocolwasrepeatedforfour sessions,eachof200s(separatedbyintervalsofabout4min).This protocolofstimulationinducedastrongincreaseinburstdischarge and,unliketheexperimentspresentedinFig.2B,thefrequencyof dischargedidnotreturntocontrollevelsafterstoppingstimula- tion,evenaftermorethan1h(Fig.4A).Toevaluatethecontribution ofoperant-stimulationcontingencyindeterminingthelongterm effects observed,anotherexperimentwascarriedout,this time onthesecondsetofpreparations,inwhichthesamenumberof stimuliwasset,however,withoutcontingency.Inthissecondset ofexperiments,theburstfrequencyincreasedonlymoderately,and returnedtocontrollevelswithinafewminutesafterstimulation wasended(Fig.4B).

(5)

Fig.4. Thefictiverespiratoryactivitywasstudiedbymeansofanoperantconditioningparadigmtotesthowsensoryinputs,bothcontingentandnon-contingent,onthe burstsdetectedinphrenicmotorrootmayaffectburstfrequency.(A)Theupperdiagramshowstheexperimentalprotocol,inwhichcontingenttrainstimuli(triangles) activatingIaandIIfibersinC4dorsalrootwereseteverytimethataburstinthehomologousventralroot(verticalbarsinthetraceabove)wasdetected.Thebarchartbelow summarizestheresultsoftheseexperiments.Theprotocolwasrepeatedin4sessionsof200sdurationeach,andseparatedbyintervalsofabout4min.Thisprotocolof stimulationinducedalongtermincreaseinburstdischargethatpersistedfortheentiredurationofobservation,whichwasmorethan1hour(n=10).(B)Controlexperiments, inwhichbrainstem–spinalcordpreparationsofthesamepostnatalday(P3)andalsothesamefictiverespiratoryfrequencyasthoseutilizedinA,weresubjectedtothe samenumberoftrainstimulisetatregularintervalsoftimeandnon-contingentontheburstsrecordedfromthephrenicroot.Stimulationonlymoderatelyincreasedthe burstfrequencyandtheeffectpersistedforonlyafewminutesaftercessationofstimulationindicatingthatcontingencywasessentialinconsolidatingthememorytrace (n=9).Eachbarrepresentsthemean(verticallines=SEM).Theasteriskindicatesasignificantdifference,^*P<0.05;^***indicatesP<0.0001;nsmeansnosignificantdifference P>0.05.

4. Discussion

4.1. Thebrainstem-spinalcordpreparationisasuitableinvitro modelforthestudyofrespiratoryrhythmconditioning

Brainstem–spinal cord preparation from newborn rats was introducedasanexperimentalmodeltostudyfictiverespiratory activityalmost30yearsago[6].Thecharacterizationoftheprepa- rationsshowedparametersinagreementwiththosereportedinthe literature.Therelativelylowfrequencyoftherespiratorybursts, comparedtothefrequencyofventilationinvivoinnewbornrats (1.5Hz),maybeexplainedbythelackofsensoryinput,asalready suggested[6,18]andconfirmedfromthedatapresentedhere.The lackofdescendingexcitatorydrivetorespiratorynucleimustalso betakenintoaccount[4,19].Therespiratorycontrolnetworkin rapiddevelopment, withits ongoing myelination, couldbethe main reason for the striking age-dependent decrease in motor burstfrequencyshowninFig.1B.Infact,anincreaseinmyelin- atedfibersandaparalleldecreaseinunmyelinatedfibershasbeen described in thephrenic nerve of the rat between P0 and P4, withlargeaxonsthatmyelinateearliercomparedtosmalleraxons [20,21].

As expected, the ﬁctive respiratory frequency promptly respondedwhenthepHoftheaCSFwasaltered(Fig.1C)[6,22]

conﬁrmingtheviabilityofthepreparations.

Thefictiverespiratoryfrequencywassignificantlyreducedby thediureticamiloride (Fig.1D), a drugknown asa nonspecific blockerofacid-sensingionchannelsASIC[23–25]andoflowvolt- ageactivatedcalciumcurrentsI_CaLVA[26–28].Bothchannelsexert anexcitatory action. Thus, we should expect that theblocking effectsofamiloridewouldreducetheburstingactivityofthepace- makerneuronsandthefictiverespiratoryfrequency.Inparticular, I_CaLVA,whichisfoundinpreBötCneuronsfromnewbornrats[29]

andknownalreadytocontributetotheburstingpacemakeractivity inotherpartsofthecentralnervoussystem[30,31],mayplayarole inburstingpacemakerneurons,byinﬂuencingtheﬁctiverespira- toryoutputatthisearlystageinsynergywithotherrhythmogenic

mechanisms[10,32,33]. However,otherdrugactions cannotbe excluded.

4.2. Therespiratorycircuitsinthepreparationretainplastic properties

The data presented suggests that the activation of the low thresholdafferent ﬁbers inthe dorsalC4 stumpincreasesven- tilation through both thenumber and amplitudeof inspiratory acts, although thelattereffect is verybriefcompared withthe former one. On the whole, these results show that the neces- saryplasticity,foroperantconditioningprotocolstobeeffectivein shapingthefunctionsofrespiratorycircuits,isretainedininvitro brainstem–spinalcordpreparation.Theyconﬁrmthatthestruc- turesabovethefacialmotornucleusandtheparafacialrespiratory group are not necessary to establish an operant conditioning- likeprocessinthespinalcord[34]andextendthisnotiontothe circuitsresponsibleforrespiratorycontrol.Moreover,thisprepa- ration,excludinganyperipheralstructure(e.g.muscles,receptors, etc.)allowsthestudyoftheneuralmechanismofconditioningand excludestheunavoidableeffectsofstimulationofthesestructures thattypicallyaffectwholeanimalexperiments.

IfthepreBötCnucleus generatestheinspiratory rhythm[8], thenwecaninferthatsensoryfiberactivationup-regulatesthe pacemakeractivityofthepreBötCnucleus,andviceversa,inagree- mentwiththeobservationsthatbilateralsensorydeafferentation andvagotomystronglyreducedtherespiratoryfrequencyinthe rat [18]. Themethodadopted todefine theintensityof stimu- lationisbasedonboth themonitoringoftheafferentvolleyon thedorsalroot, andontheventralmotor outputwhich hadto belimitedtotheH-reflexwithoutanyevidentpolysynapticactiv- ity(Fig.2A).Thisguaranteeswithreasonableconfidencethatthe effectsobservedwerederivedfromthespecificstimulationoflow thresholdfibers,putativeproprioceptivetypeIandpossiblyII.On theotherhand,anincreaseinvoltageofthestimulationtorecruit medium threshold sensory fibers induced the opposite effect

(6)

output.

Direct proprioceptive stimulation of the limbs and the low thresholdﬁberactivationincervicaldorsalrootshavebeenindi- catedtoinducesynchronizationoftherespiratoryrhythminthe newborn rat [37]. Also, this mechanism does not seem to be involvedinthephenomenashownhereforatleasttworeasons.

Oneisthatlimbmovementsorstimulationofthelimbpropriocep- tiveafferentsinducealocomotor–respiratorycouplingbutnotan increaseinrespiratoryfrequencyinnewbornrats.Thesecond,even morestringent,isthattheeffectsinducedbylimbproprioception ontherespiratoryrhythmaremediatedbythepontinePB/KFnuclei [37],butthesenucleiwerenotpresentinthebrainstem–spinalcord preparation.Inaddition,theobservationthatthephysicalremoval oftheponsincreasedtherespiratoryfrequencyisquiteinteresting [20].

Ontheotherhand,it seemsunlikely thatthediaphragmatic Golgitendonorgan afferents[21,38,39],areresponsible forthe increase in burst amplitudeor frequency. As emphasized by a viewpoint,theactivationoftheseﬁbershasinhibitoryeffectson themotorneuronsinnervating thesamemuscle.Thus, theydo notseemgoodcandidatestoexplain theexcitatoryphenomena describedhere.However,thisviewhasbeenquestioned,asstated below.

Although the role of muscle spindle afferents from the diaphragmhasbeencontroversial,intheliteraturethereisdata showingthatduringrespirationatleastpartofthediaphragmatic musclespindlesfireinphasewiththeinspiratorycontractionof thediaphragm, and intense firingis present also duringcough reflexes. These observations configure the interesting possibil- ityofaregulatorymechanismofthediaphragmaticcontractions throughthefusimotoractivation[39].Thismechanismmayalso extend its influence even onthe Golgi tendonorgans. Indeed, two important factsmust betaken into account.(1) The Golgi tendon afferentsfrom one muscle contribute tothe functional controlof musclegroups as a wholeand,vice versa, theaffer- entinformationforwardedtoindividualmotorneuronsisaffected bythelength andtension ofmany muscles. Thisis incontrast withtheviewthatthefunctionofGolgitendonorgansistocon- tributetoafeed-backsystemcontrollingthehomonymousmuscle, since muscles are not normally activated in isolation and the afferentsfromseveralmuscles arefusedtogetherbeforereach- ingspecificmotorneurons[40,41].(2)Althoughtheseorgansdo nothaveaspecificcentrifugalsystemthatcontroltheirabilityto respond,theyarefrequentlyanatomicallyassociatedwithmus- clespindles toform dyads,inparallel [42,43]orin series with Golgireceptorsattachedtointrafusalfibers[43,44].Interestingly, ithasbeensuggestedthat,fortheircloseassociation,theywork togetherunder the fusimotor control, sensing both themuscle lengthand tension[41,45]. Iftheinhibitory actionof theGolgi tendon organs from one muscle act onother muscles, namely

Fig.5.Schemeforapossibleinterpretationofthemainexperimentalresults.Theupper whitesquarerepresentsthebrainstem–spinalcordpreparationandtheexperimen- tallayout,whiletheovalbelowshowsthemissingperipheralstructures,i.e.amuscle withamusclespindle.Assumingthattheoutputrecordedfromtheventralrootrep- resentsboththemotorcommandfortherespiratorymuscles(␣motorneurons)and thepredictionoftheshorteningofthemusclesthemselvessenttoneuromuscular spindles(␥motorneurons),itfollowsthatthestimulationoflargenerveﬁbersin thehomologousdorsalrootrepresentsthesignalfromthespindlesthatindicates thediscrepancybetweenthedesiredandtheactualshortening,i.e.theerror.This signal,conveyedtothecentralnervoussystem,changestheexcitabilitythelow␣ motorneurons,whichmayaffecttherespiratoryburstintensity,andthepacemaker whichinturnadjuststheburstfrequency.

theantagonists, and thesereceptors workunder thefusimotor control,than,theymayactcoherentlywiththemusclespindles inaneuralnetworkorganizingthepatternofactivityinmuscle regions.

4.4. Atheoreticalframeforinterpretingtheexperimentalresults:

operantconditioningvs.forwardmodel

A scheme for a possible interpretation of the main experimental resultsis showninFig.5.Theinspiratoryresponsesare mediated by both a direct effect on the ␣ motorneurons, and alsobya low thresholdmechanism,representedby the␥neu- ronsandthespindleloop.Whilethe␣motorneuronsinducedirect contractionofmuscles,thespindleloopactivationdeterminesthe magnitudeofthe␣motorneuronsactivation[46]inordertoobtain thenecessarytensiontoreachdesiredmusclelength.Ifweassume a forward modelsystem [47,48]todescribe thesensory-motor systeminvolvedintherespiratorycontrol,thenthe␥neuronacti- vationmayberegardedastheeffectofa‘corollarydischarge’i.e.

asignalthat,inparallelwiththeefferencesignal,resultsinthe␣ and␥motorneuronsco-activation.Thecorollarydischargeissup- posedtorepresentthepredictionoftherespiratorymovements.If themovementiscorrect,thentheintrafusalandextrafusalmus- cleﬁberswillshortentothesameextent,andnoerrorisdetected.

Theerrorsignalhereshouldberepresentedbyachangein frequency of the discharge of theneuromuscular spindle sensory afferents.Thissignal,inthewholeanimal,isfedbackandconveyed totheCNSthroughthedorsalrootsensorynerves,whereasinthe

(7)

brainstem–spinalcordpreparationthisafferenceandrelatedmus- clespindlesareabsent.However,inthisinvitropreparationitis possibletoartificiallyproduceanerrorsignalactivatingdirectly thespindle afferentsbydirect stimulationofthelow threshold fiberspresentinthedorsalrootstump.Accordingtothemodel, themeaning(salience)ofthissignalisthattheextentoftheinspi- ratorymovementislessthanexpected;i.e.theintrafusalmuscle fibersshortenmorethantheextrafusalfibers,andthisdifference isdetectedbythestretchsensor.

As predicted by the forward model, and in analogy to the

‘length-tension inappropriateness’[49] proposedtoexplain the pathophysiologyofdyspnea(forreviewsee[50]),proprioceptor sensoryﬁberactivationcontingentonburstdischarge,represents anerrorsignalfromthemusclespindlesthat,inthisway,inform theCNSaboutanunexpectedmusclelength.Inthemodeloutlined here,theneuromuscularspindleitselfisthepostulated“difference”

calculator,atthelowesthierarchicallevelofmotorcontrol.Thesig- nalisfedbackontothemotorneurons(Fig.5)tomodifycoherently themotorpattern.Thedatapresentedsuggeststhatadjustments areinvolvedforanincreaseinboththeefferentburststrengthand frequency.

Inaddition,insupportforaforwardmodelhypothesisisthedata obtainedutilizinganoperantconditioningparadigm.Onlywhen proprioceptorsensoryfiberactivationwasinducedcontingenton therespiratorybursts,itwaseffectiveinestablishingalongterm increaseinspontaneousfictiverespiratoryfrequency(>1h).How- ever,thiseffectwasnotobservedwhenthesamenumberofstimuli wasdeliveredrandomly, withouttemporalcorrelationwiththe respiratoryburstdischarge.Confirmingproprioceptorsensoryfiber activationasa strongreinforcementofthepacemakerfunction, this suggeststhat proprioceptoractivationrepresentsa powerful unconditionedstimulussuitable in establishinga long-term operantconditioning-likeprocess (i.e.it acquires thenecessary salience)onlywhencorrelatedwiththerespiratoryrhythm,pro- vidingamechanismbywhichtheforwardmodelmaybeupdated over both short and long term time-scales [47]. On the other hand,nolongtermincreaseinburstareawasobservedinoper- antconditioningexperiments,suggestingthatthiseffectperhaps requiresothernervestructureswhich werenotincludedinthe preparation.

Inoperantconditioning,thereward/punishmentvalueisafun- damentalcharacteristicoftheconditioningstimulus.Pleasureor displeasure depends on the quality of the stimuli and on the internalstateofthesubject.Thesequalitiescorrelatetotheben- eﬁtsordangersofthestimuliwithreferencetothehomeostasis, motivatingusefulbehavior[2].Thus,aquestionarisesaboutthe reward/punishmentvalueofproprioceptorﬁberactivation.Since theinternalstateofthesubjectisnoteasilyknown,acommon assumptionisthatwhetherastimulusisorisnotrewardingor punishingisonlyrecognizedbyitseffectsonthebehavioronwhich thestimulusiscontingent:arewardincreasingtherateandvice versa.

In the present experiments low threshold fiber stimulation, theerrorsignal,contingentontherespiratorybursts, indicating thattheinspiratorymuscleshaveshortenedlessthanexpected, itshouldassumethesalienceofaninsufficientinspiration,likely responsible,atleastinpart,forthesensationofdyspnea[51,52].An apparentcontradictionarisesconsideringtheoperantconditioning paradigm,inwhichanunpleasantstimulus,theerrorsignal,aslow thresholdfiberactivationissupposedtobeinthiscase,isexpected todecreasethelikelihoodofacertainbehavior,namelythebursts.

Areasonableexplanation,thatreconcilesboththeforwardmodel hypothesisandtheoperantconditioningparadigm,maybethatthe proprioceptorsactivationisnotinterpretedbythenervoussys- temasaconsequenceoftheburstperse,butbytheeffectofan unexpectedinsufﬁcientrespiratorycommandthatresultsinapoor

inspiratoryact.Withthisperspectiveinmind,theobservationthat theburstsresultstrengthened,andalsotheirfrequencyincreased, asaresultofanerrorsignal,gainconsistency.

4.5. Conclusions

Inconclusion,thedatapresentedsuggeststhattherhythmic discharge of the respiratory pacemakers undergoes long term adaptationfollowingcontingentproprioceptiveinput.Thisoccur- rence is generally interpreted as operant conditioning,since it presents all the paradigmatic elements of this type of learning.Thisassociative learningmodelseemsofparticularinterest because,inasimpliﬁedcontextinvitro,itofferstherarepossibil- itytostudyanoperantconditioning-likeprocessofaspontaneous behaviorofmedicalrelevanceatahighmammalianphylogenetic level.

Althoughweareaware thatthetheoreticalframesuggested needsfurtherinvestigationstobevalidated,ifweassumethatthe afferencefromtheneuromuscularspindlesmayrepresentanerror signalasithasbeenproposedinaforwardmodelforthecontrol ofmovement,then,wecanhypothesizethattheoperantcondi- tioningparadigmproposedbythebehaviorists,andtheoldand evergreenforwardmodelmaycoexistinonecoherenttheoryhelp- fultobetterunderstandthemechanismsofmotorlearningandthe pathophysiologyofrespiratoryneuralcontrol.

Acknowledgements

WearegratefultoMs.ColletteO’Gradyforhelpinpreparing themanuscript.ThisresearchwassupportedbytheUniversityof Milan,DEPTfund14-13-3009009-8.

References

[1]ThorndikeEL.Aproofofthelawofeffect.Science1933;77:173–5.

[2]CabanacM.Sensorypleasure.QRevBiol1979;54:1–29.

[3]SkinnerBF.Thebehavioroforganisms,anexperimentalanalysis.NewYork, London:D.Appleton-CenturyCompany;1938.

[4]SmithJC,AbdalaAP,BorgmannA,RybakIA,PatonJF.Brainstemrespiratory networks:buildingblocksandmicrocircuits.TrendsNeurosci2013;36:152–62.

[5]Braun MH, Lukowiak K.Intermediateand long-term memoryare differ- entattheneuronallevelinLymnaeastagnalis(L.).NeurobiolLearnMem 2011;96:403–16.

[6]SuzueT.Respiratoryrhythmgenerationintheinvitrobrainstem-spinalcord preparationoftheneonatalrat.JPhysiol1984;354:173–83.

[7]Thoby-BrissonM,GreerJJ.Anatomicalandfunctionaldevelopmentofthepre- Botzingercomplexinprenatalrodents.JApplPhysiol2008;104:1213–9.

[8]FeldmanJL,DelNegroCA.Lookingforinspiration:newperspectivesonrespi- ratoryrhythm.NatRevNeurosci2006;7:232–42.

[9]SmithJC,EllenbergerHH,BallanyiK,RichterDW,FeldmanJL.Pre-Botzinger complex:abrainstemregionthatmaygeneraterespiratoryrhythminmam- mals.Science1991;254:726–9.

[10]KoizumiH,SmithJC.PersistentNa+andK+-dominatedleakcurrentscon- tributetorespiratoryrhythmgenerationinthepre-Botzingercomplexinvitro.

JNeurosci2008;28:1773–85.

[11]St-JohnWM,StornettaRL,GuyenetPG,PatonJF.Locationandpropertiesof respiratoryneuroneswithputativeintrinsicburstingpropertiesintheratin situ.JPhysiol2009;587:3175–88.

[12]PagliardiniS,RenJ,GreerJJ.Ontogenyofthepre-Botzingercomplexinperinatal rats.JNeurosci2003;23:9575–84.

[13]Johnson SM, Turner SM,Huxtable AG, Ben-MabroukF. Isolated invitro brainstem–spinalcordpreparationsremainimportanttoolsinrespiratoryneu- robiology.RespirPhysiolNeurobiol2012;180:1–7.

[14]LundbergA,WinsburyG.Selectiveadequateactivationoflargeafferentsfrom musclespindlesandGolgitendonorgans.ActaPhysiolScand1960;49:155–64.

[15]HuntCC.Relationoffunctiontodiameterinafferentﬁbersofmusclenerves.J GenPhysiol1954;38:117–31.

[16]JackJJB.Somemethods forselectiveactivationofmuscleafferentﬁbres.

In:Studiesinneurophysiology,essaysinhonourofProfessorAKMcIntyre.

Cambridge:CambridgeUniversityPress;1978.p.155–76.

[17]EldridgeFL.Relationshipbetweenphrenicnerveactivityandventilation.AmJ Physiol1971;221:535–43.

[18]MurakoshiT,SuzueT,TamaiS.Apharmacologicalstudyonrespiratoryrhythm intheisolatedbrainstem–spinalcordpreparationofthenewbornrat.BrJ Pharmacol1985;86:95–104.

(8)

[28]FormentiA,ArrigoniE,ManciaM.Low-voltageactivatedcalciumchannelsare differentlyaffectedbynimodipine.NeuroReport1993;5:145–7.

[29]ElsenFP,RamirezJM.Postnataldevelopmentdifferentiallyaffectsvoltage- activated calcium currents in respiratory rhythmic versus nonrhythmic neuronsofthepre-Botzingercomplex.JNeurophysiol2005;94:1423–31.

[30]HuguenardJR,PrinceDA.AnovelT-typecurrentunderliesprolongedCa(2+)- dependentburstﬁringinGABAergicneuronsofratthalamicreticularnucleus.

JNeurosci1992;12:3804–17.

[31]FormentiA,DeSimoniA,ArrigoniE,MartinaM.Changesinextracellular Ca2+canaffectthepatternofdischargeinratthalamicneurons.JPhysiol 2001;535:33–45.

[32]RamirezJM,TrybaAK,PenaF.Pacemakerneuronsandneuronalnetworks:an integrativeview.CurrOpinNeurobiol2004;14:665–74.

[33]PaceRW,MackayDD,FeldmanJL,DelNegroCA.Inspiratoryburstsinthepre- Botzingercomplexdependonacalcium-activatednon-speciﬁccationcurrent linkedtoglutamatereceptorsinneonatalmice.JPhysiol2007;582:113–25.

[34]Grau JW, Crown ED, Ferguson AR, Washburn SN, Hook MA, Miranda RC. Instrumentallearningwithinthespinalcord:underlyingmechanisms and implications for recovery after injury. Behav Cognit Neurosci Rev 2006;5:191–239.

[44]Barker D. The innervation of the muscle-spindle. Q J Microsc Sci 1948;89:143–86.

[45]JankowskaE,McCreaDA.SharedreﬂexpathwaysfromIbtendonorganaffer- entsandIamusclespindleafferentsinthecat.JPhysiol1983;338:99–111.

[46]AndersenP,SearsTA.Medullaryactivationofintercostalfusimotorandalpha motorneurones.JPhysiol1970;209:739–55.

[47]ProskeU,GandeviaSC.Theproprioceptivesenses:theirrolesinsignaling bodyshape,bodypositionandmovement,andmuscleforce. PhysiolRev 2012;92:1651–97.

[48]WolpertDM,GhahramaniZ.Computationalprinciplesofmovementneuro- science.NatNeurosci2000;3Suppl:1212–7.

[49]Campbell EJ, Howell JB. The sensation of breathlessness. Br Med Bull 1963;19:36–40.

[50]O’Donnell DE,Ora J, WebbKA, Laveneziana P,Jensen D.Mechanismsof activity-relateddyspneain pulmonary diseases. RespirPhysiol Neurobiol 2009;167:116–32.

[51]NishinoT.Dyspnoea:underlyingmechanismsandtreatment.BrJAnaesth 2011;106:463–74.

[52]GigliottiF.Mechanismsofdyspneainhealthysubjects.MultidiscipRespirMed 2010;5:195–201.