Cholinergic and dopaminergic effects on prediction error and uncertainty responses during sensory associative learning

(1)

ContentslistsavailableatScienceDirect

NeuroImage

journalhomepage:www.elsevier.com/locate/neuroimage

Cholinergic

and

dopaminergic

eﬀects

on

prediction

error

and

uncertainty

responses

during

sensory

associative

learning

Sandra

Iglesias

a,∗

_,

_Lars

_Kasper

a,b

_,

_Samuel

_J.

_Harrison

a

_,

_Robert

_Manka

c

_,

_Christoph

_Mathys

a,d

_,

Klaas

E. Stephan

a,e

a Translational Neuromodeling Unit (TNU), Institute for Biomedical Engineering, University of Zurich & Swiss Federal Institute of Technology (ETH Zurich), Wilfriedstr. 6, 8032 Zurich, Switzerland

b Institute for Biomedical Engineering, ETH Zurich and University of Zurich, Switzerland c Department of Cardiology, University Hospital Zurich, Switzerland

d Interacting Minds Centre, Aarhus University, Aarhus, Denmark e Max Planck Institute for Metabolism Research, Cologne, Germany

a

r

t

i

c

l

e

i

n

f

o

Keywords: Acetylcholine Dopamine Biperiden Amisulpride Basal forebrain Ventral tegmental area Substantia nigra Neuromodulation fMRI

Hierarchical Gaussian Filter

a

b

s

t

r

a

c

t

Navigatingthe physicalworldrequireslearningprobabilistic associationsbetweensensoryeventsandtheir changeintime(volatility).Bayesianaccountsofthislearningprocessrestonhierarchicalpredictionerrors(PEs) thatareweightedbyestimatesofuncertainty(oritsinverse,precision).InapreviousfMRIstudywefoundthat low-levelprecision-weightedPEsaboutvisualoutcomes(thatupdatebeliefsaboutassociations)activatedthe pu-tativedopaminergicmidbrain;bycontrast,precision-weightedPEsaboutcue-outcomeassociations(thatupdate beliefsaboutvolatility)activatedthecholinergicbasalforebrain.Thesefindingssuggestedselectivedopaminergic andcholinergicinfluencesonprecision-weightedPEsatdifferenthierarchicallevels.

Here,wetestedthishypothesis,repeatingourfMRIstudyunderpharmacologicalmanipulationsinhealthy par-ticipants.Specifically,weperformedtwopharmacologicalfMRIstudieswithabetween-subjectdouble-blind placebo-controlleddesign:study1usedantagonistsofdopaminergic(amisulpride)andmuscarinic(biperiden) receptors,study2usedenhancingdrugsofdopaminergic(levodopa)andcholinergic(galantamine)modulation. Pooled acrossallpharmacologicalconditionsof study1andstudy2,respectively,wefoundthatlow-level precision-weightedPEsactivatedthemidbrainandhigh-levelprecision-weightedPEsthebasalforebrainasin ourpreviousstudy.However,wefoundpharmacologicaleffectsonbrainactivityassociatedwiththese com-putationalquantitiesonlywhensplittingtheprecision-weightedPEsintotheirPEandprecisioncomponents: inabrainstemregionputativelycontainingcholinergic(pedunculopontineandlaterodorsaltegmental)nuclei, biperiden(comparedtoplacebo)enhancedlow-levelPEresponsesandattenuatedhigh-levelPEactivity,while amisulpridereducedhigh-levelPEresponses.Additionally,intheputativedopaminergicmidbrain,galantamine comparedtoplaceboenhancedlow-levelPEresponses(inabody-weightdependentmanner)andamisulpride enhancedhigh-levelprecisionactivity.Taskbehaviourwasnotaffectedbyanyofthedrugs.

Theseresultsdonotsupportourhypothesisofaclear-cutdichotomybetweendiﬀerenthierarchicalinference levelsandneurotransmittersystems,butsuggestamorecomplexinteractionbetweentheseneuromodulatory systemsandhierarchicalBayesianquantities.However,ourpresentresultsmayhavebeenaﬀectedbyconfounds inherenttopharmacologicalfMRI.Wediscusstheseconfoundsandoutlineimprovedexperimentaltestsforthe future.

1. Introduction

Navigatingcomplexphysicalenvironmentsrequiresarepresentation ofprobabilisticassociationsbetweeneventsintheworld.Apopular no-tionincontemporarycomputationalandcognitiveneuroscienceisthat

∗_{Corresponding}_author.

E-mailaddress:iglesias@biomed.ee.ethz.ch(S.Iglesias).

thebrainmastersthischallengebyconstructingandcontinuously updat-inganinternalmodeloftheenvironment(technicallyspeaking,a gen-erativemodelofitssensoryinputs;Dayanetal.,1995;Friston,2005). Thisgenerativemodelthenservestoinferthehiddencausesofcurrent sensationsandpredictfuturesensations.

https://doi.org/10.1016/j.neuroimage.2020.117590

Received6June2020;Receivedinrevisedform20October2020;Accepted19November2020 Availableonline4December2020

(2)

This general “Bayesianbrain” notion is reflected byseveral con-crete concepts, such as predictive coding (Rao and Ballard, 1999; Friston, 2005) or hierarchical filtering (Mathyset al., 2011). In ac-cordancewithlong-standingneuroanatomicalfindings(Fellemanand VanEssen,1991;Hilgetagetal.,2000),theseconceptsassumethatthe brain’smodelhasahierarchicalstructure,providingabasisfor hierar-chicalBayesianinference.Putsimply,thekeyideaisthateachlevelof thehierarchyholdsabelieforpredictionaboutthelevelbelowthatis updatedbyascendingpredictionerrors(PEs).

One established experimental paradigm toprobe hierarchical in-ference is probabilisticassociative learning undervolatility. This in-volvescue-outcomepairswhoseassociationstrengthschangeovertime (Behrensetal.,2007;Iglesiasetal.,2013;Diaconescuetal.,2017).In thissetting,thebrainneedstocomputetwodiﬀerenttypesofPEsfor up-datinghierarchicallycoupledbeliefs:alow-levelPEabouttheoutcome andahigh-levelPEabouttheprobabilityofthatoutcome.Importantly, thesePEsareweightedbyestimates ofuncertainty(orprecision,the inverseofuncertainty)thatmodulatethemagnitudeofbeliefupdates (Mathysetal.,2011;Mathysetal.,2014).

Wehavepreviouslyestablishedanassociativesensorylearningtask undervolatilitywhichistailoredtoanalysisbyahierarchicalBayesian model(Iglesiaset al.,2013).Afunctionalmagnetic resonance imag-ing(fMRI) studyof this task in healthyvolunteers suggested a pos-sible link between diﬀerent precision-weighted PEs and activity in distinct neuromodulatory regions. In brief, we found that low-level precision-weightedPEsaboutsensoryoutcome(𝜀2)activated the

pu-tativedopaminergicmidbrain,whereashigh-levelprecision-weighted PEsabouttheoutcome’sprobability(𝜀3)werereﬂectedbyactivityin

thecholinergicbasalforebrain,ormoreprecisely,inits septal subre-gion(areaCh1-2;Zaborszkyetal.,2008).Asubsequentstudyusinga similarcomputationalmodelreplicatedtheseeﬀectsforsociallearning, withlow-levelPEsactivatingtheputativedopaminergicmidbrainand high-levelPEsactivatingthebasalforebrain(Diaconescuetal.,2017).

Thepossibility thatactivityof twodistinct neuromodulatory sys-temscouldbe probedbya singleneuroimagingparadigm is intrigu-ing,giventheneedforcomputationalassaysofneuromodulationthat couldguidedifferentialdiagnosisandtreatmentpredictionin psychi-atry(Stephanetal.,2006;Stephanetal.,2015).However,theblood oxygenleveldependent(BOLD)signaloffMRImayrepresentamixture ofdifferent neurophysiologicalprocesses,andboththemidbrainand basalforebraincontainavarietyofdifferentneurons(seeIglesiasetal., 2013fordiscussion).Itisthusnecessarytoverifytowhatdegreeour previousfindingstrulyreflectdopaminergicandcholinergicresponses. Furthermore,adirectcomparisonbetweenlow-levelPEsandhigh-level PEsinordertotestfordifferentialactivationswithinthenucleiisnot possiblewithinthecurrentanalysisframework,astheseregressorslive ondifferentscales;standardising(e.g.z-scoring)theseregressorswould change their interpretation and result in additional complex effects (Lebretonetal.,2019).Therefore,thisstudyattemptstoaddressthe questionofaninvolvementofdopaminein low-levelPEsand acetyl-cholineinhigh-levelPEsbyusing pharmacologicalmanipulationsof dopaminergicandcholinergicreceptorsinhealthyvolunteers.

Here,wepresenttheresultsfromtwopharmacologicalfMRIstudy inhealthyvolunteers(N=81participantsineachstudy;after exclu-sion,N=75(study1)andN=69(study2)).Thesestudiesemployed selective dopaminergicandcholinergicantagonists (study1: amisul-prideandbiperiden) anddopaminergic andcholinergicpotentiating drugs(study2:levodopaandgalantamine),usingthesame computa-tionalmodelingframework asin our previous study(Iglesias et al., 2013).ThebehaviouralandfMRIdatafromoneof thethreegroups instudy1(theplacebogroup)havepreviouslybeenpublishedaspart ofIglesiasetal.(2013).Herewereportdiﬀerentialeﬀectsof amisul-pride/biperidenvs.placeboandlevodopa/galantaminevs.placeboon thebehaviouralandfMRIdata,aswellasanalyseswhereweaveraged acrossallthreepharmacologicalconditionswithineachstudy.

Basedonourpreviousﬁndings(Iglesiasetal.,2013),ourgeneral hypothesiswasthattheprecision-weightedoutcomePE(𝜀2)speciﬁcally

reﬂectsdopaminergicprocessesinthemidbrainandthatthe precision-weightedprobabilityPE(𝜀3)isspeciﬁcallyrelatedtocholinergic

pro-cessesinthebasalforebrain.

This generalhypothesisallowsfortwomorespecificandtestable predictions.Beforewedescribetheseindetail,itisworthpointingout that testing dopaminergicandcholinergic mechanismswith systemi-callyactiveantagonistsiscomplicatedbytheexistenceoftwo oppos-ingpharmacologicaleffects:ontheonehand,antagonistscanblock in-hibitory autoreceptors(locatedonsomataandaxon terminals), lead-ingtodisinhibition(CraggandGreenfield,1997).Ontheotherhand, antagonists inhibitpostsynapticreceptors; these couldbe located on neuronsinremoteprojectionsites(wheretheyareactivatedby synap-ticorvolumetransmission;Zolietal.,1999;Craggetal.,2001)oron nearbyinterneurons(wheretheyareactivatedbyparacrinereleaseof transmitters;Zolietal.,1999).Thisdualmodeofactioncanmake in-terpretationsofpharmacologicalstudieswithsystemicapplication dif-ficult.Forexample,for amisulpride,itis assumedthatlowandhigh doseshaveoppositeneteffects,withinhibitionofautoreceptors domi-natingatlowdosesandinhibitionofpostsynapticreceptorsprevailing athighdoses(Schoemakeretal.,1997;Rosenzweigetal.,2002). Sim-ilarly,thesystemicapplicationofpharmacologicalsubstanceslike lev-odopaandgalantamine,whichincreasetheavailabilityofthe respec-tiveneurotransmitter,doesnotallow forstraightforwardconclusions aboutwhichofthemechanismsmentionedabovemightdominate. Fi-nally,intheabsenceofplasmalevelmeasurementsandinorderto(at leastpartially)accountforindividualdifferencesinpharmacokinetics, weincludedbodyweightasacovariateinoursecond-levelfMRI anal-yses.Thiswasmotivatedbythefactthatbodyweightisonefactorof individualvariabilityinpharmacokinetics,forexample,becauseitcan influenceadrug’svolumeofdistribution(withlargerbodyweight typ-icallyassociatedwithlargervolumeof distribution)and,foragiven dose,plasmaconcentrationdecreaseswiththevolumeofdistribution.

Withthesecaveatsinmind,aﬁrstpredictionconcernsactivityin theneuromodulatorynucleithemselves.Speciﬁcally,acorollaryofour generalhypothesisis thatonewouldexpecttosee thatthemidbrain activationby 𝜀2 andthebasalforebrainactivationby𝜀3 arealtered

speciﬁcallybydopaminergicandcholinergicsubstances,respectively. Thiscorrespondstothefollowinginteractioneﬀects:

(i) Theeﬀectof𝜀2onmidbrainactivityshouldbealteredsigniﬁcantly

morestronglybyamisulpridecomparedtobiperidenandplacebo, andbylevodopa,comparedtogalantamineandplacebo.1

(ii) Theeﬀectof𝜀3onbasalforebrainactivityshouldbealtered

signiﬁ-cantlymorestronglyunderbiperidencomparedtoamisulprideand placebo,andundergalantaminecomparedtolevodopaandplacebo. Notably,eitheroftheseeﬀectscouldbeexertedby𝜀2 or𝜀3intoto

(ascompoundquantities)orcouldresultfromoneofthecomponents (i.e.,thePEortheprecision-weight).

Asecond(andequivalent)predictionconcernsremotetargetareasof dopaminergicandcholinergicprojections.Giventhewidespread distri-butionofcholinergicand,toalesserdegree,dopaminergicprojections totherestofthebrain,itismorediﬃculttospecifyexanteinwhich re-gionsoneexpectspossiblealterationsofPE-relatedactivityby dopamin-ergicandcholinergicdrugs(seeDiscussion).Wethereforetestedthis predictioninaspatiallylessinformedmannerusingwhole-brain analy-ses.

Inbrief,ouranalysesexaminingprecision-weightedPEsintotoand theirseparatecomponents,respectively,didnotprovideclear-cut evi-denceforanunambiguousone-to-onerelationbetweenquantitiesfrom

1_One_would_like_to_add_the_hypothesis_{“signiﬁcantly}_more_in_the_midbrain

thaninthebasalforebrain”.Testingthistypeofregion-by-conditioninteraction, however,isprohibitedinfMRIasthescalingofBOLDsignalcandiﬀeracross regionsduetodiﬀerencesinneurovascularcoupling.

(3)

ourhierarchicalBayesianmodelanddistinctneuromodulatorysystems, aswehadhopedtoﬁnd.Thereareseveralreasons– inadditiontothis relationbeingtrulyabsent,ofcourse– thatcouldexplainourresults.We discussthesepossibilitiesindetail,outliningfuturetestsbasedonthe existingdata.Moregenerally,weusethecurrentexample,toshowcase thecomplexitiesandambiguitiesinherenttopharmacologicalfMRI.

2. Methods

2.1. Subjects

Eighty-onehealthymalevolunteers(meanage±standarddeviation, SD;study1:22±2.4years;study2:22±3.0)participatedineachof thetwoplacebo-controlleddouble-blindpharmacologicalstudieswith abetween-subjectdesign.Withineachstudyvolunteerswererandomly assignedtooneofthreepharmacologicalconditionssuchthateach phar-macologicalconditioncontainedtwenty-sevenvolunteers.Toexclude variationsof hormonal effectson the BOLDsignal (Goldstein et al., 2005),weonlyrecruitedmaleparticipants.Volunteerswereall right-handed,non-smokers,withoutanypsychiatricorneurologicaldisorders intheirpastmedicalhistory,andwerenottakinganymedicationatthe time.Additionally,before includingvolunteers inthestudy,an elec-trocardiogram(ECG)wasrecordedandevaluatedbyaboard-certified cardiologist(R.M.)toexclude thepresenceof potentially arrhythmo-geniccardiacpredispositionsthatcouldhaveposedrisksfor pharma-cologicaltreatment.Notably,thesestudieswereacquiredconsecutively andseparatelyfromeachother.Whiletheinclusionandexclusion cri-teria wereidentical in both studies, participantswere not explicitly matchedbetweenstudies,andnoanalysesbetweenstudieswere per-formed.Participantswerenotexplicitlymatchedacrossdrugconditions either.However,ournarrowinclusionandexclusioncriteriaprevented differencesinmostrelevantvariablesexceptage.Agewasnot signif-icantlydifferent betweendrug groupsin eitherof thestudies(study 1:F(2,72) =0.467,p=0.629,BF10=0.164;study2:F(2,66) =0.194, p=0.824,BF10=0.141).

Ethicsapprovalwasobtainedbythelocallyresponsibleauthorities (KantonaleEthikkommission,KEK2011-0101/3).Allparticipantsgave writteninformedconsentbeforeparticipatinginthestudy.

Priortodataanalysis,eachsubject’sbehaviouraldata(seebelow) wereexaminedforinvalidresponses.Twobehaviouralmeasureswere examinedforeachsubject:trial-wisereactiontimes (RT)andpercent correctresponses(%CR).%CRmeasureswereadjustedtoaccountfor theprobabilisticnatureofthetask,thatis,theywereexpressedin rela-tiontothemaximum%CRthatanagentwithperfectknowledgeofthe probabilistictaskstructurecouldachieve(i.e.74%).Invalidtrialswere definedbythelackofanyresponse(missedresponses)orbyexcessively longreactiontimes(lateresponses;>1500msaftercuepresentation,i.e. atstartoftargetpresentation).Participantswithmorethan15%invalid trialsorlessthan65%CRwereexcludedfromfurtheranalyses.These criterialedtotheexclusionoffiveparticipantsinstudy1(twofromthe amisulpridegroupandthreefromthebiperidengroup)andsix partici-pantsinstudy2(threefromthelevodopagroup,twofromtheplacebo group,andonefromthegalantaminegroup).Furthermore,wehadto excludeoneparticipantperstudyduetoclaustrophobia(biperidenand galantaminegroup,respectively)andfromstudy2twoparticipantsdue tolargemovementartefacts(morethan40additionalregressors censor-ingscanswith≥1mm(translation)or≥1º (rotation)scan-to-scanhead movement;bothfromthegalantaminegroup),oneduetonausea (lev-odopagroup),oneduetodrop-outsinthemidbrainregioninthefMRI data(galantaminegroup),andoneduetoproblemswithmodel-fitting ofthewinningmodel(galantaminegroup).Asaconsequence,thefinal dataanalysisincluded75participants(22±2.3years)instudy1(25 participantsintheamisulpridecondition,23inthebiperiden,and27in theplacebocondition)and69participants(22±3.1years)instudy2 (22participantsinthelevodopacondition,22inthegalantamine,and 25intheplacebocondition).

2.2. Drugadministration

Drug administrationwas performedin arandomisedand double-blindfashion.

In study1 each participantreceived eithera single oraldose of thecholinergic(muscarinic)antagonistbiperiden,thedopaminergic an-tagonistamisulpride,orplacebo(lactose).Basedonpreviousstudies, dosagewaschosenas4mgbiperiden(Guthrieetal.,2000)or400mg ofamisulpride(Rosenzweigetal.,2002),respectively.Drug administra-tiontookplace90minutesbeforestartingtheﬁrsttask(seebelow),as forbothbiperidenandamisulpride,thepeakplasmalevelisattained withinapproximately1.5h(Grimaldietal.,1986;Hamon-Vilcotetal., 1998).

Instudy2subjectsreceivedeitherasingleoraldoseof8mg galan-tamine(acetylcholinesteraseinhibitor),200mglevodopa(prodrugof dopamine)combinedwith50mgoftheperipheraldecarboxylase in-hibitorbenserazide,orplacebo(lactose).Drugadministrationstarted 60 minbefore startingtheﬁrsttask,asforbothsubstancesthepeak plasma level is attained within approximately 1–2 h (Farlow, 2003; KhorandHsu,2007;NoetzliandEap,2013).

2.3. Experimentaldesign:associativelearningtask

Inboth studies,participantsperformed thesameaudio-visual as-sociative learningtask(stimulus-stimulus learning,SSL) asdescribed previously(fMRIstudy2inIglesiasetal.,2013;Iglesiasetal.,2019). Inbrief,participantshadtolearnthepredictivestrengthsofauditory cues (AC)inorder topredict,asquickly andaccuratelyas possible, whichoftwopossiblevisualtarget(VT)categorieswouldfollow(aface orahousepicture,Fig.1A).Importantly,thecue-outcomeassociation strength changedover time(volatility),includingstronglypredictive cues(probabilitiesof0.9and0.1),moderatelypredictivecues(0.7,0.3) andnon-predictivecues(0.5;Fig.1B).Participantswerenotinformed aboutthesequenceofprobabilities.Theywereinstructedexplicitlythat learningone associationwas suﬃcienttoinfertheentire probabilis-ticstructureofthetask(seeSupplementaryMaterialinIglesiasetal., 2013).

The two possible ACs, lasting for 300 ms, included high tones (576Hz)andlowtones(352Hz),werefollowedbytheVT(presented for300ms) 1200mslater.Duringthisinterval,theparticipantshad toindicatebybuttonpresswhethertheypredictedafaceorhouseto appear,providinguswithanexplicitbehaviouralreadoutofprediction (Fig.1A).TheappearanceoftheVTgaveparticipantsexplicitfeedback abouttheirpredictionsandallowedthemtoupdatetheirbeliefs trial-by-trial.Therewerenotrial-wisemonetaryrewards.Participantsreceived aﬁxedpaymentforparticipatinginthestudy,whichwasindependent oftheirtaskperformance.

To ensure that participantsperceived both tones equallyloudly, they performedapsychophysicalmatchingtask withintheMR scan-ner(denOudenetal.,2010),priortothetask.Stimuliwerepresented usingCogent2000(www.vislab.ucl.ac.uk/Cogent/index.html).

2.4. Questionnaires

To controlfor potential changes in vigilanceand arousal due to the drugs,two questionnaires wereapplied: theEpworth Sleepiness Scale (ESS; Johns, 1991) and the Karolinska Sleepiness Scale (KSS; AkerstedtandGillberg,1990).However,onlytheKSSscoreswereused ascovariatesinthefMRIgroupanalyses.

IntheKSS,thesubjectivelevelofsleepinessatdiﬀerenttimepoints duringthedayismeasured.Hereweexplicitlyaskedthemtoratetheir alertnessduringthefMRImeasurements.Subjectshadtoindicateona 9-pointscale,fromextremelyalert(1)toextremelysleepy(9),whichlevel bestreﬂectstheirpsycho-physicalstate(AkerstedtandGillberg,1990).

(4)

Fig.1. Taskandcomputationalmodel

A)Thesensory-sensoryassociativelearningtask(SSL).Subjectshadtopredictwithin1200mswhichvisualstimulus(faceorhouse)followedanauditorycue(high orlowtone).B)Black:theprobabilistictrajectoryvisualizingthechangeintheassociationstrengthbetweentheauditorycuesandthevisualtargets,including highassociations(probabilitiesof0.9and0.1),moderateassociations(0.7,0.3)andnoassociations(0.5);red:trajectoryexampleofasubject-speciﬁcposterior expectationofthevisualoutcome“Face” giventheauditoryhightone(HT).C)HGF:HierarchicalGaussianFilter:x1representstheidentityofthestimulus(stimulus

category),x2thetendencytowardsoneofthecategories(theconditionalprobabilityofthetargetgiventhecueinlogitspace),andx3representsthe(log)volatility

oftheenvironment.Theﬁgurehasbeenadapted,withpermission,fromIglesiasetal.(2013).

2.5. Computationalmodelingofbehaviouraldata

ThebehaviouraldatawereanalysedusingtheHierarchicalGaussian Filter(HGF).Thismodeldescribeshierarchicallearningatmultiple lev-elsintermsof coupledGaussianrandomwalks(Mathysetal.,2011; Mathysetal.,2014).Theupdateequationsinthismodelareanalytic andcontainclassicaldelta-ruleorreinforcementlearningasaspecial case,withprecision-weightedPEsdrivingbelief updatingatdiﬀerent levelsofthehierarchicalmodel.

Here,weusedanidenticalimplementationoftheHGFasinour pre-viousstudy(Iglesiasetal.,2013)andconsideredthreealternative mod-els:Aﬁrstmodel,hgf3l,correspondedtothestandardthree-levelHGF

asdescribedbyMathysetal.(2011);Fig.1C.Forourlearningtask, thefirstlevelofthismodelrepresentedtheoccurrenceoftheauditory andvisualstimuli,thesecondleveltheconditionalprobabilityofthe visualstimulusgiventheauditorycue,andthethirdlevelthechange inthisconditionalprobability(i.e.,log-volatility).Freesubject-specific learningparametersincludedϑ (thespeedoflearningaboutthe(log) volatilityoftheenvironment)and𝜅 (whichdeterminedhowmuchthe estimatedenvironmentalvolatilityaffectedthelearningrateatthe sec-ondlevel).AsexplainedinMathysetal.(2014),forHGFapplications wherethebelief trajectoryat thethirdlevel doesnotinformthe re-sponsemodel,notalloftheperceptualparametersoftheHGFare iden-tifiable.Therefore,inordertoensureparameteridentifiabilitywefixed thelearningparameter𝜔(whichisaconstantcomponentofthe learn-ingrateatthesecondlevel)to-4.Theperceptualmodelwascombined witharesponsemodelthatlinkedtrial-wiseestimatesoftheconditional probability(ofthevisualstimulusgiventheauditorycue)totrial-wise behaviouralresponses(i.e.,thesubjects’predictionsofvisualstimulus category)bymeansof asigmoidfunctionwithparameter𝜁.This en-abledustoinverttheHGFinordertoobtainsubject-specificparameter estimatesandbelieftrajectories(ofPEsandprecisions)foralllevelsof themodel.However,itshouldbenotedthatforthebehavioural anal-ysesoftheHGFparameters,werestrictedourselvesto𝜅 and𝜁,which

couldbewellrecoveredinsimulations,whereasϑ wasnotrecoverable (seeFigureS3,SupplementaryMaterial).

The HGF software used for the analyses in this paper (hgfTool-Box_v1.0)isavailableatwww.translationalneuromodeling.org/tapasas opensourcecode(GPLv3).Allupdateequationsaredescribedindetail byMathysetal.(2011,2014).

UsingrandomeﬀectsBayesianmodelselection(BMS;Stephanetal., 2009; Pennyetal.,2010),we comparedthis three-levelHGF(hgf_3l) to two additional models: (i)a reduced model, hgf2l, in which the

thirdlevel(thelog-volatilityoftheenvironment)wasomitted;this rep-resentedthepossibilitythatparticipantsdidnot trackandmake use ofvolatility;and(ii),anon-hierarchicalRescorla-Wagner(RW)model withfixedlearningrate.BMSoperatesonthelog-evidence,which cor-respondstothenegativesurpriseofencounteringthedatagiventhe model,andquantifiesthetrade-off betweenaccuracy(fit)andmodel complexity. Please note that we could not use model “HGF2” from

Iglesiasetal.(2013)becausethisreferredtothepredictionoftrial-wise rewards– whichdidnotexistinthepresentstudy.

UsingtheHGFweestimatedsubject-specifictrajectoriesofdifferent computationalquantitiesthatwereusedforsubsequentfMRIanalyses. Inourfirstgenerallinearmodel(GLM1)analysisoffMRIdata,the fol-lowingthreequantitieswereusedasparametricmodulators,asinour previousanalysis(Iglesiasetal.,2013):

(i) 𝜀2(= 𝜓2⋅ 𝛿1),theprecision-weightedPEaboutvisualstimulus

out-come2_(which_updates_the_estimate_of_visual_stimulus_probability_in

logitspace,𝜇2);

(ii) 𝜀3(≈ 𝜓3⋅ 𝛿2)theprecision-weightedPEaboutvisualstimulus

prob-ability3_(which_updates_the_estimate_of_{environmental}_log_volatility,

𝜇3);

2_For_the_mathematical_deﬁnition_of_precision_weight_𝜓

2andPE𝛿1,seeEq.

A.7inSupplementaryMaterialtoIglesiasetal.,2013.

3_For_the_mathematical_deﬁnition_of_precision_weight_𝜓

3andPE𝛿2,seeEq.

(5)

(iii) 𝜀_𝑐ℎ,theprecision-weightedchoicePEaboutvisualstimulusoutcome. Inasubsequentanalysis(GLM2), wesplittheprecision-weighted predictionerrorsintotheircomponents,resultinginthefollowing parametricmodulators:

(i) 𝛿1,thePEaboutvisualstimulusoutcome;

(ii) 𝛿2,thePEaboutvisualstimulusprobability;

(iii) 𝜓2, precisionweight at thesecond level; this corresponds tothe

learningratebywhichvisualstimulusprobabilityestimatesare up-dated;

(iv) 𝜓3,precisionweightatthethirdlevel; thisisproportional tothe

learningratebywhich(log)volatilityestimatesareupdated; (v) 𝛿𝑐ℎ,thechoicePEaboutvisualstimulusoutcome.

Notethatowingtotheunambiguousnatureof theoutcomes,our sensoryassociativelearningtaskinvolvedinformationaluncertainty(or estimationuncertainty,i.e.theuncertaintyaboutoutcome probabili-ties,capturedby𝜓2)andenvironmentaluncertainty(i.e.changesinthe

probabilitieswithtime,capturedby𝜓3),butdidnotinvolvesensory

uncertainty.

As thepredictions in this task were of a categorical nature and theprobabilitieswerecoupledforbothlearningtrajectories,we com-putedtheabsolutevalueofboth,theprecision-weightedpredictionerror (𝜀2),andofthepredictionerror(𝛿1)abouttheoutcome(thisis

iden-ticaltocomputing separate trajectoriesfor bothstimuluscategories; seeSupplementaryMaterialtoIglesiasetal.,2013).Thus,the lower-levelprecision-weightedpredictionerror(𝜀2)andthelower-level

pre-dictionerror(𝛿1)representthediﬀerencebetweentheactualvisual

out-comeanditsaprioriprobability.Bycontrast,theprecision-weighted choicepredictionerror(𝜀𝑐ℎ)andthelower-levelpredictionerror(𝛿𝑐ℎ) aresignedpredictionerrorsandrepresentthediﬀerencebetweenthe participant’schoicebeingcorrectandtheaprioriprobabilityofthe cor-rectnessofthischoice.Therefore,thechoicepredictionerrorispositive whentheparticipantmadeacorrectchoiceandnegativewhenthe par-ticipantwaswrong(seeSupplementaryMaterialtoIglesiasetal.,2013). 2.6. Classicalinference

Weperformedunivariateanalysesofcovariance(ANCOVA)totest whetherthereweresignificantdifferencesinbehaviouracrossthe phar-macologicalconditions.ForthisweusedthetwoHGFparameters, reac-tiontime(RT),andpercentcorrectresponses(%CR)asdependent vari-ables,weight,andKSSscoresascovariates,anddrugconditionas inde-pendentvariable.WereportsignificanteffectsataBonferroni-corrected significancethresholdof0.05/4=0.0125.BeforerunningtheANCOVAs weverifiedthattherewerenodifferencesacrossgroupsinthe covari-ates.ThesestatisticalanalyseswereperformedinIBMSPSSStatistics (Version23.0).

Atthesuggestionofareviewerwecomplementedourfrequentist analyseswiththeirBayesiancounterpartsusingthesoftwareJASP (Ver-sion0.13.1; JASP Team, 2020) andreport thecorresponding Bayes factor expressed as BF₁₀, which is computed as the ratio between the probability of the data giventhe alternative hypothesis H1 and theprobability ofthedatagiventhenullhypothesisH0.Asstatedin

Wagenmakersetal.(2018b),BF₁₀ “...grades thedegree ofevidence thatthedataprovidedforH1versusH0“.Accordingtotheir classiﬁ-cation,aBF10rangingfrom3-10reﬂects“moderate” evidence,aBF10

1030“strong” evidence,aBF1030-100“verystrong” evidence,anda

BF10>100“extreme” evidenceinfavourofthealternativehypothesis

(Wagenmakersetal.,2018a).ForallBayesiananalyses,Cauchyprior distributionswere used, as is default in JASP (Wagenmakers et al., 2018a).Asacaveat,thecurrentBayesianANOVAimplementationdoes notaccountforpotentialviolationsofsphericityandnormality, there-fore,theseresultsshouldbeinterpretedwithcaution (vandenBergh etal.,2020).

2.7. fMRIdataacquisitionandstatisticalanalysis

InbothstudiesstructuralandfunctionalMRIdatawereacquired ona3TeslaPhilipsAchievawholebodyMRScanner(PhilipsMedical Systems)equippedwithaneight-channelPhilipsSENSEhead-coil.The structuralimagewasacquiredusingaT1-weightedsequence(inversion

recoveryMPRAGEsequence;resolution=1.1×1.1×0.6mm; inver-siontime(TI)=875ms;repetitiontime(TR)=2.8s).AT2∗-weighted

echo-planarimagingsequencecoveringthewholebrainwasusedfor functionaldataacquisitionandlastedfor∼23min,ormore specifi-callyfor550volumeswithaTRof2500ms,aslicethicknessof3mm; in-planeresolutionof2×2mm;interslicegapof0.6mm;ascending continuousin-planeacquisition;TE=36ms;flipangle=90°;fieldof view=192×192×118mm;SENSEfactor=2;EPIfactor=51(as inIglesiasetal.,2013).Asecondorderpencil-beamvolumeshim (pro-videdbyPhilips)wasappliedtoaccountforfieldinhomogeneities.The fMRIsequenceweusedwasoptimisedforsignalqualityinbrainstem andbasal forebrain,resulting in signal dropoutsin theorbitofrontal cortex(seeFig.S6).Therefore,noconclusionscanbedrawnaboutthe representationofourcomputationalquantitiesinthisregion.

Inordertoenablephysiologicalnoisecorrectionofbreathingand heartbeatrelatedsignalvariance,participantsworeabreathingbelt, andanelectrocardiogramwasobtainedduringfMRIdataacquisition.

WeanalysedallfMRIdata– includingtheplacebogroupdata pre-viouslypublishedinIglesiasetal.(2013)– usingStatisticalParametric Mapping(SPM),version12(r7487).Preprocessingstepsincluded mo-tioncorrectionofthefunctionalimages(realignment),co-registration tothestructuralimage,warping ofthefunctionalandstructural im-ages toMNI spaceusing the“New Segment” toolboxin SPM12, re-sampling to 1.5 × 1.5 × 1.5 mm resolution, andsmoothing of the functionalimageswitha6mmfull-widthathalfmaximumGaussian kernel. Signal-to-noiseratiooptimizationforrelevantregions suchas the brainstem, wasperformed by correctingforphysiological noise using RETROICOR (Glover et al., 2000) based on the PhysIO tool-box(expansionorderoftheregressors:3rd,4th,1storderforcardiac, respiratoryandinteractionbetweenrespiratoryandcardiaccycle, re-spectively (Harvey etal., 2008)) andbyentering intotheﬁrst-level GLMsregressorsobtainedfromprincipalcomponentanalysis(PCA)of white matterandcerebrospinalﬂuid(CSF; Behzadietal., 2007) us-ingthePhysIOtoolbox(version2019b,v7.2.1)(Kasperetal.,2017; www.translationalneuromodeling.org/tapas).

Wespecifiedtwodifferentvoxel-wisegenerallinearmodels(GLMs) foreachparticipant.Inbothofthesefirst-levelGLMswespecified re-gressorsrepresentingthetwovisualtrialtypes(face,house),eachof whichwasmodulatedbyseveralcomputationalquantitiesthatwere es-timatedfromtheindividualbehaviour.Specifically,inGLM1weused thesubject-specifictrajectoriesofprecision-weightedPEs𝜀2,𝜀3,𝜀𝑐ℎas computational(parametric)modulators.InGLM2wesplitour computa-tionalquantitiesintoprecisionandpredictionerrorcomponents, result-inginthefollowingparametricmodulators:𝛿1,𝛿2,𝜓2,𝜓3,𝛿𝑐ℎ.Inboth GLMs,theregressorswerenotorthogonalisedtoeachotherandwere time-lockedtotheoutcomephase.Additionally,wemodeledmissedand lateresponses,respectively,byseparateregressors.Allregressorswere convolvedwithacanonicalhemodynamicresponsefunctionandits tem-poralderivative.

Inadditiontotheseregressorsofinterest,ourﬁrst-levelGLMsalso includedregressorsrepresentingpotentialconfounds.Thisincludedthe realignment parameters, their ﬁrst derivative, a regressor censoring scanswith≥1mmor≥1º scan-to-scanheadmovement,physiological confoundvariablesrelatedtocardiacactivityandbreathing(provided bythePhysIOtoolbox),andthePCAregressorsforwhitematterand CSF.

Contrastsofinterestsatthefirstlevelincludedtheaverageeffectof each computationalquantity(parametricmodulator)specifiedabove. Thesecontrastswereenteredintostudy-specificseparatesecond(group) levelANOVAs(fullfactorialdesign)thatcomparedthethreedifferent

(6)

drug groups,together withthetwo covariates(KSSscore,andbody weight).Totestourhypotheses,we comparedequivalentdrugtypes across neuromodulators (i.e.amisulpride/biperiden/placebo and lev-odopa/galantamine/placebo). KSS accounted for individual variabil-ityin drug-inducedvigilance. Thecovariatebodyweight was mean-centeredandincludedaninteractionwiththefactordrug.Bodyweight servedasaproxyforthevolumeofdistributionandthusasa(rough) approximationtoindividualdifferencesinpharmacokinetics.In addi-tiontotestingforamaineffectofdrug,wetestedwhetherdrugeffects oncomputationalquantitiesshowedabody-weightdependenteffect.

In addition to whole-brain analyses, we performed region-of-interest(ROI) analysesbasedonacombinedanatomicalmaskof pu-tativedopaminergicandcholinergicnuclei,asinour previousstudy (Iglesiasetal.,2013).Theanatomicalmaskincluded(i)the dopamin-ergic midbrain (substantia nigra, SN, and ventral tegmental area, VTA;Bunzeck andDuzel, 2006), (ii)the cholinergicbasal forebrain (Eickhoﬀ et al.,2005; Zaborszkyetal.,2008),and(iii)theputative cholinergicnucleiinthetegmentumofthebrainstem,i.e.,the peduncu-lopontinetegmental(PPT)andlaterodorsaltegmental(LDT)nuclei.The latterwerebasedonmanualdelineationusingMRICronandanatomical landmarks(Naidichetal.,2009;Zrinzoetal.,2011).

Finally,we alsohad access toinformation about two single nu-cleotidepolymorphisms (SNPs) in ourvolunteers: COMT(related to dopamine)andChAT(related toacetylcholine;formore details,see SupplementaryMaterialSection“SupplementalResults”). Inorderto exploreinteractioneffectsofthepharmacologicalinterventionand com-putationalquantitieswithgenotype,weperformedwhole-brainandROI group-levelanalysesasdescribedpreviously,butaddingCOMTorChAT asacovariate.However,asoursamplesizemaynotbesufficientfor ro-bustanalysesoftheinfluencebygenotype,wereporttheseresults in theSupplementaryMaterialonly.

Ouranalysesfocusedonouranatomicalregionsofinterest(ROIs), testingeachcomputationalquantityforgroupdiﬀerences(e.g., diﬀer-encesin 𝜀2-relatedactivitybetween biperiden andplacebo).Forthe

pharmacologicaleffectsinourROIs,weuseBayesianstatisticstoassess theevidenceforthenullhypothesisrelativetothealternative hypoth-esis.Sinceitismethodologicallychallengingtodealwithspatial de-pendenciesacrossvoxelsincurrentimplementationsofBayesiantests whencorrectingformultiplecomparisons,theBayesianprocedurewe useddiffersfromthestatisticalanalysisperformedwithSPM. Specif-ically,fortheBayesianapproach, wesummarisedthedataacross all voxelswithinagivenROI(intermsofthefirsteigenvariate)whereas fortheSPManalyseswe wereperforming voxel-wiseanalyses.More specifically,intheBayesianapproachweextractedthefirst eigenvari-ate(orprincipalcomponent)separatelyforeachregionfromourapriori anatomicalmask,i.e.,SN/VTA,basalforebrain,andPPT/LDT.This ex-tractionwasperformedforeachregionseparately,withoutapplyingany threshold,andremovingalleffectsrelatedtobodyweightand sleepi-ness.Foreachcomputationalquantityandeachofthethree anatomi-calregions,theextractedfirsteigenvariateswereenteredintoseparate BayesianANOVAs,withtherespectivefirsteigenvariateasdependent variable,andgroupasindependentvariable.Themodelcontainingthe factorgroup(H1)wasthencomparedtoanullmodel(H0).Post-hoc

tests– i.e.pairwisecomparisonswithBayesiant-testsusingaCauchy prior(0,r=1/sqrt(2))– wereonlycomputediftheANOVAdisplayedat leastmoderateevidenceforthealternativehypothesis(i.e.ifBF10>3).

InJASP,multipletestingisaccountedforbyadjustingthepriorodds. MultiplyingtheprioroddswiththeestimatedBFresultsintheposterior odds(i.e.therelativeplausibilityofthemodelafterobservingthedata (vanDoornetal.,2020).

Additionally,anumberofregionsthroughoutthebrainshowed sig-nificantdrugeffectsonactivityrelatedtothedifferentcomputational quantities.Asthesefindingsdidnotdirectlyrelatetoourhypotheses, wereporttheseresultsintheSupplementaryMaterial(TablesS32and S33).RegardlesswhetheranalyseswereconductedforROIsoracrossthe wholebrain,wealwayscorrectedstringentlyformultiplecomparisons;

intheformercase,thesearchvolumecorrespondedtothetotalvolume ofallROIscombined.Allfindingsreportedinthispapersurvived family-wiseerrorcorrectionformultiplecomparisons(p<0.05),eitheratthe peak-leveloratthecluster-levelwithacluster-definingthreshold(CDT) ofp<0.001.ThisCDTaffordsvalidcluster-levelinferenceinSPM(see Eklundetal.,2016;FlandinandFriston,2016).

3. Results

3.1. Behaviouraldata:classicalanalysis

Here,wereportbehaviouraldatafromtheN=75 participantsin study1andN=69instudy2,whosedatawereincludedinthefMRI analyses,respectively.First,withineverystudywetestedforany sig-nificant differencesin the sleepiness ratings (KSS) andbody weight betweenthepharmacologicalgroups.Wedidnotfindanysignificant groupdifferences,neitherinstudy1(KSS:F(2,72)=0.229,p=0.796,

BF10=0.137,i.e.thedatais1/0.137=7.3timesmorelikelyunderH0 thanunderH₁;bodyweight:F_(2,72)=1.354,p=0.265,BF₁₀=0.326) norin study2(KSS:F(2,66) = 1.012,p= 0.369,BF10 = 0.260;body

weight:F(2,66)=0.674,p=0.513,BF10=0.203).

We subsequentlyperformed one-wayanalyses (ANOVA) on reac-tiontimes(RT)andpercentcorrectresponses(%CR).Because normal-ityand/orhomoscedasticitywerenotalwaysmet,Welch’sANOVAwas performed for all variablesusing drug as independentvariable. No-tably,usingthis testwecannotaccountfortheeffectsofthe covari-atesKSS andweight. Wedidnot find anysignificant maineffectof drug onthese variables(study1:RT: F(2,47.675) = 0.621,p= 0.542,

BF₁₀=0.182;%CR:F_(2,44.343)=1.193,p=0.313,BF₁₀=0.308;study2: RT:F(2,43.167)=0.567,p=0.571,BF10=0.175;%CR:F(2,43.280)=1.544, p=0.225,BF10=0.431).

3.2. Behaviouraldata:computationalmodeling

Using Bayesian modelselection (BMS;Stephan etal., 2009), we tested,following ourprevious study(Iglesiaset al.,2013),which of severalalternativegenerativemodelsbestexplainedourparticipants’ behaviour.Speciﬁcally,wecomparedathree-levelHGFmodel(hgf3l),

areducedversionoftheHGF(hgf2l),andtheclassicalRescorla-Wagner

model(RW)ofassociativelearningthathasaﬁxedlearningrateand isagnosticaboutenvironmentalvolatility.ForBMSanalyseswehadto excludeparticipantsduetonumericalproblemsduringinversionofthe RWmodel(study1:ﬁveparticipantsexcluded;study2:fourparticipants excluded)andinversionofthehgf3lmodel(1participantexcludedfrom

study2aslistedinthe“Subjects” session).

Independentlyofdrugcondition,randomeﬀectsBMSyieldeda pos-teriormodelprobability(PP)of87.0%and79.9%instudy1andstudy 2,respectively,andaprotectedexceedanceprobabilityof>0.99inboth studiesinfavorofmodelhgf3l.Thissuggeststhatourparticipantsnot

onlylearnedthetask-relevantconditionalprobabilitiesofvisualstimuli, butwerecapableoftrackingthevolatilityoftheenvironmentand up-datingtheirlearningratedynamically(FigureS4AandS5A,TableS3). (PPistheexpectedprobabilitythatthemodelinquestiongeneratedthe dataforarandomlychosensubject.Theexceedanceprobability(XP)of amodeldenotestheprobabilitythatthismodelhasagreaterposterior probabilitythananyothermodeltested;andtheprotectedexceedance probability(PXP)accountsforthepossibilityofmodelshavingidentical frequencies;(Rigouxetal.,2014)).Furthermore,wheneach pharmaco-logicalconditionwasconsideredseparately,hgf3lwasthemost

com-pellingofthemodelstested(study1:placebo:PP=87.9%,PXP>99%; amisulpride:PP=81.0%,PXP>98%;biperiden:PP=71.7%,PXP> 97.9%;FigureS4B-D,TableS3;study2:placebo:PP=65.0%,PXP> 94%;levodopa:PP=85.9%,PXP> 99%;galantamine:PP=72.2%, PXP>93%;FigureS5B-D,TableS3).

Havingidentiﬁedanoptimalmodel(amongstthemodelstested),we proceededtotestingwhetherwithinstudythereweresigniﬁcant

(7)

diﬀer-encesinparameterestimatesacrossdrugconditions(seeTableS4).(In thesubsequentanalyses,weexcluded onesubject forwhich wehad encounterednumericalﬁttingproblemsforhgf3l).Tothisend,we

per-formedWelch’sone-wayanalysesofvarianceandaBayesianANOVAfor thedependentvariables𝜅 (thelearningparameterreflectinghowmuch theestimatedenvironmentalvolatilityinfluenceslearningatthesecond level)and𝜁 (theparameterencodingdecisionnoiseintheobservation model)withdrugasindependentvariable.Wedidnotfindany signifi-cantmaineffectofdrugonthesemodelparameterestimates(study1: 𝜅:F(2,44.937) =0.296,p=0.745,BF10=0.157;𝜁:F(2,45.737)=0.344, p= 0.711,BF10 =0.144;study2: 𝜅: F(2,43.368) = 1.237,p=0.300,

BF₁₀=0.275;𝜁:F_(2,41.511)=2.375,p=0.106,BF₁₀=0.514). Finally,multipleregressionwasappliedseparatelyinbothstudiesto testwhetherthemodelparameterestimatescouldexplaintask perfor-mance(percentcorrectresponses(%CR))andreactiontimes(RT)across allpharmacologicalconditions.Notably,iftheidentifiedmodelis rea-sonable,onewouldexpectthatitisassociatedwith%CRbutnotRT; thisisbecausethemodelwaschallengedtoexplainthebinarynature oftrial-by-trialdecisions,butwithoutreferencetotheirspeed.Thistest wasconductedwhilecontrollingforpotentiallyconfoundingfactorsin relationtopharmacology,i.e.KSS,andbodyweight.Inafirststepwe onlyconsideredtheconfoundvariablesandexaminedwhether,ontheir own,theyaffectedindividual%CR.Thiswasnotthecaseinstudy1:the combinedinfluenceofpotentialconfoundsonlyexplained6.5%ofthe variancein %CR(study 1;F_(2,72) =2.486,p= 0.09;BF₁₀ = 0.627). However,we founda significantinfluencein study2 (R2= 11.7%; F(2,66) = 4.388, p = 0.016;BF10 = 2.843). Notably, this effect was

onlysigniﬁcantinthegalantaminegroup(R2=30.6%;F_(2,19)=4.187, p=0.031;BF10=2.434).

Wethenadded thetwomodelparameter estimates(𝜅 and𝜁; Ta-bleS4)totheregressionmodel;thisexplainedadditional58.9%and 47.9%of the variancein %CRin study1and study2, respectively (study1:R2change=58.9;F(2,70) =59.508,p<0.001;BF10 >150;

study2:R2change=47.9;F_(2,64)=37.892,p<0.001;BF₁₀>150). Themainparameterdrivingthisresultwas𝜁 (study1:𝛽 =0.793,p< 0.001;BFInclusion>150;study2:𝛽 =0.737,p<0.001,BFInclusion>150;

BFInclusionrepresentstheevidenceinthedataforincludingapredictor

inthemodel(vandenBerghetal.,2020)).Theseresultswerenotdriven byanysinglepharmacologicalcondition,butwerefoundsimilarlywhen consideringeachdruginisolation(seeSupplementaryMaterial).

Bycontrast,asexpected,addingthemodelparameterestimatesto theregressionmodeldidnothelpexplainRT(study1:across condi-tions:R2change=0.02;F(2,70)=0.822,p=0.444,BF10=0.333;study

2:across conditions:R2 change= 0.057;F_(2,64) =2.218,p=0.117, BF10=0.844).

Finally, it is worth noting that the behavioural results reported here (both in terms of model comparison and analyses of parame-ter estimates) are perfectly consistent with the results of the non-pharmacologicalfMRIstudyinIglesiasetal.(2013).

3.3. fMRIresults

First, we tested for main eﬀects of each computational variable (withinevery study pooledacross drug conditions),both across the wholebrainandwithinouranatomicalmask(thecombinedROIsof dopaminergicandcholinergicnuclei).Thisservedtotestwhether, aver-agedacrossallpharmacologicalconditions,wewouldobtain compara-bleresultstothenon-pharmacologicalresultsasinIglesiasetal.(2013). Forthis,weusedthesamestatisticalcriteriaasinIglesiasetal.(2013), i.e.,family-wiseerrorcorrectionformultiplecomparisons(p<0.05),at thepeak-level.

Second,wetestedfordifferentialdrugeffectsonthecomputational quantitiesandinteractioneffectsbetweenbodyweight,drug,and com-putationalquantities;thistestwasappliedbothacrossthewholebrain andwithinanatomicallydefinedROIs.AsdetailedintheMethods sec-tion,weonly reportresults thatsurviveFWEcorrectionfor multiple

comparisons.Unlessmentionedotherwise,thepharmacologicalresults survivep<0.05atthecluster-levelwithacluster-deﬁningthreshold (CDT)ofp<0.001.

Third,weconductedanalternativetestofdiﬀerentialdrugeﬀects on the computationalquantities usingBayesian ANOVAs (see Meth-ods).Here,wefocusedonthethreeanatomicalregionsSN/VTA,BFand PPT/LDTwithinourapriorianatomicalmaskandassessedtheevidence foramodelincludingthepharmacologicalgroupfactorcomparedtoa nullmodel.

AsexplainedintheMethods,weusedtwodiﬀerentGLMs.InGLM1 weusedprecision-weightedPEs(𝜀2,𝜀3,𝜀𝑐ℎ)ascompoundcomputational

quantitiesforparametricmodulation.InGLM2,wesplitthesequantities intoprecisionandPEcomponents,resultinginﬁveparametric modula-tors(𝛿1,𝛿2,𝜓2,𝜓3,𝛿𝑐ℎ).

3.3.1. GLM1:averageeﬀectofprecision-weightedPEsacrossdrug conditions

Weﬁrstexaminedinbothstudiesseparatelythelow-level precision-weightedPEs,theabsoluteprecision-weightedoutcomePE𝜀2 andthe

precision-weightedchoicePE𝜀𝑐ℎ.Inbothstudies,whole-brainanalyses showedasigniﬁcant activationof numerousregionsby𝜀2, including

parietal,prefrontal,visual,insularareas,andcerebellum(seeFigs.2A and3AandTableS8).Additionally,weobservedadeactivationof op-ercular,insular,cingulate,auditory,hippocampalandprefrontalareas (seeTableS9fordetails).ThechoicePE(𝜀𝑐ℎ)notonlyactivatedawide rangeofprefrontal,cingulate,insular,temporal,andparietalregions, butalsoventralstriatum,basalforebrain,putamen,andhippocampus (Figs.2Band3B,foracompletelist,seeTableS10).Deactivationsby 𝜀𝑐ℎ werefoundinsupplementarymotorcortex,middlecingulate cor-tex,superiorparietalcortex,middlefrontalgyrus,calcarinecortexand precuneus(TableS11).

Withinouranatomicalmask,wefounda signiﬁcant𝜀2 activation

inthemidbrain(asalreadyobservedinthewholebrainanalysis;see Figs.2Dand3DfortheanatomicalROIanalysis),butalsoinPPT/LDT (TableS12)andasigniﬁcant𝜀2deactivationinthebasalforebrain

(Ta-bleS13).Similarly,thechoicepredictionerror𝜀𝑐ℎactivatedthebasal forebrain(Figs.2Eand3E,Table S14),asin thewholebrain analy-sis, anddeactivatedin bothstudiesputatively thecholinergicnuclei (PPT/LDT;TableS15)andadditionallyinstudy1themidbrain.

Subsequently,wemovedtothenexthigherlevelofthehierarchyin ourmodelandexaminedtheprecision-weightedprobabilityPE,𝜀3.In

whole-brainanalyses,wefound𝜀3activationsinthehippocampus,ACC,

MCCopercularandinsularregions(Figs.2Cand3CTableS16). Deac-tivationswerefoundinnumerousregions,includingoccipital,insular, prefrontal,parietal,temporalareasaswellascerebellum(TableS17). Restrictingtheanalysestoouranatomicalmask,wefoundtheexpected 𝜀3activationwithinthebasalforebrain(Figs.2Fand3F,TableS18)and

adeactivationofthemidbrain(TableS19).

3.3.2. GLM1:drugeﬀectsonprecision-weightedPEs– ROI(anatomical mask)

InanalysesrestrictedtoouranatomicalROIs,wedidnotﬁnd signif-icantdiﬀerencesbetweendrugconditionsforanyofthecomputational quantities(𝜀2,𝜀3,𝜀𝑐ℎ)thatsurvivedcorrectionformultiplecomparisons.

UsingBayesianANOVAwetested,foreachcomputationalquantity andineachROI(i.e.SN/VTA,BF,PPT/LDT),theevidenceforamodel thatincludedthefactorpharmacologicalgroupvs.anullmodel.For almostalltests,theBayesfactorsshowedthatthedatawerebetter ex-plainedbythenullmodel,althoughtheevidencewasonlymoderate atbest(seeTableS5).Bycontrast,evidencefordrugeﬀectswasonly foundinonecase(𝜀3inPPT/LDT,weakevidence).

3.3.3. GLM2:averageeﬀectsofPEsandprecisionsacrossdrugconditions Thepredictionerrorabouttheoutcome,𝛿1producedasimilar

whole-brainactivationpatternas𝜀2(Figs.4Aand5A),includingprominent

(8)

Fig.2. Study1-Activationsbytheprecision-weightedPEs

A)whole-brainactivationsby𝜀2;B)whole-brainactivationsby𝜀𝑐ℎ;C)whole-brainactivationsby𝜀3;A-C)Activationmapsareshownatathresholdofp<0.05,

FWEpeak-levelcorrectedformultiplecomparisonsacrossthewholebrain.D)activationsby𝜀2;E)activationsby𝜀𝑐ℎ;F)activationsby𝜀3;D-F)Activationmaps

areshownatathresholdofp<0.05,FWEpeak-levelcorrectedformultiplecomparisonsacrosstheanatomicalmask(red)overlayedonactivationthresholdedat p<0.05,FWEcluster-levelcorrectedformultiplecomparisonswithaninitialCDTofp<0.001(orange).WB:whole-brain;AM:anatomicalmask.

Fig.3. Study2-Activationsbytheprecision-weightedPEs

A)whole-brainactivationsby𝜀2;B)whole-brainactivationsby𝜀𝑐ℎ;C)whole-brainactivationsby𝜀3;A-C)Activationmapsareshownatathresholdofp<0.05,

FWEpeak-levelcorrectedformultiplecomparisonsacrossthewholebrain.D)activationsby𝜀2;E)activationsby𝜀𝑐ℎ;F)activationsby𝜀3;D-F)Activationmaps

(9)

Fig.4. Study1-ActivationsbythePEs

A)whole-brainactivationsby𝛿1;B)whole-brainactivationsby𝛿𝑐ℎ;C)whole-brainactivationsby𝛿2;A-C)Activationmapsareshownatathresholdofp<0.05,

FWEpeak-levelcorrectedformultiplecomparisonsacrossthewholebrain.D)activationsby𝛿1;E)activationsby𝛿𝑐ℎ;F)activationsby𝛿2;D-F)Activationmaps

wellasbasalganglia,thalamus,and(instudy2)midbrain(fordetailed results,seeTableS20).Deactivationswerefoundinthebasalforebrain, hippocampus,insula,aswellasinvariousothercorticalareas(Table S21).

Inouranatomicalmaskwefoundasigniﬁcant𝛿1activationwithin

themidbrainandPPT/LDT(Figs.4Dand5D).Deactivationswerefound intheseptumandbasalforebrain(spreadingtoparahippocampalgyrus butstillwithintheprobabilisticallydeﬁnedboundariesofthebasal fore-brain;TablesS22andS23).

Concerningthechoicepredictionerror,𝛿𝑐ℎ,wefoundasimilar ac-tivationpatternascomparedtoitsprecision-weightedcounterpart,𝜀𝑐ℎ, describedabove.Thisincludedactivationsinstriatum,basalforebrain, vmPFC,midandposteriorcingulatecortex(fordetailsandother acti-vations,seeFigs.4Band5B,andTableS24;fordeactivations,seeTable S25).IntheanatomicalROIanalyses,wefounda𝛿𝑐ℎactivationin bilat-eralbasalforebrain(Figs.4Eand5E,TableS26)andadeactivationin theputativelycholinergicnucleiandSN(TableS27).

Whole-brainanalysesofthehigh-levelpredictionerror(about prob-ability),𝛿2,activatedthehippocampusandanteriorcingulategyrusas

previouslyobservedforitsprecision-weightedcounterpart𝜀3,butalso

theseptumandmiddlecingulategyrus(Figs.4Cand5C;TableS28). Interestingly,theseptal𝛿2 activationwasconsiderablystrongerthan

for𝜀3,suggestingthatseptalactivitywasmorestronglydrivenbythe

PEthanbyprecision.The𝛿2deactivationsweresimilartothosefor𝜀3

(TableS29).IntheanatomicalROIanalyses,wefoundtheexpected ac-tivationofseptum(Figs. 4F and5F;TableS30),anddeactivationsin PPT/LDTandmidbrain(TableS31).

Finally,weexploredactivityrelatedtotheprecisionweights. No-tably,theseprecisionweightshaveslightlydiﬀerentformsacrossthe twolevelsofourmodel.Atthelower(second)level,theprecisionweight 𝜓2correspondstovarianceoruncertainty(aboutstimulusoutcome);see

Mathysetal.(2014).Inwhole-brainanalyses,thisquantityledto ac-tivationsof angularandfrontalgyri,parietalcortex,with additional activationsinbasalganglia,anteriorinsula(onlystudy1)and cerebel-lum(Figs.6Aand7A,TableS32).Adeactivationwasfoundintheright thalamusandprecentralgyrus(TableS33).Noresultswerefoundfor 𝜓2intheanatomicalROIanalyses.

Concerning thehigher (third) level, theprecision weight𝜓3

cor-responds to a ratio of second- and third-level uncertainties; see Mathyset al.(2014)for details.Forwhole-brainanalyses of𝜓3, we

foundsigniﬁcantactivationsoffusiformandlingualgyri,parietal, tem-poral,frontalareas,anteriorinsulaandbrainstemregions(Figs.6Band 7B;TableS34;nooverlappingdeactivationsacrossbothstudieswere found;TableS35).Withinouranatomicalmask,we foundan activa-tioninthemidbrain,andcholinergicnucleiPPT/LDT(Figs.6Dand7D; TableS36).

3.3.4. GLM2:drugeﬀectsonPEsandprecisions– ROI(anatomicalmask) Forthelow-levelPE,wefoundthat,comparedtoplacebo, galan-tamineenhancedthepositive relationshipbetween bodyweight and 𝛿1-relatedactivityin themidbrain (FWE peak-levelandcluster-level

corrected;x=0;y=-27;z=-18;tscore:4.46;clustersize:23 vox-els;Fig.8A).Furthermore,biperidenasopposedtoplaceboincreased the𝛿1-relatedactivityintherightPPT/LDT(FWEpeak-levelcorrected;

x=6;y=-36;z=-16;tscore:4.19;Fig.9A).Thesameregion(x=6; y=-36;z=-18)showedanear-signiﬁcantactivationby𝛿1under

amisul-prideascomparedtoplacebo(FWEpeak-levelcorrected;tscore:3.99; p=0.051).

Forthehigh-levelPE,wefoundthatbiperidendecreased𝛿2-related

activityintherightPPT/LDT(FWEpeak-levelcorrected;x=6;y=-36; z=-18;tscore:4.29;Fig.9B).Additionally,wefoundadecreased𝛿2

(10)

Fig.5. Study2-ActivationsbythePEs

A)whole-brainactivationsby𝛿1;B)whole-brainactivationsby𝛿𝑐ℎ;C)whole-brainactivationsby𝛿2;A-C)Activationmapsareshownatathresholdofp<0.05,

FWEpeak-levelcorrectedformultiplecomparisonsacrossthewholebrain.D)activationsby𝛿1;E)activationsby𝛿𝑐ℎ;F)activationsby𝛿2;D-F)Activationmaps

Table1

Summaryofallsignificantpharmacologicaleffectsonbrainactivityassociatedwithourcomputationalquantitiesinapriorianatomicalregionsof interest.Notethatnoeffectwasfoundforlevodopa.

Computational quantity/anatomical region 𝛿1 SN/VTA 𝜓 3 SN/VTA

galantamine ↑ vs. placebo interaction

with body weight

amisulpride ↑ vs. placebo

Computational quantity/anatomical region 𝛿1 PPT/LDT 𝛿2 PPT/LDT 𝛿2 PPT ∕ LDT 𝛿2 PPT ∕ LDT

biperiden ↑ vs. placebo ↓ vs. placebo

amisulpride ↓ vs. placebo ↑ vs. biperiden interaction

with body weight

toplacebo(FWEpeak-levelcorrected;x=6;y=-36;z=-18;tscore: 5.30;Fig.9C)andamodulationofdrugeﬀectson𝛿2-relatedactivityby

bodyweightinrightPPT/LDT(FWEpeak-levelcorrected;x=6;y=-34; z=-15;tscore:3.99):underamisulpridewefoundapositiverelation between𝛿2-relatedactivityandbodyweightwhereasthisrelationwas

inverted(negative)underbiperiden(Fig.9D).Moreprecisely,𝛿2-related

activityin thePPT/LDTincreasedwithlowerbodyweight(andthus presumablyhigher plasmalevelsofbiperiden)andwithhigher body weight(andhenceputativelylowerlevelsofamisulpride).

Finally,amisulpridecomparedtoplaceboincreasedthe𝜓3-related

activityinthemidbrain (FWEpeak-level andcluster-level corrected; x=11;y=-25;z=-21;tscore:4.26;clustersize:33voxels;Fig.8B).

Therewerenosigniﬁcantdrugeﬀectsfor𝛿𝑐ℎ, 𝜓2ineitherstudynor

foranyofthecomputationalquantitiesinstudy2,exceptfortheone ef-fectmentionedabove(ofgalantamineontherelationbetween𝛿1-related

activityinthemidbrainandbodyweight).Allsigniﬁcant pharmacologi-caleﬀectsacrosstheanatomicalmaskhavebeensummarizedinTable1.

Instudy1,aBayesianANOVAfoundmoderateevidenceforadrug eﬀecton𝛿1-relatedactivityinPPT/LDT(BF10=9.105).Inlinewiththe

SPMresults,Bayesianpost-hoctestsrevealedmoderateevidencefora diﬀerencebetweenplaceboandbiperiden(posteriorodds=4.924)and between placeboandamisulpride(posteriorodds=3.505;seeTable S7).Fortheotherregions(SN/VTAandbasalforebrain),therewasweak tomoderateevidenceinfavourofthenullmodel,i.e.absenceofdrug eﬀects(seeTableS6).

Finally,forallothercomputationalquantities(except𝛿2,werewe

foundaweakeﬀectinPPT/LDT)usingBayesianANOVAtheevidence showedthatthedata(extractedfromthethreeanatomicalregions, re-spectively)wasbestpredictedbythenullmodel(seeTableS6).

4. Discussion

Thenotionthatthediﬀerentneuromodulatorytransmitters– such asdopamine,acetylcholine,serotoninornoradrenaline– mightserveto

(11)

Fig.6. Study1-Activationsbytheprecision-weights

A)whole-brainactivationsby𝜓2;Activationmapsareshownatathresholdofp<0.05,FWEpeak-levelcorrectedformultiplecomparisonsacrossthewholebrain

(red)overlayedonactivationthresholdedatp<0.05,FWEpeak-levelcorrectedformultiplecomparisonswithaninitialCDTofp<0.001(orange).

B)whole-brainactivationsby𝜓3;Activationmapsareshownatathresholdofp<0.05,FWEpeak-levelcorrectedformultiplecomparisonsacrossthewholebrain.

C)activationsby𝜓2;D)activationsby𝜓3;C-D)Activationmapsareshownatathresholdofp<0.05,FWEpeak-levelcorrectedformultiplecomparisonsacross

theanatomicalmask(red)overlayedonactivationthresholdedatp<0.05,FWEcluster-levelcorrectedformultiplecomparisonswithaninitialCDTofp<0.001 (orange).WB:whole-brain;AM:anatomicalmask.

signaldistinctcomputationalquantitieshasalonghistory(forreviews, seeDoya,2008;Dayan,2012;Iglesiasetal.,2017).Aseminalfinding wasthatphasicdopaminereleaseappearstoreflectrewardPEsthatmay provideateachingsignalduringinstrumentallearning(Schultzetal., 1997;Montagueetal.,2004).HumanfMRIstudieshavefound represen-tationsofPEsinputativedopaminoceptiveregions,suchasthemidbrain (D’Ardenneetal.,2008;Klein-Fluggeetal.,2011;Diederenetal.,2016; HowardandKahnt,2018)orventralstriatum(Pessiglioneetal.,2006; Glascheretal.,2010; Dawetal.,2011; DanielandPollmann,2012; Hackeletal.,2015;Diederenetal.,2016;Guggenmosetal.,2016). Ad-ditionally,recent meta-analysesdifferentiating betweenabsolute PEs (i.e. “surprise PE”; deviation between prediction and outcome) and signed PEs found that the former was represented in the putative dopaminergicmidbrain,whereasthelatterwererepresentedinthe ven-tralstriatum(Garrisonetal.,2013;Fouragnanetal.,2018).Inourstudy, thelow-levelabsolutePEs(𝛿1andprecision-weighted𝜀2,respectively)

resemblethepreviouslymentionedsurprisePEandactivatedsimilar re-gions,suchasthemidbrain.Similarly,thechoicePEs(𝛿𝑐ℎand

precision-weighted𝜀𝑐ℎ,respectively)showedasimilaractivationpatternasthe signedPE.Itmustbekeptinmind,however,thatdiﬀerencesin interpre-tationexistsbetweenourBayesianPEsandPEsfromRLmodels.Whilst inRLthegoalistomaximizereward,inourBayesianframeworkthe goalistominimize(anapproximationto)surprisebybeliefupdatingat multiplelevels(Mathysetal.,2014),wheretheuncertaintyofbeliefs aﬀectsthewayPEsimpactonlearning.

ThesignedchoicePEcomputedinthisstudyisdifferentfromthe signedrewardPE(RPE)usedinRL:whileRPEsencodewhetheran out-comewasbetter(positiveRPE)orworsethanpredicted(negativeRPE; Schultzetal., 1997; Fouragnanetal., 2018),in ourcasethesigned choicePErepresentsthedifferencebetweentheparticipant’schoice be-ingcorrectandtheaprioriprobabilityofthischoicebeingcorrect(see Iglesiasetal.,2013).Finally,ourlow-levelabsolutePEcorrespondsto Bayesiansurprise,i.e.thedifferencebetweentheoutcomeanditsa pri-oriprobability.

Although RPEs andoutcome PEshave been linkedto dopamine, questionsofspeciﬁcityandcontext-dependencyhavearisen,evenfor

(12)

Fig.7. Study2-Activationsbythe precision-weights

A)whole-brainactivationsby𝜓2;Activation

mapsareshownatathresholdofp<0.05, FWEpeak-levelcorrectedformultiple compar-isonsacrossthewholebrain(red)overlayedon activationthresholdedatp<0.05,FWE peak-levelcorrectedformultiplecomparisonswith aninitialCDTofp<0.001(orange). B)whole-brainactivationsby𝜓3;Activation

mapsareshownatathresholdofp<0.05, FWEpeak-levelcorrectedformultiple compar-isonsacrossthewholebrain.C)activationsby 𝜓2;D)activationsby𝜓3;C-D)Activationmaps

areshownata thresholdofp< 0.05,FWE peak-levelcorrectedformultiplecomparisons acrossthe anatomicalmask(red)overlayed onactivationthresholded atp< 0.05,FWE cluster-level correctedfor multiple compar-isonswithaninitialCDTofp<0.001(orange). WB:whole-brain;AM:anatomicalmask.

thismostwell-establishedlinkbetweenneurotransmittersand compu-tations. Forexample,recentworkin humans using invasive voltam-metrymeasurementsofsubsecond striatalDArelease(Kishida etal., 2016)failedtofindasimplelinkbetweenrewardPEsandDArelease. Instead,thisstudysuggestedthatstriatalDAreleasereflectsthe inte-grationofrewardPEswithcounterfactualpredictionerrors.More gen-erally,dopaminewasalsofoundtosupporttheformationof associa-tionsbetweenneutralstimuli,withoutanyreinforcement(Youngetal., 1998).Furthermore,followinghumanfMRIfindingsthatsuggestedthe signallingofsensoryPEsbymidbrainneurons(Iglesiasetal.2013),this hasbeencorroboratedbysubsequentstudies(Takahashietal.,2017; Stalnakeretal.,2019),leadingtothenotionthatdopaminetransients mightencodegeneralizedPEsthatarenotnecessarilytiedtorewards (Gardneretal.,2018).

BeyondPEs,therearenumerousstudiesindicatingthatdopamine might also signal uncertainty (or its inverse, precision) in a vari-ety of contexts (Fiorillo et al., 2003; deLafuente and Romo, 2011; Fristonetal.,2012;Hartetal.,2015;Schwartenbecketal.,2015). Alto-gether,thesepotentiallydiverserolesofdopaminecouldbeinterpreted asthepossiblereflectionofamultiplexingprinciple(Nakahara,2014; Gardner et al., 2018), where different computational quantities are broadcastin different frequencybands; this maybe linkedto differ-encesineffectsofphasicvs.tonicdopaminerelease(Grace,1991).An alternativecauseofdiversityindopaminefunctionisthemarked hetero-geneityofdopamineneuronswithregardtodevelopment,physiological propertiesandanatomicalprojectionpatterns(forreview,seeRoeper, 2013).

PEsanduncertainty/precisionarefundamentalcomponentsofmany theoriesoflearningandperceptualinference,andtheirsignallinghas alsobeenlinkedtoacetylcholine(YuandDayan,2005;Okadaetal., 2011; Moranetal., 2013;Vossel etal., 2014;Marshallet al.,2016; Naude et al., 2016), noradrenaline (Yu and Dayan, 2005; Payzan-LeNestouretal.,2013),andserotonin(Cohenetal.,2015).These at-temptstounderstandthephysiologicalimplementationsof computa-tionshavereceivedconsiderableattention, notleastbecauseoftheir clinicalimplications:Ifatightlinkbetweenneuromodulatorsand spe-ciﬁccomputationalquantitiesexisted,thismightenablethe develop-mentofcomputationalassaysthatinferredpathologicalalterationsof neuromodulatory transmitters from combined behavioural and neu-roimaginganalyses(Stephanetal.,2015;Iglesiasetal.,2017).

In aprevious fMRI study usinga sensory learningparadigm un-dervolatility,wefound(andreplicatedintwoseparatesamples)that twohierarchicallyrelatedPEswerereflectedbyactivityintwo neu-romodulatorysystems:low-levelprecision-weightedPEsaboutsensory outcomesactivatedtheputativedopaminergicmidbrain,whereas high-levelprecision-weightedPEsabouttheprobabilityoftheoutcome acti-vatedthecholinergicbasalforebrain(Iglesiasetal.,2013),specifically theseptum(Ch1-2complex;Zaborszkyetal.,2008).Theseresultswere intriguingsincetheysuggestedthefeasibilityofassayingtwoclinically relevantneuromodulatorysystemswithasingleparadigm.However,as highlightedinthispreviousstudy,itwasnotpossibletoconcludewith certainty thatthesefMRI resultstruly reflectedDA andAChsignals. This is becausethemidbrain andbasalforebrainarenotexclusively composedofdopaminergicandcholinergicneurons, respectively,but

(13)

Fig.8. Signiﬁcantpharmacologicalresultsin themidbrain

A)Galantamineascompared toplacebo in-creasedthebody-weightdependent𝛿1activity

inthemidbrain;theactivationmapisshown ata thresholdof p < 0.05,FWE peak-level correctedformultiplecomparisonsacrossthe anatomicalmask(blue)overlayedon activa-tionthresholdedatp<0.05,FWEcluster-level correctedformultiple comparisonswith an initialCDTofp<0.001(yellow).B) Amisul-prideascomparedtoplaceboincreasedthe 𝜓3-relatedactivityinthemidbrain;the

activa-tionmapisshownatathresholdofp<0.05, FWEcluster-levelcorrectedformultiple com-parisonsacrossthewholebrain,withan ini-tialCDTofp<0.001.Thedatawereextracted fromthesignificantpeakvoxelatthegroup level,removingtheeffectsofgroupand sleepi-ness(A)orremovingtheeffectsofthe covari-atesbodyweightandsleepiness(B), respec-tively.

containnumeroustypesofneurons,e.g.GABAergicandglutamatergic neurons(forreviews,ZaborszkyandDuque,2003;Duzeletal.,2009). Additionally,dopaminergicmidbrainneuronsreceivecholinergic aﬀer-ents(Bolametal.,1991;Oakmanetal.,1995;Yeomansetal.,2001; KobayashiandOkada,2007),andcholinergicbasalforebrainneurons receivedopaminergicinputs(Gaykemaand Zaborszky,1996).These twofactorscomplicatetheinterpretationofmidbrain/basalforebrain activationsconsiderably,asbothdopaminergicandcholinergic mecha-nismscouldexplainactivationofeitherregion.Toaddevenmore com-plexity,cholinergicmechanismscannot onlyalter theﬁringpattern ofDAneuronsinmidbrain(e.g.,Yeomansetal.,2001;Kobayashiand Okada,2007),butcanalsotriggerstriatalDAreleaseviapresynaptic re-ceptorsonmesostriatalprojectionsandindependentlysofromDA neu-ronactivity(Threlfelletal.,2012).

Thepresentstudyrepresentsaninitialattempttoscrutinizethe inter-pretabilityofourpreviousfMRIfindingsbyinvestigatinghow pharma-cologicalperturbationsofDAandAChwouldaltertherepresentation ofprecision-weightedPEsin dopaminergicandcholinergicnuclei.In humans,thiscanonlybetestedbysystemicadministrationofdrugs,in combinationwithhumanneuroimaging.(Unfortunately,asdiscussed below, this approach hasmany possible confounds andlimitations.) Here,fortwoseparatestudies,weusedabetween-subjectdesignwhere participantsinstudy1eitherreceived400mgoftheD2/D3-receptor antagonistamisulpride,4mg ofthemuscarinic(M1)receptor antag-onistbiperiden,orplacebo,andinstudy2either200mgofthe pro-druglevodopacombinedwith50mgofbenserazide,8mgofthe acetyl-cholinesteraseinhibitorgalantamine,orplacebo. Theidealresultwe hopedtofindwasthatlow-andhigh-levelprecision-weightedPE activ-ityinthemidbrainandbasalforebrainwouldbeselectivelyaffectedby pharmacologicalmanipulationsofDAandACh,respectively.

Thishypothesiswasnotsupportedbyourdata.Inbrief,wedidnot find clear-cutevidencefor adichotomybetween low-level precision-weighted PEs and DA on the one hand, and high-level precision-weightedPEsandAChontheotherhand.Thishypothesiseddichotomy didnotemergeeitherwhensplittingtheprecision-weightedPEsintheir PEandprecisioncomponents.Wedid,however,findeffectsofinterest acrosstheanatomicalROIsthatdeservediscussion.

First,wefoundabody-weightdependentpharmacologicaleﬀecton 𝛿1-relatedactivityintheSN/VTA:comparedtoplacebo,galantamine

enhancedthepositiverelationbetweenbodyweightandlow-levelPE (𝛿1)-relatedactivityinthisregion(Fig.8A).Morespeciﬁcally,δ1-related

midbrainactivitydecreasedwithhigherbodyweightintheplacebo con-ditionbutincreasedundergalantamine.Whilethecholinergiceffecton themidbrainisnotdirectlyin linewithourprevioushypothesisand thecurrentliteraturelinkingsensoryPEstomidbrainDAsignalling,the effectsof galantaminemight haveoccurredvia cholinergicreceptors on dopaminergicmidbrainneurons. Forexample,cholinergic projec-tionsfromPPTandLDTaffectDAneuron activityviabothnicotinic andmuscarinicreceptors(Zhouetal.,2003).Galantamineisan acetyl-cholinesteraseinhibitorenhancingAChlevelsandalsoactsasallosteric modulatorofpresynapticnicotinicAChreceptors,increasingreceptor sensitivity(NoetzliandEap,2013).Moststudiesexaminingthe activa-tionofnicotinicandmuscarinicreceptorsonDAmidbrainneuronshave foundexcitatoryeffects(e.g.DaniandBertrand,2007;Schilstrometal., 2007;Blahaetal.,1996;forreview,Zhouetal.,2003).Unlessthese effectsfollowanonlineardose-responserelationship,theyshould de-crease withhigherbody weight(and thus presumablylowerplasma levelsofgalantamine), whereaswefoundtheopposite.Activationof muscarinicreceptorsondopamineneuronsintheSN/VTAcanalso ex-ertinhibitoryeffectsonmidbraindopamineneuronsbutthismaybe

(14)

re-Fig.9. Signiﬁcantpharmacologicalresultsinputativelythecholinergicnuclei

A)Biperidenascomparedtoplaceboincreasedthe𝛿1-relatedactivityinPPT/LDT;B)biperidenascomparedtoplaceboreducedthe𝛿2-relatedactivityinPPT/LDT;

C)amisulprideascomparedtoplaceboreducedthe𝛿2-relatedactivityinPPT/LDT;D)amisulprideascomparedtobiperidenincreasedthebody-weightdependent

𝛿2-relatedactivityinPPT/LDT.ThefMRIresultsoverlayedonthestructuralimageareshownatathresholdofp<0.05,FWEpeak-levelcorrectedformultiple

comparisonsacrossthewholebrain(blue)anduncorrectedatthecluster-level,withaninitialCDTofp<0.001(yellow).Thedatawereextractedfromthe significantpeakvoxelatthegrouplevel,removingtheeffectsofthecovariatesbodyweightandsleepiness(A,B,C)orremovingtheeffectsofgroupandsleepiness (D),respectively.

strictedtophasiccholinergictransmission(FiorilloandWilliams,2000; butseeMillerandBlaha,2005).Inbrief,itisnotimmediatelyclearhow theobservedincreaseinδ1-relatedmidbrainactivitywithhigherbody

weight(andthuspresumablylowergalantaminelevels)canbe recon-ciledwiththeknowninteractionsbetweenthedopaminergicmidbrain andcholinergicPPT/LDT.

Asecondﬁndingwasthatbiperidenaﬀectedlow-levelPEs(𝛿1)and

thehigh-levelPE(𝛿2)inabrainstemregioncorrespondingtoputatively

thecholinergicPPT/LDT(Fig.9AandB,regionsoverlap).Theseeffects wereindependentofbodyweight,i.e.reflectingmeanpharmacological effectsonactivityin PPT/LDTrelatedtobothpredictionerrors.This ispartly compatiblewithourpreviousstudy(Iglesiasetal.,2013) – wherewehadfoundanactivationoftheprecision-weightedhigh-level PEinacholinergicregion,thebasalforebrain– andresultsfromrecent behaviouralstudiesthatusedtheHGFandcholinergicmanipulations (Vosseletal.,2014;Marshalletal.,2016).Thebiologybehind possi-bledopaminergicandcholinergiceffectsonPPT/LDTactivity,however, iscomplexanddeservesabriefdiscussiontoillustratethenumerous waysinwhichdrugscanaffectthesenucleiandtheirinteractionswith thedopaminergicmidbrain.PPT/LDTnotonlycontaincholinergic,but alsoglutamatergic,GABAergicanddopaminergicneurons(Pahapilland Lozano,2000; KobayashiandOkada, 2007; Okadaet al.,2011), re-ceive cholinergic, glutamatergic and GABAergicafferents (Ye etal., 2010),andprojectwidelytoother(sub)corticalregions,includingthe dopaminergicmidbrain(LodgeandGrace,2006;vonBohlenund Hal-bach&Dermietzel,2006;Mena-Segoviaetal.,2008;Okadaetal.,2011). ProjectionsfromtheSNtoPPTincludebothinhibitoryandexcitatory connections (Granata and Kitai, 1991; Okada etal., 2011), andSN neuronsareendowedwithmuscarinicreceptors(Vilaró et al., 1990; Leveyetal.,1991;ZubietaandFrey,1993).Furthermore,thePPT re-ceivesinputfromtheventralpallidumwhich,inturn,receives

GABAer-gicprojectionsfromthedopaminergicallyinnervatedventralstriatum (Leeetal.,2000).ThiscircuitryoﬀersampletargetsforDAandAChin aﬀectingPPT/LDTactivity.

Athirdﬁndingwasthatamisulpridesigniﬁcantlyincreased activa-tionoftherightmidbrainby𝜓3,theprecision-weightonthethirdlevel

ofourmodel(Fig.8B);thisparticularprecision-weightcorrespondsto thedegreeoftheagent’suncertaintyaboutthelog-volatility𝑥(₃𝑘) (i.e. theprecisionthatmodulatestheinfluenceofthehigh-levelPEon log-volatilityestimates).IntheplaceboconditionSNresponseswere nega-tivelyrelatedtotheparticipant’sprecision-weightatthethirdlevel,and this negativerelationdisappearedunderamisulprideandturnedinto apositiverelation.Theprecisionofbeliefsaboutactionselectionhas beenrelatedtodopaminergicprocesses(deLafuenteandRomo,2011; Schwartenbeck etal.,2015; Parr andFriston,2017),while expected uncertainty(i.e.thedifferencebetweenthedegree(conditional prob-ability) of cue validity and certainty) has been linked to choliner-gic signalling (Yu and Dayan, 2005; Iglesiaset al., 2013; Parr and Friston, 2017) and unexpected uncertainty (uncertainty about state transitions)tonoradrenergicsignalling(YuandDayan,2005; Payzan-LeNestouretal.,2013;ParrandFriston,2017).Herewefinda dopamin-ergicmodulationof𝜓3-relatedactivityinadopaminergicregion,

sug-gestingadopaminergicroleinsignallinghigherlevelprecisionestimates (asfurthershownbytheaverageeﬀectof𝜓3).

Oursecondhypothesisconcerningpharmacologicaleﬀectson rep-resentations of the computational quantities across the whole-brain was moreexploratory.Nevertheless,giventheprominenceof certain neuroanatomicallycharacterisedprojections,forthelower-level com-putational quantities (𝜀2, 𝛿1, and 𝜓2) eﬀects would have been

ex-pectedinmajordopaminergicprojectionregionssuchasthebasal gan-glia,hippocampus,amygdala,cingulateandfrontalcortex(Wise,2004; BentivoglioandMorelli,2005;Schultz,2012)andforthehigher-level