Graphical Causal Models for Empirical Validation: a Comparative Application to a DSGE and an Agent-Based Model

(1)

Universit`

a di Pisa

Scuola Superiore Sant’Anna

master of science in economics

Graphical Causal Models for Empirical Validation:

a Comparative Application to a DSGE and an

Agent-Based Model

Supervisor

Candidate

Prof. Alessio Moneta

Elena Dal Torrione

(2)

Any macroeconomic model attempting to explain real world phenomena must be em-pirically reliable, especially if the aim is to inform economic policy. Agent-Based and DSGE models can, indeed, be regarded as artificial economies which can be used as laboratories to conduct policy experiments. Hence, it is fundamental to assess the de-gree to which they are able to represent real-world mechanisms giving rise to observable phenomena. To this aim, a common approach is to evaluate a model’s ex-post ability to reproduce a set of stylized facts, that is, a set of robust statistical properties of actual data. This method is widespread in Agent-Based modeling (Windrum et al., 2007), but is also employed for DSGEs, even when they are estimated by means of sophisticated

econometric techniques (Fern´andez-Villaverde et al., 2016).

However, this is not a sufficiently severe test. As pointed out by Brock (1999), styl-ized facts are just “unconditional objects”, or statistical properties of a stationary pro-cess, which do not convey information on the underlying data generating mechanism. Similarly, Guerini and Moneta (2017) argue that different data generating processes might produce the same empirical patterns, and devise a new method of empirical vali-dation where vector autoregressive structures are estimated from real and artificial data by means of graphical-based causal search algorithms, and compared. Such algorithms are able to retrieve causal structures in the form of Directed Acyclic Graphs, under suitable assumptions (Pearl, 2000; Spirtes et al., 2000; Shimizu et al., 2006).

This study applies a method of empirical validation based on graphical models to the Agent-Based model “Schumpeter meeting Keynes” by Dosi et al. (2015) and to the DSGE model by Smets and Wouters (2007), and evaluates their performance comparatively. It takes the approach of Guerini and Moneta and adds a contribution in that it also compares causal structures as estimated with the PCMCI (Runge et al., 2018), a recent causal search algorithm specifically designed for time series analysis. This allows to capture non-linear features of data, by relaxing the linearity constraints imposed in the vector autoregressive setting.

The main results show that the models perform similarly in reproducing causal struc-tures as estimated from U.S. data for the period 1959-2014. Nonetheless, the Smets and Wouters tends to omit existing causal links more frequently than the “Schumpeter meeting Keynes”, which, on the contrary, is more prone to connect unrelated vari-ables. These findings are robust across different assumptions on the type of statistical dependence.

The work is organized as follows: Chapter 1 introduces the theoretical foundations of DSGE and Agent-Based models, highlighting the relative differences; Chapter 2 and 3 provide an overview of the current estimation and validation techniques for Agent-Based and DSGE models, respectively; Chapter 4 gives an extensive introduction to the theory of graphical models and describes in detail the causal search algorithms employed in the present study; in Chapter 5, we discuss the results of the analysis and conclude.

(5)

Chapter 1 DSGE and Agent-Based Models

1.1 DGSE Models

Large-scale macro-econometric models dominated the macroeconomics field until the seventies. Such models provided a description of the economy in the form of behavioral equations of aggregate quantities as derived from Neoclassical Synthesis’ theories. As a matter of fact, macro-variables were modeled directly, without any explicit description of the micro-mechanisms that should give rise to the theorized relationships.

In the light of their strikingly poor forecasting performance, Lucas (1976) put for-ward his famous critique. In his view, the failure of the Keynesian models was due to their lack of a rigorous micro-foundation, which led to the estimation of non-structural,

unstable parameters. Indeed, as he pointed out, agents form expectations and

re-act to policy so that non-structural parameters may change accordingly. In order to obtain robust policy implications, any macroeconomic theory should model directly pa-rameters regarding expectation formation, preferences, technological and institutional constraints, which are by nature policy-invariant. He claimed that a general equilib-rium framework would suitably serve the purpose, since the focus would be placed on optimization problems faced by individuals and firms. The Lucas critique was a major turning point in economic modeling and paved the way to a new wave of micro-founded models.

Real Business Cycles models (RBC), developed in the eighties, were among the first responses to the Lucas’ critique. RBC models aimed at representing the dynamic evolution of the economy as affected by random shocks, in a Walrasian framework and through first-order conditions derived from consumers and firms’ inter-temporal optimization problems.

RBC models adhered well to Lucas’ recommendation that a model be built as a “fully articulated artificial economy which behaves through time so as to imitate closely the time series behavior of actual economies” (Lucas, 1977). Policy experiments, that would be otherwise unfeasible in the real world, could be performed inexpensively in such artificial economies. Indeed, Kydland and Prescott (1982) showed that, after calibration, their model was able to reproduce a rich amount of business cycles stylized

(6)

facts so that it could be used as a laboratory for policy analysis.

RBC models stressed the importance of real factors, and, in particular, of technology shocks, in driving economic fluctuations and growth. By contrast, nominal factors were relegated to a negligible role: money was considered to be neutral in both the short and the long run. Since fluctuations were derived from agents’ utility optimization in a frictionless setting, RBC models suggested that the economy is always in a socially optimal state and, consequently, policy intervention is purposeless or, at worst, detri-mental. Indeed, the main criticisms of RBC models revolved around money neutrality and the irrelevance of monetary policy, which seemed to contradict the well-established positive correlation between output and money, as well as long-held beliefs about the effectiveness of central banks’ interventions (Snowdon and Vane, 2005, Chapter 6). In addition, the empirical performance of the RBC models rested on some implausible assumptions, such as high volatility of technology and the presence of technological

regresses1.

In the light of these criticisms and given the need for concrete policy evaluation, some researchers started to introduce New Keynesian elements to the RBC framework, leading to the development of the so-called New Keynesian DSGE models (NK DSGE), which quickly became, in recent years, the standard tool in central banks and govern-mental institutions.

This wave of DSGE models incorporate key Keynesian concepts such as the in-efficiency of aggregate fluctuations, nominal price stickiness, and the non-neutrality of money. With respect to previous Keynesian macro-models, however, such features are explicitly micro-founded and derived from an optimization problem, thus making DSGEs internally consistent and, at least in principle, policy-invariant.

Despite relevant differences, which we shall highlight in a moment, NK DSGE mod-els retain the basic structure of the early RBCs (Gal`ı, 2008):

• an infinitely-lived representative household, or agent, optimizes an inter-temporal utility function including consumption and leisure, subject to a dynamic budget constraint;

• a large number of firms produce a homogeneous good with identical technology and subject to exogenous stochastic shocks;

• agents are perfectly rational and forward-looking and form their expectations exploiting all available information without systematic bias.

The distinguishing feature of DSGEs is the presence of frictions and market

imperfec-tions. Among these, three are the key elements2:

1_{In general, in order to generate plausible fluctuations, some counter-factual assumptions were}

needed. For example, to produce large variations in employment in response to a technological shock, the substitution effect in the labor supply had to be very high compared to the income effect, a hypothesis not supported by the empirical micro-evidence (Snowdon and Vane, 2005).

(7)

• money: money enters DSGE models only as a unit of account and it has no independent effect on aggregate demand. Rather money supply is varied by the central bankers to influence the short-term interest rate. Hence, it is the interest rate that influence short-run fluctuations, while money balances do not have per se any effect on real variables.

• imperfect competition: firms have some market power and faces a downward sloping demand curve. Hence, they are able to set prices in order to maximize profits.

• nominal rigidities: prices are sticky, either because firms cannot always adjust prices at the desired frequency or because they face some adjustment costs. Sim-ilarly, also wages are sluggish.

In the phase of estimation, a number of other additional frictions and shocks of various nature are usually added to DSGEs in order to improve their fit with the data. A basic DSGE model is composed of three blocks: the aggregate demand-side block, represented by the New Keynesian IS equation, which is derived from the Euler equation of the representative agent; the aggregate supply-side block, represented by the NK Phillips curve, which is derived from the price-setting behavior of firms; a monetary policy rule, such as the Taylor rule, which describes how the central bank sets the interest rate (Romer, 2011).

Although DSGE models may vary a great deal, in general they share some important implications. Firstly, because prices are sluggish, money is not neutral in the short-run: the central bank can affect output and employment by setting the short-term interest rate, which is understood to be the main monetary policy instrument. The transmission of monetary impulses depends crucially on the expectations of the private sector about the future intentions of the central banker. Hence, expectations about economic variables are part of the structural equations, since agents are forward-looking. Secondly, although the economy is subject to a plethora of frictions, there exists a natural level of output and a natural value of the interest rate that correspond to the values that would arise in a frictionless (flexible price) economy. These natural levels are not constant, but may change in response to real shocks. Importantly, policy makers cannot create “persistent departures from the natural values without inducing either inflationary or deflationary pressures” (Gal`ı and Gertler, 2007). In other terms, policy interventions cannot alter output constantly without triggering some mechanisms that bring the economy back to the steady state.

DSGE models notoriously failed to predict the financial crisis of 2008 and the sub-sequent economic depression. In explaining their failure, critics have pointed out to theoretical as well as empirical issues. The theoretical criticisms are effectively summa-rized in Fagiolo and Roventini (2017).

As we have highlighted, the NK DSGEs rest on the neoclassical apparatus of RBC models. Its main assumptions have been challenged on formal and empirical grounds. Firstly, the representative agent (RA) assumption is problematic in many respects, since

(8)

aggregate behavior cannot always be assimilated to the behavior of a single individual: aggregate rationality does not necessarily follow from individual rationality; the reac-tions of the RA to shocks or parameter changes may differ from the aggregate reacreac-tions of the represented agents; the RA may have different preferences, even if all agents’ preferences are identical. Furthermore, the RA assumption eschews all considerations related to the presence of heterogeneity and interactions among agents, which not only undermines some basic properties of DSGEs, but also affects their policy implications. For instance, distribution of wealth cannot typically be addressed in DSGEs, while the lack of interactions prevents the study of financial and credit markets and of other settings where coordination failures emerge. Unfortunately, distributional and financial issues played a major role in the crisis of 2008.

Secondly, rational expectations have been questioned for being a too strong, em-pirically invalid assumption: agents unrealistically possess all the relevant information, know the “true” model of the economy and have incredibly sophisticated computation capabilities. Arguably, individuals are more likely to follow heuristics and routines in context of radical Knightian uncertainty.

Finally, in DSGE models, business cycles arise from the propagation of exogenous shocks. The economy is conceived as a system constantly in equilibrium that is per-turbed by some exogenous shocks, which propagate through the system and eventually die out. Arguably, shocks are produced also endogenously as a natural consequence of the very functioning of the economy, as the subprime mortgage crisis has clearly shown (Fagiolo and Roventini, 2017).

Although many endeavors have been made in the after-crisis years to remedy these shortcomings, such as including financial frictions and some features of bounded ratio-nality and heterogeneity, DSGEs are still left wanting. Such new elements are often just superimposed to the models, by resorting once again to exogenous shocks or ad hoc assumptions on individual behavior, without altering the basic framework. In this sense, they just “scratch the surface” (Fagiolo and Roventini, 2017) of problems, not really addressing the core of the criticisms.

1.2 Agent-Based Models

In recent years, a new strand of macroeconomic models, called Agent-Based Models, have emerged, which embraces a radically different view with respect to the traditional DSGE framework, and attempts to provide a satisfying answer to the issues mentioned

in the previous section3_{. Agent-Based Models are the tools of a new paradigm known}

as Agent-Based Computational Economics (ACE), defined as “the computational study of economic processes modeled as dynamic systems of interacting agents” (Tesfatsion, 2006). According to this definition, economies are conceived as complex evolving sys-tems in which a multitude of agents interact with each other in a non-trivial way. The central aim of the ACE community is to develop macroeconomic models that are

(9)

micro-founded on realistic, credible assumptions on individuals’ behavior. Hence, the over-simplifying assumptions of perfect rationality and representative agent are shed in favor of behavioral rules which are supported by empirical evidence. No top-down coordination mechanism is imposed: macro-properties result from the repeated inter-actions of heterogeneous, boundedly rational individuals who have access to limited information and are engaged in a learning process of trial and error.

The approach taken by the ACE is historically rooted in evolutionary economics (Dosi and Nelson, 1994), and in the research program of the Santa Fe Institute.

Evolutionary economics conceives economies as evolving entities, where a selection mechanism operates over a population of heterogeneous and interacting firms. These produce endogenous innovations and are recognized to be only imperfectly rational. Since they face problems whose complexity prevents the formulation of efficient solu-tions, firms resort to simple heuristics and routines whose performance is robust across similar situations. Simulation is the tool employed by evolutionary economics to ex-plore industries and firms’ dynamics. While the evolutionary economics’ approach is limited in scope to settings in which selection plays a major role, the ACE approach can be seen as a generalization of such perspective to a wider range of scenarios.

The Santa Fe Institute is a private research and education center founded in 1984 in Santa Fe, New Mexico, whose objective is to investigate complex adaptive systems through a multidisciplinary approach. Complex adaptive systems are defined in the manifesto of Santa Fe as “systems comprising large numbers of coupled elements the properties of which are modifiable as a results of environmental interactions”. At the time of its foundation, the driving motivation behind the Santa Fe’s work was that nat-ural and social systems, by virtue of their inherent complexity, could be studied from a unified perspective, the so-called science of complexity, and that, therefore, different fields could benefit from ideas and methodologies coming from other disciplines. The Santa Fe’s initiative led to a consensus around the fundamental features of economic models, which are to include cognitive and structural foundations, adaptation, nov-elty and out-of-equilibrium dynamics. Moreover, its success was corroborated by the development of a platform, called SWARM, where agent-based models could be imple-mented relatively easily. Of course, this was made possible by the continual increase in computational power, which allowed to simulate complicated ABMs on small Personal Computers.

Given these premises, let us give a complete definition of what is meant by Agent-Based Models. They are models (i.e., abstract representations of the reality) in which (i) a multitude of objects interact with each other and with the environment, (ii) the objects

are autonomous, and (iii) the outcome of their interaction is numerically computed4_.

The term “agent” comprises a wide variety of entities, such as households, firm and public institutions.

ABMs share the following key features (Fagiolo and Roventini, 2017):

• Bottom-up perspective: ABMs models adhere to the idea that ”if you didn’t

(10)

grow it, you didn’t explain it” (Epstein, 1999). In ABMs, aggregate properties are emergent, i.e. they stem from micro-dynamics. In this sense, the ACE com-munity embraces the view typical of methodological individualism in its holistic declination: any social phenomena should be explained in terms of individuals’ actions, but the aggregate does not coincide with the sum of its parts. The system as whole, instead, also determines how the parts behave. In ABMs, the “whole” is computed as a sum of the part, but it may materialize as a distinct entity (Delli Gatti et al., 2018).

• Heterogeneity: Agents will typically differ in their preferences, endowments, lo-cation and abilities. Heterogeneity is attained by assigning to agents relevant parameters drawn from pre-specified distributions.

• Direct interactions: agents interact with each other by exchanging commodities, information, experiences and even expectations. Such interactions are direct and local, depending on the network structure in which agents operate.

• The evolving complex system (ECS) approach and selection: economic systems are complex and evolving over time. Moreover, selection mechanisms operate in the economies.

• Bounded rationality and learning: agents have limited cognitive capabilities and have access just to a small portion of information. They form adaptive expecta-tions and learn from past experiences, as well as from other agents. These are intrinsic features of economic systems since agents need to adapt to a mutable environment subject to innovations.

• Non-linearity: interactions are inherently non-linear. Moreover, mutual feedback between the macro level and the micro level adds further non-linearities to the models.

• True dynamics: history matters in ABM models. The state of the system is computed from the preceding period, in clear contrast to neoclassical models where the present state of the economy depends on expectations about the future. • Endogenous and persistent novelty: novelty and new patterns of behavior are constantly introduced. Importantly, such innovations are endogenous, i.e. they are produced within the system.

It is evident from this framework how ABMs are models of non-equilibrium. As a matter of fact, the very idea of neoclassical equilibrium is rejected. The interest here is rather on regularities, i.e. whether the model is capable of generating some stable patterns or statistical equilibria. Stable does not mean that the system shall display such properties indefinitely: regularities may be permanent or transient. Anyhow, they should be typical of specific circumstances.

(11)

A standard ABM starts from a population of agents of possibly variable size.

Ini-tially, each agent is endowed with a finite number of micro-economic variables xi,0

and by a vector of fixed micro-economic parameters θi . Some time-invariant

macro-parameters Θ may be specified, as well. The system is observed in discrete time. At each time step t > 0, a subset of agents are selected to collect their available infor-mation about the current and past state of their local environment as well as of the whole economy. Such knowledge is plugged into heuristics and other behavioral rules. In this way, agents update their micro-economic variables, which are fed into the sys-tem. In each period, one can then compute common aggregate variables such as GDP, unemployment, investment, consumption etc., by taking the sum or the average of mi-cro variables. This procedure allows to observe a full dynamics over the entire period of observation t = 1, ..., T . Given the stochastic components of behavioral rule and interactions, such dynamics qualifies as a Markovian stochastic process, which usually does not possess closed-form analytical expressions due to high non-linearities. Hence, ABMs are typically analyzed by means of repeated simulations, which allow to build empirical distributions of key statistics and to explore how the behavior of the system change when initial conditions and parameters are varied (Fagiolo and Roventini, 2017). In conclusion, ABMs appear to be very flexible tools. Since they do not impose any consistency requirement (optimization, rationality etc.), different assumptions can be easily embedded in ABMs, without consequences for the analysis. Such theoretical flexibility allows ABMs also to replicate a wide range of stylized facts, including micro-economic stylized facts, an option that is not viable for NK DSGEs where the micro-level is flattened onto the macro-level.

Flexibility, however, is also the critical aspect of ABMs. Since, in principle, assump-tions can be made as accurate as desired, models can be built in a way as to describe almost perfectly the real world. Yet a complete model, which is nearly a one-to-one map of the real world to itself, would be of little use (Delli Gatti et al., 2018).

A closely related issues is that ABMs are usually over-parametrized. Hence, a

researcher is often left with many free parameters. This casts doubts on whether a model which reproduces some stylized facts is actually minimal with respect to those stylized facts. Another controversial point is how one should treat free parameters. One way would be to eliminate any degree of freedom by perfectly calibrating all parameters, but a counter-argument could be that policy analysis should be robust to different parametrizations. Finally, being still in its infancy, to this day, the ACE paradigm lacks a widespread agreement on common rules and empirical validation techniques, in contrast to the DSGE tradition, with the result that models comparability is often hindered (Fagiolo and Roventini, 2017). However, many efforts are being made in the ACE community to produce a convergence of theories and methods.

(12)

1.3 The Models

In this section, we will present the two models under analysis. The first one is a popular New Keynesian DSGE, developed by Frank Smets, an economist at the ECB, and Raf Wouters, an economist at the National Bank of Belgium. The Smets and Wouters (SW) model have quickly become a modern workhorse for policy analysis both in European and foreign central banks, the related paper being one of the most cited in recent years. The second one is the Agent-Based “Schumpeter meeting Keynes” model (K+S) of Dosi et al. (2010, 2013, 2015) of Sant’Anna School of Advanced Studies. The model includes a banking sector and a public sector and has been designed with the ultimate aim of evaluating the impact of different fiscal and monetary regimes.

1.3.1 The Smets and Wouters model

The Smets and Wouters (2007) extends on a previous model, the Smets and Wouters (2003).

It features a continuum of households which maximizes a non-separable utility func-tion over an infinite horizon with respect to two arguments: consumpfunc-tion, which is relative to a time-varying external habit variable, and labor. The presence of a union allows for some monopolistic power over wages and for the introduction of sticky nom-inal wages. Households supply labor and capital rental to firms, which, in turn, decide

the quantity of labor and capital to use. Firms set prices `a la Calvo. Prices and wages

that in each period are not re-optimized are partially indexed on past inflation.

The model is solved by log-linearization of variables around the steady-state. The demand-side of the model is described by the following equations. The first one is the resource constraint:

yt= cycty + iyit+ zyzt+ gt (1.1)

Output yt is divided among consumption ct, investment it, capital-utilization costs

zt, and exogenous spending

g

t. cy and iy are the steady-state ratio to output. zy =

Rk

∗ky, where Rk∗ is the steady-state rental rate of capital and ky is the capital-to-output

ratio. The exogenous spending captures governmental spending and net exports. It is assumed to follow an AR(1) process with IID-Normal errors plus a term depending on productivity shocks:

g_t = g_t−1+ ηg_t + ρgaηta

The Euler equation for consumption is given by :

ct = c1ct−1+ (1 − c1)Et(ct+1) + c2(lt− Et(lt + 1)) + c3(rt− Et(πt+1) + b) (1.2)

Hence, c1, c2, c3 depends on structural parameters γ, λ and σc, i.e. the steady-state

growth rate, an habit formation parameter, and the intertemporal rate of substitution between labor and consumption, respectively. A value of λ 6= 0 allows current consump-tion to depend also on past consumpconsump-tion. For the remaining part, current consumpconsump-tion depends on expectation about future consumption and on work hours growth, as well

(13)

as on the ex-ante interest rate plus as error term following an AR(1) process. This latter term is a “risk premium” shock that affects the willingness of households to hold bonds.

The Euler equation for investment is given by:

it= i1it−1+ (1 − i1)Et(it+1) + i2qt+ it (1.3)

where i1 and i2 depend on the the steady-state elasticity of the capital adjustment cost

function, φ, and on the discount factor applied by households β. Hence, investment is

a function of past and expected investment and of qt, the real value of capital, which,

in turn, depends positively on its expected future value and on the expected real rental rate, and negatively, on the ex-ante real interest rate and the risk premium disturbance.

The supply side of the model is determined by a production function:

yt = φp(αkts+ (1 − α)lt+ at) (1.4) where ks t = kt−1+ zt, zt= z1rkt (1.5)

Output is produced by a combination of capital and labor, where φp represents fixed

costs and a

t is the total factor productivity following an AR(1) process. Capital is

effective after one-quarter, hence, in each period k is given by lagged capital and by

the degree of capital utilization zt. Capital utilization, in turn, is a positive function of

the rental rate of capital, since households minimize costs, where z1 controls the costs

of changing the utilization of capital through the parameter ψ. The law of capital accumulation is given by:

kt= k1kt−1+ (1 − k1)it+ k2it (1.6)

So capital accumulation depends on investment and on a disturbance term that repre-sents the relative efficiency of investment expenditures. For the remaining part, a set of equations describes the price mark-up formation, the rental rate of capital and the wage mark-up. µp_t = mplt− wt r_tk= −(kt− lt) + wt µk t = wt− mrst (1.7)

where mplt and mrst are the marginal product of labor and the marginal rate of

sub-stitution between labor and consumption. Since prices are Calvo-sticky and partially indexed on past inflation, one can obtain the following New-Keynesian Phillips curve:

πt= π1πt−1+ π2Et(πt+1) − π3µpt +

p

t (1.8)

where the disturbance term is an ARMA(1) process. π3 depends on ξp, which measures

the degree of price stickiness. The higher ξp the slower is the speed of adjustment of

(14)

Finally, the model is complemented by a monetary policy rule: rt = ρrt−1+ (1 − ρ){rππt+ ry(yt− y p t)} + r∆y[(yt− y p t) − (yt−1− y p t−1)] + r t (1.9)

which is a generalized version of the Taylor rule. The authority adjusts the interest rate according to the inflation and the output gap, defined as the difference between actual inflation and output with respect to their flexible-price counterparts. The third term in Eq. 1.9 represents a short-run feedback from the change in the output gap. The disturbance term is, once again, an AR(1) with IID-Normal errors. In sum, the system features 14 endogenous variables and seven driving exogenous shocks.

The model is estimated with US quarterly data for the period 1966:1-2004:4 on seven observables (the log difference of real GDP, real consumption, real investment, real wage, log hours worked, the log difference of the GDP deflator and the federal

funds rate) by means of Bayesian techniques5_{. Some parameters are not identified,}

hence, they are fixed at a certain value.

The main findings of SW are summarized as follows:

• The productivity and the wage mark-up shocks are the most persistent. In the long run, these two shocks together explain most of the forecast error variance of real variables.

• On the other hand, the persistence and the standard deviation of the risk premium and monetary policy shock are relatively low.

• Nominal frictions `a la Calvo seem to play an important role, while past indexation

does not. Moreover, among real frictions, the most relevant for the model fit are investment adjustment costs and habit formation, while the least relevant is variable capital utilization.

• The DSGE forecasting performance is evaluated against that of an unrestricted VAR and a Bayesian VAR, by comparing the marginal likelihood and the Root Mean Squared Error (RMSE). At different horizons, the DSGE does as well as, or outperforms, the BVAR.

• The error variance decomposition highlights that, in the short run, movements in real GDP are primarily driven by the exogenous spending shock, the risk premium shock and the investment-specific technology shock. However, in the long run, supply-side productivity and wage mark-up shocks are the prevalent sources of variations. Monetary shocks account for little variation at every time horizons. • In general, price and wage mark-ups are the most important drivers of inflation,

while monetary policy shocks contribution is quite small, although in some period (e.g. the Volcker period) the effect is stronger. The model replicates well the dy-namic cross-correlations between output and inflation, which appear to be mainly driven by the price and wage mark-up shocks, and only marginally by monetary shocks.

(15)

1.3.2 The Schumpeter meeting Keynes model

The Schumpeter meeting Keynes (K+S) model (Dosi et al., 2015) is an extended ver-sion of two previous evolutionary Agent-Based models (Dosi et al., 2010, 2013). Its distinguishing characteristic is that it combines Schumpeterian theories of firm-specific, endogenous innovations with typical Keynesian features of demand-generation. Such framework sheds light on the mutual interaction of supply-side factors with demand-side factors and on their effect in the short and in the long run.

The agents populating the economy are divided in F1 machine-producing firms, F2

consumption-good firms, LS _{consumers/workers, B commercial banks, a Central Bank}

and a public sector.

In the supply side of the economy, firms in the capital-good industry produce hetero-geneous machines using labor as the only input. They introduce innovations or imitate competitors to augment labor productivity or to reduce costs, by investing part of their revenues in research activity. Consumption-good firms buy machines and combine them with labor to produce a homogeneous good. They plan their production and inventories on the basis of the expected demand, which is formed by backward-looking, and invest in new machines when the capital stock is insufficient or obsolete.

Machines’ prices and productivity are advertised by suppliers, so the choice of new capital goods is affected by imperfect information. Moreover, consumption-good firms can invest using internal or external resources. However, since capital market are as-sumed to be imperfect, banks do not have perfect information, hence they may restrict credit to some firms. The maximum amount of credit that a firm can receive is given by a loan-to-value ratio, while the cost of credit depends on the interest rate set by the Central Bank and on the credit worthiness of the firm. As a consequence, firms prefer using internal resources first, before turning to banks’ loans.

Here prices are set by applying a mark-up which varies according to the market share. Profits are given by revenues minus costs:

Πj,t = Sj,t+ rDN Wj,t−1− cj,tQj,t− rdebj,t Debj,t (1.10)

where Sj,t = pj,tDj,t are sales, cj,tQj,t are production costs, and rj,tdebDebj,t is the total

amount of debt expenses. rD_{N W}

j,t−1 are the interests received on firms’ net liquid

assets. N W are defined by the following equation:

N Wj,t = N Wj,t−1+ (1 − tr)Πj,t− cIj,t (1.11)

where cIj,t is the amount of invested internal resources and tr is the tax rate. A

firm is eliminated from the market if it has near zero market share or goes bankrupt

(N Wj,t < 0).

The banking sector is composed of banks which are heterogeneous in the number of clients and in other characteristics. They maintain both mandatory buffers and a

strategical buffer and supply credit according to the value of their equity N Wb

(16)

to their level of financial fragility Levk,t−1 , as defined in the following equation: T Ck,t = N Wb k,t−1 τb_{(1 + βLev} k,t−1 (1.12) Banks rate their customers’ credit worthiness and divide them into percentiles q, on the base of which they determine the risk premium:

rdeb_j,t = r_tdeb(1 − (q − 1)kconst) (1.13)

where rdeb

t is the Central Banks interest rate augmented by a mark-up. The Central

Bank determines its interest rate according to a Taylor rule, which is sensitive to the output gap and the employment gap, defined relatively to target levels:

rt= rT + γπ(πt− πT) + γU(UT − Ut) (1.14)

The profits of banks are given by:

Πb_k,t =

Clk

X

cl=1

r_cl,tdebDebcl,t+ rresCashk,t+ rbondst Bondsk,t− rDDepk,t− BadDebtk,t (1.15)

where rres is the interest rate received on deposits with the Central Bank, Cashk,t are

liquidities, rbonds _{is the interest rate on governmental bonds, r}D_Dep

k,t are the interests

banks pay to depositors, and BadDebtk,t are ill-performing loans.

The net worth of a bank is given as the sum of its loans, its liquidities, its bonds and after-taxes profit minus its liabilities (deposits):

N W_k,tb = Loansk,t+ Cashk,t+ Bondsk,t− Depok,t+ N etΠbk,t (1.16)

A bank goes bankrupt when N W_k,tb < 0. In that case, the government sector intervenes

and bails out the bank at a cost, Gbailoutt.

Consumption is given by the sum of the wages wt of employed workers and the

subsidies Gt provided by the government to unemployed workers:

Ct= wtLDt + Gt

wu

t = φwt

Gt= wtu(LDt − LSt)

(1.17)

The government spends resources for repaying debts, for providing subsidies and to save banks. It collects taxes from firms and consumers. Its deficit is given by :

Deft= Debtcostt + Gbailoutt+ Gt− T axt (1.18)

The model so defined is then calibrated and empirically validated. It is shown to be able to replicate a wide variety of macro and micro stylized facts, including

(17)

self sustained-growth, relative volatilities, cross-correlations, firms size distributions, distributions of banking crisis duration and so on.

Lastly, the model is used to evaluate different monetary and fiscal policy rules. These include: the 3%-deficit rule of the European Stability and Growth Pact; the debt-reduction rule of the Fiscal Compact, which requires the ratio of public debt on GDP to be 60%, or, otherwise, to be reduced at a rate of 5% of the difference between the current and target level every year; the lender of last resort, where the CB keeps a low level of the interest rate in order to limit debt costs for the government. These rules can be combined with a single or double-mandate of the CB, i.e., the CB seeks only price stability or both price and output stability. In the benchmark model, the Central bank sets the interest rate only according to the inflation gap, while, on the fiscal side, the deficit is free to fluctuate.

The main findings are that austerity fiscal policy are self-defeating and contributes to a great extent to the deterioration of economic activities and public finances. More-over, a Taylor rule based just on inflation does not stabilize output sufficiently, but it actually worsen the state of the economy when paired with austerity rules. Instead, unconstrained fiscal regimes coupled with a double mandate of the Central Bank sta-bilize output, employment and inflation. The negative effect of austere fiscal policies is exacerbated when income inequality is high. Importantly, aggregate demand shocks do not have just short-run effects, but may influence output and employment in the long run, as well.

(18)

Chapter 2 Empirical Validation of

Agent-Based Models

The aim of a typical Agent Based Model is to approximate the mechanisms that operate in the real world and give rise to observable phenomena. The real-world data generating process, which we denote by rwDGP, is largely unknown and it can be conceived as a very complicated process of which we observe just one realization.

In essence, validating empirically a model consists in assessing its ability to represent the rwDGP. In other words, any validation procedure attempts to answer the question of how good the model is in resembling the phenomenon of interest. As a consequence, the empirical validity of a model is taken also as an indirect proof of its substantive or theoretical validity. In most cases, such assessment does not usually give a clear-cut answer. A model may mirror some characteristics of the real world and fail to replicate others. Its acceptance closely depends on the chosen validity criteria.

The usual path taken in the ACE community is to test whether the outputs of a model are in tune with some real-world stylized facts, which the modeler intends to explain. Stylized facts usually comprise distributional properties of variables, co-movements among macro-aggregates, micro-regularities and so on. The agreement of a model to reality can be assessed on four levels (Axtell and Epstein, 1994):

• Level 0: the model is a mere caricature of reality.

• Level 1: the model is in qualitative accordance with macro stylized facts. • Level 2: the model is in quantitative accordance with macro stylized facts. • Level 3: the model is in quantitative accordance with micro stylized facts. However, ABMs’ intrinsic characteristics, such as non-linearities, complex interac-tions and the unavailability of analytical soluinterac-tions for their probabilistic laws of motion, render the task of empirical validation particularly challenging. Recent years have seen the flourishing of a variety of approaches aimed at tackling issues related to empirical validation of ABMs (Fagiolo et al., 2017).

(19)

Although, in the present work, we will mainly be concerned with output validation, it must be highlighted that other types of validation exist.

Concept validation regards whether a model is consistent with the theory it is meant to represent. For instance, in a solvable model of equations, the modeler has to make sure that the theory’s implications are well represented by the mathematical expressions and that no contradicting conclusions can be drawn from them. In simulation models, such as ABMs, this also implies a further step, that is program validity, which aims at ensuring that the computer program implementing the model works as planned.

Empirical validation can also be referred to input validation. Given its strict con-nection with output validation, this topic requires a more careful examination.

2.1 Input Validation

Inputs are the building blocks of a model. They can be divided into two broad cat-egories: assumptions about the behaviors and the interactions of agents, on the one hand, and choice of parameters and initial conditions, on the other.

Inputs validation is performed by evaluating the degree of realism of such assump-tions, i.e., by checking whether they are supported by empirical evidence.

Calibration and estimation can be considered forms of input validation, as they serve the purpose of reducing the distance between actual and model-generated data. Since calibration and estimation usually involve an assessment of the empirical performance of a model, such procedures are also regarded as forms of output validation. For this reason, the discussion on these two techniques is postponed to the next section.

Clearly, input validation is a fundamental task for ACE modelers. As highlighted in Chapter 1, ABMs have been developed as alternatives to mainstream neoclassical models, on the grounds that they are founded on patently unrealistic assumptions. The aspiration of the ACE community is, by contrast, to provide models that are micro-founded on more plausible premises. Nonetheless, ABMs have been sometimes criticized by neoclassical exponents on the very fragility of their assumptions. Hence, the need for serious and convincing procedures to input validation (Delli Gatti et al., 2018, Chapter 8).

At present, a variety of approaches have been followed in the literature.

Controlled laboratory experiments on human behavior have been used, mainly in small scale ABMs, as a basis for assumptions selection. Experiments allows researchers to understand how agents behaves in controlled environment, as well as dismissing some behavioral rules as not conforming to reality (Delli Gatti et al., 2018, Chapter 8).

In some large scale, more complex ABM, where agents’ behavior is not readily testable, agents act according to adjustment heuristics, i.e., they adapt decisions and expectations based on observed outcomes of past behaviors. For instance, in a simple AB model by Gallegati et al. (2008), firms decide production and prices on the basis of the expected demand for their goods, which is formed by looking at the actual demand in the period before, and change their plans according to the realized outcome.

(20)

In other approaches, behavioral assumptions are drawn from the management liter-ature, especially for models of industries and sectors, or from empirical micro-economic evidence. This approach is followed in the “Schumpter meeting Keynes” model (Dosi et al., 2015).

2.2 Output Validation

A procedure for output validation can be schematized as in Fig. 2.1 (Windrum et

al., 2007). Let (z)i = {zi,t, t = t0, ...t1, i ∈ I} be the observed individual data in a

population of agents I over a set of finite time periods {t0, ..., t1} and let Z = {Zt, t =

t0, .., t1} be the K-vector of observed aggregate time series. Assuming that the

real-world system is ergodic, a set of statistics S = {s1, sw, ...} is computed from observed

data.

The AB model is initialized with micro and macro parameters and initial conditions

(x_i,0, θ, Θ, X₀) and the same set of statistics is then computed on the data generated by

the model’s data generating process (mDGP) {xit, t = 1, ..., T } and {Xt, t = 1, ..., T }.

At each run (m = 1, 2, ..., M ), the simulation will output a different value for each

statistics sj. Because the simulations are independent, one can construct the Monte

Carlo distribution of sj and compute its statistical moments. The statistics and their

properties, however, will be dependent on the initial condition and parameters. There-fore, the procedure should be repeated for a sufficient number of points in the space of initial conditions and parameters, in order to gain insight on the model inner func-tioning and to assess the robustness of the model’s implications. The distance between model-based and empirical moments can be evaluated by means of statistical tests.

As anticipated, calibration and estimation are the main methods to validate ABMs. More specifically, both procedures restricts the parameter space in a way as to minimize the distance between the model and the real-worlds phenomena. In this sense, output validation is used instrumentally for discriminating among different parametrizations of a model. Only later, once the relevant parameters have been singled out, output validation provides a measure of how distant the model and reality still are (Delli Gatti et al., 2018).

The ACE community has been following a variety of procedures to output vali-dation, mainly based on frequentist approach, although also Bayesian approaches are currently emerging. We provide a concise description of the most popular methods, by distinguishing between calibration techniques and estimation techniques.

(21)

Figure 2.1: A procedure for model validation (Windrum et al., 2007)

2.2.1 Calibration of ABMs

The most common approach to ABM empirical validation is a four-step procedure,

known as the Indirect Calibration Approach1_{. The approach is called “indirect”,}

be-cause the model is validated first and then indirectly calibrated, by retaining the set of parameters capable of generating some stylized facts.

The first two stages of the procedure involve selecting a set of stylized facts and building the model with special attention to the credibility of the assumptions on the agents’ behavior and interactions. In the third step, the parameters are chosen accord-ing to available empirical evidence, so that the researcher is left with few combinations of parameters, or, so to speak, possible ”worlds”, which are able to produce proper-ties consistent with the observed data. The fourth step involves exploring this residual subset of parameters in order to shed light on the casual relationships embedded in the model. This last step is crucial in that it allows the researcher to draw new implications on the underlying mechanism of the model.

As emphasized in Windrum et al. (2007), the indirect calibration approach suffers from two major drawbacks.

First, because there is no guarantee that a model’s features are actually comparable to those of the observed reality, the parameters are usually not calibrated on their

(22)

empirical equivalents. Still, the stylized fact should be quite specific in order to enable the researcher to significantly restrict the parameter space, a necessary condition for providing any meaningful theoretical explanation.

Second, problems arise as to how different points in the restricted subspace of pa-rameters should be interpreted.

A second method to model validation is the Werker-Brenner approach, proposed by Werker and Brenner (2004). Contrary to indirect calibration, the Werker-Brenner approach attempts to directly choose the parameters according to their empirical coun-terpart: output validation is performed only after parameters have been selected. The procedure works as follows.

Firstly, plausible initial conditions and ranges of parameters are calibrated using empirical evidence. Secondly, output validation is conducted in order to restrict such ranges, possibly with the aid of Bayesian techniques. The distinguishing characteristic of the Werker-Brenner procedure, however, is that it uses abduction as inference rule. Basically, the resulting subset of the parameter space is further restricted by search-ing for common characteristics and features that arise from the remainsearch-ing parameters configurations. Indeed, shared features should represent some core aspects of the in-herent mechanism of the model and they may be expected to apply also to real-world phenomena. Also differences are inquired, as they may provide useful insights on the underlying structural process (Werker and Brenner, 2004). Such approach is grounded

on the school of thought, known as Critical Realism2_.

A third procedure is the so-called History-Friendly Approach , followed most notably

by Malerba et al. (1999) and Malerba and Orsenigo (2001). The history-friendly

approach advocates the use of a wide range of empirical evidence, such as historical case studies and anecdotal accounts, as a guide to build models and to identify the set of relevant parameters and initial conditions. The key feature of the approach, however, is that models are assessed on their ability to replicate the actual history of the real-world phenomenon, i.e. on how well the simulated trace history mirrors the true history. As such, the approach is mainly concerned with micro-economic phenomena, such as the evolution of particular companies and industrial sector.

The history-friendly is problematic in many respects. Firstly, historical evidence is collected for just few agents (firms), to the detriment of the universality of the model. Moreover, the available historical sources often do not provide sufficient or adequate enough information for calibrating the model. Secondly, the possibility to retrieve the correct set of parameters by backward-looking is questionable: different combinations of parameter might in principle, replicate the same actual trace history. Thirdly, the very reliance on history is controversial. Indeed, in order for such to effectively inform judgments on a model’s validity , historical accounts need to be accurate and impartial, but such sources are often subjective and imperfect. One way out would be to focus on a collection of histories, rather than a single story, so as to build models more general in nature (Windrum et al., 2007).

(23)

2.2.2 Estimation of ABMs

The lines between estimation and calibration are quite blurred and the two terms are often used interchangeably. However, though the point is debated, there is a subtle difference. Calibration techniques are mainly concerned with improving the fit of a model to observed data and the aim is simply to prove the plausibility of a model, without too much thought to the true value of the parameters of the rwDPG.

Estimation aims at estimating quantitatively the true parameters of the rwDPG through the minimization of an appropriate distance function. Therefore, estimation is also concerned with the properties of the estimators and the uncertainty surrounding the estimates. In other words, contrary to calibration, the model is considered to be approximately correct and able to yield information about the true parameters of

real-world processes3_.

In order to appreciate how calibration differs from estimation, it must be noted that, in many cases, the usual quadratic distance function employed in calibration procedures yields inconsistent estimates of the “ true” value of parameters. For instance, in the

history-friendly approach the simulated path yt(θ) is compared to the actual path yR,t

by minimizing their quadratic distance. The path estimator is given by: ˆ θ = argmin T X t=1 [yt(θ) − yR,t]2 (2.1)

which asymptotically tends to ˆ

θinf = argmin{V (y) + V (yR) + [E(y) − E(yR)]2} (2.2)

If the distribution of both the real-world and the model path is exponential, i.e f (y) =

1 θe

−1

θy then the estimator will be inconsistent, as shown in Delli Gatti et al. (2018):

ˆ

θinf = argmin[θ2+ θ∗2+ (θ − θ∗)2] = θ∗/2 (2.3)

Estimation of ABMs is quite arduous, since, because of their inherent complexity, a closed-form solution of the likelihood function and of the moments’ conditions is not typically available, if not for very small, simple models. Hence, estimation relies mainly on simulation-based techniques which approximate numerically the analytic expressions of theoretical quantities.

A standard approach in the estimation of ABMs is called Indirect Inference (Gourier-oux et al., 1993), which has been developed specifically for models with intractable likelihood functions. The only requirement is that artificial data can be simulated from such models.

Despite some relevant differences, the logic of the Indirect Inference is based on the Method of Simulated Moments (MSM). In a nutshell, the MSM involves choosing a set of parameters in order to generate artificial time series, from which longitudinal

(24)

moments are computed. The latter are, then, combined into a distance function which is minimized over the parameters space. Usually, the MSM estimator is given by

ˆ

θ = argmin[µ∗(θ) − µR]W−1[µ∗(θ) − µR] (2.4)

where W is a weighting matrix that assigns more weight to less uncertain estimates (Delli Gatti et al., 2018).

The Indirect Inference approach makes use of an auxiliary model yt = f (β, zt)

whose estimation properties are well-understood. The auxiliary model is fitted on both real-world and artificial data and the structural parameters of interest are inferred by minimizing the distance between the auxiliary model’s parameters estimated from simulated data and the ones estimated from on real-world data. This is made possible since the parameters of the auxiliary model and the structural parameters are assumed to be related by a binding function, that maps θ into β (Gourierouz et al., 1993).

The Indirect Inference approach is more general than the MSM, which, at a closer look, is just a specific case of Indirect Inference where the auxiliary model is simply

yt= µ + ut (Delli Gatti et al., 2018, Chapter 9).

Generally, provided that the auxiliary model offers a good description of the data, its mis-specification does not hinder the correct inference of the structural parameters. In this sense, one is able to infer the correct value by an ”incorrect criterion” (Gourieroux et al., 1993).

Two important shortcomings of these methods shall be highlighted. First, both MSM and Indirect Inference might become computationally unfeasible when models’ complexity increases, since a high number of Monte Carlo simulations must be run for each parameter configuration.

Second, in the MSM, the choice of moments is somewhat arbitrary and might not adequately reflect the characteristics of a process, thus leading to biased estimates (Fagiolo et al., 2017). Similarly, Indirect Inference is valid only when the auxiliary model provides a good enough approximation of the process under inquiry, otherwise the parameters of interest cannot be correctly inferred (Delli Gatti et al., 2018).

2.3 Issues in ABM Validation

So far, we have highlighted a number of issues related to the specific validation methods for Agent-Based models. An overarching problem in ABMs’ empirical validation is that they are often over-parametrized. This renders their study particularly taxing

and might undermine their theoretical cogency, as well4. All methods described in

the previous section attempts to solve this issue by variously restricting the parameter space with mixed success: calibration methods often fail to yield a single model, but rather identify a plurality of possible “worlds”; on the other hand, estimation methods based on MSM or on Indirect Inference are affected by the arbitrariness of the chosen moments or by poor descriptive power of the auxiliary models.

(25)

Most recently, new techniques have been devised to overcome these drawbacks. For instance, Lamperti (2018) suggests using a measure of the goodness of fit based on information theory, the so-called Generalized Subtracted L-divergence (GSL-div), which, after conveniently symbolizing real and artificial time series, quantifies the degree of similarity in their patterns (trends, concavities and convexities, etc.). Therefore, the GLS-div does not suffer from the arbitrariness typical of the MSM and related approaches, and it has the additional advantage of scoring models on the basis of their whole simulated conditional distribution.

Having said that, there is one fundamental issue that equally affects all approaches: after any calibration or estimation procedure, models’ empirical validity is ultimately assessed on their ability to replicate a set of key stylized facts. However, as argued by Brock (1999), this is not a sufficiently severe test: stylized facts are just “unconditional objects”, i.e., they are properties of stationary distribution, and, as such, do not convey any information on the stochastic process that actually produced them.

As a matter of fact, several different causal mechanisms might be compatible with the same set of empirical patterns. Hence, while replication of empirical regularities should be regarded as a positive feature of a model, it is but a preliminary step and should not be taken in any way as a confirmation of the validity of the underlying causal mechanism. In this sense, “replication does not imply explanation” (Windrum et al., 2007). If anything, such test is informative in that it helps to narrow the number of possible models, especially if the set of empirical regularities is sufficiently large. Importantly, this issue is not circumscribed to Agent-Based models, but applies to any simulation economic model, including DSGEs, which are often, though not exclusively,

evaluated on their ability to mirror some stylized facts5_.

Of course, this rises the question of how one should, then, discriminate among competing models that replicate the same empirical patterns. One solution would be to validate also the micro-economic elements of models. Such approach, however, would require high-quality data, which are not always available (Windrum et al., 2007).

In response to this criticism, a promising method has been developed by Guerini and Moneta (2017), based on the comparison of real-world and model-generated causal structures. Since this approach will be followed in the present work for the analysis, we will provide a full, detailed description of the methodology.

2.3.1 Comparison of causal structures

The method of empirical validation proposed by Guerini and Moneta (2017) consists in evaluating the degree of similarity between real-world and model-based causal struc-tures.

As argued by the authors, since models incorporating dissimilar assumptions might as well reproduce the same set of empirical regularities, the mere ability of a model to replicate some stylized facts is not sufficient to prove the validity of its

(26)

ing causal mechanism. Anyhow, the latter is often of primary interest, especially if models are intended to inform economic policy. The approach of Guerini and Moneta addresses exactly this need for a more stringent criterion to assess the reliability and to discriminate among policy-oriented models.

The approach lies on the comparison of a structural Vector Autoregressive model (SVAR) estimated on real and simulated data. The key feature is that the SVARs are identified by means of causal search algorithms based on graphical models. The model-based structural coefficients so estimated are then compared to their empirical counterparts by means of some similarity measures.

Before proceeding with the description of the approach, two important differences with respect to the other methods shall be highlighted: firstly, there is no attempt here to map the model’s parameters into the coefficients of the SVAR. In this sense, it is not an estimation technique, although it could be used for guiding calibration and estimation procedures. Secondly, it differs from information-based criteria (Lamperti, 2018) in that it is mainly concerned with multivariate systems and their evolution, while the latter focus on the comparison of uni-variate time series. The steps of the

validation procedure are summarized in Fig. 2.2. Initially, K variables (v1, ..., vK)

Figure 2.2: The steps of the validation procedure (Guerini and Moneta 2017). are selected within the model under validation and two datasets are constructed, one

containing the observed realizations of the K variables (denoted by VRW ), the other

containing the realizations of M Monte Carlo simulations of the K variables (denoted

by VAB). The datasets’ dimensions are:

(

Dim(VRW ) = 1 × K × TRW

Dim(VAB) = M × K × TAB

(27)

The first step consists in rendering the two data sets uniform. The time series are

made same length, by removing the first TAB − TRW in each simulation, so that the

periods of observation coincide (TAB = TRW). The reason is twofold: first, this excludes

the effects of the transient periods which do not reflect the true causal mechanism of the model; second, keeping the whole length of simulated time series could affect lag selection because of variables persistence. Along with the length, also the magnitude of the time series is harmonized by applying a monotonic transformation, such as a logarithmic transformation, to the data.

The second step involves testing for the ergodicity of the model, i.e., whether the simulated observations are random draws from a multivariate stochastic process, and for statistical equilibrium, i.e., whether the model displays statistical properties that are time-independent. In a MxT matrix, whose rows contains the observations of the M run

for T periods, the rows are called samples and the columns are called ensembles. Ft(Yk)

denotes the empirical cumulative distribution function of an ensemble while Fm(Yk)

indicates the empirical cumulative distribution function of a sample. Ergodicity and statistical equilibrium are established by performing a Kolmogorov-Smirnov test on the following hypothesis:

Fi(Yk) = Fj(Yk), f or i, j = 1, ..., T i 6= j

Fi(Yk) = Fj(Yk), f or i = 1, ..., T j = 1, ..., M

(2.6) In the third step, a Vector-Autoregressive model of order p, denoted as VAR(p) is estimated on real-world and on simulated data. A VAR(p)is expressed in vector form as :

Yt = A1Yt−1+ A2Yt−2+ ... + ApYt−p+ ut (2.7)

where utis a zero-mean white noise process with covariance matrix Σu = E[utu0t], which

is, in general, non-diagonal. Since the error terms are typically cross-correlated, 2.7 is assumed to be the reduced-form of a Structural Vector Autoregressive model (SVAR) that includes the contemporaneous relationships among variables. A SVAR has the following form:

Yt= BYt+ Γ1Yt−1+ ... + ΓpYt−p+ t (2.8)

where B and Γi (i = 1, ..., p) are K × K matrices representing the contemporaneous

and lagged coefficients, and t is a K × 1 vector of error terms with diagonal covariance

matrix Σ = [t0t]. By moving the first term on the right-hand side to the left-hand side

of Eq. 2.8, the SVAR can be rewritten as :

Γ0Yt= Γ1Yt−1+ ... + ΓpYt−p+ t (2.9)

By multiplying Eq. 2.9 by Γ−1₀ , one obtains Eq. 2.7 , where Ai = Γ−10 Γi and ut = Γ−10 t.

The reduced-form coefficients matrices Ai can be consistently estimated. However, the

structural coefficients matrices are not identified and can be recovered only by imposing

some restrictions on Γ0.

In the fourth step, Γ0 is identified by applying to the matrix of residuals a causal

(28)

of algorithm yields a directed acyclic graph (DAG), whose nodes are variables and whose links represent a causal relationship. Absence of a link implies a zero in the structural

matrix Γ0. If residuals are normally-distributed, the so-called PC algorithm can be used

to derive zero-restrictions on Γ0 and the coefficients can be estimated by maximum

likelihood. If residuals are non-Gaussian, instead, one can perform an Independent

Component Analysis (ICA), which is able to identify Γ0 along with the independently

distributed error terms t. This relatively recent approach to SVAR identification has

been proposed by Swanson and Granger (1997), Bessler and Lee (2002), Demiralp and Hoover (2003), Moneta (2008) and Moneta et al. (2011). Graphical causal models and related algorithms, as well as their use in SVAR identification, are discussed extensively in Chapter 4.

In the fifth and final step, the validity of the model is assessed by comparing the

causal structures embedded in the SV ARRW and in the SV ARAB. Guerini and Moneta

propose a set of practical similarity measures highlighting different aspects of the causal

structures: a sign-based similarity measure, Ωsign_{, which compares the signs of the}

estimated structural coefficients; a size-based similarity measure, Ωsize_{, which compares}

the size of the causal effects as entailed by the weighted graph; and a conjunction

measure,Ωconj_{, which compares jointly the sign and the size of the causal effects.}

Any similarity measure is computed for each of the model’s Monte Carlo realizations

and averaged across. More formally, let γ_i,jkRW denote the (j, k) element of ΓRW_i for i =

0, ..., pRW, and γi,jkAB the (j, k) element of Γ

AB,m

i for i = 0, ..., pAB and for m = 1, ..., M .

Let pmax = max{pRW, pAB} denote the highest lag order between the RW VAR and

the AB VAR. Then, ΓRW

i = 0 for pRW < i < pmax if pRW < pmax = pAB, ΓABi = 0 for

pA < i < pmax if pAB < pmax = pRW. The first two similarity measures are built by

means of the following indicator functions:

ωsign_i,jk =

(

1 if sign(γ_i,jkRW) = sign(γ_i,jkAB)

0 if sign(γ_i,jkRW) 6= sign(γ_i,jkAB)

ωsize

i,jk =

(

1 if (γAB

i,jk) ∈ [γi,jkRW − 2σ(γi,jkRW), γi,jkRW + 2σ(γi,jkRW)]

0 if (γAB

i,jk) ∈ [γi,jkRW − 2σ(γi,jkRW), γi,jkRW + 2σ(γi,jkRW)]

(2.10)

For Ωconj, the indicator function takes value 1 if the coefficient has the correct sign and

fall into the specified range around the RW value.

The similarity measures are simply the sum of the indicators functions for all vari-ables and for all lags, divided by the total number of elements. Hence, they represent the percentage of coefficients’ signs and sizes (or both) correctly identified by the model with respect to the real-world. They are defined as follows:

Ωh = Ppmax i=1 PK j=1 PK k=1ω h i,jk K2_p max (2.11) where h = {sign, size, conj}.

(29)

Chapter 3 Empirical Validation of DSGE

Models

DSGE validation takes essentially two forms: estimation of the structural parameters and evaluation of the goodness of fit of the models with respect to some empirical facts or of their forecasting performance relative to a-theoretical models. While earlier RBCs were mostly calibrated, a rich literature has flourished in the last thirty years on identification and formal estimation procedures of DSGEs. The task is not an easy one: the models are usually quite complicated so that a closed-form solution does not exist. Hence, a common strategy is to find an approximate solution by log-linearizing the variables around the steady state or by other methods, such as by perturbation. Solved DSGEs can be mapped into a state-space model, from which it is possible to obtain a likelihood function of the observable variables by applying some filters, such as the Kalman Filter, for linear, Gaussian models, or the particle filter, for non-linear models. The fact that DSGE models deliver a well-defined likelihood is the key difference with respect to Agent-Based models.

Even so, however, DSGEs’ likelihood functions are most frequently ill-behaved: presence of local minima or insufficient curvature in the relevant dimensions renders the identification of the parameters fraught with difficulties. Hence, some procedures, called limited-information estimation, have been developed which do not rely on the like-lihood function for parameter estimation. Among these methods, a relevant approach retrieves parameters by matching an empirical Impulse Response Function. This is possible since many DSGE models have a VAR representation. Another popular so-lution resorts to Bayesian estimation, which consists, in a nutshell, in regularizing the likelihood function, by imposing a prior distribution on the structural parameters.

In this chapter we will describe the main techniques and their shortcomings. Before

proceeding, let us introduce the notation used by Fern´andez-Villaverde et al. (2016).

The (potentially non-linear) state-space form of a solved DSGE is denoted as:

yt= Ψ(st, t, θ) + ut u ∼ Fu(·; θ)

st = Φ(st−1, t, θ) e ∼ Fe(·; θ)

Graphical Causal Models for Empirical Validation: a Comparative Application to a DSGE and an Agent-Based Model

Universit`

a di Pisa

Scuola Superiore Sant’Anna

master of science in economics

Graphical Causal Models for Empirical Validation:

a Comparative Application to a DSGE and an

Agent-Based Model

Supervisor

Candidate

Prof. Alessio Moneta

Elena Dal Torrione

Contents

Introduction

Chapter 1

DSGE and Agent-Based Models

1.1

DGSE Models

1.2

Agent-Based Models

1.3

The Models

1.3.1

The Smets and Wouters model

1.3.2

The Schumpeter meeting Keynes model

Chapter 2

Empirical Validation of

Agent-Based Models

2.1

Input Validation

2.2

Output Validation

2.2.1

Calibration of ABMs

2.2.2

Estimation of ABMs

2.3

Issues in ABM Validation

2.3.1

Comparison of causal structures

Chapter 3

Empirical Validation of DSGE

Models