Merging process optimization and advanced control: novel algorithms and performance monitoring strategies for sustained economic efficiency

(1)

Universit`

a di Pisa

Dottorato di Ricerca in Ingegneria Industriale

Curriculum in Ingegneria Chimica e dei Materiali Ciclo XXXI

Merging process optimization and advanced control:

novel algorithms and performance monitoring strategies

for sustained economic efficiency

Author

Marco Vaccari

Supervisor(s)

Prof. Gabriele Pannocchia Prof. Claudio Scali

Coordinator of the PhD Program Prof. Giovanni Mengali

(2)

(3)

(4)

(5)

Abstract

This Thesis presents possible solutions to best obtain and maintain economic performances in industrial processes. It is generally known that sustained per-formance is seen from industrial practitioners as one of the main research goals. This PhD activity aims to achieve this goal from different perspectives and inter-mediate objectives. It is shown how current advanced control and optimization implementations are indeed far from being economically optimal, and how this gap can be reduced.

In particular, the problem of designing an Economic Model Predictive Con-trol (EMPC) algorithm is addressed. Different methods are proposed in order to achieve optimal performance despite the presence of plant-model mismatch. The most recent works in literature regarding this problematic are analyzed. Differ-ent solutions are formulated merging together differDiffer-ent techniques coming from the MPC field, Real-Time Optimization (RTO) field and system identification. We also formulate a performance monitoring technique for a sustained optimal behavior in order to maintain the economic efficiency of the process despite time-varying disturbances.

This PhD activity has been also developed with the aim of using and produc-ing open-source tools to be available for the research community. Hence, Python has been used to produce a multipurpose code for MPC simulations. This code has been the main tool used to develop results and algorithmic analysis. In ad-dition, during these three years, it has been applied both in didactic support and other MPC related research.

Scientific rigor has always been the primary key, hence the different results showed in this Thesis have been published in peer-reviewed journals or presented to international congresses.

(6)

Acknowledgements

I first want to acknowledge my supervisor Prof. Gabriele Pannocchia for his incomparable guidance started during my master thesis and continued along this PhD research. His incredible capacity to suggest always new possible solutions to whatever problem I was struggling on, still amazes me. He has also been an important role model in my life beyond the academic environment.

I want to thank Giulia for her love, support, and so much patience. This three years would have not been possible without you keeping pushing me to see the other side of the moon every time I felt defeated.

I would like to thank all my family for the support given along these years I lived in Pisa. Thanks to my mom Marinella that made me the person I am today, to Sara whom I love so much, to my grandmother Nadia who has always been so strong and to my uncle Alessandro and my aunt Claudia that have always be there for me.

I want also to acknowledge Prof. James Rawlings for hosting me in his research group at the University of Wisconsin in Madison. It has been a great honor to have the possibility to study, debate and learn from such a famous and brilliant scholar. I also had the possibility to meet Prof. Stephen Wright with whom I could chat about my research and he has always been so kind to listen and propose solutions to my problems.

I wish also to thank my thesis committee for the time and effort spent evalu-ating this work.

During my months in Madison, I enjoyed the time spent with every person in Rawlings group. In particular, I thank Michael and Nishith for having always been available at answering my silly questions, Doug and Min for their company, Joel for always being so nice every time I had a doubt about CasADi and Travis for letting me experience camping along the Mississippi river. Every cold or hot day in Madison would not had been the same without Denise and Michelle good-morning. I had a lot of fun teaching you about Italian culture and exchanging jokes.

I also want to thank the CPClab research group and in particular Prof. Clau-dio Scali, Prof. Gabriele Pannocchia and Dr. Riccardo Bacci di Capaci for providing a stimulating learning environment.

Thanks to Riccardo that has been a valid coworker but also a good friend who always try to drag me into the “labronic” world.

I really enjoyed working or simply spending time with all the students that have been part of the CPClab family, even for a small period of time. In a not particular order: Alessio, Giuseppe, Edoardo, Marina, Adele.

(7)

pro-iii

foundly happy to have had the possibility of knowing such a great guy with whom I could exchange various thoughts about Italian and Iranian cultures.

I also want to thank the CDC group for providing such great culinary breaks and making the last months a lot sweeter. Again, in a not particular order: Federica, Deborah, Davide, Ivano, Matteo, Angelanna.

Finally, I want to thank my dear friend Elia that has always shared my ups and downs along this PhD.

(8)

(9)

Chapter

1

Introduction

In the process industries especially, the current paradigm for achieving overall economic objectives is to partition the information management, decision mak-ing, and control system into several layers as depicted in Figure 1.1. From the top down, the timescale of each layer diminishes sensibly. This means that the supply chain planning and scheduling is repeated daily or even weekly. This layer elaborates decisions about sequences of production operations and plans how to apply the main control actions to pursue these sequences. The second layer, often referred to as Real-Time Optimization (RTO), performs a steady-state economic optimization of the plant variables, updated on a timescale of hours. This economic optimization addresses the targets to be achieved to fullfill the current production operations. The RTO sends the results of its optimiza-tion as a setpoint to the third layer, usually referred to as the advanced process control system. It is the task of this layer to guide the plant transient state to the setpoint and, once arrived, to reject dynamic disturbances that enter the system. This process control layer is often implemented with Model Predictive Control (MPC) because of its flexibility, performance, robustness, and its ability to directly handle hard constraints on both inputs and states. Control actions calculated by this layer are passed to the distributed control layer usually repre-sented by PID controllers. These usually are constituted by basic control logics that manage to convert the control action derived from the MPC into different actions to be passed at the various underlying actuators. Once at the bottom layer of the structure, the control action timescale is comparable to continuous time. Actuators and sensors apply a fraction of the calculated MPC control and collect signals to be converted into measurements by PI/PID controllers.

However, there are an increasing number of problems for which the hierarchical separation of economic analysis and control is either inefficient or inappropriate. Flexibility and efficiency are requirements the control system must now address.

(14)

Figure 1.1: Typical hierarchical optimization and control structure in process sys-tems.

1.1 Model Predictive Control (MPC)

Optimization-based controllers, in general, and MPC systems, in particular, rep-resent an extraordinary success case in the history of automation in the process industries [1]. MPC algorithms exploit a (linear or nonlinear) dynamic model of the process and numerical optimization algorithms to guide a process to a setpoint reliably, while fulfilling constraints on outputs and inputs. Most MPC algorithms are divided into three modules, as depicted in Figure 1.2. A state estimator receives the current output measurement y(k), and updates the predic-tions (made at the previous decision time) of the state x(k), that is ˆx−(k + 1). A steady-state optimization module computes the state, input and output targets (xs(k), us(k), ys(k)) that match as close as possible the desired external setpoints while respecting the imposed constraints. A dynamic optimization module finds the optimal trajectory from the current state ˆx(k) obtained by the estimator, to the target computed the steady-state optimization module. The two optimization modules are equipped with quadratic cost functions. The dynamic optimization problem typically takes into account the future (i.e. predicted) output tracking error and the corresponding input deviation from the target. Both error and deviation are calculated from the target values computed at the current iteration by the steady-state optimization module. Once an optimal trajectory of length equal to the prediction horizon of the controller has been calculated, only its first input u(k) is injected into the plant. The overall algorithm is then repeated again the next decision time given the new current output measurement. It has been

(15)

1.2 Real-Time Optimization (RTO) 3 Estimator Steady-state Optimization Dynamic Optimization Process u(k) xs(k) us(k) ys(k) Tuning parameters Tuning parameters y(k) ˆ x(k) ˆ x−(k + 1), u(k)

Figure 1.2: MPC algorithm architecture.

shown and proved how, under mild assumptions, if the actual process and the nominal model perfectly match, this feedback control scheme is stable, and the output is able to follow any reachable imposed setpoint without offset [2].

1.2 Real-Time Optimization (RTO)

MPC design is not usually oriented to a maximization of the plant profit, rather to maintain steady-state safe operations. Economic optimizers are more often steady-state, and provide setpoints for dynamic controllers. An example of steady-state economic performance optimizers governing hybrid energetic systems is given in Appendix A. A system model is used to calculate optimal setpoints for the different devices in the system subject to operational and economical constraints.

Also in the chemical industry, optimal operation of a large number of plants is of economic importance. However, in this case, optimal operation is particularly difficult to achieve, due to inaccurate plant models or process disturbances. RTO has become the solution to this problem for a large number of chemical and petrochemical plants [3]. RTO was born in the chemical industry in the late 1970s when computational tool where still not so developed. In a decision hierarchy the RTO level is the medium-term one and its time scale goes from hours to few days. The optimizer can be very slow due to the fact that every condition involving

(16)

economical profit has to be changed and analyzed. Various RTO techniques are available in the literature and can be classified in two broad families depending on whether a process model is used or not. Model-based RTO usually relies on nonlinear first-principles models describing the steady-state behavior of the plant. Since accurate process models are quite difficult to obtain, many RTO algorithms repeatedly estimate some uncertain parameters and use the updated model to generate inputs via optimization [4]. In this case, the plant at the current operating point should be better represented by the model. Obviously when the plant-model mismatch is too high this method cannot well reproduce the real plant operation. Process measurements are often integrated into the RTO optimization framework. In this case, the optimization can use information from these measurements and does not rely exclusively on a (possibly inaccurate) process model.

1.3 MPC and RTO

In the hierarchal structure depicted in Figure 1.1, MPC commands the computed controls as setpoints to the basic layer (PI/PID) controllers. Likewise, MPC receives setpoints that are generated by an upper layer (RTO), that is dedicated to economic optimization.

The MPC model and the RTO model (when present) are not necessarily con-sistent, the former being linear and identified form data, the latter being nonlinear and based on first-principles balances. Moreover, unmeasured disturbances can limit the operating range of MPC, i.e. setpoints calculated by the RTO cannot be reached by MPC due to constraints. In addition, for an increasing number of applications, this separation of information and purpose is no longer optimal or desirable [5]. An alternative to this decomposition is to take the economic objective directly as the objective function of the control system. In this ap-proach, known as Economic Model Predictive Control (EMPC), the controller optimizes directly, in real time, the economic performance of the process, rather than tracking a setpoint.

Many works in the literature consider a combination between RTO and MPC through a target calculation level in the middle that coordinates the communi-cation and guarantees stability to the whole structure calculating feasible target for the optimal control problem [6, 7]. There are also examples of integration be-tween modifier-adaptation technique and MPC [8]: in this way the input targets calculated by the MPC are included as equality constraints into the modified RTO problem. In other cases the target module of the MPC has been modi-fied in various ways including a new quadratic programming problem that is an approximation of the RTO problem [9].

(17)

1.4 Thesis Objectives 5

Another area of the literature aimed at merging the two layers is the so-called Dynamic Real-Time Optimization (RTO). The objective function of the D-RTO includes an economic objective, subject to a dynamic model of the plant. The optimal control profiles are determined from the solution of the above dy-namic optimization problem, and then passed to the underlined MPC layer as trajectory setpoints to follow. The advantages of this formulation in presence of disturbances have been deeply emphasized in the literature [10, 11], also in the case of model-free alternatives [12]. A combination of nonlinear MPC and D-RTO has also been proposed to form a robust online optimization algorithm for batch processes [13, 14]. The D-RTO is also seen as a solution for merging economic and control layer, while advances in nonlinear model predictive control and its generalization to deal with economic objective functions take place [15]. In this sense a receding horizon closed-loop implementation of D-RTO can be also referred to as economic model predictive control [16].

1.4 Thesis Objectives

The objectives of this PhD Thesis are twofold. Firstly we aim at studying non-linear MPC algorithms which merge MPC and RTO into a single dynamic opti-mization and control module. In particular we are interested at recent proposals, typically referred to as “economic MPC”, as well as defining new algorithms which include disturbance estimation to improve robustness and applicability in process control problems.

The other important objective of this work is to study the definition and im-plementation of suitable performance monitoring and diagnosis methods, aimed at keeping the actual performance of the process as high as possible. In partic-ular, we aim at understanding from data when a revamping of the algorithm is necessary. This could be necessary because the economic objectives could have changed along the process, because disturbances are not properly estimated, be-cause the MPC model is no longer consistent with the actual process behavior as could be seen from corrections made by modifiers, or because the MPC tuning parameters are not correctly chosen.

1.5 Major results

The major results of this Thesis can be summarized as follows.

Analyzing the major contributions to the economic MPC (EMPC) funda-ments and algorithms, a problem has been evidenced. From the literature inves-tigation, no previous works on the so called “offset-free EMPC”, that is an EMPC algorithm with steady-state optimal operation that subject to plant-model mis-match, is able to recover an offset in the controller optimal performance. Although

(18)

the offset-free formulation for tracking MPC is quite well established [17, 18], the same procedure on EMPC is likely to fail. This is why, the RTO literature has been studied with particular attention on the the technique named “Modifier-Adaptation Methodology” [19, 20]. This method aims to adjust the economic optimization problem in order to reach a point that satisfies the Necessary Condi-tion of Optimality of the process and, under constraints satisfacCondi-tion, corresponds to a local optimum of the real plant [21]. A new EMPC algorithm has been formulated using this technique in combination with the offset-free method . In this way the EMPC algorithm is able to reach the economic optimum despite plant-model mismatch.

In order to calculate the modification at each iteration, it is mandatory to know the real gradients of the system at each steady-state points. Obviously, this is not a trivial task and represents the major drawback of this algorithm. Hence, a plant gradient estimate is needed and in combination with a system iden-tification algorithm, a new methodology has been developed. Collecting transient output and input data, the identification algorithm is applied to identify a locally linearized model of the nonlinear process. This linearization is then used to obtain the system gain, hence the plant gradient approximation required. In addition, the Moving Horizon Estimation (MHE) method has been used to better estimate the system states and disturbances. In this way the offset-free economic MPC algorithm has been updated and previous assumptions could be overcome.

Based on the EMPC algorithm developed, a new performance diagnosis methodology is needed. As a matter of fact, in presence of disturbances or time-varying plant dynamics, the plant gradient estimation procedure obtained can fail because of a weakly informative dataset. As result, the identification proce-dure cannot guarantee a good quality gradient approximation. This leads to a poor functioning of the modifier-adaptation strategy and, consequently, of the all EMPC implementation. To develop a monitoring strategy we want to use some optimization problem parameters, in particular sensitivity analysis, and identifi-cation error indices. Hence, the monitoring algorithm exploits the analysis of the Karush-Kuhn-Tucker (KKT) optimality conditions of the plant and consequently is able to establish if a new data collection is needed. The event-triggering mech-anism is then tested any time conditions apply. Results shows how the proposed methodology can overcome plant-model mismatch and time-varying disturbances, converging always to the best economic equilibrium. The comparison between the monitored formulation and its counterpart non monitored is done using the al-gorithm with the MHE method with smoothing updating. Results show how the monitored version is particularly efficient against the second one. This technique seems to mark a positive step into a more reliable and robust EMPC formulation

(19)

1.6 Thesis structure 7

nearer to the industrial panorama needs.

Simultaneously to the definition of an EMPC algorithm, it has been necessary to implement an MPC algorithm code. This code is written using Python and its first purpose is to be the main tool for this PhD research over the three years. Thereafter its intention is also to be an open-source, multipurpose, easy-to-use code for MPC design, analysis and simulation. The code is able to represent both tracking and economic MPC with the possibility to have an “offset-free” control. Along the years many estimation methods have been added together with all the research implementation above described. The code has been implemented in Python 2.7 and 3.6 [22], using CasADi 3.4 [23], and its architecture is rather sim-ple: the basic idea is to have a main file containing all the MPC core functions. Everything else, such as model or cost functions, steady-state and dynamic opti-mization problems, state estimation, is defined in separate Python scripts. In this case the architecture is somehow modular and easier to develop and customize. For each case to simulate, the user has to fill in a single Python script with all the required information such as matrices or functions describing the state dynamics and cost terms, MPC horizon, simulation horizon, time step, etc. Sev-eral boolean variables are provided to help the user with the problem definition, e.g. state-feedback, nominal case, continuous cost function. Finally, there is also the possibility to specify which offset-free disturbance model to apply by simply changing a name tag. Linear and nonlinear model and process dynamics as well as discrete-time or continuous-time have been implemented. Cost functions can be as general as the user prefers: linear, quadratic, generally nonlinear in both discrete-time and continuous-time.

At the moment, the code is published on GitHub as a public repository and can be found at https://github.com/CPCLAB-UNIPI/MPC-code. Its usage re-mains open to many fields and can be adapted to problems in different areas, e.g. process control, robotics, aerospace.

This code has had many applications. It has be used as didactics support and exercise tool for teaching purposes and also in the MPC research. In particular it has been possible to investigate MPC controllers of nonlinear processes with control valves affected by stiction. Specifically three MPC formulations affected by different types of plant-model mismatches have been tested both for SISO and MIMO systems. Results have defined the best MPC formulation that is the one named “stiction embedding MPC”.

1.6 Thesis structure

This thesis is structured in two main parts that have been the basics of my research.

(20)

The first part covers all the methods and algorithms developed towards the formulation of an offset-free EMPC. A wide introduction to the problem and to its main components together with a review of the most recent works in literature and a deep background description of the various techniques used, is given in Chapter 2.

A first attempt to the EMPC offset-free formulation is then given in Chap-ter 3. Here the basic modifier-adaptation and offset-free combination is firstly introduced and analyzed. Results on different algorithm solution are proposed: standard EMPC, tracking MPC with economic cost function in the steady-state optimization module and finally a fully modified EMPC.

Chapter 4 presents a second version of the modifiers-adaptation algorithm, more reliable than the previous one on a wider range of plant-model mismatch. In this Chapter, a plant gradient identification procedure is also presented to finally formulate an offset-free EMPC with optimal steady-state behavior, overcoming assumption made in Chapter 3.

This algorithm formulation is then used in Chapter 5 to be the base upon which a performance monitoring technique is developed. Here, a wide background on performance monitoring techniques for MPC and EMPC is given. In addition, the event-triggering mechanism is developed and applying conditions are fully explained. Finally simulations shows how this methodology effectively works.

The second part of the thesis takes into consideration the MPC code developed along the three years. A fully description of the code structure, its usage and possibilities is illustrated in Chapter 6. A detailed description of the tools used to build the code is given along with motivation and main purpose. A description of how the various MPC algorithm modules have been implemented and how they fit into the main nature of the code is given. To better understand how this code interfaces with the user, a simpler version of the example presented in Chapter 3 is presented and its implementation explained. In this way the general structure explained in the first part of the chapter, can be better analyzed and commented. Chapter 7 gives an example of the code usage into the MPC algorithm field. An initial introduction and background to the main problem of dealing with static friction (stiction) in industrial valves is given. The representation of possible MPC formulations dealing with this particular case of plant-model mismatch is presented. The proposed methodology of including the valve dynamics inside the MPC model is explained for both the SISO and MIMO cases. In addition, a possible method for estimating valve dynamics parameters exploiting numerical optimization is presented. Finally, it is shown how this technique can help turn a stiction unaware MPC algorithm into an stiction embedding one, including the estimated valve parameters inside the MPC model.

(21)

1.6 Thesis structure 9

Concluding, Chapter 8 summarizes the major results and depicts possible future directions of the work.

Three appendices are presented afterwards. Appendix A presents a software tool able to optimize the decision variables of various electrical, thermal and/or storage devices in a Hybrid Renewable Energy System (HRES). Controlling the HRES in an economic efficient fashion is the aim of this software. It is also shown how this tool has been applied to a real energy system in Tuscany, Italy giving positive results.

Appendix B, instead, describes a package for MIMO system identification. This is an open-source software developed using Python and originally developed by a former master student in Chemical Engineering. It has been later modified and adapted to be better implemented to the code described in Chapter 6. This software represents also the core of the identification algorithm used in the plant gradient estimation in Chapter 4. Finally Appendix C contains a list of all the published papers together with ones accepted and waiting for publication.

(22)

(23)

Part I

(24)

(25)

Chapter

2

Background and literature review

In this chapter the various techniques used to implement an offset-free EMPC formulation are explained and detailed. The description of the plant, model and constraints considered in this work is given to describe the standard track-ing MPC algorithm. Moreover the tracktrack-ing MPC with offset-free framework is detailed together with the RTO methods with modifier-adaptation techniques. The starting EMPC standard formulation and its difference with tracking MPC algorithm are then described. In addition, the MHE method and advantages over the traditional state and disturbance estimation techniques are explained. Finally, an introduction to system identification and a brief description on how it will be used in this work are given.

In order to propose an offset-free EMPC algorithm, a review of related con-cepts and techniques is given.

2.1 Plant, model and constraints

When modeling a chemical-physical process, usually systems are represented by a set of ODE since the balance equations are first principles based. Although, when treating a numerical optimization algorithm, continuous-time systems are difficult to handle since because the computational nature of the optimizer always requires a discretization in the time domain. For this reason, many numerical integration methods exist to transform the original ODE system into a discrete-time state space system of DAE. Hence, in this work we are concerned with the control of time-invariant dynamical systems in the form:

x+_p = Fp(xp, u) (2.1)

y = Hp(xp)

in which xp∈ Rn, u ∈ Rm, y ∈ Rpare the plant state, control input and output at a given time, respectively, x+_p is the successor state. The plant output is measured

(26)

at each time k ∈ I. Functions Fp : Rn× Rm → Rn and Hp : Rn → Rp are not known precisely but are assumed to be differentiable. In order to design an MPC algorithm, a process model is known:

x+= f (x, u) (2.2)

y = h(x)

in which x, x+∈ Rn_{denote the current and successor model states. The functions} f : Rn× Rm _{→ R}n _{and h : R}n_{→ R}p _{are assumed to be differentiable. Input and} output are required to satisfy the following input and output constraints at all times:

umin ≤ u ≤ umax, ymin ≤ y ≤ ymax (2.3)

in which umin, umax∈ Rm and ymin, ymax∈ Rp are the bound vectors.

c

Remark 2.1 Issues about integration from continuous-time form to discrete-time one will be introduced in the following sections and chapters. Although, it has to be

underlined how this is not the focus of this work. v

2.2 Nominal tracking MPC

Taking into consideration Figure 1.2, let us now define the various module of the MPC algorithm. Given a sequence of inputs ˜u = {u(0), u(1), . . .} and an initial state z, we define ˜yz,˜u= {y(0), y(1), . . .} as the corresponding sequence of outputs generated by system (2.2). Moreover, the solution at time k of system (2.2) is denoted by x(k; z, ˜u). Hence the following definitions can be recalled [2]:

Definition 2.1 System (2.2) is observable if there exists a finite N0∈ I and γ(·) ∈ K such that for any two initial states z1 and z2 and control sequence ˜u, and all k ≥ N0 |z₁− z₂| ≤ γ(k˜yz1,˜u− ˜yz2,˜uk0:k) (2.4)

in which a function σ ∈ R≥0 → R≥0 belongs to the class K if it is continuous, zero at zero, and strictly increasing.

Definition 2.2 Given a sequence of output measurements ˜yx0,˜u = {y(0), . . . , y(k)} and a (prior) estimate of the initial state ¯x0, an estimate of x(k; x0, ˜u) for system (2.2) is denoted as ˆx(k). This estimate is defined to be asymptotically stable if there exists β(·) ∈ KL such that for any initial state x0 and its prior estimate ¯x0 and k ∈ I≥0:

|x(k; x₀, ˜u) − ˆx(k)| ≤ β(kx0− ¯x0k, k) (2.5)

in which a function β ∈ R≥0× I≥0 → R≥0 belongs to the class KL if for k ≥ 0, β(·, k) ∈ K and for each r ∈ R≥0 there holds limk←∞β(r, k) = 0.

(27)

2.2 Nominal tracking MPC 15

Assumption 2.1 The system (2.2) is observable.

At each time k ∈ I, given the output measurement y(k), an observer for (2.2) is defined to estimate the state x(k). For simplicity of exposition, only the current measurement of y(k) is used to update the prediction of x(k) made at the previous decision time, i.e. a “Kalman filter” like estimator is used. Alternatively other methods including more measurements to estimate the current state could be used, e.g. Moving Horizon Estimation (MHE). We define symbols ˆxk|k−1, and ˆ

y_k|k−1, as the predicted estimate of x(k), d(k) and y(k), respectively, obtained at the previous time k − 1 using model (2.2), i.e.:

ˆ

xk|k−1= f (ˆxk−1|k−1, uk−1) (2.6)

ˆ

yk|k−1= h(ˆxk|k−1)

Let us note the symbols ˆxk|k−1correspond to ˆx−k+1shown in Figure 1.2. Defining the output prediction error as:

ek= y(k) − ˆyk|k−1 (2.7)

the filtering relation can be written as follows:

ˆ

x_k|k = ˆx_k|k−1+ κx(ek) (2.8)

where ˆxk|k is the filtered estimate of x(k) in (2.2) obtained using measurement y(k). Given that, we can state this other assumption

Assumption 2.2 We assume that relations (2.6)-(2.8) form an asymptotically stable observer for the system (2.2).

As can be seen in Figure 1.2, given the current state estimate ˆxk|k, the MPC algorithm first computes the steady-state target that ensures exact setpoint track-ing in the controlled variable. Hence, in general the followtrack-ing target problem is solved: min x,u,y`s(y − ysp, u − usp) (2.9a) subject to x = f (x, u) (2.9b) y = h(x) (2.9c) umin ≤ u ≤ umax (2.9d)

(28)

in which `s: Rp×Rm → R is the steady-state cost function and ysp ∈ Rp, usp ∈ Rm are the output and input setpoints, respectively. As seen in Chapter 1, these setpoint usually comes from the RTO as depicted in Figure 1.1. We assume (2.9) is feasible and we denote its (unique) solution as (xs,k, us,k, ys,k). Typically, `s(·) is positive definite in the first argument (output steady-state error) and semidefinite in the second argument (input steady-state error), and relative input and output weights are chosen to ensure that ys,k → ysp whenever constraints allow it.

Let x = {x0, x1, . . . , xN} and u = {u0, u1, . . . , uN −1} be, respectively, a state sequence and an input sequence. The optimal control problem solved in the dynamic optimization module at each time is usually referred to as Finite Horizon Optimal Control Problem (FHOCP) and reads:

min x,u N −1 X i=0 `QP(xi− xs,k, ui− us,k) + Vf(xN − xs,k) (2.10a) subject to x0 = ˆxk|k, (2.10b) xi+1= f (xi, ui), i = 0, . . . , N − 1 (2.10c) umin ≤ ui ≤ umax, i = 0, . . . , N − 1 (2.10d) ymin≤ h(xi) ≤ ymax i = 0, . . . , N − 1 (2.10e)

in which `QP : Rn× Rm → R≥0is a strictly positive definite convex function. Vf : Rn→ R≥0 is a terminal cost function which may vary depending on the specific MPC formulation, according to the usual stabilizing conditions [2]. Assuming problem (2.10) feasible, its solution is denoted by (x0_k, u0_k) and the associated receding horizon implementation is given by:

uk = u00,k (2.11)

This is the control action passed indirectly to the process through the underlying control layers. This MPC formulation has been proved to have stability property and converge to the desired reachable setpoints when the model (2.2) is equal to the plant (2.1). That is if the setpoints calculated by the RTO lies into the MPC constraints and the nominal case is considered, then the control action computed is such that both input and output setpoint are achieved.

2.3 Offset-free tracking MPC

On the other hand, being MPC a model-based optimization algorithm, in the presence of plant-model mismatch or unmeasured disturbances it can come across offset problems. The offset correction in tracking MPC algorithms has been

(29)

2.3 Offset-free tracking MPC 17

deeply exploited and analyzed. Muske and Badgwell [18] and Pannocchia and Rawlings [17] first introduced the concept of general conditions that allow zero steady-state offset with respect to external setpoints. The general approach is to augment the nominal system with disturbances, i.e. to build a disturbance model, and to estimate the state and disturbance from output measurements. A recent review about disturbance models and offset-free MPC design can also be found in [24]. Offset-free MPC algorithms are generally based on an augmented model [18, 17, 25, 26]. The general form of this augmented model can be written as:

x+= F (x, u, d) (2.12)

d+= d y = H(x, d)

in which d ∈ Rnd _{is the so-called disturbance. The functions F : R}n_{× R}m_× Rnd→ Rn and H : Rn× Rnd → Rp are assumed to be continuous and consistent with (2.2), i.e. F (x, u, 0) = f (x, u) and H(x, 0) = h(x).

Assumption 2.3 The augmented system (2.12) is observable.

c

Remark 2.2 It has to be noted that system (2.12) is observable only if system (2.2) is observable. On the other hand, given Assumption 2.1 it is possible to build an observable augmented system of (2.2) choosing nd≤ p disturbance states. v The most important module of the MPC algorithm when dealing with non nomi-nal cases is the estimator. A good disturbance estimate can help address possible plant-model mismatch and overcome correspondent errors. Hence, as shown for the nominal case, at each time k ∈ I, given the output measurement y(k), an observer for (2.12) is defined to estimate the augmented state (x(k), d(k)). For simplicity of exposition, only the current measurement of y(k) is used to update the prediction of (x(k), d(k)) made at the previous decision time, i.e. a “Kalman filter” like estimator is used. Let us define ˆdk|k−1 as the predicted estimate of d(k) obtained at the previous time k − 1 using the augmented model (2.12), i.e.:

The “Kalman filter” like estimator shown in (2.8) is modified as follows:

ˆ

xk|k = ˆxk|k−1+ κx(ek) (2.14)

ˆ

(30)

where ˆd_k|k is the filtered estimate of d(k) in (2.12) obtained using measurement y(k). As stated in Assumption 2.2, relations (2.13)-(2.14) are supposed form an asymptotically stable observer for the augmented system (2.12).

The optimization problems are then modified as follows. Given the current es-timate of the augmented state (ˆxk|k, ˆdk|k), an offset-free tracking MPC algorithm computes the steady-state target with following problem:

min x,u,y`s(y − ysp, u − usp) (2.15a) subject to x = F (x, ˆdk|k, u) (2.15b) y = H(x, ˆdk|k) (2.15c) umin ≤ u ≤ umax (2.15d)

ymin≤ y ≤ ymax (2.15e)

Considerations about the steady-state problem are the same as for problem (2.9). The FHOCP solved at each time is the following:

min x,u N −1 X i=0 `QP(xi− xs,k, ui− us,k) + Vf(xN − xs,k) (2.16a) subject to x0 = ˆxk|k, (2.16b) xi+1= F (xi, ˆdk|k, ui), i = 0, . . . , N − 1 (2.16c) umin ≤ ui ≤ umax, i = 0, . . . , N − 1 (2.16d) ymin≤ H(xi, ˆdk|k) ≤ ymax i = 0, . . . , N − 1 (2.16e)

xN = xs,k (2.16f)

Assuming problem (2.16) feasible, the control action is calculated as in (2.11).The MPC scheme in Figure 1.2 is hence modified as depicted in Figure 2.1. It has to be noted how now the steady-state optimization problem is no more a stand-alone module due to the dependance of ˆdk|k.

As conclusion of this discussion, the following result holds true [18, 17, 27].

u

Proposition 2.1 Consider a system controlled by the MPC algorithm as described by (2.14), (2.15) and (2.16) with receding horizon implementation as in (2.11). If the closed-loop system is stable, then the output prediction error goes to zero, i.e.:

lim

k→∞y(k) − ˆyk|k−1= 0 (2.17)

Furthermore, if input constraints are not active at steady state, there is zero offset in the controlled variables, that is:

lim

(31)

2.4 Real-time optimization with modifier-adaptation 19 Estimator Steady-state Optimization ˆ d(k) Dynamic Optimization Process u(k) xs(k) us(k) ys(k) Tuning parameters Tuning parameters y(k) ˆ x(k), ˆd(k) ˆ x−(k + 1), u(k)

Figure 2.1: Offset-free MPC algorithm architecture.

2.4 Real-time optimization with modifier-adaptation

Non-economically optimum stationary points can also be the result of uncer-tainties in RTO algorithms. Hence, also in the RTO literature many works are focused on plant-model mismatch issues. RTO algorithms mainly use process measurements in the optimization framework to compensate the effect of these uncertainties. Depending on how the available measurements are used, RTO can be classified in three ways.

RTO typically proceeds improving the process model, using an iterative two-step approach [4, 28], namely an identification two-step followed by an optimization step. The idea is to repeatedly estimate selected uncertain model parameters and use the updated model to generate new inputs via optimization.

Other alternative options do not use a process model online to implement the optimization [29, 30, 31], so in this case the mismatch issue can be associated to unmeasured disturbances. In extremum-seeking control [32, 33, 34], the measure-ments are used to correctly estimate the plant cost through a modification of the inputs [35].

More others utilize a nominal fixed process model and appropriate measure-ments to guide the iterative scheme towards the optimum. In this last field the term “modifier-adaptation” indicates those fixed-model methods that adapt cor-rection terms (i.e. the modifiers) based on the observed difference between actual

(32)

and predicted functions or gradients [36, 19, 20]. In this case modifications are made at the level of cost and constraint functions. Marchetti et al. [21] formalize the concept of using plant measurements to adapt the optimization problem in response to plant-model mismatch, through modifier-adaptation. Among oth-ers, a recent second-order modifier-adaptation scheme proposed in [37] is able to ensure monotonous and feasible-side global convergence to a KKT point of the plant despite plant-model mismatch. This method pays high prices in terms of convergence time and conservatism in choosing tuning parameters. A recent work by [38] propose a methods to overcome these drawbacks.

The objective of RTO is the minimization of some steady-state operating cost function, while satisfying a number of constraints. Finding the optimal steady-state operation point for the actual process can be steady-stated as the solution of the following problem:

min

u Φp(u) (2.19a)

subject to

Cp(u) ≤ 0 (2.19b)

In the above, Φp := φ(u, yp) : Rm → R is the economic performance cost function of the process and Cp := c(u, yp) : Rm → Rnc is the process constraint function. As explained before for the MPC case, the exact process description yp(u) is unknown, and only a model can be used in the process optimization. Hence, the model-based economic optimization is represented by problem:

min

u Φ(u, θ) (2.20a)

subject to

C(u, θ) ≤ 0 (2.20b)

where Φ(u, θ) := φ(u, y(u, θ)) : Rm → R and C := c(u, y(u, θ)) : Rm _{→ R}nc repre-sent the model economic cost function and the model constraint function, which may depend on uncertain parameters θ ∈ Rnθ_{. The approximate model dynamics} is represented by the input-output relation y(u, θ). Due to plant-model mismatch, open-loop implementation of the solution to (2.20) may lead to suboptimal and even infeasible operation.

The modifier-adaptation methodology changes problem (2.20) so that in a closed-loop execution, the Necessary Conditions of Optimality (NCO) of the mod-ified problem correspond to the necessary conditions of the process (2.19), upon

(33)

2.4 Real-time optimization with modifier-adaptation 21

convergence of the algorithm. The following problem shows the model-based optimization with additional modifiers [21, 39]:

¯ uh = arg min u ΦM = Φ(u, θ) + (λ Φ h−1) T u (2.21a) subject to CM = C(u, θ) + (λCh−1) T (u − ¯uh−1) + Ch−1≤ 0 (2.21b) in which: λΦ_h−1= ∇uΦp(¯uh−1) − ∇uΦ(¯uh−1, θ) (2.22a) λC_h−1= ∇uCp(¯uh−1) − ∇uC(¯uh−1, θ) (2.22b) C_h−1= Cp(¯uh−1) − C(¯uh−1, θ) (2.22c)

In (2.21) and (2.22), ¯uh−1∈ Rm represents the operation point, calculated at the previous RTO iteration h − 1, and the modifiers λΦ_h−1∈ Rm_{, λ}C

h−1∈ Rm×nc, and C_h−1 ∈ Rnc _{are evaluated using the information available at that point. Notice} that the model parameters θ are not updated.

Marchetti et al. [21, 39] demonstrated that, upon convergence, the Karush-Kuhn-Tucker (KKT) conditions of the modified problem (2.21) match the ones of the true process optimization problem (2.19). Hence, if second-order conditions hold at this point, a local optimum of the real plant can be found by the prob-lem modified as in (2.21). Furthermore, a filtering procedure of the modifiers is also recommended in order to improve stability and convergence, and to reduce sensitivity to measurement noise. As a matter of fact, when operating distantly from the optimum, this strategy could lead to an excessive correction. Hence, the filtering step is given by the following equations:

λΦ_h = (I − K_λΦ)λΦ_h−1+ K_λΦ(∇uΦp(¯uh) − ∇uΦ(¯uh, θ)) (2.23a) λC_h = (I − K_λC)λC_h−1+ K_λC(∇_uC_p(¯u_h) − ∇_uC(¯u_h, θ)) (2.23b) C_h = (I − KC)C_h−1+ KC(C_p(¯u_h) − C(¯u_h, θ)) (2.23c)

where KλΦ, K_λC, and KC (usually diagonal matrices) represent the respective first-order filter constants for each modifier. An alternative approach to the modifier filtering step (2.23), is to directly define the modifiers as the gradient or function differences, and then filter the computed inputs to be applied to the process [40, 41]. From (2.23) and (2.22) it is clear how the process gradient estimation stage is the major requirement of this method: actually, the process gradient estimation, is hidden into both ∇uΦp and ∇uCp for calculating λΦh and λC_h. This is also the major and tightest constraint for this method [42].

(34)

2.5 Economic MPC

As can be seen from Figure 1.1, setpoints (ysp, usp) that enter in (2.15) come from the upper economic layer referred to as the RTO. This hierarchical division may limit the achievable flexibility and economic performance that many processes nowadays request. There are several proposals to improve the effective use of dynamic and economic information throughout the hierarchy. As explained in Section 1.3, the first approach to this merging is the D-RTO. While many D-RTO structures have been proposed throughout the literature [12, 43, 44], many of the two-layered D-RTO and MPC systems proposed are characterized by a lack of rigorous theoretical treatment including the constraints. However, as can be seen in the above cited literature, the D-RTO formulations, still consider the presence of both RTO and MPC in separated layers. Instead of moving the dynamic characteristic to the RTO level, the interest here is to move economic information into the control layer. This approach involves modifying the traditional tracking objective function in (2.16) and the target cost function in (2.15) directly with the economic stage cost function used in the RTO layer. In this latter case the formulation takes the name of economic MPC (EMPC) [45]. It has to be underlined, that, in this case, the economic optimization is provided only by the EMPC layer, while the RTO one is completely eliminated.

In standard MPC, the objective is designed to ensure asymptotic stability of the desired steady state. This is accomplished by choosing the stage cost to be zero at the steady-state target pair, denoted (xs, us), and positive elsewhere, i.e.

0 = `QP(xs, us) ≤ `QP(x, u) for all admissible (x, u) (2.24)

In EMPC, instead, the operating cost of the plant is used directly as the stage cost in the FHOCP objective function. As a consequence, it may happen that `e(xs, us) > `e(x, u) for some feasible pair (x, u) that is not a steady state. This possibility has significant impact on stability and convergence properties. In fact, while a common approach in the tracking MPC is to use the optimal value function as a Lyapunov function for the closed-loop system to prove its stability, in the EMPC formulation, due to the fact that (2.24) does not hold, these stability arguments fail. Hence, for certain systems and cost functions, oscillating solutions may be economically more profitable than steady-state ones, giving rise to the concept of average asymptotic performance of economic MPC which is deeply developed in [46, 45]. Despite of that, in the literature, there are also formulations of the Lyapunov-based EMPC by taking advantage of an auxiliary MPC problem solution [47, 48].

Therefore, in order to delineate the concept of convergence in EMPC, two other properties may be useful: dissipativity [49, 45] and turnpike [50, 51].

(35)

2.5 Economic MPC 23

In order to define dissipativity for a system, we need firstly to define a func-tion s : Rn× Rm _{→ R named supply rate. Hence, we can state the following} definition [49]:

Definition 2.3 The system (2.2) is defined dissipative respective to the function s, if there exists another function S : Rn× Rm_{→ R such that:}

S(f (x, u)) − S(x) ≤ s(x, u) (2.25)

Moreover, the system is said to be strictly dissipative, if there exists a function sρ: Rn→ R≥0 belonging to the class K, such that:

S(f (x, u)) − S(x) ≤ s(x, u) − sρ (2.26)

Strict dissipativity has been proved to be a sufficient condition for guaranteeing asymptotic stability of the closed-loop system for EMPC formulation without ter-minal constraint [52]. Gr¨une and Stieler [53] analyzed the same EMPC structure and demonstrated approximate transient optimality of the closed-loop system on finite time intervals based on dissipativity and controllability properties. M¨uller et al. [54] investigated whether the dissipativity property is not only sufficient, but also necessary for optimal steady-state operation. In addition, they also analyzed how changes in the constraint set reflect in a change in the considered supply rate, studying the robustness of the dissipativity property. A review of the the role of dissipativity in economic model predictive control can be found in [55]. A recent work by Olanrewaju and Maciejowski [56] pointed out the importance of the discretization method used for the objective function in the EMPC problem. If not properly designed, they show how the discretization process on the closed-loop system can transform a strictly dissipative system in continuous-time into a unstable system in discrete-time.

Turnpike, instead, is a property of the optimal control problem of the EMPC. Specifically this means that when varying initial conditions and horizon length, the resulting solutions stays near to a specific steady-state for most of the horizon and only in the end they tend to deviate. These properties play a key role in the analysis and design of schemes for D-RTO and EMPC. It is shown also that in a continuous-time form dissipativity of a system with respect to a steady state implies the existence of a turnpike and optimal stationary operation at this steady state [57, 58]. An extensive review about EMPC control methods can be found in [59, 60].

In order to define an offset-free EMPC we need our system to reach an eco-nomical optimal equilibrium. The following Assumption is then to be declared.

(36)

Assumption 2.4 In this work the steady-state operation is assumed to be more profitable than an oscillating behavior, i.e. the system is supposed to be dissipative respective to the stage cost `e(·).

The starting EMPC algorithm considered in this work is taken from [27], and includes an offset-free disturbance model as described in Section 2.3. Given the current state and disturbance estimate (ˆxk|k, ˆdk|k), the economic steady-state target is given by:

min

x,u,y`e(y, u) (2.27a)

subject to

x = F (x, ˆd_k|k, u) (2.27b)

y = H(x, ˆd_k|k) (2.27c)

umin ≤ u ≤ umax (2.27d)

ymin≤ y ≤ ymax (2.27e)

in which `e : Rp × Rm → R is the economic cost function defined in terms of output and input. Notice that the arguments of the economic cost function are measurable quantities. Let (xs,k, us,k, ys,k) be the steady-state target triple solution to (2.27). Then, the FHOCP solved at each time is given by:

min x,u N −1 X i=0 `e(H(xi, ˆdk|k), ui) (2.28a) subject to x0 = ˆxk|k, (2.28b) xi+1= F (xi, ˆdk|k, ui), i = 0, . . . , N − 1 (2.28c) umin ≤ ui ≤ umax, i = 0, . . . , N − 1 (2.28d) ymin ≤ H(xi, ˆdk|k) ≤ ymax, i = 0, . . . , N − 1 (2.28e)

xN = xs,k (2.28f)

While several formulations of economic MPC are possible, in this work we use a terminal equality constraint to achieve asymptotic stability [49]. We remark that the target equilibrium xs,k is recomputed at each decision time by the target calculation problem (2.27).

2.6 Moving Horizon Estimation

Kalman filter, like the one in (2.14), is the optimal state estimator for uncon-strained, linear systems subject to normally distributed state and measurement

(37)

2.6 Moving Horizon Estimation 25

noise. Nevertheless, when it comes to nonlinear physical systems with states sub-ject to hard constraints, Kalman filtering is no longer directly applicable. Among the many different solutions and nonlinear state estimators proposed along the years there is also the Extended Kalman Filter (EKF). Although its large and established usage in many fields, EKF has also been proved to fail, even with per-fect match between plant and model, for different chemical reaction networks in which there are the conditions that lead to the formation of multiple optima [61]. Hence, another state estimation method able to handle nonlinear dynamics and at same time able to satisfy state constraints, is evoked. This is the so called Moving Horizon Estimation [62] (MHE) which treats the estimation problem as an optimization problem. The new state estimate, at every sample time k, is the result of an optimization problem based on NT past output data. This method has demonstrated good results against other state estimation techniques for non-linear systems such the one considered in this work. Haseltine and Rawlings [63] found that MHE performances are much higher than EKF ones for the above mentioned cases, underling that the only price of this improvement is the greater computational expense required to solve the MHE optimization. However, effi-cient numerical methods proposed along the years, have allowed the formulation of real-time implementation of MHE [64, 65]. In [66, 67] MHE is shown to be an asymptotically stable observer for a nonlinear deterministic model, while the con-strained formulation has proved to be feasible for linear, state-space models [68].

Including past data that are not part of the estimation horizon at the current iteration, has been the always a non trivial task. This past data are considered in a term named “arrival cost” and Rao et al. [69] firstly defined two possible schemes. The so called “filtering” scheme penalizes deviations of the initial es-timate in the horizon from an a priori eses-timate, while the “smoothing” scheme penalizes deviations of the trajectory of states in the estimation horizon from an a priori estimate. The two different data windows sliding for both schemes can be seen in Figure 2.2. For unconstrained, linear systems, the MHE opti-mization collapses to the Kalman filter for both of these schemes. Tenny and Rawlings [64] then estimated the arrival cost by approximating the constrained, nonlinear system as an unconstrained, time-varying linear system and applying the corresponding filtering and smoothing schemes. Their conclusions is that the smoothing scheme has better performances because it is not generating oscilla-tions due to unnecessary propagation of initial error as, instead, happens for the filtering one.

In the end let us define the MHE optimization problem. Given the sys-tem (2.2) an observer is defined to estimate the augmented state. Let us define wx = {wx₀, . . . , w_Nx

T−1} , v

y _{= {v}y 0, . . . , v

y

(38)

Figure 2.2: Arrival cost updating schemes for MHE. Above the filtering scheme

consider only the first data of the NT past data window, while the

smoothing (below) considers the entire estimated trajectory at the previous iteration.

(39)

2.7 System identification 27

and measurement noise sequences. Both of them are intended to be random nor-mally distributed sequences. Hence the MHE problem to solve, at the generic time k, is the following:

min x,wx_,vy Γ(γ) + k X j=k−NT+1 `M HE(wxj, v y j) (2.29a) subject to: γ = ˆxk−NT+1|k−NT+1− ¯x0 (2.29b) ˆ xj+1= f (ˆxj, uj) + wxj (2.29c) yj = h(ˆxj) + vjy (2.29d)

ymin ≤ h(ˆxj) ≤ ymax (2.29e)

in which yj represents the measurement at time j and NT represents the horizon length of the MHE problem. The term Γ(γ) = 1₂γ0Pkγ + pk approximates the ar-rival cost of the full estimation problem for times before k−NT+1. The weighting term Pk represents the inverse of the covariance of the a priori augmented state estimate ¯x0. Until the data window is not filled, the optimization problem solved is the so called Full Information Estimation, i.e. the sum in (2.29a) becomes Pk

j=0 and the size of the problem increases at every iteration. Once NT input and output data have been collected, the horizon moves one step forward and the optimization problem does not grow anymore, becoming the one in (2.29), i.e. the first data point is discarded as the new one is added. This shifting can be done in different ways and may cause the overlapping of two different data windows. The role of the term pk is then to subtract double counted data points during this horizon movement. Its starting value is 0. Starting from chosen initial values, Pk and ¯x0 are accordingly updated only when the window of NT data has been collected, i.e. k = NT, and their updating formula may differ depending on the arrival cost updating selected [66] as seen above.

The estimation problem cost function `M HE(·) in (2.29a) is usually quadratic and defined as follows:

`M HE(wj, vj) = kwxjk2Q−1+ kvy_jk2_R−1 (2.30) in which Q ∈ Rn×n, R ∈ Rp×p represent the process and measurement noise co-variances respectively. The filtered estimate of x(k) ˆxk|k is hence finally obtained from x_N_T−1|k, solution of problem (2.29).

2.7 System identification

In Section 2.4 the role of the knowledge of the plant gradient in order to build a reliable modifier-adaptation formulation has been underlined. Obviously this

(40)

gradient cannot be known in practice unless some expensive experiments are con-ducted. Alternatively it can be estimated using a system identification algorithm in order to obtain a locally linear version of the initial nonlinear system. The route that involves designing experiments, collecting data and building models is commonly referred to as system identification. Usually the linear model identified is the one used to describe an approximate dynamic behavior of the considered system and, upon which, the optimization problems are built. As a matter of fact, very often these process models are not directly available or their representation derived from physical laws may be cumbersome for controller implementation due to nonlinearity or the presence of many unknown parameters. For such reasons, linear multi-variable models are used ubiquitously in applications, and the most common way of deriving them is to collect data of process inputs and outputs over specific time windows and to use suitable algorithms to obtain the process model parameters. In this work the linearized system is only considered to cal-culate its gain to better represent the approximation of the steady-state plant gradient, while the system used in the optimization problems is the nonlinear one already shown in (2.2).

System identification algorithms can be classified into two main cate-gories [70]: Prediction Error Methods (PEMs) and Subspace Identification Meth-ods (SIMs). The former are aimed at identifying input-output transfer function models, whereas the latter have been developed to obtain state space models. Models identified using SIMs are often less accurate than those obtained from PEMs, but on the other hand, state space models offer a simpler parametrization for large multivariable systems and they are more convenient for model predictive controller (MPC) design. Hence, in this work we are interested on SIMs since our system is in a state-space representation.

Let us consider to have collected input-output data for a general multivariable system with m inputs and p outputs, defined as follows:

u = [u0u1u2 . . . uL−1], y = [y0 y1y2 . . . yL−1] (2.31) where L is the number of samples, u ∈ Rm×L _{is the input sequence, and y ∈} Rp×L is the output sequence, being uk = [u(1)_k , u_k(2), . . . , u(m−1)_k , u(m)_k ]T and yk = [y (1) k , y (2) k , . . . , y (p−1) k , y (l)

k ]T considered at the k-th sampling time with k = 0, . . . , L − 1.

SIMs [71] identify state space models, that can be represented in the process form:

(

xk+1= Axk+ Buk+ wk yk= Cxk+ Duk+ vk

(2.32)

where A ∈ Rn×n_{, B ∈ R}n×m_{, C ∈ R}p×n_{, D ∈ R}p×m _{are the system matrices.} Two other common ways to represent MIMO systems are the so-called innovation

(41)

2.7 System identification 29 form: ( xk+1= Axk+ Buk+ Kek yk = Cxk+ Duk+ ek (2.33)

with K ∈ Rn×p, usually referred to as the Kalman predictor gain, and also the predictor form:

(

xk+1= AKxk+ BKuk+ Kyk yk = Cxk+ Duk+ ek

(2.34)

where the following relations hold:

AK = A − KC, BK = B − KD (2.35)

The objective of SIMs is to estimate the system matrices (A, B, C, D, K) from the collected data.

In this work we are interested to apply an identification method classified in the traditional methods category: N4SID [72]. The three most popular traditional methods are: the aforementioned N4SID [72], the “multivariable output-error state space” (MOESP5[73]) and the “canonical variate analysis” (CVA5 [74]). Although they have been theorized as three different algorithms, they have been later grouped into a unifying theorem as discussed in [75]. Based on this approach, the three methods can be seen as a singular value decomposition (SVD) of a weighted matrix. Defining fh as the future horizon and ph as the past horizon, with the following notation:

Lh = L − fh− ph+ 1, Xfh = [xfh xfh+1 . . . xfh+Lh−1] ∈ R n×Lh_, Yfh = [yfhyfh+1 . . . yfh+Lh−1] ∈ R p×Lh_, Ufh = [ufh ufh+1 . . . ufh+Lh−1] ∈ R m×Lh (2.36)

the algorithm proposed in [76] determines two state sequences (Xfh and Xfh+1) through the projection of input and output data. These sequences are shown to be outputs of non-steady state Kalman filter banks (for more details see [72, 76]). Once obtained the sequences, the system matrices are obtained by solving, in a least-squares sense, the following system:

Xfh+1 Yfh =A B C D Xfh Ufh (2.37)

A more efficient evaluation of the state, still based on the so-called “combined algorithm 2 ” proposed in [76], is implemented into the system identification pack-age used in this work. In the implemented procedure, instead, after the evaluation

(42)

of the state sequence Xfh, system matrices are obtained by solving the following system in a least-squares sense:

X+ Y− =A B C D X− U− (2.38)

where X+ is Xfh without the first column, X−, Y− and U− are Xfh, Yfh and Ufh without the last column, respectively. According to extensive simulation studies, using (2.38) instead of (2.37) has resulted in superior performance in terms of the variance of the residuals. Moreover, the implemented algorithm has a lower computation load given that only a single state sequence needs to be identified. More details and performance results on the software used for this work are reported in Appendix B.

(43)

Chapter

3

Towards Offset-free Economic MPC

1

In this chapter the problem of designing an EMPC algorithm that asymptot-ically achieves the optimal performance despite the presence of plant-model mismatch is addressed. An example of a continuous stirred tank reactor in which available EMPC and tracking MPC algorithms do not reach the op-timal steady state operation is presented. A new combination of offset-free disturbance model and the target optimization problem with a correction term that is iteratively computed to enforce the necessary conditions of optimality in the presence of plant-model mismatch is proposed. Finally, results of the proposed technique on the motivating example, highlighting the role of the stage cost function used in the finite horizon MPC problem, are given.

3.1 A motivating example

In order to motivate this work, we show the application of EMPC formulations to a chemical reactor to highlight how available methods are not able to achieve the optimal economic performance in the presence of modeling errors.

3.1.1 Process and optimal economic performance

The chemical reactor under consideration is a Continuous Stirred Tank Reactor (CSTR), in which two consecutive reactions take place:

A k1 −→ B k2

−→ C (3.1)

The reactor is described by the following system of ordinary differential equations (ODE): ˙cA= Q V(cA0− cA) − k1cA (3.2) ˙cB = Q V(cB0− cB) + k1cA− k2cB 1

(44)

Table 3.1: Actual reactor parameters.

Description Symbol Value Unit

Kinetic constant 1 k1 1.0 min−1

Kinetic constant 2 k2 0.05 min−1

Reactor volume V 1.0 m3

A feed concentration cA0 1.0 kmol_m3 B feed concentration cB0 0.0 kmol_m3

A price βA 1.0 _kmole

B price βB 4.0 _kmole

in which cA and cB are the molar concentrations of A and B in the reactor, cA0 and cB0 are the corresponding concentrations in the feed, Q is the feed flow rate, V is the constant reactor volume, k1 and k2 are the kinetic constants. The feed flow rate entering the reactor is regulated through a valve, i.e. Q is the manipulated variable. For the sake of simplicity, the reactor is assumed to be isothermal, so the fixed parameters of the actual system are shown in Table 3.1. The process economics can be expressed by the running cost:

`(Q, cB) = βAQcA0− βBQcB (3.3)

where βA, βB are the prices for the reactants A and B, respectively, also reported in Table 3.1.

Using the actual process parameters reported in Table 3.1, we can compute the process optimal steady-state, by solving the following optimization problem:

min Q βAQcA0− βBQcB (3.4a) subject to Q V(cA0− cA) − k1cA= 0 (3.4b) Q V(cB0− cB) + k1cA− k2cB = 0 (3.4c) 0 ≤ cA≤ cA0 (3.4d) 0 ≤ cB ≤ cA0 (3.4e)

The result of this optimization is Qopt = 1.043 m3/min, cA,opt = 0.511 kmol/m3 and cB,opt = 0.467 kmol/m3, which represents the most economic steady-state that the actual process can achieve.

(45)

3.1 A motivating example 33

3.1.2 Model and controllers

The definition of states, input, and outputs is the following:

x =cA cB , u =Q , y =cA cB . (3.5)

For controller design, the second kinetic constant is supposed to be uncertain, i.e. the value known by the controller is ¯k2, instead of k2. With these definitions the model equations become:

˙x1 ˙ x2 = u V(cA0− x1) − k1x1 u V(cB0− x2) + k1x1− ¯k2x2 (3.6)

We compare the closed-loop behavior of three EMPC algorithms, all designed according to Section 2.5 using the same nominal model (3.6), cost function, and a sampling time of τ = 2.0 min. Specifically, the target optimization problem is given in (2.27) and the FHOCP is given in (2.28), where the economic cost function is: `e(y(ti), u(ti)) = Z ti+τ ti `(u(t), y2(t))dt = Z ti+τ ti [βAu(t)cA0− βBu(t)y2(t)] dt (3.7) We note that the use of the cost function integrated over the sampling time is necessary to achieve an asymptotically stable closed-loop equilibrium. As a matter of fact, if the point-wise evaluation of `(·) was used as stage cost `e(·), the system would not dissipative [49], i.e. the closed-loop system would not be stable. The three controllers differ in the augmented model:

• EMPC0 is standard Economic MPC and uses no disturbance model, i.e. F (x, u, d) = f (x, u) and H(x, d) = h(x) = x.

• EMPC1 uses a state disturbance model, i.e. F (x, u, d) = f (x, u) + d and H(x, d) = h(x) = x.

• EMPC2 uses a nonlinear disturbance model [27], in which the disturbances act as a correction to the kinetic constants, i.e. F (x, u, d) is obtained by integration of the following ODE system:

˙cA= q V(cA0− cA) − (k1+ d1)cA (3.8) ˙cB = q V(cB0− cB) + (k1+ d1)cA− (¯k2+ d2)cB and H(x, d) = h(x) = x.

Since the state is measured, for EMPC0 we use ˆx_k|k = x(k). For EMPC1 and EMPC2 we use an extended Kalman filter (EKF) to estimate the current state ˆxk|k

Merging process optimization and advanced control: novel algorithms and performance monitoring strategies for sustained economic efficiency

Universit`

a di Pisa

Merging process optimization and advanced control:

novel algorithms and performance monitoring strategies

for sustained economic efficiency

Abstract

Acknowledgements

Contents

Chapter

1

Introduction

1.1

Model Predictive Control (MPC)

1.2

Real-Time Optimization (RTO)

1.3

MPC and RTO

1.4

Thesis Objectives

1.5

Major results

1.6

Thesis structure

Part I

Chapter

2

Background and literature review

2.1

Plant, model and constraints

c

2.2

Nominal tracking MPC

2.3

Offset-free tracking MPC

c

u

2.4

Real-time optimization with modifier-adaptation

2.5

Economic MPC

2.6

Moving Horizon Estimation

2.7

System identification

Chapter

3

Towards Offset-free Economic MPC

1

3.1

A motivating example