Universit`
a di Pisa
DIPARTIMENTO INGEGNERIA CIVILE E INDUSTRIALE Corso di Laurea Magistrale in Ingegneria Chimica
Tesi di Laurea Magistrale
Applicazione di algoritmi di controllo
avanzato per l’ottimizzazione economica
di processi chimici complessi
Relatore
Prof. Gabriele Pannocchia
Correlatore
Dott. Ing. Marco Vaccari
Controrelatore
Dott. Ing. Riccardo Bacci Di Capaci
Candidato
Federico Pelagagge
DEPARTMENT OF CIVIL AND INDUSTRIAL ENGINEERING Master’s degree in Chemical Engineering
Master degree thesis
Application of advanced control
algorithms and economic optimization to
complex chemical processes
Thesis advisor
Prof. Gabriele Pannocchia
Coadvisor
Dr. Marco Vaccari
Examiner
Dr. Riccardo Bacci Di Capaci
Candidate
Federico Pelagagge
Abstract
Current advanced control and optimization architectures in process indus-tries, based on a conventional hierarchy of economic optimization and control, can be far from being economically optimal. This thesis analyzes methods to reach and maintain optimal economic performance in industrial processes. The used offset-free Economic Model Predictive Control (EMPC) technique modifies the economic optimization problem and this modification allows convergence to a point that satisfies the necessary conditions of optimality of the actual process, i.e., the actual optimum of the plant. A major requirement of this algorithm is that, to estimate the modification required at each iteration, it is necessary to know the real gradients of the system at the current steady-state point, which is not a trivial task. Therefore, in order to build an EMPC industrially applicable, the algorithms have been enhanced with plant gradient estimation techniques. Five different modifier estimation techniques have been proposed. Four of them are based on steady-state perturbation methods, while, the last one is based on model identification. A number of examples have been selected for the application of the developed offset-free EMPC algorithm. In these examples, the actual process and the nominal MPC models are different and therefore would lead to different optimal equilibria. As result, all proposed EMPC algorithms converge to the economic optimum of the process under determinate conditions. Keywords: RTO, EMPC, Offset-free, Economic MPC, Optimization, Advanced control, Model predictive control.
Acknowledgements
Foremost, I would like to express my sincere gratitude to my advisor Prof. Gabriele Pannocchia for the continuous support in my master’s thesis internship, for his patience, motivation, enthusiasm, and immense knowledge. His guidance helped me in all the time of internship and writing of this thesis. I would like to thank my co-advisor, Dr. Marco Vaccari, he supported me greatly and was always willing to help me. You provided me with the tools that I needed to choose the right direction. Besides my advisors, I would like to thank my thesis examiner Dr. Riccardo Bacci Di Capaci for the encouragement, insightful comments, and hard questions. I would especially like to thank Prof. Dominique Bonvin, with whom I could chat about my research and he has always been so kind to listen and propose solutions to my problems.
I would like to acknowledge the research group from my internship at CPClab at Dipartimento di Ingegneria Civile e Industriale (DICI) of Università di Pisa.
I thank my study group for the stimulating discussions, the excellent cooper-ation, for the sleepless nights we were working together before all the deadlines, and for all the fun we have had in the last years.
Nobody has been more important to me than the members of my family. I would like to thank my parents and my grandparents for the support given during these years of study in Pisa.
Finally, I would like to offer my special thanks to my friends and to my housemates for providing happy distraction to rest my mind outside of my studies.
Pisa, October 2019 Federico Pelagagge
Abstract i
1 Introduction 1
2 Literature review 5
2.1 Nominal Model Predictive Control . . . 5
2.1.1 Models . . . 8
2.1.2 State estimation . . . 8
2.1.3 Target optimization and OCP . . . 13
2.1.4 Stability properties of MPC . . . 15
2.1.5 Stability properties of EMPC . . . 17
2.2 Real Time Optimization . . . 19
2.2.1 Modifier Adaptation . . . 23
2.2.2 Output modifier adaptation . . . 26
2.3 Offset-free MPC . . . 27
2.3.1 Augmented models and estimation . . . 28
2.3.2 Offset-free tracking MPC . . . 31
2.3.3 Offset-Free EMPC . . . 32
3 Process optimization: needs and challenges 38 3.1 The problem of process gradients evaluations . . . 38
3.1.1 Steady-state perturbation methods . . . 38
3.1.2 Dynamic perturbation methods (Dynamic model identifi-cation) . . . 39
3.1.3 Time analysis . . . 40
3.2 Gradient approximation methods . . . 40
3.2.1 Multivariable forward finite difference approximation . . 41
3.2.2 Secant methods (Broyden’s update) . . . 42
3.2.3 Simplex gradient . . . 43
3.2.4 System identification . . . 46
CONTENTS v 4 Proposed Methodologies 49 4.1 Method 1 . . . 49 4.2 Method 2 . . . 52 4.3 Method 3 . . . 53 4.4 Method 4 . . . 54 4.5 Method 5 . . . 55
4.6 Further modification to the algorithms . . . 57
4.7 Summary of proposed algorithms . . . 59
5 Application Examples 70 5.1 Application 1: CSTR with two consecutive reactions . . . 70
5.1.1 Actual plant dynamics . . . 70
5.1.2 Model reactor dynamics . . . 72
5.1.3 Simulations . . . 73
5.2 Application 2: Williams Otto reactor process . . . 87
5.2.1 Actual plant dynamics . . . 87
5.2.2 Model reactor dynamics . . . 89
5.2.3 Optimization Problem . . . 89
5.2.4 Simulations . . . 93
5.3 Application 3: Variation of Williams Otto reactor process . . . . 99
5.3.1 Revisiting the optimization Problem . . . 99
5.3.2 Simulations . . . 100
6 Comments and Analysis 105 6.1 Discussion of results . . . 105
6.2 Analysis of critical elements . . . 108
7 Conclusions 109 7.1 Summary of thesis contribution . . . 109
7.2 Possible future research . . . 110
Bibliography 113 A Complementary material 119 A.1 Useful function definition . . . 119
A.1.1 EMPC Stability . . . 120
A.2 Details of Williams Otto model . . . 122
A.2.1 System dynamics in mass basis . . . 122
1.1 Conventional hierarchical control and optimization structure. . . 1 2.1 MPC algorithm architecture. . . 5 2.2 MPC framework: at each time k the controller retrieves the
measurements from sensors and compute the optimal control move to be applied to the actuators, in order to obtain a suitable future behavior of the system. . . 6 4.1 Estimation of δ = ¯yk−1(¯uk−1, ˆd∗k−1) − ˆyk(uk−1) from the model
(blue line, figure on left) and estimation of ˆ¯yp,k from the
measure-ment of yp,k(uk−1) (black line, figure on right) . . . 58
4.2 Flow diagram for offset-free economic MPC algorithm. . . 61 4.3 Flow diagram for offset-free economic MPC algorithms Methods
1, 2, 3, 4. . . 62 4.4 Flow diagram for offset-free economic MPC algorithm Method 5. 63 5.1 CSTR: steady-states for plant and model (top) and steady-state
stage revenue (bottom). . . 73 5.2 Plot of input (Q), target input ( ¯Q), outputs and target outputs
(cA, ¯cA, cB, ¯cB). Comparison between nominal EMPC (EMPC0 )
on left and EMPC with linear disturbance model (EMPC1 and EMPC2 ). All controllers are unable to drive the flow rate to the optimal target. EMPC1 and EMPC2 uses a disturbance model, which guarantees offset-free tracking. . . 76 5.3 Plot of input (Q), target input ( ¯Q), outputs and target outputs
(cA, ¯cA, cB, ¯cB). Comparison between offset-free EMPC
(EMPC1-PG) and (MPC1-(EMPC1-PG) on left and between offset-free EMPC (EMPC2-PG) and (MPC2-PG) on right. All controllers are able to drive the flow rate to the optimal target. (MPC1-PG) and (MPC2-PG) uses a tracking cost on OCP. Perfect plant gradient
knowledge is assumed. Modifier filter update is set to σ = 0.25. 77
LIST OF FIGURES vii 5.4 Plot of the instant revenue. Comparison between several
offset-free EMPC: (EMPC1-PG) and (MPC1-PG) on left, (EMPC2-PG) and (MPC2-(EMPC2-PG) on right. All the controller are able to reach the optimal revenue. . . 78 5.5 Plot of actual Input (Q). Comparison between offset-free EMPC
controllers (M1, M2, M3 and EMPC2-M4 ). The red vertical lines separate the first simulation period where the modifier in set to 0 from the FDA estimation period where the input perturbations are executed to the last period where the algorithm is completely initialized. . . 79 5.6 Plot of actual input (Q) of offset-free EMPC controller
EMPC2-M5. The red vertical lines separate the first simulation period, where the modifier in set to 0 from the identification horizon, where the input dynamic perturbations are executed, to the last period where the algorithm is completely initialized. . . 80 5.7 Plot of actual input (Q) of offset-free EMPC controller
EMPC2-M3 at various σ. . . 81 5.8 Plot of actual input (Q) of offset-free EMPC controller
EMPC2-M3 at various σ. . . 82 5.9 Plot of actual input (Q) of offset-free EMPC controller
EMPC2-M3. Comparison between approximation A (no δ) and B (yes δ). . . 84 5.10 Plot of actual input (Q) of offset-free EMPC controller
EMPC2-M3. Comparison between option A, B and C at σ = 0.25. . . . 84 5.11 Plot of actual input (Q) of offset-free EMPC controller
EMPC2-M3. Comparison between option A, B and C at σ = 1. . . 85 5.12 Plot of actual input (Q), and noise outputs (cA, cB) of offset-free
EMPC controller EMPC2-M3. Comparison between different ns:
1, 3 and 8 at σ = 0.25. . . 86 5.13 Williams Otto reactor: steady-state stage revenue of model (red
plot) and actual plant (gray plot). Model optimum for ¯QB =
292.242dm3/min and ¯Tr = 78.4085 ℃; plant optimum for ¯QB=
293.587dm3/min and ¯T
r = 89.9896 ℃. . . 93
5.14 Plot of Inputs (QB, Tr), Outputs (cE,cP), with σ = 0.75.
Com-parison between a2EMPC1, a2EMPC1-PG and a2MPC1-PG. a2EMPC1 is unable to drive the inputs to their optimal tar-gets. Instead, the inputs are driven to their optimal targets by a2EMPC1-PG and a2MPC1-PG. Perfect plant gradient knowl-edge is assumed. Modifier filter update is set to σ = 0.75. . . . 95
5.15 Plot of objective revenue, with σ = 0.75 and M=1 for a2EMPC1, a2EMPC1-PG and a2MPC1-PG. . . 96 5.16 Comparison between M2, M3 and
a2EMPC1-M4. Plot of Inputs (QB, Tr), Outputs (cE,cP), with M = 15.
The red vertical lines separate the first simulation period, where the modifier in set to 0, from the FDA estimation period where the input perturbations are executed, to the last period where the algorithm is completely initialized. . . 97 5.17 Comparison between M2, M3 and
a2EMPC1-M4. Plot of objective revenue, with M=15 . . . 98 5.18 Comparison between M2, M3 and
a3EMPC1-M4. Plot of Inputs (QB, Tr), Outputs (cE,cP,cG), with M = 15
and cU
G = 0.5 kmol/m3. The bound on cG is not active. The
red vertical lines separate the first simulation period where the modifier in set to 0 from the FDA estimation period, where the input perturbations are executed to the last period where the algorithm is completely initialized. . . 101 5.19 Comparison between M2, M3 and
a3EMPC1-M4. Plot of objective revenue, with M=15 and cU
G= 0.5 kmol/m3.102
5.20 Plot of objective revenue, with M=15 and cU
G = 0.3 kmol/m3. . 103
5.21 Plot of Inputs (QB, Tr), Outputs (cE,cP,cG), with M = 15 and
cU
G= 0.3 kmol/m3. The bound on cG is active. The red vertical
lines separate the first simulation period, where the modifier in set to 0, from the FDA estimation period where the input perturbations are executed, to the last period where the algorithm is completely initialized. . . 104
List of Tables
3.1 Summary of gradient estimation techniques . . . 41
4.1 Summary of proposed gradient estimation techniques . . . 50
5.1 Application 1: System parameters for the true plant . . . 71
5.2 Application 1: Summary of EMPC controllers . . . 74
5.3 Application 1: Revenue comparison tracking and economic in 10 min. . . 78
5.4 Application 1: Revenue comparison offset-free MA EMPC in 70 min. . . 79
5.5 Application 1: Revenue comparison EMPC2-M3 at different σ in 70 min. . . 82
5.6 Application 1: Revenue comparison EMPC2-M3 at different M in 70 min. . . 83
5.7 Application 1: Revenue comparison EMPC2-M3 approximation A and B in 70 min. . . 84
5.8 Application 1: Revenue comparison EMPC2-M3 option A, B and C in 70 min. . . 85
5.9 Application 1: Revenue comparison EMPC2-M3 in 70 min (noise and different ns). . . 87
5.10 William-Otto Reactor: Process Parameters . . . 88
5.11 Williams-Otto Reactor: Model Parameters . . . 89
5.12 Application 2: Summary of EMPC controllers . . . 94
5.13 Application 2: Overall revenue, comparison in 150 min. . . 96
5.14 Application 2, simulation 2: Revenue comparison in 850 min. . . 99
5.15 Application 3, simulation 1: Revenue comparison in 850 min. . . 102
5.16 Application 3, simulation 2: Revenue comparison in 850 min. . . 102
6.1 Summary of EMPC tuning parameters . . . 106
(·)† Pseudoinverse operator
(·)? Optimal values of quantities
(·)0 Transpose operator (·)k Time index of plant
(·)p Plant quantities
¯
(·) Quantities at steady state ˆ
(·) Predicted quantities ˆ
(·)∗ Estimated quantities I Field of integer number R Field of real number
M Number of sampling times used for plant gradient estimation N Horizon length of MPC
Chapter 1
Introduction
In the process industries the conventional hierarchy of economic optimization and control is set since many years. The information management, decision making, and control system are separated into several layers as depicted in Figure 1.1.
Planning and Scheduling
Real Time Optimization
Model predictive control (MPC)
Distributed control system (DCS)
Actuators and sensors Planning and
Scheduling
Real Time Optimization
Model predictive control (MPC)
Distributed control system (DCS)
Actuators and sensors
Figure 1.1: Conventional hierarchical control and optimization structure.
The timescale of each layer decreases reasonably from the top layers to the bottom layers. The top layer supply chain planning and scheduling is typically executed every day or even every week. This layer processes the sequences of production operations and plans how to apply the main production actions to follow these sequences. The second layer, in Figure 1.1referred as Real-Time Optimization (RTO), carries out a steady-state economic optimization of the plant variables, updated on a timescale of hours. The RTO sends the results of its optimization as a setpoint to the third layer, normally referred to as the advanced
process control system. This control layer is often implemented with Model Predictive Control (MPC) due to its flexibility, performance, robustness, and its property to directly handle hard constraints on both inputs and states. This layer guides the plant from transient state to the setpoint and rejects dynamic disturbances that enter to the system. The advanced process control system calculates control actions that are passed to the distributed control systems layer (DCS), usually represented by proportional and proportional-integral (PI) controllers; or occasionally three-term PID controllers with derivative action. The DCS are constituted by basic control logics that manage to convert the control action, derived from the MPC, into different actions, that are passed at the various underlying actuators. At the very lowest level, there are control loops associated with individual actuators, where the control action timescale is comparable to continuous time. The sensors collect signals to be converted into measurements by PI/PID controllers [1].
There are an increasing number of problems for which the hierarchical separation of economic analysis and control is either inefficient or inappropriate. Flexibility and efficiency are requirements that the control system must now address. For a growing number of applications the separation of economic information and control purpose in the hierarchical control structure is no longer optimal nor desirable [2]. Methods for merging the two optimization layers of Model predictive control and Real time Optimization have been recently formulated in the literature, leading to so called Economic MPC (EMPC) algorithms. The EMPC takes the economic cost function of the RTO and substitutes it into the MPC problem. An objective of the offset free EMPC algorithms is to achieve and sustain the optimal economic performances in industrial processes despite plant-model mismatch and persistent disturbances.
Purpose of the thesis
The present thesis work is aimed at the development of offset free economic predictive control algorithms (EMPC). These algorithms are validated on ex-amples of complex chemical processes. More specifically, the very common industrial case in which the model used by the controller does not fully adhere to the dynamics of the controlled process it has been studied.
A combination of the modifier-adaptation and offset-free MPC formulations has led to a novel offset free EMPC algorithm. This technique modifies the economic optimization problem and the modification allows convergence to a point that satisfies the necessary condition of optimality of the actual process. This algorithm, under constraint satisfaction, is able to achieve the real optimum of the plant even in the presence of modeling errors and external disturbances.
3 A multipurpose code for MPC simulations that was written in Python within the Chemical Process Control Laboratory of DICI, has been the central tool used to develop results and algorithmic analysis. A number of examples of increasing complexity are selected for the application of the developed offset free EMPC algorithm.
A major requirement of this algorithm is that, to estimate the modification at each iteration, it is mandatory to know the real gradients of the system at the current steady state point, which is not a trivial task.
Therefore, in order to build an EMPC applicable to the reality, the proposed algorithms are enhanced with a number of plant gradient estimation techniques.
Outline
The rest of the thesis is structured as follows:
In Chapter 2 the state of the art of model predictive control, real time op-timization and the economic model predictive control are outlined. In particular we focus on the modifier-adaptation technique, initially formu-lated for the RTO and introduced also in offset free EMPC level in recent works.
The used offset-free EMPC technique modifies the economic optimization problem and the modification allows convergence to a point that satisfies the Necessary Condition of Optimality (NCO) of the actual process, i.e., the real optimum of the plant. In order to estimate the modification required at each iteration, it is necessary to estimate the real gradients of the system at the current steady-state point.
In Chapter 3 the problem of estimation of gradients and several gradient estimation techniques are discussed. We focus on steady-state perturbation and on dynamic perturbation methods. Stationary data-set are used by steady-state perturbation methods to calculate approximations of the Jacobians. In another method, a linear system model is identified using transient data-set, through dynamic model identification techniques. This model is used to obtain the gradient estimation.
In Chapter 4 the proposed gradient estimation techniques and the offset-free model predictive control algorithms are formulated. Five different modifier estimation techniques are proposed. Four of them are based on steady-state perturbation methods. While, the last one is based on dynamic perturbation methods and specifically on model identification. An initialization phase is required for all mentioned methods. Besides, once
the algorithms are initialized no further perturbation steps are necessary. The modifiers are directly updated in two of the proposed methods. In the others, the gradient of the process is first estimated and the modifier is subsequently calculated.
Chapter 5 describes the process examples considered and the result of the application of the offset-free EMPC algorithms, which are carried out on the simulations of the examples. A number of examples are selected for the application of the developed offset-free EMPC algorithm. In these examples, the actual process and the nominal MPC models are different and therefore would lead to different optimal equilibria. Gradient estimation techniques are validated and all proposed EMPC algorithms converge to economic optimum of the process under determinate conditions. However, in two applications two of the methods failed to converge in all tested simulations.
In Chapter 6 the various proposed estimation algorithms are compared; the influence of the various tuning parameters is evaluated. Critical elements of the algorithms are analyzed, in particular the robustness of the estimated gradients.
In Chapter 7 a summary of the work done and the contributions of the thesis has been reported. In addition, the possible ideas for future research are proposed.
Chapter 2
Literature review
2.1
Nominal Model Predictive Control
Model Predictive Control (MPC) algorithms were developed in industrial environments since the 70s [3]. It is advanced control methodology which has made a significant impact on industrial control engineering [4]. There are several reasons for the success of MPC in chemical industry. The more stringent production requests in term of economic optimization, maximum exploitation of production capacities and minimum variability in product qualities, have indeed contributed to the development of MPC. Moreover, the MPC can take into account actuator constrains leading often to more profitable operation.
Figure 2.1: MPC algorithm architecture.
The basic idea of MPC is to consider a dynamic model of the real system.
MPC solve, at every decision time, an optimal control problem for computing the best control action to produces a desirable system behavior by minimiza-tion/maximization of the objective to achieve. Besides, modeling is the main task in MPC, but also its downside.
Most MPC algorithms are divided into three modules, as depicted in Figure 2.1.
• A state estimator receives the current output measurement yp,k, and
updates the predictions, computed at the previous decision time of the state xk.
• A steady-state optimization module (target optimization problem) com-putes the state, input and output targets (¯xk, ¯uk, ¯yk) that match as close
as possible the desired external set points while respecting the imposed constraints.
• A dynamic optimization module, called Optimal Control Problem (OCP), computes the optimal trajectory from the current state estimate ˆxk, to
the target computed by the steady-state optimization module.
Output
Plant estimate
Plant prediction
Plant data
Time (k)
Input
Input seq.
Input prediction
Figure 2.2: MPC framework: at each time k the controller retrieves the measure-ments from sensors and compute the optimal control move to be applied to the actuators, in order to obtain a suitable future behavior of the system.
2.1. Nominal Model Predictive Control 7 Usually, the two optimization modules are equipped with quadratic cost functions.
The first input of the optimal control sequence, with length equal to the prediction horizon of the controller, is then injected into the plant. Finally, the overall algorithm is repeated the next decision time, given the new current output measurement [4]. The receding horizon idea of MPC is summarized in Figure 2.2.
The conventional controllers compute the control law off-line, taking into account only the tracking error and moreover, they do not handle safety, quality and operation constraints.
Dynamic Programming (DP) technique may be used to solve a feedback version of the same optimal control problem, obtaining a receding horizon control law for all the possible states. DP would be preferable, since it provides a control law that can be implemented simply; however obtaining a solution is arduous, if the state dimension is high [5].
The main benefit of MPC is that open-loop optimal control problems can often be solved quickly enough to allow its use even though the controlled system is nonlinear, and hard constraints on states and controls must be satisfied.
Economic MPC
The setpoints for the traditional tracking MPC come from the upper layer referred to the RTO, as we have seen in the Chapter 1. The quadratic cost of Tracking MPC may not be adequate for managing the process economic performance. The hierarchical division limits the achievable flexibility and economic performance that are requested nowadays by many processes. With a simple example it can be shown that in many cases a positive deviation from the target may represent a profit, while a negative deviation from the target may represent a loss (or vice versa). Consider a reactor with a steam jacket; supplying more steam to the jacket than the target is more costly in terms of energy consumption of the reactor, while supplying less steam consumes less energy [6].
In order to improve the effective use of dynamic and economic information throughout the hierarchy, the second one can be moved into the control layer. This approach replaces the traditional tracking objective function of OCP, with the economic stage cost function used in the RTO layer. This formulation takes the name of Economic MPC (EMPC) [7].
Since EMPC accounts directly for process economics, which is consistent with the main ideas of smart manufacturing, market-driven manufacturing, and real-time energy management, its popularity among researchers has significantly
increased within the last few years [6]. It has to be highlighted that the economic optimization is provided only by the EMPC layer, while the RTO one is completely eliminated.
As already mentioned in the Chapter 1, in many applications achieving a reasonable trade-off between safety, i.e. stability, and economic process operation is of key importance.
Powerful numerical methods for implementation of EMPC for large scale process control applications are now available.
2.1.1
Models
Consider a time-invariant dynamical system in the form:
x+p = fp(xp, u) (2.1a)
yp = hp(xp) (2.1b)
in which xp, x+p ∈ Rnx denote the current and successor plant states, u ∈ Rnu is
the control input, yp ∈ Rny is the output.
The plant output is measured at each time k ∈ I. The functions fp :
Rnx × Rnu × Rnw → Rnx and hp : Rnx × Rnv → Rny are not known precisely
but assumed continuous.
The model of the system can be written as a time-invariant dynamical system:
x+ = f (x, u)
y = h(x) (2.2)
in which x, x+∈ Rnx denote the current and successor model states and y ∈ Rny is the output. The functions f : Rnx × Rnu → Rnx and h : Rnx → Rny are assumed to be continuous.
We consider constraints on the states and inputs, (x, u) ∈ Z, and outputs y ∈ Y, where Z denotes the feasible set and it is assumed closed.
Z = X × U :=(x, u) ∈ Rnx+nu|g(h(x), u) ≤ 0 (2.3) Y = {y ∈ Rny|(x, u) ∈ Z and y = h(x)} (2.4) in which g : Rny× Rnu → Rng is the constraint function.
2.1.2
State estimation
Often in MPC only part of measurement of the whole state vector is measured. Besides, the output measurement is corrupted with sensor noise and the state
2.1. Nominal Model Predictive Control 9 evolution is corrupted with process noise. In this case, a state estimate ˆx∗
k is
used. In control engineering the instrument used to estimate unknown state variables based on process measurement is called observer.
Observer have been widely used in science and engineering fields and they are constructed based on the plant model. Moreover, in a noisy environment, a state observer acts also as a filter to reduce the effect of noise on the measurement [8]. In order to understand the fundamentals of state estimation, the fluctuations in the data must be addressed. The probability theory is a versatile approach to model these fluctuations. Hence, lets us introduce the nomenclature used in the analysis developed below:
x ∼ N(˜x, P)means that the random variable x is normally distributed with mean ˜x and covariance P.
Consider a plant in (2.1) to be described by the following discrete-time Linear Time Invariant (LTI) system:
xk+1 = Axk+ Gwk (2.5a)
yk = Cxk+ vk (2.5b)
x0 given (2.5c)
in which x0 is given, A and B are assumed known, process noise wk and
measurement noise vk are random variables defined as follows:
x0 ∼ N (˜x0, P0), wk∼ N (0, Q), vk ∼ N (0, R) (2.6)
The general structure of a state observer for the system (2.5) is as follows: ˆ x∗k= ˆxk+ L(yk− ˆyk) (2.7a) ˆ xk+1 = Aˆx∗k (2.7b) ˆ yk = C ˆxk (2.7c)
in witch L is the gain matrix of the system [4]. Let us define the symbol ˆx∗
k to denote the filtered estimate of xk obtained
using the output measurement at time k. Moreover, ˆxkand ˆyk,are the predicted
estimate of xk and yk,respectively, obtained at time k − 1 using (2.5).
Combining the system equations, we obtain the following equation:
ˆ
xk+1 = (A − ALC)ˆxk+ ALyk (2.8)
The state estimation error is then define as εk= xk− ˆxk.
We consider the next iteration state estimation error εk+1 that can be
Therefore for k → ∞ the condition for convergence εk → 0is that (A − ALC)
is strictly Schur, that is max
i |λi(A − ALkC)| ≤ 1, in which λi eigenvalue of
(A − ALC).
For the system (2.5) the best choice for the filter gain L can be obtained by applying the theory of probability as developed in the book [7].
Optimal linear estimator (Kalman filter)
Assume that x and y are jointly normally distributed and statistically independent as the assumption in [7]. Consider the output measurement at time k yp,k, xk∼ N (ˆxk, ˆPk)and from the previous iteration, we have that:
" xk yk # = " I 0 C 0 # " xk vk #
Since xk and yk are statistically independent, there holds:
" xk yk # ∼ N " ˆ xk Cxˆk # , " ˆ P P Cˆ 0 C ˆP C ˆP C0 + R #!
Therefore, because of x and y are jointly normally distributed, the conditional density of x given y is normal, (xk|yk) ∼ N (ˆx∗k, ˆP
∗ k) in which: ˆ x∗k= ˆxk+ Kk(yk− C ˆxk) (2.9) Kk= ˆPkC 0 (C ˆPkC 0 + R)−1 (2.10) ˆ Pk∗ = ˆPk− ˆPkC 0 (C ˆPkC 0 + R)−1C ˆPk (2.11)
when forecast from k to k + 1 using the model (2.5a), since wk is independent
of xk, it follows: xk+1 ∼ N (ˆxk+1, ˆPk+1) in which: ˆ xk+1 = Aˆx∗k, (2.12) ˆ Pk+1 = A ˆPk∗A 0 + GQG0 = A ˆPkA 0 − A ˆPkC 0 (C ˆPkC 0 + R)−1C ˆP kA+ GQG 0 (2.13) The Kk is the Kalman filter and it is the optimal value of gain L.
The Riccati equation (2.13) converges to P for k → ∞, which is the solution of the discrete algebraic Riccati equation:
2.1. Nominal Model Predictive Control 11 Moreover, the steady-state Kalman filter is defined as follows:
K = P C0(CP C0 + R)−1 (2.15)
The condition for convergence εk → 0, for k → ∞, is that (A − AKC) is
strictly Schur, i.e., max
i |λi(A − AKC)| ≤ 1, where λi is the i-th eigenvalue of
(A − AKC).
The observer theory for nonlinear models is less complete respect to the one for the linear case. Optimal state estimation problem can still be defined, but it can not be solved exactly. Some approximations that can be used are the steady-state Kalman filter of the linearized process, the Extended Kalman Filter (EKF) and the Moving Horizon Estimation (MHE).
Extended Kalman Filter (EKF)
The Extended Kalman Filter is one of the most used non-linear state-space estimator [8]. The main idea of the EKF is to first generate a linearization of the nonlinear system, and then apply the linear Kalman filter equations. Consider a plant in (2.1) to be described by the following a nonlinear system in the form of:
x+ = f (x, w) (2.16)
y = h(x) + v (2.17)
x0 given (2.18)
where f, h are assumed known.
A linearization of the non-linear state space model by making a first-order Taylor expansion is defined as follows:
f(xk, wk) ≈ f (ˆx∗k,0) + ¯Ak(xk− ˆx∗k) + ¯Gkwk (2.19) h(xk) ≈ h(ˆxk) + ¯Ck(xk− ˆxk) (2.20) in which: ¯ Ak = ∂f(x, w) ∂x (ˆx∗ k,0) ¯ Ck= ∂h(x) ∂x ˆ xk ¯ Gk= ∂f(x, w) ∂w (ˆx∗ k,0) (2.21)
The linearized state-space system is therefore defined as follows:
xk+1 = ¯Akxk+ (f (ˆxk∗,0) − ¯Akxˆ∗k) (2.22a)
Then, we can apply the linear Kalman filter equations to (2.22a). The recursion is summarized in the following equations, with x0 = ¯x0 and P0 = Q0
as initialization [7]: Filtering Step ˆ x∗k= ˆxk+ Kk(yk− h(ˆxk) + ¯Ckxˆk− ¯Ckxˆk) = ˆxk+ Kk(yk− h(ˆxk)) (2.23) Kk= ˆPkC¯ 0 k(R + ¯CkPˆkC¯ 0 k) −1 (2.24) ˆ Pk∗ = ˆPk− KkC¯kPˆk (2.25) Prediction step ˆ xk+1 = f (ˆx∗k,0) (2.26) ˆ Pk+1 = ¯AkPˆkA¯ 0 k+ ¯GkQ ¯G 0 k (2.27)
Observability. The concept of observability implies that is possible to esti-mate the state given output measurements.
xk+1 = Axk+ Buk (2.28a)
yk = Cxk (2.28b)
A LTI system (2.28) is observable if there exists a finite N measurements (y0, . . . , yN −1), that for every x0, it is possible to establish uniquely the initial
state x0. The condition for observability of a LTI system, given nxmeasurements,
is provided by the application of a fundamental theorem of linear algebra [7]: rank O = nx, in which O =
h
C CA · · · CAnx−1 i0
is the observability matrix. Moreover, it is important the following checking condition:
Lemma 2.1 (Hautus lemma for observability). A system is observable if and only if:
rank"¯λI − A C
#
for all ¯λ ∈ eig(A) (2.29)
The property of observability is important for the analysis of the convergence of the estimates. It can be demonstrated [7], that for an observable system in form of (2.28) and noise-free measurements yt = (Cx0, CAx0. . . , CATx0),the
optimal linear state estimate converges to the state ˆxT → xp,T for t → ∞.
The definition of observability for a general nonlinear system is more complex. Definition 2.2 (Observability [7]). System (2.1) is observable if there exists a finite No ∈ I and γ(·) ∈ K such that for any two initial states z1 and z2 and
2.1. Nominal Model Predictive Control 13 control sequence u, and all k ≥ No
|z1 − z2| ≤ γ kyz1,u− yz2,uk0:k
in which yz,u is the sequence of outputs {y0, y1, . . . }, corresponding to the
sequence of inputs u = {u0, u1, . . . }
2.1.3
Target optimization and OCP
Nominal tracking MPCTarget calculation. Given the current state estimate ˆx∗k, an MPC algorithm needs to compute a feasible equilibrium target. In the general case the target problem to be solved is the following:
min
x,u,y`s(y, u) (2.30a)
subject to:
x= f (x, u) (2.30b)
y= h(x) (2.30c)
y ∈ Y, u ∈ U (2.30d)
where `s : Rny × Rnu :→ R is the steady-state cost function. Problem 2.30
is assumed feasible and its unique solution is denoted as (¯xk,u¯k,y¯k).
Typical tracking cost function can be constructed as follows:
`s(y, u) = 1 2(y − ¯ysp) 0 Qtop(y − ¯ysp) + (u − ¯usp) 0 Rtop(u − ¯usp) (2.31) where ¯ysp and ¯usp are output and input setpoints, respectively and Qtop and
Rtop are definite positive weight matrices. These setpoint usually comes from
the RTO. The above problem are solved only when the setpoint changes.
Optimal Control Problem (OCP). Let x = {x0, x1, · · · , xN} and
u = {u0, u1, · · · , uN −1} be, respectively, a state and an input sequence. For
i= 0, . . . , N − 1, let yi = h xi
be the model output corresponding to a state xi and disturbance estimate ˆdk. Furthermore, let ˜xi = xi− ¯xk for i = 0, . . . , N,
and ˜ui = ui − ¯uk for i = 0, . . . , N − 1. Every decision time k, the following
finite-horizon OCP is solved:
min
x,u VN(x, u) = minx,u N −1
X
i=0
subject to: (2.32b) x0 = ˆx∗k (2.32c) xi+1= f xi, ui i= 0, . . . , N − 1 (2.32d) h xi ∈ Y, ui ∈ U i= 0, . . . , N − 1 (2.32e) ˜ xN ∈ Xf (2.32f)
in which `t : Rnx× Rnu → R≥0 is a strictly positive definite convex function.
Vf : Rnx → R≥0and Xf are, respectively, a terminal cost function and a terminal
set, designed according to the stabilizing conditions [5, sec 2.5]. Typical tracking cost to go function can be constructed as follows:
`t(x, u) = 1 2 x˜ 0 iQOCPx˜i+ ˜u 0 iROCPu˜i (2.33) where QOCP and ROCP are definite positive weight matrices.
If problem (2.32) is feasible, its solution is denoted by (x?
k,u?k) and the
associated receding horizon implementation, the control law is well defined and given by:
uk= u?0,k
Nominal EMPC
The fundamental change in the EMPC problem is the redefinition of the stage cost `(y, u) to represent the process economics. The stage cost is not always positive definite with respect to some target equilibrium point of the model as in a tracking problem.
0 = ` (¯x,u) ≤ `(x, u)¯ for all admissible (x, u) (2.34) The condition in equation (2.34) is relaxed and it can happen that `(¯x, ¯u) > `(x, u)for some feasible pair (x, u) that is not a steady state. This unconventional formulation of MPC was originally proposed in [9] in the context of MPC in the presence of unreachable setpoints.
Lets us now define the optimal steady-state solution of the system from the economic perspective. The optimal steady-state solution (¯x?,u¯?,y¯?)is defined
as the solution to the following optimization problem. (¯x?,u¯?,y¯?) = arg min
(x,u,y)
2.1. Nominal Model Predictive Control 15 subject to:
x= f (x, u) (2.35b)
y= h(x) (2.35c)
0 ≥ g(y, u) (2.35d)
Relevant factors to be considered are how much economic performance improvement is possible and how different is the closed loop dynamic behavior.
Note that in a non-linear system the operation at the steady state may not be the best possible dynamic behavior of the closed-loop system [7].
Let us show now the simplest version of an economic MPC problem, with a terminal constraint. The MPC objective function is set as a sum of stage costs along the prediction horizon:
min
x,u VN(x, u) = minx,u N −1 X i=0 `(yi, ui) (2.36a) subject to: x0 = ˆx∗k (2.36b) xi+1= f (xi, ui) i= 0, . . . , N − 1 (2.36c) yi = h (xi) i= 0, . . . , N − 1 (2.36d) 0 ≥ g (yi, ui) i= 0, . . . , N − 1 (2.36e) xN = ¯x (2.36f)
Stage-cost functions can be defined as a sum of prices of the raw materials and utilities, and the revenue of the products.
There exists at least one point (¯x, ¯u) ∈ Z satisfying ¯x = f (¯x, ¯u) . Besides, the stage cost `(y, u) is lower bounded for (x, u) ∈ Z but not in general positive definite.
The solution to the optimal control problem (2.36) exists and the control law, κN(·) = u?0 is hence well defined. The closed-loop system is given by
x+ = f (x, κN(x)) (2.37)
2.1.4
Stability properties of MPC
Every iteration of MPC the plant is optimized over the prediction horizon, without considering what happens beyond the prediction horizon. Therefore, the plant can reach a state at which is impossible to stabilize [4].
In this analysis the nominal model is assumed perfect and there are no uncertainties. An equilibrium point is defined stable if a small perturbation of the state or input results in a continuous perturbation of the consecutive state and input trajectories.
A way for ensuring stability is to add a terminal constraint which forces the state to take a particular value at the end of the prediction horizon [4]. It is easy to prove this condition by using a Lyapunov function (Definition A.6) in the case when the state is to be driven to the origin, for a given initial condition. In the case with no disturbances and with terminal constraint, the state is null at the end of the finite horizon and therefore the control action is null as well. Hence, the system stays at the origin.
A relaxation on the terminal constrain condition is the idea of using a terminal constraint set which contains the origin, rather than a single point.
Consider the state sequence x that is constrained a priori to be a solution of x+ = f (x, u). It is possible to express the OCP in the equivalent form of
minimizing, with respect to the decision variable u, as follows:
VN0(x) = min
u {VN(x, u)|u ∈ UN(x)} (2.38)
in which the control constraint set UN(x) is the set of control sequences u
satisfying the state and control constraints. The cost is a function of the initial state x and the control sequence u, therefore the OCP (2.38) is a parametric optimization problem in which the decision variable is u, and both the cost and the constraint set depend on the parameter x.
Let XN be the set of states in X for which (2.38) has a solution
XN := {x ∈ X|UN(x) 6= ∅}
In order to ensure the asymptotic stability of the origin, the following assumptions must be established.
Assumption 2.3 (Continuity of system and cost). The function f : X × U → Rn, ` : X × U → R≥0 and Vf : Xf → R≥0 are continuous and f(0, 0) = 0,
`(0, 0) = 0 and Vf(0) = 0.
Assumption 2.4. The sets X and Xf are closed, Xf ⊆ X and U are compact;
each set contains the origin.
Assumption 2.5 (Basic stability assumption). Vf(·), X and `(·) satisfied the
2.1. Nominal Model Predictive Control 17 1. For all x ∈ Xf, there exist (x, u) ∈ X × U satisfying:
f(x, u) ∈ Xf
Vf(f (x, u)) − Vf(x) ≤ −`(x, u)
2. There exist K∞ functions (Definition A.2) α1(·) and αf(·) satisfying
`(x, u) ≥ α1(|x|) ∀x ∈ XN, ∀u such that (x, u) ∈ X × U
Vf(x) ≤ αf(|x|) ∀x ∈ Xf
Assumption 2.6 (weak controllability). There exists a K∞ function α(·) such
that:
VN0(x) ≤ α(|x|) ∀x ∈ XN
where VN(x) = VN(x, u?k).
Assumption (2.6) bounds the cost of steering an initial state x to Xf,
considering only the states that can be steered to Xf in N steps and requires
that this cost to be not excessive.
The following theorem establishes asymptotic stability of a wide range of MPC systems
Theorem 2.7 (Asymptotic stability of the origin). Suppose Assumptions2.3, 2.4, 2.5, and 2.6 are satisfied. Then
1. There exists K∞ functions α1(·) and α2(·) such that for all x ∈ XN
α1(|x|) ≤ VN0(x) ≤ α2(|x|) (2.39)
VN0 (f (x, κN(x))) − VN0(x) ≤ −α1(|x|) (2.40)
2. The origin is asymptotically stable in XN for x+ = f (x, κN(x)).
In [7, sec 2.5.5] the hypotheses of the Theorem2.7 have been verified and hence, the asymptotic stability of the origin in XN may be established.
2.1.5
Stability properties of EMPC
The stability conditions for the EMPC are different with respect to tracking MPC. For a class of nonlinear systems and economic stage costs, it is possible to constructs a suitable Lyapunov function, for which the optimal steady-state solution of the economic stage cost is an asymptotically stable solution of the closed-loop system under economic MPC [10].
Lets us assume that f(·) and `(·) are Lipschitz continuous on the admissible set Z, that is there exist Lipschitz constants Lf, Ll > 0 such that for all
(x1, u1), (x2, u2) ∈ Z
|f (x1, u1) − f (x2, u2)| ≤ Lf|(x1, u1) − (x2, u2)| (2.41)
|`(x1, u1) − `(x2, u2)| ≤ Ll|(x1, u1) − (x2, u2)| (2.42)
We also assume a weak form of controllability.
Assumption 2.8(Weak Controllability). There exists a K∞ function γ(·) (Def.
A.2) such that for every x ∈ XN, there exists u such that (x, u) ∈ Z and N −1
X
k=0
|uk− ¯u| ≤ γ(|x − ¯x|) (2.43)
With Assumption 2.8, lets us confine attention to the initial states x, that can be steered to ¯x in N steps, while satisfying the control and state constraints. Moreover, Assumption 2.8requires that the cost of the input sequence is not too large. The set XN is forward invariant, that is x ∈ XN implies f(x, u(x)) ∈ XN,
because of the terminal constraint.
Moreover, we assume the following property.
Assumption 2.9 (Strong Duality of Steady-State Problem). There exists a multiplier ¯λ so that (¯x, ¯u) uniquely solves:
min x,u `(h(x), u) + [x − f (x, u)] 0 ¯ λ (2.44a) subject to: g(x, y) ≤ 0 (2.44b)
Besides, there exists a K∞-function (Def. A.2) β such that the "rotated" stage
cost function Lrot(x, u) = `(x, u) + [x − f (x, u)] 0 ¯ λ − `(¯x,u)¯ (2.45) satisfies Lrot(x, u) ≥ β(|x − ¯x|) (2.46)
for all (x, u) satisfying g(x, u) ≤ 0.
Finally it is possible to define the following theorem.
Theorem 2.10 (Lyapunov Function [10]). If Assumption 2.8 and 2.9 hold the steady-state solution of the closed-loop system x+ = f (x, u(x)) is asymptotically
2.2. Real Time Optimization 19 stable with XN as the region of attraction, and admits a Lyapunov function ˜VN =
VN(x) + [x − ¯x] 0¯
λ − N `(h(¯x), ¯u) such that ˜VN(x+) ≤ ˜VN(x) − Lrot(h(x), u(x)).
The Theorem 2.10 is demonstrated in [10] to deliver the same solution of problem (2.36) with the introduction of the following “rotated” MPC problem, that uses L as stage cost:
min x,u ˜ VN(x, u) = min x,u N −1 X i=0 Lrot(yi, ui) (2.47a) subject to: x0 = ˆxk (2.47b) xi+1= f (xi, ui) i= 0, . . . , N − 1 (2.47c) yi = h (xi) i= 0, . . . , N − 1 (2.47d) 0 ≥ g (yi, ui) i= 0, . . . , N − 1 (2.47e) xN = ¯x (2.47f)
The stability result for the EMPC can be also established with the use of the concept of dissipativity [11]. A short review of this method is reported in AppendixA.
2.2
Real Time Optimization
MPC design is usually oriented to maintain steady-state safe operations, rather than maximizing plant profit. Economic optimizer are more often steady-state and provide setpoints for dynamic controllers. A system model can be used to calculate optimal setpoints for the different devices in the system subject to operational and economical constraints.
Optimization of process operation in the presence of uncertainty has received wide attention recently. Faced with growing competition, it represents the natural option for decreasing production costs, improving product quality, and meeting safety requirements and environmental regulations [12].
In the chemical industry, optimal operation of many plants is extremely difficult to achieve, because to inaccurate plant models or process disturbances. Real Time Optimization (RTO) has become the solution to this problem for a significant number of chemical and petrochemical plants [13]. Various RTO techniques are available in the literature and can be classified in two broad families depending on which a process model is used or not. Model-based RTO usually employs nonlinear first-principles models describing the steady-state behavior of the plant [14].
An accurate process model can seldom be found with affordable effort. Real-time optimization (RTO) comprehend a group of optimization methods that combine process measurements in the optimization framework to carry a real process or plant to optimal performance, provided at the same time the satisfaction of the constraints. The classical sequence of steps for process optimization includes [15] process modeling, numerical optimization using the process model and Injection of the model-based optimal inputs to the plant. Uncertainty can have three main sources: process disturbances, structural plant-model mismatch when the structure of the plant-model is not perfect, for example, for unknown phenomena or neglected dynamics and parametric uncertainty when the values of the model parameters do not match to the reality of the process.
Repeated identification and optimization. The most classical strategy is to use process measurements to estimate some uncertain parameters to update the model. This is the main idea of the “Model-parameter adaptation two-step” approach [15]. The difference between predicted and measured outputs is employed to update the model parameters. Therefore, new inputs are computed on the basis of the updated model. The procedure is repeated until convergence is achieved. In presence of structural plant-model mismatch this method can not well reproduce the real plant operation [16]. The requirement for guaranteeing the convergence is the satisfaction of the model-adequacy conditions, i.e. criteria that determine whether a model is adequate for use in an RTO scheme, but the model-adequacy conditions are difficult to both achieve and verify [15].
Modifier adaptation. The study of a modified two-step approach, referred to as Integrated System Optimization and Parameter Estimation (ISOPE) is motivated by the hope of an easier convergence towards the plant optimum. A modification of the cost function in the optimization problem was used for the first time by Roberts [17]. ISOPE combines output measurements with estimates of the input gradients of the plant outputs. The plant cost gradient is evaluated using these gradients and it is exploited to modify the cost function of the optimization problem. The object of the use of estimates of the plant gradients is to enforce the Necessary Condition of Optimality (NCO) matching between the model and the plant. Furthermore, the modified model is a candidate to solve the plant optimization problem. The model parameters are updated on the basis of output measurements, and the cost function is modified by the addition of a term based on estimated plant gradients [15].
When RTO is equipped with a fixed process model, then the measured plant constraints are exploited to shift the predicted constraints in the model-based optimization problem [18]. Nevertheless, this is the main idea in Modifier
2.2. Real Time Optimization 21 Adaptation (MA) that exploits the plant constraints and gradients to modify the cost and constraint functions in the model-based optimization problem without updating the model parameters [19].
Direct input adaptation. This group of methods directly incorporate pro-cess measurements in the optimization framework updating the inputs in a control inspired mode.
With Extremum-Seeking Control (ESC), dither signals are inserted to the input variables such that an estimate of the plant cost gradient is acquired on-line using output measurements [20]. The NCO Tracking estimates the plant NCO with output measurements [21]. Moreover, in Neighboring-Extremal Control (NEC) the plant NCO are enforced with a combination of a variational analysis of the model with output measurements [22]. Self Optimizing Control (SOC) uses the sensitivity between the uncertain model parameters and the measured outputs to generate linear combination of outputs and inputs around setpoint values that are nearly insensitive to uncertainty [23].
The selection of a specific RTO method depends on the situation. It is desirable for RTO approaches to have certain properties: guaranteed plant optimality upon convergence, fast convergence and feasible-side convergence. MA satisfies the first requirement since the model-adequacy conditions for MA are easier to satisfy than those of the two-step method [15]. It is the constituent capability of MA to converge to the plant optimum despite structural plant-model mismatch. That makes it a very interesting tool for optimizing the operation of chemical processes in presence of inaccurate models.
Fast convergence and feasible side convergence are also critical requirements, which nevertheless are highly case dependent. These last two requirements oppose each other, because fast convergence requires large steps, while feasible-side convergence frequently needs conservative ones.
Problem Formulation. The objective of RTO is the minimization of some steady-state operating cost function, subject to the satisfaction of a set of constraints. The optimal steady-state problem for the actual process can be formulated as follows:
¯
u?p,h = arg min
u
subject to:
Cp,i(u) = cp,i(u, yp(u)) ≤ 0, i= 1, . . . , nc (2.48b)
uL≤ u ≤ uU (2.48c)
where Φ : Rnu × Rny → R is the cost function to be minimized; and cp,i : Rnu× Rny → R, i = 1, . . . , nc, is the set of inequality constraints on the
input and output variables and and uL and uU are the lower and upper bounds
on the decisions variables.
Φ and ci are not known functions of u and yp; in any practical
applica-tion, the steady-state input-output plant map yp(u) is typically unknown. An
approximate non-linear steady-state model is assumed available:
Fh(u, x, θ) = 0 (2.49a)
y = Hh(u, x, θ) (2.49b)
where Fh : Rnx× Rnx× Rnu ∈ Rnx is a set of process model equations including
mass and energy balances and thermodynamic relationships and θ ∈ Rnθ is a set of model parameters. We assume that for given u, the solution to problem (2.49a) is assumed to be be written as
x= ζ(u, θ) (2.50)
where ζ is an operator expressing the steady-state mapping between u, ξ and x. The input-output map predicted by the model can be expressed as:
y(u, θ) = Hh(u, ζ(u, θ), θ) (2.51)
The model-based optimization problem becomes: ¯
uh = arg min
u Φ(u, θ) = φ(u, y(u, θ)) (2.52a)
subject to:
Ci(u, θ) = ci(u, y(u, θ)) ≤ 0, i= 1, . . . , nc (2.52b)
uL ≤ u ≤ uU (2.52c)
Assuming that the feasible set U = u ∈ uL, uU : C
i(u, θ) ≤ 0} is
non-empty and the cost function Φ is continuous for a given θ, a minimizing solution of problem (2.52) is guaranteed to exist. The set of active constraints at ¯u is denoted by:
2.2. Real Time Optimization 23 Unfortunately, in the presence of plant-model mismatch, the model solution ¯u does not generally converge to the plant optimum ¯u?
p.
Necessary Conditions of Optimality. The local minimum of problem (2.52) can be studied via the Necessary Conditions (NCO) of Optimality
condi-tions.
The Linear Independence Constraint Qualification (LICQ) requires that the gradients of the active constraints, ∂Ci
∂u for i ∈ A(u), be linearly independent.
let us assume that (LICQ) is fulfilled and the solution point ¯u and the functions Φ and C are differentiable at ¯u. Furthermore, there exist unique Lagrange multiplier vectors µ ∈ Rnc, ξU, ξL ∈ Rnu such that the following first-order Karush-Kuhn-Tucker (KKT) conditions hold at ¯u :
∂L ∂u = ∂Φ ∂u + µ 0∂C ∂u + (ξ U )0− (ξL)0 = 0 (2.54a) C ≤0 (2.54b) uL ≤ u ≤ uU (2.54c) µ0C = 0 (2.54d) (ξU)0 u − uU = 0 (2.54e) (ξL)0 uL− u = 0 (2.54f) µ ≥0 (2.54g) ξU≥ 0 (2.54h) ξL ≥ 0 (2.54i) Where L = Φ + µTC+ (ξU )0(u − uU) + (ξL )0 uL− u is the Lagrangian function. The NCOs (2.54b) and (2.54c) are called primal feasibility conditions, (2.54d), (2.54e) and (2.54f) are called complementary slackness conditions, and those given in (2.54a), (2.54g), (2.54h) and (2.54i) are defined as dual feasibility conditions [19].
These conditions are very important for the analysis and formulation of a model-based optimization problem. As a matter of fact, the matching of NCO, between the plant and the model, guarantees convergence to the plant optimum ¯
u? p.
2.2.1
Modifier Adaptation
If modeling errors are presented, the constraint values, the cost and constraint gradient predicted by the model have a mismatch with those of the plant. MA techniques exploits correction terms for the cost and constraint functions, so that
the modified model-based optimization problem matches, upon convergences, the plant NCO. Contrariwise to two-step RTO schemes, the model parameters θ are not updated. The modifiers are updated based on measurements collected at the successive RTO iterates [24]. Correction terms of first-order are added to the cost and constraint functions of the optimization problem.
At the k − th iteration with the input ¯uk,h, the modified cost and constraint
functions are constructed as follows:
Φm,k(u, θ) = Φ(u, θ) + (λΦk) 0 u (2.55a) Cm,k(u, θ) = C(u, θ) + εCk + (λCk) 0 (u − ¯uk,h) (2.55b)
where the modifiers λΦ
k ∈ Rnu, εCk ∈ Rnc and λCk ∈ Rnu
×nc are given by:
(λΦk) 0 = ∂Φp ∂u (¯uk,h) − ∂Φ ∂u(¯uk,h, θ) (2.55c) εCk = Cp(¯uk,h) − C(¯uk,h, θ) (2.55d) (λC k) 0 = ∂Cp ∂u (¯uk,h) − ∂C ∂u(¯uk,h, θ) (2.55e) The zeroth-order modifiers εCi
k correspond to bias terms, that represent the
differences between the plant and the predicted constraint at ¯uk,h. Besides, the
first-order modifiers λΦ
k and λ Ci
k represent the differences between the plant
gradients and the gradients predicted by the model at ¯uk,h.
It should be noted that that cost modification is constituted only by linear terms in u, since the additional constant term Φp(¯uk,h) − Φ(u, θ) − (λΦ)0u¯k,h
have no effect to the solution point.
When the cost and/or constraints are perfectly known functions of the inputs u, the corresponding modifiers are equal to zero, and no model correction is needed [15]. Therefore, the upper and lower bounds on the input variables constraints do not require modification because that are perfectly known.
The implementation of the modifiers requires the cost and constraint gradi-ents of the plant ∂Φp/∂u(¯uk,h) and ∂Cp/∂uk available at ¯uk,h. These gradients
can be obtained from the measured plant outputs yp(¯uk,h) and the estimated
output gradients ∂yp/∂u(¯uk,h) :
∂Φp ∂u (u) = ∂φ ∂u(¯uk,h, yp(¯uk,h)) + ∂φ ∂y (¯uk,h, yp(¯uk,h)) ∂yp
∂u(u) (2.56a)
∂Cp ∂u (u) = ∂c ∂u(¯uk,h, yp(¯uk,h)) + ∂c ∂y(¯uk,h, yp(¯uk,h)) ∂yp ∂u(¯uk,h) (2.56b) At the k-th RTO iteration, the next optimal inputs ¯uk+1 are computed by
2.2. Real Time Optimization 25 solving the following modified optimization problem:
¯ uok+1,h = arg min u Φm,k(u, θ) = Φ(u, θ) + λΦk 0 u (2.57a) subject to: Cm,k(u) = C(u) + εCk + λ C k 0 (u − ¯uk,h) ≤ 0 (2.57b) uL≤ u ≤ uU (2.57c)
A first-order filters are applied to either the modifiers or the inputs [19]: εCk = (Inc − K ε)εC k−1+ K ε(C p(¯uk,h) − C(¯uk,h)) (2.58) λΦk = (Inu− K Φ)λΦ k−1+ K Φh∂Φp ∂u (¯uk,h) − ∂Φ ∂u(¯uk,h) i0 (2.59) λCi k = (Inu− K Ci)λCi k−1+ K Ci h∂Cp,i ∂u (¯uk,h) − ∂Ci ∂u(¯uk,h) i0 , i= 1, ..., nc (2.60)
where the filter matrices Kε, KΦ, and KGi are typically selected as diagonal matrices with eigenvalues in the interval (0, 1].
Moreover one can filters the optimal RTO inputs ¯uk+1,h with: Kmod =
diag (k1, . . . , knu) , ki ∈ (0, 1] : ¯
uk+1,h = ¯uk,h+ Kmod u¯ok+1,h− ¯uk
(2.61) We can observe that optimal inputs can also be applied directly to the plant:
¯
uk+1,h= ¯uok+1,h (2.62)
However, this strategy may result in excessive correction and be very sensitive to process noise. These phenomena can affect the convergence of the algorithm.
KKT Matching. Finally, as mentioned above, the important feature MA is its ability to reach a KKT point of the plant upon convergence [19]. In the following theorem, this characteristic is made explicit.
Theorem 2.11 ( Convergence ⇒ KKT matching). Consider MA with filters on either the modifiers or the inputs. Let u∞ = limk→∞u¯k,h be a fixed point
of the iterative scheme and a KKT point of the modified optimization problem (2.57). Then, u∞ is also a KKT point of the plant Problem (2.48).
2.2.2
Output modifier adaptation
Output modifier adaptation (MAy) is an alternative modifier scheme that can be formulated, where the first-order correction terms are applied to the output functions, rather than modifying the cost and constraint [19]. At the k − th iteration, with the inputs ¯uk,h applied to the plant, the modified output
functions are defined in the following ways:
ym,k(u) = y(u) + ε y k+ (λ y k) 0 (u − ¯uk,h) (2.63) where εy k ∈ Rny and λ y k ∈ Rnu
×ny are the zero-th and first-order output modifiers defined as follows:
εyk= yp(¯uk,h) − y (¯uk,h) (2.64a)
(λyk)0 = ∂yp
∂u (¯uk,h) − ∂y
∂u(¯uk,h) (2.64b)
With the modification (2.63) the modified cost and constraint function at the k − th iteration become:
ΦMAy,k(u) = φ (u, ym,k(u)) (2.65a)
CMAy,k(u) = c (u, ym,k(u)) (2.65b)
At the k − th iteration the operating point is updated by solving of the following optimization problem:
¯
uok+1,h= arg min
u ΦMAy,k(u) = φ (u, ym,k(u)) (2.66a)
subject to:
CMAy,k(u) ≤ 0 (2.66b)
uL≤ u ≤ uU (2.66c)
The solution of (2.66) is updated by first order filter: ¯
uk+1,h = ¯uk,h+ K ¯uok+1,h− ¯uk,h
(2.67) where the filter matrix Kmod is typically selected as a diagonal matrix with
diagonal elements in the interval (0, 1].
MAy has also the ability to match the KKT conditions of the plant upon convergence [16]. Besides, it can be demonstrated that first-order matching of ym,k(u) implies first − order matching of ΦM Ay,k and GM Ay,k. In the following
2.3. Offset-free MPC 27 of MA.
Theorem 2.12 (convergence ⇒ KKT matching). Consider MAy with filters on either the modifiers or the inputs. Let u∞ = limk→∞u¯k be a fixed point of the
iterative scheme and a KKT point of the modified optimization problem (2.66). Then, u∞ is also a KKT point of the plant Problem (2.48). If also the cost
function φ(u, y) and all the constraint functions ci(u, y), for i = 1, . . . , nc, are
affine in u and y, then MA and MAy are equivalent. Proof. The proof is given in [16]
As evidenced by Papasavvas et al. [16] the MAy scheme performs additional corrections to the curvature of the cost and constraint functions (Hessian) com-pared to MA. It is thus possible that the modified cost and constraint functions used in MAy provide better local approximations to the cost and constraint functions of the plant, than the approximations used in MA. Furthermore, Papasavvas et al. [16] show that MAy often performs better than MA in terms of convergence, but it is not possible to predict in advance which method will perform better.
2.3
Offset-free MPC
A common goal in applications is to use the input in a feedback controller to compensate for an incoming disturbance to the system. Thus, the disturbance effect on the controlled variable is reduced. This problem is known as disturbance rejection. Using feedback it is possible to mitigate the effect of disturbances and to cope with modeling errors.
In conventional linear Proportional-Integral-Derivative (PID) control design, the offset-free objective is achieved by integrating the tracking error. In the first industrial implementations of MPC, such as Dynamic Matrix Control [25], an output correction term, defined as the difference between the measured and the predicted output, was implemented. This method is probably the simplest disturbance model, usually referred to as output disturbance model, and it is applicable only to open-loop stable systems [26]. A methodology to compensate an incoming disturbance is to (i) model the disturbance, (ii) use the measurements to estimate the disturbance, and (iii) find the inputs that minimize the effect of the disturbance on the controlled variables. General formulations based on disturbance models and observers have been extensively discussed in [27,28].
The scope of offset-free MPC is to design an output feedback MPC law u = κ(y) where the input and output constraints are always satisfied, the
closed-loop system reaches an equilibrium and the following condition holds true:
lim
k→∞yk = yk
in which yk is the controlled output and yk is the required set-point.
2.3.1
Augmented models and estimation
Problem definition: Actual plant dynamicsConsider the control problem of the following discrete time invariant dynam-ical system (2.1), for completeness here reported:
x+p = fp(xp, u)
yp = hp(x)
Where fp : Rnxp × Rnu → Rnxp, hp : Rnxp → Rny, xp is the current state, x+p is
the successor state, u is the current input, and yp is the current output.
Let Gp(¯u) be the steady-state input-output map of plant. In other words,
Gp : Rnu → Rny is the plant steady-state input-output map
¯
u 7→ yp = Gp(¯u), (2.68)
i.e. the solution of:
¯
xp = fp(¯xp,u) ,¯ y¯p = hp(¯xp) (2.69)
and
DuGp : Rnu → Rny×nu (2.70)
is its Jacobian.
State and Disturbance Observer
The nominal model is defined in (2.2) and is here reported for completeness:
x+ = f (x, u) y= h(x)
Where f : Rnx × Rnu → Rnx, h : Rnx → Rny, x is the current state, x+ is the successor state, u is the current input, and y is the current output.
Let Gna(¯u)be the steady-state input-output map of the nominal model. In
other words, Gna : Rnu → Rny is the nominal model steady-state input-output
map
¯
2.3. Offset-free MPC 29 i.e. the solution of:
¯
x= f (¯x,u),¯ y¯= h(¯x) (2.72) and
DuGna : Rnu → Rny×nu (2.73)
is its Jacobian with respect to the first argument. That Jacobian can be calculated analytically through the following equation:
DuGna(¯u) = Dxh(¯x)(I − Dxf(¯x,u))¯ −1
Duf(¯x,u)¯
(2.74) An augmented system is formulated in order to obtain zero prediction error despite plant model mismatch at steady state. The nominal model is augmented as follows:
x+ = F (x, u, d) = f (x, u) + Bdd (2.75a)
d+ = d (2.75b)
y = H(x, d) = h(x) + Cdd (2.75c)
the matrices Bd ∈ Rnx×ny, Cd ∈ Rny×ny are used to model the effect of the
disturbance. Let us note that the disturbance is model with an integrator to remote steady offset. Moreover, the augmented model (2.75) is consistent with the nominal model (2.2), i.e. for all x ∈ Rnx and u ∈ Rnu there holds:
F(x, u, 0) = f (x, u), H(x, 0) = h(x) (2.76) Let G(¯u, ˆd)be the steady-state input-output map of the augmented model. In other words, G : Rnu × Rny → Rny is the augmented model steady-state input-output map
¯
u 7→ y = G(¯u, ˆd), (2.77)
i.e. the solution of:
¯
x= F (¯x,u, ˆ¯ d), y¯= H(¯x, ˆd) (2.78) and
DuG: Rnu× Rny → Rny×nu (2.79)
is its Jacobian with respect to the first argument. That Jacobian can be calculated analytically through the following equation:
DuG(¯u, ˆd) = DxH(¯x, ˆd) I −DxF(¯x,u, ˆ¯ d) −1 DuF(¯x,u, ˆ¯ d) (2.80a) = Dxh(¯x)(I − Dxf(¯x,u))¯ −1 Duf(¯x,u)¯ (2.80b)
Note that (2.80a) is equal to (2.74) only if a linear disturbance model is used, as in (2.75).
A necessary condition for which the augmented model (2.75) is observable is that the nominal model (2.2) is observable. Moreover, given an observable model (2.2), it is possible to define an augmented model (2.75) with nd ≤ ny
disturbance states, that is observable [29].
At each time k, given the output measurement yp,k, an observer for (2.75) is
used to estimate the augmented state (xk, dk).A "steady-state Kalman filter-like
estimator" for the augmented model can be formulated. Let us define the symbols ˆx∗
k and ˆd ∗
k to denote the filtered estimate of xk
and dk obtained using the output measurement at time k. Besides, ˆxk, ˆdk and
ˆ
yk,are the predicted estimate of xk, dk and yk, respectively, obtained at time
k −1 using the augmented model (2.75). ˆ xk = F (ˆx∗k−1, uk−1, ˆd∗k−1) (2.81a) ˆ dk = ˆd∗k−1 (2.81b) ˆ yk = H(ˆxk,xˆk) (2.81c)
in (2.81c), denoted by yk, we define the prediction error as:
ek = yp,k− ˆyk (2.82)
Given the state and disturbance predictions obtained from the augmented model in (2.75),denoted by (ˆxk, ˆdk), we compute the corresponding estimates as:
ˆ
x∗k = ˆxk+ Kxek (2.83a)
ˆ
d∗k= ˆdk+ Kdek (2.83b)
in which the matrices Kx ∈ Rnx×ny and Kd∈ R ny×ny
d are chosen to form a
(nominally) asymptotically stable observer, which requires nd= ny [29]. This
implies, in particular, that Kd is invertible. The Kx and Kd can be evaluated,
for example, by applying the EKF theory or using a deadbeat observer to the augmented system (2.75). The output deadbeat Kalman filter is a special case of Kalman observer, where the chosen of weight covariance matrices in Steady state Kalman Filter are:
Qkf = " Qkf,x Qkf,d # = " 0nx Ind # Rkf → 0ny (2.84)
where Qkf ∈ R(nx+nd)×(nx+nd) is the process noise covariance matrix and
2.3. Offset-free MPC 31 gain in output disturbance model becomes equal to:
Kx = 0nx×ny, Kd= Iny, Bd = 0nx×ny, Cd= Iny (2.85)
2.3.2
Offset-free tracking MPC
Target calculation. Given the current augmented state estimate (ˆx∗k, ˆd∗k), an offset-free MPC algorithm needs to compute the equilibrium target that establish exact tracking of the controlled variable. In the general case the target problem to be solved is the following:
min
x,u,y`s(y, u) (2.86a)
subject to:
x= F (x, u, ˆd∗k) (2.86b)
y= H(x, ˆd∗k) (2.86c)
y ∈ Y, u ∈ U (2.86d)
where `s: Rny× Rnu :→ R is the steady-state cost function. Problem (2.86) is
assumed feasible and its unique solution is denoted as (¯xk,u¯k,y¯k). The above
problem are solved at each decision time, and not only when the setpoint changes. Typical tracking cost function can be constructed as in (2.31), with the setpoint that comes from the RTO.
Optimal Control Problem (OCP). For i = 0, . . . , N −1, let yi = H xi, ˆd∗k
be the model output corresponding to a state xi and disturbance estimate ˆd∗k.
Moreover, the following finite-horizon optimal control problem is solved every decision time k, using the notation of the nominal OCP Problem (2.32):
min
x,u VN(x, u) = minx,u N −1 X i=0 `c(˜xi,u˜i) + Vf(˜xN) (2.87a) subject to: x0 = ˆx∗k (2.87b) xi+1 = F xi, ui, ˆd∗k i= 0, . . . , N − 1 (2.87c) H xi, ˆd∗k ∈ Y, ui ∈ U i= 0, . . . , N − 1 (2.87d) ˜ xN ∈ Xf (2.87e)
