### MATLAB Optimization Toolbox

### Presented by Chin Pei

### February 28, 2003

### Presentation Outline

### Introduction

### Function Optimization

### Optimization Toolbox

### Routines / Algorithms available

### Minimization Problems

### Unconstrained

### Constrained

### Example

### The Algorithm Description

### Multiobjective Optimization

### Function Optimization

### Optimization concerns the minimization or maximization of functions

### Standard Optimization Problem

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

~

###

~### min

*x*

*f x*

### ^{0}

*g x*

*j*

###

###

~### 0

*h x*

*i*

### Equality Constraints Subject to:

### Inequality Constraints

### Function Optimization

###

~*f x* ^{is the } objective function , which measure and evaluate the performance of a system.

### In a standard problem, we are minimizing the function.

### For maximization, it is equivalent to minimization of the –ve of the objective function.

*x*

~ ### is a column vector of design variables , which can

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

### Function Optimization

### Constraints – Limitation to the design space.

### Can be linear or nonlinear, explicit or implicit functions

###

~### 0 *g x*

*j*

###

###

~### 0

*h x*

*i*

### Equality Constraints Inequality Constraints

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

### Most algorithm require less than!!!

### Optimization Toolbox

### Is a collection of functions that extend the capability of MATLAB. The toolbox includes routines for:

### Unconstrained optimization

### Constrained nonlinear optimization, including goal attainment problems, minimax problems, and semi- infinite minimization problems

### Quadratic and linear programming

### Nonlinear least squares and curve fitting

### Nonlinear systems of equations solving

### Constrained linear least squares

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Minimization Algorithm

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Minimization Algorithm (Cont.)

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Equation Solving Algorithms

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Least-Squares Algorithms

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Implementing Opt. Toolbox

### Most of these optimization routines require the definition of an M-file containing the

### function, f , to be minimized.

### Maximization is achieved by supplying the routines with – f .

### Optimization options passed to the routines change optimization parameters.

### Default optimization parameters can be changed through an options structure.

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Unconstrained Minimization

### Consider the problem of finding a set of *values [x* _{1} * x* _{2} ] ^{T} that solves

_{1}

_{2}

^{T}

### Introduction

### Introduction Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

##

^{1}

^{} ^{}

~

### 2 2

### 1 2 1 2 2

### min ~ ^{x} 4 2 4 2 1

^{x}

*x* *f x* *e* *x* *x* *x x* *x*

### 1 2

### ~

*x* *x* *x* *T*

### Steps

### Create an M-file that returns the function value (Objective Function)

**Call it objfun.m**

### Then, invoke the unconstrained minimization routine

**Use fminunc**

### Introduction

### Introduction Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

### Step 1 – Obj. Function

### Introduction

### Introduction Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

**function f = objfun(x)**

**f=exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1);**

### 1 2

### ~

*x* *x* *x* *T*

### Objective function

### Step 2 – Invoke Routine

### Introduction

### Multiobjective Optimization Conclusion Conclusion

**x0 = [-1,1]; **

**options = optimset(‘LargeScale’,’off’);**

**[xmin,feval,exitflag,output]=**

**fminunc(‘objfun’,x0,options);**

### Output arguments

### Input arguments Starting with a guess

### Optimization parameters settings

### Results

### Introduction

### Multiobjective Optimization Conclusion Conclusion

**xmin =**

** 0.5000 -1.0000**
**feval =**

** 1.3028e-010**
**exitflag =**
** 1**

**output = **

** iterations: 7**
** funcCount: 40**
** stepsize: 1**

### Minimum point of design variables

### Objective function value

### Exitflag tells if the algorithm is converged.

### If exitflag > 0, then local minimum is found

### Some other information

**More on fminunc – Input**

**[xmin,feval,exitflag,output,grad,hessian]=**

** fminunc(fun,x0,options,P1,P2,…)**

**fun: Return a function of objective function.**

**x0: Starts with an initial guess. The guess must be a vector of size of ** number of design variables.

**option: To set some of the optimization parameters. (More after few ** slides)

**P1,P2,…: To pass additional parameters.**

### Introduction

### Multiobjective Optimization Conclusion Conclusion

**More on fminunc – Output **

### Introduction

### Multiobjective Optimization Conclusion Conclusion

**[xmin,feval,exitflag,output,grad,hessian]=**

** fminunc(fun,x0,options,P1,P2,…)**

**xmin: Vector of the minimum point (optimal point). The size is the ** number of design variables.

**feval: The objective function value of at the optimal point.**

**exitflag: A value shows whether the optimization routine is ** terminated successfully. (converged if >0)

**output: This structure gives more details about the optimization**

**grad: The gradient value at the optimal point.**

**Options Setting – optimset **

### Introduction

### Multiobjective Optimization Conclusion Conclusion

**Options = **

** optimset(‘param1’,value1, ‘param2’,value2,…)**

### The routines in Optimization Toolbox has a set of default optimization parameters.

### However, the toolbox allows you to alter some of those parameters, for example: the tolerance, the step size, the gradient or hessian values, the max. number of iterations etc.

### There are also a list of features available, for example: displaying the values at each iterations, compare the user supply gradient or

### hessian, etc.

### Options Setting (Cont.)

**Options = **

** optimset(‘param1’,value1, ‘param2’,value2,…)**

### Introduction

### Multiobjective Optimization Conclusion Conclusion

**Type help optimset in command window, a list of options setting available will be displayed.**

How to read? For example:

**LargeScale - Use large-scale algorithm if ** **possible [ {on} | off ]**

### The default is with { }

### Options Setting (Cont.)

**LargeScale - Use large-scale algorithm if ** **possible [ {on} | off ]**

### Since the default is on, if we would like to turn off, we just type:

**Options = **

** optimset(‘LargeScale’, ‘off’)** **Options = **

** optimset(‘param1’,value1, ‘param2’,value2,…)**

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Useful Option Settings

**Display - Level of display [ off | iter | ** **notify | final ]**

**MaxIter - Maximum number of iterations ** **allowed [ positive integer ]**

**TolCon - Termination tolerance on the ** **constraint violation [ positive scalar ]**

**TolFun - Termination tolerance on the ** **function value [ positive scalar ]**

**TolX - Termination tolerance on X [ positive **

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Highly recommended to use!!!

**fminunc and fminsearch**

### **fminunc** uses algorithm with gradient and hessian information .

### Two modes:

### Large-Scale: interior-reflective Newton

### Medium-Scale: quasi-Newton (BFGS)

### Not preferred in solving highly discontinuous functions .

### This function may only give local solutions .

### Introduction

### Multiobjective Optimization Conclusion Conclusion

**fminunc and fminsearch**

### **fminsearch is generally less efficient than ** **fminunc for problems of order greater than ** two. However, when the problem is highly discontinuous **, fminsearch may be more ** robust.

### This is a direct search method that does not use numerical or analytic gradients as in

**fminunc.**

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Constrained Minimization

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

**[xmin,feval,exitflag,output,lambda,grad,hessian]**

** = **

**fmincon(fun,x0,A,B,Aeq,Beq,LB,UB,NONLCON,options,** **P1,P2,…)**

### Vector of Lagrange

### Multiplier at optimal

### point

### Example

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

~

###

1 2 3

### min

~*x*

*f x* *x x x*

2

1 2

### 2 *x* *x* 0

1 2 3

1 2 3

### 2 2 0

### 2 2 72

*x* *x* *x*

*x* *x* *x*

###

###

1 2 3

### 0 *x x x* , , 30 Subject to:

### 1 2 2 0

### ,

*A* *B*

###

### 0 30

### 0 , 30

*LB* *UB*

###

###

###

**function f = myfun(x)**

**f=-x(1)*x(2)*x(3);**

### Example (Cont.)

2

1 2

### 2 *x* *x* 0

### For

**Create a function call nonlcon which returns 2 constraint vectors [C,Ceq]**

**function [C,Ceq]=nonlcon(x)** **C=2*x(1)^2+x(2);**

**Ceq=[];** Remember to return a null

### Matrix if the constraint does not apply

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Multiobjective Optimization

### Multiobjective Optimization Conclusion Conclusion

### Example (Cont.)

**x0=[10;10;10];**

**A=[-1 -2 -2;1 2 2];**

**B=[0 72]';**

**LB = [0 0 0]';**

**UB = [30 30 30]';**

**[x,feval]=fmincon(@myfun,x0,A,B,[],[],LB,UB,@nonlcon)**

### 1 2 2 0

### 1 2 2 , 72

*A* *B*

###

###

### Initial guess (3 design variables)

### CAREFUL!!!

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### 0 30

### 0 , 30

### 0 30

*LB* *UB*

###

###

###

###

###

### Example (Cont.)

**Warning: Large-scale (trust region) method does not currently solve this type of problem, switching to **
**medium-scale (line search).**

**> In D:\Programs\MATLAB6p1\toolbox\optim\fmincon.m at line 213**
** In D:\usr\CHINTANG\OptToolbox\min_con.m at line 6**

**Optimization terminated successfully:**

**Magnitude of directional derivative in search direction less than 2*options.TolFun and maximum constraint **
**violation is less than options.TolCon**

**Active Constraints:**

** 2**
** 9**
**x =**

** 0.00050378663220**
** 0.00000000000000**
** 30.00000000000000**

### Introduction

### Multiobjective Optimization Conclusion Conclusion

1 2 3

1 2 3

### 2 2 0

### 2 2 72

*x* *x* *x*

*x* *x* *x*

###

###

1 2

### 0 30

### 0 30

### 0 30

*x* *x* *x*

###

###

###

### Const. 1 Const. 2 Const. 3

### Const. 4

### Const. 5 Const. 6

### Const. 7 Const. 8

### Multiobjective Optimization

### Previous examples involved problems with a single objective function.

### Now let us look at solving problem with **multiobjective function by lsqnonlin.**

### Example is taken by designing an optimal PID controller for an plant.

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization

### Multiobjective Optimization Conclusion Conclusion

### Simulink Example

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization Conclusion Conclusion

**Goal: Optimize the control parameters in Simulink model optsim.mdl** ** in order to minimize the error between the output and input.**

### Plant description:

### • Third order under-damped with actuator limits.

### • Actuation limits are a saturation limit and a slew rate limit.

### Simulink Example (Cont.)

### Initial PID Controller Design Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization

### Multiobjective Optimization Conclusion Conclusion

### Solving Methodology

### Design variables are the gains in PID controller (K _{P} , K _{I} and K _{D} ) .

### Objective function is the error between the output and input.

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization

### Multiobjective Optimization Conclusion Conclusion

### Solving Methodology (Cont.)

### Let pid = [ Kp Ki Kd ] ^{T}

### Let also the step input is unity.

### F = yout - 1

### **Construct a function tracklsq for ** objective function.

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Objective Function

**function F = tracklsq(pid,a1,a2)** **Kp = pid(1);**

**Ki = pid(2);**

**Kd = pid(3);**

**% Compute function value**

**opt = simset('solver','ode5','SrcWorkspace','Current');**

**[tout,xout,yout] = sim('optsim',[0 100],opt);**

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization Conclusion Conclusion

### The idea is perform nonlinear least squares minimization of the errors from time 0 to 100 at the time step of 1.

### So, there are 101 objective functions to minimize.

**The lsqnonlin**

### Introduction

### Introduction Unconstrained Minimization Unconstrained Minimization Constrained Minimization Constrained Minimization Multiobjective Optimization Conclusion Conclusion

**[X,RESNORM,RESIDUAL,EXITFLAG,OUTPUT,LAMBDA,JACOBIAN]**

**= LSQNONLIN(FUN,X0,LB,UB,OPTIONS,P1,P2,..) **

### Invoking the Routine

**clear all** **Optsim;**

**pid0 = [0.63 0.0504 1.9688];**

**a1 = 3; a2 = 43;**

**options = **

**optimset('LargeScale','off','Display','iter','TolX',0.001,'TolFun** **',0.001);**

**pid = lsqnonlin(@tracklsq,pid0,[],[],options,a1,a2)** Introduction

### Multiobjective Optimization Conclusion Conclusion

### Results

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Results (Cont.)

### Initial Design

### Optimization Process Optimal Controller Result

### Introduction

### Multiobjective Optimization Conclusion Conclusion

### Conclusion

### Easy to use! But, we do not know what is happening behind the routine. Therefore, it is still important to understand the limitation of each routine.

### Basic steps:

### Recognize the class of optimization problem

### Define the design variables

### Create objective function

### Recognize the constraints

### Start an initial guess