State Space Realization of Model Predictive Controllers without Active Constraints

To enable the use of traditional tools for analysis of multivariable controllers such as model predictive control (MPC), we develop a state space formulation for the resulting controller for MPC without constraints or assuming that the constraints are not active. Such a derivation was not found in the literature. The formulation includes a state estimator. The MPC algorithm used is a receding horizon controller with infinite horizon based on a state space process model. When no constraints are active, we obtain a state feedback controller, which is modified to achieve either output tracking, or a combination of input and output tracking. When the states are not available, they need to be estimated from the measurements. It is often recommended to achieve integral action in a MPC by estimating input disturbances and include their effect in the model. We show that to obtain offset free steady state the number of estimated disturbances must equal the number of measurements. The estimator is included in the controller equation, and we obtain a formulation of the overall controller with the set-points and measurements as inputs, and the manipulated variables as outputs. One application of the state space formulation is in combination with the process model to obtain a closed loop model. This can for example be used to check the steady-state solution and see whether integral action is obtained or not. 1. Introduction In this paper, we develop a state-space formulation for a MPC without constraints or assuming that the constraints are not active. This state-space formulation of the controller enables the use of traditional tools to get insight into how the controller behaves. (independently) uses a linear formulation for a MPC controller to analyse its controller tuning for a paper machine headbox. He combines the linear controller formulation with the process model, and calculates the singular values of the sensitivity function and the complementary sensitivity function. The main idea behind MPC is that a model of the process is used to predict the response of future moves of the control inputs (the inputs that the controller can manipulate to control the process). This prediction is used to find an optimal sequence of the control inputs. Optimal means that a certain criterion containing an output vector and the vector of the control inputs is minimized. In most MPC implementations the control inputs are assumed to be held constant within a given number of time …


Introduction
In this paper, we develop a state-space formulation for a MPC without constraints or assuming that the constraints are not active.This state-space formulation of the controller enables the use of traditional tools to get insight into how the controller behaves.This is illustrated in Faanes &Skogestad, 2003a, b (or Faanes, 2003, Chapt. 4 and7).Maciejowski (2002) (independently) uses a linear formulation for a MPC controller to analyse its controller tuning for a paper machine headbox.He combines the linear controller formulation with the process model, and calculates the singular values of the sensitivity function and the complementary sensitivity function.
The main idea behind MPC is that a model of the process is used to predict the response of future moves of the control inputs (the inputs that the controller can manipulate to control the process).This prediction is used to find an optimal sequence of the control inputs.Optimal means that a certain criterion containing an output vector and the vector of the control inputs is minimized.
In most MPC implementations the control inputs are assumed to be held constant within a given number of time intervals.At a given time, the first value in the sequence of control inputs is implemented in the process.The prediction and the optimal sequence depend on the current state of the process.At the next time step, the state being reached is therefore used in the calculation of a new optimal control input sequence.This sequence will not necessarily be what was computed at the previous time step, due to the effects of model errors and unmodelled disturbances.So, at each time step we only implement the first step in the control input sequence, and discard the rest.
Normally, we include constraints in the optimization problem.These are constraints that naturally occur in a process, like the range of control valves and pump speeds (on control inputs), and safety-related constraints on the outputs.One may also restrict the rate of change of the control inputs.
For a review of industrial MPCs we refer to Qin & Badgwell, 1996;Badgwell & Qin, 2002.In this paper, we consider the MPC formulation proposed by Muske & Rawlings (1993).This MPC is based on a state-space model.Our assumption is that no constraints are active, and this also covers the case when the same constraints are active all the time and the degree of freedom is reduced.Bemporad et al. (2002) (first appeared in Bemporad et al., 1999) have shown that the controller also for the case with dynamic constraints is piecewise linear.
Since the models are not perfect, and there always are unmodelled disturbances, the MPC needs some correction from measurements.The most common approach is to estimate some output bias in the measurements, and correct for this bias.However, for integrating processes or processes with long time constants, this method has proved unsatisfactory (Muske & Rawlings, 1993;Lee et al., 1994;Lundstro ¨m et al., 1995).We therefore estimate input disturbances, which is straight forward when a state-space model is applied in the MPC.
As known, MPC without constraints is a special case of optimal control, and in Sections 2, 3 and 4 we will demonstrate how the control input can be expressed by the current state and the previous control input.The first of these sections, Section 2, covers the simple case when the reference for the output vector is zero, whereas Section 3 handles non-zero references.When the number of control inputs exceeds the number of outputs, the extra degree of freedom may also be used to give references to the control inputs (Section 4).Since the full state vector normally is not measured, we include a state estimator, which also estimates input disturbances, in Section 5.The total controller formulation, i.e. the control inputs as functions of the reference and the measurements, is given in Section 6.In Section 7 we find the number of estimated disturbances needed to obtain effective integral action.We develop the closed loop model of the system in Section 8.The main conclusions are summarized in Section 9.

Derivation of equivalent controller from receding horizon controller without active constraints
Muske & Rawlings (1993) present a model predictive control algorithm based on the following state-space model: Here ]T is a vector of N future moves of the control input, of which only the first is actually implemented.The control input, u k>j , is assumed zero for all jPN.In the criterion it is assumed that the reference for y is zero.We assume that the process is stable, and Muske & Rawlings (1993) show how this formulation can be transformed into the following finite optimization problem: where H, G and F are time independent matrices expressed by the model matrices, A, B and C, and the weight matrices, Q, R and S. Since d k is unknown in the future, the term E d d k from equation ( 1) is omitted in the derivation of equation ( 4).For normal use of this MPC algorithm, the control input is found by optimizing equation (4) subject to given constraints on the outputs, the control inputs and changes in the control inputs.Assuming no active constraints, however, the optimum of equation ( 4) can be found by setting the gradient equal to zero (Halvorsen, 1998): which implies Only the first vector u k from uN k is applied: where K and K u consist of the first r rows in ñH\1G and H\1F, respectively, and r is the number of control inputs.
Since H, G and F are constant, also K and K u are constant matrices.The first term can therefore be recognized as state feedback.The second term comes from the weight on the change in control input from the original criterion.The matrix F only contains S and zeros, so when no weight is put on the change in the control input, S is zero, and K u ó0.

The steady-state solution
Here we consider tracking of outputs.If the output reference vector, y r , is nonzero, equation ( 7) must be shifted to the steady-state values for the states and the control inputs: u s and x s can be found from the steady-state solver: where y r and u r are the references for the output and the control input, respectively.Again, we assume that the limitations are never active, and that we have no extra freedom for the control inputs (number of control inputs equals number of outputs), in which case the problem reduces to solve equation ( 11).
Assuming square systems (i.e., equal number of control inputs and references), no poles in the origin (which makes (IñA) invertible) and that C(IñA)\1B is invertible (it is at least quadratic from the first assumption), we get the following solution: where Since we have no knowledge of future disturbances, we assume that it will keep its current value, that is d s ód k .We note that y s óCx s óy r as desired, and that if we assume that the disturbance enters via the control inputs, i.e.E d óB, the expression for !d simplifies to i.e., u s óñd s and x s ó0.Now equation ( 9) can be expressed with y r and d k : where K and K u are defined in Section 2 and

Generalization with tracking of inputs
In this section, we generalize the steady-state solution to include tracking of both inputs and outputs.The total number of references that it is possible to track is limited by the number of (independent) control inputs.
We collect the inputs that we want to give a reference into the vector u 1 , and likewise the outputs we want to give a reference into y 1 . The rest of the inputs and outputs are assembled into u 2 and y 2 , respectively.The model may now be formulated as where where

State and disturbance estimator
To calculate u k from equations ( 16) or (28) one must know the state, x k , and if it is not measured, it must be estimated from the measurements.The same applies also to the disturbance vector d k .If we assume that neither the states nor the disturbances are measured, we extend the state variable with the disturbance vector As a basis for a state estimator the following model based on equations ( 1) and ( 2) is introduced: where w k and v k are zero-mean, uncorrelated, normally distributed white stochastic noise with covariance matrices of Q w and R v respectively, and ym k is the measured output vector, not necessarily the same as the output vector that shall track a reference, and Cm is the corresponding matrix in the estimator model, mapping from the states to the measured output vector.We have modelled the disturbance as constant except for the noise.
The augmented state estimator is then formulated as x x ¯k>1 óA ˜x x ˆkòB ˜uk (32) where L is the estimator gain matrix, for example the Kalman filter gain.x x ¯k>1 is called the a priori estimate (since it is prior to the measurement), and x x ˆk the a posteriori estimate (after the measurement is available).For a Kalman filter, L is given by the solution of a Ricatti equation: We want to express the estimator in a single expression, and this can be done in two ways, depending on which of the two estimates one prefers to use.
Alternative 1: A posteriori estimate, x x ˆk: Alternative 2: A priori estimate, x x ¯k>1 : Remark 1. Muske & Rawlings (1993) refer to A ˚stro ¨m (1970) who used a priori estimate (Alternative 2), (noting that their L corresponds to our A ˜L).However, according to Rawlings (1999) they actually used Alternative 1 (a posteriori) in their work.Normally, that is if the computational time is not too long, the control input is implemented directly after a new measurement has been sampled, in which case the a posteriori estimate is preferred since it utilizes this new measurement.Thus, in this paper we will use Alternative 1, the a posteriori estimate.

State-space representation of the overall controller
In this section, we will form the overall controller, containing the state feedback, the stead-state solution and the estimator on state-space form.
With the extended state vector x ˜k from equation ( 29) and the controller equations equations ( 16) and ( 28) can both be expressed by For equation ( 16) (without input resetting) róy r and K r óK y .Since x ˜k generally is not available, we use the estimate x x ˆk.Combination of the controller equation ( 39) with the estimator difference equation ( 36 where A ¯ó(IñLC ˜)(A ˜òB ˜K ˜).This is not an ordinary discrete state-space formulation.First, y k>1 and x x ˆk do not have the same index on the right side of equation (40).To overcome this, we introduce the artificial state variable z k óx x ˆkñLym k : Next, the term u k\1 is a problem.We first assume that in the optimization criterion equation (3) Só0.Then K u ó0, and we get an ordinary discrete state-space system with z k as the states, ym k as the input and u k as the output.The reference, r, can be seen as a 'disturbance' to the controller.We may express the controller as and F K óK r .For SÖ0 we have not yet obtained the controller on ordinary state-space form.We first express the controller as where in addition to the definitions above, G K ó(IñLC ˜)B ˜Ku and H K óK u .We repeat u k shifted one time step, insert for z k>1 in the expression for u k>1 and re-arrange: We now introduce the state vector and obtain Again, we have ym k>1 in the expression for z ˆk>1 , and introduce which yields the following expression for the total controller: In summary, we have shown that with no active constraints, the MPC controller with augmented state estimator can be expressed on discrete state-space form.
If we instead use the a priori estimate (Alternative 2), we get a different controller with other poles.

On the number of estimated disturbances
In this section, we will discuss the number of estimated disturbances (the dimension of d ˆk) necessary to avoid stead-state offset.According to Muske & Rawlings (1993), the number of elements in d ˆs cannot exceed the number of measurements if observability of the estimator shall be achieved.But what is the smallest number required?
We first have to specify clearer what 'no steady-state offset' means.If the process is perturbed by measurement noise and disturbances that change their value from time step to time step, the control will never be offset free, and no steady state will be obtained.Thus, we will consider the response when the noise, the model error and the disturbances are constant.(Alternatively, one may model noise, model error and disturbances as stochastic processes and consider a large number of experiments.) Using as before the a posteriori estimate, the estimate of the measurement is In order to obtain an offset free steady state, the estimator must provide a correct state estimate for the MPC despite model errors, constant measurement errors or noise and a constant input disturbance at steady state.More precisely, the prediction of the measured output must equal the actual one: We let index s to denote steady state.We want to see what this condition means for our MPC and estimator, and first we extract the expression for the estimate of x k>1 from the estimator equation ( 36): and obtain for the a posteriori state estimate Thus equations ( 54) and (55) yields which leads to the following matrix equation In equation ( 66) the number of scalar equations equals the number of measurements (the number of rows in Cm).The only free variables are the elements of d ˆs.To obtain an offset free steady-state solution of the control problem there must exist a solution of equation ( 66), which implies that the number of elements in d ˆs must be equal or greater than the number of measurements (independent of the size of the reference, r, and the number of control inputs, u).Thus, since the number of estimated disturbances cannot exceed the number of measurements (see above), we may conclude that: If offset free steady state shall be obtained, the number of estimated disturbances must be equal to the number of measurements.This was, independent, also found by Muske & Badgwell (2002), except that they do not distinguish between outputs to be controlled by the MPC and the measurements, and Pannocchia &Rawlings (2003). A ˚kesson andHagander (2003) propose to combine input disturbance and output bias estimates, and they also find that the total number must equal the number of measurements.In Faanes & Skogestad (2003b) or Faanes (2003, Chapt.7) an experimental illustration is given.
Remark 2. In the general case equation ( 66) cannot be used to determine d ˆs given r and ym s .It will often be many d ˆs that fulfills equation ( 66), and the value of d ˆs will depend on the disturbance, measurement or model error that is present.
Example 1.In the neutralization example in Faanes & Skogestad (2003a) or Faanes (2003, Chapt. 4) we use three measurements, and thus estimation of three disturbances is required.For the 'original' MPC we only estimate two input disturbances, and the result is insufficient integral action, as expected.The modified MPC with three disturbance estimates gets full integral action.

Closed loop model
The combination of the process model with the controller yields the closed loop model of the system.The process is expressed by the discrete model equation ( 1) and ( 2), which we repeat for the actual process, marked with a prime: The vector of measurements, ym, is expressed by where C@m is the matrix mapping from the states to the measured output vector, and m k is the measurement error.The controller is expressed by equations ( 44) or ( 53).u k and ym are then eliminated from the equations by combining the controller equation with equations ( 67), ( 68) and ( 69).We then get the following closed loop model (where we have omitted the tilde in the controller matrices from equation ( 53)): We combine the process states, where "ó E@ d 0 (77) and there are no significant steady-state errors.The small offsets are due to numerical inaccuracies.

Conclusions
In this paper, we have developed a state-space formulation for a MPC (for stable processes) without constraints or assuming that the constraints are not active.This state-space formulation of the controller makes it possible to use traditional tools to get insight into how the controller behave (see Faanes &Skogestad, 2003a,b or Faanes, 2003 (Chapt. 4 and7)).The controller can be extended with tracking of inputs, and also include the state estimator necessary if not all the states are measured.To obtain offset-free tracking, estimates of the input disturbances are included in the estimator and in the calculation of the steady state.We show that the length of this estimated disturbance vector must equal the number of measurements available to the estimator.
Finally, a closed loop state-space formulation is derived, assuming a state-space formulation of the process model.