//SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 272 ± [272±324/53] 9.8.2001 2:39PM 9 Optimal and robust control system design 9.1 Review of optimal control An optimal control system seeks to maximize the return from a system for the minimum cost. In general terms, the optimal control problem is to find a control u which causes the system  x  g(x(t), u(t), t)(9:1) to follow an optimal trajectory x(t) that minimizes the performance criterion, or cost function J   t 1 t 0 h(x(t), u(t), t)dt (9:2) The problem is one of constrained functional minimization, and has several approaches. Variational calculus, Dreyfus (1962), may be employed to obtain a set of differ- ential equations with certain boundary condition properties, known as the Euler± Lagrange equations. The maximum principle of Pontryagin (1962) can also be applied to provide the same boundary conditions by using a Hamiltonian function. An alternative procedure is the dynamic programming method of Bellman (1957) which is based on the principle of optimality and the imbedding approach. The principle of optimality yields the Hamilton±Jacobi partial differential equation, whose solution results in an optimal control policy. Euler±Lagrange and Pontrya- gin's equations are applicable to systems with non-linear, time-varying state equa- tions and non-quadratic, time varying performance criteria. The Hamilton±Jacobi equation is usually solved for the important and special case of the linear time- invariant plant with quadratic performance criterion (called the performance index), which takes the form of the matrix Riccati (1724) equation. This produces an optimal control law as a linear function of the state vector components which is always stable, providing the system is controllable. 9.1.1 Types of optimal control problems (a) The terminal control problem: This is used to bring the system as close as possible to a given terminal state within a given period of time. An example is an //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 273 ± [272±324/53] 9.8.2001 2:39PM automatic aircraft landing system, whereby the optimum control policy will focus on minimizing errors in the state vector at the point of landing. (b) The minimum-time control problem: This is used to reach the terminal state in the shortest possible time period. This usually results in a `bang±bang' control policy whereby the control is set to u max initially, switching to u min at some specific time. In the case of a car journey, this is the equivalent of the driver keeping his foot flat down on the accelerator for the entire journey, except at the terminal point, when he brakes as hard as possible. (c) The minimum energy control problem: This is used to transfer the system from an initial state to a final state with minimum expenditure of control energy. Used in satellite control. (d) The regulator control problem: With the system initially displaced from equilib- rium, will return the system to the equilibrium state in such a manner so as to minimize a given performance index. (e) The tracking control problem: This is used to cause the state of a system to track as close as possible some desired state time history in such a manner so as to minimize a given performance index. This is the generalization of the regulator control problem. 9.1.2 Selection of performance index The decision on the type of performance index to be selected depends upon the nature of the control problem. Consider the design of an autopilot for a racing yacth. Conventionally, the autopilot is designed for course-keeping, that is to minimise the error e (t) between that desired course d (t) and the actual course a (t) in the presence of disturbances (wind, waves and current). Since d (t) is fixed for most of the time, this is in essence a regulator problem. Using classical design techniques, the autopilot will be tuned to return the vessel on the desired course within the minimum transient period. With an optimal control strategy, a wider view is taken. The objective is to win the race, which means completing it in the shortest possible time. This in turn requires: (a) Minimizing the distance off-track, or cross-track error y e (t). Wandering off track will increase distance travelled and hence time taken. (b) Minimizing course or heading error e (t). It is possible of course to have zero heading error but still be off-track. (c) Minimizing rudder activity, i.e. actual rudder angle (as distinct from desired rudder angle)  a (t), and hence minimizing the expenditure of control energy. (d) Minimizing forward speed loss u e (t). As the vessel yaws as a result of correcting a track or heading error, there is an increased angle of attack of the total velocity vector, which results in increased drag and therefore increased forward speed loss. From equation (9.2) a general performance index could be written J   t 1 t 0 h( y e (t), e (t), u e (t),  a (t))dt (9:3) Optimal and robust control system design 273 //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 274 ± [272±324/53] 9.8.2001 2:39PM Quadratic performance indices If, in the racing yacht example, the following state and control variables are defined x 1  y e (t), x 2  e (t), x 3  u e (t), u   a (t) then the performance index could be expressed J   t 1 t 0 f(q 11 x 1  q 22 x 2  q 33 x 3 )  (r 1 u)gdt (9:4) or J   t 1 t 0 (Qx  Ru)dt (9:5) If the state and control variables in equations (9.4) and (9.5) are squared, then the performance index become quadratic. The advantage of a quadratic performance index is that for a linear system it has a mathematical solution that yields a linear control law of the form u(t) ÀKx(t)(9:6) A quadratic performance index for this example is therefore J   t 1 t 0 q 11 x 2 1  q 22 x 2 2  q 33 x 2 3 ÀÁ  r 1 u 2 ÀÁÈÉ dt (9:7) J   t 1 t 0 [ x 1 x 2 x 3 ] q 11 00 0 q 22 0 00q 33 P R Q S x 1 x 2 x 3 P R Q S  [u][r 1 ][u] P R Q S dt or, in general J   t 1 t 0 (x T Qx  u T Ru)dt (9:8) Q and R are the state and control weighting matrices and are always square and symmetric. J is always a scalar quantity. 9.2 The Linear Quadratic Regulator The Linear Quadratic Regulator (LQR) provides an optimal control law for a linear system with a quadratic performance index. 9.2.1 Continuous form Define a functional equation of the form f (x, t)  min u  t 1 t 0 h(x, u)dt (9:9) 274 Advanced Control Engineering //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 275 ± [272±324/53] 9.8.2001 2:39PM where over the time interval t 0 to t 1 , f (x, t 0 )  f (x(0)) f (x, t 1 )  0 From equations (9.1) and (9.2), a Hamilton±Jacobi equation may be expressed as @f @t Àmin u h(x, u)  @f @x  T g(x, u) 45 (9:10) For a linear, time invariant plant, equation (9.1) becomes  x  Ax  Bu (9:11) And if equation (9.2) is a quadratic performance index J   t 1 t 0 (x T Qx  u T Ru)dt (9:12) Substituting equations (9.11) and (9.12) into (9.10) @f @t Àmin u x T Qx  u T Ru  @f @x  T (Ax  Bu) 45 (9:13) Introducing a relationship of the form f (x, t)  x T Px (9:14) where P is a square, symmetric matrix, then @f @t  x T @ @t Px (9:15) and @f @x  2Px @f @x ! T  2x T P (9:16) Inserting equations (9.15) and (9.16) into (9.13) gives x T @P @t x Àmin u x T Qx  u T Ru  2x T P(Ax  Bu) Âà (9:17) To minimize u, from equation (9.17) @[@f /@t] @u  2u T R  2x T PB  0(9:18) Equation (9.18) can be re-arranged to give the optimal control law u opt ÀR À1 B T Px (9:19) Optimal and robust control system design 275 //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 276 ± [272±324/53] 9.8.2001 2:39PM or u opt ÀKx (9:20) where K  R À1 B T P (9:21) Substituting equation (9.19) back into (9.17) gives x T  Px Àx T (Q  2PA ÀPBR À1 B T P)x (9:22) since 2x T PAx  x T (A T P  PA)x then  P ÀPA À A T P À Q PBR À1 B T P (9:23) Equation (9.23) belongs to a class of non-linear differential equations known as the matrix Riccati equations. The coefficients of P(t) are found by integration in reverse time starting with the boundary condition x T (t 1 )P(t 1 )x(t 1 )  0 (9:24) Kalman demonstrated that as integration in reverse time proceeds, the solutions of P(t) converge to constant values. Should t 1 be infinite, or far removed from t 0 , the matrix Riccati equations reduce to a set of simultaneous equations PA  A T P  Q ÀPBR À1 B T P  0 (9:25) Equations (9.23) and (9.25) are the continuous solution of the matrix Riccati equation. 9.2.2 Discrete form From equation (8.76) the discrete solution of the state equation is x[(k  1)T ]  A(T )x(kT ) B(T )u(kT )(9:26) For simplicity, if (kT ) is written as (k), then x(k  1)  A(T )x(k)  B(T )u(k)(9:27) The discrete quadratic performance index is J   NÀ1 k0 (x T (k)Qx(k)  u T (k)Ru(k))T (9:28) The discrete solution of the matrix Riccati equation solves recursively for K and P in reverse time, commencing at the terminal time, where K(N À (k 1))  [TR  B T (T )P(N À k)B(T )] À1 B T (T )P(N À k)A(T )(9:29) 276 Advanced Control Engineering //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 277 ± [272±324/53] 9.8.2001 2:39PM and P(N À (k 1))  [TQ  K T (N À (k 1))TRK(N À (k  1))]  [A(T ) À B(T )K(N À(k 1))] T P(N À k)[A(T ) ÀB(T )K(N À (k  1))] (9:30) As k is increased from 0 to N À 1, the algorithm proceeds in reverse time. When run in forward-time, the optimal control at step k is u opt (k) ÀK(k)x(k)(9:31) The boundary condition is specified at the terminal time (k  0), where x T (N)P(N)x(N)  0 (9:32) The reverse-time recursive process can commence with P(N)  0 or alternatively, with P(N À 1)  T Q. Example 9.1 (See also Appendix 1, examp91.m) The regulator shown in Figure 9.1 contains a plant that is described by  x 1  x 2 !  01 À1 À2 ! x 1 x 2 !  0 1 ! u y  [1 0]x and has a performance index J   I 0 x T 20 01 ! x  u 2 ! dt Determine (a) the Riccati matrix P (b) the state feedback matrix K (c) the closed-loop eigenvalues y r =0 u x = Ax + Bu – K C + x Fig. 9.1 Optimal regulator. Optimal and robust control system design 277 //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 278 ± [272±324/53] 9.8.2001 2:39PM Solution (a) A  01 À1 À2 45 B  0 1 45 Q  20 01 45 R  scalar  1 From equation (9.25) the reduced Riccati equation is PA  A T P  Q ÀPBR À1 B T P  0 (9:33) PA  p 11 p 12 p 21 p 22 ! 01 À1 À2 !  Àp 12 p 11 À 2p 12 Àp 22 p 21 À 2p 22 ! (9:34) A T P  0 À1 1 À2 ! p 11 p 12 p 21 p 22 !  Àp 21 Àp 22 p 11 À 2p 21 p 12 À 2p 22 ! (9:35) PBR À1 B T P  p 11 p 12 p 21 p 22 45 0 1 45 1[ 0 1 ] p 11 p 12 p 21 p 22 45  p 12 p 22 45 [ p 21 p 22 ]  p 12 p 21 p 12 p 22 p 22 p 21 p 2 22 45 (9:36) Combining equations (9.34), (9.35) and (9.36) gives Àp 12 p 11 À 2p 12 Àp 22 p 21 À 2p 22 45  Àp 21 Àp 22 p 11 À 2p 21 p 12 À 2p 22 45  20 01 45 À p 12 p 21 p 12 p 22 p 22 p 21 p 2 22 45  0 (9:37) Since P is symmetric, p 21  p 12 . Equation (9.37) can be expressed as four simultan- eous equations Àp 12 À p 12  2 À p 2 12  0(9:38) p 11 À 2p 12 À p 22 À p 12 p 22  0(9:39) 278 Advanced Control Engineering //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 279 ± [272±324/53] 9.8.2001 2:39PM Àp 22  p 11 À 2p 12 À p 12 p 22  0(9:40) p 12 À 2p 22  p 12 À 2p 22  1 À p 2 22  0(9:41) Note that equations (9.39) and (9.40) are the same. From equation (9.38) p 2 12  2p 12 À 2  0 solving p 12  p 21  0:732 and À2:732 Using positive value p 12  p 21  0:732 (9:42) From equation (9.41) 2p 12 À 4p 22  1 À p 2 22  0 p 2 22  4p 22 À 2:464  0 solving p 22  0:542 and À4:542 Using positive value p 22  0:542 (9:43) From equation (9.39) p 11 À (2  0:732)À0:542 À(0:732  0:542)  0 p 11  2:403 (9:44) From equations (9.42), (9.43) and (9.44) the Riccati matrix is P  2:403 0:732 0:732 0:542 45 (9:45) (b) Equation (9.21) gives the state feedback matrix K  R À1 B T P  1[ 0 1 ] 2:403 0:732 0:732 0:542 45 (9:46) Hence K  [0:732 0:542 ] Optimal and robust control system design 279 //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 280 ± [272±324/53] 9.8.2001 2:39PM (c) From equation (8.96), the closed-loop eigenvalues are jsI À A BKj0 s 0 0 s 45 À 01 À1 À2 45  0 1 45 0:732 0:542            0 s À1 1 s  2 45  00 0:732 0:542 45            0 s À1 1:732 s  2:542            0 s 2  2:542s  1:732  0 s 1 , s 2 À1:271 Æ j0:341 In general, for a second-order system, when Q is a diagonal matrix and R is a scalar quantity, the elements of the Riccati matrix P are p 11  b 2 2 r ! p 12 p 22 À p 22 a 21 À p 12 a 22 p 12  p 21  r b 2 2 a 21 Æ  a 2 21  q 11 b 2 2 r r 45 p 22  r b 2 2 a 22 Æ  a 2 22  (2p 12  q 22 ) r s 45 (9:47) 9.3 The linear quadratic tracking problem The tracking or servomechanism problem is defined in section 9.1.1(e), and is directed at applying a control u(t) to drive a plant so that the state vector x(t) follows a desired state trajectory r(t) in some optimal manner. 9.3.1 Continuous form The quadratic performance index to be minimized is J   t 1 t 0 (r À x) T Q(r À x) u T Ru Âà dt (9:48) It can be shown that the constrained functional minimization of equation (9.48) yields again the matrix Riccati equations (9.23) and (9.25) obtained for the LQR, combined with the additional set of reverse-time state tracking equations  s  (A À BR À1 B T P) T s À Qr (9:49) 280 Advanced Control Engineering //SYS21/D:/B&H3B2/ACE/REVISES(08-08-01)/ACEC09.3D ± 281 ± [272±324/53] 9.8.2001 2:39PM where s is a tracking vector, whose boundary condition is s(t 1 )  0(9:50) and the optimal control law is given by u opt ÀR À1 B T Px À R À1 B T s If v ÀR À1 B T s and K  R À1 B T P Then u opt  v À Kx (9:51) Hence, if the desired state vector r(t) is known in advance, tracking errors may be reduced by allowing the system to follow a command vector v(t) computed in advance using the reverse-time equation (9.49). An optimal controller for a tracking system is shown in Figure 9.2. 9.3.2 Discrete form The discrete quadratic performance index, writing (kT )as(k), is J   NÀ1 k0 [(r(k) À x(k)) T Q(r(k) À x(k))  u T (k)Ru(k)]T (9:52) Discrete minimization gives the recursive Riccati equations (9.29) and (9.30). These are run in reverse-time together with the discrete reverse-time state tracking equation s(N À (k 1))  F(T )s(N À k) G(T )r(N Àk)(9:53) Optimal Controller Plant r s v + u opt y x=Ax+Bu – Reverse-Time Equations Command vector Tracking vector C – – RB 1 T K s=( )A–BR B P s–Qr –1 TT x Fig. 9.2 Optimal tracking system. Optimal and robust control system design 281 [...]... model in equation (9. 79) The initial conditions were P Q 200 x(0) ˆ R 30 S À30 (9: 97) w Optimal Controller r + – e u1 K Fig 9. 11 Optimal control of band dryer x = Ax + Bu + Cdw Dryer y x C //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 294 ± [272±324/53] 9. 8.2001 2:39PM 294 Advanced Control Engineering and the disturbance vector P Q 0 wˆR 0 S w3 (9: 98) where w3 , the clay feed-rate was set at a... content (%) Max Min Max Min 0.17 0 .99 1.524 2.05 2.42 2.86 0 0 0 À2.05 À2.42 À2.86 2. 09 1.74 1.5 1.27 1.1 0. 89 À2.11 À2.13 À2.15 À1.27 À1.1 À0. 89 //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 293 ± [272±324/53] 9. 8.2001 2:39PM Optimal and robust control system design 293 The closed-loop eigenvalues are s ˆ À0:04 49 Æ j0:0422 s ˆ À0:0033 (9: 93) Implementation: The optimal control law was implemented by... important control parameters are (i) Burner gas supply valve angle va (t) (rad) (ii) Dryer clay feed-rate fi (t) (tonnes/hour) Process Air Wet Clay In Top Band Middle Band Dry Clay Out Lower Band Exhaust Air Fig 9. 9 Band drying oven //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 290 ± [272±324/53] 9. 8.2001 2:39PM 290 Advanced Control Engineering A proposed control scheme by Drew and Burns ( 199 2) uses... index  I Jˆ (xT Qx ‡ uT Ru)dt 0 (9: 85) (9: 86) //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 292 ± [272±324/53] 9. 8.2001 2:39PM 292 Advanced Control Engineering where Q and R are diagonal matrices P q11 0 Q ˆ R 0 q22 0 0 of the form Q 0 r 0 S; R ˆ 11 0 q33 0 r22 ! (9: 87) From equations (9. 20) and (9. 21), the optimal control law is uopt ˆ ÀKx where K ˆ RÀ1 BT P (9: 88) The design procedure employed... i.e dP ˆ0 dK (9: 64) Substitution of equation (9. 62) into (9. 63) gives P ˆ K 2 PA ‡ (1 À K)2 PB (9: 65) Hence K is given by É d È 2 K PA ‡ (1 À K)2 PB ˆ 0 dK From which Kˆ PB PA ‡ PB Substitution of equation (9. 66) into (9. 62) provides & ' PA ” x ˆ Ax À (Ax À Bx ) PA ‡ PB (9: 66) (9: 67) //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 286 ± [272±324/53] 9. 8.2001 2:39PM 286 Advanced Control Engineering. .. matrix Equations (9. 71)± (9. 76) are illustrated in Figure 9. 7 which shows the block diagram of the Kalman filter The recursive equations (9. 74)± (9. 76) that calculate the Kalman gain matrix and covariance matrix for a Kalman filter are similar to equations (9. 29) and (9. 30) that //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 288 ± [272±324/53] 9. 8.2001 2:39PM 288 Advanced Control Engineering 0(k /k)T... //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 306 ± [272±324/53] 9. 8.2001 2:39PM 306 Advanced Control Engineering 9. 7.2 H¥ -optimal control With HI -optimal control the inputs V( j!) are assumed to belong to a set of normbounded functions with weight W( j!) as given by equation (9. 125) Each input V( j!) in the set will result in a corresponding error E( j!) The HI -optimal controller is designed to... in the controller, steady-state errors must be expected However, the selection of the elements in the Q matrix, equation (9. 90), focuses the control effort on control1 .4 1.2 Valve Angle (rad) 1 0.8 0.6 0.4 0.2 0 0 100 200 300 400 500 Time (s) Fig 9. 12 Time response of gas-valve angle u1 (t) 600 700 800 90 0 1000 //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 295 ± [272±324/53] 9. 8.2001 2:39PM Optimal... //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 296 ± [272±324/53] 9. 8.2001 2:39PM 296 Advanced Control Engineering Dryer Temperature (°C) and Moisture Content (%) 60 50 40 Dryer Temperature 30 Moisture Content 20 10 0 0 100 200 300 400 500 600 Time (s) 700 800 90 0 1000 Fig 9. 14 Combined response of dryer temperature td (t) and moisture content mf (t) 12 Feed-rate = 6 tonnes/hr Feed-rate = 8 tonnes/hr 10... Moisture Content (%) Feed-rate = 10 tonnes/hr 8 Upper limit 6 lower limit 4 2 0 500 550 600 650 Fig 9. 15 Effect of varying clay feed-rate 700 750 800 Time (s) 850 90 0 95 0 1000 //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 297 ± [272±324/53] 9. 8.2001 2:39PM Optimal and robust control system design 297 The thermocouples measuring the burner and dryer temperatures were relatively noise-free, with standard . Clay Out Exhaust Air Fig. 9. 9 Band drying oven. Optimal and robust control system design 2 89 //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 290 ± [272±324/53] 9. 8.2001 2:39PM A proposed control scheme. Temperature K f K d Gs () 3 Ts () d Ts () b Ms () f Ts b () – + Ts d () Moisture Ts b () Gs 1 () Ts c () Gs 2 () Fig. 9. 10 Model of band dryer system. 290 Advanced Control Engineering //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 291 ± [272±324/53] 9. 8.2001 2:39PM Define the state. equations  s  (A À BR À1 B T P) T s À Qr (9: 49) 280 Advanced Control Engineering //SYS21/D:/B&H3B2/ACE/REVISES(0 8-0 8-0 1)/ACEC 09. 3D ± 281 ± [272±324/53] 9. 8.2001 2:39PM where s is a tracking vector,