Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations Dr Abebe Geletu Ilmenau University of Technology Department of Simulation and Optimal Processes (SOP) Winter Semester 2011/12 Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Topics Newton’s Law: m¨ x = − Fl x m¨ y = mg Fl y Conservation of mechanical energy: x2 + y = l2 (DAE) x˙ = x3 x˙ = x4 F x1 ml F x˙ = g x2 l = x2 + y − l x˙ = − Numerical Methods of Ordinary Differential Equations Initial Value Problems (IVPs) Lecture Introduction1 to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Single Step Methods Multi-step Methods Initial Value Problems General form of an initial value problem (IVP) is x(t) ˙ = f (t, x(t)), t0 ≤ t ≤ tf , (1) x(t0 ) = x0 (2) (given), where x and f can be vector functions:     f1 (t, x(t)) x1 (t)  x2 (t)   f2 (t, x(t))      x(t) =   , f (t, x(t)) =       xn (t) fn (t, x(t)) If f is (explicitly) time independent; i.e f (t, x(t)) = f (x(t)), then the system (1) - (2) is said to be autonomous Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau x(t0 ) = x0 is the initial condition (start time of the motion) The expression x(t) ˙ represents the first order derivative of x(t) d with respect to time; i.e x(t) ˙ = dt x(t) Thus, the IVP (1) - (2) represents a system of first order differential equations If f (t, x(t)) = A(t)x(t), then we have x(t) ˙ = A(t)x(t), x(t0 ) = x0 is a first order linear IVP In the autonomous case we have A(t) = A Example: Equation of motion of a simple pendulum: θ˙ = ω g ω˙ = − sin(θ) L ⇒ θ˙ ω˙ = ω − Lg sin(θ) Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau , θ(0) = π4 ω(0) = Commonly, numerical methods are developed for systems of first order differential equations ⇒ any higher order differential equation should be written as a system of first order differential equations NB: The Matlab ODE Toolbox works only with systems of first order differential equations Strategy: Convert any n-th order initial value problem into a system of first-order IVP by introduction new variables dx d2 x dn−1 x , , , n−1 dt dt dt dx(t) dn−1 x(t) x(t0 ) = a1 , x(t ˙ ) = a2 , = a3 , , dt t=t0 dtn−1 dn x(t) =f dtn t, Introduce new variables: x1 = x, x2 = x, ˙ x3 = d dtx(t) , , xn = dn−1 x(t) dtn−1 Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau = an t=t0 Then the n-th order IVP reduces to x˙ = x2 , x1 (t0 ) = a1 x˙ = x3 , x2 (t0 ) = a2 x˙ n = f (t, x1 , x2 , , xn ), xn (t0 ) = an Example: The equation of motion of a simple pendulum (neglecting air resistance) with mass m is a 2nd-order equation mLθ¨ + mg sin(θ(t)) = 0, (3) ˙ θ(t0 ) = θ0 , θ = ω0 (4) ˙ Then Introduce ω = θ θ˙ = ω, θ(t0 ) = θ0 g ω˙ = − sin(θ), ω(t0 ) = ω0 L Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Objective: To determine a continuous function x∗ (t) that satisfies the dynamic equation (1); i.e x∗ (t) = f (t, x∗ (t)) and the initial condition (2); i.e x∗ (t0 ) = x0 !!! In general, the solution function x∗ (t) of the IVP is not easy to be analytically determined Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Basic idea of numerical solution of differential equations: Select a set of equally spaced time points t1 , t2 , , tN from the interval [t0 , tf ] (tf freely chosen for IVPs) That is, t1 = t0 + h , t2 = t1 + h, and so on; so that h = tk+1 − tk , k = 0, 1, , N − Approximate the continuous solution x∗ (t) of (1) & (2) by values at the discrete time points t1 , t2 , , tN ; i.e x(t1 ) = x1 , x(t2 ) = x2 , , x(tN ) = xN determine the values x1 , x2 , , xN through the IVP (1) - (2) Note: more discretization points from [t0 , tf ] implies better accuracy of the approximation; but demands more computation effort, depending on the algorithm used, it may be necessary to solve a system of linear or nonlinear equations Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Euler’s Method Motivation: for a single IVP x(t) ˙ = f (t, x(t)), x(t0 ) = x0 ; i.e x(t) ∈ R, ≤ t ≤ tf Given the point (t0 , x0 ) the value x(t ˙ 0) = dx(t) dt t=t is the slope of the tangent line to the graph of x(t) at the point (t0 , x0 ) Figure: Euler’s Method Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Since the function x(t) is not yet known, the derivative (slope) can be approximated by x(t0 + h) − x(t0 ) x(t ˙ 0) ≈ h Thus, using x(t ˙ ) = f (t0 , x0 ), we have x(t0 + h) − x(t0 ) = f (t0 , x(t0 )) h This implies x(t0 + h) = x(t0 ) + hf (t0 , x(t0 )) ⇒ x1 = x(t1 ) = x(t0 + h) = x(t0 ) + hf (t0 , x(t0 )) From this we obtain the value of x1 = x(t1 ) By the same procedure x2 can be also determined In general, once xk is known, xk+1 can be found from xk+1 = xk + hf (tk , xk ), k = 0, 1, , m − Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau We us the forward-difference numerical approximation for F (uk ): F (u(k) ) = F (u(k) + ∆) − F (u(k) ) ∆ for a given ∆ Note that F (u(k) + ∆) = x∗1 (tf , u(k) + ∆) − B • Since x∗1 (tf , u(k) + ∆) is unknown, we need to re-solve the problem (16) & (17) using x2 (t0 ) = u(k) + ∆ x˙ = x2 , x1 (t0 ) = A (18) (k) x˙ = ft (t, x1 ) + fx (t, x1 )x2 , x2 (t0 ) = u Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau + ∆ (19) Single Shooting A Single-Shooting Algorithm Select u0 , ∆ > and tolerance ε > while F (u(k) ) > ε • Compute inital value problems (16) & (17) with x2 (tf ) = u(k) and (18) & (19) with x2 (tf ) = u(k) + ∆ • Compute the vector F (u(k) ) and the Jacobian matrix F (u(k) ): F (u(k) + ∆) − F (u(k) ) ∆ • Solve the system of linear equations F (u(k) ) = F (u(k) )d = −F (u(k) ) to obtain dk as a solution • Set k ← k + • Set u(k) ← u(k) + dk end while Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Single Shooting Advantages/Disadvantages Advantages: • Simple to implement • effective when the interval [t0 , tf ] is short Disadvantages: • for some problems the algorithm can become un- stable • requires good initial guesses for the IVPs • the solution strategy is highly influenced by the length of the shooting (distance) interval [t0 , tf ] Hence, large numerber of iterations may be needed Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Multiple-Shooting Method • Consider again the problem x˙ = f (t, x), x(t0 ) = A, x(tf ) = B (20) • In the multiple-shooting method, the interval [t0 , tf ] is divided into subintervals [t0 , t1 ], [t1 , t2 ], , [tm−1 , tm ] by using time-points t0 = t1 , t2 , , tm−1 , tm = tf • On each subinterval tj ≤ t ≤ tj+1 , j = 0, 1, , m − 1, solve the initial value problems x˙ = f (t, x), x(tj ) = uj , tj ≤ t ≤ tj+1 , j = 0, 1, , m − 1, (21) to get a solution x(t; tj , uj ) Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Multiple-Shooting Method Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Multiple-Shooting Method • Test if all of the following equations hold true at the end points of each sub-interval x(t1 ; t0 , u0 ) = u1 (22) x(t2 ; t1 , u1 ) = u2 (23) x(tm−1 ; tm−2 , um−2 ) = um−1 x(tm ; tm−1 , um−1 ) = um = B • If yes, we are done The solution obtained satisfies the (BVP) • Otherwise, make a new and a better guess of the u’s Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau (24) (25) (26) • To find a better guess for the u s, solve the system of (possibly nonlinear) equations: F1 (u) = x(t1 ; t0 , A) − u1 = (27) F2 (u) = x(t2 ; t1 , u1 ) − u2 = (28) Fm−1 (u) = x(tm−1 ; tm−2 , um−2 ) − um−1 = Fm (u) = x(tm ; tm−1 , um−1 ) − B = • Put all the guess values into a variable vector u as   u1  u2    u=    um−1 • Note that, each of the uj ’s can be themselves vectors Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau (29) (30) (31) • Hence, solve the system of equations   F1 (u)  F2 (u)    F (u) =   =   Fm (u) using the Newton’s Algorithm to find a better guess for u • For a given u(k) , the vector F (u(k) ) can be computed using equations (27)-(31) Thus, we have   (k) x(t1 ; t0 , A) − u1   (k) F1 (u )   (k) (k) x(t2 ; t1 , u1 ) − u2   F2 (u(k) )       (k)   F (u ) =  =     (k) (k)   x(tm−1 ; tm−2 , um−2 ) − um−1  Fm (u(k) ) (k) x(tm ; tm−1 , um−1 ) − B Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau • The elements of the Jacobian matrix F (u(k) ) can be computed using finite differences: ∂Fj Fj (u(k) + ∆el ) − F (u(k) ) = ∂ul ∆ where el is the l-th unit vector in RK and K = length(u1 ) + + length(um ) • The value of Fj (u(k) + ∆el ) is obtained by solve the IVP x˙ = f (t, x), (k) x(tj ) = uj + ∆elj , tj ≤ t ≤ tj+1 , j = 0, , m − • Once the values F (u(k) ) and F (u(k) ) are available, solve the system of linear equaitons F (u(k) )d = −F (u(k) ) to obtain a solution d(k) , and update the guess vector as u(k+1) = u(k) + d(k) Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau • In fact, the Jacobian F (u(k) ) has  −I X2 −I   X3  (k) F (u ) =     the special structure  −I Xm−1     ,   −I  Xm where (the following are sensitivities w.r.t the initial values) X2 (t2 ; t1 , u1 ) = ∂x(t2 ; t1 , u1 ) ∂u (32) u=u(k) ∂x(tm−1 ; tm−2 , um−2 ) ∂u ∂x(tm ; tm−1 , um−1 ) Xm (tm ; tm−1 , um−1 ) = ∂u (k) Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations u=u Xm−1 (tm−1 ; tm−2 , um−2 ) = TU Ilmenau (33) (34) u=u(k) (35) A Multiple-Shooting Algorithm • Select shooting-points t0 < t1 < , tm = tf , initial guess (0) (0) (0) (u1 , u2 , , um ), ∆ > and tolerance ε > while F (u(k) ) > ε (k) • Solve the IVPs in equation (21) twice to obtain x(tj+1 ; tj , uj ) (k) and X(tj+1 ; tj , uj ), j = 0, 1, , m − • Construct the vector the vector F (u(k) ) and the Jacobian matrix F (u(k) ) • Solve the system of linear equations F (u(k) )d = −F (u(k) ) to obtain dk as a solution • Set k ← k + • Set u(k) ← u(k) + dk end while Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Multiple-Shooting Remarks Remark (Parallel Computation) The most time-consuming step in the multiple-shooting method is the computation of the values of the vector F (u) and Jacobian matrix F (u) for each Newton iteration The special structure of the Jacobian F (u) (also of F (u)) indicates that: • each j-th block row of the Jacobian can be obtained through the solution of the IVP only on the j-th shooting interval ⇒ the computation of the Jacobian can be done by solving smaller IVPs in parallel, correspond to the block rows of the Jacobian Choosing the shooting points The optimal choice of the shooting points t1 , t2 , , tm−1 can be problem dependent Hence, they can be considered as variables to be determined through some computational scheme Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Multiple-Shooting Advantages/Disadvantages Advantages: • it is efficient when the time-interval [t0 , tf ] too long; • it is easy to parallelize and, hence, applicable to problems of large dimensions; • accuracy of computations can be improved by considering additional intermediate points on the interval [tj , tj+1 ]; • it is applicable to solve stiff differential equations and DAEs Disadvantage: • For large-number of sub-intervals it can be computationally intensive; • The Newton-step may require the solution of a large set of linear equations requiring special linear-algebra strategies, • The choice of the shooting time points t1 , t2 , , tm−1 has an impact on the quality of the solution Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau References and Toolboxes References: • U M Ascherm, L R Petzold: Computer methods for ordinary differential equations and differential algebraic equations SISM 1998 • L F Shampine and M W Reichelt: The Matlab ODE Suite, SIAM J Sci Comput., V 18, pp 1-22, 1997 • R Ashino, M Nagase: Behind and beyond the Matlab ODE Toolbox Comput Math Appl V 40, pp 491 – 512, 2000 • L F Shampine, J Kierzenak: Solving boundary value problems for ordinary differential equations in Matlab bvp4c • W H Press, S A Teukolsky, W T Vetterling, B P Flannery: Numerical recipes in C++ (3r ed.) Cambridge University Press, 2007 Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau References and Toolboxes Toolboxes: The Matlab ODE-Toolbox found under the MATrix LABoratory (MATLAB) software - The Math Works SUite of Nonlinear and DIfferential/ALgebraic equation Solvers (Sundials) : https://computation.llnl.gov/casc/sundials/main.html Modelica : https://modelica.org/ Scialb: http://www.scilab.org/ EMSO (Environment for Modelling, Simulation and Optimisation): http://www.vrtech.com.br/rps/emso.html Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau ... (IVPs) Lecture Introduction1 to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Single Step Methods Multi-step Methods Initial Value Problems General form of an... possible to construct various 4th order Runge-Kutta methods Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Question: When RK methods. .. the IVP is not easy to be analytically determined Lecture Introduction to Numerical Methods for Differential and Differential Algebraic Equations TU Ilmenau Basic idea of numerical solution of