www.it-ebooks.info www.it-ebooks.info AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS www.it-ebooks.info www.it-ebooks.info AN INTRODUCTION TO NUMERICAL METHODS AND ANALYSIS Second Edition JAMES F EPPERSON Mathematical Reviews WILEY www.it-ebooks.info Copyright © 2013 by John Wiley & Sons, Inc All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representation or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose No warranty may be created or extended by sales representatives or written sales materials The advice and strategies contained herein may not be suitable for your situation You should consult with a professional where appropriate Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002 Wiley also publishes its books in a variety of electronic formats Some content that appears in print, however, may not be available in electronic formats For more information about Wiley products, visit our web site at www.wiley.com Library of Congress Cataloging-in-Publication Data: Epperson, James F., author An introduction to numerical methods and analysis / James F Epperson, Mathematical Reviews — Second edition pages cm Includes bibliographical references and index ISBN 978-1-118-36759-9 (hardback) Numerical analysis I Title QA297.E568 2013 518—dc23 2013013979 Printed in the United States of America 10 www.it-ebooks.info To Mom (1920-1986) and Ed (1917-2012) a story of love, faith, and grace www.it-ebooks.info www.it-ebooks.info CONTENTS Preface Introductory Concepts and Calculus Review 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 xiii Basic Tools of Calculus 1.1.1 Taylor's Theorem 1.1.2 Mean Value and Extreme Value Theorems Error, Approximate Equality, and Asymptotic Order Notation 1.2.1 Error 1.2.2 Notation: Approximate Equality 1.2.3 Notation: Asymptotic Order A Primer on Computer Arithmetic A Word on Computer Languages and Software Simple Approximations Application: Approximating the Natural Logarithm A Brief History of Computing Literature Review References 2 14 14 15 16 20 29 30 35 37 40 41 A Survey of Simple Methods and Tools 43 2.1 2.2 43 48 Homer's Rule and Nested Multiplication Difference Approximations to the Derivative vii www.it-ebooks.info CONTENTS 2.3 2.4 2.5 2.6 2.7 Application: Euler's Method for Initial Value Problems Linear Interpolation Application—The Trapezoid Rule Solution of Tridiagonal Linear Systems Application: Simple Two-Point Boundary Value Problems Root-Finding 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 56 62 68 78 85 89 The Bisection Method Newton's Method: Derivation and Examples How to Stop Newton's Method Application: Division Using Newton's Method The Newton Error Formula Newton's Method: Theory and Convergence Application: Computation of the Square Root The Secant Method: Derivation and Examples Fixed-Point Iteration Roots of Polynomials, Part Special Topics in Root-finding Methods 3.11.1 Extrapolation and Acceleration 3.11.2 Variants of Newton's Method 3.11.3 The Secant Method: Theory and Convergence 3.11.4 Multiple Roots 3.11.5 In Search of Fast Global Convergence: Hybrid Algorithms Very High-order Methods and the Efficiency Index Literature and Software Discussion References 90 97 103 106 110 115 119 122 126 136 143 143 147 151 155 159 165 168 168 Interpolation and Approximation 171 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 171 177 187 192 195 198 202 210 210 211 223 228 234 4.9 4.10 4.11 Lagrange Interpolation Newton Interpolation and Divided Differences Interpolation Error Application: Muller's Method and Inverse Quadratic Interpolation Application: More Approximations to the Derivative Hermite Interpolation Piecewise Polynomial Interpolation An Introduction to Splines 4.8.1 Definition of the Problem 4.8.2 Cubic B-Splines Application: Solution of Boundary Value Problems Tension Splines Least Squares Concepts in Approximation www.it-ebooks.info CLENSHAW-CURTIS QUADRATURE 577 Apply your results from Problem to approximate solutions to = = 0, ti(l,t) = 0, u(x, 0) = ut u(-l,t) a{x)uxx, COS-KX/2, for the following choices of a: (a) a{x) = (1 + x2); (b) a(x) = e~x ; (c) a(x) = (1 — x2) (Because a vanishes at the boundary, this problem is known as degenerate, but you should be able to compute solutions.) Consider the nonlinear problem Ut = uxx ++ ' Vuux = βΐηπχ = o, = o, u(-l,t) u(l,t) u0(x) Use spectral collocation to attack this problem, as suggested in Problem 12 of §9.1, by treating the nonlinearity explicitly (at time t = tn) and the differential equation implicitly (at time t — i n +i)· Compare your spectral solution to the explicit solution computed in Problem 10 of §9.1 Comment on your results 10.3 CLENSHAW-CURTIS QUADRATURE A skeptical reader might wonder why a section on quadrature appears at the end of a chapter on spectral methods for ODEs and PDEs The answer is very simple: Our spectral methods for differential equations are based on using Chebyshev expansions, and that is also the basis for Clenshaw-Curtis quadrature [4] Consider the integration problem Hf) = / f(x)dx In Gaussian quadrature (§5.6) we found N weights u>i and N abscissas &, so that the quadrature pi /_' N f(x)dxfa'^2wif(xi) i=l is exact for all polynomials of degree < 2AT — This produced some remarkably accurate quadrature rules, but the weights and abscissas are not easily computed.8 The ClenshawCurtis idea, which leads to simpler weights and abscissas, is simply to expand / in a series It is possible to compute Gaussian quadrature weights and abscissas by solving a tridiagonal eigenvalue problem; see the article by Golub and Welch [7] Trefethen gives a very tidy MATLAB code for doing this in [10] www.it-ebooks.info 578 AN INTRODUCTION TO SPECTRAL METHODS of Chebyshev polynomials, and integrate this exactly We have f(x)dx~J2wiN)MN\ f J fc=l ~ which is very similar in form to Gaussian quadrature The abscissas are the same as our collocation points, = cos ξΐ N+l The weights can be computed very easily—much more easily than for Gaussian quadrature Following Boyd [1], we have w (N) ein f Δί>ιη ^π \Ν+ι) N+l I N £-G&)(; γ ^ / jk-π \ fl-cosjn for < k < N For comparison purposes, we tested Clenshaw-Curtis quadrature against Gaussian quadrature on a variety of integrals over the range N = 4,8,16,32, , 512 Fig 10.24 shows log-log plots of the error vs the number of quadrature points The plots show that Clenshaw-Curtis (marked with an "x") is almost always slightly less accurate than Gaussian quadrature (marked with an "o"), for a given value of TV However, it is a lot easier to construct the quadrature for a given value of N, and it is similarly much easier to create an adaptive version, although we will not that here sin(exp(n x)) Figure 10.24 expf-x2) 1/(1.5 -cos(5x)) Comparison of Clenshaw-Curtis quadrature (x) with Gaussian quadrature (o) www.it-ebooks.info LITERATURE AND SOFTWARE DISCUSSION 579 Exercises: Use the appropriate change of variable to show how to apply Clenshaw-Curtis quadrature to an integral over an arbitrary interval [a, b\ Write a program to Clenshaw-Curtis quadrature on each of the integrals in Problem of §5.6 Compare your results to those obtained with Gaussian quadrature Produce a log-log plot of the error as a function of N for each integral Looking at the plots in Fig 10.24, we see that most of them show a very rapid decrease of the error, and that a "rounding error plateau" is reached for most of the examples The exception is the last one, where the integrand is given by f(x) = \ / l - x Explain why this example is the one that displays this kind of suboptimal performance Let CN = Σ"=ι wïN)f(uN)) ~ / - i f{x)dx be the Clenshaw-Curtis quadrature operator Show that C2N uses some of the same function values as CN- Why is this important? < · · ·> 10.4 LITERATURE AND SOFTWARE DISCUSSION The classic reference for the spectral method is the book by Gottlieb and Orszag [8] Trefethen's book [9], the small volume by Gheorghiu [6], and Fornberg's book [5] were all useful in preparing this chapter There is also the book by Canuto et al [3] Boyd's book [2] has an unbelievably extensive bibliography, and a very unacademic style (that comment is intended as a compliment—the book is a pleasure to read) The author has to repeat, however, that there is much to the presentation here that can be criticized in terms of omissions My goal was to present an introduction to spectral methods that could be understood and appreciated by students in their first numerical analysis course REFERENCES Boyd, John, "Exponentially convergent Fourier-Chebyshev quadrature schemes on bounded and infinite intervals," J Sei Comput., vol 2, pp 99-109, 1987 Boyd, John, Chebyshev and Fourier Spectral Methods, Dover, New York, 2001, Second Edition; originally published by Springer-Verlag, 1989 Canuto, C, Quarteroni, A., Hussaini, M Y., and Zang, T A., Spectral Methods: Fundamentals in Single Domains, Springer-Verlag, Berlin, 2006 Clenshaw, C W., and Curtis, A R., "A method for numerical integration on an automatic computer," Numer Math.,\o\ 2, pp 197-205, 1960 Fornberg, Bengt, A Practical Guide to Pseudospectral Methods, Cambridge University Press, Cambridge, UK, 1995 Gheorghiu, C I., Spectral Methods for Differential Problems, Casa Cartii de Stiinta, ClujNapoca, Romania, 2007 (Out of print, available online at: http://www.ictp.acad.ro/gheorghiu/spectral.pdf www.it-ebooks.info 580 AN INTRODUCTION TO SPECTRAL METHODS Golub, G H., and Welsch, J H., "Calculation of Gauss quadrature rules," Math Comp., vol 23, pp 221-230, 1969 Gottleib, David, and Orszag, Steven A., Numerical Analysis of Spectral Methods: Theory and Applications, SIAM, Philadelphia, 1977 Trefethen, L N., Spectral Methods in MATLAB, SIAM, Philadelphia, 2000 10 Trefethen, L N., "Is Gauss quadrature better than Clenshaw-Curtis?" SIAM Rev., vol 50, no l,pp 67-87,2008 www.it-ebooks.info APPENDIX A PROOFS OF SELECTED THEOREMS, AND ADDITIONAL MATERIAL A.1 PROOFS OF THE INTERPOLATION ERROR THEOREMS The interpolation error theorems all depend on applying a generalization of Rolle's Theorem to a very carefully constructed function First, we will state and prove the general version of Rolle's Theorem Theorem A.l (Generalized Rolle's Theorem) Let f G Cn([a, b]) be given, and assume that there are n points, ZkA < k < n in [a, b] such that f{zk) = Then there exists at least one point ξ G [a, b] such that / ( n _ ) ( = Proof: By Rolle's Theorem, there exists at least one point ηι^ between each Zk and Zk+i such that /'(ïjfc) = Thus there are n — points where g\{x) — f'{x) is zero By the same argument, then, there are n — points where 52(2) = f"(x) is zero Continuing onward, then, we end up with a single point where gn-i(x) = f^n~^(x) is zero · This result allows us to prove, rather easily, both of Theorems 4.3 and 4.5 Theorem A.2 (Lagrange Interpolation Error Theorem) Let f G Cn+1([a, b]) and let the nodes Xk G [a, b] for < k < n Then, for each x G [a, b], there is a ξχ G [a, b] such that An Introduction to Numerical Methods and Analysis, Second Edition By James F Epperson Copyright © 2013 John Wiley & Sons, Inc www.it-ebooks.info 581 582 Proof: PROOFS OF SELECTED THEOREMS.AND ADDITIONAL MATERIAL For any t G [a, 6], t ψ Xk, define the function G(x) = E(x)-^E(t), where E(x) = f(x) — pn{x) and n w x ( ) = Y\_(x-Xk)· fc=0 Then G(xfc) = 0, for < k < n and G(t) = 0; thus, there are n + points where G is zero Therefore, by the Generalized Rolle's Theorem, there is a point ξ G [a, b] such that G n + ( £ ) = · But G (»+D (a ) E^n+1\x)-w{n^{x)E(t), = Therefore, G n + (£) = implies that /("+1)(0 - {j = => f(x) - pn(x) = ^ ^ / ( n + D f e ) , ^E(t) and we are done · Theorem A.3 (Hermite Interpolation Error Theorem) Let f G C2n([a, b]) and let the nodes Xk G [a, b] for all k, < k < n Then, for each x G [a, b], there is a ξχ G [a, 6] such that ηχ)-Ηη(χ) =^ ^ ( ξ χ ) , (A.1) n V>n(z) = JJ(x-Xfe)2 fc=l Proof: Essentially the same as for the Lagrange theorem This time we define the auxiliary function as where E(x) = f(x) - Hn(x) ξ G [a, b] such that Then the Generalized Rolle's Theorem gives us a point = G(2">(O = / ( " ) - - ^ U ( * ) , 1pn{t) from which the result follows · www.it-ebooks.info PROOF OF THE STABILITY RESULT FOR ODES A.2 583 PROOF OF THE STABILITY RESULT FOR ODES The theorem as stated in the text (Theorem 6.2) is the following Theorem A.4 Let f{t,y) be continuous for all (t,y) in a rectangle R, and smooth and uniformly monotone decreasing in y Then for all (io,2/o) € R there exists a unique solution to the initial value problem y' = f(t,y), y{t0) = yo Moreover, ifz(t) is the solution to the same problem with initial data z(to) — ZQ, then \y{t) - z(t)| < e - ^ ' - ^ l y o - z0\ (A.2) Proof: The existence and uniqueness follows from Theorem 6.1 and was demonstrated in §6.1; all we need to here is demonstrate the stability result (A.2) We have y'= f{t,y), y{to) = y0, z' = f(t,z), z(t0) = z0 and Subtract and use the Mean Value Theorem on / to get (y - *)' = fy(t, v){y - z), y(t0) - z(t0) = y0~ z0 Here η will be a function of y, z, and t, with value between y(t) and z(t) To simplify things, let's introduce some notation: φ(ί) = y(t) - z[t), p{t) = fy(t, η), o=yo- z0 We therefore have φ' - P(t){t) = 0, φ{ί0) = Φο, which is a simple first-order linear differential equation We can solve it by introducing the function P(t) = - f p(s)ds Jto p and the integrating factor e ^\ We then have (epW*(t))'=0, so we integrate both sides (from io to t) to get ep('V(i) - φ0 Hence, φ(ί) = φ0β-ρν, so we have y(t)-z(t) = (y0-z0)e-Xop{s)d3, www.it-ebooks.info 584 PROOFS OF SELECTED THEOREMS.AND ADDITIONAL MATERIAL and therefore, |y(t) - *WI < lift) - *ol max e " p W to