234 ALGEBRA Lay, D. C., Linear Algebra and Its Applications, 3rd Edition, Addison Wesley, Boston, 2002. Lial, M. L., Student’s Solutions Manual for College Algebra, 3rd Sol. Mn Edition, Addison Wesley, Boston, 2004. Lial,M.L.,Hornsby,J.,andSchneider,D.I.,College Algebra, 9th Edition, Addison Wesley, Boston, 2004. Lipschutz, S., 3,000 Solved Problems in Linear Algebra, McGraw-Hill, New York, 1989. Lipschutz, S. and Lipson, M., Schaum’s Outline of Linear Algebra, 3rd Edition, McGraw-Hill, New York, 2000. MacDuffee, C. C., The Theory of Matrices, Phoenix Edition, Dover Publications, New York, 2004. McWeeny, R., Symmetry: An Introduction to Group Theory and Its Applications, Unabridged edition, Dover Publications, New York, 2002. Meyer, C. D., Matrix Analysis and Applied Linear Algebra, Package Edition, Society for Industrial & Applied Mathematics, University City Science Center, Philadelphia, 2001. Mikhalev, A. V. and Pilz, G., The Concise Handbook of Algebra, Kluwer Academic, Dordrecht, Boston, 2002. Perlis, S., Theory of Matrices, Dover Ed. Edition, Dover Publications, New York, 1991. Poole, D., Linear Algebra: A Modern Introduction, Brooks Cole, Stamford, 2002. Poole, D., Student Solutions Manual for Poole’s Linear Algebra: A Modern Introduction, 2nd Edition, Brooks Cole, Stamford, 2005. Rose, J. S., A Course on Group Theory (Dover Books on Advanced Mathematics), Dover Publications, New York, 1994. Schneider, H. and Barker, G. Ph., Matrices and Linear Algebra, 2nd Edition (Dover Books on Advanced Mathematics), Dover Publications, New York, 1989. Scott, W. R., Group Theory, Dover Publications, New York, 1987. Shilov, G. E., Linear Algebra, Rev. English Ed. Edition, Dover Publications, New York, 1977. Strang, G., Introduction to Linear Algebra, 3rd Edition, Wellesley Cambridge Pr., Wellesley, 2003. Strang, G., Linear Algebra and Its Applications, 4th Edition, Brooks Cole, Stamford, 2005. Sullivan, M., College Algebra, 7th Edition, Prentice Hall, Englewood Cliffs, New Jersey, 2004. Tobey, J. and Slater, J., Beginning Algebra, 6th Edition, Prentice Hall, Englewood Cliffs, New Jersey, 2004. Tobey, J. and Slater, J., Intermediate Algebra, 5th Edition, Prentice Hall, Englewood Cliffs, New Jersey, 2005. Trefethen, L. N. and Bau, D., Numerical Linear Algebra, Society for Industrial & Applied Mathematics, University City Science Center, Philadelphia, 1997. Turnbull, H. W. and Aitken, A. C., An Introduction to the Theory of Canonical Matrices, Phoenix Edition, Dover Publications, New York, 2004. Vygodskii, M. Ya., Mathematical Handbook: Higher Mathematics, Mir Publishers, Moscow, 1971. Zassenhaus, H. J., The Theory of Groups, 2nd Ed. Edition, Dover Publications, New York, 1999. Zhang, F., Matrix Theory, Springer, New York, 1999. Chapter 6 Limits and Derivatives 6.1. Basic Concepts of Mathematical Analysis 6.1.1. Number Sets. Functions of Real Variable 6.1.1-1. Real axis, intervals, and segments. The real axis is a straight line with a point O chosen as the origin, a positive direction, and a scale unit. There is a one-to-one correspondence between the set of all real numbers R and the set of all points of the real axis, with each real x being represented by a point on the real axis separated from O by the distance |x| and lying to the right of O for x > 0, or to the left of O for x < 0. One often has to deal with the following number sets (sets of real numbers or sets on the real axis). 1. Sets of the form (a, b), (–∞, b), (a,+∞), and (–∞,+∞) consisting, respectively, of all x R such that a < x < b, x < b, x > a,andx is arbitrary are called open intervals (sometimes simply intervals). 2. Sets of the form [a, b] consisting of all x R such that a ≤ x ≤ b are called closed intervals or segments. 3. Sets of the form (a, b], [a, b), (–∞, b], [a,+∞) consisting of all x such that a < x ≤ b, a ≤ x < b, x ≤ b, x ≥ a are called half-open intervals. A neighborhood of a point x ◦ R is defined as any open interval (a, b) containing x ◦ (a < x ◦ < b). A neighborhood of the “point” +∞,–∞,or ∞ is defined, respectively, as any set of the form (b,+∞), (–∞, c)or(–∞,–a) ∪ (a,+∞)(here,a ≥ 0). 6.1.1-2. Lower and upper bound of a set on a straight line. The upper bound of a set of real numbers is the least number that bounds the set from above. The lower bound of a set of real numbers is the largest number that bounds the set from below. In more details: let a set of real numbers X R be given. A number β is called its upper bound and denoted sup X if for any x X the inequality x ≤ β holds and for any β 1 < β there exists an x 1 X such that x 1 > β 1 . A number α is called the lower bound of X and denoted inf X if for any x X the inequality x ≥ α holds and for any α 1 > α there exists an x 1 X such that x 1 < α 1 . Example 1. For a set X consisting of two numbers a and b (a < b), we have inf X = a,supX = b. Example 2. For intervals (open, closed, and half-open), we have inf(a, b)=inf[a, b]=inf(a, b]=inf[a, b)=a, sup(a, b)=sup[a, b]=sup(a, b]=sup[a, b)=b. 235 236 LIMITS AND DERIVATIVES One can see that the upper and lower bounds may belong to a given set (e.g., for closed intervals) and may not (e.g., for open intervals). The symbol +∞ (resp., –∞) is called the upper (resp., lower) bound of a set unbounded from above (resp., from below). 6.1.1-3. Real-valued functions of real variable. Methods of defining a function. 1 ◦ .LetD and E be two sets of real numbers. Suppose that there is a relation between the points of D and E such that to each x D there corresponds some y E, denoted by y = f (x). In this case, one speaks of a function f defined on the set D and taking its values in the set E.ThesetD is called the domain of the function f, and the subset of E consisting of all elements f(x) is called the range of the function f. This functional relation is often denoted by y = f(x), f : D → E, f : x → y. The following terms are also used: x is the independent variable or the argument; y is the dependent variable. 2 ◦ . The most common and convenient way to define a function is the analytic method:the function is defined explicitly by means of a formula (or several formulas) depending on the argument x; for instance, y = 2 sin x + 1. Implicit definition of a function consists of using an equation of the form F (x, y)=0, from which one calculates the value y for any fixed value of the argument x. Parametric definition of a function consists of defining the values of the independent variable x and the dependent variable y by a pair of formulas depending on an auxiliary variable t (parameter): x = p(t), y = q(t). Quite often functions are defi ned in terms of convergent series or by means of tables or graphs. There are some other methods of defining functions. 3 ◦ .Thegraph of a function is the representation of a function y = f(x) as a line on the plane with orthogonal coordinates x, y, the points of the line having the coordinates x, y = f (x), where x is an arbitrary point from the domain of the function. 6.1.1-4. Single-valued, periodic, odd and even functions. 1 ◦ . A function is single-valued if each value of its argument corresponds to a unique value of the function. A function is multi-valued if there is at least one value of its argument corresponding to two or more values of the function. In what follows, we consider only single-valued functions, unless indicated otherwise. 2 ◦ . A function f (x) is called periodic with period T (or T-periodic)iff (x + T )=f (x)for any x. 3 ◦ . A function f(x) is called even if it satisfies the condition f(x)=f(–x)foranyx.A function f (x) is called odd if it satisfies the condition f(x)=–f(–x)foranyx. 6.1.1-5. Decreasing, increasing, monotone, and bounded functions. 1 ◦ . A function f(x) is called increasing or strictly increasing (resp., nondecreasing)onaset D ⊂R if for any x 1 , x 2 D such that x 1 > x 2 ,wehavef(x 1 )>f(x 2 ) (resp., f (x 1 ) ≥ f(x 2 )). A function f(x) is called decreasing or strictly decreasing (resp., nonincreasing)onaset D if for all x 1 , x 2 D such that x 1 > x 2 ,wehavef(x 1 )<f(x 2 ) (resp., f(x 1 ) ≤ f(x 2 )). All such functions are called monotone functions. Strictly increasing or decreasing functions are called strictly monotone. 6.1. BASIC CONCEPTS OF MAT H E M AT I C AL ANALYSIS 237 2 ◦ . A function f(x) is called bounded on a set D if |f(x)| < M for all x D,whereM is a finite constant. A function f(x) is called bounded from above (bounded from below)ona set D if f(x)<M (M < f(x)) for all x D,whereM is a real constant. 6.1.1-6. Composite and inverse functions. 1 ◦ . Consider a function u = u(x), x D, with values u E,andlety = f (u) be a function defined on E. Then the function y = f  u(x)  , x D, is called a composite function or the superposition of the functions f and u. 2 ◦ . Consider a function y = f(x)thatmapsx D into y E.Theinverse function of y = f (x) is a function x = g(y)defined on E andsuchthatx = g(f(x)) for all x D.The inverse function is often denoted by g = f –1 . For strictly monotone functions f(x), the inverse function always exists. In order to construct the inverse function g(y), one should use the relation y = f(x) to express x through y. The function g(y) is monotonically increasing or decreasing together with f(x). 6.1.2. Limit of a Sequence 6.1.2-1. Some definitions. Suppose that there is a correspondence between each positive integer n and some (real or complex) number denoted, for instance, by x n . In this case, one says that a numerical sequence (or, simply, a sequence) x 1 , x 2 , , x n , is defined. Such a sequence is often denoted by {x n }; x n is called the generic term of the sequence. Example 1. For the sequence {n 2 – 2},wehavex 1 =–1, x 2 = 2, x 3 = 7, x 4 = 14,etc. A sequence is called bounded (bounded from above, bounded from below) if there is a constant M such that |x n | < M (respectively, x n < M , x n > M )foralln = 1, 2, 6.1.2-2. Limit of a sequence. A number b is called the limit of a sequence x 1 , x 2 , , x n , if for any ε > 0 there is N = N(ε) such that |x n – b| < ε for all n > N. If b is the limit of the sequence {x n }, one writes lim n→∞ x n = b or x n → b as n →∞. The limit of a constant sequence {x n = c} exists and is equal to c, i.e., lim n→∞ = c.Inthis case, the inequality |x n – c| < ε takes the form 0 < ε and holds for all n. Example 2. Let us show that lim n→∞ n n + 1 = 1. Consider the difference    n n + 1 – 1    = 1 n + 1 . The inequality 1 n + 1 < ε holds for all n > 1 ε – 1 = N(ε). Therefore, for any positive ε there is N = 1 ε – 1 such that for n > N we have    n n + 1 – 1    < ε. It may happen that a sequence {x n } has no limit at all, for instance, the sequence {x n } = {(–1) n }. A sequence that has a finite limit is called convergent. T HEOREM (BOLZANO–CAUCHY). A sequence x n has a finite limit if and only if for any ε > 0 ,thereis N such that the inequality |x n – x m | < ε holds for all n > N and m > N . 238 LIMITS AND DERIVATIVES 6.1.2-3. Properties of convergent sequences. 1. Any convergent sequence can have only one limit. 2. Any convergent sequence is bounded. From any bounded sequence one can extract a convergent subsequence.* 3. If a sequence converges to b, then any of its subsequence also converges to b. 4. If {x n }, {y n } are two convergent sequences, then the sequences {x n y n }, {x n ⋅ y n }, and {x n /y n } (inthisratio,itisassumedthaty n ≠ 0 and lim n→∞ y n ≠ 0) are also convergent and lim n→∞ (x n y n ) = lim n→∞ x n lim n→∞ y n ; lim n→∞ (cx n )=c lim n→∞ x n (c = const); lim n→∞ (x n ⋅ y n ) = lim n→∞ x n ⋅ lim n→∞ y n ; lim n→∞ x n y n = lim n→∞ x n lim n→∞ y n . 5. If {x n }, {y n } are convergent sequences and the inequality x n ≤ y n holds for all n, then lim n→∞ x n ≤ lim n→∞ y n . 6. If the inequalities x n ≤ y n ≤ z n hold for all n and lim n→∞ x n = lim n→∞ z n = b,then lim n→∞ y n = b. 6.1.2-4. Increasing, decreasing, and monotone sequences. A sequence {x n } is called increasing or strictly increasing (resp., nondecreasing)ifthe inequality x n+1 > x n (resp., x n+1 ≥ x n ) holds for all n. A sequence {x n } is called decreasing or strictly decreasing (resp., nonincreasing) if the inequality x n+1 < x n (resp., x n+1 ≤ x n ) holds for all n. All such sequences are called monotone sequences. Strictly increasing or decreasing sequences are called strictly monotone. T HEOREM. Any monotone bounded sequence has a finite limit. Example 3. It can be shown that the sequence  1 + 1 n  n  is bounded and increasing. Therefore, it is convergent. Its limit is denoted by the letter e: e = lim n→∞  1 + 1 n  n (e ≈ 2.71828). Logarithms with the base e are called natural or Napierian,andlog e x is denoted by ln x. 6.1.2-5. Properties of positive sequences. 1 ◦ . If a sequence x n (x n > 0) has a limit (finite or infinite), then the sequence y n = n √ x 1 ⋅ x 2 x n has the same limit. *Let{x n } be a given sequence and let {n k } be a strictly increasing sequence with k and n k being natural numbers. The sequence {x n k } is called a subsequence of the sequence {x n }. 6.1. BASIC CONCEPTS OF MAT H E M AT I C AL ANALYSIS 239 2 ◦ . From property 1 ◦ for the sequence x 1 , x 2 x 1 , x 3 x 2 , , x n x n–1 , x n+1 x n , , we obtain a useful corollary lim n→∞ n √ x n = lim n→∞ x n+1 x n , under the assumption that the second limit exists. Example 4. Let us show that lim n→∞ n n √ n! = e. Taking x n = n n n! and using property 2 ◦ ,weget lim n→∞ n n √ n! = lim n→∞ x n+1 x n = lim n→∞  1 + 1 n  n = e. 6.1.2-6. Infinitely small and infinitely large quantities. A sequence x n converging to zero as n →∞is called infinitely small or infinitesimal. A sequence x n whose terms infinitely grow in absolute values with the growth of n is called infinitely large or “tending to infinity.” In this case, the following notation is used: lim n→∞ x n = ∞. If, in addition, all terms of the sequence starting from some number are positive (negative), then one says that the sequence x n converges to “plus (minus) infinity,” and one writes lim n→∞ x n =+∞  lim n→∞ x n =–∞  . For instance, lim n→∞ (–1) n n 2 = ∞, lim n→∞ √ n =+∞, lim n→∞ (–n)=–∞. T HEOREM (STOLZ). Let x n and y n be two infinitely large sequences, y n → +∞ ,and y n increases with the growth of n (at least for sufficiently large n ): y n+1 > y n .Then lim n→∞ x n y n = lim n→∞ x n – x n–1 y n – y n–1 , provided that the right limit exists (finite or infinite). Example 5. Let us find the limit of the sequence z n = 1 k + 2 k + ···+ n k n k+1 . Taking x n = 1 k + 2 k + ···+ n k and y n = n k+1 in the Stolz theorem, we get lim n→∞ z n = lim n→∞ n k n k+1 –(n – 1) k+1 . Since (n – 1) k+1 = n k+1 –(k + 1)n k + ··· ,wehaven k+1 –(n – 1) k+1 =(k + 1)n k + ···, and therefore lim n→∞ z n = lim n→∞ n k (k + 1)n k + ··· = 1 k + 1 . 240 LIMITS AND DERIVATIVES 6.1.2-7. Upper and lower limits of a sequence. The limit (finite or infinite) of a subsequence of a given sequence x n is called a partial limit of x n . In the set of all partial limits of any sequence of real numbers, there always exists the largest and the least (finite or infinite). The largest (resp., least) partial limit of a sequence is called its upper (resp., lower) limit. The upper and lower limits of a sequence x n are denoted, respectively, lim n→∞ x n , lim n→∞ x n . Example 6. The upper and lower limits of the sequence x n =(–1) n are, respectively, lim n→∞ x n = 1, lim n→∞ x n =–1. A sequence x n has a limit (finite or infinite) if and only if its upper limit coincides with its lower limit: lim n→∞ x n = lim n→∞ x n = lim n→∞ x n . 6.1.3. Limit of a Function. Asymptotes 6.1.3-1. Definition of the limit of a function. One-sided limits. 1 ◦ . One says that b is the limit of a function f (x)asx tends to a if for any ε > 0 there is δ = δ(ε)>0 such that |f (x)–b| < ε for all x such that 0 < |x – a| < δ. Notation: lim x→a f(x)=b or f (x) → b as x → a. One says that b is the limit of a function f (x)asx tends to +∞ if for any ε > 0 there is N = N(ε)>0 such that |f (x)–b| < ε for all x > N . Notation: lim x→+∞ f(x)=b or f (x) → b as x → +∞. In a similar way, one defines the limits for x → –∞ or x →∞. T HEOREM (BOLZANO–CAUCHY 1). A function f(x) has a finite limit as x tends to a ( a is assumed finite) if and only if for any ε > 0 there is δ > 0 such that the inequality |f(x 1 )–f(x 2 )| < ε (6.1.3.1) holds for all x 1 , x 2 such that |x 1 – a| < δ and |x 2 – a| < δ . THEOREM (BOLZANO–CAUCHY 2). A function f(x) has a finite limit as x tends to +∞ if and only if for any ε > 0 there is Δ > 0 such that the inequality (6.1.3.1) holds for all x 1 > Δ and x 2 > Δ . 2 ◦ . One says that b is the left-hand limit (resp., right-hand limit)ofafunctionf(x)asx tends to a if for any ε > 0 there is δ = δ(ε)>0 such that |f(x)–b| < ε for a – δ < x < a (resp., for a < x < a + δ). Notation: lim x→a–0 f(x)=b or f (a – 0)=b (resp., lim x→a+0 f(x)=b or f (a + 0)=b). 6.1.3-2. Properties of limits. Let a be a number or any of the symbols ∞,+∞,–∞. 1. If a function has a limit at some point, this limit is unique. 2. If c is a constant function of x, then lim x→a c = c. . |x| and lying to the right of O for x > 0, or to the left of O for x < 0. One often has to deal with the following number sets (sets of real numbers or sets on the real axis). 1. Sets of. bound of X and denoted inf X if for any x X the inequality x ≥ α holds and for any α 1 > α there exists an x 1 X such that x 1 < α 1 . Example 1. For a set X consisting of two numbers a and. definition of a function consists of using an equation of the form F (x, y)=0, from which one calculates the value y for any fixed value of the argument x. Parametric definition of a function consists of