192 ALGEBRA 5.3.3-2. Relations between coordinate transformations and basis transformations. Suppose that in a linear n-dimensional space V, the transition from its basis e 1 , , e n to another basis  e 1 , ,  e n is determined by the matrix A (see Paragraph 5.3.3-1). Let x be any element of the space V with the coordinates (x 1 , , x n ) in the basis e 1 , , e n and the coordinates (x 1 , , x n ) in the basis  e 1 , ,  e n , i.e., x = x 1 e 1 + ···+ x n e n = x 1  e 1 + ···+ x n  e n . Then using formulas (5.3.3.1), we obtain the following relations between these coordinates: x j = n  i=1 x i a ij , x k = n  l=1 x l b lk , j, k = 1, , n. In terms of matrices and row vectors, these relations can be written as follows: (x 1 , , x n )=(x 1 , , x n )A,(x 1 , , x n )=(x 1 , , x n )A –1 or, in terms of column vectors, (x 1 , , x n ) T = A T (x 1 , , x n ) T ,(x 1 , , x n ) T =(A –1 ) T (x 1 , , x n ) T , where the superscript T indicates the transpose of a matrix. 5.4. Euclidean Spaces 5.4.1. Real Euclidean Space 5.4.1-1. Definition and properties of a real Euclidean space. A real Euclidean space (or simply, Euclidean space) is a real linear space V endowed with a scalar product (also called inner product), which is a real-valued function of two arguments x V, y V called the scalar product of these elements, denoted by x ⋅ y, and satisfying the following conditions (axioms of the scalar product): 1. Symmetry: x ⋅ y = y ⋅ x. 2. Distributivity: (x 1 + x 2 ) ⋅ y = x 1 ⋅ y + x 2 ⋅ y. 3. Homogeneity: (λx) ⋅ y = λ(x ⋅ y) for any real λ. 4. Positive definiteness: x ⋅ x ≥ 0 for any x,andx ⋅ x = 0 if and only if x = 0. If the nature of the elements and the scalar product is concretized, one obtains a specific Euclidean space. Example 1. Consider the linear space B 3 of all free vectors in three-dimensional space. The space B 3 becomes a Euclidean space if the scalar product is introduced as in analytic geometry (see Paragraph 4.5.3-1): x ⋅ y = |x||y| cos ϕ, where ϕ is the angle between the vectors x and y. Example 2. Consider the n-dimensional coordinate space R n whose elements are ordered systems of n arbitrary real numbers, x =(x 1 , , x n ). Endowing this space with the scalar product x ⋅ y = x 1 y 1 + ···+ x n y n , we obtain a Euclidean space. 5.4. EUCLIDEAN SPACES 193 T HEOREM. For any two elements x and y of a Euclidean space, the Cauchy–Schwarz inequality holds: (x ⋅ y) 2 ≤ (x ⋅ x)(y ⋅ y). A linear space V is called a normed space if it is endowed with a norm,whichisa real-valued function of x V, denoted by x and satisfying the following conditions: 1. Homogeneity: λx = |λ|x for any real λ. 2. Positive definiteness: x ≥ 0 and x = 0 if and only if x = 0. 3. The triangle inequality (also called the Minkowski inequality) holds for all elements x and y: x + y ≤ x + y.(5.4.1.1) The value x is called the norm of an element x or its length. T HEOREM. Any Euclidean space becomes a normed space if the norm is introduced by x = √ x ⋅ x.( 5 .4.1.2) C OROLLARY. In any Euclidean space with the norm (5.4.1.2), the triangle inequality (5.4.1.1) holds for all its elements x and y . The distance between elements x and y of a Euclidean space is defined by d(x, y)=x – y. One says that ϕ [0, 2π]istheangle between two elements x and y of a Euclidean space if cos ϕ = x ⋅ y xy . Two elements x and y of a Euclidean space are said to be orthogonal if their scalar product is equal to zero, x ⋅ y = 0. P YTHAGOREAN THEOREM. Let x 1 , x m be mutually orthogonal elements of a Eu- clidean space, i.e., x j ⋅ x j = 0 for i ≠ j .Then x 1 + ···+ x m  2 = x 1  2 + ···+ x m  2 . Example 3. In the Euclidean space B 3 of free vectors with the usual scalar product (see Example 1), the following relations hold: a = |a|,(a ⋅ b) 2 ≤ |a| 2 |b| 2 , |a + b| ≤ |a| + |b|. In the Euclidean space R n of ordered systems of n numbers with the scalar product defi ned in Example 2, the following relations hold: x =  x 2 1 + ···+ x 2 n , (x 1 y 1 + ···+ x n y n ) 2 ≤ (x 2 1 + ···+ x 2 n )(y 2 1 + ···+ y 2 n ),  (x 1 + y 1 ) 2 + ···+(x n + y n ) 2 ≤  x 2 1 + ···+ x 2 n  y 2 1 + ···+ y 2 n . 5.4.1-2. Orthonormal basis in a finite-dimensional Euclidean space. For elements x 1 , , x m of a Euclidean space, the mth-order determinant det[x i ⋅ x j ]is called their Gram determinant. These elements are linearly independent if and only if their Gram determinant is different from zero. 194 ALGEBRA One says that n elements i 1 , , i n of an n-dimensional Euclidean space V form its orthonormal basis if these elements have unit norm and are mutually orthogonal, i.e., i i ⋅ i j =  1 for i = j, 0 for i ≠ j. T HEOREM. In any n -dimensional Euclidean space V , there exists an orthonormal basis. Orthogonalization of linearly independent elements: Let e 1 , , e n be n linearly independent vectors of an n-dimensional Euclidean space V. From these vectors, one can construct an orthonormal basis of V using the following algorithm (called Gram–Schmidt orthogonalization): i i = g i √ g i ⋅ g i ,whereg i = e i – i  j=1 (e i ⋅ i j )i j (i = 1, 2, , n). (5.4.1.3) Remark. In any n-dimensional (n > 1) Euclidean space V,thereexistinfinitely many orthonormal bases. Properties of an orthonormal basis of a Euclidean space: 1. Let i 1 , , i n be an orthonormal basis of a Euclidean space V. Then the scalar product of two elements x = x 1 i 1 + ···+ x n i n and y = y 1 i 1 + ···+ y n i n is equal to the sum of products of their respective coordinates: x ⋅ y = x 1 y 1 + ···+ x n y n . 2. The coordinates of any vector x in an orthonormal basis i 1 , , i n are equal to the scalar product of x and the corresponding vector of the basis (or the projection of the element x on the axis in the direction of the corresponding vector of the basis): x k = x ⋅ i k (k = 1, 2, , n). Remark. In an arbitrary basis e 1 , , e n of a Euclidean space, the scalar product of two elements x = x 1 e 1 + ···+ x n e n and y = y 1 e 1 + ···+ y n e n has the form x ⋅ y = n  i=1 n  j=1 a ij x i y j , where a ij = e i ⋅ e j (i, j = 1, 2, , n). Let X, Y be subspaces of a Euclidean space V. The subspace X is called the orthogonal complement of the subspace Y in V if any element x of X is orthogonal to any element y of Y and X ⊕ Y = V. T HEOREM. Any n -dimensional Euclidean space V can be represented as the direct sum of its arbitrary subspace Y and its orthogonal complement X . Two Euclidean spaces V and  V are said to be isomorphic if one can establish a one-to-one correspondence between the elements of these spaces satisfying the following conditions: if elements x and y of V correspond to elements  x and  y of  V, then the element x + y corresponds to  x +  y; the element λx corresponds to λ  x for any λ; the scalar product (x ⋅ y) V is equal to the scalar product (  x ⋅  y)  V . T HEOREM. Any two n -dimensional Euclidean spaces V and  V are isomorphic. 5.4. EUCLIDEAN SPACES 195 5.4.2. Complex Euclidean Space (Unitary Space) 5.4.2-1. Definition and properties of complex Euclidean space (unitary space). A complex Euclidean space (or unitary space) is a complex linear space V endowed with a scalar product (also called inner product), which is a complex-valued function of two arguments x V and y V called their scalar product, denoted by x ⋅ y, satisfying the following conditions (called axioms of the scalar product): 1. Commutativity: x ⋅ y = y ⋅ x. 2. Distributivity: (x 1 + x 2 ) ⋅ y = x 1 ⋅ y + x 2 ⋅ y. 3. Homogeneity: (λx) ⋅ y = λ(x ⋅ y) for any complex λ. 4. Positive definiteness: x ⋅ x ≥ 0;andx ⋅ x = 0 if and only if x = 0. Here y ⋅ x is the complex conjugate of a number y ⋅ x. Example 1. Consider the n-dimensional complex linear space R n ∗ whose elements are ordered systems of n complex numbers, x =(x 1 , , x n ). We obtain a unitary space if the scalar product of two elements x =(x 1 , , x n )andy =(y 1 , ,y n ) is introduced by x ⋅ y = x 1 ¯y 1 + ···+ x n ¯y n , where ¯y j is the complex conjugate of y j . THEOREM. For any two elements x and y of an arbitrary unitary space, the Cauchy– Schwarz inequality holds: |x ⋅ y| 2 ≤ (x ⋅ x)(y ⋅ y). T HEOREM. Any unitary space becomes a normed space if the norm of its element x is introduced by x = √ x ⋅ x.(5.4.2.1) C OROLLARY. For any two elements x and y of a normed Euclidean space with the norm (5.4.2.1), the triangle inequality (5.4.1.1) holds. The distance between elements x and y of a unitary space is defined by d(x, y)=x – y.(5.4.2.2) Two elements x and y of a unitary space are said to be orthogonal if their scalar product is equal to zero, x ⋅ y = 0. 5.4.2-2. Orthonormal basis in a finite-dimensional unitary space. Elements x 1 , , x m of a unitary space V are linearly independent if and only if their Gram determinant is different from zero, det[x i ⋅ x j ] ≠ 0. One says that elements i 1 , , i n of an n-dimensional unitary space V form an or- thonormal basis of that space if these elements are mutually orthogonal and have unit norm, i.e., i i ⋅ i j =  1 for i = j, 0 for i ≠ j. Given any n linearly independent elements of a unitary space, one can construct an orthonormal basis of that space using the procedure described in Paragraph 5.4.1-2 (see formulas (5.4.1.3)). 196 ALGEBRA Properties of an orthonormal basis of a unitary space: 1. Let i 1 , , i n be an orthonormal basis in a unitary space. Then the scalar product of two elements x = x 1 i 1 + ···+ x n i n and y = y 1 i 1 + ···+ y n i n is equal to the sum x ⋅ y = x 1 ¯y 1 + ···+ x n ¯y n . 2. The coordinates of any vector in an orthonormal basis i 1 , , i n are equal to the scalar products of this vector and the vectors of the bases (or the projections of this element on the axes in the direction of the corresponding basis vectors): x k = x ⋅ i k (k = 1, 2, , n). Two unitary spaces V and  V are said to be isomorphic if there is a one-to-one corre- spondence between their elements satisfying the following conditions: if elements x and y of V correspond to elements  x and  y of  V,thenx + y corresponds to  x +  y; the element λx corresponds to λ  x for any complex λ; the scalar product (x ⋅ y) V is equal to the scalar product (  x ⋅  y)  V . T HEOREM. Any two n -dimensional unitary spaces V and  V are isomorphic. 5.4.3. Banach Spaces and Hilbert Spaces 5.4.3-1. Convergence in unitary spaces. Banach space. Any normed linear space is a metric space with the metric (5.4.2.2). A sequence {b s } of elements of a normed space V is said to be convergent to an element b V as s →∞if lim s→∞ b s – b = 0. Aseriesx 0 +x 1 +···with terms in a normed space is said to be convergent to an element x (called its sum; one writes x = lim n→∞ n  k=0 x k = ∞  k=0 x k ) if the sequence of its partial sums forms a sequence convergent to x, i.e., lim n→∞    x – n  k=0 x k    = 0. A normed linear space V is said to be complete if any sequence of its elements s 0 , s 1 , satisfying the condition lim n,m→∞ s n – s m  = 0 is convergent to some element s of the space V. A complete normed linear space is called a Banach space. Remark. Any finite-dimensional normed linear space is complete. 5.4.3-2. Hilbert space. A complete unitary space is called a Hilbert space. Any complete subspace of a Hilbert space is itself a Hilbert space. P ROJECTION THEOREM. Let V 1 be a complete subspace of a unitary space V . Then for any x V , there is a unique vector x p V 1 such that min y V 1 x – y = x – x p . 5.5. SYSTEMS OF LINEAR ALGEBRAIC EQUATIONS 197 Moreover, the vector x p is the unique element of V 1 for which the difference x – x p is orthogonal to any element x 1 of V 1 , i.e., (x – x p ) ⋅ x 1 = 0 for all x 1 V 1 . The mapping x → x p is a bounded linear operator from V to V 1 called the orthogonal projection of the space V to its subspace V 1 . 5.5. Systems of Linear Algebraic Equations 5.5.1. Consistency Condition for a Linear System 5.5.1-1. Notion of a system of linear algebraic equations. A system of m linear equations with n unknown quantities has the form a 11 x 1 + a 12 x 2 + ···+ a 1k x k + ···+ a 1n x n = b 1 , a 21 x 1 + a 22 x 2 + ···+ a 2k x k + ···+ a 2n x n = b 2 , a m1 x 1 + a m2 x 2 + ···+ a mk x k + ···+ a mn x n = b m , (5.5.1.1) where a 11 , a 12 , , a mn are the coefficients of the system; b 1 , b 2 , , b m are its free terms; and x 1 , x 2 , , x n are the unknown quantities. System (5.5.1.1) is said to be homogeneous if all its free terms are equal to zero. Other- wise (i.e., if there is at least one nonzero free term) the system is called nonhomogeneous. If the number of equations is equal to that of the unknown quantities (m = n), sys- tem (5.5.1.1) is called a square system. A solution of system (5.5.1.1) is a set of n numbers x 1 , x 2 , , x n satisfying the equations of the system. A system is said to be consistent if it admits at least one solution. If a system has no solutions, it is said to be inconsistent. A consistent system of the form (5.5.1.1) is called a determined system—it has a unique solution. A consistent system with more than one solution is said to be underdetermined. It is convenient to use matrix notation for systems of the form (5.5.1.1), AX = B,(5.5.1.2) where A ≡ [a ij ] is a matrix of size m × n called the basic matrix of the system; X ≡ [x i ]is a column vector of size n; B ≡ [b i ] is a column vector of size m. 5.5.1-2. Existence of nontrivial solutions of a homogeneous system. Consider a homogeneous system AX = O m ,(5.5.1.3) where A ≡ [a ij ] is its basic matrix of size m × n, X ≡ [x i ] is a column vector of size n, and O m ≡ [0] is a column vector of size m. System (5.5.1.3) is always consistent since it always has the so-called trivial solution X ≡ O n . T HEOREM. A homogeneous system (5.5.1.3) has a nontrivial solution if and only if the rank of the matrix A is less than the number of the unknown quantities n . It follows that a square homogeneous system has a nontrivial solution if and only if the determinant of its matrix of coefficients is equal to zero, det A = 0. 198 ALGEBRA Properties of the set of all solutions of a homogeneous system: 1. All solutions of a homogeneous system (5.5.1.3) form a linear space. 2. The linear space of all solutions of a homogeneous system (5.5.1.3) with n unknown quantities and a basic matrix of rank r is isomorphic to the space A n–r of all ordered systems of (n – r) numbers. The dimension of the space of solutions is equal to n – r. 3. Any system of (n–r) linearly independent solutions of the homogeneous system (5.5.1.3) forms a basis in the space of all its solutions and is called a fundamental system of solutions of that system. The fundamental system of solutions corresponding to the basis i 1 =(1, 0, , 0), i 2 =(0, 1, , 0), , i n–r =(0, 0, , 1)ofthespaceA n–r is said to be normal. 5.5.1-3. Consistency condition for a general linear system. System (5.5.1.1) or (5.5.1.2) is associated with two matrices: the basic matrix A of size m×n and the augmented matrix A 1 of size m× (n+1) formed by the matrix A supplemented with the column of the free terms, i.e., A 1 ≡ ⎛ ⎜ ⎜ ⎝ a 11 a 12 ··· a 1n b 1 a 21 a 22 ··· a 2n b 2 . . . . . . . . . . . . . . . a m1 a m2 ··· a mn b m ⎞ ⎟ ⎟ ⎠ .(5.5.1.4) K RONECKER–CAPELLI THEOREM. A linear system (5.5.1.1) [or (5.5.1.2)] is consistent if and only if its basic matrix and its augmented matrix (5.5.1.4) have the same rank, i.e. rank (A 1 )=rank(A) . 5.5.2. Finding Solutions of a System of Linear Equations 5.5.2-1. System of two equations with two unknown quantities. A system of two equations with two unknown quantities has the form a 1 x + b 1 y = c 1 , a 2 x + b 2 y = c 2 . (5.5.2.1) Depending on the coefficients a k , b k , c k , the following three cases are possible: 1 ◦ .IfΔ = a 1 b 2 – a 2 b 1 ≠ 0, then system (5.5.2.1) has a unique solution, x = c 1 b 2 – c 2 b 1 a 1 b 2 – a 2 b 1 , y = a 1 c 2 – a 2 c 1 a 1 b 2 – a 2 b 1 . 2 ◦ .IfΔ = a 1 b 2 – a 2 b 1 = 0 and a 1 c 2 – a 2 c 1 = 0 (the case of proportional coefficients), then system (5.5.2.1) has infinitely many solutions described by the formulas x = t, y = c 1 – a 1 t b 1 (b 1 ≠ 0), where t is arbitrary. 3 ◦ .IfΔ = a 1 b 2 – a 2 b 1 = 0 and a 1 c 2 – a 2 c 1 ≠ 0, then system (5.5.2.1) has no solutions. . the determinant of its matrix of coefficients is equal to zero, det A = 0. 198 ALGEBRA Properties of the set of all solutions of a homogeneous system: 1. All solutions of a homogeneous system (5.5.1.3) form. dimension of the space of solutions is equal to n – r. 3. Any system of (n–r) linearly independent solutions of the homogeneous system (5.5.1.3) forms a basis in the space of all its solutions and. matrix notation for systems of the form (5.5.1.1), AX = B,(5.5.1.2) where A ≡ [a ij ] is a matrix of size m × n called the basic matrix of the system; X ≡ [x i ]is a column vector of size n; B ≡