376 COMPUTER BASED NUMERICAL AND STATISTICAL TECHNIQUES largest coefficient of y. We continue this process till last equation. This procedure is known as partial pivoting. In general, the rearrangement of equation is done even if pivot element is non-zero to improve the accuracy of solution by reducing the round off errors involved in elimination process, by getting a larger determinant, which is done by finding a largest element of the row as the pivotal element. Complete Pivoting: If the order of elimination of x 1 , x 2 , x 3 , is not important, then we may choose at each stage the largest coefficient of the whole matrix of coefficients. We may search the largest value, not only in rows but also in columns. After searching largest value, we bring at the diagonal position. This method of elimination is known as complete pivoting. The superiority of this method is that it gives the solution of a system, provided its determinant does not vanish in finite number of steps. 8.4 ILL-CONDITIONED SYSTEM OF EQUATIONS A system of equations A X = B is said to be ill-conditioned or unstable if it is highly sensitive to small changes in A and B i.e., small change in A or B causes a large change in the solution of the system. On the other hand if small changes in A and B give small changes in the solution, the system is said to be stable, or, well conditioned. Thus in a ill-conditioned system, even the small round off errors effect the solutions very badly. Unfortunately it is quite difficult to recognize an ill-conditioned system. For example, consider the following two almost identical systems. 12 1 xx−= 12 1 xx−= 1 1.00001 x − 2 0 x = and 1 0.99999 x − 2 0 x = Respective solutions are: (100001, 100000) and (–99999, –100000) obviously the two solutions differ very widely. Therefore the system is ill conditioned. Example 3. Show that the following system of linear equations is ill-conditioned. 7x 10y = 1− 5x + 7y = 0.7 Sol. On solving the given equations we get 0x = and 0.1.y = Now, we make slight changes in the given system of equations. The new system becomes. 7101.01xy+= 5 7 0.69xy+= Here we get =−0.17x and 0.22.y = Hence the given system is ill-conditioned. 8.5 ITERATIVE REFINEMENT OF THE SOLUTION BY GAUSS ELIMINATION METHOD The solution of system of equations will have some rounding error, we will discuss a technique called as ‘iterative refinement’ which leads to reduced rounding errors and often a reasonable solution for some ill-conditioned problems is obtained. SOLUTION OF SIMULTANEOUS LINEAR EQUATION 377 Consider the system of equations: 111 1 222 2 333 3 ax by cz d ax by cz d ax by cz d ++=   ++=   ++=  (1) Let x′, y′, z′ be an approximate solution, Substituting these values on the left-hand sides, we get new values of d 1 , d 2 , d 3 as d′ 1 , d′ 2 , d′ 3 , so their new system becomes; 111 1 222 2 333 3 ax by cz d ax by cz d ax by cz d ′′′′ ++=   ′′′′ ++=   ′′′′ ++=  (2) Subtracting each equation in () 2 from the corresponding equations in () 1 , we get 111 1 222 2 333 3 eee eee eee ax by cz k ax by cz k ax by cz k ++=   ++=   ++=  (3) where, ,, ee e x xxy yyz zz ′′ =− =− =− and iii k dd ′ =− We now solve the system () 3 for ,, eee xyz giving , e zxx ′ =+ , e yyy ′ =+ e zz z ′ =+ which will be better approximations for ,,.xyz We can repeat the process for improving the accuracy. Example 4. An approximate solution of the system 2x + 2y – z = 6, x – y + 2z = 8; – x + 3y + 2z = 4 is given by x = 2.8, y′ = 1, z = 1.8. Using the iterative method improve this solution. Sol. Substituting the approximate value x′ = 2.8, y′ = 1, z′ = 1.8 in the given equations, We get ()() 2 2.8 2 1 1.8 5.8 +−= () ++ = 2.8 1 2 1.8 7.4 (1) () ( ) −+ + = 2.8 3 1 2 1.8 3.8 Subtracting each equation in () 1 from the corresponding given equations, we get 22 0.2 eee xyz +−= 20.6 ee e xy z ++ = (2) 320.2 eee xyz −+−= where x e = x – 2.8, y e = y –1, z e = z –1.8 Solving the equations (2), we get x e = 0.2, y e = 0, z e = 0.2 This gives the better solution = 3,x = 1,y = 2,z which incidentally is the exact solution. Ans. 8.6 ITERATIVE METHOD FOR SOLUTION OF SIMULTANEOUS LINEAR EQUATION All the previous methods seen in solving the system of simultaneous algebraic linear equations are direct methods. Now we will see some indirect methods or iterative methods.           378 COMPUTER BASED NUMERICAL AND STATISTICAL TECHNIQUES This iterative methods is not always successful to all systems of equations. If this method is to succeed, each equation of the system must possess one large coefficient and the large coefficient must be attached to a different unknown in that equation. This condition will be satisfied if the large coefficients are along the leading diagonal of the coefficient matrix. When this condition is satisfied, the system will be solvable by the iterative method. The system, 11 1 12 2 13 3 1 a xaxax b ++= 21 1 32 2 23 3 2 a xaxax b ++= 31 1 32 2 33 3 3 a xaxaxb ++= will be solvable by this method if 11 12 13 aaa>+ 22 21 23 aaa>+ 33 31 32 aaa>+ In other words, the solution will exist (iterating will converge) if the absolute values of the leading diagonal elements of the coefficient matrix A of the system AX = B are greater than the sum of absolute values of the other coefficients of that row. The condition is sufficient but not necessary. Under the category of iterative method, we shall describe the following two methods: (i) Jacobi’s method (ii) Gauss-Seidel method. 8.6.1 Jacobi’s Method or Gauss-Jacobi Method Let us consider the system of simultaneous equations. 111 1 a xbycz d ++= 222 2 a xbycz d ++= (1) 333 3 a xbycz d ++= such that 12 ,, a b and 3 c are the largest coefficients of x,y,z, respectively. So that convergence is assured. Rearranging the above system of equations and rewriting in terms of ,,,xyz as: () 11 1 1 1 x dbycz a = −− () 22 2 2 1 y daxcz b = −− (2) () 33 3 3 1 z daxby c = −− let x 0 , y 0 , z 0 be the initial approximations of the unknowns x, y and z. Then, the first approximation are given by () 111010 1 1 x dbycz a =−− () 122020 2 1 y daxcz b =−−                SOLUTION OF SIMULTANEOUS LINEAR EQUATION 379 () 1 3 30 30 3 1 z daxby c =−− Similarly, the second approximations are given by () 211111 1 1 xdbycz a =−− () 222121 2 1 ydaxcz b =−− () 233131 3 1 zdaxby c =−− Proceeding in the same way, if ,, nnn x yz are the nth iterates then () 1111 1 1 nnn xdbycz a + =−− 1222 2 1 () nnn ydaxcz b + =−− 1333 3 1 () nnn zdaxby c + =−− The process is continued till convergency is secured. Note: In the absence of any better estimates, the initial approximations are taken as 0 0, x = 0 0, y = 0 0 z = . Example 5. Solve the following system of equation using Jacobi’s method 5x y + z = 10− 2x + 4y = 12 x+y+5z= 1− Start with the solution (2, 3, 0). Sol. Given system of equation can be written in the folliwng form, if we assume, x 0 , y 0 , z 0 as initial approximation: {} =++ 100 1 10 5 x yz {} =− 10 1 12 2 4 y x {} =−−− 100 1 1 5 z xy Now if 0 2, x = 0 3, y = 0 0 z = , then First approximation: {} =+−= 1 1 10 3 0 2.6 5 x 380 COMPUTER BASED NUMERICAL AND STATISTICAL TECHNIQUES {} =−= 1 1 12 4 2.0 4 y {} =−−−=− 1 1 123 1.2 5 z Second approximation: {} =++= 2 1 10 2 1.2 2.64 5 x {} 2 1 12 5.2 1.70 4 y =−= {} 2 1 12.62 1.12 5 z =−− −=− Third approximation: {} =++= 3 1 10 1.7 1.12 2.564 5 x {} 3 1 12 5.28 1.680 4 y =−= {} =−− − =− 3 1 12.641.7 1.068 5 z Fourth approximation: {} =++ = 4 1 10 1.68 1.068 2.5496 5 x =− = 4 1 {12 5.128} 1.7180 4 y {} 4 1 1 2.564 1.68 1.0488. 5 z =−− − =− Fifth approximation: x 5 = 1 5 {10 + 1.718 + 1.0428} = 2.553 y 4 = 1 4 {12 – 5.0992} = 1.725, z 5 = 1 5 {–1 – 2.5496 – 1.718} = – 1.054 Hence, approximating solution after having some other approximations is (up to 3 decimal places) x = 2.556 y = 1.725 z = –1.055. Ans. Example 6. Solve the following system of equations by Jacobi iteration method. 3x + 4y + 15z = 54.8, x + 12y + 3z = 39.66 and 10x + y – 2z = 7.74. Sol. The coefficient matrix of the given system is not diagonally dominant. Hence we rearrange the equations, as follows, such that the elements in the coefficient matrix are diagonally dominant. 10x + y – 2z = 7.74 x + 12y + 3z = 39.66 3x + 4y + 15z = 54.8 SOLUTION OF SIMULTANEOUS LINEAR EQUATION 381 Now, we write the equations in the form () () () 1 7.74 2 10 1 39.66 3 12 1 54.8 3 4 15 xyz yxz zxy  =−+    =−−    =−−   (1) We start from an approximation = x 0 = y 0 = z 0 = 0 Substituting these on RHS of (1), we get First approximation: x 1 = 1 10 [7.74 – 0 + 2(0)] = 0.774 y 1 = 1 12 [39.66 – 0 – 3(0)] = 1.1383333 z 1 = 1 15 [54.8 – 3(0) – 4(0)] = 3.6533333 Second approximation: x 2 = 1 10 [7.74 – 1.1383333 + 2(3.6533333)] = 1.3908333 y 2 = 1 12 [39.66 – 0.744 – 3(3.6533333)] = 2.3271667 z 2 = 1 15 [54.8 – 3(0.744) – 4(1.1383333)] = 3.1949778 Third approximation: x 3 = 1 10 [7.74 – 2.3271667 + 2(3.1949778)] = 1.1802789 y 3 = 1 12 [39.66 – 1.3908333 – 3(3.1949778)] = 2.3903528 z 3 = 1 15 [54.8 – 3(1.3908333) – 4(2.3271667)] = 2.7545889 Fourth approximation: x 4 = 1 10 [7.74 – 2.3903528 + 2(2.7545889)] = 1.0858825 y 4 = 1 12 [39.66 – 1.1802789 – 3(2.7545889)] = 2.5179962 z 4 = 1 15 [54.8 – 3(1.1802789) – 4(2.3903528)] = 2.7798501 Fifth approximation: x 5 = () 1 7.74 2.5179962 2 2.7798501 1.0781704 10 −+ =   y 5 = () 1 39.66 1.0858825 3 2.7798501 2.5195473 10 −− =   z 5 = ()() 1 54.8 3 1.0858825 4 2.5179962 2.7646912 15 −− =   382 COMPUTER BASED NUMERICAL AND STATISTICAL TECHNIQUES Sixth approximation: () 6 1 7.74 2.5195473 2 2.7646912 1.0749835 10 x =− + =   () 6 1 39.66 1.0781704 3 2.7646912 2.5239797 12 y =− − =   ()() 6 1 54.8 3 1.0781704 4 2.5195473 2.76582 15 z =− − =   Seventh approximation: () 7 1 7.74 2.5239797 2 2.76582 1.074766 10 x =− + =   () =− − =   7 1 39.66 1.0749835 3 2.76582 2.523963 12 y ()() 7 1 54.8 3 1.0749835 4 2.5239797 2.7652754 15 z =− − =   From the sixth and seventh approximations: 1.075,x = 2.524y = and 2.765z = correct to three decimals. Ans. 8.6.2 Guass-Seidel Method This is a modification of Gauss-Jacobi method. As before, the system of the linear equations. 111 1 ax by cz d++= 222 2 ax by cz d++= 333 3 ax by cz d++= is written as () 11 1 1 1 xdbycz a =−− (1) () 22 2 2 1 ydaxcz b =−− (2) () 33 3 3 1 zdaxby c =−− (3) and we start with the intial approximation x 0 , y 0 , z 0 . Substituting y 0 and z 0 in Eqn. () 1 , we get () 111010 1 1 xdbycz a =−− Now substituting 10 , xxzz== in Eqn. () 2 , we get () 122120 2 1 ydaxcz b =−− Substituting 1, xx= 1 yy= in Eqn. () 3 , we get () 1 3 31 31 3 1 zdaxby c =−− SOLUTION OF SIMULTANEOUS LINEAR EQUATION 383 This process is continued till the value of ,,,xyz are obtained to the desired degree of accuracy. In general, kth iteration can be written as () 11 11 1 1 kkk xdbycz a + =−− () 22 211 2 1 kkk ydaxcz b ++ =−− () 33 3111 3 1 kkk zdaxby c +++ =−− The rate of convergence of Gauss-Seidel method is roughly twice that of Gauss-Jacobi method. Example 7. Solve by Gauss-Seidel iteration method the system of equations 8x – 3y + 2z = 20; 6x + 3y + 12z = 35 and 4x + 11y – z = 33. Sol. From the given equations, we have () 1 20 3 2 8 xyz =+− (1) () 1 33 4 11 yxz =−+ (2) () 1 35 6 3 12 zxy =−− (3) Putting 0,y = 0z = in RHS of () 1 , we get 20 2.5 8 x == Putting 2.5,x = 0z = in RHS of () 2 , we get () 1 33 4 2.5 2.0909091 11 y =− =   Putting 2.5,x = 2.0909091y = in RHS of () 3 , we get ()( ) 1 1 35 6 2.5 3 2.0909091 1.1439394 12 z =−− =   For the second approximation: () 211 1 20 3 2 8 xyz =+− ()() 1 20 3 2.0909091 2 1.1439394 2.9981061 8 =+ − =   [] 221 1 33 4 11 yxz =−+ () 1 33 4 2.9981061 1.1439394 2.0137741 11 =− + =   [] 222 1 35 6 3 12 zxy =−− ()() 1 35 6 2.9981061 3 2.0137741 0.9141701 12 =− − =   384 COMPUTER BASED NUMERICAL AND STATISTICAL TECHNIQUES Third approximation: ()() 3 1 20 3 2.0137741 2 0.9141701 3.0266228 8 x =+ − =   () 3 1 33 4 3.0266228 0.9141701 1.9825163 11 y =− + =   ()() 3 1 35 6 3.0266228 3 1.9825163 0.9077262 12 z =− − =   Fourth approximation: ()() 4 1 20 3 1.9825163 2 0.9077262 3.0165121 8 x =+ − =   () 4 1 33 4 3.0165121 0.90777262 1.9856071 11 y =− + =   ()() 4 1 35 6 3.0165121 3 1.9856071 0.9120088 12 z =− − =   Fifth approximation: ()() 5 1 20 3 1.9856071 2 0.9120088 3.0166005 8 x =+ − =   () 5 1 33 4 3.0166005 0.9120088 1.9859643 11 y =− + =   ()() 5 1 35 6 3.0166005 3 1.9859643 0.9118753 12 z =− − =   Sixth approximation: ()() 6 1 20 3 1.9859643 2 0.9118753 3.0167568 8 x =+ − =   () 6 1 33 4 3.0167568 0.9118753 1.9858913 11 y =− + =   () 6 1 35 6 3.0167568 3(1.9858913 0.9118099 12 z =− − =   Seventh approximation: x 7 = 1 18 [20 + 3(1.9858913) –2(0.9118099)] = 3.0167568 y 7 = 1 11 [33 – 4(3.0167568) + 0.9118099] = 1.9858894 z 7 = 1 12 [35 – 6(3.0167568) –3(1.9858894)] = 0.9118159 Since at the sixth and seventh approximations, the values of ,,,xyz are the same, correct to four decimal places, we can stop the iteration process. ∴ 3.0167,x = = 1.9858,y 0.9118.z = We find that 12 iteration are necessary in Gauss-Jacobi Method to get the same accuracy as achieved by 7 iterations in Gauss-Seidel method. SOLUTION OF SIMULTANEOUS LINEAR EQUATION 385 Example 8. Solve the following system of equations using Gauss-Seidel method: 10x+y+2z=44 2x +10y + z = 51 x+2y+10z=61 Sol. Given system of equations can be written as: () 1 44 2 10 xyz =−− () 1 51 2 10 yxz =−− () 1 61 2 10 zxy =−− If we start by assuming 00 0 yz== then, we obtain () 1 1 44 0 0 4.4 10 x =−−= Now we susbtitute 4.4x = and 0 0 z = for 1 y and we obtain () 1 1 51 8.8 0 4.22 10 y =−−= Similarly, we obtain () 1 1 61 4.4 2 4.22 4.816 10 z =−−×= Now for second approximation, we obtain 2 4.0154 x = 2 3.0148 y = 2 5.0955 z = Third approximation is given by 3 3.0794 x = 3 3.9746 y = 3 4.9971 z = Similarly, if we proceed up to eighth approximation, then, we obtain 8 3.00 x = 8 4.00 y = 8 5.00 z = PROBLEM SET 8.1 1.1. 1.1. 1. Apply Gauss-Elimination method to solve the system of equations. 45xyz+−=− 612zy z+− =− 34xyz−−= [Ans. 117 , 71 x = 81 , 71 y − = 148 71 z = ] . 376 COMPUTER BASED NUMERICAL AND STATISTICAL TECHNIQUES largest coefficient of y. We continue this process till last equation. This procedure is known as partial pivoting. In general, the rearrangement. SYSTEM OF EQUATIONS A system of equations A X = B is said to be ill-conditioned or unstable if it is highly sensitive to small changes in A and B i.e., small change in A or B causes a large change. methods.           378 COMPUTER BASED NUMERICAL AND STATISTICAL TECHNIQUES This iterative methods is not always successful to all systems of equations. If this method is to succeed, each equation of the system