[...]... cost -1 5.1079 -8 .5437 -1 0.7228 -1 3.5379 -7 .2760 -1 0.1134 -1 5.4150 -1 5.4150 -8 .5437 -1 1.9682 -4 .3269 -1 3.8461 Ending Point x y 9.0390 5.9011 0.0000 9.0390 0.0000 7.4696 7.4696 5.9011 5.9011 5.9011 4.3341 4.3341 -1 1.2347 2.4567 2.4566 8.6682 5.5428 5.5428 2.4566 8.6682 8.6682 2.4566 5.5428 0.0000 8.6622 cost -1 1.6835 -8 .5437 -9 .5192 -1 5.1079 -6 .0724 -1 0.1134 -1 6.9847 -1 6.9847 -8 .5437 -1 1.9682 -4 .3269 -1 3.8461... baseball and a solution to a linear first-order differential equation Other problems have various minimum or maximum solutions known as optimal points or extrema, and best may be a relative definition Examples include best piece of artwork or best musical composition Practical Genetic Algorithms, Second Edition, by Randy L Haupt and Sue Ellen Haupt ISBN 0-4 7 1-4 556 5-2 Copyright © 2004 John Wiley & Sons,... zero yields —f (x) = —f (x n ) + (x - x n )H = 0 (1.16) Starting with a guess x0, the next point, xn+1, can be found from the previous point, xn, by x n+1 = x n - H -1 —f (x n ) (1.17) Rarely is the Hessian matrix known A myriad of algorithms have spawned around this formulation In general, these techniques are formulated as MINIMUM-SEEKING ALGORITHMS x n+1 = x n - a n A n—f (x n ) 17 (1.18) where an... when An = I, the identity matrix (1.18) becomes the method of steepest descent, and when An = H-1, (1.18) becomes Newton’s method Two excellent quasi-Newton techniques that construct a sequence of approximations to the Hessian, such that lim A n = H -1 (1.19) nÆ• The first approach is called the Davidon-Fletcher-Powell (DFP) algorithm (Powell, 1964) Powell developed a method that finds a set of line minimization... plot of the movement of the simplex down hill to surround the minimum Figure 1.7 minimum Mesh plot of the movement of the simplex down hill to surround the MINIMUM-SEEKING ALGORITHMS TABLE 1.1 Comparison of Nelder-Meade and BFGS Algorithms Nelder-Mead x Starting Point y 8.9932 3.4995 0.4985 8.4066 0.8113 6.8915 7.3021 5.6989 6.3245 5.6989 4.0958 4.2815 3.7830 8.9932 7.3803 5.9238 6.3148 1.8475 9.5406 8.2893... geological upheaval, are also constantly changing These changes act to revise the optimization equation That is what makes life (and genetic algorithms) interesting 1.5 THE GENETIC ALGORITHM The genetic algorithm (GA) is an optimization and search technique based on the principles of genetics and natural selection A GA allows a population composed of many individuals to evolve under specified selection rules... mating Number of processors to optimize speedup of a parallel GA Probability of the schema being selected to survive to the next generation Initial guess for Nelder-Mead algorithm Quantized version of Pn Quantization vector = [ 2-1 2-2 2 -N gene] Number of different values that variable i can have Probability of the schema being selected to mate Uniform random number Probability of the schema not being... form of Newton’s method Newton’s method is based on a mul- 16 INTRODUCTION TO OPTIMIZATION y start x Figure 1.11 Possible path that the steepest descent algorithm might take on a quadratic cost surface tidimensional Taylor series expansion of the function about the point xk given by T f ( x) = f (x n ) + —f (x n )(x - x n ) + (x - x n ) 2! T H(x - x n ) + (1.15) where xn = point about which Taylor... any value A constrained variable often converts into an unconstrained variable through a transforma- MINIMUM-SEEKING ALGORITHMS 5 tion of variables Most numerical optimization routines work best with unconstrained variables Consider the simple constrained example of minimizing f(x) over the interval -1 £ x £ 1 The variable converts x into an unconstrained variable u by letting x = sin(u) and minimizing... the other two points (B and C) As shown in Figure 1.5, D is found by D = B+C - A (1.10) 3 Expansion If the cost of D is smaller than that at A, then the move was in the right direction and another step is made in that same direction as shown in Figure 1.5 The formula is given by E= 3(B + C ) 2 - 2A (1.11) MINIMUM-SEEKING ALGORITHMS 11 C A I G F D E H B Figure 1.5 Manipulation of the basic simplex, . available in electronic format. Library of Congress Cataloging-in-Publication Data: Haupt, Randy L. Practical genetic algorithms / Randy L. Haupt, Sue Ellen Haupt. —2nd ed. p. cm. Red. ed. of: Practical. Practical genetic algorithms. c1998. “A Wiley-Interscience publication.” Includes bibliographical references and index. ISBN 0-4 7 1-4 556 5-2 1. Genetic algorithms. I. Haupt, S. E. II. Haupt, Randy L. Practical. Genetic Algorithms 136 5.14 Parallel Genetic Algorithms 137 5.14.1 Advantages of Parallel GAs 138 5.14.2 Strategies for Parallel GAs 138 5.14.3 Expected Speedup 141 5.14.4 An Example Parallel