190 MARKOV CHAIN MONTE CARL0 is thus to minimize (6.23) Note that the number of elements in X is typically very large, because I XI = n!. The TSP can be solved via simulated annealing in the following way. First, we define the target pdf to be the Boltzmann pdf f(x) = ce-s(x)/T. Second, we define a neighborhood structure on the space of permutations X called 2- opt. Here the neighbors of an arbitrary permutation x are found by (1) select- ing two different indices from { 1, . . . , n} and (2) reversing the path of x between those two indices. For example, if x = (1,2,. . . ,lo) and indices 4 and 7 are selected, then y = (1,2,3,7,6,5,4,8,9,10); see Figure 6.13. Another exam- ple is: if x = (6,7,2,8,3,9,10,5,4, 1) and indices 6 and 10 are selected, then y = (6,7,2,8,3,1,4,5,10,9). 1 2 3 4 5 1 2 3 4 5 10 9 8 7 6 ix 10 9 8 7 6 Figure 6.13 Illustration of the 2-opt neighborhood structure. Third, we apply the Metropolis-Hastings algorithm to sample from the target. We need to supply a transition function 9(x, y) from x to one of its neighbors. Typically, the two indices for the 2-opt neighborhood are selected uniformly. This can be done, for example, by drawing a uniform permutation of (1, . . . , n) (see Section 2.8) and then selecting the first two elements of this permutation. The transition function is here constant: q(x, y) = 9(y, x) = 1/ (i) . It follows that in this case the acceptance probability is By gradually decreasing the temperature T, the Boltzmann distribution becomes more and more concentrated around the global minimizer. This leads to the following generic simulated annealing algorithm with Metropolis-Hastings sampling. SIMULATED ANNEALING 191 Algorithm 6.8.1 (Simulated Annealing: Metropolis-Hastings Sampling) 1. Initialize the starting state XO and temperature TO. Set t = 0. 2. Generate a new state Y from the symmetric proposal q(X1, y). 3. IfS(Y) < S(Xt) let Xt+l = Y. IfS(Y) 2 S(Xt), generate U - U(0,l) and let Xt+l = Y if otherwise, let Xt+l = Xt. (y < e-(s(Y)-s(xc))/~c . 4. Select a new temperature Tt+l < Tt, increase t by I, and repeat from Step 2 until stopping. A common choice in Step 4 is to take Tt+l = flTt for some fl < 1 close to 1, such as ,D = 0.99. EXAMPLE 6.13 n-Queens Problem In the n-queens problem the objective is to arrange n queens on a n x n chess board in such a way that no queen can capture another queen. An illustration is given in Figure 6.14 for the case n = 8. Note that the configuration in Figure 6.14 does not solve the problem. We take n = 8 from now on. Note that each row of the chess board must contain exactly one queen. Denote the position of the queen in the i-th row by xi; then each configuration can be represented by a vector x = (21, . . . ,Q). For example, x = (2,3,7,4,8,5,1,6) corresponds to the large configuration in Figure 6.14. Two other examples are given in the same figure. We can now formulate the problem of minimizing the function S(x) representing the number of times the queens can capture each other. Thus S(x) is the sum of the number of queens that can hit each other minus 1; see Figure 6.14, where S(x) = 2 for the large configuration. Note that the minimal S value is 0. One of the optimal solutions is ~'=(5,1,8,6,3,7,2,4). x = (2,5,4,8,3,7,3,5) S(X) = 6 x= (1,8,3,1,5,8,4,2) S(x) = 7 Figure 6.14 Position the eight queens such that no queen can capture another. 192 MARKOV CHAIN MONTE CARL0 We show next how this optimization problem can be solved via simulated annealing using the Gibbs sampler. As in the previous TSP example, each iteration of the algorithm consists of sampling from the Boltzmann pdf f(x) = e-S(x)/T via the Gibbs sampler, followed by decreasing the temperature. This leads to the following generic simulated annealing algorithm using Gibbs sampling. Algorithm 6.8.2 (Simulated Annealing: Gibbs Sampling) 1. Initialize the starting state XO and temperature TO. Set t = 0. 2. For a given Xt, generate Y = (Y1, . . . , Y,) as follows: i. Draw Y1 from the conditionalpdff(x1 I Xt,2,. . . Xt,n). ii. Draw Yi from j(xi I Y1,. . . , x-1, Xt,i+l,. . . , Xt,n), iii. Draw Y,from j(z, 1 Y1,. . . , Yn-l). i = 2,. . . ,TI - 1. 3. Let Xt+l = Y. 4. rfS(Xt) = 0 stop and display the solution; otherwise, select a new temperature Tt+l < Tt, increase t by I, and repeat from Step 2. Note that in Step 2 each Y, is drawn from a discrete distribution on { 1, . . . , n} with probabilities proportional to e-s(zl)/Tt, . . . , e-s(zv,)/Tt, where each Zk is equal to the vector (Y1,. . . , Yi-1, k, Xt,i+ll. . . ,Xt,,). Other MCMC samplers can be used in simulated annealing. For example, in the hide- and-seek algorithm [20] the general hit-and-run sampler (Section 6.3) is used. Research motivated by the use of hit-and-run and discrete hit-and-run in simulated annealing, has resulted in the development of a theoretically derived cooling schedule that uses the recorded values obtained during the course of the algorithm to adaptively update the temperature [22, 231. 6.9 PERFECT SAMPLING Returning to the beginning of this chapter, suppose that we wish to generate a random variable X taking values in { 1, . . . , m} according to a target distribution x = { xi}. As mentioned, one of the main drawbacks of the MCMC method is that each sample Xt is only asymptotically distributed according to x, that is, limt+m P(Xt = i) = xi. In contrast, perfect sampling is an MCMC technique that produces exact samples from K. Let {X,} be a Markov chain with state space { 1,. . . , m}, transition matrix P, and stationary distribution K. We wish to generate the {Xt, t = 0, -1, -2>. . .} in such a way that Xo has the desired distribution. We can draw XO from the rn-point distribution corresponding to the X-l-th row of P, see Algorithm 2.7.1. This can be done via the IT method, which requires the generation of a random variable UO - U(0, 1). Similarly, X-1 can be generated from X-2 and U-1 - U(0,l). In general, we see that for any negative time -t the random variable Xo depends on XWt and the independent random variables Next, consider m dependent copies of the Markov chain, starting from each of the states 1, . . . , m and using the same random numbers { Uz} - similar to the CRV method. Then, if two paths coincide, or coalesce, at some time, from that time on, both paths will be identical. Cl_t+l,. . . , vo N U(0,l). PERFECT SAMPLING 193 The paths are said to be coupled. The main point of the perfect sampling method is that if the chain is ergodic (in particular, if it is aperiodic and irreducible), then withprobabiliv I there exists a negative time -T such that all m paths will have coalesced before or at time 0. The situation is illustrated in Figure 6.15. 6 ' 2' 1' Lt -T T 0 Figure 6.15 All Markov chains have coalesced at time -7. Let U represent the vector of all Ut, t 6 0. For each U we know there exists, with probability 1, a -T(U) < 0 such that by time 0 all m coupled chains defined by U have coalesced. Moreover, if we start at time -T a stationaly version of the Markov chain, using again the same U, this stationary chain must, at time t = 0, have coalesced with the other ones. Thus, any of the m chains has at time 0 the same distribution as the stationary chain, which is T. Note that in order to construct T we do not need to know the whole (infinite vector) U. Instead, we can work backward from t = 0 by generating U-1 first, and checking if -T = -1. If this is not the case, generate U-2 and check if -T = -2, and so on. This leads to the following algorithm, due to Propp and Wilson [ 181, called coupling from the past. Algorithm 6.9.1 (Coupling from the Past) I. Generate UO - U(0,l). Set UO = Uo. Set t = -1. 2. Generate m Markov chains, starting at t from each of the states 1, . . . , m, using the same random vector Ut+l. 3. Check if all chains have coalesced before or at time 0. If so, return the common value of the chains at time 0 andstop; otherwise, generate Ut - U(0, l), let Ut = (Ut, UL+l), set t = t - 1, and repeat from Step 2. Although perfect sampling seems indeed perfect in that it returns an exact sample from the target x rather than an approximate one, practical applications of the technique are, presently, quite limited. Not only is the technique difficult or impossible to use for most continuous simulation systems, it is also much more computationally intensive than simple MCMC. 194 MARKOV CHAIN MONTE CARL0 PROBLEMS 6.1 Verify that the local balance equation (6.3) holds for the Metropolis-Hastings algo- rithm. 6.2 When running an MCMC algorithm, it is important to know when the transient (or burn-in) period has finished; otherwise, steady-state statistical analyses such as those in Section 4.3.2 may not be applicable. In practice this is often done via a visual inspection of the sample path. As an example, run the random walk sampler with normal target distribution N(10,l) and proposal Y - N(z,0.01). Take a sample size of N = 5000. Determine roughly when the process reaches stationarity. 6.3 A useful tool for examining the behavior of a stationary process { X,} obtained, for example, from an MCMC simulation, is the covariance function R(t) = Cov(Xt, XO); see Example 6.4. Estimate the covariance function for the process in Problem 6.2 and plot the results. In Matlab’s signal processing toolbox this is implemented under the M-function xc0v.m. Try different proposal distributions of the form N(z, g2) and observe how the covariance function changes. 6.4 Implement the independence sampler with an Exp( 1) target and an Exp( A) proposal distribution for several values of A. Similar to the importance sampling situation, things go awry when the sampling distribution gets too far from the target distribution, in this case when X > 2. For each run, use a sample size of lo5 and start with z = 1. a) For each value X = 0.2,1,2, and 5, plot a histogram of the data and compare it with the true pdf. b) Foreach value of the above values of A, calculate the sample mean and repeat this for20 independent runs. Make a dotplot of the data (plot them on a line) and notice the differences. Observe that for X = 5 most of the sample means are below 1, and thus underestimate the true expectation 1, but a few are significantly greater. Observe also the behavior of the corresponding auto-covariance functions, both between the different As and, for X = 5, within the 20 runs. 6.5 Implement the random walk sampler with an Exp( 1) target distribution, where Z (in the proposal Y = z + 2) has a double exponential distribution with parameter A. Carry out a study similar to that in Problem 6.4 for different values of A, say X = 0.1, 1,5: 20. Observe that (in this case) the random walk sampler has a more stable behavior than the independence sampler. 6.6 Let X = (X, Y)T be a random column vector with a bivariate normal distribution with expectation vector 0 = (0, O)T and covariance matrix a) Show that (Y I X = x) - N(ex, 1 - e2) and (XI Y = y) - N(ey, 1 - e2). b) Write a systematic Gibbs sampler to draw lo4 samples from the bivariate distri- 6.7 A remarkable feature of the Gibbs sampler is that the conditional distributions in Algorithm 6.4.1 contain sufficient information to generate a sample from the joint one. The following result (by Hammersley and Clifford [9]) shows that it is possible to directly bution N(O,2’) and plot the data for e = 0,0.7 and 0.9. PROBLEMS 195 express the joint pdf in terms of the conditional ones. Namely, Prove this. Generalize this to the n-dimensional case, 6.8 In the Ising model the expected magnetizationper spin is given by where KT is the Boltzmann distribution at temperature T. Estimate M(T), for example via the Swendsen-Wang algorithm, for various values of T E [0,5], and observe that the graph of M(T) changes sharply around the critical temperature T z 2.61. Take n = 20 and use periodic boundaries. 6.9 Run Peter Young's Java applet in http://bartok.ucsc.edu/peter/java/ising/keep/ising.html to gain a better understanding of how the king model works. 6.10 AsinExample6.6,letZ* = {x : ~~="=,, = m, z, E (0,. . .,m}, i = I,. . . ,n}. Show that this set has (m:!F1) elements. 6.1 1 In a simple model for a closed queueing network with n queues and m customers, it is assumed that the service times are independent and exponentially distributed, say with rate /I,% for queue i, i = 1, . . . , n. After completing service at queue i, the customer moves to queue j with probability pZ3. The {pv} are the so-called routingprobabilities. Figure 6.16 A closed queueing network. It can be shown (see, for example, [ 121) that the stationary distribution of the number of customers in the queues is of product form (6. lo), with fi being the pdf of the G( 1 - yi/pi) distribution; thus, ji(zi) 0: (yi/pi)=i. Here the {yi} are constants that are obtained from the following set offrow balance equations: (6.25) which has a one-dimensional solution space. Without loss of generality, y1 can be set to 1 to obtain a unique solution. 196 MARKOV CHAIN MONTE CARL0 Consider now the specific case of the network depicted in Figure 6.16, with n = 3 queues. Suppose the service rates are p1 = 2, p2 = 1, and p3 = 1. The routing probabilities are given in the figure. a) Show that a solution to (6.25) is (y1, y2, y3) = (1,10/21,4/7). b) For m = 50 determine the exact normalization constant C. c) Implement the procedure of Example 6.6 to estimate C via MCMC and compare Let XI,. . . , X, be a random sample from the N(p, 02) distribution. Consider the the estimate form = 50 with the exact value. 6.12 following Bayesian model: 0 f(p,u2) = l/2; 0 (xt I p, g) - N(p, a2), i = 1,. . . n independently. Note that the prior for (p, 02) is improper. That is, it is not a pdf in itself, but by obstinately applying Bayes' formula, it does yield a proper posterior pdf. In some sense it conveys the least amount of information about p and 02. Let x = (51, . . . , 2,) represent the data. The posterior pdf is given by We wish to sample from this distribution via the Gibbs sampler. a) Show that (p I u2, x) N N(Zl n2/n), where 3 is the sample mean. b) Prove that (6.26) where V, = Cr(xi - ~)~/n is the classical sample variance for known p. In other words, (1/02 I p, x) - Garnrna(n,/2, n.V,/2). c) Implement a Gibbs sampler to sample from the posterior distribution, taking 'n = 100. Run the sampler for lo5 iterations. Plot the histograms of j(p 1 x) and f(02 I x) and find the sample means of these posteriors. Compare them with the classical estimates. d) Show that the true posterior pdf of p given the data is given by fb I x) 0: ((P - + v) -n/2 1 where V = c,(zi - Z)2/n. (Hint: in order to evaluate the integral f(P I x) = Lrn IiP, 2 I x) do2 write it first as (2~)-4~ Jr tnI2-' exp( - t c) dt, where c = n V,, by applying the change of variable t = l/a2. Show that the latter integral is proportional to c-"/~. Finally, apply the decomposition V, = (3 - p)2 + V.) 6.13 Suppose f(O I x) is the posterior pdf for some Bayesian estimation problem. For example, 0 could represent the parameters of a regression model based on the data x. An important use for the posterior pdf is to make predictions about the distribution of other PROBLEMS 197 random variables. For example, suppose the pdf of some random variable Y depends on 0 via the conditional pdf f(y 10). Thepredictivepdfof Y given x is defined as which can be viewed as the expectation of f(y I 0) under the posterior pdf. Therefore, we can use Monte Carlo simulation to approximate f(y I x) as where the sample {Otl i = 1,. . . , N} is obtained from f(O I x); for example, via MCMC. As a concrete application, suppose that the independent measurement data: -0.4326, -1.6656,0.1253,0.2877, -1.1465 come from some N(p, 02) distribution. De- fine 0 = (p, g2). Let Y - N(p, c2) be a new measurement. Estimate and draw the predictive pdf f(y I x) from a sample 01,. . . , 0N obtained via the Gibbs sampler of Prob- lem 6.12. Take N = 10,000. Compare this with the “common-sense” Gaussian pdf with expectation Z (sample mean) and variance s2 (sample variance). 6.14 In the zero-inflated Poisson (ZIP) model, random data XI, . . . , X, are assumed to be of the form X, = R, K, where the { yZ} have a Poi(A) distribution and the { Ri} have a Ber(p) distribution, all independent of each other. Given an outcome x = (z1, . . . , zn), the objective is to estimate both A and p. Consider the following hierarchical Bayes model: 0 p - U(0,l) 0 (A I p) - Garnrna(a, b) 0 (T, I p, A) - Ber(p) independently 0 (xi I r, A, p) - Poi(A T,) independently (from the model above), where r = (TI, . . . , T,) and a and b are known parameters. It follows that (prior for p), (prior for A), (from the model above), We wish to sample from the posterior pdf f(X, p, r I x) using the Gibbs sampler. Show that 1. (Alp,r,x) -Garnma(o+C,z,, b+C,r,). 2. (p I A, r, x) - Beta(1 + c, r,, n + 1 - c, T,). 3. (Ta I A,P?X) - Ber (pc-*+Yp;I{,,=a)). Generate a random sample of size n = 100 for the ZIP model using parameters p = 0.3 and X = 2. Implement the Gibbs sampler, generate a large (dependent) sample from the pos- terior distribution and use this to construct 95% Bayesian CIS for p and X using the data in b). Compare these with the true values. 6.15 * Show that p in (6.15) satisfies the local balance equations p(x, y) R[(x, Y), (XI, Y’)] = ~(x’, Y’) R[(x‘, (X7 Y)] 198 MARKOV CHAIN MONTE CARL0 Thus pis stationary with respect to R, that is, p,R = p. Show that respect to Q. Show, finally, that p is stationary with respect to P = QR. 6.16 * This is to show that the systematic Gibbs sampler is a special case of the generalized Markov sampler. Take 9 to be the set of indices { 1, . . . , n}, and define for the Q-step is also stationary with 1 ify’=y+Iory’= 1,y=n QX(y,y’) = { 0 otherwise. Let the set of possible transitions 9(x, y) be the set of vectors {(XI, y)} such that all coordinates of x’ are the same as those of x except for possibly the y-th coordinate. a) Show that the stationary distribution of Qx is qx(y) = l/n, for y = 1,. . . , n. b) Show that (z,Y)-Yx,Y) c) Compare with Algorithm 6.4.1. 6.17 * Prove that the Metropolis-Hastings algorithm is a special case of the general- ized Markov sampler. (Hint: let the auxiliary set 9 be a copy of the target set x, let Qx correspond to the transition function of the Metropolis-Hastings algorithm (that is, Qx(., y) = q(x, y)), and define 9(x, y) = { (x, y), (y, x)}. Use arguments similar to those for the Markov jump sampler (see (6.20)) to complete the proof.) 6.18 Barker’s and Hastings’ MCMC algorithms differ from the symmetric Metropolis sampleronly in thatthey define theacceptance ratioa(x, y) toberespectively f(y)/(f(x)+ f(y)) and s(x, y)/(l + l/~(x, y)) instead of min{f(y)/f(x), 1). Here ~(x, y) is defined in (6.6) and s is any symmetric function such that 0 < a(x, y) < 1. Show that both are special cases of the generalized Markov sampler. (Hint: take 9 = X.) 6.19 in Example 6.13. How many solutions can you find? 6.20 TSP in Example 6.12. Run the algorithm on some test problems in Implement the simulated annealing algorithm for the n-queens problem suggested Implement the Metropolis-Hastings based simulated annealing algorithm for the http://www.iwr.uni-heidelberg.de/groups/comopt/software/TSPLIB95/ 6.21 mize the function Write a simulated annealing algorithm based on the random walk sampler to maxi- sin’(10z) + cos5(5z + 1) S(X) = s*-z+1 Use a N(z, u2) proposal function, given the current state 2. Start with z = 0. Plot the current best function value against the number of evaluations of S for various values of CT and various annealing schedules. Repeat the experiments several times to assess what works best. Further Reading MCMC is one of the principal tools of statistical computing and Bayesian analysis. A com- prehensive discussion of MCMC techniques can be found in [ 191, and practical applications REFERENCES 199 are discussed in [7]. For more details on the use of MCMC in Bayesian analysis, we refer to [5]. A classical reference on simulated annealing is [I]. More general global search algorithms may be found in [25]. An influential paper on stationarity detection in Markov chains, which is closely related to perfect sampling, is [3]. REFERENCES 1. E. H. L. Aarts and J. H. M. Korst. Simulated Annealing and Boltzmann Machines. John Wiley & Sons, Chichester, 1989. 2. D. J. Aldous and J. Fill. ReversibleMarkov Chains andRandom Walks on Graphs. In preparation. http://ww.stat.berkeley.edu /users/aldous/book.htrn1,2007. 3. S. Asmussen, P. W. Glynn, and H. Thorisson. Stationary detection in the initial transient problem. ACM Transactions on Modeling and Computer Simulation, 2(2): 130-1 57, 1992. 4. S. Baumert, A. Ghate, S Kiatsupaibul, Y. Shen, R. L. Smith, and Z. B. Zabinsky. A discrete hit-and-run algorithm for generating multivariate distributions over arbitrary finite subsets of a lattice. Technical report, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, 2006. 5. A. Gelman, J. B. Carlin. H. S. Stem, and D. B. Rubin. Bayesian Data Analysis. Chapman & Hall, New York, 2nd edition, 2003. 6. S. Geman and D. Geman. Stochastic relaxation, Gibbs distribution and the Bayesian restoration 7. W.R. Gilks, S. Richardson, and D. J. Spiegelhalter. Markov Chain Monte Carlo in Practice. 8. P. J. Green. Reversible jump Markov chain Monte Carlo computation and Bayesian model 9. J. Hammersley and M. Clifford. Markov fields on finite graphs and lattices. Unpublished 10. W. K. Hastings. Monte Carlo sampling methods using Markov chains and their applications. 11. J. M. Keith, D. P. Kroese, and D. Bryant. A generalized Markov chain sampler. Methodology 12. F. P. Kelly. Reversibility and Stochastic Networks. Wiley, Chichester, 1979. 13. J. S. Liu. Monte Carlo Strategies in Scientifi c Computing. Springer-Verlag, New York, 2001. 14. L. Lovasz. Hit-and-run mixes fast. Mathematical Programming, 86:443461, 1999. 15. L. Lovasz and S. S. Vempala. Hit-and-run is fast and fun. Technical report, Microsoft Research, 16. L. Lovisz and S. Vempala. Hit-and-run from a comer. SlAMJournalon Computing, 35(4):985- 17. M. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller. Equations of state calculations by fast computing machines. Journal of Chemical Physics, 21 :1087-1092, 1953. 18. 1. G. Propp and D. B. Wilson. Exact sampling with coupled Markov chains and applications to 19. C. P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer, New York, 2nd edition, of images. IEEE Transactions on PAM, 6:721-741, 1984. Chapman & Hall, New York, 1996. determination. Biometrika, 82(4):7 11-732, 1995. manuscript, 1970. Biometrika, 57:92-109, 1970. and Computing in Applied Probability, 6( 1):29-53, 2004. SMS-TR, 2003. 1005,2006. statistical mechanics. Random Structures and Algorithms, 1 & 2:223-252, 1996. 2004. [...]... Hessians, etc.) of the response function ! u) ( with respect to parameter vector u, and it is based on the score function and the Fisher inforSimulation and the Monte Carlo Method, Second Edition By R.Y Rubinstein and D P Kroese Copyright @ 2007 John Wiley & Sons, Inc 201 202 SENSITIVITY ANALYSIS AND MONTE CARL0 OPTIMIZATION mation It provides guidance for design and operational decisions and plays an important... value functions in the deterministic program (PO)by their sample average equivalents 216 SENSITIVITY ANALYSIS AND MONTE CARL0 OPTIMIZATION and then solve the latter by standard mathematical programming techniques The resultant optimal solution provides an estimator of the corresponding true optimal one for the original program (PO) If not stated otherwise, we shall consider here the unconstrained program... different types of queueing and inventory models, the reader is referred to [ 161 7.2 THE SCORE FUNCTION METHOD FOR SENSITIVITY ANALYSIS OF DESS In this section we introduce the celebratedscorefunction (SF)methodfor sensitivity analysis of DESS The goal of the SF method is to estimate the gradient and higher derivatives of !(u) with respect to the distributional parameter vector u, where the expected performance... show how the stochastic counterpart method can approximate quite efficiently the true unknown optimal solution using a single simulation Our results are based on [15, 17, 181 , where of the program (PO) theoretical foundations of the stochastic counterpart method are established It is interesting to note that Geyer and Thompson [2] independently discovered the stochastic counterpart method in 1995 They... comprehensive study of both DESS and DEDS the reader is referred to [ 1 I], [ 161, and [201 Because of their complexity, the performance evaluation of discrete-event systems is usually studied by simulation, and it is often associated with the estimation of the performance or response function !(u) = E,[H(X)], where the distribution of the sample performance H(X) depends on the control or reference parameter... with respect to the parameter vector u of the pdf f ( x ;u), while in the latter case we are interested in sensitivities of the expected performance (7.2) with respect to the parameter vector u in the sample performance H ( x ;u) As an example, consider a G I / G / l queue In the first case u might be the vector of the interarrival and service rates, while in the second case u might be the buffer size... Tables A 1 and 7.1 Note that in Table 7.2 we change one parameter only, which is denoted by u and is changed to v The values of E,, [ W 2 ( X u, are calculated via (A.9) ; v)] 210 SENSITIVITY ANALYSIS AND MONTE CARL0 OPTIMIZATION and (7.23) In particular, we first reparameterize the distribution in terms of (A.9) with 0 = $ ( u )and 7 = $(v), and then calculate (7.24) At the end, we substitute u and 71... seen that the estimator e^(u; performs reasonably well for 6 not exceeding 0.1, w) that is, when the relative perturbation in u is within 10% For larger relative perturbations, the term E,[W2]“blows up” the variance of the estimators Similar results also hold for the derivatives of [ ( u ) The above (negative) results on the unstable behavior of the likelihood ratio W and the rapid decrease of the trust... of the following holds true: 1 It is too expensive to store long samples XI, , X N and the associated seX2, quences {e^,(u)> 212 SENSITIVITY ANALYSIS AND MONTE CARL0 OPTIMIZATION 2 The sample performance, &(u),cannot be computed simultaneously for different values of u However, we are allowed to set the control vector, u, at any desired ) value u ( ~and then compute the random variables &(u(" )and. .. that is, the feasible set Y coincides with the whole space, then this projection is the identity mapping and can be omitted from (7.29) It is readily seen that (7.29) represents a gradient descentprocedure in which the exact of gradients are replaced by their estimates Indeed, if the exact value V!(U(~)) the gradient was available, then -V!(U(~)) would give the direction of steepest descent at the point . Hessians, etc.) of the response function !( u) with respect to parameter vector u, and it is based on the score function and the Fisher infor- Simulation and the Monte Carlo Method, Second. m dependent copies of the Markov chain, starting from each of the states 1, . . . , m and using the same random numbers { Uz} - similar to the CRV method. Then, if two paths coincide,. Richardson, and D. J. Spiegelhalter. Markov Chain Monte Carlo in Practice. 8. P. J. Green. Reversible jump Markov chain Monte Carlo computation and Bayesian model 9. J. Hammersley and M. Clifford.