CHAPTER 5 Specific Random Variables 5.1. Binomial We will b e gin with mean and variance of the binomial variable, i.e., the number of successes in n independent repetitions of a Be rnoulli trial (3.7.1). The binomial variable has the two parameters n and p. Let us look first at the case n = 1, in which the binomial variable is also called indicator variable: If the event A has probability p, then its complement A  has the probability q = 1 − p. The indicator variable of A, which assumes the value 1 if A occurs, and 0 if it doesn’t, has expected value p and variance pq. For the binomial variable with n observations, which is the sum of n independent indicator variables, the expected value (mean) is np and the variance is npq. 139 140 5. SPECIFIC RANDOM VARIABLES Problem 79. The random variable x assumes the value a with probability p and the value b with probability q = 1 −p. Show that var[x] = pq(a −b) 2 . Answer. E[x] = pa + qb; var[x] = E[x 2 ] − (E[x]) 2 = pa 2 + qb 2 − (pa + qb) 2 = (p − p 2 )a 2 − 2pqab + (q − q 2 )b 2 = pq(a − b) 2 . For this last equality we need p −p 2 = p(1 −p) = pq.  The Negative Binomial Variable is, like the binomial variable, derived from the Bernoulli experiment; but one reverses the question. Instead of asking how many successes one gets in a given number of trials, one asks, how m any trials one must make to get a given number of successes, say, r successes. First look at r = 1. Let t denote the number of the trial at which the first success occurs. Then (5.1.1) Pr[t=n] = pq n−1 (n = 1, 2, . . .). This is called the geometric probability. Is the probability derived in this way σ-additive? The sum of a geometrically declining sequence is easily computed: 1 + q + q 2 + q 3 + ··· = s Now multiply by q:(5.1.2) q + q 2 + q 3 + ··· = qs Now subtract and write 1 −q = p:(5.1.3) 1 = ps(5.1.4) 5.1. BINOMIAL 141 Equation (5.1.4) means 1 = p + pq + pq 2 + ···, i.e., the sum of all probabilities is indeed 1. Now what is the expected value of a geometric variable? Use definition of ex- pected value of a discrete variable: E[t] = p  ∞ k=1 kq k−1 . To evaluate the infinite sum, solve (5.1.4) for s: s = 1 p or 1 + q + q 2 + q 3 + q 4 ··· = ∞  k=0 q k = 1 1 − q (5.1.5) and differentiate both sides with respect to q: 1 + 2q + 3q 2 + 4q 3 + ··· = ∞  k=1 kq k−1 = 1 (1 − q) 2 = 1 p 2 .(5.1.6) The expected value of the geometric variable is therefore E[t] = p p 2 = 1 p . Problem 80. Assume t is a geometric random variable with parameter p, i.e., it has the values k = 1, 2, . . . with probabilities (5.1.7) p t (k) = pq k−1 , where q = 1 −p. The geometric variable denotes the number of times one has to perform a Bernoulli experiment with success probability p to get the first success. 142 5. SPECIFIC RANDOM VARIABLES • a. 1 point Given a positive integer n. What is Pr[t>n]? (Easy with a simple trick!) Answer. t>n means, the first n trials must result in failures, i.e., Pr[t>n] = q n . Since {t > n} = {t = n + 1} ∪{t = n + 2} ∪···, on e can also get the same result in a more tedious way: It is pq n + pq n+1 + pq n+2 + ··· = s, say. Therefore qs = pq n+1 + pq n+2 + ···, and (1 −q)s = pq n ; since p = 1 − q, it follows s = q n .  • b. 2 points Let m and n be two positive integers with m < n. Show that Pr[t=n|t>m] = Pr[t=n − m]. Answer. Pr[t=n|t>m] = Pr[t=n] Pr[t>m] = pq n−1 q m = pq n−m−1 = Pr[t=n − m].  • c. 1 point Why is this property called the memory-less property of the geometric random variable? Answer. If you have already waited for m periods wi thou t succe ss, the probability that success will come in the nth period is the same as the probability that it comes in n − m periods if you start now. Obvious if you remember that geometric random variable is time you have to wait until 1st success in Bernoulli trial.  Problem 81. t is a geometric random variable as in the preceding problem. In order to compute var[t] it is most convenient to make a detour via E[t(t − 1)]. Here are t he steps: 5.1. BINOMIAL 143 • a. Express E[t(t − 1)] as an infinite sum. Answer. Just write it down according to the definition of expected values:  ∞ k=0 k(k − 1)pq k−1 =  ∞ k=2 k(k −1)pq k−1 .  • b. Derive the formula (5.1.8) ∞  k=2 k(k −1)q k−2 = 2 (1 − q) 3 by the same trick by which we derived a similar formula in class. Note that the sum starts at k = 2. Answer. This is just a second time differentiating the geometric series, i.e., first time differ- entiating (5.1.6).  • c. Use a. and b. to derive (5.1.9) E[t(t − 1)] = 2q p 2 Answer. (5.1.10) ∞  k=2 k(k −1)pq k−1 = pq ∞  k=2 k(k −1)q k−2 = pq 2 (1 − q) 3 = 2q p 2 .  144 5. SPECIFIC RANDOM VARIABLES • d. Use c. and the fact that E[t] = 1/p to derive (5.1.11) var[t] = q p 2 . Answer. (5.1.12) var[t] = E[t 2 ] − (E[t]) 2 = E[t(t − 1)] + E[t] − (E[t]) 2 = 2q p 2 + 1 p − 1 p 2 = q p 2 .  Now let us look at the negative binomial with arbitrary r. What is the probability that it takes n trials to get r successes? (That means, with n−1 trials we did not yet have r successes.) The probability that the nth trial is a success is p. The probability that there are r − 1 successes in the first n − 1 trials is  n−1 r−1  p r−1 q n−r . Multiply those to get: (5.1.13) Pr[t=n] =  n − 1 r −1  p r q n−r . This is the negative binomial, also called the Pascal probability distribution with parameters r and p. 5.1. BINOMIAL 145 One easily gets the mean and variance, because due to the memory-less property it is the sum of r independent geometric variables: (5.1.14) E[t] = r p var[t] = rq p 2 Some authors define the negative binomial as the number of failures before the rth success. Their formulas will look slightly different than ours. Problem 82. 3 points A fair coin is flipped until heads appear 10 times, and x is the number of times tails appear before the 10th appearance of heads. Show that the expected value E[x] = 10. Answer. Let t be the number of the throw which gives the 10th head. t is a negative binomial with r = 10 and p = 1/2, therefore E[t] = 20. Since x = t − 10, it follows E[x] = 10.  Problem 83. (Banach’s match-box problem) (Not eligible for in-class exams) There are two restaurants in town serving hamburgers. In the morning each of them obtains a shipment of n raw hamburgers. Every time someone in that town wants to eat a hamburger, he or she selects one of the two restaurants at random. What is the probability that the (n + k)th customer will have to be turned away because the restaurant selected has run out of hamburgers? Answer. For each restaurant it is the negative binomial probability distribution in disguise: if a restaurant runs out of hamburgers this is like having n successes in n + k tries. 146 5. SPECIFIC RANDOM VARIABLES But one can also reason it out: Assume one of the restaurantes must turn customers away after the n + kth customer. Write down all the n + k decisions made: write a 1 if the customer goes to the first restaurant, and a 2 if he goes to the second. I.e., write down n + k ones and twos. Under what conditio ns will such a sequence result in the n +kth move eating the last hamburgerthe first restaurant? Exact ly if it has n ones and k twos, a n + kth move is a one. As in the reasoning for the negative binomial probability distribution, there are  n+k−1 n−1  possibilitie s, each of which has probability 2 −n−k . Emptying the second restaurant has the same probability. Together the probability is therefore  n+k−1 n−1  2 1−n−k .  5.2. The Hypergeometric Probability Distribution Until now we had independent events, such as, repeated throwing of coins or dice, sampling with replacement from finite populations, ar sampling from infinite populations. If we sample without replacem ent from a finite population, the prob- ability of the second element of the sample depends on what the first element was. Here the hypergeometric probability distribution applies. Assume we have an urn with w white and n −w black balls in it, and we take a sample of m balls. What is the probability that y of them are white? We are not interested in the order in which these balls are taken out; we may therefore assume that they are taken out simultaneously, therefore the set U of outcomes is the set of subsets containing m of the n balls. The total number of such subsets is  n m  . How many of them have y white balls in them? Imagine you first 5.2. THE HYPERGEOMETRIC PROBABILITY DISTRIBUTION 147 pick y white balls from the set of all white balls (there are  w y  possibilities to do that), and then you pick m − y black balls from the set of all black balls, which can be done in  n−w m−y  different ways. Every union of such a set of white balls with a set of black balls gives a set of m elements with exactly y white balls, as desired. There are therefore  w y  n−w m−y  different such sets, and the probability of picking such a set is (5.2.1) Pr[Sample of m elements has exactly y white balls] =  w y  n−w m−y   n m  . Problem 84. You have an urn with w white and n−w black balls in it, and you take a sample of m balls with replacement, i.e., after pulling each ball out you put it back in before you pull out the next ball. What is the probability that y of these balls are white? I.e., we are asking here for the counterpart of formula (5.2.1) if sampling is done with replacement. Answer. (5.2.2)  w n  y  n − w n  m−y  m y   148 5. SPECIFIC RANDOM VARIABLES Without proof we will state here that the expected value of y, the number of white balls in the sample, is E[y] = m w n , which is the same as if one would select the balls with replacement. Also without proof, the variance of y is (5.2.3) var[y] = m w n (n − w) n (n − m) (n − 1) . This is smaller than the variance if one would choose with replacement, which is represented by the above formula without the last term n−m n−1 . This last term is called the finite population correction. More about all this is in [Lar82, p. 176–183]. 5.3. The Poisson Distribution The Poisson distribution counts the number of events in a given time interval. This number has the Poisson distribution if each event is the cumulative result of a large number of independent possibilities, e ach of which has only a small chance of occurring (law of rare events). The expected number of occurrences is proportional to time with a proportionality factor λ, and in a short time span only zero or one event can occur, i.e., for infinitesimal time intervals it becomes a Bernoulli trial. Approximate it by dividing the time from 0 to t into n intervals of length t n ; then the occurrences are approximately n independent Bernoulli trials with probability of [...]... the standard Normal distribution Problem 104 2 points Compare [Gre97, p 68]: Assume x ∼ N (3, 4) (mean is 3 and variance 4) Determine with the help of a table of the Standard Normal Distribution function Pr[2 575 1 − Ψ(0 .3) 5.9 The Chi-Square... independent random variables distributed uniformly over the interval [0, 1] Let u be their minimum u = min(x, y) (i.e., u takes the value of x when x is smaller, and the value of y when y is smaller), and v = max(x, y) • a 2 points Given two numbers q and r between 0 and 1 Draw the events u≤q and v≤r into the unit square and compute their probabilities • b 2 points Compute the density functions fu (u) and fv... area between 0 and all positive values; in this case it is 0 .34 13 + 0.1915 The moment generating function of a standard normal z ∼ N (0, 1) is the following integral: +∞ (5.8.2) mz (t) = E[etz ] = −∞ 1 −z2 etz √ e 2 dz 2π To solve this integral, complete the square in the exponent: (5.8 .3) tz − z2 t2 1 = − (z − t)2 ; 2 2 2 168 5 SPECIFIC RANDOM VARIABLES t2 2 Note that the first summand, t2 , no longer... $0 Therefore (5.8.12) Pr[y>$ 630 0 + m|y>$ 630 0] = Pr[y>$ 630 0 + m] 1 = , Pr[y>$ 630 0] 2 172 5 SPECIFIC RANDOM VARIABLES or Pr[y>$ 630 0 + m] = (1/2) Pr[y>$ 630 0] = (1/2) .38 21 = 1910 I.e., Pr[ y−6000 > 630 0−6000+m = 1000 1000 30 0 m 30 0 m + 1000 ] = 1910 For this we find in the table 1000 + 1000 = 0.875, therefore 30 0 + m = 875, 1000 or m = $575 • e 3 points Is the expected value of the consumer surplus of all...5 .3 THE POISSON DISTRIBUTION 149 success λt (This is an approximation since some of these intervals may have more n than one occurrence; but if the intervals become very short the probability of having two occurrences in the same interval becomes negligible.) In this discrete approximation, the probability to have k successes in time t is (5 .3. 1) (5 .3. 2) (5 .3. 3) n λt k λt (n−k) 1−... parameters λ and r The following definite integral, which is defined for all r > 0 and all λ > 0 is called the Gamma function: ∞ (5.5.6) λr tr−1 e−λt dt Γ(r) = 0 Although this integral cannot be expressed in a closed form, it is an important function in mathematics It is a well behaved function interpolating the factorials in the sense that Γ(r) = (r − 1)! Problem 95 Show that Γ(r) as defined in (5.5.6) is independent... toss a coin If the coin shows head, Herbert serves 154 5 SPECIFIC RANDOM VARIABLES the customer, and if it shows tails, Karl does Compute the probability that Herbert has to serve exactly one customer during the hour Hint: 1 1 1 (5 .3. 12) e = 1 + 1 + + + + ··· 2! 3! 4! • c For any integer k ≥ 0, compute the probability that Herbert has to serve exactly k customers during the hour Problem 89 3 points Compute... they threw a coin 2 13 times and counted the number of heads This number, they pretended, was the number of cars in the first half hour (5 .3. 8) • a 6 points Did the probability distribution of the number gained in this way differ from the distribution of actually counting the number of cars in the first half hour? Answer First a few definitions: x is the total number of occurrences in the interval [0, 1] y... is independent of λ, i.e., instead of (5.5.6) one can also use the simpler equation ∞ (5.5.7) tr−1 e−t dt Γ(r) = 0 160 5 SPECIFIC RANDOM VARIABLES Problem 96 3 points Show by partial integration that the Gamma function satisfies Γ(r + 1) = rΓ(r) Answer Start with ∞ (5.5.8) λr+1 tr e−λt dt Γ(r + 1) = 0 and integrate by parts: and v = rλr tr−1 : (5.5.9) uv dt with u = λe−λt and v = λr tr , therefore u . Random Variables 5.1. Binomial We will b e gin with mean and variance of the binomial variable, i.e., the number of successes in n independent repetitions of a Be rnoulli trial (3. 7.1). The binomial variable. hiding in the back, and whenever a customer steps up to the counter and rings the bell, they toss a coin. If the coin shows head, Herbert serves 154 5. SPECIFIC RANDOM VARIABLES the customer, and. − λt n  (n−k) (5 .3. 1) = 1 k! n(n − 1) ···(n − k + 1) n k (λt) k  1 − λt n  n  1 − λt n  −k (5 .3. 2) → (λt) k k! e −λt for n → ∞ while k remains constant(5 .3. 3) (5 .3. 3) is the limit because the se cond and