PROBABILITY AND MATHEMATICAL STATISTICS Prasanna Sahoo Department of Mathematics University of Louisville Louisville, KY 40292 USA v THIS BOOK IS DEDICATED TO AMIT SADHNA MY PARENTS, TEACHERS AND STUDENTS vi vii Copyright c 2008. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the author. viii ix PREFACE This book is both a tutorial and a textbook. This book presents an introduc- tion to probability and mathematical statistics and it is intended for students already having some elementary mathematical background. It is intended for a one-year junior or senior level undergraduate or beginning graduate level course in probability theory and mathematical statistics. The book contains more material than normally would be taught in a one-year course. This should give the teacher flexibility with respect to the selection of the content and level at which the book is to be used. This book is based on over 15 years of lectures in senior level calculus based courses in probability theory and mathematical statistics at the University of Louisville. Probability theory and mathematical statistics are difficult subjects both for students to comprehend and teachers to explain. Despite the publication of a great many textbooks in this field, each one intended to provide an im- provement over the previous textbooks, this subject is still difficult to com- prehend. A good set of examples makes these subjects easy to understand. For this reason alone I have included more than 350 completely worked out examples and over 165 illustrations. I give a rigorous treatment of the fun- damentals of probability and statistics using mostly calculus. I have given great attention to the clarity of the presentation of the materials. In the text, theoretical results are presented as theorems, propositions or lemmas, of which as a rule rigorous proofs are given. For the few exceptions to this rule references are given to indicate where details can be found. This book contains over 450 problems of varying degrees of difficulty to help students master their problem solving skill. In many existing textbooks, the examples following the explanation of a topic are too few in numb er or too simple to obtain a through grasp of the principles involved. Often, in many books, examples are presented in abbreviated form that leaves out much material between steps, and requires that students derive the omitted materials themselves. As a result, students find examples difficult to understand. Moreover, in some textbooks, examples x are often worded in a confusing manner. They do not state the problem and then present the solution. Instead, they pass through a general discussion, never revealing what is to be solved for. In this book, I give many examples to illustrate each topic. Often we provide illustrations to promote a better understanding of the topic. All examples in this book are formulated as questions and clear and concise answers are provided in step-by-step detail. There are several good books on these subjects and perhaps there is no need to bring a new one to the market. So for several years, this was circulated as a series of typeset lecture notes among my students who were preparing for the examination 110 of the Actuarial Society of America. Many of my students encouraged me to formally write it as a book. Actuarial students will benefit greatly from this book. The book is written in simple English; this might be an advantage to students whose native language is not English. I cannot claim that all the materials I have written in this book are mine. I have learned the subject from many excellent books, such as Introduction to Mathematical Statistics by Hogg and Craig, and An Introduction to Prob- ability Theory and Its Applications by Feller. In fact, these books have had a profound impact on me, and my explanations are influenced greatly by these textb ooks. If there are some similarities, then it is due to the fact that I could not make improvements on the original explanations. I am very thankful to the authors of these great textbooks. I am also thankful to the Actuarial Society of America for letting me use their test problems. I thank all my students in my probability theory and mathematical statistics courses from 1988 to 2005 who helped me in many ways to make this book possible in the present form. Lastly, if it weren’t for the infinite patience of my wife, Sadhna, this book would never get out of the hard drive of my computer. The author on a Macintosh computer using T E X, the typesetting system designed by Donald Knuth, typeset the entire book. The figures were gener- ated by the author using MATHEMATICA, a system for doing mathematics designed by Wolfram Research, and MAPLE, a system for doing mathemat- ics designed by Maplesoft. The author is very thankful to the University of Louisville for providing many internal financial grants while this book was under preparation. Prasanna Sahoo, Louisville xi xii TABLE OF CONTENTS 1. Probability of Events . . . . . . . . . . . . . . . . . . . 1 1.1. Introduction 1.2. Counting Techniques 1.3. Probability Measure 1.4. Some Properties of the Probability Measure 1.5. Review Exercises 2. Conditional Probability and Bayes’ Theorem . . . . . . . 27 2.1. Conditional Probability 2.2. Bayes’ Theorem 2.3. Review Exercises 3. Random Variables and Distribution Functions . . . . . . . 45 3.1. Introduction 3.2. Distribution Functions of Discrete Variables 3.3. Distribution Functions of Continuous Variables 3.4. Percentile for Continuous Random Variables 3.5. Review Exercises 4. Moments of Random Variables and Chebychev Inequality . 73 4.1. Moments of Random Variables 4.2. Expected Value of Random Variables 4.3. Variance of Random Variables 4.4. Chebychev Inequality 4.5. Moment Generating Functions 4.6. Review Exercises xiii 5. Some Special Discrete Distributions . . . . . . . . . . . 107 5.1. Bernoulli Distribution 5.2. Binomial Distribution 5.3. Geometric Distribution 5.4. Negative Binomial Distribution 5.5. Hypergeometric Distribution 5.6. Poisson Distribution 5.7. Riemann Zeta Distribution 5.8. Review Exercises 6. Some Special Continuous Distributions . . . . . . . . . 141 6.1. Uniform Distribution 6.2. Gamma Distribution 6.3. Beta Distribution 6.4. Normal Distribution 6.5. Lognormal Distribution 6.6. Inverse Gaussian Distribution 6.7. Logistic Distribution 6.8. Review Exercises 7. Two Random Variables . . . . . . . . . . . . . . . . . 185 7.1. Bivariate Discrete Random Variables 7.2. Bivariate Continuous Random Variables 7.3. Conditional Distributions 7.4. Independence of Random Variables 7.5. Review Exercises 8. Product Moments of Bivariate Random Variables . . . . 213 8.1. Covariance of Bivariate Random Variables 8.2. Independence of Random Variables 8.3. Variance of the Linear Combination of Random Variables 8.4. Correlation and Independence 8.5. Moment Generating Functions 8.6. Review Exercises [...]... Review Exercises Probability and Mathematical Statistics 1 Chapter 1 PROBABILITY OF EVENTS 1.1 Introduction During his lecture in 1929, Bertrand Russel said, Probability is the most important concept in modern science, especially as nobody has the slightest notion what it means.” Most people have some vague ideas about what probability of an event means The interpretation of the word probability involves... Thus, from the above, we get P (B) ≥ P (A) and the proof is complete Theorem 1.10 If A is any event in S, then 0 ≤ P (A) ≤ 1 Probability and Mathematical Statistics 19 Proof: Follows from axioms (P1) and (P2) and Theorem 1.8 Theorem 1.10 If A and B are any two events, then P (A ∪ B) = P (A) + P (B) − P (A ∩ B) Proof: It is easy to see that A ∪ B = A ∪ (Ac ∩ B) and A ∩ (Ac ∩ B) = ∅ S A B Hence by (P3),... objectively by means of probability statements Probability theory is used to evaluate the reliability of conclusions and inferences based on data Thus, probability theory is fundamental to mathematical statistics For an event A of a discrete sample space S, the probability of A can be computed by using the formula P (A) = N (A) N (S) where N (A) denotes the number of elements of A and N (S) denotes the... n2 , , nm 1 2 is known as the multinomial theorem and it generalizes the binomial theorem The sum is taken over all positive integers n1 , n2 , , nm such that n1 + n2 + · · · + nm = n, and n n1 , n2 , , nm = n! n1 ! n2 !, , nm ! Probability and Mathematical Statistics 11 This coefficient is known as the multinomial coefficient 1.3 Probability Measure A random experiment is an experiment whose outcomes... probability out of many In fact, we will study probability theory based on the theory developed by the late Kolmogorov There are many applications of probability theory We are studying probability theory because we would like to study mathematical statistics Statistics is concerned with the development of methods and their applications for collecting, analyzing and interpreting quantitative data in such... get 0.05 ≤ P (A ∩ B) (1.4) Probability and Mathematical Statistics 21 From (1.3) and (1.4), we get 0.05 ≤ P (A ∩ B) ≤ 0.25 Example 1.20 Let A and B be events in a sample space S such that P (A) = 1 = P (B) and P (Ac ∩ B c ) = 1 Find P (A ∪ B c ) 2 3 Answer: Notice that A ∪ B c = A ∪ (Ac ∩ B c ) Hence, P (A ∪ B c ) = P (A) + P (Ac ∩ B c ) 1 1 = + 2 3 5 = 6 Theorem 1.11 If A1 and A2 are two events such... we have P (∅) = 0 Probability of Events 14 and the proof of the theorem is complete This theorem says that the probability of an impossible event is zero Note that if the probability of an event is zero, that does not mean the event is empty (or impossible) There are random experiments in which there are infinitely many events each with probability 0 Similarly, if A is an event with probability 1, then... + i=1 n = i=n+1 ∞ i=n+1 P (Ai ) + 0 i=1 n = P (Ai ) i=1 and the proof of the theorem is now complete P (Ai ) P (∅) Probability and Mathematical Statistics 15 When n = 2, the above theorem yields P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) where A1 and A2 are disjoint (or mutually exclusive) events In the following theorem, we give a method for computing probability of an event A by knowing the probabilities of... basic counting techniques They are multiplication rule, permutation and combination 1.2.1 Multiplication Rule If E1 is an experiment with n1 outcomes and E2 is an experiment with n2 possible outcomes, then the experiment which consists of performing E1 first and then E2 consists of n1 n2 possible outcomes Probability and Mathematical Statistics 3 Example 1.1 Find the possible number of outcomes in a... space of a random experiment A probability measure P : F → [0, 1] is a set function which assigns real numbers to the various events of S satisfying (P1) P (A) ≥ 0 for all event A ∈ F, (P2) P (S) = 1, Probability and Mathematical Statistics (P3) P if ∞ Ak k=1 A1 , A2 , A3 , , = ∞ 13 P (Ak ) k=1 Ak , are mutually disjoint events of S Any set function with the above three properties is a probability . courses in probability theory and mathematical statistics at the University of Louisville. Probability theory and mathematical statistics are difficult subjects. 669 Probability and Mathematical Statistics 1 Chapter 1 PROBABILITY OF EVENTS 1.1. Introduction During his lecture in 1929, Bertrand Russel said, “Probability