Class Notes in Statistics and Econometrics Hans G. Ehrbar Economics Department, University of Utah, 1645 Campus Center Drive, Salt Lake City UT 84112-9300, U.S.A. URL: www.econ.utah.edu/ehrbar/ecmet.pdf E-mail address: ehrbar@econ.utah.edu Abstract. This is an attempt to make a carefully argued set of class notes freely available. The source code for these notes can be downloaded from www.econ.utah.edu/ehrbar/ecmet-sources.zip Copyright Hans G. Ehrbar un- der the GNU Public License Contents Chapter 1. Preface xxiii Chapter 2. Probability Fields 1 2.1. The Concept of Probability 1 2.2. Events as Sets 12 2.3. The Axioms of Probability 20 2.4. Objective and Subjective Interpretation of Probability 26 2.5. Counting Rules 28 2.6. Relationships Involving Binomial Coefficients 32 2.7. Conditional Probability 34 2.8. Ratio of Probabilities as Strength of Evidence 45 iii iv CONTENTS 2.9. Bayes Theorem 48 2.10. Independence of Events 50 2.11. How to Plot Frequency Vectors and Probability Vectors 57 Chapter 3. Random Variables 63 3.1. Notation 63 3.2. Digression about Infinitesimals 64 3.3. Definition of a Random Variable 68 3.4. Characterization of Random Variables 70 3.5. Discrete and Absolutely Continuous Probability Measures 77 3.6. Transformation of a Scalar Density Function 79 3.7. Example: Binomial Variable 82 3.8. Pitfalls of Data Reduction: The Ecological Fallacy 85 3.9. Independence of Random Variables 87 3.10. Location Parameters and Dispersion Parameters of a Random Variable 89 3.11. Entropy 100 Chapter 4. Random Number Generation and Encryption 121 4.1. Alternatives to the Linear Congruential Random Generator 125 4.2. How to test random generators 126 4.3. The Wichmann Hill generator 128 CONTENTS v 4.4. Public Key Cryptology 133 Chapter 5. Specific Random Variables 139 5.1. Binomial 139 5.2. The Hypergeometric Probability Distribution 146 5.3. The Poisson Distribution 148 5.4. The Exponential Distribution 154 5.5. The Gamma Distribution 158 5.6. The Uniform Distribution 164 5.7. The Beta Distribution 165 5.8. The Normal Distribution 166 5.9. The Chi-Square Distribution 172 5.10. The Lognormal Dis tribution 174 5.11. The Cauchy Distribution 174 Chapter 6. Sufficient Statistics and their Distributions 179 6.1. Factorization Theorem for Sufficient Statistics 179 6.2. The Exponential Family of Probability Distributions 182 Chapter 7. Chebyshev Inequality, Weak Law of Large Numbers, and Central Limit Theorem 189 vi CONTENTS 7.1. Chebyshev Inequality 189 7.2. The Probability Limit and the Law of Large Numbers 192 7.3. Central Limit Theorem 195 Chapter 8. Vector Random Variables 199 8.1. Expected Value, Variances, Covariances 203 8.2. Marginal Probability Laws 210 8.3. Conditional Probability Distribution and Conditional Mean 212 8.4. The Multinomial Distribution 216 8.5. Independent Random Vectors 218 8.6. Conditional Expectation and Variance 221 8.7. Expected Values as Predictors 226 8.8. Transformation of Vector Random Variables 235 Chapter 9. Random Matrices 245 9.1. Linearity of Expected Values 245 9.2. Means and Variances of Quadratic Forms in Random Matrices 249 Chapter 10. The Multivariate Normal Probability Distribution 261 10.1. More About the Univariate Case 261 10.2. Definition of Multivariate Normal 264 CONTENTS vii 10.3. Special Case: Bivariate Normal 265 10.4. Multivariate Standard Normal in Higher Dimensions 284 10.5. Higher Moments of the Multivariate Standard Normal 290 10.6. The General Multivariate Normal 299 Chapter 11. The Regression Fallacy 309 Chapter 12. A Simple Example of Estimation 327 12.1. Sample Mean as Estimator of the Location Parameter 327 12.2. Intuition of the Maximum Likelihood Estimator 330 12.3. Variance Estimation and Degrees of Freedom 335 Chapter 13. Estimation Principles and Classification of Estimators 355 13.1. Asymptotic or Large-Sample Properties of Estimators 355 13.2. Small Sample Properties 359 13.3. Comparison Unbiasedness Consistency 362 13.4. The Cramer-Rao Lower Bound 369 13.5. Best Linear Unbiased Without Distribution Assumptions 386 13.6. Maximum Likelihood Estimation 390 13.7. Method of Moments Estimators 396 13.8. M-Estimators 396 viii CONTENTS 13.9. Sufficient Statistics and Estimation 397 13.10. The Likelihood Principle 405 13.11. Bayesian Inference 406 Chapter 14. Interval Estimation 411 Chapter 15. Hypothesis Testing 425 15.1. Duality between Significance Tests and Confidence Regions 433 15.2. The Neyman Pearson Lemma and Likelihood Ratio Tests 434 15.3. The Runs Test 440 15.4. Pearson’s Goodness of Fit Test. 447 15.5. Permutation Tests 453 15.6. The Wald, Likelihood Ratio, and Lagrange Multiplier Tests 465 Chapter 16. General Principles of Econometric Modelling 469 Chapter 17. Causality and Inference 473 Chapter 18. Mean-Variance Analysis in the Linear Model 481 18.1. Three Versions of the Linear Model 481 18.2. Ordinary Least Squares 484 18.3. The Coefficient of Determination 499 CONTENTS ix 18.4. The Adjusted R- Square 509 Chapter 19. Digression about Correlation Coefficients 513 19.1. A Unified Definition of Correlation Coefficients 513 19.2. Correlation Coefficients and the Associated Least Squares Problem 519 19.3. Canonical Correlations 521 19.4. Some Remarks about the Sample Partial Correlation Coefficients 524 Chapter 20. Numerical Methods for c omputing OLS Estimates 527 20.1. QR Decomposition 527 20.2. The LINPACK Impleme ntation of the QR Decomposition 530 Chapter 21. About Computers 535 21.1. General Strategy 535 21.2. The Emacs Editor 542 21.3. How to Enter and Exit SAS 544 21.4. How to Transfer SAS Data Sets Between Computers 545 21.5. Instructions for Statistics 5969, Hans Ehrbar’s Section 547 21.6. The Data Step in SAS 557 Chapter 22. Specific Datasets 563 x CONTENTS 22.1. Cobb Douglas Aggregate Production Function 563 22.2. Houthakker’s Data 580 22.3. Long Term Data about US Economy 592 22.4. Dougherty Data 594 22.5. Wage Data 595 Chapter 23. The Mean Squared Error as an Initial Criterion of Precision 629 23.1. Comparison of Two Vector Estimators 630 Chapter 24. Sampling Properties of the Least Squares Estimator 637 24.1. The Gauss Markov Theorem 639 24.2. Digression about Minimax Estimators 643 24.3. Miscellaneous Properties of the BLUE 645 24.4. Estimation of the Variance 666 24.5. Mallow’s Cp-Statistic as Estimator of the Mean Squared Error 668 24.6. Optimality of Variance Estimators 670 Chapter 25. Variance Estimation: Should One Require Unbiasedness? 675 25.1. Setting the Framework Straight 678 25.2. Derivation of the Best Bounded MSE Quadratic Estimator of the Variance 682 [...]... the exams In the on-line version of the notes they are printed in a different color A After some more cleaning out of the code, I am planning to make the AMS-L TEX source files for these notes publicly available under the GNU public license, and upload them to the TEX-archive network CTAN Since I am using Debian GNU/Linux, the materials will also be available as a deb archive The most up-to-date version... the web site of the Economics Department of the University of Utah www.econ .utah. edu/ehrbar/ecmet.pdf You can contact me by email at ehrbar@econ .utah. edu Hans Ehrbar CHAPTER 2 Probability Fields 2.1 The Concept of Probability Probability theory and statistics are useful in dealing with the following types of situations: • Games of chance: throwing dice, shuffling cards, drawing balls out of urns • Quality... are class notes from several different graduate econometrics and statistics classes In the Spring 2000 they were used for Statistics 6869, syllabus on p ??, and in the Fall 2000 for Economics 7800, syllabus on p ?? The notes give a careful and complete mathematical treatment intended to be accessible also to a reader inexperienced in math There are 618 exercise questions, almost all with answers The R-package... Formulas for the Test Statistics Interpretation in terms of Studentized Mahalanobis Distance 921 921 929 931 932 Chapter 42.1 42.2 42.3 42.4 42.5 42 Three Principles for Testing a Linear Constraint Mathematical Detail of the Three Approaches Examples of Tests of Linear Hypotheses The F-Test Statistic is a Function of the Likelihood Ratio Tests of Nonlinear Hypotheses Choosing Between Nonnested Models 941... Arbitrary Constraint into a Zero Constraint Lagrange Approach to Constrained Least Squares Constrained Least Squares as the Nesting of Two Simpler Models Solution by Quadratic Decomposition Sampling Properties of Constrained Least Squares Estimation of the Variance in Constrained OLS Inequality Restrictions 737 738 740 742 748 750 752 755 763 xii CONTENTS 29.9 Application: Biased Estimators and Pre-Test Estimators... recognizes here the openness and stratification of the world: If many different in uences come together, each of which is governed by laws, then their sum total is not determinate, as a naive hyperdeterminist would think, but indeterminate This is not only a condition for the possibility of science (in a hyper-deterministic world, one could not know anything before one knew everything, and science would also... uncertainty is as hard to generate as pure certainty; it is needed for encryption and numerical methods Here is an encryption scheme which leads to a random looking sequence of numbers (see [Rao97, p 13]): First a string of binary random digits is generated which is known only to the sender and receiver The sender converts his message into a string of binary digits He then places the message string below... string below the key string and obtains a coded string by changing every message bit to its alternative at all places where the key bit is 1 and leaving the others unchanged The coded string which appears to be a random binary sequence is transmitted The received message is decoded by 10 2 PROBABILITY FIELDS making the changes in the same way as in encrypting using the key string which is known to the... anything), but also for practical human activity: the macro outcomes of human practice are largely independent of micro detail (the postcard arrives whether the address is written in cursive or in printed letters, etc.) Games of chance are situations which deliberately project this micro indeterminacy into the macro world: the micro in uences cancel each other out without one enduring in uence taking... epistemic uncertainty but ontological indeterminacy, since the polarized photons form a pure state, which is atomic in the algebra of events In this case, the distinction between epistemic uncertainty and ontological indeterminacy is operational: the two alternatives follow different mathematics • Statistical mechanics: the velocity distribution of molecules in a gas at a given pressure and temperature . Class Notes in Statistics and Econometrics Hans G. Ehrbar Economics Department, University of Utah, 1645 Campus Center Drive, Salt Lake City UT 8411 2-9 300, U.S.A. URL: www.econ .utah. edu/ehrbar/ecmet.pdf E-mail. Inequality 1043 48.1. Web Resources about Income Inequality 1043 48.2. Graphical Representations of Inequality 1044 48.3. Quantitative Measures of Income Inequality 1045 48.4. Properties of Inequality. Ratio, and Lagrange Multiplier Tests 465 Chapter 16. General Principles of Econometric Modelling 469 Chapter 17. Causality and Inference 473 Chapter 18. Mean-Variance Analysis in the Linear Model