Generalized Least Squares Generalized Least Squares Takeaki Kariya and Hiroshi Kurata  2004 John Wiley & Sons, Ltd ISBN: 0-470-86697-7 (PPC) WILEY SERIES IN PROBABILITY AND STATISTICS Established by WALTER A SHEWHART and SAMUEL S WILKS Editors: David J Balding, Peter Bloomfield, Noel A C Cressie, Nicholas I Fisher, Iain M Johnstone, J B Kadane, Geert Molenberghs, Louise M Ryan, David W Scott, Adrian F M Smith, Jozef L Teugels; Editors Emeriti: Vic Barnett, J Stuart Hunter, David G Kendall A complete list of the titles in this series appears at the end of this volume Generalized Least Squares Takeaki Kariya Kyoto University and Meiji University, Japan Hiroshi Kurata University of Tokyo, Japan Copyright 2004 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England Telephone (+44) 1243 779777 Email (for orders and customer service enquiries): cs-books@wiley.co.uk Visit our Home Page on www.wileyeurope.com or www.wiley.com All Rights Reserved No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to permreq@wiley.co.uk, or faxed to (+44) 1243 770620 This publication is designed to provide accurate and authoritative information in regard to the subject matter covered It is sold on the understanding that the Publisher is not engaged in rendering professional services If professional advice or other expert assistance is required, the services of a competent professional should be sought Other Wiley Editorial Offices John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA Wiley-VCH Verlag GmbH, Boschstr 12, D-69469 Weinheim, Germany John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1 Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be available in electronic books Library of Congress Cataloging-in-Publication Data Kariya, Takeaki Generalized least squares / Takeaki Kariya, Hiroshi Kurata p cm – (Wiley series in probability and statistics) Includes bibliographical references and index ISBN 0-470-86697-7 (alk paper) Least squares I Kurata, Hiroshi, 1967-II Title III Series QA275.K32 2004 511 42—dc22 2004047963 British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN 0-470-86697-7 (PPC) Produced from LaTeX files supplied by the author and processed by Laserwords Private Limited, Chennai, India Printed and bound in Great Britain by TJ International, Padstow, Cornwall This book is printed on acid-free paper responsibly manufactured from sustainable forestry in which at least two trees are planted for each one used for paper production To my late GLS co-worker Yasuyuki Toyooka and to my wife Shizuko —Takeaki Kariya To Akiko, Tomoatsu and the memory of my fathers —Hiroshi Kurata Contents Preface xi Preliminaries 1.1 Overview 1.2 Multivariate Normal and Wishart Distributions 1.3 Elliptically Symmetric Distributions 1.4 Group Invariance 1.5 Problems 1 16 21 Generalized Least Squares Estimators 2.1 Overview 2.2 General Linear Regression Model 2.3 Generalized Least Squares Estimators 2.4 Finiteness of Moments and Typical GLSEs 2.5 Empirical Example: CO2 Emission Data 2.6 Empirical Example: Bond Price Data 2.7 Problems 25 25 26 33 40 49 55 63 67 67 68 73 82 90 95 Nonlinear Versions of the Gauss–Markov Theorem 3.1 Overview 3.2 Generalized Least Squares Predictors 3.3 A Nonlinear Version of the Gauss–Markov Theorem in Prediction 3.4 A Nonlinear Version of the Gauss–Markov Theorem in Estimation 3.5 An Application to GLSEs with Iterated Residuals 3.6 Problems SUR and Heteroscedastic Models 4.1 Overview 4.2 GLSEs with a Simple Covariance Structure 4.3 Upper Bound for the Covariance Matrix of a GLSE 4.4 Upper Bound Problem for the UZE in an SUR Model 4.5 Upper Bound Problems for a GLSE in a Heteroscedastic Model vii 97 97 102 108 117 126 viii CONTENTS 4.6 4.7 Empirical Example: CO2 Emission Data 134 Problems 140 Serial Correlation Model 5.1 Overview 5.2 Upper Bound for the Risk Matrix of a GLSE 5.3 Upper Bound Problem for a GLSE in the Anderson Model 5.4 Upper Bound Problem for a GLSE in a Two-equation Heteroscedastic Model 5.5 Empirical Example: Automobile Data 5.6 Problems Normal Approximation 6.1 Overview 6.2 Uniform Bounds for Normal Approximations to the Probability Density Functions 6.3 Uniform Bounds for Normal Approximations to the Cumulative Distribution Functions 6.4 Problems 143 143 145 153 158 165 170 171 171 176 182 193 Extension of Gauss–Markov Theorem 7.1 Overview 7.2 An Equivalence Relation on S(n) 7.3 A Maximal Extension of the Gauss–Markov Theorem 7.4 Nonlinear Versions of the Gauss–Markov Theorem 7.5 Problems 195 195 198 203 208 212 Some Further Extensions 8.1 Overview 8.2 Concentration Inequalities for the Gauss–Markov Estimator 8.3 Efficiency of GLSEs under Elliptical Symmetry 8.4 Degeneracy of the Distributions of GLSEs 8.5 Problems 213 213 214 223 233 241 Growth Curve Model and GLSEs 9.1 Overview 9.2 Condition for the Identical Equality between the GME and the OLSE 9.3 GLSEs and Nonlinear Version of the Gauss–Markov Theorem 9.4 Analysis Based on a Canonical Form 9.5 Efficiency of GLSEs 9.6 Problems 244 244 249 250 255 262 271 CONTENTS ix A Appendix 274 A.1 Asymptotic Equivalence of the Estimators of θ in the AR(1) Error Model and Anderson Model 274 Bibliography 281 Index 287 Preface Regression analysis has been one of the most widely employed and most important statistical methods in applications and has been continually made more sophisticated from various points of view over the last four decades Among a number of branches of regression analysis, the method of generalized least squares estimation based on the well-known Gauss–Markov theory has been a principal subject, and is still playing an essential role in many theoretical and practical aspects of statistical inference in a general linear regression model A general linear regression model is typically of a certain covariance structure for the error term, and the examples are not only univariate linear regression models such as serial correlation models, heteroscedastic models and equi-correlated models but also multivariate models such as seemingly unrelated regression (SUR) models, multivariate analysis of variance (MANOVA) models, growth curve models, and so on When the problem of estimating the regression coefficients in such a model is considered and when the covariance matrix of the error term is known, as an efficient estimation procedure, we rely on the Gauss–Markov theorem that the Gauss–Markov estimator (GME) is the best linear unbiased estimator In practice, however, the covariance matrix of the error term is usually unknown and hence the GME is not feasible In such cases, a generalized least squares estimator (GLSE), which is defined as the GME with the unknown covariance matrix replaced by an appropriate estimator, is widely used owing to its theoretical and practical virtue This book attempts to provide a self-contained treatment of the unified theory of the GLSEs with a focus on their finite sample properties We have made the content and exposition easy to understand for first-year graduate students in statistics, mathematics, econometrics, biometrics and other related fields One of the key features of the book is a concise and mathematically rigorous description of the material via the lower and upper bounds approach, which enables us to evaluate the finite sample efficiency in a general manner In general, the efficiency of a GLSE is measured by relative magnitude of its risk (or covariance) matrix to that of the GME However, since the GLSE is in general a nonlinear function of observations, it is often very difficult to evaluate the risk matrix in an explicit form Besides, even if it is derived, it is often impractical to use such a result because of its complication To overcome this difficulty, our book adopts as a main tool the lower and upper bounds approach, xi xii PREFACE which approaches the problem by deriving a sharp lower bound and an effective upper bound for the risk matrix of a GLSE: for this purpose, we begin by showing that the risk matrix of a GLSE is bounded below by the covariance matrix of the GME (Nonlinear Version of the Gauss–Markov Theorem); on the basis of this result, we also derive an effective upper bound for the risk matrix of a GLSE relative to the covariance matrix of the GME (Upper Bound Problems) This approach has several important advantages: the upper bound provides information on the finite sample efficiency of a GLSE; it has a much simpler form than the risk matrix itself and hence serves as a tractable efficiency measure; furthermore, in some cases, we can obtain the optimal GLSE that has the minimum upper bound among an appropriate class of GLSEs This book systematically develops the theory with various examples The book can be divided into three parts, corresponding respectively to Chapters and 2, Chapters to 6, and Chapters to The first part (Chapters and 2) provides the basics for general linear regression models and GLSEs In particular, we first give a fairly general definition of a GLSE, and establish its fundamental properties including conditions for unbiasedness and finiteness of second moments The second part (Chapters 3–6), the main part of this book, is devoted to the detailed description of the lower and upper bounds approach stated above and its applications to serial correlation models, heteroscedastic models and SUR models First, in Chapter 3, a nonlinear version of the Gauss–Markov theorem is established under fairly mild conditions on the distribution of the error term Next, in Chapters and 5, we derive several types of effective upper bounds for the risk matrix of a GLSE Further, in Chapter 6, a uniform bound for the normal approximation to the distribution of a GLSE is obtained The last part (Chapters 7–9) provides further developments (including mathematical extensions) of the results in the second part Chapter is devoted to making a further extension of the Gauss–Markov theorem, which is a maximal extension in a sense and leads to a further generalization of the nonlinear Gauss–Markov theorem proved in Chapter In the last two chapters, some complementary topics are discussed These include concentration inequalities, efficiency under elliptical symmetry, degeneracy of the distribution of a GLSE, and estimation of growth curves This book is not intended to be exhaustive, and there are many topics that are not even mentioned Instead, we have done our best to give a systematic and unified presentation We believe that reading this book leads to quite a solid understanding of this attractive subject, and hope that it will stimulate further research on the problems that remain The authors are indebted to many people who have helped us with this work Among others, I, Takeaki Kariya, am first of all grateful to Professor Morris L Eaton, who was my PhD thesis advisor and helped us get in touch with the publishers I am also grateful to my late coauthor Yasuyuki Toyooka with whom 282 BIBLIOGRAPHY Breusch TS and Pagan AR 1980 The Lagrange multiplier test and its applications to model specification in econometrics Review of Economic Studies 47, 239–253 Brockwell PJ and Davis RA 2002 Introduction to Time Series and Forecasting, 2nd edn Springer-Verlag Cambanis S, Huang S and Simons G 1981 On the theory of elliptically contoured distributions Journal of Multivariate Analysis 11, 368–385 Davis AW 1989 Distribution theory for some tests of independence of seemingly unrelated regressions Journal of Multivariate Analysis 31, 69–82 Eaton ML 1981 On the projections of isotropic distributions Annals of Statistics 9, 391–400 Eaton ML 1982 A review of selected topics in multivariate probability inequalities Annals of Statistics 10, 11–43 Eaton ML 1983 Multivariate Statistics: A Vector Space Approach John Wiley & Sons Eaton ML 1985 The Gauss-Markov theorem in multivariate analysis In Multivariate Analysis VI (ed Krishnaiah PR), pp 177–201 Elsevier Science Publishers B.V (North Holland) Eaton ML 1986 A characterization of spherical distributions Journal of Multivariate Analysis 20, 272–276 Eaton ML 1987 Group induced orderings with some applications in statistics CWI Newsletter 16, 3–31 Eaton ML 1988 Concentration inequarities for Gauss-Markov estimators Journal of Multivariate Analysis 25, 119–138 Eaton ML 1989 Group Invariance Applications in Statistics (Regional Conference Series in Probability and Statistics 1) Institute of Mathematical Statistics Ezekiel M and Fox FA 1959 Methods of Correlation and Regression Analysis John Wiley & Sons Fang KT, Kotz S and Ng KW 1990 Symmetric Multivariate and Related Distributions Chapman & Hall Feller W 1966 An Introduction to Probability Theory and its Applications John Wiley & Sons Ferguson TS 1967 Mathematical Statistics Academic Press Geisser S 1970 Baysian analysis of growth curve Sankhya A 32, 53–64 Goldberger AS 1962 Best linear unbiased prediction in the generalized linear regression model Journal of American Statistical Association 57, 369–375 Harville DA and Jeske DR 1992 Mean squared error of estimation or prediction under a general linear model Journal of American Statistical Association 87, 724–731 Hasegawa H 1995 On small sample properties of Zellner’s estimator for the case of two SUR equations with compound normal disturbances Communications in Statistics: Simulation and Computation 24, 45–59 Hillier GH and King ML 1987 Linear regression with correlated errors: bounds on coefficient estimates and t-values In Specification Analysis in the Linear Model (In Honour of Donald Cochrane) (eds King ML and Giles DEA), pp 74–80 Routledge & Kegan Paul Hwang JT 1985 Universal domination and stochastic domination: Estimation simultaneously under a broad class of loss functions Annals of Statistics 13, 295–314 Jensen DR 1996 Symmetry and unimodality in linear inference Journal of Multivariate Analysis 60, 188–202 Judge GG, Griffiths WE, Hill RC, Lăutkepohl M and Lee TC 1985 The Theory and Practice of Econometrics, 2nd edn John Wiley & Sons BIBLIOGRAPHY 283 Kanda T 1994 Growth curve model with covariance structures Hiroshima Mathematical Journal 24, 135–176 Kariya T 1980 Note on a condition for equality of sample variances in a linear model Journal of American Statistical Association 75, 701–703 Kariya T 1981a Bounds for the covariance matrices of Zellner’s estimator in the SUR model and the 2SAE in a heteroscedastic model Journal of American Statistical Association 76, 975–979 Kariya T 1981b Tests for the independence between two seemingly unrelated regression equations Annals of Statistics 9, 381–390 Kariya T 1985a A nonlinear version of the Gauss-Markov theorem Journal of American Statistical Association 80, 476–477 Kariya T 1985b Testing in the Multivariate General Linear Model Kinokuniya, Tokyo Kariya T 1988 An identity in regression with missing observations In Statistical Theory and Data Analysis II (ed Matusita K), pp 365–368 Elsevier Science Publishers B.V (North Holland) Kariya T 1993 Quantitative Methods for Portfolio Analysis Kluwer Academic Publishers Kariya T, Fujikoshi Y and Krishnaiah PR 1984 Tests for independence of two multivariate regression equations with different design matrices Journal of Multivariate Analysis 15, 383–407 Kariya T, Fujikoshi Y and Krishnaiah PR 1987 On tests for selection of variables and independence under multivariate regression models Journal of Multivariate Analysis 21, 207–237 Kariya T and Kurata H 2002 A maximal extension of the Gauss-Markov theorem and its nonlinear version Journal of Multivariate Analysis 83, 37–55 Kariya T and Sinha BK 1989 Robustness of Statistical Tests Academic Press Kariya T and Toyooka Y 1985 Nonlinear versions of the Gauss-Markov theorem and GLSE In Multivariate Analysis VI (ed Krishnaiah PR), pp 345–354 Elsevier Science Publishers B.V (North Holland) Kariya T and Toyooka Y 1992 Bounds for normal approximations to the distributions of generalized least squares predictors and estimators Journal of Statistical Planning and Inference 30, 213–221 Kariya T and Tsuda H 1994 New bond pricing models with applications to Japanese data Financial Engineering and the Japanese Markets 1, 1–20 Kelker D 1970 Distribution theory of spherical distributions and a location-scale parameter Sankhya A 32, 419–430 Khatri CG 1966 A note on MANOVA model applied to problems in growth curves Annals of Institute of Statistical Mathematics 18, 75–86 Khatri CG and Shah KR 1974 Estimation of location parameters from two linear models under normality Communications in Statistics: Theory and Methods 3, 647–663 Khatri CG and Srivastava MS 1971 On exact non-null distributions of likelihood ration criteria for sphericity test and equality of two covariance matrices Sankhya A 33, 201– 206 Knott M 1975 On the minimum efficiency of least squares Biometrika 62, 129–132 Kobayashi M 1985 Comparison of efficiencies of several estimators for linear regressions with autocorrelated errors Journal of American Statistical Association 80, 951–953 Kruskal W 1968 When are Gauss-Markov and least squares estimators identical? A coordinate-free approach Annals of Mathematical Statistics 39, 70–75 284 BIBLIOGRAPHY Kubokawa T 1998 Double shrinkage estimation of common coefficients in two regression equations with heteroscedasticity Journal of Multivariate Analysis 67, 169–189 Kunitomo N 1977 A note on the efficiency of Zellner’s estimator for the case of two seemingly unrelated regression equations Economic Studies Quarterly 28, 73–77 Kurata H 1998 A generalization of Rao’s covariance structure with applications to several linear models Journal of Multivariate Analysis 67, 297–305 Kurata H 1999 On the efficiencies of several generalized least squares estimators in a seemingly unrelated regression model and a heteroscedastic model Journal of Multivariate Analysis 70, 86–94 Kurata H 2001 A note on an upper bound for the covariance matrix of a generalized least squares estimator in a heteroscedastic model Scientiae Mathematicae Japonicae 53, 65–74 Kurata H and Kariya T 1996 Least upper bound for the covariance matrix of a generalized least squares estimator in regression with applications to a seemingly unrelated regression model and a heteroscedastic model Annals of Statistics 24, 1547–1559 Kuritsyn YG 1986 On the least-squares method for elliptically contoured distributions Theory of Probability and its Applications 31, 738–740 Lehmann EL 1983 Theory of Point Estimation John Wiley & Sons Lehmann EL 1986 Testing Statistical Hypotheses, 2nd edn Springer-Verlag Liski EP and Zaigraev A 2001 A stochastic characterization of Loewner optimality design criterion in linear models Metrika 53, 207–222 Lu CY and Shi NZ 2000 Universal inadmissibility of least squares estimator Journal of Multivariate Analysis 72, 22–29 Magee L 1985 Efficiency of iterative estimators in the regression model with AR(1) disturbances Journal of Econometrics 29, 275–287 Magee L, Ullah A and Srivastava VK 1987 Efficiency of estimators in the regression model with first-order autoregressive errors In Specification Analysis in the Linear Model (In Honour of Donald Cochrane) (eds King ML and Giles DEA), pp 81–98 Routledge & Kegan Paul Magnus JR 1978 Maximum likelihood estimation of the GLS model with unknown parameters in the disturbance covariance matrix Journal of Econometrics 7, 281–312 Mathew T 1983 Linear estimation with an incorrect dispersion matrix in linear models with a common linear part Journal of American Statistical Association 78, 468–471 McCulloch JH 1971 Measuring the term structure of interest rates Journal of Business 44, 19–31 McCulloch JH 1975 The tax-adjusted yield curve Journal of Business, 30, 811–830 McElroy FW 1967 A necessary and sufficient condition that ordinary least-squares estimators be best linear unbiased Journal of American Statistical Association 62, 1302–1304 Mehta JS and Swamy PAVB 1976 Further evidence on the relative efficiencies of Zellner’s seemingly unrelated regressions estimator Journal of American Statistical Association 71, 634–639 Muirhead RJ 1982 Aspects of Multivariate Statistical Theory John, Wiley & Sons Nawata K 2001 Excel Toukei Kaiseki Box niyoru Data Kaiseki (in Japanese) (Data Analysis by Excel Statistical Analysis Box) Asakura Shoten Ng VM 2000 A note on predictive inference for multivariate elliptically contoured distributions Communications in Statistics: Theory and Methods 29, 477–483 BIBLIOGRAPHY 285 Ng VM 2002 Robust Baysian inference for seemingly unrelated regressions with elliptical errors Journal of Multivariate Analysis 83, 409–414 Pan JX and Fang KT 2002 Growth Curve Models with Statistical Diagnostics SpringerVerlag Perron F 1992 Equivariant estimator of the covariance matrix Canadian Journal of Statistics 43, 16–28 Potthoff RF and Roy SN 1964 A generalized multivariate analysis of variance model useful especially for growth curve problems Biometrika 51, 313–326 Puntanen S and Styan GPH 1989 The equality of the ordinary least squares estimator and the best linear unbiased estimator American Statistician 43, 153–161 Rao CR 1967 Least squares theory using an estimated dispersion matrix and its application to measurement of signals In Proceedings of the Fifth Berkely Symposium on Mathematical Statistics and Probability (eds Lecam LM and Neyman J), Vol 1, pp 255–372 University of California Press Rao CR 1973 Linear Statistical Inference and its Applications, 2nd edn John Wiley & Sons Rao CR and Mitra SK 1971 Generalized Inverse of Matrices and its Applications John Wiley & Sons Rao CR and Rao MB 1998 Matrix Algebra and its Applications to Statistics and Econometrics World Scientific Publishing Revankar NS 1974 Some finite sample results in the context of two seemingly unrelated regression equations Journal of American Statistical Association 68, 187–190 Revankar NS 1976 Use of restricted residuals in SUR systems: some finite sample results Journal of American Statistical Association 69, 183–188 Schoenberg IJ 1938 Metric spaces and completely monotone functions Annals of Mathematics 39, 811–841 Sen A and Srivastava M 1990 Regression Analysis: Theory, Methods, and Applications Springer-Verlag Srivastava VK and Giles DEA 1987 Seemingly Unrelated Regression Equation Models Marcel Dekker Srivastava VK and Maekawa K 1995 Efficiency properties of feasible generalized least squares estimators in SURE models under non-normal disturbances Journal of Econometrics 66, 99–121 Sugiura N and Kubokawa T 1988 Estimating common parameters of growth curve models Annals of Institute of Statistical Mathematics 40, 119–135 Swamy PAVB and Mehta JS 1979 Estimation of common coefficients in two regression equations Journal of Econometrics 10, 1–14 Takemura A 1984 An orthogonally invariant minimax estimator of the covariance matrix of a multivariate normal population Tsukuba Journal of Mathematics 8, 367–376 Taylor WE 1977 Small sample properties of a class of two stage Aitken estimators Econometrica 45, 497–508 Taylor WE 1978 The heteroscedastic linear model: exact finite sample results Econometrica 46, 663–675 Tong YL 1990 The Multivariate Normal Distribution Springer-Verlag Toyooka Y 1987 An iterated version of the Gauss-Markov theorem in generalized least squares estimation Journal of Japan Statistical Society 17, 129–136 Toyooka Y and Kariya T 1986 An approach to upper bound problems for risks of the generalized least squares estimators Annals of Statistics 14, 679–690 286 BIBLIOGRAPHY Toyooka Y and Kariya T 1995 A note on sufficient condition for the existence of 2nd moments of generalized least squares estimator in a linear model Mathematica Japonica 42, 509–510 Usami Y and Toyooka Y 1997a On the degeneracy of the distribution of a GLSE in a regression with a circularly distributed error Mathematica Japonica 45, 423–431 Usami Y and Toyooka Y 1997b Errata of Kariya and Toyooka (1992), Bounds for normal approximations to the distributions of generalized least squares predictors and estimators Journal of Statistical Planning and Inference 58, 399–405 von Rosen D 1991 The growth curve model: a review Communications in Statistics: Theory and Methods 20, 2791–2822 Wang SG and Shao J 1992 Constrained Kantorovich inequalities and relative efficiency of least squares Journal of Multivariate Analysis 42, 284–298 Wang SG and Ip WC 1999 A matrix version of the Wielandt inequality and its applications to statistics Linear Algebra and its Applications 296, 171–181 Wu L and Perlman MD 2000 Lattice conditional independence model for seemingly unrelated regressions Communications in Statistics: Simulation and Computation 29, 361– 384 Zellner A 1962 An efficient method of estimating seemingly unrelated regressions and tests for aggregation bias Journal of American Statistical Association 57, 348–368 Zellner A 1963 Estimators for seemingly unrelated regressions: some finite sample results Journal of American Statistical Association 58, 977–992 Zyskind G 1967 On canonical forms, non-negative covariance matrices and best and simple least squares estimators in linear models Annals of Mathematical Statistics 38, 1092– 1109 Zyskind G 1969 Parametric augmentations and error structures under which certain simple least squares and analysis of variance procedures are also best Journal of American Statistical Association 64, 1353–1368 Index Anderson model, 29, 40, 53, 182, 187, 234, 237 Anderson’s theorem, 217 AR(1) error model, 28, 79 Automobile data, 165 conditional distribution, 13 marginal distribution, 13 Equi-correlated model, 29 Equivalence theorem, 215 Equivariant estimator, 124, 132 location-equivariant estimator, 37, 197, 220 Bartlet decomposition, Best linear unbiased estimator (BLUE), 34 Best linear unbiased predictor (BLUP), 70 Bond price data, 55 Gamma — function, Multivariate — function, Gauss–Markov estimator (GME), 34, 70, 98, 144, 172, 196, 214 Gauss–Markov predictor (GMP), 71, 172 Gauss–Markov theorem, 34, 196 — in prediction, 70 maximal extension of —, 206 nonlinear version of — in estimation, 83, 209, 260 nonlinear version of — in prediction, 76 General linear regression model, 26, 34, 41, 82, 97, 171, 195, 214, 244 General multivariate analysis of variance (GMANOVA) model, 245 Generalized least squares estimator (GLSE), 35, 72, 91, 98, 144, 173, 220, 245 Cauchy–Schwarz inequality, 41, 43 Chi-square (χ ) distribution, Cholesky decomposition, Circularly distributed error model, 234 CO2 emission data, 135, 181 Concentration inequality, 215 Concentration probability, 215 Conditional covariance matrix, 15 Convex set, 215 Covariance matrix, Degeneracy, 233 Elasticity, 49, 136 Elliptically symmetric distribution — with finite moments, 11 definition, 11 characterization, 87 Generalized Least Squares Takeaki Kariya and Hiroshi Kurata  2004 John Wiley & Sons, Ltd ISBN: 0-470-86697-7 (PPC) 287 288 Generalized least squares predictor (GLSP), 72, 173 Group — of nonsingular lower-triangular matrices, 123 — of nonsingular lower-triangular matrices with positive diagonal elements, 16 — of nonsingular matrices, 16 — of orthogonal matrices, 16 definition, 16 group action, 17 group invariance, 16 subgroup, 17 transitivity, 17 Growth curve model, 244 Heteroscedastic model, 30, 43, 84, 93, 105, 127, 158, 165 Hypergeometric function, 161 Indicator function, 14 Invariance property — of GLSE, 43, 125 — of loss function, 115, 125 Invariant function, 18 Inversion formula, 176 Kantorovich inequality, 111 Kronecker product, 32 Linear unbiased estimator, 34 Linear unbiased predictor, 70 Location-equivariant estimator, 37, 197, 220 Log-concavity, 218 Loss function, 114, 125, 132, 150, 160 Maximal invariant, 18 Maximum likelihood estimator (MLE), 39 Mean vector, INDEX Moore–Penrose generalized inverse, 66 Most concentrated, 215 Multivariate linear regression model, 21 Multivariate normal distribution characteristic function, 21 characterization, conditional distribution, marginal distribution, moment generating function, 21 probability density function, Nonlinear version of Gauss–Markov theorem — in estimation, 83, 209, 260 — in prediction, 76 Normal approximation, 171 Orbit, 17 Ordinary least squares estimator (OLSE), 35, 104, 145, 194, 197 Ordinary least squares (OLS) residual, 20, 245, 250 Rao’s covariance structure, 152, 200, 249 Restricted GLSE, 45 Restricted Zellner estimator (RZE), 48 Risk function, 125, 133 Risk matrix, 70, 98, 144, 196, 245 Seemingly unrelated regression (SUR) model, 31, 47, 85, 94, 101, 107, 118, 135, 180, 187, 224 Serial correlation model, 143 Simple covariance structure, 102 Spectral decomposition, Spherically symmetric distribution — with finite moments, 10 definition, INDEX Symmetric convex set, 215 Symmetric inverse property, 115, 150, 160 Symmetric set, 215 Symmetry about the origin, 217 Uniform bound — for cdf, 186 — for pdf, 176 Uniform distribution on the unit sphere, 4, 10 Unimodal, 217 289 Unrestricted GLSE, 46, 106, 129, 160, 168 Unrestricted Zellner estimator (UZE), 48, 107, 118, 225 Upper bound for risk (covariance) matrix, 109, 144 Upper bound problem, 102 Wishart distribution marginal distribution, probability density function, Generalized Least Squares Takeaki Kariya and Hiroshi Kurata  2004 John Wiley & Sons, Ltd ISBN: 0-470-86697-7 (PPC) WILEY SERIES IN PROBABILITY AND STATISTICS ESTABLISHED BY WALTER A SHEWHART AND SAMUEL S WILKS Editors: David J Balding, Noel A C Cressie, Nicholas I Fisher, Iain M Johnstone, J B Kadane, Geert Molenberghs, Louise M Ryan, David W Scott, Adrian F M Smith, Jozef L Teugels Editors Emeriti: Vic Barnett, J Stuart Hunter David G Kendall The Wiley Series in Probability and Statistics is well established and authoritative It covers many topics of current research interest in both pure and applied statistics and probability theory Written by leading statisticians and institutions, the titles span both state-of-the-art developments in the field and classical methods Reflecting the wide range of current research in statistics, the series encompasses applied, methodological and theoretical statistics, ranging from applications and new techniques made possible by advances in computerized practice to rigorous treatment of theoretical approaches This series provides essential and invaluable reading for all statisticians, whether in academia, industry, government, or research ABRAHAM and LEDOLTER · Statistical Methods for Forecasting AGRESTI · Analysis of Ordinal Categorical Data AGRESTI · An Introduction to Categorical Data Analysis AGRESTI · Categorical Data Analysis, Second Edition ALTMAN, GILL, and McDONALD · Numerical Issues in Statistical Computing for the Social Scientist ˇ · Mathematics of Chance ANDEL ANDERSON · An Introduction to Multivariate Statistical Analysis, Third Edition ∗ ANDERSON · The Statistical Analysis of Time Series ANDERSON, AUQUIER, HAUCK, OAKES, VANDAELE, and WEISBERG · Statistical Methods for Comparative Studies ANDERSON and LOYNES · The Teaching of Practical Statistics ARMITAGE and DAVID (editors) · Advances in Biometry ARNOLD, BALAKRISHNAN, and NAGARAJA · Records ∗ ARTHANARI and DODGE · Mathematical Programming in Statistics ∗ BAILEY · The Elements of Stochastic Processes with Applications to the Natural Sciences BALAKRISHNAN and KOUTRAS · Runs and Scans with Applications BARNETT · Comparative Statistical Inference, Third Edition BARNETT · Environmental Statistics: Methods & Applications BARNETT and LEWIS · Outliers in Statistical Data, Third Edition BARTOSZYNSKI and NIEWIADOMSKA-BUGAJ · Probability and Statistical Inference BASILEVSKY · Statistical Factor Analysis and Related Methods: Theory and Applications BASU and RIGDON · Statistical Methods for the Reliability of Repairable Systems BATES and WATTS · Nonlinear Regression Analysis and Its Applications BECHHOFER, SANTNER, and GOLDSMAN · Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons BELSLEY · Conditioning Diagnostics: Collinearity and Weak Data in Regression BELSLEY, KUH, and WELSCH · Regression Diagnostics: Identifying Influential Data and Sources of Collinearity BENDAT and PIERSOL · Random Data: Analysis and Measurement Procedures, Third Edition ∗ Now available in a lower priced paperback edition in the Wiley Classics Library BERRY, CHALONER, and GEWEKE · Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner BERNARDO and SMITH · Bayesian Theory BHAT and MILLER · Elements of Applied Stochastic Processes, Third Edition BHATTACHARYA and JOHNSON · Statistical Concepts and Methods BHATTACHARYA and WAYMIRE · Stochastic Processes with Applications BILLINGSLEY · Convergence of Probability Measures, Second Edition BILLINGSLEY · Probability and Measure, Third Edition BIRKES and DODGE · Alternative Methods of Regression BLISCHKE AND MURTHY (editors) · Case Studies in Reliability and Maintenance BLISCHKE AND MURTHY · Reliability: Modeling, Prediction, and Optimization BLOOMFIELD · Fourier Analysis of Time Series: An Introduction, Second Edition BOLLEN · Structural Equations with Latent Variables BOROVKOV · Ergodicity and Stability of Stochastic Processes BOULEAU · Numerical Methods for Stochastic Processes BOX · Bayesian Inference in Statistical Analysis BOX · R A Fisher, the Life of a Scientist BOX and DRAPER · Empirical Model-Building and Response Surfaces ∗ BOX and DRAPER · Evolutionary Operation: A Statistical Method for Process Improvement BOX, HUNTER, and HUNTER · Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building ˜ · Statistical Control by Monitoring and Feedback Adjustment BOX and LUCENO BRANDIMARTE · Numerical Methods in Finance: A MATLAB-Based Introduction BROWN and HOLLANDER · Statistics: A Biomedical Introduction BRUNNER, DOMHOF, and LANGER · Nonparametric Analysis of Longitudinal Data in Factorial Experiments BUCKLEW · Large Deviation Techniques in Decision, Simulation, and Estimation CAIROLI and DALANG · Sequential Stochastic Optimization CHAN · Time Series: Applications to Finance CHATTERJEE and HADI · Sensitivity Analysis in Linear Regression CHATTERJEE and PRICE · Regression Analysis by Example, Third Edition CHERNICK · Bootstrap Methods: A Practitioner’s Guide CHERNICK and FRIIS · Introductory Biostatistics for the Health Sciences ` and DELFINER · Geostatistics: Modeling Spatial Uncertainty CHILES CHOW and LIU · Design and Analysis of Clinical Trials: Concepts and Methodologies, Second Edition CLARKE and DISNEY · Probability and Random Processes: A First Course with Applications, Second Edition ∗ COCHRAN and COX · Experimental Designs, Second Edition CONGDON · Applied Bayesian Modelling CONGDON · Bayesian Statistical Modelling CONOVER · Practical Nonparametric Statistics, Second Edition COOK · Regression Graphics COOK and WEISBERG · Applied Regression Including Computing and Graphics COOK and WEISBERG · An Introduction to Regression Graphics CORNELL · Experiments with Mixtures, Designs, Models, and the Analysis of Mixture Data, Third Edition COVER and THOMAS · Elements of Information Theory COX · A Handbook of Introductory Statistical Methods ∗ COX · Planning of Experiments CRESSIE · Statistics for Spatial Data, Revised Edition ă ă and HORVATH ´ CSORG O · Limit Theorems in Change Point Analysis DANIEL · Applications of Statistics to Industrial Experimentation DANIEL · Biostatistics: A Foundation for Analysis in the Health Sciences, Sixth Edition ∗ DANIEL · Fitting Equations to Data: Computer Analysis of Multifactor Data, Second Edition ∗ Now available in a lower priced paperback edition in the Wiley Classics Library DASU and JOHNSON · Exploratory Data Mining and Data Cleaning DAVID and NAGARAJA · Order Statistics, Third Edition ∗ DEGROOT, FIENBERG, and KADANE · Statistics and the Law DEL CASTILLO · Statistical Process Adjustment for Quality Control DENISON, HOLMES, MALLICK and SMITH · Bayesian Methods for Nonlinear Classification and Regression DETTE and STUDDEN · The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis DEY and MUKERJEE · Fractional Factorial Plans DILLON and GOLDSTEIN · Multivariate Analysis: Methods and Applications DODGE · Alternative Methods of Regression ∗ DODGE and ROMIG · Sampling Inspection Tables, Second Edition ∗ DOOB · Stochastic Processes DOWDY, WEARDEN, and CHILKO · Statistics for Research, Third Edition DRAPER and SMITH · Applied Regression Analysis, Third Edition DRYDEN and MARDIA · Statistical Shape Analysis DUDEWICZ and MISHRA · Modern Mathematical Statistics DUNN and CLARK · Applied Statistics: Analysis of Variance and Regression, Second Edition DUNN and CLARK · Basic Statistics: A Primer for the Biomedical Sciences, Third Edition DUPUIS and ELLIS · A Weak Convergence Approach to the Theory of Large Deviations ∗ ELANDT-JOHNSON and JOHNSON · Survival Models and Data Analysis ENDERS · Applied Econometric Time Series ETHIER and KURTZ · Markov Processes: Characterization and Convergence EVANS, HASTINGS, and PEACOCK · Statistical Distributions, Third Edition FELLER · An Introduction to Probability Theory and Its Applications, Volume I, Third Edition, Revised; Volume II, Second Edition FISHER and VAN BELLE · Biostatistics: A Methodology for the Health Sciences ∗ FLEISS · The Design and Analysis of Clinical Experiments FLEISS · Statistical Methods for Rates and Proportions, Second Edition FLEMING and HARRINGTON · Counting Processes and Survival Analysis FULLER · Introduction to Statistical Time Series, Second Edition FULLER · Measurement Error Models GALLANT · Nonlinear Statistical Models GIESBRECHT and GUMPERTZ · Planning, Construction, and Statistical Analysis of Comparative Experiments GHOSH, MUKHOPADHYAY, and SEN · Sequential Estimation GIFI · Nonlinear Multivariate Analysis GLASSERMAN and YAO · Monotone Structure in Discrete-Event Systems GNANADESIKAN · Methods for Statistical Data Analysis of Multivariate Observations, Second Edition GOLDSTEIN and LEWIS · Assessment: Problems, Development, and Statistical Issues GREENWOOD and NIKULIN · A Guide to Chi-Squared Testing GROSS and HARRIS · Fundamentals of Queueing Theory, Third Edition ∗ HAHN and SHAPIRO · Statistical Models in Engineering HAHN and MEEKER · Statistical Intervals: A Guide for Practitioners HALD · A History of Probability and Statistics and their Applications Before 1750 HALD · A History of Mathematical Statistics from 1750 to 1930 HAMPEL · Robust Statistics: The Approach Based on Influence Functions HANNAN and DEISTLER · The Statistical Theory of Linear Systems HEIBERGER · Computation for the Analysis of Designed Experiments HEDAYAT and SINHA · Design and Inference in Finite Population Sampling HELLER · MACSYMA for Statisticians HINKELMAN and KEMPTHORNE: · Design and Analysis of Experiments, Volume 1: Introduction to Experimental Design ∗ Now available in a lower priced paperback edition in the Wiley Classics Library HOAGLIN, MOSTELLER, and TUKEY · Exploratory Approach to Analysis of Variance HOAGLIN, MOSTELLER, and TUKEY · Exploring Data Tables, Trends and Shapes ∗ HOAGLIN, MOSTELLER, and TUKEY · Understanding Robust and Exploratory Data Analysis HOCHBERG and TAMHANE · Multiple Comparison Procedures HOCKING · Methods and Applications of Linear Models: Regression and the Analysis of Variance, Second Edition HOEL · Introduction to Mathematical Statistics, Fifth Edition HOGG and KLUGMAN · Loss Distributions HOLLANDER and WOLFE · Nonparametric Statistical Methods, Second Edition HOSMER and LEMESHOW · Applied Logistic Regression, Second Edition HOSMER and LEMESHOW · Applied Survival Analysis: Regression Modeling of Time to Event Data HØYLAND and RAUSAND · System Reliability Theory: Models and Statistical Methods HUBER · Robust Statistics HUBERTY · Applied Discriminant Analysis HUNT and KENNEDY · Financial Derivatives in Theory and Practice HUSKOVA, BERAN, and DUPAC · Collected Works of Jaroslav Hajek— with Commentary IMAN and CONOVER · A Modern Approach to Statistics JACKSON · A User’s Guide to Principle Components JOHN · Statistical Methods in Engineering and Quality Assurance JOHNSON · Multivariate Statistical Simulation JOHNSON and BALAKRISHNAN · Advances in the Theory and Practice of Statistics: A Volume in Honor of Samuel Kotz ă JUDGE, GRIFFITHS, HILL, LUTKEPOHL, and LEE · The Theory and Practice of Econometrics, Second Edition JOHNSON and KOTZ · Distributions in Statistics JOHNSON and KOTZ (editors) · Leading Personalities in Statistical Sciences: From the Seventeenth Century to the Present JOHNSON, KOTZ, and BALAKRISHNAN · Continuous Univariate Distributions, Volume 1, Second Edition JOHNSON, KOTZ, and BALAKRISHNAN · Continuous Univariate Distributions, Volume 2, Second Edition JOHNSON, KOTZ, and BALAKRISHNAN · Discrete Multivariate Distributions JOHNSON, KOTZ, and KEMP · Univariate Discrete Distributions, Second Edition ˇ ´ and SEN · Robust Statistical Procedures: Asymptotics and Interrelations JURECKOV A JUREK and MASON · Operator-Limit Distributions in Probability Theory KADANE · Bayesian Methods and Ethics in a Clinical Trial Design KADANE AND SCHUM · A Probabilistic Analysis of the Sacco and Vanzetti Evidence KALBFLEISCH and PRENTICE · The Statistical Analysis of Failure Time Data Second Edition KARIYA and KURATA · Generalized Least Squares KASS and VOS · Geometrical Foundations of Asymptotic Inference KAUFMAN and ROUSSEEUW · Finding Groups in Data: An Introduction to Cluster Analysis KEDEM and FOKIANOS · Regression Models for Time Series Analysis KENDALL, BARDEN, CARNE, and LE · Shape and Shape Theory KHURI · Advanced Calculus with Applications in Statistics, Second Edition KHURI, MATHEW, and SINHA · Statistical Tests for Mixed Linear Models KLEIBER and KOTZ · Statistical Size Distributions in Economics and Actuarial Sciences KLUGMAN, PANJER, and WILLMOT · Loss Models: From Data to Decisions KLUGMAN, PANJER, and WILLMOT · Solutions Manual to Accompany Loss Models: From Data to Decisions KOTZ, BALAKRISHNAN, and JOHNSON · Continuous Multivariate Distributions, Volume 1, Second Edition ∗ Now available in a lower priced paperback edition in the Wiley Classics Library KOTZ and JOHNSON (editors) · Encyclopedia of Statistical Sciences: Volumes to with Index KOTZ and JOHNSON (editors) · Encyclopedia of Statistical Sciences: Supplement Volume KOTZ, READ, and BANKS (editors) · Encyclopedia of Statistical Sciences: Update Volume KOTZ, READ, and BANKS (editors) · Encyclopedia of Statistical Sciences: Update Volume KOVALENKO, KUZNETZOV, and PEGG · Mathematical Theory of Reliability of Time-Dependent Systems with Practical Applications LACHIN · Biostatistical Methods: The Assessment of Relative Risks LAD · Operational Subjective Statistical Methods: A Mathematical, Philosophical, and Historical Introduction LAMPERTI · Probability: A Survey of the Mathematical Theory, Second Edition LANGE, RYAN, BILLARD, BRILLINGER, CONQUEST, and GREENHOUSE · Case Studies in Biometry LARSON · Introduction to Probability Theory and Statistical Inference, Third Edition LAWLESS · Statistical Models and Methods for Lifetime Data, Second Edition LAWSON · Statistical Methods in Spatial Epidemiology LE · Applied Categorical Data Analysis LE · Applied Survival Analysis LEE and WANG · Statistical Methods for Survival Data Analysis, Third Edition LEPAGE and BILLARD · Exploring the Limits of Bootstrap LEYLAND and GOLDSTEIN (editors) · Multilevel Modelling of Health Statistics LIAO · Statistical Group Comparison LINDVALL · Lectures on the Coupling Method LINHART and ZUCCHINI · Model Selection LITTLE and RUBIN · Statistical Analysis with Missing Data, Second Edition LLOYD · The Statistical Analysis of Categorical Data MAGNUS and NEUDECKER · Matrix Differential Calculus with Applications in Statistics and Econometrics, Revised Edition MALLER and ZHOU · Survival Analysis with Long Term Survivors MALLOWS · Design, Data, and Analysis by Some Friends of Cuthbert Daniel MANN, SCHAFER, and SINGPURWALLA · Methods for Statistical Analysis of Reliability and Life Data MANTON, WOODBURY, and TOLLEY · Statistical Applications Using Fuzzy Sets MARDIA and JUPP · Directional Statistics MASON, GUNST, and HESS · Statistical Design and Analysis of Experiments with Applications to Engineering and Science, Second Edition McCULLOCH and SEARLE · Generalized, Linear, and Mixed Models McFADDEN · Management of Data in Clinical Trials McLACHLAN · Discriminant Analysis and Statistical Pattern Recognition McLACHLAN and KRISHNAN · The EM Algorithm and Extensions McLACHLAN and PEEL · Finite Mixture Models McNEIL · Epidemiological Research Methods MEEKER and ESCOBAR · Statistical Methods for Reliability Data MEERSCHAERT and SCHEFFLER · Limit Distributions for Sums of Independent Random Vectors: Heavy Tails in Theory and Practice ∗ MILLER · Survival Analysis, Second Edition MONTGOMERY, PECK, and VINING · Introduction to Linear Regression Analysis, Third Edition MORGENTHALER and TUKEY · Configural Polysampling: A Route to Practical Robustness MUIRHEAD · Aspects of Multivariate Statistical Theory MULLER and STOYAN · Comparison Methods for Stochastic Models and Risks ∗ Now available in a lower priced paperback edition in the Wiley Classics Library MURRAY · X-STAT 2.0 Statistical Experimentation, Design Data Analysis, and Nonlinear Optimization MURTHY, XIE, and JIANG · Weibull Models MYERS and MONTGOMERY · Response Surface Methodology: Process and Product Optimization Using Designed Experiments, Second Edition MYERS, MONTGOMERY, and VINING · Generalized Linear Models With Applications in Engineering and the Sciences NELSON · Accelerated Testing, Statistical Models, Test Plans, and Data Analyses NELSON · Applied Life Data Analysis NEWMAN · Biostatistical Methods in Epidemiology OCHI · Applied Probability and Stochastic Processes in Engineering and Physical Sciences OKABE, BOOTS, SUGIHARA, and CHIU · Spatial Tesselations: Concepts and Applications of Voronoi Diagrams, Second Edition OLIVER and SMITH · Influence Diagrams, Belief Nets and Decision Analysis PALTA · Quantitative Methods in Population Health: Extensions of Ordinary Regressions PANKRATZ · Forecasting with Dynamic Regression Models PANKRATZ · Forecasting with Univariate Box-Jenkins Models: Concepts and Cases ∗ PARZEN · Modern Probability Theory and It’s Applications ˜ TIAO, and TSAY · A Course in Time Series Analysis PENA, PIANTADOSI · Clinical Trials: A Methodologic Perspective PORT · Theoretical Probability for Applications POURAHMADI · Foundations of Time Series Analysis and Prediction Theory PRESS · Bayesian Statistics: Principles, Models, and Applications PRESS · Subjective and Objective Bayesian Statistics, Second Edition PRESS and TANUR · The Subjectivity of Scientists and the Bayesian Approach PUKELSHEIM · Optimal Experimental Design PURI, VILAPLANA, and WERTZ · New Perspectives in Theoretical and Applied Statistics PUTERMAN · Markov Decision Processes: Discrete Stochastic Dynamic Programming ∗ RAO · Linear Statistical Inference and Its Applications, Second Edition RENCHER · Linear Models in Statistics RENCHER · Methods of Multivariate Analysis, Second Edition RENCHER · Multivariate Statistical Inference with Applications RIPLEY · Spatial Statistics RIPLEY · Stochastic Simulation ROBINSON · Practical Strategies for Experimenting ROHATGI and SALEH · An Introduction to Probability and Statistics, Second Edition ROLSKI, SCHMIDLI, SCHMIDT, and TEUGELS · Stochastic Processes for Insurance and Finance ROSENBERGER and LACHIN · Randomization in Clinical Trials: Theory and Practice ROSS · Introduction to Probability and Statistics for Engineers and Scientists ROUSSEEUW and LEROY · Robust Regression and Outlier Detection RUBIN · Multiple Imputation for Nonresponse in Surveys RUBINSTEIN · Simulation and the Monte Carlo Method RUBINSTEIN and MELAMED · Modern Simulation and Modeling RYAN · Modern Regression Methods RYAN · Statistical Methods for Quality Improvement, Second Edition SALTELLI, CHAN, and SCOTT (editors) · Sensitivity Analysis ∗ SCHEFFE · The Analysis of Variance SCHIMEK · Smoothing and Regression: Approaches, Computation, and Application SCHOTT · Matrix Analysis for Statistics SCHUSS · Theory and Applications of Stochastic Differential Equations SCOTT · Multivariate Density Estimation: Theory, Practice, and Visualization ∗ SEARLE · Linear Models ∗ Now available in a lower priced paperback edition in the Wiley Classics Library SEARLE · Linear Models for Unbalanced Data SEARLE · Matrix Algebra Useful for Statistics SEARLE, CASELLA, and McCULLOCH · Variance Components SEARLE and WILLETT · Matrix Algebra for Applied Economics SEBER and LEE · Linear Regression Analysis, Second Edition SEBER · Multivariate Observations SEBER and WILD · Nonlinear Regression SENNOTT · Stochastic Dynamic Programming and the Control of Queueing Systems ∗ SERFLING · Approximation Theorems of Mathematical Statistics SHAFER and VOVK · Probability and Finance: Its Only a Game! SMALL and MCLEISH · Hilbert Space Methods in Probability and Statistical Inference SRIVASTAVA · Methods of Multivariate Statistics STAPLETON · Linear Statistical Models STAUDTE and SHEATHER · Robust Estimation and Testing STOYAN, KENDALL, and MECKE · Stochastic Geometry and Its Applications, Second Edition STOYAN and STOYAN · Fractals, Random Shapes and Point Fields: Methods of Geometrical Statistics STYAN · The Collected Papers of T W Anderson: 1943–1985 SUTTON, ABRAMS, JONES, SHELDON, and SONG · Methods for MetaAnalysis in Medical Research TANAKA · Time Series Analysis: Nonstationary and Noninvertible Distribution Theory THOMPSON · Empirical Model Building THOMPSON · Sampling, Second Edition THOMPSON · Simulation: A Modeler’s Approach THOMPSON and SEBER · Adaptive Sampling THOMPSON, WILLIAMS, and FINDLAY · Models for Investors in Real World Markets ˜ TIAO, BISGAARD, HILL, PENA, and STIGLER (editors) · Box on Quality and Discovery: with Design, Control, and Robustness TIERNEY · LISP-STAT: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics TSAY · Analysis of Financial Time Series UPTON and FINGLETON · Spatial Data Analysis by Example, Volume II: Categorical and Directional Data VAN BELLE · Statistical Rules of Thumb VESTRUP · The Theory of Measures and Integration VIDAKOVIC · Statistical Modeling by Wavelets WEISBERG · Applied Linear Regression, Second Edition WELSH · Aspects of Statistical Inference WESTFALL and YOUNG · Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment WHITTAKER · Graphical Models in Applied Multivariate Statistics WINKER · Optimization Heuristics in Economics: Applications of Threshold Accepting WONNACOTT and WONNACOTT · Econometrics, Second Edition WOODING · Planning Pharmaceutical Clinical Trials: Basic Statistical Principles WOOLSON and CLARKE · Statistical Methods for the Analysis of Biomedical Data, Second Edition WU and HAMADA · Experiments: Planning, Analysis, and Parameter Design Optimization YANG · The Construction Theory of Denumerable Markov Processes ∗ ZELLNER · An Introduction to Bayesian Inference in Econometrics ZELTERMAN · Discrete Distributions: Applications in the Health Sciences ZHOU, OBUCHOWSKI, and McCLISH · Statistical Methods in Diagnostic Medicine ∗ Now available in a lower priced paperback edition in the Wiley Classics Library ... 1 16 21 Generalized Least Squares Estimators 2.1 Overview 2.2 General Linear Regression Model 2.3 Generalized Least Squares Estimators 2.4 Finiteness... Takeaki Generalized least squares / Takeaki Kariya, Hiroshi Kurata p cm – (Wiley series in probability and statistics) Includes bibliographical references and index ISBN 0-470-86697-7 (alk paper) Least. .. Kendall A complete list of the titles in this series appears at the end of this volume Generalized Least Squares Takeaki Kariya Kyoto University and Meiji University, Japan Hiroshi Kurata University