Đang tải... (xem toàn văn)
THE MULTIVARIATE LINEAR REGRESSION MODEL
CHAPTER 24 The multivariate linear regression model 24.1 Introduction The multivariate linear regression model is a direct extension of the linear regression model to the case where the dependent variable is an mx random vector y, That is, the statistical GM takes the form y,= Bx, + (24.1) tel, Wy, where y,: mx 1, B: kx m, x, k x 1, u,: mx system of m linear regression equations: Ve = BX, tuy, 1=1,2, ,m, The system (1) is effectively a teT, (24.2) with B=(B,, B2,. Bn) In direct analogy with the m= case (see Chapter 19) the multivariate linear regression model will be derived from first principles based on the joint distribution of the observable random variables involved, D(Z,; ý) where Z,=(yi, Xi), (m+k)x distributed vector, Le Y: H ~N Assuming os ((c 2i 2) that Z, for allreT, T2; is an HD normally (24.3) we can proceed to define the systematic and non-systematic components by: and u.=E(y,/X,=x)=Bx, u,=y, —Ely,/X,=x,), 571 B=E;;E,, te T 244) (24.5) - $72 The multivariate linear regression model Moreover, by construction, u, and y, satisfy the following properties: (i) E(u,) = ELE(u,/X, = x,)] =0; i) Eluw)=E[EuayX,=x)]=1° '° u,u,) = E[E(uu,/X, = x,)] = ts: (iii) EUsw)=E[Eua/X,=xJ]=E[w,Etw/X,=x)]=0 (it) where O=X,¡—¡¿:ŠX;; *¿;¡ (compare 19.2) reT, these with the results in Section The similarity between the m= case and the general case allows us to consider several loose ends left in Chapter 19 The first is the use of the joint distribution D(Z,; ys) in defining the model instead of concentrating exclusively on D(y,/X,; w,) The loss of generality in postulating the form of the joint distribution is more than compensated for by the additional insight provided In practice it is often easier to ‘judge’ the plausibility of assumptions relating to the nature of D(Z,; y) rather than D(y,/X,; p,) Moreover, in misspecification analysis the relationship between the assumptions underlying the model and those underlying the random vector process {Z,, t¢ 1} enhances our understanding of the nature of the possible departures An interesting example of this is the relationship of the assumption that {Z,,reT} isa (1) normal (N); (2) independent (J); and (3) identically distributed (ID) process; and [6] [7] [8] (i) (1) (ili) ——_Dly,/X,; 4) is normal; E(y,/X,=X,) 18 linear in x,; Cov(y,/X,=x,) is homoskedastic (free of x,); 6=(B, Q) are time-invariant; {¥,/X,,t€T} ts an independent process The relationship between below: these components is shown diagrammatically (i) (N)> (i), (ID) >[7], (D> [8] (iii) The question which naturally arises is whether (i)}(ii) imply (N) or not The following lemma shows that if (i)-{ili) are supplemented by the assumption 24.1 Introduction 373 that X,~ N(0,Z,,), det(X,,)40, the reverse implication holds Lemma 24.1 Z,~ N(0,X) for te TT if and only if (i) X,~N(0,E,,), det(L,) #0; ti) E(y,/X,=X,) = 2252x,; (iii) Covly,/X;=X,) =, —Ly.E57Z>, (iii) (y,/X,)~ N(B‘X,, Q) (see Barra (1981)) The statistical GM (1) for the sample period f=1.2 T is written as Y=XB+U (24.6) whereY:T xm.X:T x k.B:k x m.U: x m The system in (1) can be viewed as the tth row of (6) The ith row taking the form y,=XB,t+u, i= 1.2 m (24.7) represents all T observations on the ith regression in (2) In order to define the conditional distribution D(Y/X; w,) we need the special notation of Kronecker products (see Appendix 2) Using this notation the matrix distribution can be written in the form (Y/# =X)~ N(XB, @ II), (24.8) where Q @I, represents the covariance of ti vec(Y)= %2 : Tmx Ym The vectoring operator vec( -) transforms a matrix into a column vector by stacking the columns of the matrix one beneath the other Using the vectoring operator we can express (6) in the form vec(Y) =(l„ @ X) vec(B) + vec(U) or y*=X,B, +, in an obvious notation (24.9) (24.10) 374 The multivariate linear regression model The multivariate linear regression (MLR) model is of considerable interest in econometrics because of its direct relationship with the simultaneous equations formulation to be considered in Chapter 25 In particular, the latter formulation can be viewed as a reparametrisation of the MLR model where the statistical parameters of interest @=(B, Q) not coincide with the theoretical parameters of interest € Instead, the two sets of parameters are related by some system of implicit equations of the form: h,(0.6)=0 i=1,2, ,p (24.11) These equations can be interpreted as providing an alternative parametrisation for the statistical GM in terms of the theoretical parameters of interest In view of this relationship between the two statistical models a sound understanding of the MLR model will pave the way for the simultaneous equations formulation in Chapter 25 24.2 Specification and estimation In direct analogy to the linear regression model (m= 1) the multivariate linear regression model is specified as follows: qd) Statistical GM: y:mxl, [1] y,=B’x,+u,, Xx;:kx1, teT B:kxm The systematic and non-systematic components are: H,= E(y,X,=x,)=Bx, u,=y,— Ety,/X,=x,), and by construction E(u,) = EL E(u,/X, =x,)]=9, E(uu,) = E[LE(wu,/X, [2] The statistical parameters X2; E¿y, Q=%,, = x,)] =0, reT of interest are 6=(B,Q) —2,2%37'E), where B= [3] [4] X, is assumed to be weakly exogenous with respect to No a prion information on Ø [5] Rank(X)=k, X =(x,, X5, , X7): T xk, for T>k 24.2 ap Specification and estimation 575 Probability model 0=| DivX a Or Pí— 3ÍW,— Bx)OQ_ 0c [6] ty c—B%,)}, **xC”"†,íe vf (i) — Dly,/Xs @) — normal; [7] (1) Ety,/X, = x,) = B’x, — linear in x,; (ili) Cov(y,/X,= X,) = — homoskedastic (free of x,); is time invariant (IW) Sampling model {8} Y=(¡,Y¿ Yr} 1S an independent sample sequentially drawn from D(y,/X,; 6), t=1,2, , T.and T2=m+k The above specification is almost identical with that of m= considered in Chapter 19 The discussion of the assumptions in the same chapter applies to [1]-[8] above with only minor modifications due to m> The only real change brought about by m> | is the increase in the number of statistical parameters of interest being mk +4m(m + 1) It should come as no surprise to learn that the similarities between the two statistical models extend to estimation, testing and prediction From assumptions [6] to [8] we can deduce function takes the form that the likelihood r 10; Y) )=c(Y) IP (y,/X,; 9) and the log likelihood is log L=const ~5, lost (det Q)— S (y, )@~!(y,—Bx,) (24.12) , B =const —4[T log(det 2) + trQ~!(Y —XB(Y—XB)] (24.13) (see exercise 1) The first-order conditions are ÈÍ19§ È _ yy xxB)Q-'=0, cB ê T Clog _T2 sy _xpy(y—xB)=0 ãQ@"! (24.14) (24.15) + C™ denotes the space of all real positive definite symmetric matrices of rank m 576 The multivariate linear regression model These first-order conditions lead to the following MLE’s: B=(XX)'!XY G=— T (24.16) CU, (24.17) where U = Y —XB For Q to be positive definite we need to assume that T > m+k (see Dykstra (1970)) It is interesting to note that (16) amounts to estimating cach regression equation separately by ,=(XX) lXy, i=1,2 m (24.18) Moreover, the residuals from these separate regressions ủ,= y,— Xổ, can be used to derive QD via @¡;=(1/T)ùjâ,, ij=1,2, m As in the case of B in the linear regression model, the MLE B preserves the original orthogonality between the systematic and non-systematic components That is, for #,=B’x, and a, =y, —Bx, V,=f8,+úủ, /=1,2 T (24.19) and #, ũ, This orthogonality can be used measure by extending R?= 1—(a'd) (yy) to to deñne a goodness-of-lit G=I—(Ù Õ)\(Y'Y)!=(Y'YT-ÙÔ)J(Y'Y)T1, (24.20) The matrix G varies between the identity matrix when U =0 and zero when Y=U (no measure explanation) In to a scalar we can [ d,=-trG, m order to d,=det(G) [ m } Mm = and this matrix goodness-of-fit (24.21) (see Hooper (1959)) In terms of the eigenvalues (4,, goodness of fit take the form dị= reduce use the trace or the determinant , ,4,,) of G the above measures of m dạ=[|] 2, ¿=1 (24.22) The orthogonality extends directly to M=XB and U and can be used to show that B and Q are independent random matrices In the present context this amounts to Cov(B @ Ô) =0, where E(:) is relative to D(V/X: 0) (24.23) 24.2 Specification and estimation 577 Finite sample properties of B and Oo From the fact that B and Q are MLE’s we can deduce that they enjoy the invariance property of such estimators (see Chapter 13) and they are functions of the minimal sufficient statistics, if they exist Using the Lehman— Scheffe result (see Chapter 12) we can see that the ratio D(Y/X; 6) | DIV gi 8) RPL 20767 1[YY —Y0Yu —(Y —YuJXB— BX(Y-Yạ]; is independent of if YY=Y’Y, a(Y)=(t,(¥),t,(¥)), defines the set of minimal and Y’'X = YX (24.24) This implies that where t,(Y)=Y’Y, t,(¥Y) =Y'X sufficient statistics and B=(X'X) '2,(YY, (24.25) ~ (24.26) Ô=.r(Y)—r(VIXX) tra), In order to discuss the other properties of B and distributions Since © let us derive their B=B+(X’X) 'X'U =B+LU, L=(XX) 1X, we can deduce that B~ N(B, Q @ (X'X)~') This is because B is (24.27) a linear function of Y where (Y/X) ~ N(XB,Q © I) (24.28) Given that TO = Y(I — M,)Y’, its distribution is the matrix equivalent to the chi-square, known as the Wishart distribution with T—k degrees of freedom and scale matrix Q and written as TQ ~ W,(Q, T—k) (see Appendix (24.29) 1) In the case where m= TÔ =ữủ TƠ~ø2z?(T—k), E(TƠ)=ø?(T—ĐI and (24.30) The Wishart distribution enjoys most of the attractive properties of the multivariate normal distribution (see Appendix 1) In direct analogy to (30), E[(TQ) =(T—k)]Q, (24.31) 578 The multivariate linear regression model and thus Õ=[1/(T—k)]ỮÔ is an unbiased estimator of Q In view of (25}-(31) we can summarise the finite sample properties of the MLE’s B and © of B and Q respectively: (1) B and are invariant (with respect to Borel functions of the form (3) (4) g(-): ©— (1