5 An unordered multinomial dependent variable In the previous chapter we considered the Logit and Probit models for a binomial dependent variable. These models are suitable for modeling bino- mial choice decisions, where the two categories often correspond to no/yes situations. For example, an individual can decide whether or not to donate to charity, to respond to a direct mailing, or to buy brand A and not B. In many choice cases, one can choose between more than two categories. For example, households usually can choose between many brands within a product category. Or firms can decide not to renew, to renew, or to renew and upgrade a maintenance contract. In this chapter we deal with quantita- tive models for such discrete choices, where the number of choice options is more than two. The models assume that there is no ordering in these options, based on, say, perceived quality. In the next chapter we relax this assumption. The outline of this chapter is as follows. In section 5.1 we discuss the representation and interpretation of several choice models: the Multinomial and Conditional Logit models, the Multinomial Probit model and the Nested Logit model. Admittedly, the technical level of this section is reasonably high. We do believe, however, that considerable detail is relevant, in particular because these models are very often used in empirical marketing research. Section 5.2 deals with estimation of the parameters of these models using the Maximum Likelihood method. In section 5.3 we discuss model evaluation, although it is worth mentioning here that not many such diag- nostic measures are currently available. We consider variable selection pro- cedures and a method to determine some optimal number of choice categories. Indeed, it may sometimes be useful to join two or more choice categories into a new single category. To analyze the fit of the models, we consider within- and out-of-sample forecasting and the evaluation of forecast performance. The illustration in section 5.4 concerns the choice between four brands of saltine crackers. Finally, in section 5.5 we deal with modeling of unobserved heterogeneity among individuals, and modeling of dynamic choice behavior. In the appendix to this chapter we give the EViews code 76 An unordered multinomial dependent variable 77 for three models, because these are not included in version 3.1 of this statis- tical package. 5.1 Representation and interpretation In this chapter we extend the choice models of the previous chapter to the case with an unordered categorical dependent variable, that is, we now assume that an individual or household i can choose between J categories, where J is larger than 2. The observed choice of the individual is again denoted by the variable y i , which now can take the discrete values 1; 2; ; J. Just as for the binomial choice models, it is usually the aim to correlate the choice between the categories with explanatory variables. Before we turn to the models, we need to say something briefly about the available data, because we will see below that the data guide the selection of the model. In general, a marketing researcher has access to three types of explanatory variable. The first type corresponds to variables that are differ- ent across individuals but are the same across the categories. Examples are age, income and gender. We will denote these variables by X i . The second type of explanatory variable concerns variables that are different for each individual and are also different across categories. We denote these variables by W i;j . An example of such a variable in the context of brand choice is the price of brand j experienced by individual i on a particular purchase occa- sion. The third type of explanatory variable, summarized by Z j , is the same for each individual but different across the categories. This variable might be the size of a package, which is the same for each individual. In what follows we will see that the models differ, depending on the available data. 5.1.1 The Multinomial and Conditional Logit models The random variable Y i , which underlies the actual observations y i , can take only J discrete values. Assume that we want to explain the choice by the single explanatory variable x i , which might be, say, age or gender. Again, it can easily be understood that a standard Linear Regression model such as y i ¼  0 þ  1 x i þ " i ; ð5:1Þ which correlates the discrete choice y i with the explanatory variable x i , does not lead to a satisfactory model. This is because it relates a discrete variable with a continuous variable through a linear relation. For discrete outcomes, it therefore seems preferable to consider an extension of the Bernoulli dis- tribution used in chapter 4, that is, the multivariate Bernoulli distribution denoted as 78 Quantitative models in marketing research Y i $ MNð1; 1 ; ; J Þð5:2Þ (see section A.2 in the Appendix). This distribution implies that the prob- ability that category j is chosen equals Pr½Y i ¼ j¼ j , j ¼ 1; ; J, with  1 þ  2 þÁÁÁþ J ¼ 1. To relate the explanatory variables to the choice, one can make  j a function of the explanatory variable, that is,  j ¼ F j ð 0;j þ  1;j x i Þ: ð5:3Þ Notice that we allow the parameter  1;j to differ across the categories because the effect of variable x i may be different for each category. If we have an explanatory variable w i;j , we could restrict  1;j to  1 (see below). For a binomial dependent variable, expression (5.3) becomes  ¼ Fð 0 þ  1 x i Þ. Because the probabilities  j have to lie between 0 and 1, the function F j has to be bounded between 0 and 1. Because it also must hold that P J j¼1  j equals 1, a suitable choice for F j is the logistic function. For this function, the probability that individual i will choose category j given an explanatory variable x i is equal to Pr½Y i ¼ jjX i ¼ expð 0;j þ  1;j x i Þ P J l¼1 expð 0;l þ  1;l x i Þ ; for j ¼ 1; ; J; ð5:4Þ where X i collects the intercept and the explanatory variable x i . Because the probabilities sum to 1, that is, P J j¼1 Pr½Y i ¼ jjX i ¼1, it can be understood that one has to assign a base category. This can be done by restricting the corresponding parameters to zero. Put another way, multiplying the numera- tor and denominator in (5.4) by a non-zero constant, for example expðÞ, changes the intercept parameters  0;j into  0;j þ  but the probability Pr½Y i ¼ jjX i  remains the same. In other words, not all J intercept para- meters are identified. Without loss of generality, one usually restricts  0;J to zero, thereby imposing category J as the base category. The same holds true for the  1;j parameters, which describe the effects of the individual- specific variables on choice. Indeed, if we multiply the nominator and denominator by expðx i Þ, the probability Pr½Y i ¼ jjX i  again does not change. To identify the  1;j parameters one therefore also imposes that  1;J ¼ 0. Note that the choice for a base category does not change the effect of the explanatory variables on choice. So far, the focus has been on a single explanatory variable and an inter- cept for notational convenience, and this will continue in several of the subsequent discussions. Extensions to K x explanatory variables are however straightforward, where we use the same notation as before. Hence, we write Pr½Y i ¼ jjX i ¼ expðX i  j Þ P J l¼1 expðX i  l Þ for j ¼ 1; ; J; ð5:5Þ An unordered multinomial dependent variable 79 where X i is a 1 ÂðK x þ 1Þ matrix of explanatory variables including the element 1 to model the intercept and  j is a ðK x þ 1Þ-dimensional parameter vector. For identification, one can set  J ¼ 0. Later on in this section we will also consider the explanatory variables W i . The Multinomial Logit model The model in (5.4) is called the Multinomial Logit model. If we impose the identification restrictions for parameter identification, that is, we impose  J ¼ 0, we obtain for K x ¼ 1 that Pr½Y i ¼ jjX i ¼ expð 0;j þ  1;j x i Þ 1 þ P JÀ1 l¼1 expð 0;l þ  1;l x i Þ for j ¼ 1; ; J À 1; Pr½Y i ¼ JjX i ¼ 1 1 þ P JÀ1 l¼1 expð 0;l þ  1;l x i Þ : ð5:6Þ Note that for J ¼ 2 (5.6) reduces to the binomial Logit model discussed in the previous chapter. The model in (5.6) assumes that the choices can be explained by intercepts and by individual-specific variables. For example, if x i measures the age of an individual, the model may describe that older persons are more likely than younger persons to choose brand j. A direct interpretation of the model parameters is not straightforward because the effect of x i on the choice is clearly a nonlinear function in the model parameters  j . Similarly to the binomial Logit model, to interpret the parameters one may consider the odds ratios. The odds ratio of category j versus category l is defined as  jjl ðX i Þ¼ Pr½Y i ¼ jjX i  Pr½Y i ¼ ljX i  ¼ expð 0;j þ  1;j x i Þ expð 0;l þ  1;l x i Þ for l ¼ 1; ; J À1;  jjJ ðx i Þ¼ Pr½Y i ¼ jjX i  Pr½Y i ¼ JjX i  ¼ expð 0;j þ  1;j x i Þ ð5:7Þ and the corresponding log odds ratios are log  jjl ðX i Þ¼ð 0;j À  0;l Þþð 1;j À  1;l Þx i for l ¼ 1; ; J À 1; log  jjJ ðX i Þ¼ 0;j þ  1;j x i : ð5:8Þ Suppose that the  1;j parameters are equal to zero, we then see that positive values of  0;j imply that individuals are more likely to choose category j than the base category J. Likewise, individuals prefer category j over category l if ð 0;j À  0;l Þ > 0. In this case the intercept parameters correspond with the 80 Quantitative models in marketing research average base preferences of the individuals. Individuals with a larger value for x i tend to favor category j over category l if ð 1;j À  1;l Þ > 0 and the other way around if ð 1;j À  1;l Þ < 0. In other words, the difference ð 1;j À  1;l Þ measures the change in the log odds ratio for a unit change in x i . Finally, if we consider the odds ratio with respect to the base category J, the effects are determined solely by the parameter  1;j . The odds ratios show that a change in x i may imply that individuals are more likely to choose category j compared with category l. It is important to recognize, however, that this does not necessarily mean that Pr ½Y i ¼ jjX i  moves in the same direction. Indeed, owing to the summation restriction, a change in x i also changes the odds ratios of category j versus the other categories. The net effect of a change in x i on the choice probability follows from the partial derivative of Pr½Y i ¼ jjX i  with respect to x i , which is given by @ Pr½Y i ¼ jjX i  @x i ¼ 1 þ P JÀ1 l¼1 expð 0;l þ  1;l x i Þ  expð 0;j þ  1;j Þ 1;j x i 1 þ P JÀ1 l¼1 expð 0;l þ  1;l x i Þ  2 À expð 0;j þ  1;j x i Þ P JÀ1 l¼1 expð 0;l þ  1;l x i Þ 1;l 1 þ P JÀ1 l¼1 expð 0;l þ  1;l x i Þ  2 ¼ Pr½Y i ¼ jjX i   1;j À X JÀ1 l¼1  1;l Pr½Y i ¼ lj X i  ! : ð5:9Þ The sign of this derivative now depends on the sign of the term in parenth- eses. Because the probabilities depend on the value of x i , the derivative may be positive for some values of x i but negative for others. This phenomenon can also be observed from the odds ratios in (5.7), which show that an increase in x i may imply an increase in the odds ratio of category j versus category l but a decrease in the odds ratio of category j versus some other category s 6¼ l. This aspect of the Multinomial Logit model is in marked contrast to the binomial Logit model, where the probabilities are monoto- nically increasing or decreasing in x i . In fact, note that for only two cate- gories (J ¼ 2) the partial derivative in (5.9) reduces to Pr½Y i ¼ 1jX i ð1 À Pr½Y i ¼ 1jX i Þ 1;j : ð5:10Þ Because obviously  1;j ¼  1 , this is equal to the partial derivative in a bino- mial Logit model (see (4.19)). An unordered multinomial dependent variable 81 The quasi-elasticity of x i , which can also be useful for model interpreta- tion, follows directly from the partial derivative (5.9), that is, @ Pr½Y i ¼ jjX i  @x i x i ¼ Pr½Y i ¼ jj X i   1;j À X JÀ1 l¼1  1;l Pr½Y i ¼ ljX i  ! x i : ð5:11Þ This elasticity measures the percentage point change in the probability that category j is preferred owing to a percentage increase in x i . The summation restriction concerning the J probabilities establishes that the sum of the elasticities over the alternatives is equal to zero, that is, X J j¼1 @ Pr½Y i ¼ jjX i  @x i x i ¼ X J j¼1 Pr½Y i ¼ jjX i  1;j x i À X J j¼1 ðPr½Y i ¼ jjX i  X JÀ1 l¼1  1;l Pr½Y i ¼ ljX i x i Þ ¼ X JÀ1 j¼1 Pr½Y i ¼ jjX i  1;j x i À X JÀ1 l¼1 ðPr½Y i ¼ ljX i  1;l x i ð X J j¼1 Pr½Y i ¼ jjX i ÞÞ ¼ 0; ð5:12Þ where we have used  1;J ¼ 0. Sometimes it may be useful to interpret the Multinomial Logit model as a utility model, thereby building on the related discussion in section 4.1 for a binomial dependent variable. Suppose that an individual i perceives utility u i;j if he or she chooses category j, where u i;j ¼  0;j þ  1;j x i þ " i;j ; for j ¼ 1; ; J ð5:13Þ and " i;j is an unobserved error variable. It seems natural to assume that individual i chooses category j if he or she perceives the highest utility from this choice, that is, u i;j ¼ maxðu i;1 ; ; u i;J Þ: ð5:14Þ The probability that the individual chooses category j therefore equals the probability that the perceived utility u i;j is larger than the other utilities u i;l for l 6¼ j, that is, Pr½Y i ¼ jjX i ¼Pr½u i;j > u i;1 ; ; u i;j > u i;jÀ1 ; u i;j > u i;jþ1 ; ; u i;j > u i;J jX i : ð5:15Þ 82 Quantitative models in marketing research The Conditional Logit model In the Multinomial Logit model, the individual choices are corre- lated with individual-specific explanatory variables, which take the same value across the choice categories. In other cases, however, one may have explanatory variables that take different values across the choice options. One may, for example, explain brand choice by w i;j , which denotes the price of brand j as experienced by household i on a particular purchase occasion. Another version of a logit model that is suitable for the inclusion of this type of variable is the Conditional Logit model, initially proposed by McFadden (1973). For this model, the probability that category j is chosen equals Pr½Y i ¼ jjW i ¼ expð 0;j þ  1 w i;j Þ P J l¼1 expð 0;l þ  1 w i;l Þ for j ¼ 1; ; J: ð5:16Þ For this model the choice probabilities depend on the explanatory variables denoted by W i ¼ðW i;1 ; ; W i;J Þ, which have a common impact  1 on the probabilities. Again, we have to set  0;J ¼ 0 for identification of the intercept parameters. However, the  1 parameter is equal for each category and hence it is always identified except for the case where w i;1 ¼ w i;2 ¼ ¼ w i;J . The choice probabilities in the Conditional Logit model are nonlinear functions of the model parameter  1 and hence again model interpretation is not straightforward. To understand the effect of the explanatory variables, we again consider odds ratios. The odds ratio of category j versus category l is given by  jjl ðW i Þ¼ Pr½Y i ¼ jjW i  Pr½Y i ¼ ljW i  ¼ expð 0;j þ  1 w i;j Þ expð 0;l þ  1 w i;l Þ for l ¼ 1; ; J ¼ expðð 0;j À  0;l Þþ 1 ðw i;j À w i;l ÞÞ ð5:17Þ and the corresponding log odds ratio is log  jjl ðW i Þ¼ð 0;j À  0;l Þþ 1 ðw i;j À w i;l Þ for l ¼ 1; ; J: ð5:18Þ The interpretation of the intercept parameters is similar to that for the Multinomial Logit model. Furthermore, for positive values of  1 , individuals favor category j more than category l for larger positive values of ðw i;j À w i;l Þ. For  1 < 0, we observe the opposite effect. If we consider a brand choice problem and w i;j represents the price of brand j, a negative value of  1 means that households are more likely to buy brand j instead of brand l as brand l gets increasingly more expensive. Due to symmetry, a unit change in w i;j leads to a change of  1 in the log odds ratio of category j versus l and a change of À 1 in the log odds ratio of l versus j. An unordered multinomial dependent variable 83 The odds ratios for category j (5.17) show the effect of a change in the value of the explanatory variables on the probability that category j is chosen compared with another category l 6¼ j. To analyze the total effect of a change in w i;j on the probability that category j is chosen, we consider the partial derivative of Pr½Y i ¼ jjW i  with respect to w i;j , that is, @ Pr½Y i ¼ jjW i  @w i;j ¼ P J l¼1 expð 0;l þ  1 w i;l Þexpð 0;j þ  1 w i;j Þ 1 P J l¼1 expð 0;l þ  1 w i;l Þ  2 À expð 0;j þ  1 w i;j Þexpð 0;j þ  1 w i;j Þ 1 P J l¼1 expð 0;l þ  1 w i;l Þ  2 ¼  1 Pr½Y i ¼ jjW i ð1 À Pr½Y i ¼ jjW i Þ: ð5:19Þ This partial derivative depends on the probability that category j is chosen and hence on the values of all explanatory variables in the model. The sign of this derivative, however, is completely determined by the sign of  1 . Hence, in contrast to the Multinomial Logit specification, the probability varies monotonically with w i;j . Along similar lines, we can derive the partial derivative of the probability that an individual i chooses category j with respect to w i;l for l 6¼ j, that is, @ Pr½Y i ¼ jjW i  @w i;l ¼À 1 Pr½Y i ¼ jjW i Pr½Y i ¼ lj W i : ð5:20Þ The sign of this cross-derivative is again completely determined by the sign of À 1 . The value of the derivative itself also depends on the value of all explanatory variables through the choice probabilities. Note that the sym- metry @ Pr½Y i ¼ jjW i =@w i;l ¼ @ Pr½Y i ¼ lj W i =@w i;j holds. If we consider brand choice again, where w i;j corresponds to the price of brand j as experi- enced by individual i, the derivatives (5.19) and (5.20) show that for  1 < 0 an increase in the price of brand j leads to a decrease in the probability that brand j is chosen and an increase in the probability that the other brands are chosen. Again, the sum of these changes in choice probabilities is zero because 84 Quantitative models in marketing research X J j¼1 @ Pr½Y i ¼ jjW i  @w i;l ¼  1 Pr½Y i ¼ ljW i ð1 À Pr½Y i ¼ ljW i Þ þ X J j¼1;j6¼l À 1 Pr½Y i ¼ jjW i Pr½Y i ¼ ljW i ¼0; ð5:21Þ which simply confirms that the probabilities sum to one. The magnitude of each specific change in choice probability depends on  1 and on the prob- abilities themselves, and hence on the values of all w i;l variables. If all w i;l variables change similarly, l ¼ 1; ; J, the net effect of this change on the probability that, say, category j is chosen is also zero because it holds that X J l¼1 @ Pr½Y i ¼ jjW i  @w i;l ¼  1 Pr½Y i ¼ ljW i ð1 À Pr½Y i ¼ ljW i Þ þ X J l¼1;l6¼j À 1 Pr½Y i ¼ jjW i Pr½Y i ¼ ljW i ¼0; ð5:22Þ where we have used P J l¼1;l6¼j Pr½Y i ¼ ljW i ¼1 À Pr½Y i ¼ ljW i . In marketing terms, for example for brand choice, this means that the model implies that an equal price change in all brands does not affect brand choice. Quasi-elasticities and cross-elasticities follow immediately from the above two partial derivatives. The percentage point change in the probability that category j is chosen upon a percentage change in w i;j equals @ Pr½Y i ¼ jjW i  @w i;j w i;j ¼  1 w i;j Pr½Y i ¼ jj W i ð1 À Pr½Y i ¼ jjW i Þ: ð5:23Þ The percentage point change in the probability for j upon a percentage change in w i;l is simply @ Pr½Y i ¼ jjW i  @w i;l w i;l ¼À 1 w i;l Pr½Y i ¼ jjW i Pr½Y i ¼ ljW i : ð5:24Þ Given (5.23) and (5.24), it is easy to see that X J j¼1 @ Pr½Y i ¼ jjW i  @w i;l w i;l ¼ 0 and X J l¼1 @ Pr½Y i ¼ jjw i  @w i;l w i;l ¼ 0; ð5:25Þ and hence the sum of all elasticities is equal to zero. An unordered multinomial dependent variable 85 A general logit specification So far, we have discussed the Multinomial and Conditional Logit models separately. In some applications one may want to combine both models in a general logit specification. This specification can be further extended by including explanatory variables Z j that are different across categories but the same for each individual. Furthermore, it is also possible to allow for different  1 parameters for each category in the Conditional Logit model (5.16). Taking all this together results in a general logit speci- fication, which for one explanatory variable of either type reads as Pr½Y i ¼ jjX i ; W i ; Z¼ expð 0;j þ  1;j x i þ  1;j w i;j þ z j Þ P J l¼1 expð 0;l þ  1;l x i þ  1;l w i;l þ z l Þ ; for j ¼ 1; ; J; ð5:26Þ where  0;J ¼  1;J ¼ 0 for identification purposes and Z ¼ðz 1 ; ; z J Þ. Note that it is not possible to modify  into  j because the z j variables are in fact already proportional to the choice-specific intercept terms. The interpretation of the logit model (5.26) follows again from the odds ratio Pr½Y i ¼ jjx i ; w i ; z Pr½Y i ¼ ljx i ; w i ; z ¼ expðð 0;j À  0;l Þþð 1;j À  1;l Þ x i þ  1;j w i;j À  1;l w i;l þ ðz j À z l ÞÞ: ð5:27Þ For most of the explanatory variables, the effects on the odds ratios are the same as in the Conditional and Multinomial Logit model specification. The exception is that it is not the difference between w i;j and w i;l that affects the odds ratio but the linear combination  1;j w i;j À  1;l w i;l . Finally, partial deri- vatives and elasticities for the net effects of changes in the explanatory vari- ables on the probabilities can be derived in a manner similar to that for the Conditional and Multinomial Logit models. Note, however, that the sym- metry @ Pr½Y i ¼ jj X i ; W i ; Z=@w i;l ¼ @ Pr½Y i ¼ ljX i ; W i ; Z=@w i;j does not hold any more. The independence of irrelevant alternatives The odds ratio in (5.27) shows that the choice between two cate- gories depends only on the characteristics of the categories under considera- tion. Hence, it does not relate to the characteristics of other categories or to the number of categories that might be available for consideration. Naturally, this is also true for the Multinomial and Conditional Logit mod- els, as can be seen from (5.7) and (5.17), respectively. This property of these models is known as the independence of irrelevant alternatives (IIA). [...]... the corresponding parameters to zero Put another way, multiplying the numerator and denominator in (5. 4) by a non-zero constant, for example expðÞ, changes the intercept parameters 0;j into 0;j þ  but the probability Pr½Yi ¼ jjXi Š remains the same In other words, not all J intercept parameters are identified Without loss of generality, one usually restricts 0;J to zero, thereby imposing category J... is, we impose J ¼ 0, we obtain for Kx ¼ 1 that Pr½Yi ¼ jjXi Š ¼ Pr½Yi ¼ JjXi Š ¼ expð0;j þ 1;j xi Þ PJÀ1 1 þ l¼1 expð0;l þ 1;l xi Þ 1þ PJÀ1 l¼1 1 expð0;l þ 1;l xi Þ for j ¼ 1; ; J À 1; : 5: 6Þ Note that for J ¼ 2 (5. 6) reduces to the binomial Logit model discussed in the previous chapter The model in (5. 6) assumes that the choices can be explained by intercepts and by individual-specific variables... variable where Xi is a 1  ðKx þ 1Þ matrix of explanatory variables including the element 1 to model the intercept and j is a ðKx þ 1Þ-dimensional parameter vector For identification, one can set J ¼ 0 Later on in this section we will also consider the explanatory variables Wi The Multinomial Logit model The model in (5. 4) is called the Multinomial Logit model If we impose the identification restrictions... the effects of the individualspecific variables on choice Indeed, if we multiply the nominator and denominator by expðxi Þ, the probability Pr½Yi ¼ jjXi Š again does not change To identify the 1;j parameters one therefore also imposes that 1;J ¼ 0 Note that the choice for a base category does not change the effect of the explanatory variables on choice So far, the focus has been on a single explanatory... been on a single explanatory variable and an intercept for notational convenience, and this will continue in several of the subsequent discussions Extensions to Kx explanatory variables are however straightforward, where we use the same notation as before Hence, we write expðXi j Þ Pr½Yi ¼ jjXi Š ¼ PJ l¼1 expðXi l Þ for j ¼ 1; ; J; 5: 5Þ 79 An unordered multinomial dependent variable where Xi is a 1... For example, if xi measures the age of an individual, the model may describe that older persons are more likely than younger persons to choose brand j A direct interpretation of the model parameters is not straightforward because the effect of xi on the choice is clearly a nonlinear function in the model parameters j Similarly to the binomial Logit model, to interpret the parameters one may consider... that individual i will choose category j given an explanatory variable xi is equal to expð0;j þ 1;j xi Þ ; Pr½Yi ¼ jjXi Š ¼ PJ l¼1 expð0;l þ 1;l xi Þ for j ¼ 1; ; J; 5: 4Þ where Xi collects the intercept and the explanatory variable xi Because the P probabilities sum to 1, that is, J Pr½Yi ¼ jjXi Š ¼ 1, it can be understood j¼1 that one has to assign a base category This can be done by restricting . ; u i;j > u i;J jX i : 5: 15 82 Quantitative models in marketing research The Conditional Logit model In the Multinomial Logit model, the individual choices are corre- lated with individual-specific. detail is relevant, in particular because these models are very often used in empirical marketing research. Section 5. 2 deals with estimation of the parameters of these models using the Maximum Likelihood. three models, because these are not included in version 3.1 of this statis- tical package. 5. 1 Representation and interpretation In this chapter we extend the choice models of the previous chapter to