Đang tải... (xem toàn văn)
... 48106-1346 The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools Anna Rosefsky Saavedra Ad hoc committee Meira Levinson Richard Murnane... that the IB Diploma Program is a cost-effective way to increase high-school graduation rates 1 The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of. .. of Contents Abstract v The International Baccalaureate Diploma Program The IB Diploma Program in Chicago Public Schools Research Questions Research Design DATASET SAMPLE MEASURES ANALYTIC METHODS
The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools Anna Rosefsky Saavedra Advisor Meira Levinson A Thesis Presented to the Faculty of the Graduate School of Education of Harvard University in Partial Fulfillment of the Requirements for the Degree of Doctor of Education 2011 UMI Number: 3485998 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. UMI Dissertation Publishing UMI 3485998 Copyright 2011 by ProQuest LLC. All rights reserved. This edition of the work is protected against unauthorized copying under Title 17, United States Code. ProQuest LLC 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106-1346 The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools Anna Rosefsky Saavedra Ad hoc committee Meira Levinson Richard Murnane John Willett A Thesis Presented to the Faculty of the Graduate School of Education of Harvard University in Partial Fulfillment of the Requirements for the Degree of Doctor of Education 2011 Vita Anna Rosefsky Saavedra 1994-98 Yale University New Haven, CT 1998-99 High School Teacher, History Pinewood School Los Altos, CA 1999-05 Educational Programs Manager EF Educational Tours Boston, MA 2005-06 Graduate School of Education Harvard University 2006-11 Doctor of Education candidate Graduate School of Education Harvard University 2006-09 Teaching Fellow Graduate School of Education Harvard University 2009-10 Fulbright Student Scholar Bogota, Colombia B.A. June 1998 Ed.M. May 2006 © Anna Rosefsky Saavedra 2011 All Rights Reserved Acknowledgements Without the support of many people, I would not have been able to begin or finish this project. I will always be thankful to every one of them for the roles they played in enabling me to write this dissertation. Even after studying the impact of enrollment in the International Baccalaureate Diploma Program for past three years, I still thoroughly enjoy research of this topic and am keen to continue it as part of my future research agenda! Specifically, I am grateful to Sara Leven from Chicago Public Schools (CPS) for providing the support necessary to begin this project in the first place and for addressing my ongoing questions throughout its duration. Thank you as well to Amy Novell for approving my use of CPS data. Many thanks to Sue Sporte and Vanessa Coca from the Consortium on Chicago School Research for providing me with the data I use in this project and for also addressing my ongoing questions. I am grateful for generous financial support from the Center for the Advancement and Study of International Education, the Fulbright Foundation, the Harvard Frederick Sheldon Fund Travel Fellowship and the Harvard Graduate School of Education (HGSE). Financial resources from these organizations made it possible for me to dedicate a substantial portion of my time over the past two years to this dissertation. Several of my doctoral colleagues and friends offered valuable feedback to previous versions of this dissertation, including Liz Dawes Duraisingh, Rebecca Holcombe, Qian Guo, Maria Elena Ortega and Justin Reich. I also feel gratitude toward particularly supportive HGSE professors David Perkins and Eileen McGowan, who, over my years at HGSE, became friends as well as people for whom I have the utmost respect. To my dear friend Leila Morsy Eckert, I will forever be thankful the fates threw us ii together as statistics partners. I am so fortunate to have gone through HGSE with Leila and look forward to our annual "road race," no matter where we both happen to be living. I especially thank my advisors Meira Levinson, Richard Murnane and John Willett for supporting me through the dissertation, from beginning to end. Individually, each helped me with specific parts of the process and, in sum, their advice, time and support were critical to my completion of the doctoral program and this dissertation. I will miss each of them but will stay in touch and will keep their advice with me wherever I go. To Meira in particular, I will always be grateful for taking me on as her first doctoral student and then for guiding me through the doctoral program with enthusiasm and style. I could not have found a more encouraging, loyal and caring advisor, who will always by my role model, mentor and friend. I am also so thankful to my parents and sister. It was only by modeling my father's work ethic and determination that I was able to successfully finish my dissertation. For his example and for his support in so many other ways I will always be thankful. I thank my mother for many, many hours of listening, emphasizing and understanding. She boosted me up when I was down and her face shone with happiness and pride when I shared good news. I thank my sister Heather for her unwavering confidence in my ability to finish my dissertation. She is and has always been there for encouragement or a celebration. To each one of you, I am thankful for your love and support, I am so fortunate to have you as my family. And most of all, I am thankful to my dearly beloved husband, Juan Esteban, who has played all of the roles I describe above and is my best friend and lifelong partner in all things. In addition to editing drafts, writing code, and otherwise helping me with the iii technical aspects of writing a dissertation, his steadfast love and confidence in me provided me with the foundation from which I was able to progress through my doctoral studies and eventually complete my dissertation. He writes at the end of his dissertation that he is thankful to me for conquering every bit of his dissertation milestone with me. I could not feel more similarly with regards to my dissertation and all we have experienced, enjoyed and conquered together. iv Table of Contents Abstract v The International Baccalaureate Diploma Program 1 The IB Diploma Program in Chicago Public Schools 4 Research Questions 7 Research Design 8 DATASET SAMPLE MEASURES ANALYTIC METHODS Results THE IB ENROLLMENT PROCESS RQ #1: THE IMPACT OF IB ENROLLMENT ON STUDENTS' ACADEMIC ACHIEVEMENT RQ #2: THE IMPACT OF IB ENROLLMENT ON STUDENTS' PROBABILITY OF HIGH-SCHOOL GRADUATION RQ #3: THE IMPACT OF IB ENROLLMENT ON STUDENTS' PROBABILITY OF COLLEGE ENROLLMENT RQ #4: GENDER DIFFERENCES IN THE IMPACT OF IB ENROLLMENT Threats to Validity 8 9 9 13 17 17 21 22 23 23 24 CHALLENGE #1: SELECTION BIAS 24 CHALLENGE #2: IB ENROLLMENT INCREASES THE PROBABILITY THAT STUDENTS TAKE THE ACT EXAMINATION 29 CHALLENGE #3: IB ENROLLMENT INCREASES THE PROBABILITY THAT STUDENTS GRADUATE FROM HIGH SCHOOL 30 CHALLENGE #4: DIFFERENTIAL PROBABILITY OF MISSING SEVENTH-GRADE ITBS SCORES ..31 CHALLENGE #5: EXTERNAL VALIDITY 32 Discussion 32 Cost of the IB Diploma Program Effects 36 Conclusion 38 References 40 Tables and Figures 44 Vita 52 V Abstract In this study, I examine whether eleventh-grade students' enrollment in the Chicago Public Schools (CPS) International Baccalaureate (IB) Diploma Program improves their academic achievement as measured by their ACT examination scores, probability of high-school graduation and probability of college enrollment. CPS offers the IB Diploma Program in thirteen high schools, more than twice any other U.S. school district. Using data on the IB enrollment status of 20,422 students attending these thirteen high schools from 2002-2008,1 estimate that IB enrollment increases students' academic achievement by as much as 0.5 standard deviations and their probability of high-school graduation and college enrollment by as much as 17 and 22 percentage points respectively. All of my estimates are highly robust to validity threats posed by self-selection into IB enrollment. All estimates are greater for boys than for girls. I also calculate that the IB Diploma Program is a cost-effective way to increase high-school graduation rates. 1 The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools Originally founded as a private means for diplomats' children to earn an internationally recognized high-school diploma, today the International Baccalaureate (IB) Diploma Program is one of the fastest growing curricular innovations in U.S. publicschool districts. In schools accredited as "IB World Schools" by the International Baccalaureate Organization (IBO), teachers use IB curriculum and pedagogy to teach a range of courses that are intended to prepare IB-enrolled students for college. In this study, I examine whether IB enrollment does in fact prepare students for college. Specifically, I ask whether enrollment in the IB Diploma Program increases students' academic achievement as measured by ACT college admissions examination scores,1 probability of high-school graduation and probability of college enrollment and whether my estimates differ by gender. The International Baccalaureate Diploma Program Over the past fifteen years, the number of U.S. schools that implement IB programs has increased nearly tenfold, from 133 in 1994 to 1,218 in 2011. By 2009, public schools offered over 90% of U.S. high-school level IB programs (IBO, 2010). What began as a program that a few private schools offered to wealthy children, today reaches a much broader student audience. This growth in IB implementation is based predominantly on the program's perceived academic rigor and success as a college-preparatory intervention (Byrd, Ellington, Gross, Jago, & Stern, 2007; Cech, 2008; Mathews & Hill, 2005). This 1 The ACT organization administers the ACT college admissions examination, which includes English, mathematics, reading and science sections (ACT, 2010). 2 reputation, augmented through recent governmental support, suggests that the growth trend will continue. For example, through the reauthorization of the Elementary and Secondary Schools Act, the U.S. Department of Education has proposed to fund IB programs as part of its objective to "increase access to accelerated learning opportunities" (U.S. Department of Education, 2011a, 29). State-level education departments are also promoting IB implementation as part of their "Race to the Top" strategies (e.g. FL, MA, ME, NH; U.S. Department of Education, 201 lb). The IB school-level accreditation process and subsequent requirements for students and teachers are rigorous (Byrd et al., 2007; Mathews & Hill, 2005; U.S. Department of Education, 2010). To earn accreditation from the IBO, schools must demonstrate adherence to IBO's curricular, pedagogical, mission-based and ongoing professional-development requirements as stated in the most current version of the "Handbook of procedures for the Diploma Program" (IBO, 2009). Typically, the IB accreditation process requires extensive faculty and staff participation, takes several years to complete and, if successful, results in authorized "IB World School" status. IB WorldSchool teachers must use IB curricular and pedagogical materials to teach IB courses, and participate in ongoing professional development and self-review processes. From a financial perspective, as of 2010, IB World-School accreditation costs schools approximately $9,500 per year during the two years of candidacy. Thereafter, accredited schools pay annual fees—of $9,6000 USD in 2010—to access IB curriculum and support. These fees do not differ according to the number of students estimated to enroll in IB courses, once the school is accredited. IB World Schools offering the Diploma Program must also provide a way for students to take the IB Diploma 3 examinations at the end of their twelfth grade. Per-student, as of 2010, registration to take IB examinations costs $135 and the examinations themselves cost $92 per subject (IBO, 2010).2 In some schools, the students pay for the examination fees, while in others, the schools themselves pay the fees. Students enroll in the official IB Diploma Program in eleventh grade. Over the course of their final two years of high school, to be eligible to earn the IB Diploma, students must: enroll in six core IB courses and the IB Theory of Knowledge (TOK) epistemology course; participate in the weekly Creativity, Action and Service (CAS) requirement; and write a 4,000-word "extended essay." They must also score above a defined threshold on IBO-created and administered examinations in six subjects including languages, social studies, experimental sciences and mathematics. 9,000 IBcertified examiners, in 121 countries worldwide, assess the examinations (IBO, 2010). Despite the IB program's rapid expansion, little is known about whether enrollment in the IB Diploma Program improves students' academic outcomes, including their high-school academic achievement, probability of high-school graduation and/or subsequent probability of college enrollment (i.e.: Foust, Hertberg-Davis & Callahan, 2009; Jackson, 2010; Kyberg, Davis & Callahan, 2007; Roderick, Nagaoka, Coca & Moeller, 2009). To date, only one study has examined the relationship between IB Diploma Program enrollment and students' high-school academic achievement and subsequent college entry (Roderick et al, 2009). The purpose of that study, however, was to examine whether Chicago Public Schools' (CPS) students who enrolled in the IB Diploma Program, Advanced Placement or other honors-level courses, then enrolled in 2 The IBO tends to increase its fees on an annual basis. 4 colleges that were appropriately competitive. In contrast to the present study, according to Roderick et al, "[their] report is not intended to be a rigorous evaluation of selective enrollment schools, IB programs or AP initiatives (2009, 2)." The IB Diploma Program in Chicago Public Schools In this study, I examine whether eleventh-grade CPS students' enrollment in the IB program improves their academic achievement, probability of high-school graduation and probability of college enrollment. CPS is an ideal setting in which to address this issue for three reasons. First, CPS, at the forefront of IB implementation nationwide, offers the IB Diploma Program in thirteen high schools, more than twice any other U.S. school district.3 Next, the system retains complete historical records of its students' IB enrollment status, achievement test scores, high-school graduation status and collegeenrollment information. Finally, since the average CPS student is low income and minority, evidence documenting the IB enrollment impact in this setting would enhance our knowledge of ways to improve secondary education for disadvantaged urban youth. In the early 1990s, levels of academic achievement in most CPS high schools were dismal. Ninth- and eleventh-grade students' achievement scores revealed academic performance that was, on average, more than a year behind grade level when compared to national averages (Chicago School Reform Board of Trustees, 1999). The district's highschool graduation rate in 1995 was less than 50% of an entering cohort, relative to the national average of 76% (Heckman & LaFontaine, 2010).4 Among Illinois students who 3 www.ibo.org. retrieved 5-30-09. Indianapolis currently has six high school level IB programs, followed by five in Philadelphia and four in St. Paul, Minnesota. See Heckman & LaFontaine (2010) for a discussion of U.S. high school graduation rates. 5 took the 1994-1995 ACT college-entrance examination—50% of CPS seniors and 57% statewide—CPS students lagged behind their peers statewide by nearly five points out of a maximum score of thirty-six (Rice, 1995). Boys who attended CPS neighborhood schools fared particularly poorly. For example, in the population of CPS students who were thirteen years old in 1998, only 39%o of African-American, 51% of Latino and 55% of White boys had graduated from high school by 2004, when this cohort was nineteen years old. In comparison, 57% of African-American girls, 65% of Latino girls and 71% of White girls in the 1998 cohort graduated by 2004 (Allensworth, 2005). Arguably, the need for school reform in CPS was even greater for boys than for girls. From 1981 through 1997, Lincoln Park was the only public, non-magnet high school, out of more than 90 in CPS, to offer the IB Diploma Program. From its inception, Lincoln Park IB students' academic achievement far surpassed that of students from other neighborhood high schools and even from most of the CPS selective-enrollment high schools (Roderick et al, 2009). Driven by Lincoln Park's IB success and support from the Chicago mayor and business community, beginning in 1997, CPS increased rapidly the number of IBaccredited programs offered in Chicago neighborhood high schools. CPS's primary goal for widely implementing the IB Diploma Program was to increase predominantly lowincome and minority students' access to academically rigorous curricula in neighborhood high schools and, thereby, to increase their academic achievement and college preparedness (Spittle, Leven, & Roderick, 2008). 6 The thirteen CPS high-school level IB programs serve primarily low-income, minority students: CPS administrative data show that 65% of CPS students enrolled in IB are eligible to receive free or reduced price lunch and 74% are racial minorities. Among non-IB (eleventh grade) students in the district, in comparison, 78% are eligible to receive free- or reduced-price lunch and 85% are racial minorities. Therefore, both groups are majority-minority and low-income, but compared to non-IB CPS students, the probability is greater that CPS IB students will be White and higher income. The IB Organization (IBO) does not dictate how schools determine student eligibility for enrollment in IB programs. The CPS policy is to require all students who think they would like to enroll in the official eleventh- through twelfth-grade IB Diploma Program in their geographically determined neighborhood high school to apply in their spring of eighth grade so that they can participate in an IB-preparation track during ninth and tenth grade. The nature of the preparation track differs by CPS high school. Several offer accredited IB-preparation programs (the IB Middle Years Program), while others offer AP courses or other honors classes. CPS requires that students submit their seventh-grade Illinois Test of Basic Skills (ITBS) mathematics and reading scores as part of the IB application requirement. For the first few years of IB Diploma Program implementation, the stated CPS policy was to admit only students scoring above the sixtieth percentile on both tests. This policy has evolved such that today the "unofficial" school-level policy is to admit only students scoring above the fiftieth percentiles on both tests. Research Questions Currently, policy makers, administrators, teachers, parents and students lack credible evidence on whether enrollment in the IB Diploma Program increases students' academic achievement, probability of high-school graduation and probability of college enrollment and whether effects might differ by gender. Such evidence could inform decisions about future investments in the IB Diploma Program throughout Chicago, and more broadly, in U.S. public schools. This deficiency motivates my research questions: • RQ#1: A) Does CPS students' enrollment in the IB Diploma Program in eleventh grade increase the probability that they will take the ACT college entrance examination? B) Does CPS students' enrollment in the IB Diploma Program in eleventh grade improve their high-school achievement, as measured by their ACT scores? • RQ #2: Does CPS students' enrollment in the IB Diploma Program in eleventh grade increase the probability that they will graduate from high school? • RQ #3: Does CPS students' enrollment in the IB program in eleventh grade increase the probability that they will enroll in a two- or four-year college at some point during the first two years after their graduation from high school? • RQ #4: Do potential effects of IB enrollment on students' probability of taking the ACT examination, academic achievement as measured by ACT scores, probability of graduation from high school and probability of enrollment in college, respectively, differ by gender? 8 Research Design Dataset The data that I analyze here contain information merged from two sources, from the Chicago Public Schools (CPS) themselves and from the National Student Clearinghouse (NSC), a non-profit organization that verifies students' enrollment in, and graduation from, 92 percent of U.S. colleges (National Student Clearinghouse, 2011).5 The CPS dataset contains records for the 20,422 students who attended one of the thirteen CPS high schools that offered the IB Diploma Program to the 2002-2003, 2003-2004, 2004-2005 and 2006-2007 eleventh-grade cohorts.6 It contains measures of students' IB enrollment status, previous academic performance, gender, race/ethnicity and family income, as well as students' mathematics, English, reading and science scores on the ACT test, and indicators of whether they graduated from high school. Data on students are included in the NSC dataset if they enrolled in a college that is included among the ninety-three percent of colleges whose admissions data is collated 5 The majority of CPS students who enroll in college attend postsecondary institutions located in Illinois (Roderick, 2006). According to the NSC's Assistant Director of Research Services, "the NSC post-secondary enrollment data coverage for students in Illinois is higher than the national average of 92%" (personal correspondence, February 23, 2011). By cross-referencing NSC data with CPS exit survey data, Roderick, Nagaoka, Allensworth, Stoker, Correa, & Coca (2006) estimate that the NSC data "misses approximately 5 percent of CPS graduates who may be enrolled in college" (cited in Roderick, 2006, footnote 56). Roderick et al also characterize the Illinois colleges for which the NSC does not verify enrollment as "primarily local proprietary and technical institutions" (2006). Contingent upon the assumption that compared to IB students, the probability is greater that non-IB students will attend local proprietary and technical institutions, missing NSC enrollment data could contribute to a slight understatement of the percentage of non-IB students who enroll in college. I do not have access to records for the 2005-2006 eleventh grade cohort. I have no reason to believe that the results of my analysis would differ if I could include those records in my full sample. 9 in the National Student Clearinghouse at some point during the first four semesters postgraduation from high school. I located fifty percent of the 20,422 students in the CPS sample in the NSC dataset, indicating that at least half of CPS students in my full sample enrolled in two- or four-year college for at least one semester during the first four semesters after their graduation from high school. Sample My full sample of CPS students (n=20,422) contains every eleventh-grade student in each of the thirteen CPS high schools that offered the IB program to the eleventh-grade cohort in academic years 2002-2003 (n=5042), 2003-2004 (n=5,097), 2004-2005 (n=5,181) and 2006-2007 (n=5,102). I use data from these years because college enrollment data is available on each of these cohorts. I combine four annual cohorts of students to improve the statistical power of my analyses and enhance the external validity of my findings. Measures Outcomes In Table 1, I present descriptive statistics on my outcomes, question predictors and covariates for the full sample of students included in my CPS dataset (n=20,422). The first of my outcome variables is a measure of student achievement, which I label ACT, and which I formed by conducting a principal-components analysis (PCA) of eleventh-grade students' mathematics, English, reading and science scores on the ACT college entrance examination. The resulting continuous variable is a symmetrically distributed, weighted linear composite with an estimated Cronbach's alpha internal- 10 consistency reliability coefficient of 0.89. In total, 2,667 students—1 percent of IB students and 14 percent of non-IB students—are missing on at least one of the four constituent achievement tests and thus are also missing on the academic achievement outcome itself. After compositing, for ease of interpretation, I standardized the obtained first principal component score to a mean of zero and a standard deviation (S.D.) of unity in my analytic sample of students (n=14,368) who were not missing seventh-grade mathematics or reading scores nor one of the ACT subject-level tests. Students take the constituent achievement tests during the late spring of their junior year, and so my estimated "treatment effects" represent the impact of only one year of official IB enrollment on the academic-achievement outcome. «Insert Table 1 h e r e » My second outcome, HSGRAD, is a dichotomous indicator that I coded 1 if students graduated on time from high school with a regular CPS degree or an alternate Illinois degree (0 otherwise).7 I possess this information for every student in the full sample, of which 99 percent of IB and 78 percent of non-IB students graduated from high school.8 My third outcome, COLLENROLL, is a dichotomous indicator that I coded 1 for students who enrolled in either two- or four-year college for at least one semester during one of the first four semesters post-graduation from high school (0 otherwise). The NSC only matches CPS high-school graduates to its college enrollment database. Therefore, 7 The requirements for the CPS high school degree are more demanding than the requirements for an Illinois degree. My data does not indicate whether students later earned the General Education Development equivalency degree (GED). 8 Though these proportions seem high given the disadvantaged context, these students persisted beyond ninth grade, the year in which CPS students are at the highest risk of dropout (Allensworth, 2005). 11 students' COLLENROLL status is conditional on whether they graduated from high school, such that the COLLENROLL=0 for students for whom HSGRAD=0. 83% of IB students and 48% of non-IB students enrolled in college. If IB enrollment affects the probability that students will graduate from high school, then relative to non-IB students, a greater proportion of IB students will be included in the sub-sample of students that I use to estimate the impact of IB enrollment on students' probability of college enrollment. The overrepresentation of IB students in this sub-sample would then contribute to biasing my estimation of the impact of IB enrollment on students' probability of college enrollment. I discuss this potential threat to a causal interpretation of my estimates of the impact of IB enrollment on students' probability of college enrollment in the "Threats to Validity" section of this paper. Question predictor I define my question predictor, IBTREAT, as whether students enrolled in the core IB Theory of Knowledge course for at least a semester (l=yes, 0=no). According to the CPS IB Diploma Program Administrator (personal correspondence, July 6, 2010), students typically only enroll in TOK if they are attempting to fulfill the IB program requirements and only rarely as a way to take honors courses in a piecemeal fashion. The students that enroll in TOK, on average, enroll in eleven IB courses (SD=4), confirming this characterization of TOK course enrollment. According to my definition, seven percent (1,403 students) of the full sample of 20,422 students was enrolled in the IB treatment. Covariates I include in my analysis students' seventh-grade mathematics (MATH) and 12 reading (READ) test percentiles, which are based on their national percentile rank on the ITBS test, as measures of their previous academic achievement. CPS administrators use these test percentiles to determine IB eligibility. 26% (5,261) of students in my full sample and 20% (279) of IB students are missing these scores. I eliminate students missing on MATH or READ, leaving an analytic subsample of 15,148 students. As I show in Table 1, among IB students, the mean ITBS mathematics-achievement percentile is 79.47 (SD=16.18) and the mean reading-achievement percentile is 73.86 (SD=17.71). In comparison, the non-IB mean mathematics percentile of 50.27 (SD=24.95) and mean reading percentile of 47.13 (SD=22.97) are each almost thirty percentiles lower. I also include as covariates in my analyses measures of students' demographic characteristics, including their gender, family income and race. I coded MALE as 1 for the boys, who make up 48% of non-IB students and 35% of IB students. I represent family income using LUNCH, a dichotomous indicator that I coded 1 for students who were eligible to receive free or reduced-price lunch. Over all, 65% of IB students were eligible, compared to 78% of non-IB students, indicating that on average, both non-IB and IB students are from low-income families, though the probability that non-IB students are from poorer families is greater. I represent student race using a vector of dichotomous indicators that describe whether a student was Asian-American (ASIAN), African-American (AFAM), Latino (LATINO), Native American or Alaskan American (NATIVEAM), or White (WHITE), dropping the last to create a reference category. Among all IB students, 30 percent are Latino, 28 percent are African-American, 26 percent are White, 17 percent are Asian-American and less than 1 percent are Native 13 American or Alaskan, while the corresponding proportions are 42%, 37%, 15%, 6% and under 1%, respectively, among non-IB students. In following section, I use the generic variable name STUDENTDEM to represent the vector of covariates that record students' ITBS test percentiles, gender, family income and race. Analytic Methods My objective is to estimate the causal impact of enrollment in the IB Diploma Program on CPS eleventh-grade students' academic achievement, probability of highschool graduation and probability of college enrollment. I use the same analytic strategy to address each of my research questions, which I describe for outcome, Y, for student /' in high school j in cohort c. I first assume selection into the IB Diploma Program based on students' observed characteristics by using a. propensity score approach to estimate the impact of IB enrollment on my three measures of students' academic success. Then, as I discuss in the "Threats to Validity" section of this paper, following Rosenbaum (2002), I test the sensitivity of my estimates to different levels of selection bias. In fitting my propensity-score selection model, I use logistic regression analysis to model students' probability of enrolling in the IB program as a function of a set of measured characteristics that I have determined theoretically should predict enrollment (Murnane & Willett, 2011 311). I include students' seventh-grade mathematics and reading test percentiles as the principal predictors in my propensity score selection model because I anticipate that these test results will be strong predictors of whether students 14 enroll in the IB Diploma Program, given the CPS IB-admission policy. 1 also include an 9 indicator of gender because CPS boys' tendency to fail classes, earn lower grade-point averages and drop out of school at higher rates than girls (Allensworth, 2007) suggests that they have a lower theoretical probability of CPS IB enrollment eligibility than girls. I include measures of race and family income as predictors because historically IB has attracted White, Asian-American and economically advantaged students, while AfricanAmerican, Latino and economically disadvantaged students have not had the academic preparation necessary to enter the rigorous Diploma Program (Burris, Welner, Wiley, & Murphy, 2007). I therefore fit the following logistic regression model (1) to represent the probability that student / in schooly in cohort c chooses to enroll in the IB program: ¥r(IBTREATlJC = 1) = (I) -P0+P,STUDENTDEMIJC+SJ+\ ' e I include school (S,) and cohort (Xc) fixed effects to allow for the probability that student i will enroll in IB to differ by school and cohort. The fixed effects address the concern that schools' and cohorts' unmeasured characteristics might contribute to determining whether students will enroll in IB. Based on the fitted propensity score selection model, the propensity scores (PHAT) are the predicted probabilities that students will enroll in the IB program. Once I have obtained PHAT for each of the students in my analytic sample, I follow Imbens and Wooldridge (2009) in eliminating from my analytic sample all non-IB students with 9 I use quadratic functional forms of the mathematics and reading percentiles because this more flexible form ought to be a better fit for IB students' ITBS scores. Estimates of the IB enrollment impact are statistically indistinguishable in models in which I include respective nested linear, quadratic, cubic and quartic functional forms of the ITBS percentiles. 15 estimated propensity scores that fall outside of the range of estimated propensity scores for the IB students. This process aligns the IB and non-IB groups explicitly on the selection model covariates, ensuring that the non-IB students in my comparison sample are as similar as possible in terms of measured characteristics to IB students. The resulting comparison sample includes 11,592 student records.10 Then, following Hirano, Imbens and Ridder (2003), in my subsequent analysis of program impact, I use PHAT io provide an inverse-probability weighting, IPW, for each student in my comparison sample. I create the weights as follows: For IB students: IPW=\IPHAT; for non-IB students: IPW=\l(l-PHAT). The purpose of this weighting strategy is to allocate the least weight to the IB students with the highest propensity for selection into IB and the greatest weight to the IB students with the lowest propensity for selection into IB (and vice versa for non-IB students) in order to impose identical (observed) covariate distributions for students who enrolled versus did not enroll in the IB program. The final step is to fit my weighted least squares (WLS) estimation model (2), which represents the relationship between my academic outcomes and membership in the IB treatment, weighting each student i in school j in cohort c by his or her IPW as follows11: 10 1 define the 3,469 students who were missing on an IB treatment status as being non-IB rather than dropping them. My rationale is that according to model (1), the distribution of propensity scores for students missing on IB status (mean=0.053, SD=0.113) is to the left of the distribution of propensity scores for students who explicitly did not choose the IB option (mean=0.065, SD=0.113). Therefore, students missing on IB status, if anything, look more like non-IB students than IB students, for whom the mean propensity score is 0.294 (SD=0.210). 11 In practice, I weight each observation by the square of the weights (Imbens & Wooldridge, 2009). 16 (2) Yijc = /30 + BJBTREATijc + B2STUDENTDEM ijc +6j+Yc+ eijc Where Y is the outcome in question (probability of taking the ACT examination, student academic achievement, probability of high-school graduation, or probability of college enrollment). I control for the vector of student characteristics, STUDENTDEM and include school (6,) and cohort ( yc) fixed effects to increase the precision of my estimates (Imbens & Wooldridge, 2009). I represent the residuals with the term EyC and estimate standard errors to account for co-variation in students' outcomes within schools, the unit at which the IB treatment is administered. Assuming selection into IB based only on students' observed characteristics, I can interpret Bi as the impact of IB enrollment on outcome Y. To determine if the impact of IB enrollment differs by gender, I fit model (3) below, including the main effect of MALE in the STUDENTDEM term, the two-way interaction term IB*MALE and again weighting each student i in school j in cohort c by IPW: (3) Yljc = B0 + pjBTREATijc + 62IB*MALEijc + 63STUDENTDEMijc +dj +Yc +sijc I interpret the Bi parameter estimate as the impact of IB enrollment on girls and the B2 parameter estimate as the extent to which the impact of enrollment in IB differs according to gender. Compared to using Ordinary Least Squares (OLS) methods to address my research questions, the propensity-score approach is a more effective, robust and partially non-parametric way of controlling for observed bias attributable to measured covariates (Murnane & Willett, 2011, 310). By using the propensity-score approach, I must first model and examine explicitly the process of students' selection into the IB program. The 17 extent to which I am able to capture the selection process successfully in my selection model then improves my ability to remove selection bias from my subsequent estimates of the causal impact of IB enrollment on academic outcomes (Murnane & Willett, 2011, 288). The weighting strategy—as compared to stratifying or matching strategies—is my preferred use of the propensity scores because it allows me to impose the identical covariate distributions for IB and non-IB groups and to include in my comparison sample all non-IB students who are most comparable to IB students based on observed characteristics (Murnane & Willett, 2011). Finally, propensity scores constitute the basis of Rosenbaum's sensitivity analysis, to which I subject my results and describe later in the "Threats to Validity" section of this paper. Results The IB enrollment process In Table 2, I show the results of fitting Model (I), which describes the fitted relationship between students' probability of enrolling in the IB Diploma Program and predictors that include their demographic characteristics and previous academic achievement. In the table, I present parameter estimates and standard errors for each of the demographic and ITBS test percentile covariates that, I have argued, participate in the IB selection process. «Insert Table 2 h e r e » In addition, to demonstrate the substantive nature of the IB selection process visually, in Figure 1, I plot the fitted probability that prototypical students will enroll in the IB program as a function of their seventh-grade ITBS reading percentiles. In the figure, I present fitted lines for students who attended one CPS high school that offered 18 the IB program to the eleventh-grade 2006-2007 cohort, who scored at the fiftieth percentile on the ITBS mathematics test, and who were eligible to receive free or reduced price lunch (n=303). The eight fitted lines compare the fitted probabilities that Latino, African-American, White and Asian-American girls and boys enrolled in the IB program, as a function of their seventh-grade reading test percentiles. I use solid lines to represent the fitted selection probabilities for boys and dashed lines for girls. I use black to represent Latino students, dark grey to represent African-American students, medium grey to represent White students and the lightest grey to represent Asian-American students. «Insert Figure 1 h e r e » Among Title 1 U.S. high schools that offer the IB program, approximately twothirds require that students demonstrate strong previous academic ability to be eligible to enroll. The most commonly used selection criterion is grade point average, followed by standardized test performance (Siskin & Weinstein, 2008). Like enrollment policies in the majority of IB-offering Title 1 schools, the CPS IB enrollment policy requires that students demonstrate their academic ability to be eligible to enroll. Specifically, students must score over the fiftieth percentile on both the mathematics and reading seventh-grade tests to be eligible. Given this enrollment requirement, I had anticipated and the CPS IB Diploma Program Administrator concurred that it would be more probable that students who have demonstrated stronger prior academic achievement would enroll in the IB Diploma Program (personal correspondence, July 6, 2010). The results that I present in Table 2 and Figure 1 verify my expectation about the relationship between students' previously demonstrated academic strength and the 19 probability that they will enroll in the IB program. Highlights of Table 2 include that holding constant gender, race, lunch-eligibility status, school and cohort, the fitted probability that students will enroll in the IB program is greater for students with higher seventh-grade mathematics (p=0.087) and reading (p=0.001) scores. Correspondingly, in Figure 1, I show that for all students who score below the fiftieth percentile on the seventh-grade ITBS reading score, there is close to a zero fitted probability that they will enroll in an IB program. The fitted enrollment probability is then higher for students whose prior reading scores are higher. Gender is also an important predictor of IB enrollment. As I show in Table 2, holding constant seventh grade test scores, race and lunch-eligibility status, school and cohort, the fitted odds that a boy will enroll in the IB program is half that of a girl (e"701=0.50; p[...]...1 The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools Originally founded as a private means for diplomats' children to earn an internationally recognized high-school diploma, today the International Baccalaureate (IB) Diploma Program is one of the fastest growing curricular innovations in U.S publicschool districts In schools accredited... that they have a lower theoretical probability of CPS IB enrollment eligibility than girls I include measures of race and family income as predictors because historically IB has attracted White, Asian-American and economically advantaged students, while AfricanAmerican, Latino and economically disadvantaged students have not had the academic preparation necessary to enter the rigorous Diploma Program... taking the ACT examination may affect the composition of unobserved characteristics among the subsample of IB and non-IB students who took the ACT examination and may contribute to biasing my estimation of the impact of IB enrollment on students' ACT scores I discuss this potential threat to the causal interpretation of my estimate of the impact of IB enrollment on students' ACT scores in the following... Among students in my comparison sample, 96%o took the four test sections, as indicated by having their ACT measure information In the first column of Table 4, I show that enrollment in the IB Diploma Program increases—by three percentage points (p=0.009) the fitted probability of taking the ACT examination In the second column, I show that, on average, accounting for measured characteristics that affect... Latino and African-American boys will enroll I had anticipated this result, given that the theoretical probability that boys are eligible to enroll in the IB Diploma Program in the first place is lesser than that of girls The race-based differential in the fitted probability of enrollment in the IB program aligns with the Burris et al (2007) argument that White and Asian-American students, on average,... underestimation of the IB enrollment impact 25 compare the outcomes of students enrolled in the IB program to those of students who are not.13 In this study, in the absence of random assignment data, I have used observational data to estimate the IB enrollment impact Since my estimates are likely overestimates of the true impact of enrollment in the IB program, I assess the sensitivity of my estimates of. .. the same color line, each of the dashed fitted lines (representing prototypical girls) indicates that there is a greater fitted probability of enrollment in the IB program than do the solid fitted lines representing boys In fact, as the dashed lines show, the fitted probability that all White, Latino and African-American girls will enroll in the IB program is greater than the probability that all White,... that on average, both non-IB and IB students are from low-income families, though the probability that non-IB students are from poorer families is greater I represent student race using a vector of dichotomous indicators that describe whether a student was Asian-American (ASIAN), African-American (AFAM), Latino (LATINO), Native American or Alaskan American (NATIVEAM), or White (WHITE), dropping the. .. use the propensityscore weighting approach to predict the impact of IB enrollment on students': (1) probability of taking the ACT test; (2) academic achievement as measured by the variable ACT, (3) probability of high-school graduation; (4) and probability of college enrollment In each column, I provide the outcome's mean and S.D in the comparison sample and then the parameter estimate and associated... twelfth-grade IB Diploma Program in their geographically determined neighborhood high school to apply in their spring of eighth grade so that they can participate in an IB-preparation track during ninth and tenth grade The nature of the preparation track differs by CPS high school Several offer accredited IB-preparation programs (the IB Middle Years Program), while others offer AP courses or other honors classes