ACADEMIC SKILLS PROBLEMS Fourth Edition Direct Assessment and Intervention Ed w a r d S. Sh a p i r o THE GUILFORD PRESS New York London ©2011 The Guilford Press A Division of Guilford Publications, Inc. 72 Spring Street, New York, NY 10012 www.guilford.com All rights reserved No part of this book may be reproduced, translated, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without written permission from the Publisher. Printed in the United States of America This book is printed on acid-free paper. Last digit is print number: 9 8 7 6 5 4 3 2 1 Library of Congress Cataloging-in-Publication Data is available from the Publisher ISBN 978-1-60623-960-5 ix Preface People always make the difference. I have a hard time believing that 20 years have passed since I shared my vision and methodology for linking assessment to instructional outcomes with our field. It’s hard to put into words the gratification I feel personally that these processes have become such a routine part of the everyday experience of so many educational professionals. As I came to write the preface to this edition of the text, I wanted to put into words where my ideas came from that have become such an intrinsic part of what we do in the assessment process. The concepts I put forward in the first edition were derived from the work of many key individuals—Stan Deno and Phyllis Mirkin’s (1979) seminal little spiral-bound book, Data-Based Program Modification, had an incredible influence in shaping my model of assessment. The influences from the University of Minnesota Institute for Learning Disabilities in the 1980s, led by key researchers such as Jim Ysseldyke and Stan Deno, comprised another wave that impacted my thinking. As curriculum-based measurement (CBM) became a legitimate methodology for conducting academic assessments (especially in reading), the ongoing, incredible con- tributions of some of Stan Deno’s disciples, such as Lynn and Doug Fuchs in special education, continue to have strong influences on my thinking. I found “kindred spirits” in school psychology in those early years of develop- ment, such as Dan Reschly, Sylvia Rosenfield, and Mark Shinn, who equally shared my passion for trying to improve academic assessment. Together, we all were trying to find a way to break away from the longstanding tradi- tion of standardized norm-referenced testing that so dominated our field. Again, people, not just ideas, make the difference. And it was not just the researchers and academicians who had an impact on my thinking. Opportunities to work with influential practitio- ners such as Jeff Grimes, Greg Robinson, Jim Stumme, and Randy Allison x Preface in Iowa, who already saw the future and began changing directions through Project ReAIM in the mid-1980s, provided the evidence that what I and others were advocating resulted in real improvements for children, began to change learning trajectories for struggling students, and began to show that prevention science could indeed be extended to academic skills problems. We could solve the big problems of literacy development and effective early mathematical knowledge by changing a system, not just children. The publication of the first edition of this volume in 1989 was like being a fortune teller, having a vision for where our field needed to go. So what happened over those 20-plus years? Essentially, we have gone from instructional support teams (IST) to response to intervention (RTI). I have been lucky to have spent those years in my native Pennsylvania, where I watched the development of the IST model in the early 1990s. The model was implemented through the visionary leadership of Jim Tucker, who was appointed State Director of Special Education, and through the careful guidance of Joe Koveleski, who was engaged in leading the IST process while he was a practitioner and district administrator. The concepts of cur- riculum-based assessment (CBA), direct assessment, prereferral interven- tion, and behavioral consultation became a routine part of what all educa- tors in Pennsylvania began to implement. What Tucker and Kovaleski put in place remains the reason why Pennsylvania has moved ahead of many states in the country in the implementation of RTI today. Indeed, RTI is essentially IST on steroids! Over the course of those 20 years, I have watched the coming of accountability, through No Child Left Behind, a law that codified the requirements of what schools have always needed to do—recognize that they have an obligation to get kids to learn, even those with tough back- grounds that can impede learning. Schools still must be held accountable to show that they are places where children learn to read, learn to do math, learn to write. I watched the coming of Reading First and Early Reading First, the incredible emphasis in this country on the importance of early lit- eracy. I watched the recognition of prevention as an important way to solve many academic skills problems. I watched the evolution in our field from solving problems “one kid at a time” to attacking problems at the level of systems change. I watched the evolution to RTI, efforts to bring about early intervention so we can head off learning problems before children’s learn- ing processes become embedded in continual failure for the rest of their school lives. I have watched schools understand that letting children fail until they are deemed eligible for special education services is just wrong and illogical. I have watched the emergence of a new way to think about, assess, identify, and understand learning disabilities. Although we still have a lot to learn and understand about this new approach to identifying Preface xi children who are in need of learning support special education services, it is clear that more and more schools are recognizing the logical appeal of the RTI method. But it is always the people in your life who make the difference. Look- ing back, I have asked myself the question, “Has all that I have written about, advocated, researched, and envisioned, actually influenced oth- ers?” My own early doctoral students Chris Skinner, Tanya Eckert, and John Hintze went on to become well-recognized academicians in our field as contributors to the area of academic assessment and intervention. If one traces some of their doctoral students (my “grandchildren”), you find a number of influential academicians who have continued the study of aca- demic skills problems: Ted Christ (John Hintze) and Chris Riley-Tillman (Tanya Eckert). I am heartened by the new generation of researchers and leaders who have pushed CBA to levels yet unknown—Amanda VanDer- Heyden, Matt Burns, Ted Christ, and Scott Ardoin, just to name a few. The current volume maintains close connections with the past, where there are solid foundations for what I have been advocating for well over 20 years. Readers of past editions will recognize the continuation of the same concepts, themes, and methods that have evolved into a well-established process for conducting academic assessment. The volume also introduces the process of RTI, a systematic method of changing schools in which direct assessment of academic skills plays a crucial role and is likely to be the mechanism for a long future that will sustain the type of assess- ment processes I have advocated for more than 20 years. The volume ties together the assessment processes I have defined and detailed over this and previous volumes and places them in the context of an RTI model. An entire chapter is devoted to two key assessment methods that are core com- ponents of RTI—universal screening and progress monitoring—showing how CBA naturally links into this process. I end the preface, as always, by offering sincere gratitude and admira- tion to those who continue to mean the most in my life—my wife, Sally, my partner of now 33 years who just gets better and more radiant as the years go on, and my children, Dan and Jay, who continue to amaze me at the turn of every corner. I’ve watched Dan move forward in a success- ful career, as a new husband in a life that gives joy to his parents. I’ve watched Jay continue with his passions for film making, Africa, and base- ball. I remain very proud of both of my boys. I also acknowledge with sad- ness those who have left our field and those who have left my immediate life since the previous edition. Our field lost two important people—Ken Kavale and Michael Pressley—whose views I may not have agreed with but whom I always respected for their empirical approach to argument. I will miss reading their rants that always got me “up for the intellectual fight.” I am sure others will emerge to take their place. On a personal level, I lost xii Preface two dear members of my own family, my wife’s parents, Earl and Binky. I was so lucky to have them as part of my life for the past 30 years, and they continue to be missed every day. Finally, thanks to the new crop of doctoral and specialist students with whom I work, who continue to “keep me young in thought.” I love the challenge and the collaboration that continue to make every day fun to go to the office. My final thought is to express thanks that I have been able to offer a piece of myself to improve the lives of children who struggle in school. They remain my passion, my reason for the work I do, and the reason why all of us are so dedicated to this field. Ed w a r d S. Sh a p i r o xiii Contents CHAPTER 1 . Introduction 1 Background, History, and Rationale for Academic Assessment and Intervention 5 Assessment and Decision Making for Academic Problems 7 Types of Individual Assessment Methods 8 Intervention Methods for Academic Skills 24 CHAPTER 2 . Choosing Targets for Academic Assessment and Remediation 31 Selecting Targets for Assessment 33 Selecting Targets for Remediation: Linking Assessment to Intervention 55 Selecting Intervention Procedures 57 Summary of Procedures for Choosing Interventions 65 CHAPTER 3 . Step 1: Assessing the Academic Environment 67 What CBA Is 69 What CBA Is Not 71 Assessing the Academic Environment: Overview 72 Teacher Interviews 74 Direct Observation 95 Student Interview 112 Permanent Product Review 115 Summary and Conclusions 117 CHAPTER 4 . Step 2: Assessing Instructional Placement 133 Reading 134 Mathematics 148 xiv Contents Written Expression 158 Spelling 164 Summarizing the Data- Collection Process 166 CHAPTER 5 . Step 3: Instructional Modification I: General Strategies 178 Background 178 General Strategies for Academic Problems 184 Summary 211 CHAPTER 6 . Step 3: Instructional Modification II: Specific Skills Areas 212 Reading 212 Mathematics 237 Spelling 244 Written Language 247 Conclusions and Summary 252 CHAPTER 7. Step 4: Progress Monitoring 254 Reading 256 Mathematics 272 Spelling 281 Written Language 285 Summary 288 CHAPTER 8 . Academic Assessment within a Response-to- Intervention Framework 289 Assessment for Universal Screening 292 Assessment for Progress Monitoring within RTI 308 Conclusions 319 CHAPTER 9 . Case Illustrations 321 Case Examples for Academic Assessment 321 Case Examples for Academic Interventions 360 A Case Example of the Four-Step Model of Direct Academic Assessment 371 A Case Example of CBA Assessment within an RTI Model 379 Conclusions 384 References 387 Index 435 1 CHAPTER 1 Introduction Br i a n , a S E c o n d -g r a d E S t u d E n t at Salter Elementary School, was referred to the school psychologist for evaluation. The request for evalu- ation from the multidisciplinary team noted that he was easily distracted and was having difficulty in most academic subjects. Background informa- tion reported on the referral request indicated that he was retained in kindergarten and was on the list this year for possible retention. As a result of his difficulties sitting still during class, his desk has been removed from the area near his peers and placed adjacent to the teacher’s desk. Brian currently receives remedial math lessons. Susan was in the fifth grade at Carnell Elementary School. She had been in a self- contained classroom for students with learning disabilities since second grade and was currently doing very well. Her teacher referred her to determine her current academic status and potential for increased inclusion. Jorgé was in the third grade at Moore Elementary School. He was referred by his teacher because he was struggling in reading and had been in the English as a second language (ESL) program for the past year since arriving from Puerto Rico. Jorgé’s teacher was concerned that he was not achieving the expected level of progress compared to other students with similar backgrounds. All of these cases are samples of the many types of referrals for aca- demic problems faced by school personnel. How should the team proceed to conduct the evaluations? The answer to this question clearly lies in how the problems are conceptualized. Most often, the multidisciplinary team will view the problem within a diagnostic framework. In Brian’s case, the primary question asked would be whether he is eligible for special educa- tion services and if so, in which category. In Susan’s case, the question would be whether her skills have improved sufficiently to suggest that she 2 AC A DE M IC S K I LL S PRO BL E MS would be successful in a less restrictive setting. Jorgé’s case raises questions of whether his difficulties in reading are a function of his status as an ESL learner, or whether he has other difficulties requiring special education services. In all cases, the methodology employed in conducting these types of assessments is similar. Typically, the school psychologist would administer an individual intel- ligence test (usually the Wechsler Intelligence Scale for Children— Fourth Edition [WISC-IV; Wechsler, 2003]), an individual achievement test (such as the Peabody Individual Achievement Test— Revised/Normative Update [PIAT-R/NU; Markwardt, 1997], or the Wechsler Individual Achievement Test–II [WIAT-II; Wechsler, 2001]), and a test of visual–motor integration (usually the Bender– Gestalt). Often, the psychologist would add some mea- sure of personality, such as projective drawings. Other professionals, such as educational consultants or educational diagnosticians, might assess the child’s specific academic skills by administering norm- referenced achieve- ment tests such as the Woodcock– Johnson Psychoeducational Battery–III (Woodcock, McGrew, & Mather, 2001), the Key Math–3 Diagnostic Assess- ment (Connoley, 2007), or other diagnostic instruments. Based on these test results, a determination of eligibility (in the cases of Brian and Jorgé) or evaluation of academic performance (in the case of Susan) would be made. When Brian was evaluated in this traditional way, the results revealed that he was not eligible for special education. Not surprisingly, Brian’s teacher requested that the multidisciplinary team make some recommen- dations for remediating his skills. From this type of assessment, it was very difficult to make specific recommendations. The team suggested that since Brian was not eligible for special education, he was probably doing the best he could in his current classroom. They did note that his phonetic analysis skills appeared weak and recommended that some consideration be given to switching him to a less phonetically oriented approach to reading. When Susan was assessed, the data showed that she was still substan- tially below grade levels in all academic areas. Despite having spent the last 3 years in a self- contained classroom for students with learning disabilities, Susan had made minimal progress when compared to peers of similar age and grade. As a result, the team decided not to increase the amount of time that she be mainstreamed for academic subjects. When Jorgé was assessed, the team also administered measures to evaluate his overall language development. Specifically, the Woodcock– Muñoz Language Survey Revised (Woodcock & Muñóz- Sandoval, 2005) was given to assess his Cognitive Academic Language Proficiency (CALP), which measures the degree to which students’ acquisition of their second language enables them to effectively use the second language in cognitive processing. The data showed that his poor reading skills were a function of [...]... indirect methods of intervention In contrast, direct interventions attempt to improve the area of academic skill by directly teaching that particular skill These types of interventions usually are based on examination of variables that have been found to have direct relationships to academic performance Indirect Interventions for Academic Skills Most indirect interventions for remediating academic skills. .. conceptual issues of academic assessment and remediation is provided The framework upon which behavioral assessment and intervention for academic problems is based is described First, however, the current state of academic assessment and intervention is examined Types of Individual Assessment Methods Norm-Â�Referenced Tests One of the most common methods of evaluating individual academic skills involves... text is on the use of a direct assessment and intervention methodology for the evaluation and remediation of academic problems with students like Greg Specifically, detailed descriptions of conducting a behavioral assessment of academic skills (as developed by Shapiro, 1990, 2004; Shapiro & Lentz, 1985, 1986) are presented Direct interventions focused on teaching the skills directly assessed are also... contingencies, and so forth (Lentz & Shapiro, 1986) What is needed in the assessment of academic skills is a methodology that can more directly assess both the student’s skills and the academic environment This methodology also needs to be able to address most or all of the types of educational decisions identified by Salvia et al (2007) Direct Assessment of Academic Skills A large number of assessment. .. 24 A C A DEMIC S KILL S P ROBLEM S Intervention Methods for Academic Skills Remediation procedures developed for academic problems can be conceptualized on a continuum from indirect to direct procedures Those techniques that attempt to improve academic performance by improving underlying learning processes can be characterized as indirect interventions In particular, interventions based on “aptitude–Â�treatment... consumers (teachers and school psychologists) and is increasingly being taught as part of the curriculum in training school psychologists, it is slow in assuming a prominent role in the assessment methods of practicing psychologists Conclusions and Summary Strategies for the assessment of academic skills range from the more indirect norm- and criterion-Â�referenced methods through direct assessment, which... change in academic performance (e.g., Arter & Jenkins, 1979; Good, Vollmer, Creek, Katz, & Chowdhri, 1993) Assessment and Decision Making for Academic Problems Salvia et al (2007) define assessment as “the process of collecting data for the purpose of (1) specifying and verifying problems, and (2) making decisions about students” (p.€5) They identify five types of decisions that can be made from assessment. .. behavior problems by almost five to one Clearly, there are significant needs for effective assessment and intervention strategies to address academic problems in school-age children Indeed, the number of commercially available standardized achievement tests (e.g., Salvia, Ysseldyke, & Bolt, 2007) suggests that evaluation of academic progress has been a longstanding concern among educators Goh, Teslow, and. .. between what is taught and what is tested on reading subtests is questionable, the data examined by us (Shapiro & Derr, 1987) and by Jenkins and Pany (1978) were hypothetical It certainly is possible that such poor overlap does not actually exist, since the achievement tests are designed only as samples of skills and not as direct assessments Good and Salvia (1988) and Bell, Lentz, and Graden (1992) have... from birth through age 7, and the Comprehensive Inventory of Basic Skills II (CIBS-II; Brigance, 2009) providing inventories for skills development between prekindergarten and grade 9 Each measure includes skills in academic areas such as readiness, speech, listening, reading, spelling, writing, mathematics, and study skills The inventories cover a wide range of subskills, and each inventory is linked . Academic Assessment and Intervention 5 Assessment and Decision Making for Academic Problems 7 Types of Individual Assessment Methods 8 Intervention Methods for Academic Skills 24 CHAPTER 2 Examples for Academic Assessment 321 Case Examples for Academic Interventions 360 A Case Example of the Four-Step Model of Direct Academic Assessment 371 A Case Example of CBA Assessment within. Choosing Targets for Academic Assessment and Remediation 31 Selecting Targets for Assessment 33 Selecting Targets for Remediation: Linking Assessment to Intervention 55 Selecting Intervention Procedures