Thông tin tài liệu
Improving Learning in South African Schools:
The Quality Learning Project (QLP)
Summative Evaluation (2000 to 2004)
A Kanjee & CH Prinsloo
August 2005
HUMAN SCIENCES RESEARCH COUNCIL
Free download from www.hsrcpress.ac.za
Compiled by the Assessment Technology and Education Evaluation Research Programme,
Human Sciences Research Council
Funded by the Business Trust
Intervention Programme Managed by JET Education Services
Evaluation by the Human Sciences Research Council
Published by HSRC Press
Private Bag X9182, Cape Town, 8000, South Africa
www.hsrcpress.ac.za
© 2005 Human Sciences Research Council
First published 2005
All rights reserved. No part of this book may be reprinted or reproduced or utilised in
any form or by any electronic, mechanical, or other means, including photocopying
and recording, or in any information storage or retrieval system, without permission
in writing from the publishers.
ISBN 0-7969-2145-8
Cover Design and Layout: Vinesh Naidoo
Production: Shereno Printers
Distributed in Africa by Blue Weaver
PO Box 30370, Tokai, Cape Town, 7966, South Africa
Tel: +27 (0) 21 701 4477
Fax: +27 (0) 21 701 7302
email: orders@blueweaver.co.za
www.oneworldbooks.com
Distributed in Europe and the United Kingdom by Eurospan Distribution Services (EDS)
3 Henrietta Street, Covent Garden, London, WC2E 8LU, United Kingdom
Tel: +44 (0) 20 7240 0856
Fax: +44 (0) 20 7379 0609
email: orders@edspubs.co.uk
www.eurospanonline.com
Distributed in North America by Independent Publishers Group (IPG)
Order Department, 814 North Franklin Street, Chicago, IL 60610, USA
Call toll-free: (800) 888 4741
All other enquiries: +1 (312) 337 0747
Fax: +1 (312) 337 5985
email: frontdesk@ipgbook.com
www.ipgbook.com
Free download from www.hsrcpress.ac.za
CONTENTS
LIST OF FIGURES AND TABLES vi
LIST OF ANNEXURES vii
ACKNOWLEDGEMENTS viii
PREFACE x
ABBREVIATIONS USED FOR THE QLP DISTRICTS xi
Map showing location of Quality Learning Project (QLP) districts xi
EXECUTIVE SUMMARY xii
Background 1
Framework for the Evaluation Study 1
Evaluation Methodology and Research Design 3
Evaluation Criteria 6
Success of the Quality Learning Project 7
Results of the QLP Evaluation 10
District-level Functioning 10
School-level Functioning 11
Classroom-level Functioning 13
Learner Context and Performance 18
Intervention Coverage and Quality 20
Effects of Interventions on Functioning 21
Effects of Improved Functioning on Learner Performance 22
Effects of Interventions on Learner Performance 22
Additional Observations Emanating from the Path Analysis 23
Concluding Statements and Recommendations 24
Recommendations 24
Recommendations on Methodology and Design 24
District-level Recommendations 26
School-level Recommendations 27
Classroom-level Recommendations 28
Causal Modelling Recommendations 28
Conclusion 29
NOTES 37
HUMAN SCIENCES RESEARCH COUNCIL
v
CONTENTS
QLP SUMMATIVE EVALUATION
Free download from www.hsrcpress.ac.za
LIST OF FIGURES AND TABLES
Figures
Map showing location of Quality Learning Project (QLP) districts xi
Figure A: The QLP model at district, school and teacher level 2
Figure B: Number of schools in the evaluation survey and case-study samples 3
Figure C: District-functionality levels in 2004 and change from 2002 to 2004 10
Figure D: School-functionality levels in 2004 and change from 2002 to 2004 12
Figure E: Grade 9 mathematics classroom-functionality levels in 2004 and change
from 2002 to 2004 14
Figure F: Grade 11 mathematics classroom-functionality levels in 2004 and change
from 2002 to 2004 14
Figure G: Grade 9 language classroom-functionality levels in 2004 and change
from 2002 to 2004 15
Figure H: Grade 11 language classroom-functionality levels in 2004 and change
from 2002 to 2004 15
Figure I: National mean mathematics scores for QLP and control schools
by year and grade 18
Figure J: National mean language scores for QLP and control schools
by year and grade 19
Figure K: Index scores for district-intervention coverage and quality by year 20
Figure L: Index scores for school-intervention coverage and quality by year 21
Figure M: Index scores for teacher-intervention coverage and quality by year,
subject and grade 21
Tables
Table I: Indicators at Grade 12 level of the success of the QLP (from 2000 to 2004) xiii
Table A: Total sample obtained for mid-term and summative evaluations 4
Table B: Number of schools sampled per district 4
Table C: Change in Grade 12 learner performance between 2000 and 2004
across QLP and control schools by province 8
Table D: Indicators at Grade 12 level of the success of the QLP (from 2000 to 2004) 9
HUMAN SCIENCES RESEARCH COUNCIL
vi
FIGURES AND TABLES
QLP SUMMATIVE EVALUATION
Free download from www.hsrcpress.ac.za
LIST OF ANNEXURES
(Graphs comprising findings from the path analyses)
Annexure 1:
Standardised path weights for modelling Grade 9 mathematics causal patterns 30
Annexure 2:
Standardised path weights for modelling Grade 9 language causal patterns 31
Annexure 3:
Standardised path weights for modelling Grade 11 mathematics causal patterns 32
Annexure 4:
Standardised path weights for modelling Grade 11 language causal patterns 33
Annexure 5:
Standardised path weights for modelling matriculation pass rate causal patterns 34
Annexure 6:
Standardised path weights for modelling matriculation English pass rate causal patterns 35
Annexure 7:
Standardised path weights for modelling Grade 12 (SG) mathematics pass rates 36
HUMAN SCIENCES RESEARCH COUNCIL
vii
ANNEXURES
QLP SUMMATIVE EVALUATION
Free download from www.hsrcpress.ac.za
ACKNOWLEDGEMENTS
The hard work and commitment of many dedicated individuals and organisations have over five years made it
possible to bring the formal QLP evaluation to a close by publishing the final, summative report. All contributions
are appreciated and are hereby acknowledged.
The Business Trust is acknowledged for providing the funding for this project.
Acknowledgement is made of JET Education Services and the Business Trust for managing the QLP and for
providing valuable support and assistance through the project Steering Committee and other mechanisms. Nick
Taylor, Anele Davids, Hemant Waghmarae, Jackie Moyana, and Leigh-May Moses, all from the QLP programme
management team of JET Education Services and Charles Barnard, Brian Whittaker, Mdu Ndhlovu and Nomfundo
Mqadi, with Theuns Eloff at the outset, from Business Trust, all deserve special mention. They contributed
continued guidance throughout the project, with considerable effort and time spent on many rounds of comment
to draft versions of instruments and documents of many kinds, and especially the baseline, mid-term and current
summative reports.
The Department of Education is acknowledged for its continually increasing role in direct communications and
meetings between the HSRC and participants at national and provincial level (at the Director General’s office and
through strategic planning sessions, respectively).
Consortium members and service providers are thanked for valuable inputs at various stages and through
feedback sessions, Partners Forum meetings, and otherwise.
Acknowledgement is made of the fieldworkers and observers, especially through contracts with AC Nielsen,
assisted by Mictert in 2002, and the many qualified teachers, as well as the data-capturing team of Datanet under
the guidance of Pio Combrink.
Professor Johann Mouton is thanked for his comments and ongoing advice, mainly on methodology, during the
first half of the study.
Special mention has to be made of the contributions by district managers, regional or circuit managers
(institutional development officials), learning area specialists, school principals, teachers, learners and their
caregivers, for allowing researchers into their institutions, offices, classrooms and lives, and for making
themselves available for observations, interviews, the completion of questionnaires, and for making learners
available for sitting for performance tests.
The role of and contributions by the official QLP co-ordinators in the office of JET Education Services are also
acknowledged. The QLP co-ordinators became an institutionalised channel through which certain business was
conducted just so much more efficiently. Access to the districts and schools, and district-level and intervention
information collection are two cases in point. [The co-ordinators were: Alfred Mabina (Gauteng); Kedibone Boka
(Mpumalanga); Darwin Solomon (Northern Cape); Noel Daniels (Western Cape); Samuel Nkosi (KwaZulu-Natal,
Inanda and Ixopo districts); Thulani Dlamini (KwaZulu-Natal, Ubombo district); Rose Machobane (North West
Province); Nosipho Nxiweni (Free State); Vuyani Mrwetyana (Eastern Cape); and Maxwell Malatji (Limpopo).]
Marcel Croon, professor at Tilburg University in the Netherlands, is thanked for providing invaluable assistance
and training in 2002/3, and again in 2004/5, with the data modelling and analysis, and with related software.
HUMAN SCIENCES RESEARCH COUNCIL
viii
ACKNOWLEDGEMENTS
QLP SUMMATIVE EVALUATION
Free download from www.hsrcpress.ac.za
Shereno Printers must be mentioned for printing and packing research instruments within very tight schedules
during the three evaluation years, and also for producing the mid-term technical report, this summative report,
and its technical companion report.
Wordsmiths English Consultancy is acknowledged for language editing, formatting and laying out the manuscript
of the mid-term technical report in 2003, and for language editing this summative report and its technical
companion report in 2005.
Professor H.S. Bhola, Professor Brahm Fleisch, and Hersheela Narsee are thanked for reviewing the final
manuscript and for their helpful comments and assistance in improving it.
The following HSRC team members (listed alphabetically) are also noted with gratitude for their respective roles
as part of the evaluation team at different points in time during the final evaluation phase following 2003:
1
Brutus
Malada, Carla Pheiffer, Elsie Venter, Gerda Diedericks, Godwin Khosa, Heidi Paterson, Hendrik de Kock, Lerato
Mashego, Lolita Winnaar, Makola Phurutse, Matthews Makgamatha, Natalie le Roux, Nicolaas Claassen, Sannie
Reyneke, Sophie Strydom, Vijay Reddy, Xola Mati and Zinhle Kgobe.
HUMAN SCIENCES RESEARCH COUNCIL
ix
ACKNOWLEDGEMENTS
QLP SUMMATIVE EVALUATION
1 Participation and contributions during the mid-term and baseline evaluations are acknowledged in the appropriate reports following those periods.
Free download from www.hsrcpress.ac.za
PREFACE
This report marks the end of a unique, long-term and extensive teaching and learning intervention programme,
the Quality Learning Project (QLP). In concluding the evaluation activities of the QLP, a reflection on the
evaluation processes and findings is desirable. For this reason, some of the most important functions and roles of
the QLP and its evaluation are placed in perspective. In doing so, the report emphasises the crucial nature and
function of evaluation for teaching and learning, more so in view of the transformation context of the South
African education system.
At the most apparent and immediate level, this summative report provides a conclusive account to the sponsors
of the QLP of how successfully the funds of the project have been spent.
Additional value also lies in reflecting more deeply on the complexities inherent in large-scale and lengthy
endeavours such as the QLP. These reflections carry a positive verdict about the methods and models selected for
the QLP intervention programme and its evaluation. Finally, the reflections allow affirmation of the policy
decision implicit in undertaking the QLP work at the outset. In this regard, professional and policy experts can
find justification in the soundness, replicability and sustainability of the road travelled by the QLP.
This report also provides background information on the interventions as well as on the evaluation design and
methodology of the QLP, its findings, and the conclusions and recommendations derived from the findings.
HUMAN SCIENCES RESEARCH COUNCIL
x
PREFACE
QLP SUMMATIVE EVALUATION
Free download from www.hsrcpress.ac.za
ABBREVIATIONS USED FOR THE QLP DISTRICTS
Lu = Lusikisiki
Fl = Flagstaff
Li = Libode
ThM = Thabo Mofutsanyana
JSM = Johannesburg South Mega
SeW = Sedibeng West
In = Inanda
Ix = Ixopo
Ub = Ubombo
Mo = Moretele
Ma = Mafikeng
Ze = Zeerust
Ka = Karoo
Bo = Bolobedu
Ko = Konekwena
Zb = Zebediela
WCME = Western Cape Metropole East
Map showing location of Quality Learning Project (QLP) districts*
* District labels appear next to markers indicating the centroid of the particular district. It has to
be noted that for certain districts, such as Zeerust and Karoo, actual district areas may be quite large.
HUMAN SCIENCES RESEARCH COUNCIL
xi
ABBREVIATIONS
QLP SUMMATIVE EVALUATION
Free download from www.hsrcpress.ac.za
HUMAN SCIENCES RESEARCH COUNCIL
xii
EXECUTIVE SUMMARY
EXECUTIVE SUMMARY
The summative report of the Quality Learning Project (QLP) encapsulates the evaluation work of the project
spanning the past five years. In addition, it focuses on the successes and findings of the QLP, and the implications
and recommendations flowing from the evaluation study. The details underpinning the summative report can be
found in the technical companion report.
The success of the QLP
The QLP adopted a specific theoretical model for interventions designed to improve learning and teaching in
schools, and for evaluating the success of these interventions. As such, the hierarchical levels of the system
(districts, schools and classrooms) were taken into account. Observations were made at three points in time to
study trends and causal patterns. Comparisons were also made between project and control schools.
Performance targets for the QLP were set at the outset. QLP schools were to show an improvement, measured by
overall learner performance, against a comparable sample, by the end of 2004. What was required was:
• A 10% improvement in mean overall matriculation pass rate;
• A 10% improvement in mean mathematics pass rate; and
• A 10% improvement in mean English Second Language pass rate.
However, pass rates, when used as sole indicators, have certain weaknesses. For example, small increases from
low baselines (previous poor matriculation results) appear as large improvements. Moreover, schools are able to
artificially inflate Grade 12 pass rates by holding back potentially unsuccessful Grade 11 learners or by requiring
learners to take subjects at the Standard Grade (SG). These targets were therefore refined after the 2002 mid-term
evaluation, using categories that more reliably reflected school-performance outcomes. These categories were:
(a) The increase in the absolute number of learners passing, as an indication of the quantitative improvement
of learner results;
(b) The increase in the number of learners passing with university exemption, and with mathematics at
Higher Grade (HG), rather than Standard Grade (SG), as an indication of an improvement in quality of
the learner results; and
(c) The increase in matriculation pass rate, as an indication of improved efficiency in learner results.
Table I compares the performance of QLP evaluation schools with that of control schools in terms of the
final evaluation criteria adopted. It shows that the matriculation results of QLP schools consistently
improved more than those of control schools with respect to quantity, quality and efficiency.
QLP SUMMATIVE EVALUATION
Free download from www.hsrcpress.ac.za
[...]... between: (1) the QLP schools in all nine provinces and the control schools in the four provinces; and (2) the control schools in the four provinces and the corresponding QLP schools in these provinces Table C provides information on the change in Grade 12 learner performance, aggregated by province, from 2000 to 2004 in terms of: • The number of learners passing their matriculation examinations (e.g., 53... dramatic in QLP schools, at 332 percentage points (for QLP schools from all nine provinces against control schools in four provinces) and at 924 percentage points (direct comparison across four provinces) higher than control schools However, very low baselines contributed to some inconsistency and to the seemingly large changes The QLP and control schools were rather similar in terms of the increases in. .. points (94% improvement in QLP schools above the 87% in control schools) in direct comparisons made in the four provinces The figure changed to three percentage points in favour of the control schools when the full QLP group in all nine provinces was considered (the 84% improvement in QLP schools is below the 87% in the control schools) Results of the QLP Evaluation Analysis was conducted to determine... to schools is required in the Eastern Cape, and the North West and Limpopo provinces On the index for curriculum leadership, schools in nine of the 17 districts from five provinces fell within the low category, indicating that a great deal of improvement is required in this area In addition, the index for professional development indicated a widespread need for further intervention across schools in. .. QLP schools in all nine provinces and control schools located in the four provinces Such analyses would also be in the spirit of initial intentions to keep a close watch on the improvement in the matriculation results of the schools participating in the QLP as one of the criteria set for evaluating the project’s success Another basis for such a comparison is the initial criteria set for evaluating... in QLP schools above the 15% in control schools) when direct comparisons were made in the four provinces The figure dropped to 11 percentage points when the full QLP group in all nine provinces was considered (26% improvement in QLP schools above the 15% in control schools) In the case of Grade 12 mathematics (SG), improvements in QLP schools exceeded those in control schools by HUMAN SCIENCES RESEARCH... are many indications that service providers targeted interventions dynamically and interactively in areas that needed them most • Interventions improved functioning in areas targeted by the QLP This is evidenced in improved school functioning driven by good classroom and teacher interventions District interventions also played a role in improving school functioning • Improved functioning within the QLP... materials in teaching and learning The control schools seemed to have deteriorated in terms of classroom adequacy and the effect of shortages of classroom materials on teaching and learning, suggesting that the QLP schools have experienced improved physical conditions as a result of the interventions employed at these schools As revealed in the technical report, an increase in the monitoring of curriculum... approximately 17 percentage points higher than those in control schools For learners passing English (HG, Second Language), the difference, when compared across the four provinces that comprised both QLP and control schools, was 36 percentage points in favour of the QLP schools However, when comparing QLP schools in all nine provinces to the control schools from the four provinces, the increase was slightly... of the experimental schools undergoing interventions against the control schools, which did not receive any interventions As a result, the initial disadvantages of the loss of continuity in monitoring programme impact at the mid-point of the programme were turned into advantages for the final summative evaluation phase in 2004 Final 2002 and 2004 evaluation sample figures are shown in Figure B and Tables . entailed improving: • Learning outcomes in the languages of instruction and mathematics in Grades 8 to 12 in 524 schools; • The teaching of mathematics, and reading and writing skills in 524 schools; •. language in learning, given that reading and writing skills, abstract thinking, and producing meaning are central learning objectives. In addition, resource allocation should be prioritised according. Improving Learning in South African Schools: The Quality Learning Project (QLP) Summative Evaluation (2000 to 2004) A Kanjee & CH Prinsloo August 2005 HUMAN SCIENCES
Ngày đăng: 29/03/2014, 04:21
Xem thêm: Improving Learning in South African Schools pdf, Improving Learning in South African Schools pdf