[...]... accept, discard, or revise items The appropriate selection and arrangement of suitable MC items on a test can best be accomplished by measuring items against three indices: • Item facility (IF) (or Item difficulty) • Item discrimination (ID)/Item differentiation • Distractor efficiency Item facility (IF) • is the extent to which an item is easy or difficult for the proposed group of test- takers • A too easy... proposed group of test- takers • A too easy item (e.g., 99% of the test- takers get it right) • A too difficult item (99% get it wrong) → Does nothing to separate high-ability and low-ability test- takers Item facility (IF) The formula looks like this Item Facility (IF) • There is no absolute IF value that must be met • The appropriate test items will have IFs that range between 15 and 85 Item Discrimination... acceptable and unacceptable ID indices Distractor Efficiency is the extent to which • the distractors “lure” a sufficient number of test- takers, especially lower-ability ones, and • those responses are somewhat evenly distributed across all distractors Note: C is the correct response Choices A B C* D E High-ability students (10) 0 1 7 0 2 Low-ability students (10) 3 5 2 0 0 DE (cont.) The item might be improved... and creative prose; • Artwork, photos, newspapers or magazine clippings; • Audio and/or video recordings of presentations, demonstrations, etc • Journals, diaries, and other personal reflections; • Tests, test scores, and written homework exercises; • Notes on lectures; and • Self- and peer-assessments – comments, evaluations, and checklists Attributes of Portfolios • • • • • • Collecting Reflecting... absolute IF value that must be met • The appropriate test items will have IFs that range between 15 and 85 Item Discrimination (ID) • is the extent to which an item differentiates between highand low-ability test- takers • An item on which high-ability students and low-ability students score equally well would have poor ID because it did not discriminate between the two groups • An item that garners correct... Provide tangible evidence of a student’s work, • Facilitate critical thinking, self-assessment, and revision processes, • Offer opportunities for collaborative work with peers, • Permit assessment of multiple dimension of language learning Guidelines on portfolios development • • • • State objectives clearly, Give guidelines on what materials to include, Communicate assessment criteria to students,... essays and reports Reviewing portfolios Responding to journals Advising on a student’s plan for an oral presentation Assessing a proposal for a project Giving feedback on the results of performance on a test Clarifying understanding of a reading Exploring strategies-based options for enhancement or compensation Focusing on aspects of oral production Checking a student’s self-assessment of a performance... assessment of (general) competence 3 Metacognitive assessment more strategic in nature setting goals and maintaining an eye on the process of their pursuit) 4 Socioaffective assessment 5 Student-generated tests Indirect assessment Metacognitive assessment . Year's Day will "cut off fortune." Designing multiple choice test items Week 8 Multiple choice • MC items are all receptive, or selective: test- takers choose from a set of responses. Weaknesses in multiple- choice items • The technique tests only recognition knowledge. • Guessing may have a considerable effect on test scores. • The technique severely restricts what can be tested. • . the correct response, while the others serve as distractors. Multiple choice items • A preferred mode for large-sale tests, co’z MC items provide an “objective’ means for determining correct