Slide _ LOGO _ _ _ Dinh Cong Triet _ _ _ Slide _ TEST DESIGNING & TEST SPECIFICATION BUILDING _ _ _ Summer training course- August 2011 _ _ _ Slide 10 Questions for thought _ _ _ _ _ _ _ Slide Speaking Reading _ Skills Listening Writing _ WHAT Knowledge _ to test? Language lexis semantics Grammar morphology _ syntax _ _ _ Slide _ Reading WHAT Reading aloud to test? Silent reading Subskills pronunciation • Identify main facts & details, Stresses & intonation _ • Relate cause & effect _ • Identify sequence of events, predicting outcomes • inferring meaning from contextual clues _ _ _ _ Slide Listening WHAT _ _ to test? Subskills • Identify main facts & details, _ • Relate cause & effect • Identify sequence of events, predicting outcomes • inferring meaning from contextual clues _ _ _ _ Slide _ Grammar Syntax (sentence structure) WHAT to test? Mechanics: e.g Punctuation, capital letter, comma, etc _ pronouns nouns Rules of grammar _ tenses verbs prepositions subject & verb agreement _ comparatives adjectives adverbs _ _ _ Slide _ Speaking quantity quality vocabulary Sentence complexity Content / ideas _ Language WHAT clarity syntax to test? grammar fluency confidence _ Pronunciation Presentation skills Organization accuracy appropriacy Eye contact format style _ _ _ _ Slide _ Writing quantity clarity Simple quality voc compound complex Sentence complexity Content / ideas _ Language WHAT to test? syntax accuracy Organization Paraphrasing appropriacy Topic sentence _ _ Supportive sentences _ _ _ Slide 10 Speaking Reading _ Skills Listening Writing _ WHAT Knowledge _ to test? Language lexis semantics Grammar morphology _ syntax _ _ _ Slide 11 _ Reading WHAT Reading aloud to test? Silent reading Subskills pronunciation • Identify main facts & details, Stresses & intonation _ • Relate cause & effect _ • Identify sequence of events, predicting outcomes • inferring meaning from contextual clues _ _ _ _ Slide 12 Listening WHAT _ _ to test? Subskills • Identify main facts & details, _ • Relate cause & effect • Identify sequence of events, predicting outcomes • inferring meaning from contextual clues _ _ _ _ Slide 13 _ Grammar Syntax (sentence structure) WHAT to test? Mechanics: e.g Punctuation, capital letter, comma, etc _ pronouns nouns Rules of grammar _ tenses verbs prepositions subject & verb agreement _ comparatives adjectives adverbs _ _ _ Slide 14 _ Speaking quantity quality vocabulary Sentence complexity Content / ideas _ Language WHAT clarity syntax to test? grammar fluency confidence _ Pronunciation Presentation skills Organization accuracy appropriacy Eye contact format style _ _ _ _ Slide 15 _ Writing quantity clarity Simple quality voc compound complex Sentence complexity Content / ideas _ Language WHAT to test? syntax accuracy Organization Paraphrasing appropriacy Topic sentence _ _ Supportive sentences _ _ _ Slide 16 What makes test questions difficult? _ _ _ THINKING LEVELS _ _ _ _ Slide 17 _ Designing test questions in accordance with the levels of thinking _ _ _ _ _ _ Slide 18 Bloom’s taxonomy _ _ _ _ Old version, 1956, Benjamin Bloom New version, 1990’s , Lorin Anderson _ _ _ Slide 19 BLOOM’S TAXONOMY Evaluation Argue, assess, justify, judge, rate, support, value, evaluate Synthesis Analysis _ _ Arrange, collect, compose, plan, construct, create, design, develop Analyse, calculate, categorise, criticise, contrast, distinguish Application Comprehension Classify, describe, discuss, explain, express, recognise, identify, report _ Apply, choose, demonstrate, illustrate, solve, use, interpret Knowledge _ Arrange, define, list, memorise, name, relate, recall, repeat, state _ _ _ Slide 20 Bloom’s taxonomy – Knowledge level Useful verbs Tell List Describe Relate Locate Write Find State name _ Sample questions stems • • • • • • • • • • What happened after….? How many… ? Who was it that…… ? Can you name the …….? Describe what happened at… ? Who spoke to… ? Can you tell why….? Find the meaning of … ? What is …….? Which is true or false…….? _ _ _ _ _ _ Slide 21 Bloom’s taxonomy – Comprehension level Useful verbs Sample questions stems Explain Interpret Outline Discuss Distinguish Predict Restate Compare Translate Describe • Can you write in your own words….? • Can you write a brief outline….? • What you think could happen next….? • Who you think….? • What was the main idea… ? • Who was the key character… ? • Can you distinguish between….? • Can you provide an example of … ? • What you mean ….? • Can you provide a definition for….? _ _ _ _ _ _ _ Slide 22 Bloom’s taxonomy – Application level Useful verbs Solve Show Use illustrate Construct Complete Examine Classify _ Sample questions stems • Do you know another instance where….? • Could this have happened in ….? • Can you group by charateristics such as….? • What factors would you change if… ? • What questions would you ask of… ? • From the information given, can you develop a set of instructions about……? • Would this information be useful if you had….? _ _ _ _ _ _ Slide 23 Bloom’s taxonomy – Analysis level Useful verbs Analyse Examine Compare Contrast Investigate Categorise Identify Explain Separate _ Sample questions stems • Which events could have happened….? • What you see as other possible outcomes…? • How was this similar to….? • Can you explain what must have happened when…? • What are some of the problems of ….? • Can you distinguish between….? • What were some of the motives behind….? _ _ _ _ _ _ Slide 24 Bloom’s taxonomy – Synthesis level Useful verbs Create Invent Compose Predict Plan Construct Design Imagine Propose devise _ Sample questions stems • Can you design a… to….? • Can you see a possible solution to ….? • If you had access to all resources how would you deal with….? • What would happen if … ? • How many ways can you……? • Can you write a new recipe for a tasty dish? • Why not compose a song about… ? _ _ _ _ _ _ Slide 25 _ Bloom’s taxonomy – Evaluation level Useful verbs Judge Select Choose Decide Justify Debate Verify Argue Recommend determine Sample questions stems • Is there a better solution to … • Judge the value of… • Can you defend your position about… ? • Do you think……is a good or bad thing? • What changes to ……would you recommend? • Do you believe….? • Are you a … person? • How would you feel if….? • How effective are… ? • What you think about…….? _ _ _ _ _ _ Slide 26 _ BLOOM’S TAXONOMY Skills Subskills Question types Levels _ _ _ _ _ _ Slide 27 _ BLOOM’S TAXONOMY Skills Subskills Question types Levels _ _ _ _ _ _ Slide 28 _ Skills Reading Sub-skills       Listening        Question types Identifying main facts & details (FD) Recognizing cause & effects relationship (CE) Inferring meaning from contextual context ( ICC) Predicting outcomes (PO) Sequence of events (SOE) Evaluating writer’s attitude & view     Identifying facts & details relate cause & effect identify sequence of events Inferring meaning from contextual clues Predicting out comes Inferring speaker’s attitude & emotion (sad, happy, worried) Inferring relationship between speakers         Question levels True/ false Re-arrangement Structured Open-ended (subjective response, free writing) MCQ (wh-question, matching, completion with options) Cloze procedure MCQ with pictures or without pictures Fill in the blanks/ completion item Labelling Matching True / false sequencing _ _ _ _ _ _ Slide 29 _ BARRETT’S TAXONOMY _ _ _ _ _ _ Slide 30 (MCQ, T/F, gap-fill, matching, sequencing, open-ended) Barrett’s (LC, RO, IC, Eva, Appr) _ Question levels Bloom’s(Know,Comp, App, Ana, Syn, Eva) _ Question types knowledge Table of Specifications LSRW Curriculum specs Time allocation Skills & subskills _ Mark allocation Question Sections/numbers _ _ _ _ Slide 31 _ LOGO _ _ _ Dinh Cong Triet _ _ _ ... _ Slide 22 Bloom? ?s taxonomy – Application level Useful verbs Solve Show Use illustrate Construct Complete Examine Classify _ Sample questions stems • Do you know another instance... _ Slide 21 Bloom? ?s taxonomy – Comprehension level Useful verbs Sample questions stems Explain Interpret Outline Discuss Distinguish Predict Restate Compare Translate Describe • Can you write... 24 Bloom? ?s taxonomy – Synthesis level Useful verbs Create Invent Compose Predict Plan Construct Design Imagine Propose devise _ Sample questions stems • Can you design a… to….?