CHAPTER METHODS OF MEASURING BEHAVIOR Nguyen Tien Dzung Hanoi University of Science and Technology Email: dungnt-fem@mail.hut.edu.vn Website: http://dungnt.tk Chapter Overview Deciding on a Method Tests and Their Development Types of Tests Observational Techniques Questionnaires © 2012 NTD Research Methods - Chapter Deciding on a Method to Measure Behavior: Some Important First Considerations ● Is the tool you propose to use reliable and valid? ● Base your choice of research tools on how you have asked the research question © 2012 NTD Research Methods - Chapter Tests and Their Development ● A test should measure the nature and extent of individual differences ● A good test differentiates people based on true scores © 2012 NTD Research Methods - Chapter Why Use Tests? ● ● ● ● ● Help determine outcomes of experiments Can be used to diagnose strengths and weaknesses Assist in placing individuals in appropriate programs Assist in selecting applicants Used to evaluate a program’s effectiveness © 2012 NTD Research Methods - Chapter Types Of Tests: Overview Types of Tests What It Does Achievement Test Assesses an individual’s knowledge of a specific area Attitude Test Assesses an individual’s feelings about an issue Personality Test Assesses stable individual behavior patterns © 2012 NTD Research Methods - Chapter 6 Achievement Test Sources ● Standardized ● Commercially prepared for wide use ● Scoring instructions included ● Researcher-made ● Designed by user for specific purpose ● Scoring instructions specific to test © 2012 NTD Research Methods - Chapter Achievement Tests Referencing What Comparison Do Tests Make? ● Norm-referenced—Individual’s scores are compared to the scores of other individuals ● Criterion-referenced—Individual’s scores are compared to defined performance standards © 2012 NTD Research Methods - Chapter Multiple-choice Achievement Items Anatomy of a Multiple-Choice Item 12 Intelligence tests that are given to preschool children a favor middle-class children STEM Clearly written CORRECT ANSWER b have questionable construct validity c are based on motor skills d are no fun at all DISTRACTERS Should be plausible (b & c), not easily ruled out (d) © 2012 NTD Research Methods - Chapter To Use Or Not To Use? ● Advantages ● ● ● ● ● ● ● Disadvantages Versatile Easy to score Simple to take Poor writers not penalized Good items used again Good distracters are diagnostic ● Hard to fake correct answer © 2012 NTD ● Limit student’s options ● No opportunity to practice writing ● Some people don’t well on them ● Limits content to be assessed ● Items must be well written Research Methods - Chapter 10 Thurstone Scales Method of Equal Appearing Intervals ● Statements indicating an attitude toward a topic are written ● Judges rank the statements from least favorable to most favorable ● Statements receiving consistent ratings are given the average score ● A set of statements covering the entire range of attitudes is selected © 2012 NTD Research Methods - Chapter 17 Thurstone Scales Administration ● Respondents check items with which they agree ● Well-formed attitudes indicated by consistently checking either high or low items ● Poorly-formed or inconsistent attitudes indicated by inconsistent pattern or by checking off many neutral items © 2012 NTD Research Methods - Chapter 18 Likert Scales ● Statements indicating an attitude toward a topic are written ● Items with clearly positive or negative attitudes are selected ● Statements are listed with a space for respondent to indicate degree of agreement © 2012 NTD Research Methods - Chapter 19 A Likert Scale Directions: Indicate to what extent you agree or disagree with the statements listed below by circling one of the following: SA means that you strongly agree with the statement A means that you agree with the statement U means that you are undecided about the statement D means that you disagree with the statement SD means that you strongly disagree with the statement Item Rating Government has no business funding child care programs SD D U A SA Child care should be supported by federal, state, and local tax dollars SD D U A SA © 2012 NTD Research Methods - Chapter 20 Scoring Likert Responses Method Of Summated Ratings Item Rating Government has no business funding child care programs SD D U A SA Child care activities are supported by federal, state, and local tax dollars SD D U A SA ● Items are weighted ● Weights of unfavorable items are reversed ● Average score is computed © 2012 NTD Research Methods - Chapter 21 Personality Tests ● Projective tests ● Present respondent with ambiguous stimulus ● Structured tests ● Questions are objective ● Example: true-false, multiple choice, yes-no © 2012 NTD Research Methods - Chapter 22 Observational Techniques ● Researcher observes and records ● Does not interfere with behavior © 2012 NTD Research Methods - Chapter 23 Techniques For Recording Behavior Technique How it Works Example Duration recording The researcher records the length of time that a behavior occurs How much time is spent in verbal interaction between two children? Frequency recording The researcher records the number of times a behavior occurs How often are questions asked? Interval recording The researcher observes a subject for a fixed amount of time Within a 60-second period, how many times members of the group talk to another person? Continuous recording The researcher records everything that happens During a 1-hour period, all the behavior of a 6-year-old boy is recorded © 2012 NTD Research Methods - Chapter 24 Observational Techniques? Be Careful! ● Pitfalls to Avoid ● ● ● ● © 2012 NTD Observer effects Observer bias Fatigue Changing definitions Research Methods - Chapter 25 Questionnaires (Mail Survey) ● What they are ● Paper and pencil tests with structured questions ● Self-administered © 2012 NTD Research Methods - Chapter 26 Questionnaires ● Advantages ● ● ● ● ● Can be mailed out Survey broad geographic area Cheaper than one-on-one interview Respondents may be more honest Data easy to share with other researchers ● Disadvantage ● Low return rate © 2012 NTD Research Methods - Chapter 27 Basic Assumptions of a Questionnaire ● Does not make unreasonable demands upon the respondent ● Does not have a hidden purpose ● Requests information that respondents presumably have ● Contains interesting questions ● Does not request information that could be obtained by other means © 2012 NTD Research Methods - Chapter 28 The Questions ● The questionnaire contains questions that can be answered ● The questionnaire contains questions that are straightforward © 2012 NTD Research Methods - Chapter 29 The Format ● The presentation is attractive, professional, and easy-tounderstand ● Questions and pages are clearly numbered ● Directions are clear and explicit ● Questions are objective ● Questions are ordered sensibly ● Transitions are used from one topic to the next © 2012 NTD Research Methods - Chapter 30 The Importance of Using a Cover Letter ● Informs the recipient about the research ● Establishes the importance of the research ● Makes the recipient a part of the research © 2012 NTD Research Methods - Chapter 31 [...]... number of times a behavior occurs How often are questions asked? Interval recording The researcher observes a subject for a fixed amount of time Within a 60-second period, how many times do members of the group talk to another person? Continuous recording The researcher records everything that happens During a 1-hour period, all the behavior of a 6-year-old boy is recorded © 2012 NTD Research Methods - Chapter. .. true-false, multiple choice, yes-no © 2012 NTD Research Methods - Chapter 6 22 4 Observational Techniques ● Researcher observes and records ● Does not interfere with behavior © 2012 NTD Research Methods - Chapter 6 23 Techniques For Recording Behavior Technique How it Works Example Duration recording The researcher records the length of time that a behavior occurs How much time is spent in verbal interaction... don’t ● Item analysis provides two measures of a question’s ability to discriminate ● Difficulty index ● Discrimination index © 2012 NTD Research Methods - Chapter 6 11 Computing Indices First Steps 1 Rank scores from highest to lowest 2 Choose top 27% of scores for ―high‖ group 3 Choose bottom 27% of scores for ―low‖ group © 2012 NTD Research Methods - Chapter 6 12 Computing Indices 12 Intelligence... NTD Research Methods - Chapter 6 20 Scoring Likert Responses Method Of Summated Ratings Item Rating Government has no business funding child care programs SD D U A SA Child care activities are supported by federal, state, and local tax dollars SD D U A SA ● Items are weighted ● Weights of unfavorable items are reversed ● Average score is computed © 2012 NTD Research Methods - Chapter 6 21 Personality... Research Methods - Chapter 6 13 Computing Indices Difficulty index ● Proportion who answered item correctly Discrimination index ● Proportion in high group who answered correctly – proportion in low group who answered correctly NCh  NCl D T NCh  NCl d 0.5T NCh = number of people in high group answering correctly NCl = number of people in low group answering correctly T = total number of people in high... average score ● A set of statements covering the entire range of attitudes is selected © 2012 NTD Research Methods - Chapter 6 17 Thurstone Scales Administration ● Respondents check items with which they agree ● Well-formed attitudes indicated by consistently checking either high or low items ● Poorly-formed or inconsistent attitudes indicated by inconsistent pattern or by checking off many neutral items... © 2012 NTD Research Methods - Chapter 6 29 The Format ● The presentation is attractive, professional, and easy-tounderstand ● Questions and pages are clearly numbered ● Directions are clear and explicit ● Questions are objective ● Questions are ordered sensibly ● Transitions are used from one topic to the next © 2012 NTD Research Methods - Chapter 6 30 The Importance of Using a Cover Letter ● Informs... should serve gourmet food _ _ _ My parents don’t appreciate how smart I am _ _ _ My professors don’t appreciate how smart I am, either _ _ _ The day before Thanksgiving should be a holiday Final exams should be elective © 2012 NTD Research Methods - Chapter 6 16 Thurstone Scales Method of Equal Appearing Intervals ● Statements indicating an attitude toward a topic are written ● Judges... return rate © 2012 NTD Research Methods - Chapter 6 27 Basic Assumptions of a Questionnaire ● Does not make unreasonable demands upon the respondent ● Does not have a hidden purpose ● Requests information that respondents presumably have ● Contains interesting questions ● Does not request information that could be obtained by other means © 2012 NTD Research Methods - Chapter 6 28 The Questions ● The questionnaire... or by checking off many neutral items © 2012 NTD Research Methods - Chapter 6 18 Likert Scales ● Statements indicating an attitude toward a topic are written ● Items with clearly positive or negative attitudes are selected ● Statements are listed with a space for respondent to indicate degree of agreement © 2012 NTD Research Methods - Chapter 6 19 A Likert Scale Directions: Indicate to what extent you