Home Page

Previous Chapter 7 Collecting Research Data with Tests and Self-Report Measures

p. 221: #1(Describe the relative advantages and limitations of questionnaires and interview in educational research).

Questionnaires and interviews can be used to collect data about phenomena that is not directly observable (e.g. inner experiences, opinions, values, interests, etc.) They are more convenient to use than direct observation when used for collecting data on observable behavior. The advantages of using questionnaires are as follows: 1) can be given to large groups, 2) respondents can complete the questionnaire at their own convenience, answer questions out of order, skip questions, take several sessions to answer the questions, and write in comments. The cost and time involved in using questionnaires is less than with interviews. The disadvantages include inability to probe deeply into respondents’ beliefs, attitudes and inner experiences. Modifications to the questions can’t be made once the questionnaire has been distributed. Interviews typically ask oral questions of individuals. Notes are taken using audio/visual equipment, computers, or are hand written during or following the interview. The major advantage of the interview is its adaptability in controlling the response situation, scheduling a mutually convenient time and place, and controlling the sequence and pacing of the questions asked. Questions can be modified as needed. Interviews can probe deeply into respondents' beliefs, attitudes and inner experiences by following up with questions to obtain more information and clarify vague statements. Interviewers can build trust and support with respondents, making it possible to obtain information that might not have been revealed using another data collection. The limitations of interviews include difficulty in standardizing the interview situation so the interviewer doesn’t influence the respondents answers. Interviews can’t provide anonymity for the respondent, but reporting of responses can be anonymous.

#2 (Describe each step in constructing and administering a research questionnaire--e.g., questionnaire design guidelines)

Step 1: Define research objectives: Start with a broad topic, then narrow it by asking five questions about: the time frame, geographical location, conducting a broad descriptive study versus specifying and comparing different subgroups, what aspect of the topic you want to study, and how abstract is your interest.

Step 2: Selecting a sample: Identify the target population from which the sample will be selected.

Step 3: Designing the questionnaire (Figure 8.1, p. 226): keep it short; avoid technical terminology; don’t use the term questionnaire or checklist on the form; make it attractive; organize the items so they are easy to read and complete; numbers pages and items; include return address information on the form and include a postage paid addressed envelope; directions should be clear, brief, in bold print; organize questionnaire in a logical sequence; use a transitional sentence to change topics; begin with interesting, non-threatening topics; put threatening or difficult items near the end; don’t put important items at the end of a long questionnaire; provide a rationale for the test items; provide examples of responses for difficult or hard to understand items; avoid terms like several, most and usually; be brief in stating each item; avoid negatively stated items; avoid double-barreled items that require a response to two separate ideas with a single answer; when a general question and a related specific question are to be asked together, ask the general one first in order to avoid the respondents narrowing their focus; avoid biased or leading questions.

Additional considerations: Use coding to keep respondent identity anonymous. Design items with respondents in mind so terminology is understood. Define any terms that may lead to confusion. Use a closed form with pre-specified (i.e. multiple choice) response choices. Open forms require short or long written responses. Use a scale rather than a one-item test when measuring attitudes. The Likert scale uses a five-point scale ranging from strongly disagree to strongly agree. Web questionnaires are easy to use, but measures need to be taken to avoid sampling bias and assure anonymity.

Step 4: Pilot-testing the questionnaire: Use a sample from your target population to pilot test the questionnaire. Provide space for them to criticize or make suggestions for improving the items. Ask them to state in their own words what they think each item means. Revise and retest the questionnaire.

Step 5: Pre-contacting the sample: A greater response rate occurs when the researcher identifies herself and discusses the purpose of the study and requesting cooperation

Step 6: Writing a cover letter: Be brief, explaining the purpose of the study, assuring how confidentiality will be maintained. Include flattery and mention professional affiliations. Mention rewards if you plan to provide them and specify a return date. Attend to the appearance of the cover letter.

Step 7: Following up with non-respondents: Send a follow-up letter with another copy of the questionnaire. Stress the importance of the study and the contribution the respondent can make.

Step 8: Analyzing questionnaire data: Can use qualitative or quantitative measures to analyze data

#5(Explain the effects of respondents’ knowledge and the number of items on attitude measurement in questionnaires)

A questionnaire that measures attitudes should be constructed as an attitude scale and should have at least 10 items. A pilot test should be used to determine if respondents have sufficient knowledge and understandings to express a meaningful opinion about the topic. Other wise their responses to the attitude scale would be of questionable value. Including a “no opinion” option or including several information gathering questions out the beginning of the questionnaire would screen out those respondents who have little or no knowledge about the subject.

#7(Describe several features of a cover letter that are likely to increase the response rate to a mailed questionnaire)

Be brief, explaining the purpose of the study, assuring how confidentiality will be maintained. Explain the conditions that have been established for informed consent. Include flattery and mention professional affiliations. Mention rewards if you plan to provide them and specify a return date. Attend to the appearance of the cover letter.

#8(Describe several strategies for following up with nonrespondents in order to maximize the response rate to a questionnaire)

Try a making a different, more professional appeal in the follow-up letter. Mention that you are confident the respondent wanted to participate in the study but perhaps some oversight on the part of the researcher prevented them from completing and returning the questionnaire. Stress the importance of their contribution to the research study. Include another copy of the questionnaire with a stamped self-addressed return envelope.

#9(Describe each step in preparing and conducting a research interview)

Step 1: Define the purpose of the study: Determines the nature of the interview. Three major types of research interviews. The key informant interview collects data from individuals who have special knowledge or perceptions that would not otherwise be available to the researcher. Survey interviews are used to supplement data that have been collected by other methods. Projective techniques use ambiguous stimuli to elicit subconscious perceptions that cannot be observed in the natural setting or solicited through regular interviewing.

Step 2: Selecting a sample: Use quantitative or qualitative sampling techniques. Use individual respondents or a focus group.

Step 3: Designing the interview format: In quantitative research all respondents have the same interview experience so all data can be compared. All variables are pre-specified. In qualitative research, there is more flexibility because the goal is to help respondents express their view of phenomena in their own terms. Questions are not as specific, but are broader in nature

Step 4: Developing questions: Questions are either developed in advance or during the interview. An interview guide can be used for more structured interviews. An interviewer needs to be able to think fast on his feet to conduct an unstructured interview.

Step 5: Selecting and training interviewers: The most important criteria are the interviewer’s ability to relate to respondents positively. Using variables as social class, gender, race and age to match interviewers with respondents such are likely to produce more valid responses. Interviewers can be selected from the respondent target population. Amount of training for interviewers depends on the depth and structure of the interview. Training should familiarize the interviewers with the interview guide so that they will be able to conduct the interview in a conversational manner without being too dependent on the guide. The second phase of training involves practice interviews receiving feedback until their performance is polished and reaches the desired level of structure, objectivity and reliability.

Step 6: Pilot-testing the interview: The interview guide and procedures should be pilot tested to ensure they will yield reasonably unbiased data. Questions may need to be rephrased or procedures may need to be revised. Threatening questions need to be altered. Opening statements and methods of recoding responses need to be evaluated to ensure rapport is maintained.

Step 7: Conducting the interview: (see guidelines Figure 8.5 p. 247). Interviewing tasks include deciding how to present oneself, establishing an appropriate level of rapport, gaining trust when sensitive topics are being discussed, understanding the respondent’s language and culture, being sensitive to nonverbal language, and recording interview data. Note taking using the interview guide facilitates data analysis. However it may disrupt the effectiveness of the communication between the interviewer and the respondent. Tape recoding provides a more accurate record of the interview. However, it may inhibit the responses. Computer software can also be used for recording responses.

Step 8: Analyzing interview data: For closed- form interview questions calculate the percentage of respondents who indicated each response option for each item. Analysis of responses to open-form questions requires the development of a category system. Interview data collected for a qualitative research study can use the grounded-theory approach (Ch. 14). The choice of data analysis approach is determined by the type of qualitative research being conducted

Home Page

Next Chapter 9 Collecting Research Data through Observation and Content Analysis

Planning Page for this Wiki Check here to see who is doing what!