Home Page
Previous Chapter 8 Collecting Research Data with Questionnaires and Interviews
Chapter 9 Complete
p. 253-4: #7(Explain the differences between criterion-related observer reliability, intra-observer reliability, and inter-observer reliability)
Criterion related observer reliability is the extent to which a trained observer’s scores agree with those of an expert observer such as the researcher who developed the observation instrument. Intra-observer reliability is the extent to which the observer is consistent in her observational codings. Both criterion related and intra related observer reliability use coding videotapes or audiotapes of events similar to those she will be seeing in the field. Inter-observer reliability is the extent to which the observers agree with each other during actual data collection. Pairs of observers collect data on the same events.
#8(Describe seven types of observer effects that weaken the validity and reliability of quantitative observational data, and procedures that can be used to minimize or avoid each effect)
An observer effect is any action by the observer that has a negative effect on the validity or reliability of the data they collect. Following are observer effects and steps that can be taken to control them. 1) Effect of the observer on the observed. Being distracted by the observer can result in the production of nonrepresentational data. Making several visits beforehand will result in the students and teacher taking the visit for granted reducing the effect. 2) Observer personal bias refers to errors in observational data that are traceable to characteristics of the observer. Reduce this by looking for and eliminating obvious sources of personal bias. 3) Rating errors occur when observational rating scales are used. Some observers form a response set that produces errors in their ratings on these scales. The three response sets are error of leniency (giving the majority high marks), error of central tendency (give majority midpoint scores), and halo effect (make decisions based on early impressions). To prevent these rating errors, either reconceptualize the rating scale or select and train observers more carefully. 4) Observer contamination occurs when the observer’s knowledge of certain data in a study influences the data he records about other variables. Keeping possibly contaminating information from the observers might reduce this effect. 5) Observer omission is the failure to record the occurrence of a behavior that fits one of the categories on the observational schedule. Cause is personal bias or when behaviors being observed occur too frequently or rarely. Ways to avoid this is simplifying the observation schedule or assign multiple observers to a setting. Providing cues and reminders may help maintain the observer’s vigilance. 6) Observer drift is the tendency for observers gradually to redefine the observational variables, so that the data they collect no longer reflect the definitions they learned during training.. Observer drift can be avoided by starting to collect data immediately following training and for long term observation, hold weekly refresher training sessions 7) Reliability decay is the tendency for observational data recorded during the later phases of data collection to be less reliable than those collected earlier. Avoid this by frequently checking on observers during the course of study to keep them performing at a satisfactory level. Maintaining observer motivation should prevent most of the above effects.
#9(State three ways in which observation in quantitative research differs from observation in qualitative research)
1) Observers in a qualitative study do not seek to remain neutral or objective about the phenomena being observed. They may include their own feelings and experiences in interpreting their observations. 2) The focus of qualitative observation is much more emergent and focus can change to new phenomena as new research questions emerge while data collection in quantitative research generally is driven by a priori hypotheses, questions, or objectives. 3) The focus of observation is much wider in qualitative research.
#14(Describe four types of observer effects in qualitative research and procedures that can be used to minimize each effect)
1) Reactions of program participants and staff to the observer’s presence. Observers should describe and analyze these effects as part of the research.
2) Effects on the observer during the course of study. Observers should describe and analyze these effects of their own reactions as part of the research.
3) The observer’s personal predispositions or biases. The researcher should follow established procedures for validating and verifying data analyses so as to reduce any distortions that may have been introduced by the researcher’s predispositions. These procedures include testing alternative explanations for research findings, use both qualitative and quantitative research methods to examine a phenomenon, use multiple observers and researchers, examine findings from various theoretical perspectives, and provide detailed reports on the findings so readers can “audit” the findings.
4) Observer incompetence. Observers need sufficient preparation and training to collect the data required by the research problem and approach.
#15(State the advantages and limitations of unobtrusive measures and the study of material culture).
Unobtrusive measures or non-reactive measures are used to avoid observer effects in quantitative research. Data is collected in a natural setting and individuals are not aware they are being observed. Advantages of using unobtrusive measures are especially useful then used in conjunction with conventional reactive measures because they involve using a different approach to measurement. Using several different kinds of instruments to measure the same variables, yielding similar results, increase confidence that the results are valid. (see figure 9.4 on p. 276 for examples of unobtrusive measures). The limitations of using unobtrusive measures follow: 1) the validity of some unobtrusive measures is uncertain. 2) The reliability of unobtrusive measures often is difficult to establish. Even for unobtrusive measures for which reliability can be computed, the data are of limited use to other researchers because most such measures are designed to study a very specific attitude of behavior pattern and are rarely used more than once. 3) Use of unobtrusive measures raises ethical issues involving informed consent and invasion of privacy. Informed consent poses a problem because making individuals aware that they are being studies jeopardizes the nonreactivity of any subsequent measurement. Therefore it becomes essential in some conditions to forego informed consent. An instructional review board may OK this if the individuals being studied will incur no risk, their anonymity will be maintained, it is impossible to conduct the study under the condition of informed consent, and the study promises to produce significant results.
Material culture refers to the various objects (artifacts) created by different groups throughout history. The first type of material culture is one that serves a communicative, representational function such as the study of text and signs. The second type is called practice-oriented material culture and is distinguished by its association with particular practices and meanings. Tools and symbolic associations about tools are examples of practice-oriented material culture. Observation of practice-oriented material culture is important because it gives qualitative researchers access to groups of people who have not left a written history. A disadvantage is that researchers can’t check the validity of their interpretations of the social reality it represents by interviewing people who produced it.
Home Page
Next Chap 10 Nonexperimental Research: Descriptive and Causal-Comparative Designs
Planning Page for this Wiki Check here to see who is doing what!
Comments (0)
You don't have permission to comment on this page.