Home Page
Previous Chapter 3 Ethical, Legal, and Human Relations Issues in Educational Research
Chapter 4 Complete
p. 89: #1(Explain the various ways that a literature review helps a researcher design a study)
Unless your study builds on the work of other researchers in your area of inquiry--it isnt likely to contribute much to body of research knowledge. Lit. reviews should take 3-6 months to do right.
1.A literature review helps a researcher design a study by narrowing the research problem. By looking at other studies you can find out how other researchers found fruitful lines of inquiry within a broad field of interest.
2. A literature review helps a researcher design a study by helping them seek new lines of inquiry. You can determine what has already been done in your area of interest and what possibilities have been overlooked.
3. A literature review helps a researcher design a study by helping him/her to avoid fruitless approaches. A lit. review will show studies that have been repeated using basically the same methodology and yielded no significant experiement of correlation result--so to repeat it would be useless.
4. A literature review helps a researcher design a study by giving him/her methodological insights. By paying attention to methodology you can find tips for how to design your study.
5. A literature review helps a researcher design a study by helping them do a grounded study. A grounded study is where data is collected first and then a theory is derived from that data. The lit review is done after so that the researcher can have a fresh perspective in how they think about the data.
#2(Describe the four major steps involved in conducting a literature review)
Prior to doing a lit review write a preliminary statement of your problem.
Step 1 Search preliminary sources- (indexes to bodies of literature)for studies, books, articles, that are related to study.
Step 2 Use secondary sources- (other researchers reviews of oliterature relevant to the problem and written by someone who didnt do the research, develop the theories, or express the opinions.)
Step 3 Read primary resources - (documents wtritten by the person who conducted the study, formulated the theory, or gave the opinions described in the document.)
Step 4 Synthesize the literature- synthesize what you have read. As you read you may reformulate your problem which takes you into a new area to review.
Purpose of the Review
1. Determine what is already known
2. Determine what is not known yet
3. Refine the problems or questions you plan to investigate
4. Determine how this study relates to and builds upon the existing knowledge base.
In his Pearls....Purpose of the Review
• Delimiting the research problem.
• Seeking new lines of inquiry.
• Avoiding fruitless approaches.
• Gaining methodological insights - "In reviewing research reports some individuals give scant attention to anything but the results reported."
• Identifying recommendations for further research."
• Seeking support for grounded theory. Pg. 90-91
#3(Describe the difference between preliminary, secondary, and primary sources, and the role of each type of source in the literature-review process)
Preliminary-- Index of resources
"The Thesaurus of ERIC Descriptors is a manual that helps you identify appropriate descriptors to use in searching CIJE and RIE for documents that are relevant to your research problem. A descriptor is a term that is used to classify all documents that contain information about the topic denoted by the term." Pg. 97-98
Secondary Resources- Publication written by authors who were not direct observers or participants in the events being described. Example textbooks, encylopedia, handbook
-They are useful because they compare primary source material.
-Most comprehesive secondary resource is Review of Educational Research.
-A good secondary resource not only tells you about the problem you are investigating, but provides a structure for positioning your proposed research so that others can appreciate its significance.
-A secondary resource that relies on a statistical procedure is called a meta-analysis.
(from his Pearls below)
"A secondary source is a document written by someone who did not actually do the research, develop the theories, or express the opinions that they have synthesized into a literature review.....secondary sources review research studies but not in detail.... A primary source is a document that was written by the individuals who actually conducted the research study or who formulated the theory or opinions that are described in the document." Pg. 92
Primary Resources- A direct report of an event by an individual who actually observed or participated in the research. In education research, a primary source generally is a report of a study by one or more of the persons who conducted it. Or a report by the authors of their own theory or opinion.
-Results and theories are reported in primary.
-Always better to read the most recent research first as they are built on the foundation of previous research.
-when you read primary resources try to find ways to classify them (usually by the research questions)
#7(Interpret the results of a meta-analysis of a set of quantitative research studies on a particular problem)
"Since the late 1970s, meta-analysis has been the most widely used method for synthesizing the statistical results of a group of studies on the same research problem....three major advantages:
1. it focuses on the magnitude of the effect observed in each study to be synthesized....
2. it provides a metric, called an effect size, that can be applied to any statistic and any measure....
3. it allows the reviewer to dertermine whether certain features of the studies included in the review affected the results that were obtained" Pg. 117
This is from Powerpoint
Meta-Analysis
-A quantitative technique to compare outcomes among a wide range of research studies
Effect sizes (ES) are calculated to express the strength of a reported relationship in these research studies.
-Meta-analysis uses Effect Sizes instead of “statistical significance” because ES estimates are not influence by sample size.
Basic Formula
-Mean Score for Experimental Group MINUS Mean Score for Control Group DIVIDED BY Standard Deviation of scores for Control Group
Interpretation
-Think of ES scores as z-scores in a normal distribution
ES scores can be translated into percentiles using the z-scores tables
-On the table find the ES number in the z-score column and use the percentage in the accompanying column.
Click for graph & table (the web page he has linked doesnt work anymore- http://www.cemcentre.org/ebeuk/research/effectsize/ESbrief.htm
But I think this is the table.. in this section on effect size on the same site.
http://www.cemcentre.org/renderpage.asp?linkid=30325016
Interpretation Example
-An ES of .2 would mean the “percentage of control group who would be below the average person in experimental group” would be 57.9%
-An ES of .5 would mean the “percentage of control group who would be below the average person in experimental group” would be 69.1%
-An ES of .8 would mean the “percentage of control group who would be below the average person in experimental group” would be 78.8%
Criticisms (Rosenthal)
-Bias in Sampling the Findings
-Garbage In and Garbage Out
Singularity and Nonindependence of Effects
-An Overemphasis on Individual Effects
-Combining Apples and Oranges
Bias in Sampling the Findings
-The File Drawer Problem
-Significant research results are published while non-significant results are not.
Solutions?
-Make every effort to contact all investigators who might have done research on the topic
-Analyze differences in different kinds of research: journal ES, dissertation ES, and presentations ES
Garbage In and Garbage Out
-The Quality Control Problem
-Research quality may vary greatly -- sampling; measurement; operationalizing the independent variable; statistical tools, etc.
Solutions?
-Use quality judgment scale on all studies and compute ES for different quality research
-Compute ES for different types of methodology (experimental, quasi-experiemental, etc)
Singularity and Nonindependence of Effects
-The Same Investigation (or Investigator) Problem
-Research studies may have more than one Effect Size OR many similar studies may originate from the same researchers
Solutions?
-Use different ES combination methods for independent and nonindependent studies
Overemphasis on Individual Effects
-The Multiple Influences Problem
-Real world effects are usually best explained by multifactorical models while meta-analysis usually only examines an individual effect relationship
Solutions?
-Need a clear understanding of single effects before more complicated multi-effect models can be constructed
Combining Apples and Oranges
-The Fruit Salad Problem
-Combining very different types of research obscures the uniqueness of each study by producing one “meaningless” number
{You no longer have distinctive tasting apples and oranges; you have a hodgepodge of tastes you cannot identify!}
Solutions?
-Pay attention to differences in sampling, measurement instruments; operationalization of independent variables and aggregate carefully.
Bottom Line
-The use of meta-analysis and Effect Sizes can be a very helpful way to identify and synthesize the magnitude of research results.
-It should NOT be done in a mechanical “by the formula” approach.
-It SHOULD be done by carefully considering the quality and methodologies of the collected research studies.
#10(State several criteria that are useful in judging the merits of a quantitative or qualitative study)
1. Relevance to the research problem
2. Quality 40-60% of research published is bad-- give weight to better research and how results might be effected by flaws.
pg 118 Qualitative
1. Is there a clear audit trail. This could be in the Lit review describing all procedures and decision rules used by the researcher.
2. Did they define the focus of the review- Did they define their constructs because this really defines the scop of the review. Only review doc. that match your definition of the construct.
3. Thorough review of literature.
4. Classify documents...
5. Create summary databases
6. Identify constructs and hypothesized causal links
7. Search for contrary findings and rival interpretations
8. Use colleagues or informants to corroborate findings
#11(Describe several flaws that weaken the reporting of a literature review)
1. Lit. review stands alone from other parts of dissertation or article. No clear path of how the study relates to other studies.
2. The review focuses on findings without looking at soundness of methodology-- no sense of how much confidence to place in the conclusions.
3. The review does not include a description of search procedures or the preliminary or secondary resources reviewed.
4. Writes isolated opinions and ideas-- the paragraphs seem disconnected...obvious need to fit ideas and opinions into a conceptual framework or a theoretical framework.
Note: Need to reflect on how info in document relates to a theoretical or conceptual framework and tie to your proposed study. Take note of the methodological problems reviewed in the study and connect to theory in your report-- discuss how you will deal with them. Also a major shortcoming in narrative reviews is subjectivity.
Home Page
Next Chapter 6 Selecting a Sample
Planning Page for this Wiki Check here to see who is doing what!
Comments (0)
You don't have permission to comment on this page.