Inquiry Synthesis Project Publications

Introduction

Phases of the Study

Introduction

The Center for Science Education (CSE) at Education Development Center, Inc., (EDC) completed a four-year study funded by the National Science Foundation that addressed the question: what is the impact of inquiry science instruction on student outcomes? This project synthesized research that met the following criteria:

• Was completed between 1984 and 2002
• Was conducted with K–12 students
• Had at least one research question that was about the effect of an instructional intervention in science on student outcomes
• Focused on at least one student instructional intervention in science
• Described the student instructional intervention with sufficient specificity
• Reported student outcomes

The synthesis included 138 studies with a dependent variable of science subject matter content knowledge. Of these, 105 were conducted in the United States; 41 studies focused on elementary grades; 48 on middle grades; and 49 on high school grades. There were 339 schools, 589 classrooms, and 13,066 students included as subjects of these studies; however, a number of studies did not report on these characteristics, and so these totals underreport the actual numbers. Of the 138 studies, 83 (60%) focused on physical science; 35 (25%) focused on life science; and 16 (12%) focused on earth/space science. A very small number (3%) examined the multiple content areas of physical and life science; and physical and life and earth science. The dependent variable—science-subject-matter content knowledge—was divided into six different finding types, which were expressed in physical science, life science, or earth/space science. The finding types are listed below, followed by the number of studies that had such a finding type as its dependent variable (some studies had more than one finding type):

1. Understanding(s) related to science concepts (104 studies, or 75%)
2. Understanding(s) related to science facts and vocabulary (28 studies, or 20%)
3. Understanding(s) related to science principles and theories (19 studies, or 14%)
4. Retention of science facts and vocabulary (9 studies, or 7%)
5. Retention of science concepts (11 studies, or 8%)
6. Retention of science principles and theories (3 studies, or 2%) 

The study provides a full account of what the field learned between 1984 and 2002 about the impact of inquiry science on student outcomes.

top

Phases of the Study

This project included three broad phases.

• Phase I—Report Collection: This phase involved conducting a search for research reports that met the criteria. The search was shaped by the objective of “casting a broad net” in order to identify and retrieve all reports that could potentially be part of the synthesis (a description of this process can be found in Technical Report 1: Generating the Synthesis Sample of Studies.
  
  

  There is a purposeful distinction made throughout this technical report between a “report” and a “study.” Often authors disseminate multiple reports from a single study. From Phase I through Phase II–stage one, reports are the primary unit of interest. From Phase II, stage two through Phase III, studies (which could be represented by multiple reports) are the primary unit of interest. Details of how this reconciliation process was done can be found in Technical Report 4: Report-Study Reconciliation Process.

• Phase II—Coding Process: Once retrieved, reports entered this phase, which comprised three stages.
  1. In stage one—inclusion/exclusion—each report was carefully screened and coded to determine if the inclusion criteria were met. These criteria include the following: the report focused on a science instructional intervention; the report included student outcomes; the outcomes were directly related to the instructional intervention; the report was completed between January 1, 1984, and December 31, 2002; the study was conducted with K–12 students; and the intervention was described with sufficient specificity. Reports that met all of these criteria were included in the synthesis dataset. Details about the development of the codebook used for the inclusion/exclusion coding and the process for establishing interrater agreement can be found in Technical Report 3: Operationalizing the Inclusion/Exclusion Coding Process.
  2. Stage two of the coding process, inquiry instruction description, involved developing and applying a detailed coding schema to describe the instructional intervention. This descriptive methodology is the subject of Technical Report 5: Operationalizing the Inquiry Science Instruction Coding Process. Technical Report 2: Conceptualizing Inquiry Science Instruction, articulates the theoretical underpinnings of the coding protocol for describing inquiry instruction.
  3. Stage three of coding, research rigor, context, and study findings, involved capturing all other relevant information about the methodological integrity of the research, the context of the study, covariates, comparison treatments, and study findings. The third stage of coding is detailed in Technical Report 6: Operationalizing the Coding of Research Rigor, Context, and Study Findings.

Each stage of Phase II afforded an opportunity to exclude a report or study due to misalignment with the data requirements of the synthesis. To remain included, reports/studies were required to include the appropriate variables, to provide sufficient information to code the variables of interest and to present evidence of a minimum threshold for inquiry science instruction. While the search for reports in Phase I was broad to better ensure that all eligible studies would be found, it was understood that a broad search would result in the collection of some number of reports that would not meet the minimum criteria. Thus, this multi-step process was designed to refine and narrow the sample of studies included in this synthesis in order to ensure that the analyses would be based on only those studies that adequately reported clear descriptions of an inquiry science intervention and were conducted with sufficient rigor to allow for drawing credible conclusions.

• Phase III of the project includes the analysis and dissemination of results. The findings of this study appear in the Journal of Research in Science Teaching. For more information,see Technical Report 7: Bibliography of Studies Included in Final Inquiry Synthesis Project Analyses.

Contact Us

Contact Jackie DeLisis at jdelisi@edc.org.

top

This material is based upon work supported by the National Science Foundation under Grant No. 0101766.