 |
|
NCES: 98025 May 1998 |
PDF & Related InfoThe sampling frame for the FRSS Public School Survey on Education Reform was constructed from the 1993-94 NCES Common Core of Data (CCD) public school universe file and included over 82,000 public elementary, middle, and high schools. Excluded from the frame were special education, vocational, and alternative/other schools, schools in the territories, and schools with the highest grade lower than grade 1.
A stratified sample of 1,360 schools-534 elementary schools, 375 middle schools, and 451 high schools-was selected for the survey. To select the sample, the schools in the frame were stratified by the three instructional levels, poverty status (based on the percent of students eligible for the free or reduced-price lunch program as recorded in the CCD file) within level, and enrollment size class within poverty status. Within these primary strata, schools were sorted by region and locale to induce limited additional stratification.
Since free or reduced-price lunch program participation data were missing for about 24 percent of the schools in the CCD, such schools were placed in a separate stratum for sampling purposes. (Note that an item on the survey questionnaire updated this information for all schools.) High poverty schools were oversampled. Such a design is reasonably efficient for the analysis of the survey results by poverty group within instructional level. Within each instructional level and poverty status group, the sample of schools was selected within size classes with probabilities roughly proportional to the square root of the enrollment of the school. The use of the square root of enrollment to determine the sample allocation is reasonably efficient for estimating both school-level characteristics and quantitative measures correlated with enrollment. Further, the proposed sample allocation permits limited analysis (along a single dimension) by instructional level, locale, and poverty status within level (table A- 1).
In April of 1996, questionnaires (see appendix D) were mailed to 1,360 public school principals. Seven schools were found to be out of scope (no longer at the same location or not serving the same population), leaving 1,353 eligible schools in the sample. Telephone followup was initiated in mid-May and data collection was completed on July 31, with 1,216 respondents. Principals completed 90 percent of the returned questionnaires; the remaining 10 percent were completed by other administrators at the school. Fifty-five percent of the surveys were returned by mail and 30 percent by fax, and about 15 percent of the responses were taken over the telephone. The final unweighted response rate was 90 percent. The weighted response rate was also 90 percent. Item nonresponse rates ranged from 0.0 to 1.0 percent.
For estimation purposes, sampling weights were used that reflect each school's overall probability of selection. These weights are also adjusted to compensate for differential nonresponse in the survey. The findings in this report are estimates based on the sample selected and, consequently, are subject to sampling variability. The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse or noncoverage) errors, errors of reporting, and errors made in the collection of the data. These errors can sometimes bias the data. Nonsampling errors include such problems as the differences in the respondents' interpretations of the meaning of the questions; memory effects; misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; and errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used.
To minimize the potential for nonsampling errors, the questionnaire was pretested with knowledgeable respondents like those who completed the survey. During the design of the survey and the survey pretest, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous terms. The questionnaire and instructions were extensively reviewed by the Planning and Evaluation Service and the National Center for Education Statistics. Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Imputations for item nonresponse were not implemented, as item nonresponse rates were very low. Data were keyed with 100 percent verification.
The standard error is a measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size.
Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. For example, the estimated percentage of public schools that use content standards to a great extent in reading/language arts is 50 percent and the estimated standard error is 2.3 percent. The 95 percent confidence interval for this statistic extends from [50 - (2.3 x 1.96) to 50 + (2.3 x 1.96)], or from 45.5 to 54.5.
Estimates of standard errors were computed using a technique known as jackknife replication. As with any replication method, jackknife replication involves constructing a number of subsamples (replicates) from the full sample and computing the statistic of interest for each replicate. The mean square error of the replicate estimates around the full sample estimate provides an estimate of the variance of the statistic. To construct the replications, 50 stratified subsamples of the full sample were created and then dropped, one at a time, to define 50 jackknife replicates. A proprietary computer program (WESVAR), available at Westat, Inc., was used to calculate the estimates of standard errors.
The survey was conducted under contract with Westat, Inc., using the NCES Fast Response Survey System (FRSS). Westat's project director was Elizabeth Farris, and the survey manager was Carin Celebuski. Judi Carpenter and Shelley Burns were the NCES project officers. The data were requested by Nancy Loy and Daphne Hardcastle of the Planning and Evaluation Service (PES) of the U.S. Department of Education. The report was reviewed by the following individuals:
Outside NCES
Inside NCES
For more information about the Fast Response Survey System or the Public School Survey on Education Reform, contact Shelley Burns, Elementary/Secondary Statistics Division, Office of Educational Research and Improvement, National Center for Education Statistics.
Comprehensive reform: Efforts to improve education for all students by establishing high content and performance standards and redesigning the various components of the education system in a coordinated and coherent fashion to support students learning to the standards.
Disability: An impairment that substantially limits one or more of the major life activities of individuals.
ERIC: Educational Resources Information Center. ERIC is an education database, clearinghouse, and document reproduction service financed by the U.S. Department of Education.
High standards: Refers to recent and current education reform activities that seek to establish more challenging expectations for student achievement and performance, such as the National Council of Teachers of Mathematics standards for math, state- or local initiated standards in various subjects, and those outlined in Goals 2000.
School-parent compact: Voluntary written agreements between the school and parents on what each will do to help students succeed in school.
SSI/USI:: National Science Foundation's Statewide Systemic Initiatives and Urban Systemic Initiatives programs. For these programs, NSF has cooperative agreements with states and urban areas to undertake comprehensive initiatives for education reform in science, mathematics, and technology.
Classification Variables
Locale
City - a central city of a Metropolitan Statistical Area (MSA).
Urban fringe - a place within an MSA of a central city, but not within its central city.
Town - a place not within an MSA, but with a population greater than or equal to 2,500, and defined as urban by the U.S. Bureau of the Census.
Rural - a place with a population less than 2,500 and defined as rural by the U.S. Bureau of the Census.
Eligibility for free or reduced-price lunches through the National School Lunch Program: (available for 75 percent of the sample from the CCD-data for remaining schools taken from survey questionnaire)
Less than 35 percent - of students in the school eligible
35-49 percent - of students in the school eligible
50-74 percent of students in the school eligible
75 percent or more students in the school eligible
Title I funding
No Title I - School principal reported on the questionnaire that the school did not receive Title I funds in school year 1995-96.
Title I nonschoolwide program - School principal reported on the questionnaire that the school received Title I funds in school year 1995-96, but did not operate a schoolwide program.
Title I schoolwide program - School principal reported on the questionnaire that the school received Title I funds in school year 1995-96 and operated a schoolwide program.
