Skip Navigation
small header image
Occupational Programs and the Use of Skill Competencies at the Secondary and Postsecondary Levels, 1999
NCES: 2000023
February 2000

Appendix A: Methodology and Technical Notes

Surveys and Samples

The tabular statistics in this E.D. TAB report present a subset of the data collected from the surveys Vocational Programs in Secondary Schools and Occupational Programs in Postsecondary Education Institutions. The surveys were conducted through the Fast Response Survey System (FRSS) and the Postsecondary Education Quick Information System (PEQIS), respectively. To select a nationally representative sample of public secondary schools for the FRSS survey, a stratified sample of 1,200 public secondary schools, including 600 vocational schools and 600 comprehensive (regular) schools, was selected from the 1996-97 Quality Education Data (QED) National Education Database. The QED database is compiled from a variety of sources, including the National Center for Education Statistics' Common Core of Data (CCD) public school universe file. Almost 16,000 comprehensive secondary schools and 1,300 vocational schools met the eligibility requirement for this study; that is, they had 11th or 12th grade. Excluded from the sampling frame were private schools (the CCD does not include private schools), nonregular schools such as special education and alternative/other schools, and schools in the outlying U.S. territories.

For the PEQIS survey, the sample of postsecondary institutions was restricted to 2-year and less-than-2-year institutions that were eligible for federal financial aid under Title IV. A stratified random sample of 1,289 institutions was selected, including 689 2-year institutions and 600 less-than- 2-year institutions. The sample of 2-year postsecondary institutions was drawn from the Postsecondary Education Quick Information System (PEQIS) panel, which was constructed from the 1995-96 Integrated Postsecondary Education Data System (IPEDS) Institutional Characteristics (IC) file. The PEQIS frame included 4-year, 2-year, and less-than-2-year institutions of higher education located in the 50 states and the District of Columbia: a total of 5,353 institutions. Only 2-year institutions that were eligible for federal financial aid were included for selection from the PEQIS panel.

The sampling frame for the supplementary sample of less-than-2-year institutions was the 1996-97 Integrated Postsecondary Education Data System (IPEDS) Institutional Characteristics file. The institutions eligible for the supplementary sample were all less-than-2-year institutions in the 50 states and the District of Columbia (the same geographic area used for the PEQIS panel) that reported eligibility for Title IV. A total of 1,898 institutions met these requirements.

Respondents and Response Rates

For the FRSS survey, questionnaires with letters explaining the purpose of the study were mailed to school principals in early April 1999. The questionnaires were to be completed by the person who was most knowledgeable about vocational education at the school. Telephone followup of nonrespondents was conducted during May and June 1999. Of the 1,200 schools selected for the survey, 50 were found to be out of scope for the study (29 of these were postsecondary institutions). A total of 1,078 eligible schools completed the survey for an overall unweighted response rate of 94 percent. The weighted response rate was 95 percent. The postsecondary questionnaires were mailed in mid-April 1999 to PEQIS coordinators at 2-year institutions and administrators at less-than-2-year institutions. As with the FRSS survey, the questionnaire was to be completed by the person most knowledgeable about occupational programs at the institution. Telephone followup of nonrespondents started in late May, and data collection ended in early July 1999. Of the 1,289 postsecondary institutions sampled for the study, 103 were out of scope for the study; 57 of these institutions were closed, and 38 did not have Title IV eligibility. The survey was completed by 1,100 2-year and less-than-2-postsecondary institutions, yielding an overall unweighted response rate of 94 percent. The weighted response rate was also 94 percent.

Sampling and Nonsampling Errors

The responses were weighted to produce national estimates. The weights were designed to adjust for the variable probabilities of selection and differential nonresponse. The findings in this report are based on the sample selected and, consequently, are subject to sampling variability. The survey estimates are also subject to nonsampling errors that can arise because of nonobservation (nonresponse and noncoverage) errors, errors of reporting, and errors made in data collection. These errors can sometimes bias the data. Nonsampling errors may include such problems as misrecording of responses; incorrect editing, coding, and data entry; differences related to the particular time the survey was conducted; or errors in data preparation. While general sampling theory can be used in part to determine how to estimate the sampling variability of a statistic, nonsampling errors are not easy to measure and, for measurement purposes, usually require that an experiment be conducted as part of the data collection procedures or that data external to the study be used.

A number of actions were taken to minimize nonsampling error. The questionnaire was pretested with respondents like those who completed the survey. During the design of the survey and survey pretest, an effort was made to check for consistency of interpretation of questions and to eliminate ambiguous items. The questionnaire and instructions were extensively reviewed by the National Center for Education Statistics and the Office of Vocational and Adult Education, U.S. Department of Education. Manual and machine editing of the questionnaire responses were conducted to check the data for accuracy and consistency. Cases with missing or inconsistent items were recontacted by telephone. Data were keyed with 100 percent verification.

Variances

The standard error is a measure of the variability of estimates due to sampling. It indicates the variability of a sample estimate that would be obtained from all possible samples of a given design and size. Standard errors are used as a measure of the precision expected from a particular sample. If all possible samples were surveyed under similar conditions, intervals of 1.96 standard errors below to 1.96 standard errors above a particular statistic would include the true population parameter being estimated in about 95 percent of the samples. This is a 95 percent confidence interval. Estimates of standard errors for this report were computed using the jacknife replication method.

Terms and Variables

For the secondary school survey, a vocational program was defined as a sequence of courses designed to prepare students for an occupation (e.g., nurses' aide) or occupation area (e.g., health care) that typically requires education below the baccalaureate level. Because the focus of the surveys is on preparation for jobs within specific occupations, the definition of vocational programs did not include career exploration or other introductory courses that prepare students for adult life or for work in general (e.g., consumer and homemaking, industrial arts). At the postsecondary level, an occupational program was defined as a sequence of courses designed to prepare students for an occupation (e.g., nurses' aide) that typically requires education below the baccalaureate level. To allow institutions to report noncredit courses, a noncredit occupational program could have included only one course or more than one course. For both surveys, a skill competency was defined as a concept, skill, or attitude that is essential to an occupation; the level of attainment or performance established for a skill competency is a skill standard. Because these terms tend to be used interchangeably in practice, "skill competencies" was used to refer to both skill competencies and skill standards. The main classification variable was school type (vocational, comprehensive) for the secondary school survey, and level of institution (2-year, less-than-2-year) for the postsecondary survey. For the tabular analyses, several variables were constructed to measure the number of programs offered, skill competencies used for the programs offered, and skill certificates or industry-related credentials available for programs offered. The variables were constructed for all programs offered and for programs offered within the six broad occupation areas examined in the study.

Background Information

The secondary survey was conducted under contract with Westat, using the Fast response Survey System (FRSS), and the postsecondary survey was conducted using the Postsecondary Education Quick Information System (PEQIS). Westat's Project Director was Elizabeth Farris, and the Survey Manager was Basmat Parsad. Bernie Greene was the NCES Project Officer. The data were requested by the Office of Vocational and Adult Education, U.S. Department of Education.

The following individuals reviewed this report:

Outside NCES

  • Vickie Schray, Office of Vocational and Adult Education

  • Doris Werwie, Office of Vocational and Adult Education

  • Cynthia Brown, Office of Vocational and Adult Education

  • Sharon Belli, Office of Vocational and Adult Education

Inside NCES

  • Lisa Hudson, Early Childhood, International, and Crosscutting Studies Division
  • Ellen Bradburn, Early Childhood, International, and Crosscutting Studies Division

  • Michael Cohen, Statistical Standards Program

  • John Wirt, Early Childhood, International, and Crosscutting Studies Division

  • Jeff Williams, Postsecondary Studies Division

For more information about the surveys, Vocational Programs in Secondary Schools and Occupational Programs in Postsecondary Education Institutions, contact:

Bernie Greene
Early Childhood, International, and Crosscutting Studies Division
National Center for Education Statistics
Office of Educational Research and Improvement
U.S. Department of Education E-mail: Bernard_Greene@ed.gov

Top