Dr. Gary W. Phillips
Deputy Commissioner of the National Center for Education Statistics
The Release of the National Assessment of Educational Progress (NAEP) The Nation's Report Card: U.S. History 2001
May 9, 2002
In 2001, NCES conducted the National Assessment of Educational Progress in U.S. history among fourth-, eighth-, and twelfth-grade students. Results for the 2001 assessment are available for the nation only, with no state data. Students in both public and nonpublic schools participated in the assessment.
The NAEP 2001 U.S. history assessment was based on a framework developed by the National Assessment Governing Board (NAGB), which establishes policy for NAEP. Previously, an assessment in U.S. history was conducted in 1994 using the same framework. Therefore, the results for the 2001 assessment can be compared to the 1994 assessment results.
The NAEP 2001 U.S. history assessment provides results for subgroups of students defined by various background and contextual characteristics. This assessment also provides results for a second sample in which testing accommodations were provided to students with special learning needs (students with disabilities and/or limited English proficient students).
The number of students assessed nationally in 2001 totaled about 29,000 students. The fourth-grade sample included about 7,000 students in 365 schools, while the eighth- and twelfth-grade samples were approximately 11,000 students each in 369 and 374 schools, respectively.
A Cautionary Note on Interpreting the Results
Please note that because NAEP scores are based on samples, there is a margin of error associated with each score. When comparisons are made between scores, or between the percentages of students at the various achievement levels, the differences must be tested to see if they are larger than the margin of error involved-these differences are referred to as "statistically significant." NCES only discusses statistically significant differences. Furthermore, NAEP is not designed to indicate the reasons for these differences. NAEP is like a thermometer. It tells what the temperature is, but doesn't tell why the temperature changes.
The NAEP U.S. History Framework
The content of the NAEP 2001 U.S. history assessment is based upon the NAEP U.S. history framework, which is organized around three concepts or dimensions. The three concepts are major themes of U.S. History, chronological periods, and ways of knowing and thinking about U.S. history. The four themes provide the core organizing structure of the framework. The four historical themes for the assessment are:
The issues included in the four themes are chronologically structured in the assessment according to eight different historical periods:
The framework also considers ways of thinking and the kinds of knowledge that historical study requires. These include "Historical Knowledge and Perspective" and "Historical Analysis and Interpretation."
Students answered both multiple-choice and constructed-response questions that corresponded to the historical themes and periods. Students devoted more than 50 percent of their time to "constructed-response" questions that required at least a brief written response and sometimes several paragraphs to explain their answers.
Results Reported from the 2001 U.S. History Assessment
Student performance is reported in two ways: 1) as average scale scores, and 2) as percents of students performing at or above various achievement levels. An average scale score reflects the overall U.S. history performance of a particular group or subgroup of students. The achievement levels are intended to describe standards for what students should know and be able to do. There are three U.S. history achievement levels established by the National Assessment Governing Board. The three achievement levels are defined as Basic, Proficient, and Advanced. NAGB believes that all students should be performing at or above the Proficient level.
National Scale Score Results Across Years
The NAEP U.S. History assessment uses a single 500-point composite scale. All scores, regardless of grade, are on a scale that ranges from 0 to 500. In 2001, the average score for fourth- and eighth-grade students was higher than in 1994, while scores for twelfth-grade students did not show a change. Fourth-grade scores rose from 205 in 1994 to 209 in 2001, while eighth-grade scores rose from 259 to 262.
National Scale Score Percentiles Across Years
In addition to presenting average scores, the results are presented for students at various levels of performance, or percentiles. The report indicates performance trends for students scoring at the 10th, 25th, 50th, 75th, and 90th percentiles in each grade. The improvement in performance at the fourth-grade level was seen among the students scoring in the lower percentiles. Fourth-graders at or near the 10th and 25th percentiles showed higher scores, increasing from 147 to 158 and from 180 to 186, respectively. Scores for students at or near the 50th, 75th, and 90th did not change. At the eighth-grade, students at or near the 25th, 75th, and 90th percentiles showed a change. Specifically, scores increased from 239 to 241 (25th percentile), from 282 to 285 (75th percentile), and from 299 to 303 (90th percentile). Scores for the twelfth-grade students showed no change at any of the percentiles.
National Achievement Level Results Across Years
At the fourth grade, the percentage of students who scored below Basic fell from 36 percent in 1994 to 33 percent in 2001, while the percentage at or above Basic, which includes the Proficient and Advanced levels, rose from 64 percent to 67 percent. However, neither the percentage of students at or above Proficient (18 percent) nor at Advanced (2 percent) showed an increase.
At grade 8, there were increases in the percentage of students who were at or above Basic (rising from 61 percent to 64 percent), at or above Proficient (rising from 14 percent to 17 percent), and at Advanced (from 1 percent to 2 percent). There was also a corresponding decline in the percentage of students who were below Basic, from 39 percent to 36 percent. At grade 12, 43 percent of students were at or above Basic, 11 percent were at or above Proficient, and 1 percent was Advanced. The percentages of twelfth-grade students attaining the Basic, Proficient, and Advanced achievement levels showed no changes from 1994 to 2001.
National Scale Scores Across Years by Gender
At the fourth grade, both male and female students showed an increase in average scale scores. Scores for male fourth-graders rose from 203 to 209, while scores for females rose from 206 to 209. At the eighth grade, male scores increased, from 259 in 1994 to 264 in 2001, while scores for females did not show a change (259 and 261, respectively). At grade 12, the average score for males was 288 in 1994 and 2001, and the average scores for females were 285 and 286, respectively. Average performance patterns for male and female students did not show a change at the twelfth-grade. For all three grades, there was no significant difference in scores for males and females in 2001.
National Achievement Level Results Across Years by Gender
At the fourth and twelfth grades, neither males nor females showed any changes in the percentage students that attained any of the achievement levels. At the eighth grade, the percentage of males who were at or above Basic increased from 61 percent to 65 percent, while those at or above Proficient rose from 15 percent to 18 percent. The percentage of male students at Advanced increased from 1 percent to 2 percent. Percentages for female eighth-graders did not show any increases at any of the achievement levels.
Average Scale Scores Across Years by Race/Ethnicity
Across the assessment grades, there were some changes in average performance patterns for some of the racial/ethnic subgroups. Among fourth-graders, White and Black students showed increases in U.S. history scale scores in 2001, rising from 215 to 220 and from 177 to 188, respectively. Scores for other racial/ethnic subgroups did not show a change in 2001. Among eighth-graders, only White students showed an increase, as average scale scores rose from 267 in 1994 to 271 in 2001. For the twelfth-grade students, only Hispanic students showed improvement, rising from 267 to 274.
National Achievement Level Results Across Years by Race/Ethnicity
At the fourth grade, both White and Black students showed increases in the percentages of students at or above Basic, from 74 percent in 1994 to 79 percent in 2001 and from 36 percent in 1994 to 44 percent in 2001, respectively. There were no significant changes in the percentages of White and Black students that attained the Proficient and Advanced achievement levels.
At the eighth grade, only the percentages of White students changed, increasing their percentages at or above the Proficient level (from 17 percent to 21 percent) and at Advanced (1 percent to 2 percent). At the twelfth grade, the percentages of students attaining the three achievement levels do not show change for any racial/ethnic subgroup.
Racial/Ethnic Gaps in Average Scale Scores Across Years
Comparisons of the differences in average scores for White and Black students and White and Hispanic students showed two changes from 1994 to 2001. At the fourth-grade, the average score gap between White and Black students decreased from 38 points to 31 points. The average score gap between twelfth-grade White and Hispanic students fell from 26 points to 19. All other differences in average scores did not show a change.
One of the goals of the "No Child Left Behind Act of 2001" is to close the racial/ethnic performance gap. The decrease seen in the racial/ethnic performance gap at the fourth grade is a good example of this type of improvement. NCES has developed a new graphic to display and explain the achievement gaps information. The graph shows the difference between the percentage of students from two different racial/ethnic backgrounds that scored at or above each achievement level for 1994 and 2001. For example, in 1994, 74 percent of White students and 36 percent of Black students scored at or above the Basic level. This is a 39 point difference (when rounded). The gap fell to 35 points in 2001 (79 percent of White students and 44 percent of Black students). The performance gap can be seen by the space between the two lines of the graph. Therefore, the graph shows that the gap is mostly among students at the Basic achievement level and below.
Average Scale Scores by Parents' Highest Level of Education
NAEP provides information on student performance according to parents' highest level of education for grades 8 and 12 only. Students are asked to report the highest level of education attained by either parent. Fourth-graders are not asked this question due to the poor reliability of the responses.
At the eighth grade, students who reported having a parent who had graduated from college showed an increase in average score (from 270 in 1994 to 275 in 2001), as did those who reported that they did not know their parents' level of education (238 to 244). However, the average score of students that did not know their parents' level of education was among the lowest.
At the twelfth grade, students who reported that neither of their parents finished high school showed an increase in average score, from 263 in 1994 to 269 in 2001. For both grades, the higher the parental education level reported, the higher the average score attained.
Average Scale Scores by Type of School (Public and Nonpublic)
NAEP collects data for students in both public and nonpublic schools. Results are presented for public and nonpublic schools, where nonpublic schools are divided into the categories of Catholic and "Other" nonpublic schools. In 2001, average scale scores increased at the fourth-grade for public schools and for Catholic schools, rising from 203 to 207 and 221 to 229, respectively. However, the average score for all nonpublic schools did not show an increase, nor did the score for "Other" nonpublic schools.
At the eighth-grade, scores increased for public schools only, rising from 257 in 1994 to 260 in 2001. Scores for nonpublic students (279), Catholic students (280), and "Other" nonpublic students (278) did not show a change.
At the twelfth-grade level, there were no increases for public students (286), nonpublic students (298), Other nonpublic students (293), or Catholic students (302). At all three grades, students in nonpublic schools showed higher average performance than those in public schools, a pattern that is visible in achievement level data as well.
Background Factors
In addition to assessing performance in U.S. history, NAEP also collected data from students and teachers about classroom and instructional activities that may be related to performance. Some activities that showed a relationship to student performance are discussed below.
Time spent on Social Studies: Grade 4
The NAEP data indicate that at the fourth-grade, about one-fifth of the students had teachers who reported spending more than 180 minutes on social studies in a typical week. These students attained a higher average score than students whose teachers reported spending less time on social studies.
State and Local Standards: Grades 4 & 8
In 2001, teachers of fourth- and eighth-grade students were asked about the extent to which they used state or local standards in planning their history or social studies instruction. More than half of the students had teachers who reported that standards were used to a large extent in planning instruction (63 percent at grade 4, and 69 percent at grade 8). Only a small percentage of students had teachers who reported there were no state or local standards that applied to teaching social studies (3 percent at grade 4, and 1 percent at grade 8). However, there were no statistically significant differences in students' performance at either grade.
Frequency of Classroom Activities: Grades 4, 8 & 12
Most fourth-graders had teachers who reported that they required their students to read from a textbook on daily or weekly basis. Students whose teachers required their students to read on a daily basis had a higher average score in 2001 than those who required their students to read on a weekly or a monthly basis.
Eighth-graders whose teachers reported requiring their students to use primary historical documents on a weekly basis had a higher average score than those whose teachers said their students used primary historical documents on a monthly basis or less. About a quarter of the students used these documents on a weekly basis.
Twelfth-graders who reported that they never read extra materials in addition to their textbook had a lower average score than those who did read extra material, even if they only did so a few times a year.
Use of Technology: Grades 4, 8 & 12
At grade four, students who said they used computers at school on a daily basis for social studies had a lower average score than those who reported less frequent general use. In addition, the percentage of fourth-graders reporting such frequent general use was low (3 percent). However, students who reported using computer programs in which they pretend to be an explorer, travel to new places, or do things like build cities, obtained a higher average score (212) than students who did not use such computer programs (207).
The same pattern was evident at grade 8¾1 percent of students reported using a computer every day, and these students had an average score (239) that was lower than that of other students. However, students who reported doing research projects using a compact disc or the Internet to a "large extent" had a higher average score than those who did so to a smaller extent.
Outcomes were similar at grade twelve. A small percentage of students reported general use of a computer at school every day (2 percent), and they had an average score (265) that was lower than students with less frequent general use of computer. Students who reported using computers to a "large extent" to write reports had a higher average score (300) than those who report less frequent or no use at all.
Days absent from school: Grades 4, 8 & 12
Across all grades, students who reported being absent from school two or less days in the last month obtained higher average scores than students who reported being absent 3 or more days. At grade 4, students who reported never being absent (213) or being absent one to two days (210) in the last month had higher average scores than students who reported being absent three to four days (201), five to ten days (198), or more than 10 days (176). At grade 8, students who reported never being absent (267) or being absent one to two days (265) in the last month had higher average scores than students who reported being absent three to four days (257), five to ten days (255), or more than 10 days (237). For twelfth-grade students, those who reported never being absent (292) or being absent one to two days (289) in the last month had higher average scores than students who reported being absent three to four days (280), five to ten days (280), or more than 10 days (264).
Accommodations Permitted in the 2001 U.S. History Assessment
In recent years, NAEP has been investigating the impact of permitting testing accommodations for students with disabilities and limited English proficient students. These students are unable to fully participate in the NAEP assessment without accommodations. NAEP only allows accommodations that do not affect the integrity of the assessment. For example, NAEP allowed some students to use a bilingual dictionary for the 2001 U.S. history assessment, but not for the 2000 reading assessment, because NAEP took the position that the reading assessment is intended to assess students' ability to read English. In a few instances, including accommodated students has led to lower average scores. However, NAEP has concluded that providing accommodations has only a minor impact on statistical results overall.
NAEP used two samples in the 2001 U.S. history assessment. One sample was assessed with accommodations permitted for students who needed them to participate and one sample was assessed without accommodations. Average scores for the non-accommodated sample were used to allow comparisons with the 1994 assessment, which did not allow accommodations. There were no significant differences in the average scores between the two sets of results at grades 4 and 12. At grade 8, however, the average score when accommodations were permitted was lower than the average score when accommodations were not permitted. Scores from the two samples appear in the main body of the NAEP 2001 U.S. History Report Card.
Complete results for the Nation's Report Card: U.S. History 2001 are available on the web at http://nces.ed.gov/nationsreportcard/ushistory/results/.
