Create the Assessment

Once the item pool of cognitive and contextual questionnaires are created, they are then reviewed by

• content experts and teachers for political sensitivity and bias, and
• standing committees who assess the appropriateness, representativeness, and quality of items.

For contextual questionnaires, NCES submits them to the Office of Management and Budget (OMB) for clearance. OMB checks to make sure the questions comply with government policies. When developing the questionnaires, NAEP ensures that the questions are grounded in educational research and that the answers provide information relevant to the subject being assessed. The questionnaires are also designed to minimize the burden on respondents while meeting the needs of the NAEP program.

The cognitive items are then placed in blocks, or groups of items created by dividing the item pool for an age or grade into subsets. Using the BIB spiral sample design, item blocks are then placed in a booklet, along with one-block of a student survey questionnaire. Booklets are loaded to tablets using the following process:

1. The test developer sends assessment content (including the item blocks and appropriate block order) to the technology team.
2. The technology team loads the image of the entire assessment (system, foundation, and content) into a software program, and a master image is created.
3. Finally, the image is loaded onto imaging USB drives for cloning onto thousands of tablets in preparation for the digital assessment.

In general, students who are sampled to answer two 30-minute cognitive blocks will also receive 15 minutes of survey questions (10 minutes related to the subject assessed and 5 minutes of general information questions). Starting in 2021, students who are sampled to receive a form with three 30-minute cognitive item blocks will be asked to answer one five-minute general student survey questionnaire along with questionnaires about the subjects assessed and computer access and familiarity.

Items are then ready for pilot testing. The purpose of national-level pilot testing is to gain information about

• item clarity,
• item difficulty,
• timing,
• feasibility, and
• administration logistical considerations.

Items are revised based on student data, and the item set is once again subjected to review following the same procedure described above. Then, a final set of test items is assembled for NCES and the Governing Board review and approval. Once NCES and the Governing Board have approved, the assessment is deemed operational.