Direct Cognitive Assessments

Overview

The ECLS-K:2011 assessed children's cognitive, socioemotional, and physical development through direct and indirect methods. The direct child assessments are cognitive assessment batteries and socioemotional items developed or adapted for use in the ECLS-K:2011 and administered directly to children. Physical development was also measured directly. The indirect child assessments are parent and teacher ratings of children's cognitive and socioemotional development. Summary information for the child assessments and other measures is provided below.

Direct Cognitive Assessments

The direct cognitive assessments were designed to measure children's knowledge and skills at given time points, as well as track their academic growth in different subject areas across time. Results from the assessments for reading, mathematics, and executive function enable researchers to measure growth from the fall of children's kindergarten year (fall 2010) through the spring of 2016, when most ECLS-K:2011 students were in the fifth grade. Science knowledge and skills can be examined beginning in the spring of the children's kindergarten year.

The ECLS-K:2011 assessed knowledge and skills that are typically taught and developmentally important. The assessment frameworks were derived from national and state standards, including selected states’ curriculum standards, as well as the assessment frameworks for the National Assessment of Educational Progress (NAEP) and the ECLS-K. The ECLS-K:2011 assessments include items that were specifically created for the ECLS studies, items adapted from commercial assessments with copyright permission, and items from other NCES studies.

In the kindergarten and first-grade rounds, the components of the ECLS-K:2011 assessment administered to children who spoke a language other than English at home depended on the children's performance on a language screener, which was administered as the first component of the assessment. The screener consisted of two tasks from the Preschool Language Assessment Scale (preLAS 2000). The "Simon Says" task required children to follow simple, direct instructions given by the assessor in English. The "Art Show" task was a picture vocabulary assessment that tested children's expressive vocabulary. All children, regardless of home language, were administered the language screener as the first component of the direct cognitive assessment. For children whose home language was English, the screener primarily served as a warm-up or practice for the rest of the assessment since the items were of low difficulty. While the screener also served as a warm-up for children whose home language was one other than English, it also determined whether the children understood English well enough to receive the full direct child assessment in English. All children received the first set of items on the reading assessment in English, regardless of their home language or performance on the language screener. Once this first set of items was administered, the cognitive assessments in English ended for children whose home language was not English and who did not achieve at least a minimum score on the language screener. Spanish-speaking children who did not achieve at least the minimum score on the screener were then administered a short reading assessment in Spanish that measured Spanish early reading skills (SERS), as well as the mathematics and executive function assessments that had been translated into Spanish. Children whose home language was one other than English or Spanish and who did not achieve at least the minimum score on the screener were not administered any of the remaining cognitive assessments beyond the first set of reading items. All children had their height and weight measured. Beginning in second grade, all study children were administered the entire assessment in English, regardless of home language.

The ECLS-K:2011 direct cognitive assessments in reading, mathematics, and science were individually administered, two-stage adaptive tests, with the exception of the spring kindergarten science assessment, which was a relatively short assessment comprising one set of items administered to all children. Assessors asked the children questions related to images (such as pictures, letters of the alphabet, words, or short sentences for reading or numbers and number problems for mathematics), which were presented on a small easel. Children responded by pointing or telling the assessor their answers. They were not required to write their answers or explain their reasoning.

For each two-stage assessment, the first stage was a routing section that included items covering a broad range of difficulty. A child's performance on the routing section determined which one of three second-stage tests (low, middle, or high difficulty) the child was administered. The second-stage tests vary by level of difficulty so that a child was administered questions appropriate to his or her demonstrated level of ability for each of the cognitive domains. The purpose of this adaptive assessment design is to maximize accuracy of measurement and minimize both administration time and the potential for floor and ceiling effects.

The second-stage forms include some items that overlapped (e.g., some items in the low-level form were are included in the middle-level form). The common routing test and the item overlap between second-stage forms help to ensure that there was a sufficient number of items to precisely measure the child's skills.

Reading

The reading assessment included questions measuring basic skills such as print familiarity, letter recognition, beginning and ending sounds, recognition of common words (sight vocabulary), and decoding multisyllabic words; vocabulary knowledge such as receptive vocabulary and vocabulary in context; and reading comprehension. Reading comprehension questions asked the child to identify information specifically stated in text (e.g., definitions, facts, supporting details), make inferences within texts, and consider the text objectively and judge its appropriateness and quality. The selected reading passages represented a variety of literary genres, such as poetry, letters, fiction, and nonfiction, with the content, length, and language complexity appropriate for the grade level being assessed. Assessments in the earlier grades began with relatively more emphasis on basic reading skills, while greater emphasis was placed on comprehension in the assessments for the later grades. As noted above, in the kindergarten and first-grade rounds, all children received the first set of items on the reading assessment in English, regardless of their home language or performance on the language screener. These items, plus two items from the preLAS "Art Show" task (a total of 20 items), make up the section of the reading assessment referred to as the English basic reading skills (EBRS) section because they measure such skills.

Mathematics

The mathematics assessment included questions measuring conceptual knowledge, procedural knowledge, and problem solving. The assessment included questions on number sense, properties, and operations; measurement; geometry and spatial sense; data analysis, statistics, and probability; and patterns, algebra, and functions.

Science

The science assessment included questions about physical sciences, life sciences, environmental sciences, and scientific inquiry. The spring kindergarten science assessment was composed of 20 items that all children received; a two-stage assessment was not used for this domain until the fall first-grade round.

Executive Function

The executive function component of the cognitive assessment obtained information on important cognitive processes associated with learning. Cognitive flexibility and working memory were assessed in each round of data collection, and a measure of inhibitory control was included in the fourth- and fifth-grade rounds. To assess cognitive flexibility, children were administered a card sort task, which was administered as a physical, tabletop card sort in the kindergarten and first-grade rounds and as a computer-administered card sort in the second-grade and later rounds. To assess working memory, children responded to a backward digit span task. For the computer-administered inhibitory control task, children were asked to focus attention on a central stimulus on the computer screen while ignoring or inhibiting attention to stimuli presented on either side of the central stimulus.

Direct Physical Measures

Children’s height and weight were measured at each round of data collection. Additionally, in the fall of second grade, spring of third grade, and spring of fifth grade, a subsample of children had their hearing evaluated.

Indirect Cognitive Assessments

The Academic Rating Scale was developed for the ECLS-K to obtain teachers' evaluations of children's academic achievement in three domains: language and literacy, science, and mathematical thinking. The Academic Rating Scale developed for the ECLS-K was included in the kindergarten and first-grade rounds of the ECLS-K:2011, with some modifications to the item text. The Academic Rating Scale was designed both to overlap with and to augment the information gathered through the direct cognitive assessment battery. Although the direct and indirect instruments measured children's skills and behaviors within the same broad curricular domains with some intended overlap, several of the constructs were designed to differ in significant ways. Most importantly, the Academic Rating Scale included items that were designed to measure both the process and products of children's learning in school, whereas the direct cognitive battery was more limited. Because of time and space limitations, the direct cognitive assessment battery was less able to measure the process of children's thinking, including the strategies they use to read, solve mathematical problems, or investigate a scientific phenomenon.

Socioemotional Measures

The ECLS-K:2011 socioemotional development assessments focused on aspects of social competence, including social skills (e.g., social interaction, attentional focus, and self-control) and problem behaviors (e.g., impulsivity and externalizing problem behaviors). Both the child-level teacher questionnaires and parent interviews included the Social Rating Scale (SRS), originally developed for the ECLS-K, which included Approaches to Learning items that asked teachers and parents to report how often the ECLS-K:2011 children exhibited a selected set of learning behaviors. For more information and to view the Approaches to Learning items, please see the Approaches to Learning Items tip sheet. Teachers also reported on closeness and conflict between them and each of the study children. Parents and teachers were the primary sources of information on children’s social competence and skills in kindergarten, first grade, and second grade.

In the third, fourth, and fifth grades, ECLS-K:2011 students independently completed a self-administered questionnaire. Topics on the child questionnaire included interest and perceived competence in reading, math, and science; relationships with peers; social distress; occurrences of peer victimization; and life satisfaction and prosocial behavior.

Top