The Use of a Benchmark Assessment to Track Progress of a Reading Intervention:
A Pilot Study with an Urban Elementary School  

Tiffany A. Jenkins, M.A., Antoinette Halsell Miranda, Ph.D., and Valerie E. Cook, M.Ed.  

Abstract:  The use of a benchmark assessment to track the general progression of reading acquisition skills for kindergarten students in an urban elementary school was investigated. The benchmark assessment used was the Dynamic Indicators of Basic Early Literacy Skills, which has been shown to be highly predictive of early literacy (Good, Gruba, & Kaminski, 2002). The results of this study indicated that early literacy development in kindergarten is consistent with the research literature demonstrating that African American students are placed at-risk for reading failure. Less than 50% of the students were on track to becoming established readers. The need for early identification of children at-risk for reading difficulty as well as suggestions for overcoming barriers implementing prevention reading programs in urban schools are discussed.  

Introduction  

The federally-mandated and reauthorized Elementary and Secondary Education Act, also known as the No Child Left Behind Act of 2001 (NCLB), is having a considerable impact on education, particularly in urban areas. Heavy emphasis is being placed on reading and closing the achievement gap between ethnic minority students and their White counterparts. As a result, effective interventions are seen as key to making this happen as well as providing on-going assessment to determine if children are making adequate yearly progress and to make data-based decisions regarding a student’s education (McGee & Richgels, 2003). Using appropriate instruments to assess the needs of children is an educational mandate. Moreover, implementing teaching strategies in urban districts to prevent literacy failure is equally critical to the success of many ethnic minority students, especially African American students.  

In 1998, the National Reading Council produced a comprehensive report on the nation’s reading achievement status and found that children of color and those raised in poverty were continually documented as being at risk for academic failure. It is estimated that approximately one-third of the children in the United States are at-risk of academic failure by the time they enter kindergarten due to socially and economically impoverished conditions (National Assessment of Educational Progress, 2000). Unfortunately, African American students are far more likely to be raised in poverty than White children (McGee & Richgels, 2003).  

While 40% of the White fourth graders scored at or above proficiency in reading, only 12% of their African American counterparts reached the same level (National Assessment of Educational Progress, 2000). Learning to read well is believed to be the principal goal of first grade instruction (Farkas, 2003). Children from low socioeconomic environments, especially African Americans, consistently perform below their peers in reading, mathematics, and science, which is in part a result of inadequate attainment of literacy skills (Farkas, 2003).  Unfortunately, the achievement gap continues and widens as these students proceed through formal schooling (Ferguson, 2003).  

Children who do not receive proper early reading intervention often face harsh consequences. In 1988, Juel reported that 88% of the children who scored in the lowest quartile for reading comprehension at the end of first grade continued to score below the 50th percentile at the end of fourth grade. These findings also strongly indicated that many children do not receive instruction or effective interventions to successfully facilitate remediation for difficulties noted at the end of the first grade year. In 1986, Stanovich presented the “Matthew Effect” phenomenon – the idea that the rich get richer and the poor get poorer. This notion is found to be an on-going occurrence in urban school systems. Stanovich also found that students who develop early literacy skills acquire tools needed to grow exponentially in their knowledge and skill over time. Conversely, students who fail to develop early literacy skills fall further behind with the progression of time. In addition, this downward spiraling of reading achievement has been proposed as a major determinant of school failure (Davis, 2003; Farkas, 2003; Ferguson, 2003). When children begin to transition from “learning to read” to “reading to learn,” those who have difficulty reading have a harder time exploring and comprehending science, literature, history, mathematics and other core academic subjects. Additionally, poor skills in reading are associated with poverty, unemployment, and dropping out of high school, which are all predictive of a bleak future (Chandler, 2000; McGee & Richgels, 2003). Fourteen percent of the dropouts, from 16- to19-year-olds reside in urban neighborhoods. Unfortunately, this is twice the rate for teens in suburban settings (7%). In high poverty neighborhoods located within large cities (e.g., poverty rates above 20%), one-fifth of 16- to 19-year-olds drop out of school (Annie E. Casey Foundation, 2000).  

Class size and teacher quality have also been identified as a risk factor for reading difficulties. The U.S. Department of Education (2000) found that teachers in schools with a high ethnic minority enrollment and/or high percentage of students eligible for free or reduced-price lunch were less likely to have master’s degrees than teachers in schools with low ethnic minority enrollment and/or a low percentage of students eligible for free or reduced-price lunch. Since most urban school systems have high ethnic minority enrollment and high percentages of students receiving free or reduced lunch, it is plausible to conclude, on average, that the percentage of teachers in suburban districts have received more training than teachers in urban districts. However, this statement is not meant to imply that urban schools are at an automatic disadvantage, but simply to point out that teacher education and training varies greatly, and therefore, an extra effort must be exerted by less prepared teachers to ensure their students receive the best instruction possible. In addition to teacher quality, educational researchers have also examined the influence of teachers’ perceptions and expectations on the academic achievement of African Americans (Farkas, 2003; Ferguson, 1998; Flowers, Milner, & Moore, 2003). It has generally been concluded that negative teacher expectations and perceptions play some part in the lower academic achievement of African Americans.  

In addition to the variation in teacher training, it was also found that class size varies between schools with high versus low ethnic minority enrollment (Pritchard, 1999). Kindergarten classrooms with less than 10% ethnic minority children were more likely to have 15 or fewer children than classrooms where 75% or greater of the children were minorities (Pritchard, 1999). If class size is reduced to below 20 students, research indicates that disadvantaged children of color have the greatest increase in student achievement (Pritchard, 1999).  

There has been a substantial amount of research conducted on phonemic awareness instruction of students enrolled in pre-school and kindergarten programs. Lundberg, Frost, and Peterson (1988) found that direct instruction of phonemic awareness in preschool had a positive effect on reading and spelling acquisition. In 1995, Kozminsky and Kozminsky found that explicit instruction of phonological awareness in kindergarten resulted in statistically significant gains for students in reading versus students who did not receive explicit instruction. Furthermore, they found that the explicit instruction positively influenced students’ scores on measures of reading comprehension in the first grade. In 2000, the National Reading Panel conducted a review of research and found that many children required direct instruction of phonemic awareness. Therefore, Good et al. (in press) recommended that these skills be taught in kindergarten and first grade, when children first begin to acquire reading skills. This is particularly important for African American children who, on average, enter school with lower-levels of school readiness, oral language, and prereading skills than do White and Asian children (Farkas, 2003).  

Before assessing early literacy skills, it is important to understand the underlying concepts a student must possess in order to succeed. According to Simmons and Kame’enui (1998), there are five main concepts that must be considered: (a) phonemic awareness, (b) alphabetic principle, (c) fluency with text, (d) vocabulary, and (e) comprehension. In order for a reading program or intervention to be effective, it is suggested that all five components are included within the structure of the program. The use of an assessment that includes these five components can inform instruction and provide guidance for an effective beginning reading program.  

Dynamic Indicators of Basic Early Literacy Skills (DIBELS)  

DIBELS is a set of tasks designed to assess fundamental reading skills through the use of seven assessment measures: (a) Letter-naming Fluency, (b) Initial Sound Fluency, (c) Phoneme Segmentation Fluency, (d) Nonsense Word Fluency, (e) Oral Reading Fluency, (f) Story Retell (optional), and (g) Word Use Fluency (optional). These measures are specifically designed to assess three of the five main concepts of early literacy: Phonemic Awareness, Alphabetic Principle, and Fluency with Connected Text. The measures are linked to one another, both psychometrically and theoretically, and have strong psychometric properties for predicting later reading proficiency (Good & Kaminski, 2001).     

DIBELS enables educators to quickly and efficiently, with a high degree of confidence, identify students who require additional reading intervention. The general assessment is completed three times per year, preferably once during the academic fall, winter, and spring terms. Once the student has completed the assessment, data can be easily analyzed to determine individual performance. By utilizing the data, it is possible to identify students who are at-risk for reading failure. If a student is identified as being at-risk for reading difficulties, it is advised that interventions be provided to the student in order to assist in achieving proficient literacy in problematic areas (McGee & Richgels, 2003).  

Good and Kaminski (2002) indicated several advantages of using DIBELS, including that the measures are: (a) reliable and valid measures of early literacy skills; (b) simple and efficient to administer; (c) inexpensive to produce in terms of time and resources; (d) sensitive to improvement in student’s skills over time; (e) available in multiple forms of short duration to facilitate frequent administration by educators; and (f) based on production-type responses, so that students’ skills can be observed. Over the years, DIBELS has been implemented in numerous school districts, including large urban districts and has been shown to be a successful assessment tool for identifying children at-risk for being established readers (Burns, Lateer-Huhn, Poth, & Stollar, 2004). Thus, prevention is considered to be the key in helping children obtain proficient literacy skills. As educators and educational scholars, teachers and researchers strive to prevent reading failure in order to combat reading difficulties for students who do not possess these skills.  

Method  

A pilot study was initiated with a large urban elementary school in which the DIBELS was administered three times during the school year to kindergarten, first, and second grade students. This was intended as a baseline and to determine the reading performance of the school population. The purpose of this study was two-fold: (a) to track the general progression of reading acquisition skills for all kindergarten students and (b) to evaluate the effectiveness of a supplemental reading program with a selected subsample of low performing readers in kindergarten using the DIBELS as the benchmark assessment tool. 

Participants and Setting  

Participants in this study were two kindergarten classrooms in a large urban elementary school in the Midwest. The school had 480 students enrolled, with approximately 97% of the school population being of minority status (e.g., 96.2% identified as Black, 0.2% Hispanic, 0.2% Asian) and 98% of the school eligible for free/reduced lunch. The median household income in the surrounding area was less than $25,000 per year. At the time of this study, the school received the rating of “Academic Emergency,” which is the lowest rating a school can receive from the state. As a result, only 21.3% of their fourth graders passed the State of Ohio Reading Proficiency Test.   

In an effort to focus on ways to increase the reading achievement of all students in this elementary school, a project was initiated that used the DIBELS. It has shown to be highly predictive of early literacy skills (Good, Gruba, & Kaminsky, 2002). Kindergarten, first, and second grade students were individually administered the measures three times during the school year (e.g., fall, winter, and spring). For this pilot study, all 44 kindergarten students participated. A sub-sample of seven students, two boys and five girls, were selected to participate in an early reading intervention. All seven students were African American.  

Students were selected based on two criteria. First, students were identified as at-risk by their performance on the fall administration of the DIBELS. Each of the seven children received a DIBELS instructional recommendation of “Intensive Support,” based on their performance. To receive this recommendation, students had to score in the “Deficit” range for Initial Sound Fluency and Letter Naming Fluency. The DIBELS recommendations were based on the patterns of performance of over 30,000 kindergarten children across the country (Good et al., in press). These seven children were the lowest performing children in the kindergarten classes involved in the study. Second, the parent or legal guardian provided written consent for their child to participate in the intervention study. Two boys and three girls met these criteria and made up the intervention sub-sample. The remaining two students, both girls, served as the control group.  

Materials             

The DIBELS, developed by Good and Kaminski (2002), was used to assess early literacy skills. Again, it is a set of tasks designed to target specific skill areas, which are linked to the “Big Ideas” in early literacy development. They represent constructs that can be described as the foundational skills of early reading and are prerequisites of reading success (Good & Kaminski, 2001). The four DIBELS measures administered to the kindergarten class (fall, winter, or spring terms) included the Initial Sound Fluency (ISF), Phoneme Segmentation Fluency (PSF), Letter Naming Fluency (LNF), and Nonsense Word Fluency (NWF).   Alternate-form reliability of the DIBELS measures are adequate and range from .65 to .93 (Good, Grupa, & Kaminski, 2002). Concurrent criterion-related validity of DIBELS measures with other standardized measures of early literacy ranged from .36 for ISF in the beginning of kindergarten to .81 for LNF. Predictive validity correlations with outcomes 1-year later ranged from .36 to .82.   

Phonemic awareness is assessed with two DIBELS subtests, ISF, and PSF. ISF is a standardized, individually administered measure of phonological awareness that assesses a child's ability to recognize and produce the initial sound in an orally presented word (Kaminski & Good, 1996). The PSF measure assesses a student's ability to segment three- and four-phoneme words into their individual phonemes fluently. The PSF measure has been found to be a good predictor of later reading achievement (Kaminski & Good, 1996).  

The alphabetic principle is measured with the NWF and LNF measures. The NWF measure is a standardized, individually administered test of the alphabetic principle and of the ability to blend letters into words in which the letters represent their most common sounds in the English language (Kaminski & Good, 1996). The LNF is a standardized, individually administered test that is intended for students from the fall of kindergarten through the fall of first grade. The measure assesses a student’s ability to rapidly name letters, which is a skill thought to be predictive of later reading outcomes (Kaminski & Good, 1996).  

Benchmarks for the subtests are cumulative. In other words, if students met the PSF benchmark by the end of kindergarten, they were likely to meet the benchmark in mid-first grade for NWF (Kaminiski & Good, 1996). In 2001, Good and Kaminski demonstrated that the odds of being an established reader when meeting the PSF benchmark by the end of kindergarten was 84%. Results of the DIBELS are categorized as low risk, some risk, at-risk, established, emerging or deficit. The low risk, some risk and at-risk categories are utilized when the skill is not yet expected to be mastered. Low risk indicates that the student is well on track for mastering the goal. Some risk indicates that the student may benefit from some intervention. At-risk indicates that the student needs intensive intervention in order to learn the skills necessary, if they are to master the skill. Additionally, the other categories, such as established, emerging and deficit are used when mastery of the skill is expected. Established means the skill should be mastered at that time and that the student has mastered the goal. Emerging indicates the skill should be mastered at that time, but the student may require some intervention in order to master the skill. Deficit indicates the skill should be mastered, but the student’s performance indicates that intensive intervention is needed before mastery can occur. Based on data from previous studies with 30,000 kindergarten children which established correlations between success on the measures and future reading performance, these categories were developed (Good et al., in press).  

Supplemental Reading Program           

The Scott Foresman Early Reading Intervention Program (ERI) is a program designed for children who need early, intensive instruction for phonological awareness. The program teaches letter names, letter sounds, word reading, spelling and simple reading (Simmons & Kame’enui, 2003). Although the program is designed for use with kindergarten populations, it can be used with first-grade students who have not mastered early literacy skills. Because the program teaches skills prior to the time when most students become proficient readers, it is often referred to as a prevention program rather than an intervention program. The ERI teaches and reinforces early literacy skills including: (a) phonemic awareness; (b) letter-sound coordination; (c) word reading; (d) spelling and (e) sentence reading. The program uses a combination of carefully scripted, teacher-directed instruction and game format activities to teach and reinforce phonemic awareness, letter-sound recognition, word reading, spelling, and sentence reading. The entire curriculum consists of 126 lessons, in which students move sequentially through basic to higher order skills. More importantly, it is intended to be a supplement for a reading curriculum.

Procedures  

Grade Level Administration. Six school psychology graduate students were trained in DIBELS measurement procedures by a school psychologist with expertise on DIBELS. All researchers were provided with a demonstration of the procedures as well as written instructions. The researchers practiced administering the measures and scoring the probes, until there was 100% accuracy.  

All 44 kindergarten students were individually administered the DIBELS measurements three times during the school year (e.g., fall, winter, and spring). Administration of the DIBELS occurred in the morning. The researcher escorted each student to the assessment room and instructed the student to sit at the table across from the researcher. The assessment room was an empty classroom that was well lit and quiet. Each student was asked to complete the designated tasks for that period. After the student completed the DIBELS measurement tasks, they were given a sticker and escorted back to their classroom.  

            Intervention Program. Students identified as needing intensive interventions were assigned to one of two groups. The groups were developed based on class assignment as there were restraints placed on when students could be removed for the supplemental instruction. In other words, the students were not able to be randomly assigned to a group. After being identified by the DIBELS, each intervention student was administered the ERI to determine the starting point for the lessons in the intervention. Results indicated that all five kindergarten students were to begin at Lesson One. At the end of every six lessons, students were given progress monitoring probes as indicated by the ERI in order to determine if they could proceed to the next lesson.  

The ERI was conducted by two of the six school psychology graduate students trained on the DIBELS measurements. The graduate students received three hours of training on the ERI.  This included an overview of the program, instruction regarding administration guidelines, and practice with implementation and data collection. A videotape was provided for each graduate student to review, if necessary. The intervention was conducted for 16 weeks (e.g., from mid-January to mid-May). For Group One, which consisted of three students, the intervention was implemented twice a week. For Group Two, which consisted of two students, the intervention was implemented three times per week. Each intervention session was 30 minutes in length. The 30 minute lesson consisted of two 15-minute components delivered consecutively. The graduate students followed the scripted lessons in order as designated by the ERI.  The intervention occurred during times identified by the teacher. Intervention sessions did not occur during the reading block time. However, the intervention was conducted in an empty classroom that was well lit and very quiet.  

The two students in the control group received no special intervention; however, all the kindergarten children participated in an extra reading assistance program entitled HOSTS, Helping Our Students to Succeed. The program required that interventionists pair kindergarten students with an adult who worked with them individually for 30 minutes on reading. It most often consisted of helping the child read aloud.  

Dependent Variables  

            Letter Naming. Rapid letter naming was measured, a skill thought to be highly predictive of later reading outcomes. The student is provided a page with rows of both upper and lower-case letters and asked to name as many letters as possible within one minute. The number of letters named correctly in one minute is the dependent variable.           

Initial Sound. Students were shown four pictures in which the examiner says an initial sound orally and asks the students to match it to one of the pictures. The child was then asked to orally produce the beginning sound for an orally presented word that matches one of the given pictures. The examiner then calculates the amount of time taken to produce the correct sound and converts the score into the number of initial sounds correctly identified in one minute.  

            Phonemic Segmentation. This measure assesses a student's ability to segment three- and four-phoneme words into their individual phonemes fluently. The PSF task is administered by the examiner orally presenting words of three to four phonemes. It requires that the student to produce verbally the individual phonemes for each word. After the student responds, the examiner presented the next word, and the number of correct phonemes produced in one minute determined the final score. Moreover, the PSF measure has been found to be a good predictor of later reading achievement (Kaminski & Good, 1996).   

           Nonsense Word. On this task, the student was presented an 8.5” x 11” sheet of paper with randomly ordered vowel-consonant (VC) and consonant-vowel-consonant (CVC) patterns and asked to produce verbally the individual letter sound of each letter or verbally produce, or read, the whole nonsense word. Later, the students were allowed one minute to produce as many letter-sounds as they could. Thus, the final score was the number of letter-sounds produced correctly in one minute. 

Results  

Kindergarten Classes  

Of the five students who received intervention, two met benchmark on three of the four measures in the spring term. Interestingly, Good et al. (in-press) have found that meeting only one goal is not sufficient in becoming a successful reader. Therefore, achieving all benchmark goals is the best predictor for achieving literacy success. For the two control group students only one met one benchmark and that was in LNF.   

The results of this study indicated that as a grade, a sizable number of kindergarten students were in the “some risk” or “at-risk” category. Children who are low risk are considered to be well on track for mastering the goal. Table 1 displays the risk level for future reading for all kindergarten students on four DIBELS’ measures: (a) letter naming fluency, (b) initial sound fluency, (c) nonsense word fluency, and (d) phoneme segmentation fluency. The DIBELS Benchmark goals are the minimal level students need to achieve and are viewed as a stepping stone toward literacy (Good et al., 2002).   

Table 1  

Summary of Kindergarten students’ Risk Factor on Letter Naming Fluency, Initial Sound Fluency, Nonsense Word Fluency, and Phonemic Segmentation Fluency  

 Note.  The number of students varied during assessments due to student attrition.

Research using the DIBELS indicates that in the beginning of kindergarten, students should be able to name at least eight letters of the alphabet per minute (Good et al., 2002). By the middle of kindergarten, students should be able to name at least 27 letters, and by the end, students should be able to name 40 or more letters in one minute (Good et al.). Results in this study indicate that while there was some improvement between fall and winter administrations, the performance of the group, as a whole, regressed in the spring for Letter Naming Fluency. While the class average for LNF met benchmark (see Table 2), Table 1 shows that individually, there were 17 (41%) students that were designed as “some risk” or “at risk” for difficulty in achieving the early literacy benchmark.

Table 2  

Comparison of Kindergarten Class Mean Scores to DECIBELS Benchmarks

Note.  * Met Benchmark.

 With respect to the ISF proficiency, students should be learning initial sounds in words at the beginning of kindergarten and identifying at least 25 to 35 sounds per minute by the middle of kindergarten (Good et al., 2002). The results indicated that as a group, there was a general regression in the acquisition of this skill as the year progressed. With the at-risk category increasing by 12% between fall and winter terms, more students were in need of intensive intervention in order to learn the skills necessary if they were to master it. In the winter term, more children were in the “some risk” or “at-risk” (30 of 42) than the “low risk” range (Table 1).             

With respect to NWF proficiency, students should begin to learn some letter-sound correspondence in the middle of kindergarten and should be able to score 25 or higher on the measure in one minute in order to be on track for mastery of this skill in first grade (Good et al., 2002). Slightly more than 50% of the children have met benchmark, which means there were still a significant number of students who have not. Finally, on phoneme segmentation, students should be able to identify 35-45 correct phonemes said aloud in a 1-minute period by the end of kindergarten (Good et al., 2002). Results indicated that as a group, most kindergarten students (i.e. 30 of 42) had not met benchmark.  

This was significant since meeting benchmark on PSF increases a student’s odds of being an established reader at the end of first grade. Results also suggest that a significant segment of the kindergarten class was at-risk for being established readers at the end of first grade if they did not receive intensive intervention. Neither of the classes, based on their mean scores, met the ISF or PSF, which assessed Phonemic Awareness, an essential skill for reading.  

Intervention Students  

Table 3 displays the results for the students who received the ERI as well as the two control students who did not receive the intervention. Students “A” received two days of intervention weekly while Students “B” received three days of intervention weekly. On LNF, three of the five students met benchmark. However, a fourth student was close to making the goal.  On the ISF, none of the students met benchmark. It should be noted that two of the five intervention students were close to benchmark. Three of the five students met benchmark on the NWF task, both winter and spring terms.     

On the PSF task, two of the five students met benchmark with two more students very close to benchmark.  Student 2A and 1B also met benchmark, during the winter quarter administration of DIBELS.  Thus, it would be expected that they would continue to make progress in this area (Good et al., in press).  

Table 3

Comparison of Intervention and Control Students on DIBELS Benchmark