
| Home | Purpose | Editors | Board of Reviewers | Author Guidelines | Review Process | Issues |
The Use of a Benchmark Assessment to Track Progress of a Reading
Intervention: Tiffany A. Jenkins, M.A., Antoinette
Halsell Miranda, Ph.D., and Valerie E. Cook, M.Ed. Abstract:
The use of a benchmark assessment to
track the general progression of reading acquisition skills for
kindergarten students in an urban elementary school was investigated.
The benchmark assessment used was the Dynamic Indicators of Basic Early
Literacy Skills, which has been shown to be highly predictive of early
literacy (Good, Gruba, & Kaminski, 2002). The results of this study
indicated that early literacy development in kindergarten is consistent
with the research literature demonstrating that African American
students are placed at-risk for reading failure. Less than 50% of the
students were on track to becoming established readers. The need for
early identification of children at-risk for reading difficulty as well
as suggestions for overcoming barriers implementing prevention reading
programs in urban schools are discussed. Introduction The
federally-mandated and reauthorized Elementary and Secondary Education
Act, also known as the No Child Left Behind Act of
2001 (NCLB), is having a considerable impact on education,
particularly in urban areas. Heavy emphasis is being placed on reading
and closing the achievement gap between ethnic minority students and
their White counterparts. As a result, effective interventions are seen
as key to making this happen as well as providing on-going assessment to
determine if children are making adequate yearly progress and to make
data-based decisions regarding a student’s education (McGee &
Richgels, 2003). Using appropriate instruments to assess the needs of
children is an educational mandate. Moreover, implementing teaching
strategies in urban districts to prevent literacy failure is equally
critical to the success of many ethnic minority students, especially
African American students. In 1998, the
National Reading Council produced a comprehensive report on the
nation’s reading achievement status and found that children of color
and those raised in poverty were continually documented as being at risk
for academic failure. It is estimated that approximately one-third of
the children in the United States are at-risk of academic failure by the
time they enter kindergarten due to socially and economically
impoverished conditions (National Assessment of Educational Progress,
2000). Unfortunately, African American students are far more likely to
be raised in poverty than White children (McGee & Richgels, 2003). While 40% of the
White fourth graders scored at or above proficiency in reading, only 12%
of their African American counterparts reached the same level (National
Assessment of Educational Progress, 2000). Learning to read well is
believed to be the principal goal of first grade instruction (Farkas,
2003). Children from low socioeconomic environments, especially African
Americans, consistently perform below their peers in reading,
mathematics, and science, which is in part a result of inadequate
attainment of literacy skills (Farkas, 2003).
Unfortunately, the achievement gap continues and widens as these
students proceed through formal schooling (Ferguson, 2003). Children who do not
receive proper early reading intervention often face harsh consequences.
In 1988, Juel reported that 88% of the children who scored in the lowest
quartile for reading comprehension at the end of first grade continued
to score below the 50th percentile at the end of fourth grade. These
findings also strongly indicated that many children do not receive
instruction or effective interventions to successfully facilitate
remediation for difficulties noted at the end of the first grade year.
In 1986, Stanovich presented the “Matthew Effect” phenomenon – the
idea that the rich get richer and the poor get poorer. This notion is
found to be an on-going occurrence in urban school systems. Stanovich
also found that students who develop early literacy skills acquire tools
needed to grow exponentially in their knowledge and skill over time.
Conversely, students who fail to develop early literacy skills fall
further behind with the progression of time. In addition, this downward
spiraling of reading achievement has been proposed as a major
determinant of school failure (Davis, 2003; Farkas, 2003; Ferguson,
2003). When children begin to transition from “learning to read” to
“reading to learn,” those who have difficulty reading have a harder
time exploring and comprehending science, literature, history,
mathematics and other core academic subjects. Additionally, poor skills
in reading are associated with poverty, unemployment, and dropping out
of high school, which are all predictive of a bleak future (Chandler,
2000; McGee & Richgels, 2003). Fourteen percent of the dropouts,
from 16- to19-year-olds reside in urban neighborhoods. Unfortunately,
this is twice the rate for teens in suburban settings (7%). In high
poverty neighborhoods located within large cities (e.g., poverty rates
above 20%), one-fifth of 16- to 19-year-olds drop out of school (Annie
E. Casey Foundation, 2000). Class
size and teacher quality have also been identified as a risk factor for
reading difficulties. The U.S. Department of Education (2000) found that
teachers in schools with a high ethnic minority enrollment and/or high
percentage of students eligible for free or reduced-price lunch were
less likely to have master’s degrees than teachers in schools with low
ethnic minority enrollment and/or a low percentage of students eligible
for free or reduced-price lunch. Since most urban school systems have
high ethnic minority enrollment and high percentages of students
receiving free or reduced lunch, it is plausible to conclude, on
average, that the percentage of teachers in suburban districts have
received more training than teachers in urban districts. However, this
statement is not meant to imply that urban schools are at an automatic
disadvantage, but simply to point out that teacher education and
training varies greatly, and therefore, an extra effort must be exerted
by less prepared teachers to ensure their students receive the best
instruction possible. In addition to teacher quality, educational
researchers have also examined the influence of teachers’ perceptions
and expectations on the academic achievement of African Americans (Farkas,
2003; Ferguson, 1998; Flowers, Milner, & Moore, 2003). It has
generally been concluded that negative teacher expectations and
perceptions play some part in the lower academic achievement of African
Americans. In addition to the
variation in teacher training, it was also found that class size varies
between schools with high versus low ethnic minority enrollment
(Pritchard, 1999). Kindergarten classrooms with less than 10% ethnic
minority children were more likely to have 15 or fewer children than
classrooms where 75% or greater of the children were minorities
(Pritchard, 1999). If class size is reduced to below 20 students,
research indicates that disadvantaged children of color have the
greatest increase in student achievement (Pritchard, 1999). There has been a
substantial amount of research conducted on phonemic awareness
instruction of students enrolled in pre-school and kindergarten
programs. Lundberg, Frost, and Peterson (1988) found that direct
instruction of phonemic awareness in preschool had a positive effect on
reading and spelling acquisition. In 1995, Kozminsky and Kozminsky found
that explicit instruction of phonological awareness in kindergarten
resulted in statistically significant gains for students in reading
versus students who did not receive explicit instruction. Furthermore,
they found that the explicit instruction positively influenced
students’ scores on measures of reading comprehension in the first
grade. In 2000, the National Reading Panel conducted a review of
research and found that many children required direct instruction of
phonemic awareness. Therefore, Good et al. (in press) recommended that
these skills be taught in kindergarten and first grade, when children
first begin to acquire reading skills. This is particularly important
for African American children who, on average, enter school with
lower-levels of school readiness, oral language, and prereading skills
than do White and Asian children (Farkas, 2003). Before assessing early literacy skills, it is important to understand the underlying concepts a student must possess in order to succeed. According to Simmons and Kame’enui (1998), there are five main concepts that must be considered: (a) phonemic awareness, (b) alphabetic principle, (c) fluency with text, (d) vocabulary, and (e) comprehension. In order for a reading program or intervention to be effective, it is suggested that all five components are included within the structure of the program. The use of an assessment that includes these five components can inform instruction and provide guidance for an effective beginning reading program. Dynamic
Indicators of Basic Early Literacy Skills (DIBELS) DIBELS
is a set of tasks designed to assess fundamental reading skills through
the use of seven assessment measures: (a) Letter-naming Fluency, (b)
Initial Sound Fluency, (c) Phoneme Segmentation Fluency, (d) Nonsense
Word Fluency, (e) Oral Reading Fluency, (f) Story Retell (optional), and
(g) Word Use Fluency (optional). These measures are specifically
designed to assess three of the five main concepts of early literacy:
Phonemic Awareness, Alphabetic Principle, and Fluency with Connected
Text. The measures are linked to one another, both psychometrically and
theoretically, and have strong psychometric properties for predicting
later reading proficiency (Good & Kaminski, 2001).
DIBELS
enables educators to quickly and efficiently, with a high degree of
confidence, identify students who require additional reading
intervention. The general assessment is completed three times per year,
preferably once during the academic fall, winter, and spring terms. Once
the student has completed the assessment, data can be easily analyzed to
determine individual performance. By utilizing the data, it is possible
to identify students who are at-risk for reading failure. If a student
is identified as being at-risk for reading difficulties, it is advised
that interventions be provided to the student in order to assist in
achieving proficient literacy in problematic areas (McGee & Richgels,
2003). Good
and Kaminski (2002) indicated several advantages of using DIBELS,
including that the measures are: (a) reliable and valid measures of
early literacy skills; (b) simple and efficient to administer; (c)
inexpensive to produce in terms of time and resources; (d) sensitive to
improvement in student’s skills over time; (e) available in multiple
forms of short duration to facilitate frequent administration by
educators; and (f) based on production-type responses, so that
students’ skills can be observed. Over the years, DIBELS has been
implemented in numerous school districts, including large urban
districts and has been shown to be a successful assessment tool for
identifying children at-risk for being established readers (Burns,
Lateer-Huhn, Poth, & Stollar, 2004). Thus, prevention is considered
to be the key in helping children obtain proficient literacy skills. As
educators and educational scholars, teachers and researchers strive to
prevent reading failure in order to combat reading difficulties for
students who do not possess these skills. Method A pilot study was initiated with a large urban elementary school in which the DIBELS was administered three times during the school year to kindergarten, first, and second grade students. This was intended as a baseline and to determine the reading performance of the school population. The purpose of this study was two-fold: (a) to track the general progression of reading acquisition skills for all kindergarten students and (b) to evaluate the effectiveness of a supplemental reading program with a selected subsample of low performing readers in kindergarten using the DIBELS as the benchmark assessment tool. Participants and Setting Participants in
this study were two kindergarten classrooms in a large urban elementary
school in the Midwest. The school had 480 students enrolled, with
approximately 97% of the school population being of minority status
(e.g., 96.2% identified as Black, 0.2% Hispanic, 0.2% Asian) and 98% of
the school eligible for free/reduced lunch. The median household income
in the surrounding area was less than $25,000 per year. At the time of
this study, the school received the rating of “Academic Emergency,”
which is the lowest rating a school can receive from the state. As a
result, only 21.3% of their fourth graders passed the State of Ohio
Reading Proficiency Test. In an effort to
focus on ways to increase the reading achievement of all students in
this elementary school, a project was initiated that used the DIBELS. It
has shown to be highly predictive of early literacy skills (Good, Gruba,
& Kaminsky, 2002). Kindergarten, first, and second grade students
were individually administered the measures three times during the
school year (e.g., fall, winter, and spring). For this pilot study, all
44 kindergarten students participated. A sub-sample of seven students,
two boys and five girls, were selected to participate in an early
reading intervention. All seven students were African American. Students were
selected based on two criteria. First, students were identified as
at-risk by their performance on the fall administration of the DIBELS.
Each of the seven children received a DIBELS instructional
recommendation of “Intensive Support,” based on their performance.
To receive this recommendation, students had to score in the
“Deficit” range for Initial Sound Fluency and Letter Naming Fluency.
The DIBELS recommendations were based on the patterns of performance of
over 30,000 kindergarten children across the country (Good et al., in
press). These seven children were the lowest performing children in the
kindergarten classes involved in the study. Second, the parent or legal
guardian provided written consent for their child to participate in the
intervention study. Two boys and three girls met these criteria and made
up the intervention sub-sample. The remaining two students, both girls,
served as the control group. Materials The DIBELS,
developed by Good and Kaminski (2002), was used to assess early literacy
skills. Again, it is a set of tasks designed to target specific skill
areas, which are linked to the “Big Ideas” in early literacy
development. They represent constructs that can be described as the
foundational skills of early reading and are prerequisites of reading
success (Good & Kaminski, 2001). The four DIBELS measures
administered to the kindergarten class (fall, winter, or spring terms)
included the Initial Sound Fluency (ISF), Phoneme Segmentation Fluency (PSF),
Letter Naming Fluency (LNF), and Nonsense Word Fluency (NWF).
Alternate-form reliability of the DIBELS measures are adequate
and range from .65 to .93 (Good, Grupa, & Kaminski, 2002).
Concurrent criterion-related validity of DIBELS measures with other
standardized measures of early literacy ranged from .36 for ISF in the
beginning of kindergarten to .81 for LNF. Predictive validity
correlations with outcomes 1-year later ranged from .36 to .82.
Phonemic awareness is assessed with two DIBELS subtests, ISF,
and PSF. ISF is a standardized, individually administered measure of
phonological awareness that assesses a child's ability to recognize and
produce the initial sound in an orally presented word (Kaminski &
Good, 1996). The PSF measure assesses a student's ability to segment
three- and four-phoneme words into their individual phonemes fluently.
The PSF measure has been found to be a good predictor of later reading
achievement (Kaminski & Good, 1996). The alphabetic principle is measured with the NWF and LNF
measures. The NWF measure is a standardized, individually administered
test of the alphabetic principle and of the ability to blend letters
into words in which the letters represent their most common sounds in
the English language (Kaminski & Good, 1996). The LNF is a
standardized, individually administered test that is intended for
students from the fall of kindergarten through the fall of first grade.
The measure assesses a student’s ability to rapidly name letters,
which is a skill thought to be predictive of later reading outcomes
(Kaminski & Good, 1996). Benchmarks for the subtests are cumulative. In other words,
if students met the PSF benchmark by the end of kindergarten, they were
likely to meet the benchmark in mid-first grade for NWF (Kaminiski &
Good, 1996). In 2001, Good and Kaminski demonstrated that the odds of
being an established reader when meeting the PSF benchmark by the end of
kindergarten was 84%. Results of the DIBELS are categorized as low risk,
some risk, at-risk, established, emerging or deficit. The low risk, some
risk and at-risk categories are utilized when the skill is not yet
expected to be mastered. Low risk indicates that the student is well on
track for mastering the goal. Some risk indicates that the student may
benefit from some intervention. At-risk indicates that the student needs
intensive intervention in order to learn the skills necessary, if they
are to master the skill. Additionally, the other categories, such as
established, emerging and deficit are used when mastery of the skill is
expected. Established means the skill should be mastered at that time
and that the student has mastered the goal. Emerging indicates the skill
should be mastered at that time, but the student may require some
intervention in order to master the skill. Deficit indicates the skill
should be mastered, but the student’s performance indicates that
intensive intervention is needed before mastery can occur. Based on data
from previous studies with 30,000 kindergarten children which
established correlations between success on the measures and future
reading performance, these categories were developed (Good et al., in
press). Supplemental Reading Program
The Scott Foresman Early Reading Intervention Program (ERI)
is a program designed for children who need early, intensive instruction
for phonological awareness. The program teaches letter names, letter
sounds, word reading, spelling and simple reading (Simmons &
Kame’enui, 2003). Although the program is designed for use with
kindergarten populations, it can be used with first-grade students who
have not mastered early literacy skills. Because the program teaches
skills prior to the time when most students become proficient readers,
it is often referred to as a prevention program rather than an
intervention program. The ERI teaches and reinforces early literacy
skills including: (a) phonemic awareness; (b) letter-sound coordination;
(c) word reading; (d) spelling and (e) sentence reading. The program
uses a combination of carefully scripted, teacher-directed instruction
and game format activities to teach and reinforce phonemic awareness,
letter-sound recognition, word reading, spelling, and sentence reading.
The entire curriculum consists of 126 lessons, in which students move
sequentially through basic to higher order skills. More importantly, it
is intended to be a supplement for a reading curriculum. Procedures
Grade
Level Administration.
Six school psychology
graduate students were trained in DIBELS measurement procedures by a
school psychologist with expertise on DIBELS. All researchers were
provided with a demonstration of the procedures as well as written
instructions. The researchers practiced administering the measures and
scoring the probes, until there was 100% accuracy. All 44 kindergarten
students were individually administered the DIBELS measurements three
times during the school year (e.g., fall, winter, and spring).
Administration of the DIBELS occurred in the morning. The researcher
escorted each student to the assessment room and instructed the student
to sit at the table across from the researcher. The assessment room was
an empty classroom that was well lit and quiet. Each student was asked
to complete the designated tasks for that period. After the student
completed the DIBELS measurement tasks, they were given a sticker and
escorted back to their classroom.
Intervention Program. Students
identified as needing intensive interventions were assigned to one of
two groups. The groups were developed based on class assignment as there
were restraints placed on when students could be removed for the
supplemental instruction. In other words, the students were not able to
be randomly assigned to a group. After being identified by the DIBELS,
each intervention student was administered the ERI to determine the
starting point for the lessons in the intervention. Results indicated
that all five kindergarten students were to begin at Lesson One. At the
end of every six lessons, students were given progress monitoring probes
as indicated by the ERI in order to determine if they could proceed to
the next lesson. The ERI was
conducted by two of the six school psychology graduate students trained
on the DIBELS measurements. The graduate students received three hours
of training on the ERI. This
included an overview of the program, instruction regarding
administration guidelines, and practice with implementation and data
collection. A videotape was provided for each graduate student to
review, if necessary. The intervention was conducted for 16 weeks (e.g.,
from mid-January to mid-May). For Group One, which consisted of three
students, the intervention was implemented twice a week. For Group Two,
which consisted of two students, the intervention was implemented three
times per week. Each intervention session was 30 minutes in length. The
30 minute lesson consisted of two 15-minute components delivered
consecutively. The graduate students followed the scripted lessons in
order as designated by the ERI. The
intervention occurred during times identified by the teacher.
Intervention sessions did not occur during the reading block time.
However, the intervention was conducted in an empty classroom that was
well lit and very quiet. The two students in
the control group received no special intervention; however, all the
kindergarten children participated in an extra reading assistance
program entitled HOSTS, Helping Our Students to Succeed. The program
required that interventionists pair kindergarten students with an adult
who worked with them individually for 30 minutes on reading. It most
often consisted of helping the child read aloud. Dependent Variables Letter
Naming. Rapid letter naming was measured, a skill thought to be
highly predictive of later reading outcomes. The student is provided a
page with rows of both upper and lower-case letters and asked to name as
many letters as possible within one minute. The number of letters named
correctly in one minute is the dependent variable. Initial
Sound.
Students were shown four pictures in which the examiner says an initial
sound orally and asks the students to match it to one of the pictures.
The child was then asked to orally produce the beginning sound for an
orally presented word that matches one of the given pictures. The
examiner then calculates the amount of time taken to produce the correct
sound and converts the score into the number of initial sounds correctly
identified in one minute. Phonemic Segmentation. This measure assesses a student's ability to segment three- and four-phoneme words into their individual phonemes fluently. The PSF task is administered by the examiner orally presenting words of three to four phonemes. It requires that the student to produce verbally the individual phonemes for each word. After the student responds, the examiner presented the next word, and the number of correct phonemes produced in one minute determined the final score. Moreover, the PSF measure has been found to be a good predictor of later reading achievement (Kaminski & Good, 1996). Nonsense Word. On this task, the student was presented an 8.5” x 11” sheet of paper with randomly ordered vowel-consonant (VC) and consonant-vowel-consonant (CVC) patterns and asked to produce verbally the individual letter sound of each letter or verbally produce, or read, the whole nonsense word. Later, the students were allowed one minute to produce as many letter-sounds as they could. Thus, the final score was the number of letter-sounds produced correctly in one minute. Table 1 Summary of Kindergarten students’ Risk
Factor on Letter Naming Fluency, Initial Sound Fluency, Nonsense Word
Fluency, and Phonemic Segmentation Fluency
Research using the DIBELS indicates that in the beginning of kindergarten, students should be able to name at least eight letters of the alphabet per minute (Good et al., 2002). By the middle of kindergarten, students should be able to name at least 27 letters, and by the end, students should be able to name 40 or more letters in one minute (Good et al.). Results in this study indicate that while there was some improvement between fall and winter administrations, the performance of the group, as a whole, regressed in the spring for Letter Naming Fluency. While the class average for LNF met benchmark (see Table 2), Table 1 shows that individually, there were 17 (41%) students that were designed as “some risk” or “at risk” for difficulty in achieving the early literacy benchmark. Table 2 Comparison of Kindergarten Class Mean Scores to DECIBELS Benchmarks
Note. * Met Benchmark. With respect to NWF
proficiency, students should begin to learn some letter-sound
correspondence in the middle of kindergarten and should be able to score
25 or higher on the measure in one minute in order to be on track for
mastery of this skill in first grade (Good et al., 2002). Slightly more
than 50% of the children have met benchmark, which means there were
still a significant number of students who have not. Finally, on phoneme
segmentation, students should be able to identify 35-45 correct phonemes
said aloud in a 1-minute period by the end of kindergarten (Good et al.,
2002). Results indicated that as a group, most kindergarten students
(i.e. 30 of 42) had not met benchmark. This was
significant since meeting benchmark on PSF increases a student’s odds
of being an established reader at the end of first grade. Results also
suggest that a significant segment of the kindergarten class was at-risk
for being established readers at the end of first grade if they did not
receive intensive intervention. Neither of the classes, based on their
mean scores, met the ISF or PSF, which assessed Phonemic Awareness, an
essential skill for reading. Intervention Students Table 3 displays
the results for the students who received the ERI as well as the two
control students who did not receive the intervention. Students “A”
received two days of intervention weekly while Students “B” received
three days of intervention weekly. On LNF, three of the five students
met benchmark. However, a fourth student was close to making the goal. On
the ISF, none of the students met benchmark. It should be noted that two
of the five intervention students were close to benchmark. Three of the
five students met benchmark on the NWF task, both winter and spring
terms. On the PSF task,
two of the five students met benchmark with two more students very close
to benchmark. Student 2A and
1B also met benchmark, during the winter quarter administration of
DIBELS. Thus, it would be
expected that they would continue to make progress in this area (Good et
al., in press). Table 3 Comparison of Intervention and
Control Students on DIBELS Benchmark
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||