Introductory Statement

As directed, Board staff conducted research and analysis of the District's Reading Recovery/ Descubriendo La Lectura (RR/DLL) Program. The research project compared the academic performance of participating and non-participating first grade students based on two established assessments: norm reference tests and literacy assessment portfolios. The purpose of the research project is to determine whether San Diego City Schools should continue primarily financing the RR/DLL Program through Title I funding.

Background

History of Reading Recovery-

New Zealand educator and psychologist, Dr. Marie M. Clay, is the acknowledged founder of Reading Recovery. During the 1960's, Dr. Clay conducted observational research, which laid the foundation for techniques in detecting early reading and writing difficulties of children. During the 1970's, Dr. Clay worked with a team of experienced teachers in New Zealand to develop program procedures and conduct field trials. Based on test results of the pilot studies, in 1979, New Zealand adopted the RR Program nation-wide. In 1984, Reading Recovery was introduced in the United States. In 1990, Reading Recovery was introduced in California. In 1992, Descubriendo La Lectura, an application of Reading Recovery in Spanish, was constructed for Spanish-speaking students.

In San Diego City Schools, the RR Program was introduced during the 1991-92 school year. The impetus was provided by the San Diego County Reading Recovery Consortium that included three school districts: San Diego Unified, Poway Unified and Escondido Union. The Spanish version of Reading Recovery was included during the 1995-96 school year. Since then, the RR/DLL Program has expanded to 48 school sites with a staffing level of 117.5 teachers. Provided below is a table illustrating this expansion:

Overview of Reading Recovery-

Reading Recovery is an early intervention program designed to assist children in first grade who are having difficulty learning to read and write. The RR/DLL Program entails the use of trained teachers providing one-to-one tutoring in 30-minute daily sessions to the lowest 20% of a first grade class who have the prerequisite skills.¹ The primary goal of Reading Recovery is to bring the lowest 20% performing students up to the average reading proficiency level of their classmates by the end of first grade within 60 lessons, or approximately 12 weeks.

The first two weeks of the student's participation are designed to promote engagement with literacy activities. This period is referred to as "roaming around the known" and is comprised of a variety of reading and writing activities. The student's individualized lesson plans include the following seven components:

Generally, when students successfully complete 60 RR/DLL lessons AND reach the average reading proficiency level of their classmates, they are discontinued,² at which time the RR/DLL Teacher can take another student into the 30-minute slot. Each RR/DLL Teacher, working a half-day with Reading Recovery, is expected to be able to tutor eight students in one year. The other half a day is devoted to conducting "guided reading literacy groups."

Professional development is an integral part of the RR/DLL Program. Training consists of an intense, year-long graduate course for teachers made up of weekly classes affiliated with a university-based Regional Training Center. Teachers from San Diego City Schools attend the Center located at California State University, San Bernardino. In August 1994, San Diego City Schools established its own training facility located at Valencia Park Elementary. As teachers learn how to implement the program, they work simultaneously with children in their home schools. Through clinical and peer-critiquing experiences guided by Teacher Leaders, RR/DLL Teachers learn to use diagnostic techniques and teaching procedures for conducting lessons.

The criteria for selecting classroom teachers to become RR/DLL teachers has been: 1) at least three years of teaching experience; 2) experience at the primary level; and 3) recommendation by principal, administrator and/or other teachers.

²The 1996-97 Reading Recovery Site Report (page 17) further states: "Decisions concerning whether or not children could be discontinued were made by examining a variety of data for each dhild: 1) highest level of text reading at 90% accuracy or better; 2) scores on three Observation Survey assessments: Writing Vocabulary, Dictation and Text Reading Level; 3) reading behavior as shown in recent running records and text reading tests; and 4) achievement in the classroom instructional program."

Demographic Profile of RR/DLL Participant-

The research project focuses on first grade students who participated in the RR/DLL Program during 1995-96 and 1996-97. Provided below is a table listing the number of students distinguished between those who successfully completed the 60 lessons AND then performed at the average level of their classmates (discontinued) and those who did not (not discontinued).

Provided below is a table reflecting the race/ethnicity of those first grade students who participated in the RR/DLL Program:

For first grade students participating during 1995-96, analysis was conducted to ascertain the reasons why certain participating students DID NOT successfully complete the RR/DLL Program. Similar analysis could not be conducted for 1996-97. A summary table³ is provided below:

Research Methodology-

The research project compares the reading proficiency levels of first grade students who participated in the RR/DLL Program during 1995-96 and 1996-97 with a control group of non-participating students from each school site. In fact, the control group consists of the same students included in the Reading Recovery Site Reports⁴ authored by the District's RR Program Teacher Leaders.

As previously stated, Reading Recovery asserts that, through targeted intervention, the reading level of the lowest performing 20% of students will improve to the average reading level of their non-participating classmates once the student has successfully completed the program. For example, the 1996-97 Site Report states at least 96% of its RR participants and 98% of its DLL participants accomplished this task (pages 19-20). The "sustained effect" of Reading Recovery into the third grade for participating, first grade students was ascertained by re-administering the Text Reading component of the Observation Survey. The 1996-97 Site Report (pages 42-43) states that 88% of RR discontinued students and 95% of DLL discontinued students were reading at or above grade level.

Thus, to further evaluate the efficacy of the RR/DLL Program the above-referenced tenets of Reading Recovery are examined by relying upon norm reference and reading assessment test results for participating vs. non-participating first grade students one and two years after participating in the internal program study. By confining the elapsed time to two years, the study focuses on the initial period of the student's academic career when they are still "learning to read."

⁴Prepared by SDUSD Educational Services Division, Humanities Department. The Reading Recovery Site Reports refer to these students as a "site random sample" established by compiling class lists of first grade students from sites with a RR/DLL Program. This control group was established to determine "a site average band." At the end of each school year, participating first grade students along with the control group are tested with the same three parts of the Observation Survey. The three parts include writing vocabulary, dictation and text reading.

The research project assesses the reading proficiency levels of participating, first grade students distinguished between three groups that comprise the study cohorts:

1. Those who successfully completed the RR/DLL Program (discontinued),
2. Those who did not successfully complete the RR/DLL Program (non-discontinued), and
3. Those who did not participate (control group) from the same school site.

The summary table below outlines the academic indicators used to assess the reading proficiency levels of the study cohorts:

The SAT 9 Total Reading test scores included sub-tests assessing Reading Vocabulary and Reading Comprehension. The Literacy Assessment Portfolio included sub-tests assessing Oral Accuracy, Oral Comprehension and Oral Fluency.

Using the data from SAT 9 and Literacy Portfolios, the performance levels of the first grade students included in this research project were assessed based on the following methodology:

As previously suggested, the academic indicators and methodology used in this research project are substantively different than those used in previous studies of the District's RR/DLL Program.

For example, over the past years, the District's Reading Recovery Program Teacher Leaders have prepared annual Reading Recovery Site Reports.⁵ These Reports focused on participating, first grade students who completed the required 60 lessons. Academic performance was evaluated using three of the six measures included in the program-designed Observation Survey. The evaluation process included selected text materials in graded levels of difficulty, based on a scale of 1-30. The efficacy of the RR/DLL Program was based on comparative data using the Observation Survey. Participating students were tested at the beginning and end of the school year. Non-participating students randomly selected from each school site were tested at the end of the school year. Program success was based on the number of participating, first grade students who discontinued from the RR/DLL Program AND attained the average reading level of the site random sample.

Other district reports have been prepared as well. One particular report, entitled the "1996-97 Progress Report on the Reading Recovery Program⁶" also evaluated the sustained effect of Reading Recovery. The study analyzed second and third grade mid-year English Language Arts academic marks of first grade students who participated in Reading Recovery during 1994-95 and 1995-96. It is important to note that this report states that 1994-95 was the first year in which the District's RR Program adhered to the criterion of selecting first grade students in the lowest 20% of their classes. In previous years, students were selected from an upper range within the lowest 30%. This study did not include a control group per se. Rather, participating students served as their own control group distinguished between discontinued vs. not discontinued. The report concluded:

"Thus, the program's adherence to the selection criterion has been accompanied by three important outcomes: a noticeably low percentage of students who are discontinued; a noticeably low percentage who are at least "moderately experienced readers" in Grades 2 or 3; and a considerable number of students who are not discontinued because they have not completed sufficient RR lessons or they have not reached the average level of their first-grade classmates." (Executive Summary)

Research Findings-

The study cohort for the research project includes 678 participating first grade students, along with 106 first grade students who comprise the control group of non-participants. A summary table is provided below:

⁶Prepared by SDUSD Planning, Assessment, Accountability & Development Division. This report was submitted to the former Senior Management Council, but was never presented to the Board of Education for review and comment.

⁷The 1996-97 Progress Report (page 6) indicated 237 RR participants, and 127 DLL participants for a total of 364 participating, first grade students. Based on the state 364 students, the study cohort reflects a difference of 98.

⁸The 1996-97 Site Report (page 8) indicated 303 RR participants, and 150 DLL participants, totaling 453 participating, first grade students. However, analysis was conducted on 361 students only. Thus, 92 students were unaccounted for in the study's research findings. Based on the stated 453 students, the study cohort reflects a difference of only 41.

Attachment B details the norm reference and literacy assessment test scores between participating and non-participating first grade students who comprise the 1995-96 and 1996-97 study cohorts. A review of the test scores reveal the following:

RR/DLL Program Funding-

Since its inception, the primary source of funds used to finance the District's RR/DLL Program has been Title I federal categorical funding. During 1991-92, Title I constituted 50% of Reading Recovery's total funding, which increased to 87% in 1998-89. During 1998-99, 20.4% of the District's total Title I funding was expended on the RR/DLL Program. Provided below is a table outlining the historical growth rate in funding dedicated to the District's RR/DLL Program:

An expressed concern of the District's RR/DLL Program has been the high per capita cost of delivering this intervention strategy. Provided below is a summary table outlining the per capita cost, based on the number of participating students:

Instructional Implications

This evaluation of the District's RR/DLL Program using SAT 9 and Literacy Portfolio test scores of participating vs. non-participating first grade students indicates a dire need to conduct an independent, comprehensive analysis of the District's Reading Recovery/Descubriendo La Lectura (RR/DLL) Program.

Beginning with the literacy framework that was introduced during the 1998-99 school year, the District's Institute for Learning has prioritized reading as the "gatekeeper" core subject upon which to elevate the overall academic performance of all students. Consequently, new initiatives have been implemented to address student reading proficiency levels, including After-School Reading (Grade 3), and Two-Period Literacy Blocks (Grades 6, 7 and 9).

Consequently, it must be recognized the District needs to increase its capacity to systematically conduct comprehensive studies involving major qualitative and quantitative analyses of its instructional programs - especially those dedicated to increasing reading proficiency levels.

Facilities Implications

There are no facilities implications.

Budget Implications

Since its introduction in 1991-92, the level of Title I funding devoted to Reading Recovery has increased by over $6.8 million. In 1998-99, Reading Recovery encumbered 20% of the District's total Title I categorical funding.

The financing of support programs and intervention strategies to address social promotion/retention coupled with the Superintendent's edict that Title I school site funding allotments will be based on 40% dedicated to literacy and 20% to academic achievement will create increasing competition for limited school site discretionary funding. The high per capita cost to deliver Reading Recovery to address the lowest 20% of performing first grade students precludes elementary school sites from enlisting alternative strategies that may (or may not) prove more cost-effective and cost-efficient. Thus, before expanding Reading Recovery critical comprehensive research and analysis should be conducted to validate the efficacy of this intervention strategy.

Public Support and Engagement Implications

There are no direct implications regarding public support and engagement as a result of the research findings from this independent study of Reading Recovery.

Board Policy Implications This Board Report complies with board policies F-8400 (Assessment and Training), F-8800 (Program Evaluation) and F-9400 (Research).

Policy Recommendation

Based on the findings of this research project conducted on the District's RR/DLL Program, it is recommended that your Board direct Board Staff to:

It is further recommended that your Board direct the Superintendent to provide a supplemental report concerning operational improvements to enhance the District's RR/DLL Program in conjunction with the 2000-2001 Title I/State Compensatory Education Participation Plan (tentatively scheduled for April 2000).

Report prepared by Jerome Torres, Board Policy Analyst. Technical assistance provided by Nick Nicholas.

d:\readbdrpt1.doc

SOURCES

"Reading Recovery Site Report -Year Three 1996-97," prepared by Ann Fulmer, Billie Greene, Helena Sabala, SDUSD Educational Services Division, Humanities Department

"1996-97 Progress Report on the Reading Recovery Program," prepared by Barry Fass-Holmes and Frank Ciriza, SDUSD Evaluation Unit, September 8, 1997

"Report on the Reading Recovery Program in San Diego Unified School District," prepared by Barry Fass-Holmes and Donna Davis, SDUSD Evaluation Unit, August 21, 1995

"1995-96 Progress Report on the Reading Recovery Program," prepared by Barry-Fass Holmes and Frank Ciriza, SDUSD Evaluation Unit, December 16, 1996

"Reading Recovery in Relation to Language Factors, Reading Self-Perceptions, Classroom Behavior Difficulties and Literacy Achievement: A Longitudinal Study," prepared by James W. Chapman, William E. Tummer & Jane E. Prochnow, Massey University, New Zealand. (Paper presented at the annual meeting of the American Educational Research Association, San Diego, California - April 1998.)

'Touching the Future: Executive Summary, 1991-97," prepared by Judith Neal, Patricia Kelly, Adria Klein, and Barbara Schubert, California Reading Recovery

"Reading Recovery Review: Understandings, Outcomes & Implications," prepared by Billie Askew, Irene Fountas, Carol Lyons, Gay Su Pinnell and Maribeth Schmitt, Reading Recovery Council of North America, Inc.

"Reading Recovery: An Evaluation of Benefits and Costs," prepared by Bonnie Grossen, Gail Coulter and Barbara Ruggles, Effective School Practices, 15(3), Summer 1996

"Questions and Conclusions from a Discussion of Reading Recovery," prepared by Patrick Groff, Effective School Practices 15(3), Summer 1996

"Reading Recovery in the San Diego Unified School District 1998: A Summary of Findings and Recommendations," prepared by Jacqueline L. Chaparro, March 1998

Home | About Us | About Phonics | Resources
Research | Topics | Reading Reform | Links | Search