SRI Conducts Evaluation of National Writing Project Professional Development and Its Affect on Student Writing Achievement

Date: November 1, 2007

Summary: SRI International has been commissioned by the NWP to conduct an independent five-year study to examine changes in students' writing performance, students' writing practices, teachers' instructional practices, and teachers' professional communities in middle schools forming partnerships with local National Writing Project sites.

SRI International has been commissioned by the National Writing Project (NWP), to conduct an independent five-year study, the National Evaluation of Writing Project Professional Development. The study was launched in November 2006.

This evaluation employs a randomized control trial design to examine changes in students' writing performance, students' writing practices, teachers' instructional practices, and teachers' professional communities in middle schools forming partnerships with local National Writing Project sites.

SRI has drawn a random sample of eligible local NWP sites and schools and is collecting baseline data on them during the 2007–08 school year. The baseline data—and data to be collected throughout the study—come from on-demand writing samples; artifacts of instruction, including teacher assignments, instructional logs, and student work; annual teacher surveys; interviews and focus groups; and school records from 40 schools.

During this initial year of participation in the study, half of the schools have been randomly chosen to form "partnerships" (i.e., to begin receiving services from a local NWP site). The other half will remain in the study as controls ("delayed treatment" schools) and will receive NWP services after the study is completed. Data collection will continue through the 2010–2011 school year in both "treatment" and "control" schools.

Comprehensive case studies will be conducted in a subset of these schools to help reveal how partnerships respond to local conditions and needs and to track the complexities of implementation, as well as to triangulate findings and trace patterns of influence among various factors. Integrating the randomized trial data with in-depth case studies will increase the validity of causal inferences about the effects of the intervention.

The sample for the study comprises 41 schools located in the service areas of 15 local NWP sites across nine of the ten geographic regions formally established by the US Department of Education. The schools in the study are comparable with respect to their size, demographics, and student achievement levels. The sites represent different regions across the United States and reflect the diversity of sites in the NWP network with respect to experience and capacity.

SRI will produce annual progress reports and deliver presentations to the NWP. The Year 1 Report documents an initial survey of the 15 NWP sites and describes the data collection design. Reports for years 2–4 will provide interim analyses. The final report, to be written in 2011, will provide a comprehensive description of the evaluation questions, methodology, and findings.

The National Writing Project conducts many research studies of its professional development programs; some of these are done by the NWP Research Unit, some are done at local sites, and some, like this national evaluation, are conducted by outside independent firms.

Local site-based research conducted in twelve states shows significantly greater gains in writing achievement for students of teachers who have participated in NWP programs in comparison to students whose teachers have not. Previous national evaluations and other research studies have also found similarly positive results. Because of its broad scope and rigorous methodology, this study promises to provide the most conclusive evidence to date on the effectiveness of NWP professional development.