SWEPT Multi-site Student Outcomes Study
Study Genesis
Participating School Demographics
Sloane-Young Report (PDF 147K)
Genesis of the SWEPT Multi-site
Student Outcomes Study
Program Managers of Science Work Experience Programs for Teachers (SWEPTs) have long recognized the importance of determining whether teacher participation in these professional development programs affects student outcomes. However, efforts to obtain such data have heretofore proved unsuccessful. This is not surprising since statewide 1, 2 and national studies 3 have documented the difficulties of identifying relationships between teacher quality and student performance. Nonetheless, from its' inception, Columbia University's Summer Research Program for Science Teachers sought ways to evaluate the effects of teacher participation on the performance of students.
Like most SWEPTs, Columbia's Program encourages constructivist approaches to teaching and learning. The effects of these approaches on students may not be captured by standardized tests. However, the changes in teaching attitudes and styles fostered by these programs may affect student attendance, interest in science, graduation, and/or pursuit of a college education.
To examine these questions, Columbia's Program, with assistance from the New York City Board of Education, obtained data on the achievement of students in the classes of participating public high school teachers for the year preceding each teacher's entry into the Program, and subsequent to the teacher's involvement with it. Performance of students who have studied with the Program's teachers is compared with performance of students in the science classes of other teachers in the same schools during the same years.
At the 3rd national Scientific Work Experience Program for Teachers conference (March 1996), Jay Dubner, Program Manager of Columbia's SWEPT, described the design of his Program's student outcomes study. The talk led to the Industry Initiatives for Science and Math Education (IISME) program obtaining funds from the Intel Foundation to study the feasibility of conducting a multi-site SWEPT student outcomes study. Seven SWEPT managers and three experts on SWEPT programs, met in Bethesda, MD, in October 1996. The congruence of program goals and activities encouraged these seven SWEPTs (Columbia's Summer Research Program, the Bay Area's IISME, Oregon's Business Education Compact, Georgia's Project GIFT, Texas Alliance for Science and Math Education; Ohio's Alliance for Education Project, and Washington's Science Education Partnership) to develop a plan for a multi-site student outcomes evaluation project. In 1997 these seven SWEPTs received a National Science Foundation (NSF) Special Grant for Exploratory Research to develop plans and instruments for a multi-site SWEPT student outcomes study, naming Dr. Samuel Silverstein, Director of Columbia's SWEPT, as Principal Investigator. The grant enabled the leaders of these SWEPTs to meet to plan such a study. These discussions indicated that to obtain statistically significant results, such a study would require a slightly larger number of teachers than that encompassed by the seven participating SWEPTs. Therefore, the group invited the participation of an eighth SWEPT, the Idaho National Engineering and Environmental Laboratory (INEEL). The eight SWEPTs developed a full proposal which outlined the rationale for a study of the effects of teacher participation in a SWEPT on student performance and detailed the way the eight cooperating SWEPTs proposed to organize, implement, analyze and disseminate the results of the study. Westat, Inc., a firm with extensive experience in teacher evaluation, was contracted to develop the study's instruments, provide guidance to the SWEPTs in administration of the instruments, analyze the data and produce the final report.
The full proposal received favorable reviews, and as a result, NSF agreed to fund the project for four years at a total cost of $1.6 million. Dr. Silverstein is the Principal Investigator, and Mr. Dubner its Project Coordinator. Dr. Silverstein plans to seek support from non-governmental foundations so a fifth year of the study can be implemented. The Directors and Managers of the eight participating SWEPTs met in December 1998 and April 1999 to initiate the project. At that time, The Texas Alliance for Science and Math Education program dropped out of the study because they felt they were unable to provide teachers that fit the study's criteria. The study's Advisory Board met in March 1999 to discuss the instruments designed for the study. Instrument administration began in June 1999 with pre-program surveys administered to the teachers first entering the SWEPTs in the summer of 1999 (now known as Cohort I). Post-program teacher surveys were administered to the same teachers in August 1999. Student instruments and comparison teacher surveys were administered beginning September 1999. In June 2000 the Arkansas STRIVE Program joined the study and contributed to Cohort II. Ohio's Alliance for Education discontinued their participation in the study as of September 2000 due to changes in their programmatic goals. Additional funding is being sought to support data collection and analysis for Cohort III. The Texas STARs program has agreed to participate in Cohort III. They have already administered surveys to the teachers first participating in their program in June 2001.
On April 18, 2001 Mr. Dubner and Dr. Frechtling presented the study's design and latest findings at the Materials Research Society's Director's meeting in San Francisco, CA. Following the meeting, the study's partners submitted a manuscript of the meeting presentation. The manuscript is available in PDF format (112K). The study concluded in 2002.
Award # 9812142
____________________________________
1D. Armor, P. Coonry-Oseguera, M. Cox, N. King, L. McDonnell, A. Pascal, E. Pauly & G. Zellman, Analysis of the School Preferred Reading Program in Selected Los Angeles Minority Schools, Rand Corporation Report No. R-2007-LAUSD, 1976
2R.F. Furgeson, Harvard Journal on Legislation, Vol. 28:465, 1991.
3J. Coleman, Equality of Educational Opportunity, 1966; F. Mosteller & D.P. Moynihan, On Equality of Educational Opportunity, 1972; R. Murnane, Impact of School Resources on the Learning of Inner City Children, 1975.