Partner Organizations:
Westat, Inc.

Business Education Compact

Texas Stars

Activities and findings:

Research and Education Activities: 

Science Work Experience Programs for Teachers (SWEPTs): SWEPTs provide
teachers with opportunities to participate in basic and applied
research in academic and industrial laboratories for periods ranging
from two weeks to two months during the summer months. Teachers
receive stipends during the period they are engaged in research.  Some
SWEPTs provide teachers with additional educational supports during
the school year (e.g.; consultations with scientists in academia or
industry as well as graduate students, school or classroom visits,
laboratory visits, kits and/or equipment for use in school classrooms
and laboratories.

SWEPTs are based on the premise that teachers who are experienced in
the practice of science and with its practitioners are better able to
communicate the concepts, vitality, and utility of science than
teachers who are not. Simply put, there are few basketball coaches who
have never played basketball, but only a minor proportion of science
teachers have had any experience with the practice of science and with
its practitioners. SWEPTs provide teachers with opportunities to
engage in the practice of science under the supervision and mentorship
of professional scientists. By this means, SWEPTs seek to improve K-
12 science education in the U.S.

Several reports indicate that teachers value their SWEPT experiences
and that participation in a SWEPT improves teacher retention (Frazier,
W.; Sloane K. & Young, J.; Weisbaum, K. & Huang, D.). However, except
for data reported informally by Columbia University's Summer Research
Program for Secondary School Science Teachers there have been no
studies that assess whether teacher participation in a SWEPT impacts
student interest or academic achievement.  This was the primary
research question addressed by this study. Namely, does teacher
participation in a SWEPT affect student achievement and attitudes in
science and mathematics?  This report provides evidence that teacher
participation in a SWEPT has a significant positive effect on student
achievement in science.

Organization of this Study: Recognizing the importance of this primary
research question, the managers of eight SWEPTs (Arkansas STRIVE,
Business Education Compact, Georgia Industrial Fellowships for
Teachers, IISME, INEEL, Science Education Partnership, Columbia
University's Summer Research Program for Science Teachers, Texas STARS
Program), joined together to organize and conduct the multi-site study
described in this report. They did so for three principal reasons.
First, to assemble the number of teachers and students required for
obtaining statistically significant data. Second, to bring together
educators and other personnel with the diverse array of viewpoints and
skills needed to design such a study and the instruments needed for
its conduct. Third, to obtain the economies of scale and the resources
needed to hire an organization capable of collecting and analyzing
these instruments. 

As planning proceeded, SWEPT managers realized that in the process of
determining whether teacher participation in a SWEPT affected student
interest and achievement in science, it would be necessary to obtain
data about the demographic characteristics, licensure, educational
background, teaching experience and practices, and attitudes toward
teaching of all teachers who enrolled in a SWEPT during the study
period. In addition, SWEPT managers thought it would be important to
survey both teachers and their mentors about what teachers did and
learned during their SWEPT placements, and about their satisfaction
with SWEPTs, and their evaluation of them as professional development
programs for teachers. Therefore, we surveyed all teachers about these
matters, as appropriate, at the beginning or end of their SWEPT
participation, and all mentors at the end of each teacher's residence
in the mentor's laboratory or workplace. Data obtained in some of
these surveys are directly relevant to the study's major focus - the
impact of teacher participation in a SWEPT on student interest and
achievement in science – and therefore are described in Section I of
this report. Data on teacher and mentor evaluations of SWEPTs, and on
matters unrelated to the study's primary focus, comprise Sections II
through V of this report.
 
To investigate the study's primary question - Does teacher
participation in a SWEPT affect student attitudes and achievement in
science and mathematics? – we utilized a quasi-experimental design in
which a subset of 59 of the 117 math and science teachers who
completed one of the participating SWEPTs in the summer of 1999 or
2000 were paired with Comparison teachers of the same subject in the
same high school who had not participated in a SWEPT. This subset
contained 25 mathematics and 34 science teachers. The subjects
targeted were regular algebra, geometry, biology and chemistry
courses. Teachers who had participated in a SWEPT in the preceding 5
years, taught in magnet schools, or taught only advanced placement
courses, remedial courses, or courses for ESL students were excluded
from the study. School administrators informed parents of students of
SWEPT and Comparison teachers of the plan to participate in the study,
and students whose parents objected to their child's participation
were exempted. In the event, no parental objections were reported.  

All students in targeted algebra, geometry, biology and chemistry
classes of SWEPT and Comparison teachers completed demographic
surveys, attitudinal surveys, and subject-specific cognitive tests at
the beginning of each academic year and attitudinal and
subject-specific cognitive tests at the end of the first, and in some
cases, the second year SWEPT teachers participated in this study.
Surveys and test instruments were encoded to assure students
anonymity. 

Research Questions Addressed by the Study:

The study addressed four classes of research questions. 
•	First, did teacher participation in a SWEPT have a positive impact
on student interest and achievement in science? Results from the
quasi-experimental portion of the study (Section I of this report),
show that it did. 
•	Second, are there any characteristics that distinguish teachers who
elect to participate in a SWEPT? Results from teacher surveys and
interviews (Section II of this report), indicate that there were none.

•	Third, did SWEPTs provide participating teachers with experiences
that might be expected to affect their classroom and teaching
practices? Results from teacher surveys (Section III of this report),
show that they did. 
•	Fourth, what did mentors report about their SWEPT experiences?
Results from Mentor surveys (Section IV of this report), showed that
mentors felt the experience was valuable for teachers and themselves
and would participate in future years. 
•	Finally, two unanticipated findings, described in Section V, emerged
from this analysis. The first relates to differences in teaching
practices of mathematics and science teachers. The second, relates to
the frequency with which students are assigned to a different science
or mathematics teachers over the course of a year-long mathematics or
science course.

REFERENCES

Summer Research Program for Secondary School Science Teachers, Annual
Reports 1998-2002, Columbia University, New York, NY

Frazier, Wendy M., The Effect of a Science Work Experience Program for
Teachers on the Classroom Environment: A Qualitative Program
Evaluation, Teachers College, Columbia University, New York, NY 2001

Sloane, K., Young, J., Evaluation of Scientific Work Experience
Programs for Teachers: Current Practice and Future Directions, IISME,
Santa Clara, CA 1996

Weisbaum, K, Huang, D., IISME Retention and Program Impact Evaluation
1985-2000

Findings:

SECTION I. (All data in this section reflect findings for the 59 SWEPT
and 65 Comparison teachers and the more than 4,000 students enrolled
in their classes). 

Research Question #1. Was there a significant positive correlation
between student performance on cognitive tests and their assignment to
a class taught by a SWEPT teacher?

•	Students in year-long biology and chemistry classes of SWEPT
teachers in the second year following their teachers' SWEPT
participation showed significantly (p>0.05) greater cognitive gains in
biology and chemistry than students in biology classes of Comparison
teachers.
•	In contrast, students in year-long algebra, geometry, biology, and
chemistry classes of SWEPT teachers in the first year following their
teachers' SWEPT participation, and students in year-long algebra and
geometry classes of SWEPT teachers in the second year following their
teachers' SWEPT participation, showed cognitive gains in these
subjects that were comparable to those of students in classes of
Comparison teachers. 

The finding that improved student cognitive performance in a science
taught by a SWEPT teacher did not occur until the second year
following teacher SWEPT participation is consistent with reports that
it often takes more than one academic year for teachers to implement
successfully newly acquired educational practices. It also is
consistent with data obtained by Columbia University's Summer Research
Program for Secondary School Science Teachers, which show that in the
second and third years following teacher entry into Columbia's
program, approximately one-third more students in classes of these
teachers pass a New York State Regents examination in science than
students in classes of non-participating teachers in the same schools.


Conclusion: Science teacher participation in a SWEPT had a significant
(p <0.05) positive impact on student achievement in biology and
chemistry as measured by student gains on pre-post course standardized
tests in these subjects in the second year following a teacher's
completion of a SWEPT.   

Were SWEPT and Comparison teachers, and the students assigned to their
classes, comparable at the beginning of the study? 

The findings reported above that students in classes of SWEPT biology
and chemistry teachers showed significantly greater gains on both the
standardized subject-matter achievement tests administered in the
present study, and on the New York State Regents examination, provide
objective evidence that teacher participation in a SWEPT has a
positive impact on student achievement. These findings have important
implications for educators and policy makers, assuming fulfillment of
the following conditions: 

•	First, if the students were comparable in terms of socio-economic
status, educational expectations, motivation, interest in science or
mathematics, and in cognitive ability at the time they were assigned
to classes of SWEPT and Comparison teachers.

•	Second, if SWEPT and Comparison teachers were comparable in
educational background, teaching experience, and educational practices
at the time they entered the study. 

As the data reported below show, all of these conditions were
fulfilled. 

Comparability students of SWEPT and Comparison teachers in cognitive
ability in science and mathematics at the beginning of the study. 

It was possible that parents requested, and/or that schools assigned,
better-prepared and/or more able students to classes of Study or
Comparison teachers. To control for bias of this sort we compared
pre-course cognitive test scores of students in classes of SWEPT and
Comparison teachers. There were no differences in cognitive
performance on these pre-course tests of students of SWEPT and
Comparison teachers, ruling out the possibility of selective
assignment of better-prepared or more able students to classes of
SWEPT or Comparison teachers.       

Conclusion: Students in classes of SWEPT and Comparison teachers were
comparable in terms of gender, home learning resources, parents'
education, and parents' and student educational expectations at the
beginning of the study.

Comparability of students of SWEPT and Comparison teachers in
attitudes about science and mathematics at the time they began a class
taught by a SWEPT or comparison teacher. It was possible that schools
assigned students to classes of SWEPT or Comparison teachers based on
gender or on parents' or student educational expectations. Demographic
surveys administered to students at the beginning of each academic
year showed there were no significant initial differences in gender,
home learning resources, parents' education, or parents' and student
educational expectations of students in classes of SWEPT and
Comparison teachers. 

Conclusion: Students of SWEPT and Comparison teachers were comparable
in attitudes about science and mathematics at the beginning of the
study. 

It was possible that schools assigned the more confident or interested
students to classes of SWEPT or Comparison teachers. We compared
attitudes of students in science and mathematics classes of SWEPT and
Comparison teachers on three scales (i.e., Belief in Own
Science/Mathematics Ability scale, Interest and Involvement in
Science/Mathematics scale, Importance and Usefulness of
Science/Mathematics scale) constructed using students' responses to
survey instruments administered at the beginning of each school year.


Conclusion: There were no significant differences between students in
mathematics classes of SWEPT vs. Comparison teachers on any scale, and
no significant differences between students in science classes of
SWEPT vs. Comparison teachers on the Interest and Involvement in
Science, and the Importance and Usefulness of Science scales. Students
in classes of Comparison teachers scored significantly (p<0.05) higher
on the Belief in Own Science Ability than students in classes of SWEPT
teachers.  

Comparability in gender, licensure, education, teaching practices,
attitudes toward teaching, and school responsibilities of the 59 SWEPT
teachers who participated in the quasi-experimental protion of the
study and the 58 SWEPT teachers who did not.

It was possible that SWEPT managers selected a subset of the most
highly qualified SWEPT teachers to participate in the
quasi-experimental portion of the study. 

Conclusion: There were no significant differences in gender,
licensure, educational background, teaching practices, attitudes
toward teaching, and school responsibilities of the 59 SWEPT teachers
who participated in the quasi-experimental portion of the study and
the 58 SWEPT teachers who did not.     

Comparability of SWEPT and Comparison teachers in gender, licensure,
education, and teaching practices at the beginning of the study. 

It was possible that there were pre-existing differences between SWEPT
and Comparison teachers that would complicate subsequent comparisons
of impact on student attitudes and achievement. 

Conclusion: Comparison of the two groups showed no significant
differences by gender, licensure, level of educational attainment, or
teaching practices. SWEPT teachers of mathematics and science had 2
and 4.6 fewer years of total teaching experience, respectively, than
Comparison teachers of mathematics and science, and fewer course
credit hours beyond their highest degree than Comparison teachers.
These differences in teaching experience of SWEPT and Comparison
teachers notwithstanding, 83% of SWEPT teachers and 84% of Comparison
teachers were experienced teachers with 4 or more years of teaching
experience. 

Comparability of SWEPT and Comparison teachers in attitudes toward
teaching, teaching objectives and strategies, and classroom practices
at the beginning of the study. 

Surveys administered to SWEPT teachers at the time they entered a
SWEPT, and to Comparison teachers at the beginning of the academic
year, included a number of questions about teachers' attitudes toward
teaching, teaching objectives and strategies, and classroom practices.
We derived six scales from these questions (i.e., Traditional Goals
and Objectives, Inquiry-based Goals and Objectives, Traditional
Teaching Methods, Traditional Student Activities, and Inquiry-based
Student Activities scales), and used them to determine if there was
any systematic difference between SWEPT and Comparison teachers at the
beginning of the evaluation period. Scale scores were available for 58
of the 59 pairs of SWEPT and Comparison teachers, and all of the
scales represent standardized factor scores with a mean of 250 and a
standard deviation of 50. Using this metric, none of the mean scores
differed significantly, at the p< 0.05 level. 

Conclusion: Aside from a difference in years of mathematics and
science teaching experience, there were no important differences in
the backgrounds, teaching practices, or attitudes toward teaching of
Study and Comparison teachers that might be expected to result in
subsequent variations in student attitudes or achievement in science
or mathematics. 


Research Question #2. Were there significant differences in attitudes
toward science or mathematics of students at the time of their
assignment to a class taught by a Study (SWEPT) vs. Comparison
teachers?
 
Surveys administered to students at the beginning and end of a
year-long mathematics course (algebra or geometry) or science course
(biology or chemistry) taught by a SWEPT or Comparison teacher
included questions about the students' attitudes toward mathematics or
science. We derived three scales from these questions (i.e., Belief in
Own Math or Science Ability, Interest and Involvement in Math or
Science, Importance and Uses of Math or Science), and used them to
determine if there were any systematic differences between attitudes
of students in classes of SWEPT and Comparison teachers at the
beginning and end of a year-long mathematics or science course. All of
the scales use standardized factor scores with a mean of 250 and a
standard deviation of 50. 

With respect to students' attitudes about mathematics: There were
significantly (p <0.05) smaller declines in the pre-post-course
'Interest and Involvement in Math' and the 'Importance and Uses of
Math' scales among students in classes of SWEPT teachers in the first
year following the teachers' SWEPT participation than among students
in classes of Comparison teachers. Students in mathematics classes of
SWEPT teachers in the second year following the teachers' SWEPT
participation also showed a smaller decline in the 'Interest and
Involvement in Math' scale than students in mathematics classes of
Comparison teachers, but there were many fewer teachers and students
in this second year cohort and this difference was not significant at
the p<0.05 level.   

With respect to students' attitudes about science: Student attitudes
toward science remained relatively constant from the beginning to the
end of the school year, and this pattern was essentially the same for
the students of SWEPT and Comparison teachers.

Conclusions: Students in mathematics classes of SWEPT participants
showed smaller declines in attitudes about mathematics than students
in classes of Comparison teachers, as measured by pre-post course
attitudinal surveys.  These differences were observed in both the
first and second years following teacher SWEPT participation but were
statistically significant at the p <0.05 level only in the first year.
 In contrast, similar surveys showed student attitudes about science
remained relatively constant throughout the school year, and there
were no significant differences in attitudes of students in science
classes of SWEPT vs. Comparison teachers.  

Research Question #3. Did SWEPT participation affect teachers'
attitudes and teaching practices?  

1. Study teachers reported a very small increase in use of traditional
teaching methods in their classrooms while Comparison teachers
reported a decline in the use of traditional teaching methods. The
combination of the two patterns of change produced a net change of
about 13 points, which was significant at the p <0.05 level.

2. Study teachers reported a small increase in the adoption of
inquiry-based goals and objectives for their teaching during the first
year after their SWEPT experience, while Comparison teachers reported
a large decline in the use of inquiry-based goals and objectives
during the same time period. The effect of the two patterns produced a
net difference of approximately 32 points, which was significant at
the p <0.05 level. 
 
3. Study teachers reported a 10-point gain in the use of inquiry-based
student activities in their classrooms during the first year after
SWEPT participation. During the same period, Comparison teachers
reported a 14-point decline in the use of these methods, producing a
net change of 24 points, which was significant at the p <0.05 level. 

4. The End-of-Course surveys of students in classes of Study and
Comparison teachers corroborated the teachers' self-assessments.
Students in classes of Study teachers were significantly more likely
to report engaging in four constructivist practices encouraged by
SWEPTs (i.e., 'reflecting on course material by writing in a
notebook,' 'using primary sources such as journals,' 'listening to
guest speakers or going on field trips,' 'exploring career
opportunities in math or technology,' and being encouraged by their
teacher to consider a career in math or science') than students of
Comparison teachers.  

Conclusion: In the academic year immediately following their SWEPT
experiences, Study teachers undertake more inquiry-based
constructivist educational practices than Comparison teachers. Thus,
participation in a SWEPT significantly affects teachers' teaching
practices. 

Research Question #4. Were there significant differences in the
evolution of attitudes about science or mathematics of students in
classes of SWEPT vs Comparison teachers over the course of an academic
year?
 
Student attitudes about mathematics: There were significantly (p
<0.05) smaller declines in the pre-post-course 'Interest and
Involvement in Math' and the 'Importance and Uses of Math' scales
among students in classes of SWEPT teachers in the first year
following the teachers' SWEPT participation than among students in
classes of Comparison teachers. Students in mathematics classes of
SWEPT teachers in the second year following the teachers' SWEPT
participation also showed a smaller decline in the 'Interest and
Involvement in Math' scale than students in mathematics classes of
Comparison teachers, but there were many fewer teachers and students
in this second year cohort and this difference was not significant at
the p<0.05 level.   

Student attitudes about science: While students in classes of SWEPT
teachers reported significantly more exposure to inquiry-based
constructivist practices than students in classes of Comparison
teachers, student attitudes toward science remained relatively
constant from the beginning to the end of the school year. This
pattern was essentially the same for the students of SWEPT and
Comparison teachers. Consequently, there were no significant
differences in attitudes about science of students in classes of SWEPT
vs. Comparison teachers. 

Conclusion: Students in mathematics classes of SWEPT participants
showed smaller declines in attitudes about mathematics than students
in classes of Comparison teachers, as measured by pre-post course
attitudinal surveys.  These differences were observed in both the
first and second years following teacher SWEPT participation but were
statistically significant at the p <0.05 level only in the first year.
 In contrast, similar surveys showed student attitudes about science
remained relatively constant throughout the school year, and there
were no significant differences in attitudes of students in science
classes of SWEPT vs. Comparison teachers.  


Sections II - V. (Data reported in these sections are derived from
post-SWEPT program surveys administered to the 117 mathematics and
science teachers who completed one of the SWEPT programs participating
in this study in the summer of 1999 or 2000, and from their mentors.
Survey response rates were ~95%.  Many survey questions used a four
point Likert-type scale in which 'strongly agree,' 'great extent,'
'very successful,' and 'excellent' were the highest ratings.)

SECTION II
Research Question #1: 
Do teachers who elect to participate in a SWEPT exhibit any
characteristics in terms of gender, race, licensure, educational
background, teaching experience, school responsibilities or time
invested in preparing for teaching that distinguish them from other
teachers? 

A. Teachers who participate in SWEPTs are in general experienced
teachers (i.e., average 7.8 years of teaching experience).  
B. The demographic characteristics, licensure, educational background,
attitudes toward teaching, teaching practices, and school
responsibilities of mathematics and science teachers who entered one
of the participating SWEPTs in the summers of 1999 and 2000 are
generally similar to those reported for U.S. high school teachers, and
for other teachers (Comparison teachers, see Section I) teaching the
same subjects in their schools.  

Conclusion: Teachers who elect to participate in a SWEPT do not
exhibit any distinguishing characteristics in terms of gender, race,
licensure, educational background, teaching experience, attitudes
toward teaching, time invested in preparing for teaching, or school
responsibilities from other mathematics and science teachers in their
schools.   

SECTION III

Overall research question: Did SWEPTs provide participating teachers
with experiences that might be expected to affect their classroom and
teaching practices?
 
Research Question 1. What activities did SWEPT teachers participate in
during the SWEPT? 

Sixty-four to 79% of teachers reported that to a 'great' or 'moderate
extent' they operated instruments, observed research activities of
others, read academic literature and journals, used the Internet in
their work, wrote about their work, shared their reports with others
at their work sites, and gave an oral presentation on their work. 
Conclusion: SWEPTs succeed in their primary goal of providing teachers
with opportunities to gain hands-on experience in the practice of
basic or applied science. 
 
Research Question 2. 
What did teachers report learning from their SWEPT experiences?

About half of the teachers reported that to a 'great extent' they
learned about applications of science, mathematics and technology in
everyday life and about current issues in science or mathematics
research, and gained familiarity with new materials and equipment to
use in teaching.

Conclusion: Teachers gain new insights about the practical
applications of science, mathematics, and technology; and become
familiar with new materials and equipment for use in their classrooms
through their SWEPT experiences.
   
Research Question 3. Did teachers believe their SWEPT experiences
affected their efficacy as teachers and self-learners, their interest
in research and/or the applications of science, mathematics and
technology, and/or their professional development as teachers? Over
95% of teachers 'strongly agreed' or 'agreed' that SWEPT participation
contributed positively to their efficacy as teachers and
self-learners, stimulated their interest in research and its
applications, and increased their motivation to participate in
professional development programs.

Conclusion: Nearly all teachers felt their SWEPT experiences increased
their efficacy as teachers and self-learners, elevated their interest
in science/mathematics/technology, and made a positive contribution to
their professional development as teachers.
  
Research Question 4. Did teachers feel SWEPTs were responsive to their
professional development needs and appropriate to their knowledge,
skills and interests? 

Sixty-one percent of teachers rated their SWEPT experience as 'very
successful' with respect to their professional development needs, and
65% found it 'very successful' with respect their knowledge, skills
and interests.

Conclusion: Nearly two-thirds of teachers reported that SWEPTs were
'very successful' in responding to their professional development
needs and appropriate to their knowledge, skills and interests.  

Research Question 5. Were teachers satisfied with their work
placements, interactions with work-site staff, and the availability
and quality of materials and equipment at their work sites? 

Sixty-five percent of teachers rated their work placements 'very
successful,' 63-73% of teachers felt 'very satisfied' with their
interactions with work site staff, and 69% were 'very satisfied' with
the quality of material and equipment at their work sites. 

Conclusion: Two-thirds to three-quarters of teachers were 'very
satisfied' (the highest ranking with their work placements,
interactions with work-site staff, and the availability and quality of
materials and equipment at their work sites.

Research Question 6.  Were SWEPTs successful from the perspective of
teachers and their Mentors? 

Over 90% of teachers 'strongly agreed'/'agreed' that their SWEPT
experiences positively affected their efficacy as teachers and
self-learners, that SWEPTs were responsive to their professional
development needs, and that they would recommend SWEPTs to their
colleagues. Similarly, 95% of mentors 'strongly agreed'/'agreed' that
SWEPTs are worthwhile for teachers and 83% of mentors said they would
participate again. 

Conclusion: SWEPTs succeed from the standpoint of both teachers and
mentors. 
 
Research Question 7. What did teachers report about their interactions
with their mentors? 

Sixty-nine percent of teachers were 'very satisfied' with their
relationships with their mentors, and 68% rated their mentor's
commitment to providing opportunities to learn and gain new expertise
as 'excellent.'

Conclusion: More than two-thirds of teachers were 'very satisfied'
with their mentors.
  
Research Question 8. Were teachers satisfied with the materials and
information they received prior to beginning their work experiences,
with their SWEPT orientation, and with their mentor's preparation for
their arrival? 

While most teachers were at least 'moderately satisfied' with their
SWEPT and mentor's performance in these areas, only 47% of teachers
reported being 'very satisfied' with materials received prior to
beginning their work experiences, only 53% reported being 'very
satisfied' with their SWEPT orientation, and only 49% rated their
mentor's preparation for their arrival 'excellent.'   

Conclusion: Teachers found the materials provided prior to beginning
their work experiences, their SWEPT orientation, and their mentor's
preparation for their arrival as the least successful aspects of the
SWEPT programs. Teacher reports of   lack of mentor preparation for
their arrival was confirmed in the mentor surveys (see Point 1,
Section IV, below). 

SUMMARY OF CONCLUSIONS DRAWN FROM DATA REPORTED IN  SECTION III. 
Teachers participated in all activities and aspects of research at
their assigned laboratories and worksites. Specifically, they obtained
hands-on experience in designing and performing research, using
equipment and information technology, reporting on research, and
working as a member of a research team. These are the types of
experiences that might be expected to affect their classroom and
teaching practices. Indeed, as described in Sections I and III of this
report, data collected in this study showed that in the year following
their SWEPT participation teachers increased use of constructivist
practices in their classes.    

SECTION IV.  What did mentors report about their SWEPT experiences?

Research Question 1. What did mentors report about their preparations
for a teacher's arrival and about their interactions with teachers
during the summer?  

Thirty-eight percent of mentors had not met their assigned teacher
prior to the teacher's arrival at their laboratory or work-site, 20%
did not provide any preparatory material prior to the teacher's
arrival, and 45% spent 0 -1 hr per week meeting with their assigned
teacher. 

Conclusion: A significant proportion of mentors make no effort to meet
teachers or to provide them with preparatory materials prior to the
teacher's arrival in the mentor's laboratory or work site. The finding
that nearly half of the mentors spent an hour or less per week with
their assigned teacher is difficult to reconcile with the high level
of teacher satisfaction with their mentors (see point 7 above). 

Research Question 2.What did mentors report about
their assigned teacher? 

Ninety-three percent of Mentors reported that to a 'great' (61 – 65%)
or 'moderate' (27 – 33%) extent their assigned teacher performed well
in their laboratory or work place (i.e., adjusted to work challenges,
was able to apply new skills to work, became proficient with materials
and technology, and finished assignments in the expected time frame),
84% reported the teacher contributed positively at staff meetings and
made a positive contribution to the organization, and 93% would
recommend the teacher for another fellowship.  

Conclusion: Teachers adapt quickly and perform at a high level in
their assigned research settings. In general, they meet their mentor's
expectations.  

Research Question 3. What did mentors report about the value of SWEPT
for teachers and for mentors? 

Ninety-five percent of mentors 'strongly agreed' (57%), or 'agreed'
(38%), that 'SWEPTs are worthwhile for teachers,' 86% 'strongly
agreed' (44%), or 'agreed' (42%), that 'SWEPTs are worthwhile for
mentors,' and 83% indicated they 'would participate again.'

Conclusion: Most mentors who accept a teacher into their laboratory or
work site were sufficiently impressed by their assigned teacher and
their experiences with  SWEPTs to participate again in the program. 

SECTION V.  Unanticipated Findings.

1. Differences in teaching practices of mathematics and science
teachers. 

Our surveys showed that science and mathematics teachers differ
significantly in their teaching practices. Science teachers report
covering 62% of the material in their assigned textbook while math
teachers report covering 77%.  Thirty-six percent of science teachers
report using the textbook publishers tests 'sometimes' or 'frequently'
vs. 72% for mathematics teachers. Science teachers report spending an
average of 11.2 hrs per week outside of school preparing for class vs.
9.2 hrs per week for math teachers. Almost twice as many math teachers
as science teachers reported formally mentoring beginning teachers,
while nearly twice as many science teachers as math teachers reported
making presentations to school boards, parents or community groups and
conducting research on a topic of interest to themselves.

Conclusion:  Science and mathematics teachers differ significantly in
their teaching practices.  We suspect that the work placements of many
mathematics teachers did not result in their engagement in work that
challenged and/or extended their mathematics skills.  Further analysis
of the responses of mathematics teachers and of their mentors to
post-SWEPT surveys are needed to assess the appropriateness of their
SWEPT work assignments.

2.  One-third of high school students taking algebra, geometry,
biology or chemistry were in a class taught by a different teacher
every trimester or semester.  Teachers have different teaching styles.
Students have different learning styles. It may take several weeks for
students to accommodate to the expectations and teaching practices of
a new   teacher. This is especially the case in the first year of high
school, where students must adjust to a new school and a new level of
expectations. Difficult as these adjustments may be for students, they
are even greater for teachers. Most high school science and
mathematics teachers teach three classes, each with 30-35 students.
Each student has a slightly different learning style. Many students
need remedial help to overcome deficiencies in their preparation for
high school algebra, geometry, biology or chemistry. And there are
always a few students who have very unique social or educational
problems. Teachers need time to assess the best ways to approach each
student. As Darling-Hammond and Falk (Phi Delta Kappan 1997) report,
'….students experience much greater success in school settings that
are structured to create close, sustained relationships between
students and teachers….Schools that personalize instruction by keeping
the same teachers with the same students for extended periods of time
have fewer behavior problems and higher achievement than schools that
fragment student learning with highly departmentalized structures in
which students move continuously from one teacher to another, never
becoming very well known by any.' Darling-Hammond and Falk cite
research showing that '….in high-achieving countries such as Japan,
Germany, Sweden, and Switzerland teachers often stay with the same
students for two or more years….Thus their teaching is informed by
greater knowledge of the students and how they learn.' 

Conclusion: One-third of all high school students taking algebra,
geometry, biology and chemistry have a different teacher every
trimester or semester. This is a pedagogically unsound practice that
surely contributes to the difficulties many students encounter in high
school mathematics and science courses.

Training and Development:

1. SWEPT managers gained significant experience in the design,
organization and administration of an evaluation of their programs and
of their impacts on teachers and students. 

2. SWEPT managers gained significant experience in gaining consent of
school administrators, and cooperation of non-SWEPT teachers, in
performing evaluation studies in local schools.

3. SWEPT personnel were specifically trained in maintenance of
Sampling Logs to monitor problems encountered with the Treatment and
Comparison groups. These procedures were repeated for the two new
programs (Arkansas STRIVE and Texas STARS) that joined the consortium
after the study began.

4. SWEPT managers and lead personnel learned about the organization,
operation, and capabilities of other U.S. SWEPTs.

Outreach Activities:

In addition to the sweptstudy.org website mentioned elsewhere in this
report, the P.I. (Dr. Samuel Silverstein), Project Coordinator (Mr.
Jay Dubner) and lead Westat partner (Dr. Joy Frechtling) have
presented the methodology and interim findings at various professional
conferences. 

Even though the grant award has ended, Dr. Silverstein and Mr. Dubner
continue to report the findings and disseminate information about the
availability of the instruments developed and used for the study.

Other Specific Products:

Instruments or equipment developed Teacher, student and mentor Surveys have been developed. The Survey covers the following areas: • Instructional practices: Integration across subject areas • Use of technology to support learning • Engagement of students in hands-on/minds-on activities • Use of cooperative learning in a judicious combination with independent work • In-depth study of a limited number of topics rather than limited coverage of a wide range of topics • Contextualization of learning in the experiences of daily life; use of community resources, mentors from the community • Use of 'real world' problems as the basis for research, analysis and interpretation • Integration of the learning of terms and concepts • Combination of coaching and student-led discussion with lecture and teacher-directed activities Teacher leadership • Participation in professional meetings • Professional development involvement, both leading and not leading to additional academic degrees • Assumption of new roles in the school or district, such as serving on curriculum committees, being a teacher-mentor or master teacher • Disseminating knowledge gained through professional development activities through workshops, demonstration lessons, observation/coaching of others • Providing material for parents that reflect the outcomes of the new skills and/or practices • Confidence in content knowledge When the study was completed, many teacher professional development programs found these instruments useful in conducting assessments of their programs. We are certain that NSF funded RET sites have used the pre- and post-program and mentor surveys. Software (or netware) A new website was developed providing those interested in the study with the latest information. The website can be found at: www.scienceteacherprogram.org/sweptstudy 1. The website is available on all major search engines. 2. Those inquiring about the study are directed to the website. 3. Joy Frechtling (Westat) and Jay Dubner (Columbia U.) have given talks at national conferences. The website's address is included in the talks.

Internet Dissemination:

http://www.sweptstudy.org

This site contains the current project information. Included on the
site are lists of the partners, advisory board members and instruments
used during the data collection process.

It is our understanding that NSF supported RET sites are now using the
pre- and post-program teacher surveys and the mentor survey developed
for this study and available on the site's website.

Contributions:

Contributions within Discipline:

 Contributions within the discipline of secondary school science
education:

This project has made five specific contributions to knowledge and
understanding of secondary school science education.

1. It provides the first demonstration that teacher participation in a
SWEPT has a significant (p <0.05) positive impact on student
achievement in biology and chemistry.  This finding is buttressed by
data obtained by Columbia University's Summer Research Program for
Secondary School Science Teachers that one-third more students of
teachers who have completed Columbia's program pass a New York State
Regents exam in science than students in classes of other teachers in
the same schools and science departments.

2.  It provides the first systematic evidence that teachers value
their SWEPT experiences, and that teacher participation in a SWEPT has
a significant effect (p<0.05) on their attitudes toward teaching and
their teaching practices.  

These results indicate that SWEPTs provide a complementary form of
professional development for science teachers that has a positive
impact on student achievement in science.  

3.  It provides the first evidence that science and mathematics
teachers who participate in a SWEPT are experienced teachers who are
no different in demographic characteristics, licensure, educational
background, attitudes toward teaching, teaching practices, and other
school activities than other science and mathematics teachers in the
same schools.  

4. It indicates that SWEPTs should improve their practices with
respect to introducing teachers to their mentors prior to their
arrival at the mentor's laboratory or workplace,  providing background
materials that help prepare teachers for their laboratory or workplace
experience, and in orienting teachers when they first arrive to begin
a SWEPT program. 

5. The study has created and validated the following instruments which
are have been available on the study's website (www.SweptStudy.org)
for use by others since 2002, and have been used in modified form by
the RET Network Working Committee since 2004.

A.  Pre-SWEPT teacher survey and Pre-teaching Comparison teacher
survey that provide information on demographic characteristics of
teachers, licensure, educational background, teaching experience,
teaching practices, and school associated activities. 

B. A Post-SWEPT teacher survey that assess how teachers feel about
their SWEPT experiences, and what they learned during them.

C. A Post-teaching SWEPT and Comparison teacher survey that assesses
teachers attitudes about teaching at the end of a targeted course.

D. A mentor's survey that provides information about SWEPT teacher
performance in their mentor's laboratory or workplace, and can be used
to assess each mentor's opinion about SWEPT programs. 

E. A pre-course student demographic survey that can be used to assess
student home environment, parental educational level, and student and
parental educational expectations.  

F. A pre- and post-course student attitude survey that can be used to
assess changes in student attitudes about science or mathematics from
the time students begin a science or mathematics course until they
complete it.  The post-course student survey also contains questions
about classroom activities initiated by the student's teacher.  

G. Cognitive pre-course post-course tests to assess student
achievement in algebra, geometry, biology and chemistry.

 The study's results demonstrates the value of SWEPTs as an important
professional development program. With the No Child Left Behind Act's
(NCLB) reliance on standardized tests, SWEPTs help schools achieve
those goals.

 1.	For students: Passing standardized tests is essential for student
achievement and graduation from high school. By measuring student
achievement in science, SWEPTs increase the likelihood they will take
additional science courses, graduate from high school, and succeed in
post-secondary education.
2.	For teachers: SWEPTs enlarge teacher contacts with and knowledge of
educational resources in their own communities and in the larger
world.
3.	For mentors: SWEPTs increase awareness, interest, and engagement in
local schools of scientists and executives in not-for-profit and
for-profit organizations.