Practice Activity: Leading a Problem Clarification Process
.
.
Needs Assessment for Montgomery Elementary School

Data Analysis

Process

At the February staff meeting we began to examine the results from the 2001 MSPAP administered to our third and fifth grade students last May. Grade level teams were assigned a number of graphs to examine and discuss using the attached worksheets. The teams brought back their findings to the entire staff at the March staff meeting. Parents examined and discussed the same data during the PTA Board meeting in March.

Findings

Our area of need continues to be reading with only 31% of our 5th grade students and 38. 8% of our third grade students scoring satisfactory. The only area lower in grade 3 and 5 was social studies, which is not a priority area for our district. Writing is also an area of need in 5th grade with only 32. 9 % of students scoring satisfactory in writing. In grade 5 over 39% of the students scored satisfactory in language usage, math and science and in grade 3 over 40% of the students scored satisfactory in writing, language usage, mathematics and science.

When we examined progress in reading across time, 5th grade scores have remained essentially flat over the last nine years with the exception of 1999 when scores showed progress beyond chance when compared to 1997 and 1998. Third grade scores have also remained flat except for a decline beyond chance from 1998 to 1999.

When we examined disaggregated data on reading, we found the following disparities:

  • Girls outperformed boys in grade 3 by a little more than 1%
  • Boys outperformed girls in grade 5 by 10. 9%
  • In grades 3 and 5 Hispanic and African American students lagged behind White and Asian students. In grade 3, 53. 8% of Asian students, 47. 1% White, 37. 1% African American, and 25% Hispanic students scored satisfactory on reading. In grade 5, 69. 2% of White students, 44. 4% Asian, 25% Hispanic, and 14. 3% African American students scored satisfactory on reading.
  • Students participating in the Free and Reduced Meals Program performed at a lower level than students not participating in FARMS. On 19% or FARMS 5th graders scored satisfactory as compared to 48. 3% of non-Farms 5th graders. In grade 3, 30% of FARMS students and 46. 7% of non-FARMS students scored satisfactory.
  • Fifth graders in special education outperformed regular education students by 2. 7% whereas third graders in regular education outperformed special education students by 36. 7%. There were only 9 fifth grade special education students tested and 13 third grade special education students.

When we examined the outcome level data in reading, we did not find a substantial difference in performance on the three outcome areas. Less than 50% of the students scored satisfactory on any of the three outcomes in both 3rd and 5th grades.

When I examined how our school performed in relation to schools with similar poverty, I found that 13% of similar schools outperformed us in reading in grade 3 and 40% of similar schools outperformed us in grade 5. Three schools outperformed us by 20% points in both 3rd and 5th grades. Flintstone Elementary in Allegany County and Glenmar and Dundalk Elementaries in Baltimore County outperformed us by over 20 percentage points in grade 5. George's Creek Elementary in Allegany County, Gladys N. Spellman Elementary in Prince George's County, and Salem Avenue Elementary in Washington County outperformed us by over 20 percentage points in grade 3.

Problem Clarification

Process

To address the question of why our data is low performing in reading, we engaged staff in the following problem clarification activities:

We asked grade level teams to discuss where they thought their team was in relationship to five key processes:

  • Understanding the target (what is assessed and how it is scored)
  • Teaching the indicators
  • Assessing the indicators
  • Monitoring student progress
  • Intervening with students not succeeding

Teams were given the attached fishbone and additional questions to focus their discussion. They were asked to bring back to the whole staff their findings and the evidence they had to support their findings.

On an ongoing basis, our teams have been examining sample student work, learning to use scoring tools, and analyzing classroom data on selected indicators. Teams have reached a number of decisions based on that data as well.

Our leadership team participated in two classroom walk through activities to examine the extent to which student work in classrooms was aligned with the reading indicators.

Parents were involved in the problem clarification process at the May board meeting when data was presented and disparities were a focus topic.

Findings

In the grade level team examination of the five key processes, all six teams identified all five processes as areas they needed to work. No team felt they had 100% of their teachers fully understanding or implementing the processes. Two of the teams felt they had 75% of their teachers understanding the target. However, findings from the classroom walk throughs and the team discussion of student work indicated their was still much confusion on the target and in particular how student work is assessed in reading using the MSPAP scoring tool.

Though all teams are currently using a school developed monitoring tool, they all agreed they needed to much better understand how to write good formative assessments and how to analyze the student response to make instructional decisions. No team felt they could honestly answer the question, "Where are each of my students in relation to each of the reading indicators and what evidence do I have of that?"

Classroom walk throughs indicated that half of the student work on display did not have a clear relationship to any indicator, and that student work identified as exemplary appeared to be exemplary in its neatness or artwork rather than exemplary based on demonstrated proficiency on an indicator.

Ongoing team discussions have resulted in team awareness that they were not all on the same page when it came to assessing student responses, that they did not all understand how to score with a scoring tool, and that they were not sure what to do with the data that they had collected.

Consequently, the consensus of staff was that two major problems for us were 1) ensuring that staff understood how reading was assessed and what a satisfactory answer looked like and 2) monitoring individual student progress so that we could identify where each of our students were and how we could use that information to increase their achievement.

Parents concluded that they did not understand the target well enough to determine where their children were in relation to the indicators. In conferences with teachers, they had not heard a clear explanation of where their child was performing in relation to the content standard indicators. Those parents with male children were concerned about the disparity between male and female students and reported a general lack of interest in completing the reading assignments given by teachers.

.
.
...
.
Course Overview Module 3 Overview Workshop 1 Overview Workshop Activity site map mdk12