Well, finally, Andy Thomas got a response from the state. The response did not come from State Superintendent of Education Michael Flanagan himself, but rather from Deputy Superintendents Joseph Martineau.
Read Martineau's response here, in which he says that:
In order to make meaningful, valid, and reliable accountability calculations, a stringent assessment participation requirement is necessary (and required under NCLB). Without the 95% requirement, schools and districts could selectively choose not to assess lower-performing students (a practice we have seen). This is why the participation audit is stringent – not testing 95% of students in the school as a whole and in all valid groups will cause a school to drop in the overall color scale, even if its academic measures are strong. In addition, schools have three ways to make the participation target: the most recent year, averaging across the most recent two years, and averaging across the most recent three years. Only if a school misses the participation target on all three measures is it marked as missing the target. As you have pointed out, low participation rates at Pioneer High School were the ultimate factor in its red scorecard. A more appropriate question for a parent to ask would be, why did Ann Arbor Pioneer not achieve an adequate participation rate among its economically disadvantaged students?
Actually, I don't think that is the key question, Mr. Martineau. The key question is, "Why are we using these tests as measures of whether schools are failing?" Most kids in Ann Arbor Public Schools graduate high school and go on to college. That is not a measure of a failing school. And as a parent who would like to opt my child out of ridiculous testing (but don't because I don't want to hurt our public schools' funding), this seems like even more of a ridiculous measure.
And here Andy Thomas responds to Martineau:
Purpose of ScorecardFirst, Dr. Martineau asserts that the accountability scorecard, unlike the “top to bottom” ranking, is not intended to be used to compare schools. He states that the primary purpose of the TTB ranking is to compare all schools in the state with one another, while the purpose of the scorecard is to compare schools against their own baseline performance rather than against other schools.
I find this distinction highly disingenuous. Each school receives an overall color rating. These ratings are a shorthand way of evaluating schools. They are widely used by the media, which publish lists of schools according to color code. Inevitably, schools are compared based on their color ratings alone, and few people have the time or inclination to delve into the details behind those color ratings.
Size and DiversitySecond, Dr. Martineau does not respond to one of my main concerns: The scorecard has a built-in bias that favors small, homogeneous schools over larger, more diverse schools. Perhaps I did not make that point as clearly as I might have in my original letter, for which I apologize. I hope the following data will emphasize my point.
I looked at the data for over 800 Michigan high schools and found a strong correlation between a school’s size and its overall color rating. The average enrollment at “green” high schools is 74 students. The average enrollment at “lime” high schools is 491, and for “yellow” high schools, 1,347.
One might be tempted to jump to the conclusion that smaller schools are better than larger schools due, perhaps, to the advantages of a more intimate learning environment. This may or may not be the case, but if it is, it is not supported by this data. The reason that small schools do better than large schools on the scorecard system is that large schools are evaluated on far more criteria than small ones, and thus are more likely to have visible, measurable areas for improvement.
Furthermore, the smaller the school, the less likely it is that there will be a sufficient number of students in the various sub-groups to meet the 30-student threshold for capturing evaluation data. Of the “green” high schools, not a single school had an identifiable ethnic sub-group (other than white). Among the “lime” high schools, only nine out of 152 schools were evaluated on more than one ethnic sub-group. In addition, all of the “green” schools and 40% of the “lime” schools did not receive proficiency ratings for economically disadvantaged students because they did not evaluate 30 or more economically disadvantaged students in any academic area...
It should also be noted that students may fall into more than one cell. For example, a student might be counted in the “bottom 30%” sub-group, and also in the “Hispanic”, “economically disadvantaged”, “English language learner” and “students with disabilities” sub-groups. So a student who somehow falls through the assessment crack may have an impact on multiple cells at a large, diverse school, but would have an impact on only one cell in a small, non-diverse school.
ConclusionIn my original letter, I characterized the color rating system as arbitrary, meaningless, useless and destructive. Nothing in Dr. Martineau’s letter has led me to a different conclusion. I have presented a variety of data taken from the MDE website showing that the color ratings are clearly biased in favor of small, homogeneous schools and biased against large, diverse schools. Dr. Martineau has not refuted these findings. Dr. Martineau states that the accountability scorecard is a requirement of Michigan’s flexibility waiver to the No Child Left Behind law. While I have no reason to question this, I would question whether the waiver requires the specific methodology implemented by the Department of Education. I would strongly suggest that this methodology needs to be revised to eliminate the bias I have documented.
Personally, I'd go a little bit further, and say that maybe it's time to throw out NCLB...
(Emphases added throughout.)
Read these in the following order:
1. Andy Thomas' original letter to Superintendent Flanagan.
2. Deputy Superintendent Martineau's response to Thomas.
3. Thomas' reply to Martineau.
Consider subscribing to Ann Arbor Schools Musings by Email!
No comments:
Post a Comment