Guest Post: Teachers, Statistics, and Teacher Evaluation

Have I mentioned that I love guest posts? 

Priti Shah, an AAPS parent and a UM psychology professor read a version of this during public commentary at a school board meeting, and she followed her comments up as a formal letter. I liked it so much that I asked if I could post it here. The reason I asked is that I think we need to understand what good evaluation would mean, and why the system being imposed on teachers by the school district is not a good system. And by the way, if you have never spoken at public comment (or haven't recently), I encourage it!

Dear Ann Arbor School Board Members:

This letter follows up on my comments during the public comment period of the Ann Arbor School Board meeting in January 2016. I spoke about the new teacher evaluation system.

As a reminder, I’m the parent of two children in the Ann Arbor Public Schools (11th and 6th grade). I am also a Professor of Psychology at the University of Michigan, and my research areas are in cognition and cognitive neuroscience and educational psychology. I base my comments on my feelings as a parent as well as based on the research evidence regarding teacher evaluations.

Priti Shah

The reason I wanted to speak was because I am very concerned about the climate of respect and collaboration teachers and administration that has been eroding in the Ann Arbor Public Schools and the impact on our children.

I start with three assumptions: 

(1) we all want the very best teachers possible,  

(2) we all want them to have the resources they need to provide the best possible educational experiences for each of our children, and 

(3) we want to be able to do all that without wasting our hard-earned resources. 

I strongly believe in setting high expectations and rewarding high quality work.  And as an educational scientist, I believe very much in high quality, research-supported teacher evaluation.  High quality evaluation should be valid (that is, someone who is rated as a “good” teacher should actually be a good teacher and someone who is rated as a “bad” teacher should actually be a bad teacher) and reliable (that is, evaluation shouldn’t change too much depending on who is in one classroom or which day the assessment occurs). Validity is a very hard nut to crack, because it depends fundamentally on one’s definition of what a good teacher is.

The new teacher evaluation system relies on two components: (1) student growth on a menu of standardized tests and (2) the Charlotte Danielson teacher evaluation system.  I would like to outline my concerns with respect to both of these approaches in terms of validity and reliability.

Student Growth

While I understand that incorporating student growth into teachers’ evaluations is mandated by state law, I want to highlight that the use of student growth—and how a teacher contributes to that growth--is problematic from a statistical perspective.  The American Statistical Association, in their policy statement on the issue, point to numerous concerns with respect to using student growth data for teacher evaluation purposes.  Most studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Student growth measures are not highly reliable, in other words. 

Most studies find that teachers account for about 1% to 14% of the variability in 

test scores, and that the majority of opportunities for quality improvement are found 

in the system-level conditions. Student growth measures are not highly 

reliable, in other words.  

A good teacher may look like a bad teacher depending on the composition of students in his or her class.  A group of Ann Arbor students in AP English may not show huge growth on a standardized English test because those students are already performing at ceiling on the test; their teacher might be rated as ineffective because there was no growth.  A teacher whose students may need safety and security (and warm coats and breakfast) may do an outstanding job and yet the circumstances that they are dealing with might lead to minimal growth on a standardized test. 

Another problem with using test scores to evaluate teachers is that relevant test scores are not available for many subjects taught by teachers-- my children have taken outstanding courses in subjects for which there are no standardized tests used: engineering design; communications, media and public policy; orchestra; art.  Some of these teachers will only interact with students once a week for an hour.  Evaluating these teachers on the performance of their students in subjects that they do not teach, and students that they rarely see, is absurd.

Furthermore, there is good support for the idea that teachers change their practices in light of these high stakes evaluations, often removing activities that promote critical thinking and creativity to spend more time on tested materials.

Most importantly, growth rates for different years for the same teachers vary widely, suggesting that these measures are not very reliable indicators of teacher quality and highly influenced by exactly which random kids they are teaching. And unfortunately, students will spend increasing amounts of time, and the district increasing amounts of money on high stakes tests that assess learning to the detriment of resources spent on other activities.

The Ann Arbor Public Schools would like to focus on growth for the bottom 1/3 of students in hopes that this will be an incentive to reducing the achievement gap.  Unfortunately, having 1/3 of the data to work with will mean a massive reduction in the possible reliability of the data because of smaller sample size.  And the bottom 1/3 is a dramatically different benchmark standard across teachers (i.e., you cannot compare growth across teachers if one is using the bottom 33% of the students in AP English and another the bottom 33% of students in guitar).

The Charlotte Danielson Framework

The second proposed component of the new teacher evaluation system is the Charlotte Danielson Framework. On the surface, this is a reasonable measure that involves administrators evaluating teachers on a systematic set of 76 items that are likely to be positively associated with teacher quality. 

Again, a good measure of teaching quality an assessment requires two key features: it needs to be reliable – in that the same teacher would be rated the same across time by different people—and valid—that is, that a good score on the means someone really is a good teacher.  Unfortunately, the reliability or validity of this framework is just not clear, based on the extant evidence.  Sure, you’ll hear some relatively high numbers from the people who sell the Danielson system but they are based on expert coders watching the same lessons on video.  Consider rating a baseball player for 15 minutes during a game.  If he makes a home run that day, your two independent raters will rate him similarly. If he strikes out, the two independent raters will rate him low. It’ll look like your rating system is highly reliable. That’s how reliability of these observational methods is tested. This is just one of many problems associated with such classroom observation methods.  

I point the board to a 2012 article in Education Researcher by Harvard School of Education Professor and University of Michigan PhD Heather Hill for a more technical discussion of these and related concerns. And at the same time I appeal to your common sense: Look at the rubrics and ask yourself—have you ever had a terrible teacher who could check off all the boxes and look like an “effective” teacher because they could use the right lingo and implement the criteria superficially?  Have you ever had a stellar educator who inspired and motivated you to succeed but didn’t see eye to eye with the administrators’ views on how classroom seating could be organized? Might there be a teacher who can shine during such a formal evaluation process but shows active disdain for some students throughout the school year?

I appreciate the extreme difficulty but necessity of evaluating teacher effectiveness, but I can confidently state that just by moving from rating teacher on one subset of the criteria annually to rating them on all four will not necessarily positively impact the reliability or validity of the measure. Indeed, it is likely to reduce the quality of the ratings, the validity of the measures, while simultaneously increasing burden on teachers and administrators. Just because there are more items does not mean an assessment is better.   Neither do I think that the vast majority of highly effective experienced teachers are going to change and become less effective. At my own job, our evaluations become less frequent with greater seniority; this makes sense to me.

Recommendation

Given that teachers must be evaluated, and that none of the proposed methods are particularly reliable or valid, I would probably use a combination of metrics as proposed by the school board. However, I would (1) try to minimize burden on the teachers and administrators (as in, not that many hours of time), (2) involve teachers in decision making at all phases (to get input on what they think should be included and what is reasonable and won’t distract them from their real work), (3) include not just administrator evaluations but peer evaluations (that is, ratings of other teachers, who often know more about what goes on in classrooms), and (4) consider also input of parents and students.   

A proud mama moment: my son wrote an article advocating the inclusion of student ratings of teachers for the Skyline Skybox (http://readtheskybox.com/201601/why-students-are-the-best-tools-when-it-comes-to-teacher-evaluations/); while I think student evaluations can be problematic in some situations, he makes an excellent point.   Student evaluations, based on specific questions regarding teaching effectiveness (not just “was this a good class” but whether the teacher seemed to care, whether students respect the teacher, and so forth) can actually be better predictors of student growth than observational methods.  And I can tell you that parents in our community are pretty well informed regarding which teachers seem engaged, caring, and effective. Parent and student surveys are cheap.

Conclusion

We need to start with some basic assumptions in revamping the teacher evaluation system in Ann Arbor.

My first assumption is that most of our teachers are smart, hard working, and caring professionals. I have observed far, far, more excellence in the Ann Arbor Schools classrooms on my many visits and interactions with teachers than I have experienced ineffective teaching.

Second, the Ann Arbor school system needs to maintain its leadership position regarding school administration and governance as well as quality schools.  The reason we have such outstanding teachers is that they want to work in our district.  We want to attract the very best teachers, not drive them away with unnecessary busywork.  Let’s interpret our state’s laws in a manner best suited to our teachers and students instead of jumping through hoops that may well be unnecessary.

Finally, let’s all agree that we want to expend our time and money on what helps our children learn, and that we do not want more and more of our money go to for profit testing companies, consultants to train administrators and run workshops teachers on evaluation rubrics, software so that administrators can rapidly rate teachers on numerous criteria quickly in the classroom at the press of a button.

Thanks for your time, and I’m happy to have a longer conversation with anyone who would like to talk to me.

Sincerely,

Priti Shah

A few references:

http://www.amstat.org/policy/pdfs/asa_vam_statement.pdf

https://edpolicy.stanford.edu/sites/default/files/publications/getting-teacher-evaluation-right-challenge-policy-makers.pdf

Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough teacher observation systems and a case for the generalizability study. Educational Researcher, 41(2), 56-64.

Guest Post: A Parent Reviews Her Child's M-STEP Results, and Learns...

A guest post by Naomi Zikmund-Fisher about the M-STEP results, and what they mean.

Last week, we finally received my children’s scores from the M-STEP test they took last spring. My son, a fourth grader at the time (now 5th) and my daughter, a high school Junior (now senior) both took the test. For more on that decision, you can read here.

In the interest of maintaining some of their privacy, I’m not going to share how my kids did on the test. More to the point, it probably doesn’t matter how they did on the test, as their performance on this first round appears to be being largely discarded.

As a former teacher and administrator, I probably know more about how to read a score report than most parents. Theoretically, I should be able to get all there is to get out of these scores. So, here’s what I learned from looking at my children’s score reports:

1. Last spring, my children were doing about as well in their academic progress as their teachers said they were. There were no real surprises. You could have looked at their report cards and gotten the same information that M-STEP gives you.

2. That information is wildly out of date. They took this test in a window from March to May. I got the scores in January. Whatever new information may have been useful in the scores is no longer pertinent.

3. The science and social studies tests measure curriculum alignment more than anything else. They are broken out by different smaller subjects (e.g. physical science, life science or economics, geography). You can see that in this sample of a child’s 4th grade science scores.

Sample M-Step information provided to parents, in this case for the science test.

When they say a child is proficient, what does that mean?

My children did best in areas that they had studied recently and worst in those from previous years. In other words, this test measured what classes they were taking, not anything about my children or about whether their teachers were teaching well.

4. The target area for “proficient” is, in some cases, shockingly small. Scores are reported graphically (among other ways) on a continuum of four ranges. Proficient is the second to the top and is the smallest area, sometimes by quite a bit.

But shouldn’t “just fine” be a fairly broad range of kids? When did we stop recognizing that “normal” isn’t a single point, it’s a spectrum?

Sample of information provided to parents. Note that the grey "margin of error" overlaps both the "partially proficient" and advanced categories, meaning that a child who scores in the yellow/gray overlap as "partially proficient" might actually be "proficient" on another day. Note also that the green ball of "proficient" is a much smaller area than the bars for not proficient, partially proficient, or advanced.

This picture shows the score graphic for the same student whose subject scores were above. This child is supposedly proficient in 4th grade science [the score is right in the middle of the green bubble]. As you can see, this is quite a feat, since the “Proficient” range is about 5.5% of the total.

What’s more, while it’s great that the score report acknowledges a “margin of error” around the score, that margin is substantially larger than the target itself. This means that three kids who score as “partially proficient,” “proficient,” and “advanced” might all know exactly the same amount of science. We sing the praises of one (and the wonderful teacher who taught her) while wringing our hands about another (and the mediocre educator she had) when there is truly no difference at all.

In the end, what I realize once again is that this data is designed to measure districts and schools much more than to give us any useful information about individual children. Even without the huge delay in score reporting, the amount of useful information, that you can’t find more easily somewhere else, about a single child is minimal.

It’s reasonable to say that the measure of a school or a district is how well its children are prepared for the next phase of life. The problem is, we’re substituting this test for the real answer to that question. We’re asking our kids to take hours upon hours of tests – time they could spend actually learning something – in service of measuring their school system.

If we already know how they’re going to do on the tests, then we already know what the answer we’re going to get will be. And if we don’t already know how they’re going to do on the tests, it’s either a really bad test or a school so out of touch with students that it should be obvious in multiple other ways.

I can say unequivocally, however, as an educator and as a parent, that the M-STEP given last spring was just plain a waste of my children’s time.

Wednesday 1/27/2016: AAEA Press Conference, then AAPS Board Meeting

Ann Arbor teachers have a new evaluation system that involves more testing (of students) and much more paperwork, and they are not happy about it.

Tomorrow, Wednesday, January 27, 2016, the Ann Arbor Education Association (teachers' union) is having a press conference at 6 p.m., at Forsythe Middle School in the Media Center.

After that, the school board meeting will also be at Forsythe Middle School.

The board meeting is supposed to start at 7 p.m., also at Forsythe.

Look at the agenda and board packet here. [The system is not at all intuitive. To see the meeting packet, click on the agenda. The agenda opens up and on the left side of the page, there is a navigation panel that has documents attached. If a document has been uploaded in advance of the meeting.]

Can't be there in person? A nice new feature is that you can live stream the board meetings. Go to this web page and look for the live streaming link.

Want to see what the teachers are talking about? 

What catches my eye is that in a recent survey of over 600 teachers, over 90% of teachers don't believe the administration or school board supports them.

There are lots of links to documents on this web page, and here is an infographic they have shared.

Consider subscribing to Ann Arbor Schools Musings by Email!

State Legislation: Thank You, Local School Boards, Superintendents, Representatives

I think I would be remiss if I didn't thank the school board and superintendent (and, in fact, not just of Ann Arbor, but of Lincoln schools and maybe some other local ones as well), for taking policy issues to the legislators.

1. Ann Arbor Superintendent Jeanice Swift testified at a Senate Committee hearing against the idea of having guns, whether open carry or concealed carry, in schools. 

Here is an excerpt of her statement:

My remarks today are directed specifically toward the question of allowing concealed carry in pre K-12 schools.  

We recognize the proposed legislation is considered by some as a ‘fix,’ a compromise, an effective way to close the ‘open carry loophole’ that currently exists in Michigan law. Clearly, some consider ‘concealed carry’ as an improvement over ‘open carry.’ 

We understand that the stated intent of the legislation may in part be designed to remove the concerns with weapons that are visibly displayed in school and so prove a disruption to ensuring a safe, secure, learning environment. However, it is overwhelmingly clear that guns, visible or concealed, pose a significant risk to the safety and wellbeing of students, staff, and families at school.

You should read the rest, because Dr. Swift gives some shocking examples that happened in real life, that explain why guns in schools are a bad idea.

2. On the "third grade retention bill," which would provide interventions for struggling readers but also would require kids to be retained, the bill has passed out of the House more or less on party lines. My representative, Adam Zemke, originally was a co-sponsor but withdrew his support. According to this article,

The proposal was approved in a 57-48 vote, mostly along party lines, and now heads to the Senate. Democratic Rep. Adam Zemke of Ann Arbor, an original co-sponsor of the bill and key player in negotiations, withdrew support on the floor and removed his name from the measure.
Zemke had proposed an amendment to allow struggling readers to advance to fourth grade if they were working to improve under an individualized reading plan and had support from school administrators and parents. The amendment was rejected.
"This bill, without that amendment, then tells Johnny none of that (work) matters," Zemke said. "We're going to hold you back regardless. I am not going to remove the hope of a 9-year-old, period."

Thank you Lincoln Consolidated Schools Board for opposing this bill!
Thank you, Representative Zemke! The bill now goes to the Senate.

The bills go to the Senate next. The Capitol is

pretty. What's going on inside? Not so much.

3. The Teacher Evaluation bill passed the House. It's better than it was, but it's still (in  my opinion) bad, and I appreciate Rep. Jeff Irwin's opposition to this bill. He wrote on facebook:

I also have concerns about SB 103, the educator evaluation policy. My opposition stems in part from my opposition to the changes made to the tenure act in 2011. But, my opposition is deeper than my desire to stunt the effect of those changes. Mainly, I'm opposed to the bill because it accepts the toxic notion that education will be improved by more testing and more motivation for the teachers. This bill accelerates the problem we have with teaching to the test. If we want educators to teach to the test, the best way is to approve legislation like this that bases their employment and promotion on testing.
Also, I don't think the tests we're mandating produce consistent and reliable results. In other states that have adopted similar policies relying on testing growth (or value added), teachers are rated highly effective one year and then ineffective the next. The assessments bounce all over and this legislation will provide unreliable information to parents and school leaders. Our students and educators deserve better. (Emphasis added.)

Consider subscribing to Ann Arbor Schools Musings by Email!

Guess Who's Evaluating Those Standardized Tests? (I Wish This Were a Joke!)

From an ad on Craigslist:

"The starting pay is $10.70 per hour." For that sum, I'm sure they are getting
some highly skilled staff. Screenshot from Craigslist.

Who? Measurement Incorporated

We are a diverse company engaged in educational research, test development, and the scoring of tests administered throughout the world. 

What? Reader/Evaluator Position

If you qualify as a reader/evaluator, you will be eligible to work on a number of our projects. Many projects require readers to score essays for content, organization, grammatical convention, and/or the student's ability to communicate and to respond to a specific directive. Other projects involve scoring test items in reading, math, science, social studies, or other subject areas. The tests you will score come from many different states and from students at all grade levels, elementary through college, depending on the project.

Where and when? 

Starting in March of 2015 , day and evening shift, in Ypsilanti!

REQUIREMENTS

Bachelor's degree in any field
Ability to perform adequately on a placement assessment

HOURS: Temporary, but 5 days/week.

PAY: The starting pay is $10.70 per hour. 

Yet we are going to use these temporary employees to evaluate our students' work, and our teachers. Yippee.



Consider subscribing to Ann Arbor Schools Musings by Email!

Five Updates: Who's Covering the First Elected Board, What's Your First Choice, Where's the Money Gone, Who's Listening Anyway, Why Keep Testing

1. Ypsilanti Community Schools board voted 4-3 not to do a superintendent search--their superintendent's contract will automatically renew in April for another year if they don't do anything.

It's not that I have an opinion about whether the school board should continue with Laura Lisiscki as superintendent, but I first heard about this issue on a facebook group. As a rumor. The rumor turned out to be true.

It also turned out that heritage.com did cover the school board meeting, but that is not always a sure thing.

Who is routinely covering the school board meetings now? (Good question.) Does it matter that a discussion like this ends up with a 4-3 vote? (I would say that it does.) What is the best way to keep the community informed?

2. Ann Arbor schools has opened its doors to students from other districts. Part of me feels that this is poaching. On the other hand, the other school districts had already opened their doors to schools of choice.

But also--perhaps more interesting--you can also choose to send your kids to a different school within the district. Here's some information, the "window" for applications ends February 27th: http://a2schools.org/aaps/ins.child_accounting/space_available_transfers.

3. Local state representative Jeff Irwin explained how we can follow the money the other day on Facebook: 

I just finished my first meeting of the Appropriations committee. Chairman Al Pscholka offered some thoughtful and well-offered commentary at the beginning of the meeting focused on encouraging committee members to get educated on Michigan's budget. Then, we received a report from the House Fiscal Agency on state revenues.
This report highlights the impact of the massive changes that were made to business taxes in 2011. In summary, business taxes are expected to net just over $180M in the next budget. This represents only about 2% of the state's general fund. Just a few years ago, corporate taxes brought in around $2B per year in state taxes (about 25% of the GF). If you're wondering why tuition is going up, class sizes are going up and taxes on individuals are going up, now you have your answer. (Emphasis added.)

4. Ann Arbor Superintendent Jeanice Swift is on her second listening tour. I thought the first one was pretty successful, and I'm planning to attend at least one of these. I hope you will go to one too--so you can bring up your most important issues, whether they be transportation, testing, trimesters, trigonometry, tenth grade, or (I'm running out of "t" words here)...

All discussions run from 6:30-8 pm unless otherwise noted:

Clip art taken from:
http://www.clker.com/cliparts/0/6/8/6/11971488431079343407barretr_Pencil.svg.hi.png

Monday, February 9 at the Administration Building hosted by the PTO Council
Tuesday, February 10 at Scarlett Middle School
Thursday, February 12 at Slauson Middle School
Monday, February 23 at the Downtown Library, 4th Floor at 12noon-1:30pm
Tuesday, March 3 at Tappan Middle School
Monday, March 9 at Peace Neighborhood Center
Thursday, March 12 at A2 STEAM at Northside
Thursday, March 26 at Pathways to Success Academic Campus
Monday, March 30 at Community High School
Tuesday, March 31 at Ann Arbor Open

5. Last, but not least, we come to testing. 
This spring there will be the M-Step, which is Michigan's "not" Smarter Balance and "not" the MEAP and "not" the ACT [but 11th graders will still ALSO have to take the ACT]...the last minute, untested, unvalidated, uncomparable to last year's MEAP but being used because state officials couldn't agree on using Smarter Balance standardized test. And this will probably be replaced by something else next year. Read all about M-Step here.

Well--if you want to know what is going on with testing at the state level, then I suggest you subscribe to the Michigan Department of Education's Spotlight on Student Assessment and Accountability. 

If you were paying attention a few months ago, you might remember that several local superintendents voiced the concern that there was too much testing in the 11th grade. And so I was interested in this tiny concession that was published in Spotlight: 

Due to concerns around testing time, the Classroom Activity andPerformance Task components of the 11th grade M-STEP are optionalfor high schools. While there are some benefits to administeringthese components, they will not be required. There will be noaccountability penalty in terms of participation or scoring for highschools that choose to not administer the classroom activities andperformance tasks. This does not apply to the classroom activitiesand performance tasks in grades 3 through 8 — those are required.

Anyone want to opt out? Here is an article about some strategies.
Also, you can find some local resources (people!) at the Facebook page, Ann Arbor STOP: Stop Overtesting.

Consider subscribing to Ann Arbor Schools Musings by Email!

Smarter Balanced Test: Try It Out Before Your Kids Take It

Wondering what Smarter Balanced is?

It sounds like margarine, right?

It's not. It's a test. It's the test that is supposed to replace the MEAP test (but be given in the spring, and on computers). And it's supposed to be "Common Core aligned." And it's going to be longer than the MEAP. [It's state-mandated. So whatever school district or charter school you are in, this test should be of interest to you.]

But hey, it's got a logo that looks like it belongs to a forestry group.

Read the smarterbalanced.org web page here.






According to this fact sheet by Fair Test, "Two multi-state consortia — the Smarter Balanced Assessment Consortium (SBAC) and the Partnership for Assessment of Readiness for College and Careers (PARCC) — won federal grants to develop Common Core tests, which are due to be rolled out in 2014-15."

In Michigan, we will be using Smarter Balanced.
Here's a nice article on how the Smarter Balanced test is going to fund for-profit corporations. (OK, really--not so "nice." But it's worth paying attention to this!)

As Fair Test points out,

Proponents initially hyped new assessments that they said would measure – and help teachers promote – critical thinking. In fact, the exams will remain predominantly multiple choice. Heavy reliance on such items continues to promote rote teaching and learning. Assessments will generally include just one session of short performance tasks per subject. Some short-answer and “essay” questions will appear, just as on many current state tests. Common Core math items are often simple computation tasks buried in complex and sometimes confusing “word problems” (PARCC, 2012; SBAC, 2012). The prominent Gordon Commission of measurement and education experts concluded Common Core tests are currently “far from what is ultimately needed for either accountability or classroom instructional improvement purposes” (Gordon Commission, 2013).

Oh yeah, and also? It's a computer-based test which means it is going to hog up the school computers. I sure am glad I voted for that technology millage...

Curious about the test? You should be. Take a sample of the test here. [Just log in as a "guest."]

And then? Please share your observations about the test sample in the comments section.

Consider subscribing to Ann Arbor Schools Musings by Email!

Missed Opportunities: Ann Arbor and the Washtenaw IB/WAY/ECA Consortium

I wasn't really paying attention to the big brouhaha as to whether the Ann Arbor schools should sign the contract to continue to participate in the International Baccalaureate program, the WAY (Washtenaw Alternatives for Youth) program, or the Early College Alliance. But now I am. And mostly what strikes me is that there have been several missed opportunities. Sure, I know that hindsight is perfect, but looking back and evaluating is also a good way to learn.

So here are five missed opportunities.

1. Missed Meetings: The Ann Arbor News reported that the Ann Arbor school representatives missed many of the consortium meetings. I don't know if Supt. Pat Green or Deputy Supt. Alesia Flye was to blame for that--maybe it was both of them. They're both gone now, so I'm not sure if it matters if we figure that out. Going forward though, if we have a seat at the table, we need to take it. It's pretty clear that we can have more influence if we are there early in the process.

2. Anti-union contract: Someone called me to say that he was worried that the contract the Ann Arbor school board was discussing was anti-union. Given that the contract (click on the link to see it) specifies that if a teacher is tenured in a district and goes to work for the IB, WAY, or ECA schools they are not operating under or accumulating tenure (among other things), you could describe it that way fairly, I think. But here's the thing--this same contract was already voted on by the Ann Arbor schools for this current year in August--and by the other school districts as well. Does the Ann Arbor Education Association or the Washtenaw Education Association not care, or did they just miss this? They probably could have influenced the terms and conditions...

3. Failure to Track: When the Ann Arbor school's Count Day numbers came out, and they were below expectations, much of the attention went to the number of AAPS high school students who were enrolled in the IB, WAY, and ECA programs. And the district seemed surprised by this. To my mind, either they weren't surprised, but wanted the public to feel that they were (which would be misleading), or they were surprised. And if they were surprised, then I have to ask why that is. You might remember that my son applied to the IB program at the Washtenaw International High School--and he found out that he was accepted sometime in late winter or early spring. Now surely, as consortium members, the district could find out how many Ann Arbor students had gotten in to--and later, decided to go to--these alternative programs. The question is, why didn't they take those numbers into consideration as they constructed this year's budget?

4. Transportation Thinking: I don't think the school board and administration really took into account the way that threatening to cut high school transportation could affect the way students looked at schools. I'll probably never be able to prove this, but to my mind, when the district said--at the same time that students were looking at high schools--that high school transportation might not be available, it changed the equation for many parents. I know for myself that I was intimidated by the idea of transporting my son to the IB school. On the other hand, if I lived far from my district high school, and would have to transport my child anyway, then I would not be comparing "drive my child to one school or have him take a bus to the other" but rather "drive my child to school A or school B?" So even the threat of the transportation being cut may have influenced the debate for students at the time when the choices were being made.

5. Going it alone: I believe the ECA, the IB program, and the WAY program are all very worthwhile. But Dexter--which decided to do its own IB program, and which decided not to join the WISD transportation consortium--may have done the best job in looking out for Dexter. I am glad to see the Ann Arbor school board now considering doing its own IB program, even if the consortium IB program continues.



Subscribe to Ann Arbor Schools Musings by Email

Testing, Testing, Testing: Tale #1

1. About cartooning

I admit, I love my little cartoon, The Parable of the Hammer. But I would hate to be as obscure as Bezonki (although I wouldn't mind drawing/painting as well as Alvey Jones). My husband thought the moral of the story of my cartoon, "When all you have is a hammer, everything looks like a nail" was obvious. And he sort of understood the references to testing being overused. But, he said, it wasn't obvious why you couldn't use a test like the NWEA MAP test--that was built for assessing individual students--to also assess teachers. Why shouldn't teachers be assessed by their students' improvement?  And this is, in fact, such a common misunderstanding of testing that I thought maybe I should address it again. [For this explanation, assume that the MAP test does a good job of assessing students, ok? A hammer, after all, does a good job pounding in nails.]

And I started thinking about this (again) because in an mlive.com article school board member Andy Thomas said that "NWEA testing that the district has engaged in will help show if there are advantages or disadvantages to a combined class." 

So in fact Andy Thomas got stuck in the same type of thinking that my husband was stuck in--that if a test can assess individual students, it can assess something else. In my husband's case, he was thinking it could assess the teacher; in Andy's case, he was thinking that it could assess types of classrooms. 

But it can't. Here's why. Students are not randomly assigned to classrooms. If a principal thinks a teacher is competent, (s)he might get--or volunteer to take--students who are perceived as more difficult.

In addition, students are not randomly assigned to schools. [Andy was talking about comparing Ann Arbor Open to other schools, and that is obviously not random since Ann Arbor Open is a magnet school.] But even if we look only at neighborhood schools, it is pretty clear that neighborhoods like Burns Park and the area near King School are in general wealthier than the area near Pittsfield or Mitchell schools.

And last, but not least, sample size is important and in general the sample sizes are too small and not at all random. Read the Northwest Evaluation Association's own memo on the subject.

So we can debate whether we should be using the NWEA MAP test to evaluate individual student progress [and you know that I'm a NO voice for that] but let's try to use the right tools for the right jobs.