Evaluation of the NWEA MAP Test (in Ann Arbor)

Back in March of 2012 (around 18 months ago) my friend Julie and I went to the AAPS school board meeting to talk about our concerns about the NWEA MAP test. In 2011-2012, the MAP test was first used in the Ann Arbor schools, at the K-5 level throughout the district, and only for grades 6-8 at Ann Arbor Open and at Scarlett Middle School.

Before I get into talking about evaluation, it's worth noting that the Ann Arbor Chronicle has writeups of the initial discussions at the school board meetings in 2011, and you can find them here:

March 2011 Performance Committee discussions, as reported to the board

April 2011 Performance Committee discussions, as reported to the board

May 17th, 2011 report on board discussions (first briefing)

May 30th, 2011 report on board discussions (second briefing)

[Side note: Boy do I miss those Chronicle reports on the school board!]

By the time I realized what was going on in the school testing world, it was nearly a year after the school board had voted on getting the NWEA MAP test. And I felt that we had been sold a bill of goods. That's why Julie and I went and shared our concerns with the school board (links are on "Julie" and "I").

One of the main requests that I made was to have the use of the NWEA MAP test evaluated. I wrote:

5.     Since this is a pilot, do a rigorous evaluation of MAP’s usefulness. Evaluate whether the MAP test is working for everyone else. And do that by surveying principals, administrators, teachers, parents and students—anonymously—so that they can share their true opinions.

Shortly after that I met with Alesia Flye, Dawn Linden, and a woman from the technology department whose name I've forgotten. It's worth noting that none of them were at the Ann Arbor Public Schools when the decision was made to use the NWEA. I didn't keep good notes of the meeting, but I did leave with the impression that they would be evaluating the use of the MAP test.

Also around this same time, I met with several school board trustees at a technology bond meeting. I told them, I don't like the NWEA MAP test, I don't want my money going for technology that will support it. I was told by Deb Mexicotte (and again I didn't write this down) that she thought the MAP test would be around for at least 3 years, but it would be evaluated.

Meanwhile, my friend Angela also complained, and had a meeting with curricular staff.
In June of 2012, she had the following email exchange with Dawn Linden:

Angie to Dawn, 6/5/2012:

Thank you so much for taking the time to meet with Pam and I today.  I had no idea we would talk for so long, and still feel like there was still more to say.  I appreciate hearing your take on the NWEA tool.
On the drive home, however, one part of our conversation gave me pause.  You said that a parent and teacher feedback survey is in the works, as this is a pilot program.  You also mentioned that the NWEA is NOT going away.  What is the puprose of the survey, then?

Dawn to Angie, 6/5/2012:

Thank you, Angie.  I appreciated the meeting and welcome further discussion, too.  We've been seeking information from teachers and principals informally throughout the process.  That feedback has been overwhelmingly positive.  We do plan to share a brief survey with parents seeking information about parent understanding of the tool and its uses as well as how we can improve reports. I hope that helps to answer your question and please don't hesitate to call. (Emphasis added.)

Angie to Dawn, 6/8/2012:

So even though this test is in its pilot year, the district has all ready decided that implementation of it will continue as is. And this has happened before any feedback has fbeen collected from the test-givers, test-takers, tax payers, teachers, or AAPS families that are all supposed to benefit from it?

Dawn then offered to meet with Angie again.

This came from an October 2012 AAPS student achievement report. Does it add to
your insight as to how students are doing? Yup, that's what I thought too. Found at:
http://annarborchronicle.com/wp-content/uploads/2012/10/101012studentachreport.pdf

Understand, too, that as an Ann Arbor Open parent, I was and am extremely aggravated that Ann Arbor Open and Scarlett were (and still are) the only middle schools getting tested. If it's such a great test, shouldn't everyone else get a chance to take it? And if it's not such a great test, why should we suffer? Shouldn't that be evaluated?

But I'm patient. (No, not really. But I do believe that good evaluations take time. And so I waited.)

Meanwhile. . . the only change that the district made (that we could see) was that kindergartners didn't have to take the test until the winter. Instead, they got to take "practice" tests. (Are you aggravated yet?)

So in April 2013, Heather sent a FOIA to the district. Yup, that Heather. Here is what she sent, and what she got.

1. Any plan for evaluation of the NWEA MAP in the Ann Arbor schools, in final form
or if not finalized in draft form, with the date when the plan was developed. 

a. Currently there is not a formal written evaluation plan. 

2. Any discussion via email of plans for evaluating the NWEA MAP (for example,
between and among the Superintendent, her cabinet, other administrators (AAAA
group), the school board, and/or the technology department. 

a. No emails exist on this topic.

Now, we have a situation where there is no formal written evaluation, in draft or final form, and that is disappointing. But if no emails exist on this topic, then that is--in my opinion--inexcusable--because it means there is really no intent to evaluate, no discussion of the topic even. 

The school district is using a test that has evaluation in its name, to evaluate students--but we're not evaluating if it does a good job at that. The district is using a test that has evaluation in its name, to evaluate teachers--even though the organization that developed the test says it shouldn't be used that way. The district is using a test that takes a tremendous amount of staff time and technology time to use and administer, but we don't account for that time and effort to see if it is worth it. The district is using a test at two middle schools, and not evaluating whether it is worth extending or cancelling. We haven't been able to improve the reporting to parents in any way that would be meaningful. We have been told that the test would be evaluated, but it hasn't been. 

And that's why--if anyone asks me if they should opt their kids out of the NWEA MAP test--I do not discourage them at all.

So--that's the bad news.
The good news? There is a new superintendent in town, and maybe she is open to new ideas. It's worth a try. Let her know how you feel. There are a whole lot of community meetings coming up in the next two months!