The (financial) costs of (NWEA) testing in Ann Arbor

Because I no longer have a child in K-8 grade, I missed the start of the NWEA MAP testing period. I only heard it had begun when I heard through the grapevine about a technology problem in the testing at Ann Arbor Open during the first week of the testing period.

You might recall that my friends and I have raised a lot of questions about the NWEA MAP testing over the past couple of years. You can read more about our concerns in some back posts:

Testing the Year Away: Julie's Comments

Testing the Year Away: Ruth's Comments

Ann Arbor: Stop Overtesting Our Kids! (the petition was submitted; the facebook group is still alive and well if you want to join us)

Real Life NWEA Experiences, Part I

Kindergarten Blues

Are You Following the NWEA MAP Controversy in Seattle?

Standardized Testing Season: Did You Want to Opt Out?

In any case--because of those concerns, we decided to FOIA some materials  from the Ann Arbor Public Schools around costs and evaluation. This post is only about costs.

Obviously, there are costs involved with testing--both direct and indirect. What are they? We asked "invoices, bills, or contracts" for 2011, 2012, and 2013, for the NWEA MAP test  (and no others, even though there are several others).

For 2011-2012, the bill was $96,401.50 (including $3,700 for an on-site administrative workshop)

For the 2012-2013 year, the bill was $77,701.50.

And on May 6, 2013, while we were still discussing the budget, AAPS got a bill from NWEA for $94,024 for the 2013-2014 year.

Picture of the 2013 NWEA MAP test invoice

[Here, by the way, is the "Master Subscription Agreement" if you are interested in knowing what AAPS has agreed to in the NWEA license agreement.]

In other words, over the past three years, we have spent more than a quarter of a million dollars ($268,127 if you are counting) on purchasing the licensing for the NWEA MAP test.

Now obviously, software doesn't install and manage itself; teachers don't learn to use the test by themselves. In fact, it turned out that our technology wasn't even adequate to properly run the test, and so the rationale for computer upgrades (and the cost) should appropriately be--at least partially--attributed to the use of the NWEA MAP test.

In a separate FOIA (one that was more focused on evaluation--coming next) we asked for information about some of the staff costs, and were told:

3. The amount of staff time from the curriculum and technology departments that has
been devoted to addressing implementation and deployment of the NWEA/MAP over
the past two years (fiscal 2011-2013) 

a. This information is difficult to isolate given that NWEA MAP training has occurred in conjunction with other topics surrounding the use of data to differentiate instruction. There is no separate document that provides this information. 

4. The number of inservices given (total number of hours, along with dates) to teachers
and administrators, the number of hours spent on these inservices and the number of
teacher/administrators trained for the purposes of implementing the NWEA MAP
test. These inservice hours could, for example, relate to how to administer the MAP
test, how to understand MAP test results, and how to use that information to inform
teaching. 

a. This information is difficult to isolate given the NWEA MAP training has occurred in conjunction with other topics surrounding the use of data to differentiate instruction. There is no separate record of NWEA MAP inservices or professional development.

Which means that I don't actually know the costs. But I am going to take some educated guesses as to the amount of staff time spent on the NWEA. These are rough estimates.

In Year 1, I am going to estimate that every K-5 teacher and all of the 6-8 teachers at Scarlett and Ann Arbor Open, spent at least 4 hours on inservices (trainings) related to the NWEA. With approximately 20 teachers at each school, and 20 elementary schools (so 400 teachers, PLUS another 40 middle school teachers), we get 440 x 4=1760 hours x $50/hour=$88,000.

Half of those teachers spent at least 5 hours preparing for and proctoring the MAP test (and yes, it was supposedly much less time than that, but I don't think it turned out that way). 220 x 5=1100 hours x $50/hour=$55,000.

In Year 1, I am going to estimate that at least one technology person spent hundreds of hours on dealing with technology issues around the MAP test (there were many), and that between the various curriculum staff, they also spent several hundred hours on the MAP test, troubleshooting, training teachers, etc. So if between them, they spent 750 hours on the MAP test (and I think that is low), that would be 750 x $50/hour=$37,500.

And then, of course, the technology we had was not adequate, and one of the rationales for the tech bond was that we needed to upgrade our computers for the MAP test. And that was really a lot of money.

I will mention, but not count the costs of, the additional sheets of paper that we parents got with (completely useless and unintelligible) test results--three times/year. But if there were 8,000 students involved, and therefore 8,000 pieces of paper each time, that is 24,000 sheets of paper (8,000 x 3) and the ink related to printing them.

I know, you are thinking, "But Ruth, we would be paying those teachers, tech people, curriculum staff anyway." Sure, that's true. "And we needed better computers anyway." OK, that might be true as well.

But those teachers, tech people, curriculum staff could be spending their time on other things if not for the MAP tests, so I think we have to ask if that time-money (over $180,000/year) was well-spent. Was that time-money well-spent?

Was that hard cash well-spent? Or could we find a different way to spend a quarter-million dollars a year? This is not just a thought experiment. It's a serious question. I know there are people who believe this is an important task. But let's admit that when we "estimate" the cost of the MAP test, it is not just the $75,000-$100,000/year. It's a lot more.