3 Weeks In The Marking Factory: An Insider's Look at How Standardized Tests are Really Graded

By:  Anonymous Canadian BAT

I'm sitting in the marking factory with 3,000 other teachers from across Ontario where, for the past three weeks, I've been marking the Grade 6 reading component of the provincial standardized achievement test. Before the start of my first day here, a representative from the Ontario Teachers' Federation flagged down a group of us and adjured us to refuse to mark the test, since the Ontario Teachers' Federation doesn't support standardized testing. But, of course, neither do any of us. We're just here because we need the money. I might have been warned of what was to come by the coffee break that first morning: 10:30 by my clock rolls around and I make for the door, only to be barred by my group leader. We aren't allowed to leave until the chimes ring. So a group of teachers silently line up single file, waiting to be dismissed. Kind of makes me remember how I felt as a student and why I don't insist on this lineup bullshit from my Grade 8 students. The bell rings. The hallway floods with 3,000 people trying to get to the cafeteria. It takes 10 minutes to negotiate the line, another five to get to the coffee pots, then the chimes ring again. No time even to smoke. Some break.

The test is administered by the Education Quality and Accountability Office (EQAO), supposedly to give parents concrete results on how their children are performing in school. At the elementary level, Grade 3 and 6 students are assessed on a four-point scale in reading, writing and mathematics with level one being 50-60 per cent and level four being 80-100 (notwithstanding those who just don't have a clue, who are marked NE1). There's no nuance; a high level two and a low level three may be a hair's breadth apart and separated only by the markers mood that day. It's probably important to note that, taking the Grade 6 test cold with no preparation, I -- an Honours Bachelor of Arts graduate, summa cum laude -- was graded a level two last week. Now, there's an important difference between assessment and evaluation. Assessment seeks to test learning in order to better direct it: find out what you need to teach, then teach it. The EQAO was established with this ideal in mind. Evaluation, on the other hand, seeks to draw conclusions and render judgments on the quality of the performance task: in this case, the test. This is how the tests are actually used. Originally, almost 18 years ago, they were going to be the first step in a complicated and flawed teacher effectiveness evaluation process.

The EQAO says the results mean nothing in the ranking of schools, as all schools in Ontario are delivering a quality education. Nonetheless, it publishes the ranked results every year, best schools to worst. The best -- surprise! -- tend to be in affluent areas and confer bragging rights, often precipitating a rush for optional attendance at the school. The worst -- surprise! -- tend to be in socio-economically disadvantaged communities. I teach in such a community.

Students and parents set great store by the results of this test. So do schools; at my school alone, tens of thousands of dollars have been spent to improve the comparative ranking of our school. This results in a hell of a lot of test-teaching. In Eduspeak, this is called "best practices." We are told to leave the Level 1s and 4s alone; just raise the Level 2 students to Level 3 work and the Level 3 students to 4.

Back to the marking site, where they are telling us to improve our daily tally of marked test booklets. Each test takes between five and 10 minutes to mark. By the fourth day here, I'm just skimming the tests looking for key words and going with my gut feeling. What Malcolm Gladwell called “Thin slicing,” is surprisingly effective. Other colleagues with whom I have furtive conversations also skim. Is the child obviously learning disabled? Give him a level one instead of NE1. Made you laugh? Level three. Real howler? Level four. I figure this is sabotaging a flawed process by removing some of its credibility. As long as statistical extremes are avoided, they don't really check your work -- they couldn't properly check the work of 3,000 markers even if they wanted to.

I haven’t mentioned how, last year, during a push to demonstrate how bad our teachers were still doing, we teacher/graders were told, even after almost a week of PD, that we were grading the tests to easy—to be more realistic, which most of us read to mean, be artificially heartless. Whereas yesterday, we were told, in the wake of a press release describing how under the comparatively new government, the test scores were demonstrably better, that we were told we needed to be a little less harsh, and to remember that they’re just kids. The site manager has been known to interrupt cellphone calls -- notably from a colleague who was in a tight bidding situation on a house -- to tell them to wrap it up quick or face the consequences. If, as another colleague did, you have a family emergency that requires you to take two consecutive days off, you are dismissed. It was the site manager who told a middle-aged teacher who was on her way to the bathroom to "walk with a purpose."   My principal instructed teachers not teach to the higher performing students or the lower performing students. Get it? Do you understand the implications here? We were instructed to focus less on certain children in our classrooms either because they will get it on their own, or they won't and we can't control for either situation. There is a bright independent future for the high performers--not much hope for the under-performers.

The site manager has just walked in, so it's time to wrap this up. I have already exceeded the site average for daily numbers by five booklets, so I can coast. Here is some math to make you think: 3,000 markers at $150 per day equals $450,000 per day, not including per diems, mileage or hotel expenses. Multiply by 15 days and you get $6.75 million. This doesn't include management, printing, processing and payroll costs or the huge dollars being poured into schools to improve their standing on the list. Put these figures against, say, running mothballed outdoor-education centres here in Toronto, for instance. Put these figures against the shafting that teachers just got at the hands of the government a couple of years ago, where we were forced to take 2 unpaid days off, and had all our accumulated sick-leave days stolen from us. This is what the Ontario Government has decided to keep spending money on; what parents love: meaningless numbers used to compare their children to those of others. Oh well. Back to the booklets. The things we do for money.