NY BATs Reflections of the Lederman v. King Trial

 

"Sheri G. Lederman has been teaching for 17 years as a fourth-grade teacher  in New York’s Great Neck Public School district. Her students consistently  outperform state averages on math and English standardized tests, and  Thomas Dolan, the superintendent of Great Neck schools, signed an affidavit saying “her record is flawless” and that “she is highly regarded as an educator.”

"Yet somehow, when Lederman received her 2013-14 evaluation, which is based in part on student standardized test scores, she was rated as “ineffective.” Now she has sued state officials over the method they used to make this determination in an action that could affect New York’s controversial teacher evaluation system."

http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/10/31/high-achieving-teacher-sues-state-over-evaluation-labeling-her-ineffective/

On August 12th several NY BATs traveled to Albany to support Sheri and her husband Bruce as they argued their case in NY Supreme Court.  Here are the reflections some of them had regarding the proceedings and a twitter story tweeted in real time as the case was being argued.

Jake Jacobs

Last Spring, Regent Judith Johnson told parents and teachers that APPR was “a lawsuit waiting to happen”. Signing a  “7 Regents letter” calling for suspension of the evaluation system pending expert review, Regent Johnson, a former principal and superintendent added that she had been looking to get her hands on "that formula" for a long time.

Today in NY Supreme Court, LI teacher Sheri Lederman and her husband Bruce argued NY's evaluation system used “irrational and unreasonable” actions, comprising several specific violations of statute:

- no one (apparently in any state) knows how students tests scores are converted into teacher ratings after the computer algorithms consider poverty, language and disability status. This lack of transparency violated the requirement that policies be “transparent and available” so teacher’s know what to do throughout to the school year in order to be successful.

- the use of a “bell curve” to deem exactly 7% of teachers inefficient is not just making the results “blunt and inaccurate” but isn’t in the language of the law

- the APPR model uses computer generated “prediction” to assign target scores, instead of basing teacher rankings on evidentiary fact

State tried to deny that their APPR formula was “a prediction” because it was based on actual students scored, but that “based on” becomes the key factor. The Lederman's call the process of turning the actual student test scores into a VAM sausage a “statistical black box”, responding to the complaint only by referring to their paid subcontractor’s reaffirmation.

In responding to the Lederman’s suit, the state representatives Ira Schwartz, Assistant Commissioner of Accountability and (?) Sherman, a "Quality Control” official did not provide affidavits from independent experts, rather they asserted that the Lederman's misunderstood the meaning of “growth”, providing language from promotional brochures. But the response also conceded that the policy was derived in pursuit of federal Race to the Top funding (another case of “Thanks Obama?”)

The judge teased this out of the state’s lawyer by asking so the students can perform well and the teacher be deemed ineffective? The Lawyer eventually answered “yes!” but tried to remind the court that the formula is only used as part of the overall evaluation. This caused the judge to ask aloud why “discordant” and “inappropriate” results would be used in any percentage.

He repeatedly asked the state’s lawyer if use of a bell curve was subjective, finally stating that the decision itself to use the bell curve meant the policy was subjective. The Lederman's felt they’d shown this was not only the use of discretion, but an unreasonable application of it, resulting in “arbitrary and capricious” results across, supporting the case through affidavits showing volatile shifts across various districts. The Lederman’s case was supported by several expert affidavits, and even included communications from Chetty et al group, the very researchers behind the practice.

In answering Lederman’s request to see the secret formulas, the Judge asked Mr. Lederman if he had researched NY case law for precedent on discovery for such policies. Lederman noted that multiple requests to the state and the third party contractor AIR who received in excess of $3 million to develop the secret formula went unanswered, leaving legal remedies the only option.

One supporting affidavit by Pallas that explains that even tiny downward growth measured against a perfect or high score in a prior year leads to disproportionally precipitous drops in the teachers growth score. The judge put it in practical terms, essentially asking how Lederman’s rating goes from an A to an F if her students grades went from an A+ to an A. In fact, Sheri’s state growth score went from a 75% to a 5% on the state measures portion, worth 20% of her overall evaluation and thereby dropping her ranking one whole notch lower.

State lawyer: The purpose is to identify “outlier” teachers by comparing student results over time relative to others “with similar histories” noting the system was “not perfect” but was “the best the state can come up with”, and could always be modified.

Deb Escobar

Lederman v. King et al

Plaintiff had an hour’s worth of valid argument on three main points:  (1) that the VAM computer model is “arbitrary and capricious.”  (2) that there is wide variance in scores from year to year, proving it is unreliable as a measuring tool.  (3)  This model must not be used until it can be proven valid.  He also argued that “there is no tool that can be finely enough refined to serve this purpose,” the purpose being a “rank and yank” style of management that has been widely discredited as a business model.  Another pertinent argument was that the plaintiff was not given information about the test and scoring at the beginning of the year, as required by law (defense disputed this and said teachers had access to the VAM model).

The fact that the state ignored plaintiff’s discovery request for raw data on the “statistically comparable” other students used for comparison with Sheri Lederman’s students was surprising.  Lederman intimated that these students could well be fictitious, and stated that “raw data should be part of the public record.” 

In comparison to Lederman’s pointed and constructed argument, the assistant attorney general did a minimal response to argument, defending the definition and use of the growth model.  The judge asked over and over how a teacher could go from a 14 one year to a 1 the following year.  No answer was really given except for a poor attempt to explain the model’s comparison to other students. 

Quote of the day – from Lederman – went something like this:  Are we living in a science fiction world where Hal the Computer gets to make decisions and there is no opportunity for human input or appeal?

My takeaway was that yes, the model is flawed (as we all knew); and, as it appeared from the judge’s questions, it was understood that the bell curve was in itself a subjective and flawed measurement.

I am in awe of the Lederman's, true heroes for all of the downtrodden teachers being judged by a flawed and unfair system of measurement.  If they win, all teachers and all students win.

Jeremy Dudley

Looking back on the arguments this morning one of the main things that stood out to me was the judge.  He seemed impatient at times and a bit frustrated as well.  He threw some tough questions toward Bruce and at times I recall feeling like he really didn't get it.  He asked what should be done in place of the current system and didn't seem thrilled with the answer.  It was when the state started their argument that I realized that the frustrated, antagonistic style of the judge was more about the way he conducts his business than it was about anything else.  He fired many tough questions at the state, and the state struggled to find the words to answer.  He demonstrated through his questioning that he saw a flaw in the system.  It gave many of us hope.

Another thing that stood out was just how deeply flawed the system is.  When a teacher like Sheri goes from 14 points to 1, yet her students are doing very well and meeting proficiency, it's easy to see that something is deeply wrong.  I believe everyone in the courtroom saw that today.  The argument that it's just a portion of an overall score doesn't matter to me.  If any part of it is flawed, the whole thing should not be used.  Let's hope the judge agrees.

Lastly what stood out was how much Bruce shined, and the state faltered.  Prepared and eloquent, Bruce laid out the arguments point by point and handled challenging questions with thorough and thoughtful explanations.  The same could not be said about the state's representation and it was wonderfully obvious.

Lorri Gumanow

I arrived after Bruce Lederman’s presentation, so I cannot comment on that part of the hearing.

I was very surprised by how seemingly unprepared NY State and Ms. Galligan were for this hearing. Ms. Galligan was not able to answer the judge’s request for clarification, even though he reworded the same question three times. He eventually gave up, and seemed pretty amused that she couldn’t give a coherent answer.

My overall impression was that the judge felt there is merit in this case. He had read all the affidavits offered by the petitioner, and his questions to the state were based on his understandings gleaned from these documents. He had difficulty comprehending that there could be any other form of measuring the achievement of children than the bell curve that could be reliable. He couldn’t understand that even if a student achieved what would be equivalent to a 95 on a test, that that score could be equated to a 1 when rating a teacher. I feel the same way.

While in the courtroom, I noticed many of the attendees nodding their heads as the judge asked his questions, seeming to agree with his confusion at the answers given by Ms. Galligan. I felt that the judge was leaning towards supporting Dr. Lederman’s petition, given the lack of explanation from the state that such an APPR rating was reliable and valid.

It was exciting for me to witness this hearing, and I feel that the outcome of this case could be very historic in our fight to save public education.

Luz Christina Mooney

One of the roles I fill in my district includes being a mentor teacher. As a result, I see firsthand how  teachers are affected by educational reform.  Since it is NYSED that applies the labels of effective and ineffective, they carry a sense of authority that destroys morale. Mr. and Mrs. Lederman's bravery inspired me to make the drive and be a witness to history in the making. Below are some of my takeaways.

The state claims Mrs. Lederman's score was valid because her student’s scores were compared to other students throughout NYS, of similar demographics. Mr. Lederman argues the state fails to show proof of how the state calculates this. Furthermore, NYSED is making decisions based on a computer program's prediction, instead of common sense, and real growth. Mr. Lederman argued the rating system was actually breaking the law because it did not allow the teacher to earn every possible point, as the law specifically states.

A  few things left me feeling as if the judge understood the issues. The judge seemed well informed from his line of questioning. My favorite was when the judge asked why MUST a test be part of the evaluation score at all?

The state continued to argue the rating computer system was valid. NYSED admitted that you can have a teacher whose kids are successful yet the teacher will still get a low rating. Overall, they seemed okay with that reality. The state explained the evaluation system will let them get rid of 'outliers'.

After the proceedings I was moved to tears when The Lederman's shared their motivation. Mrs. Lederman's evaluation had her distraught, she was ready to quit and leave the profession that she loves.  After many late night discussions, she and her husband decided, to challenge this unjust system. They certainly know they have a world of educators and parents on their side.

Gail Richmond

* It was encouraging to have the public see for themselves that even the state could not explain the impact of test scores on teachers.

* The common sense scenarios as presented by the judge made it clear to everyone in the courtroom just how ridiculous VAM is.

* Even if Sheri Lederman does not win light was shed on how teachers of excellence will be fired in the coming few years.

Marla Kilfoyle

Many things struck me as interesting in this proceeding:

1.  At first when Bruce Lederman was methodically going through his arguments the judge seemed bored and disinterested.  At one point I thought he was going to fall asleep and my hopes were falling but an amazing thing happened….It seemed to me the judge GOT IT!  He seemed to be well read and assured Bruce that he had read the “expert” submissions on behalf of Sheri Lederman (Linda Darling-Hammond and Carol Burris submitted expert testimony on Sheri’s behalf)

2.  In my mind Bruce’s arguments for Sheri were simple; that the computer model NYSED uses to figure growth scores is “arbitrary and capricious.”  The other argument Bruce used that was brilliant is the ability for humans to question the computer generated math.   Bruce often referred to the statistics used to figure a teachers growth score as a “statistical blackbox.”   The Lederman’s questions to AIR, the firm that contracted the “math formula” went unanswered.  Thankfully, in my mind, the judge  seemed to get this.

3.  The State then argued that Sheri’s growth model scores did indeed put her at the bottom of the state prediction.  The judge proceeded to question the State on the idea that Sheri went from a 14  to a 1 in her growth score.  He wanted to know specifically how that happened.  The state had NO response but to refer to the AIR saying that the math formula was indeed accurate.  The state also had NO expert witnesses and as I recall had 2 affidavit’s from officials who worked for the state saying the model was valid.  When finally pressed by the judge for close to an hour, the state finally stated that model was not perfect but the best that the state could come up with.   The judge was also not impressed that the Bell Curve used to deem 7% of teachers ineffective was NOT objective but subjective. 

I am biased, for sure, but I have to say Bruce and Sheri Lederman have done an amazing job laying out the faults of this system.  They have lined up the best in the field to validate their claims.  My overall feeling was, that as gruff as this judge seemed to be, he got it.  He understood that this system is not transparent, is not valid, and should not be used to judge teachers.   We hope that his findings, which will be rendered in about 6 weeks, will state just that. 

Here are some tweets sent out in real time as the proceedings were going on