L. A. Times uses value-added analysis to rate public school teachers

Public school students are graded and tested all the time. Schools are scored too — California rates them in an annual index.

Not so with teachers.

Nationally, the vast majority who seek tenure get it after a few years on the job, practically ensuring a position for life. After that, pay and job protections depend mostly on seniority, not performance.

That’s from The Los Angeles Times, which recently published a fascinating article about evaluating teachers (read it here). They used a statistical method known as “value-added analysis” to rate teacher effectiveness in Los Angeles public schools. They explained that, “Value-added analysis offers a rigorous approach. In essence, a student’s past performance on tests is used to project his or her future results. The difference between the prediction and the student’s actual performance after a year is the ‘value’ that the teacher added or subtracted.”

The Times obtained seven years of math and English test scores from the Los Angeles Unified School District and used the information to estimate the effectiveness of L.A. teachers — something the district could do but has not.[…]

Among the findings:

  • Highly effective teachers routinely propel students from below grade level to advanced in a single year. There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. The fortunate students ranked 17 percentile points higher in English and 25 points higher in math.
  • Some students landed in the classrooms of the poorest-performing instructors year after year — a potentially devastating setback that the district could have avoided. Over the period analyzed, more than 8,000 students got such a math or English teacher at least twice in a row.
  • Contrary to popular belief, the best teachers were not concentrated in schools in the most affluent neighborhoods, nor were the weakest instructors bunched in poor areas. Rather, these teachers were scattered throughout the district. The quality of instruction typically varied far more within a school than between schools.
  • Although many parents fixate on picking the right school for their child, it matters far more which teacher the child gets. Teachers had three times as much influence on students’ academic development as the school they attend. Yet parents have no access to objective information about individual instructors, and they often have little say in which teacher their child gets.
  • Many of the factors commonly assumed to be important to teachers’ effectiveness were not. Although teachers are paid more for experience, education and training, none of this had much bearing on whether they improved their students’ performance.

I highly recommend reading the whole article. A question for teachers and administrators who read my blog: what do you think of value-added analysis, and using this as a tool to evaluate teachers? The article concedes that it should not be the sole method of evaluation, but suggests that it would be beneficial if it made up 30-50% of a teacher’s review.

195 thoughts on “L. A. Times uses value-added analysis to rate public school teachers”

  1. At first glance–this sounds like a very expensive evaluation tool–but I guess no more expensive than student testing.
    If teacher evaluations are to be based on student test scores, I would think that there should be some incentive for students to do well on tests. There are no consequences to students for low scores on NCLB tests and the reading and math tests currently given in 150. I have seen some of the results of these math and reading tests given in the fall and again in the spring–a significant number of the scores drop considerably for the second test. If teachers are to be evaluated on the basis of these tests, I would hope the students had reason to do their best.
    Also, I believe a students’ attendance rate should be considered. After compiling the attendance records for Manual’s seniors (which didn’t surprise me), I wouldn’t think that the teachers should be held accountable for what the students didn’t learn on the days they were absent.

  2. The American Federation of Teachers is pushing for valid means for measuring teacher effectiveness. The AFT is concerned with protecting the teaching profession — ensuring only people cut out to teach become teachers. You will find (if you explore the AFT’s literature regarding evaluation systems) that the push is for higher expectations and the highest possible teacher performance — not for blanket tenure based solely upon the number of years a given educator has been teaching.

    Locally, our union, the Peoria Federation of Teachers, has been pushing for a new evaluation system for around a year. The current evaluation system is absolutely useless and needs to be thrown out. I don’t think you’ll find many teachers who disagree with that statement.

    We have been exploring the Danielson model (http://www.danielsongroup.org/) — which clearly defines good teaching and, further, gives a valid means for measuring good teaching. We have been exploring peer evaluation as well. We have been kept up to date on out of the box ideas regarding various forms of merit pay by Dr. Patrick Dolan — who is an expert in school restructuring. (One interesting method involves giving every employee in a given school building a raise if students’ test scores go up — everyone from the janitor to the lunch lady to the secretary to the teachers to the principal. I’m not sure how I feel about this particular method . . . but it is interesting . . . and something to ponder. I imagine that any slackers would be summarily beaten to death in the parking lot. 🙂 )

    Although it’s not my place to go into specifics, I do want people to know that the AFT, the IFT, and the PFT aren’t fighting for blanket tenure for people who can’t teach. Actually, quite the opposite is true. We need a system that will push our good teachers to become great and our great teachers to remain great . . . a system that will, at the same time, allow for summarily counseling out people not cut out to be teachers.

    Again, check out the AFT’s stance on this issue. Here’s one link:

    http://www.aft.org/newspubs/press/2010/030310.cfm

  3. At second glance–and I did read the article quickly: Two statements impressed me.
    “Each student’s performance is compared with his or her own in past years, which largely controls for outside influences often blamed for academic failure: poverty, prior learning and other factors.
    This is certainly a better basis for evaluation than is the NCLB that pits one junior class against the next year’s junior class, for instance.
    This next sentence answers the question about attendance, etc.
    “Any single student’s performance in a given year could be due to other factors — a child’s attention could suffer during a divorce, for example. But when the performance of dozens of a teacher’s students is averaged — often over several years — the value-added score becomes more reliable, statisticians say.”
    The article mentioned that one principal (I think in Washington, D.C.) who made headlines recently fired 26 teachers on the basis of this evaluation. I’m not sure how she acquired the statistics “over several years” to determine these teachers were failures. Also, I can’t accept that test scores alone should be the deciding factor. But it does seem to have some merit as part of an evaluation.

  4. I was so pleased read the Sunday article about how teachers in low or socioeconomic disadvantage neighborhoods are making positive changes when doing what is necessary for the students to learn. I was a product of Los Angeles Unified School district, and I am now working as a substitute teacher.
    I know that the teacher can have the most positive influences on academics when they differenciate instruction to meet the needs of all their students. I do know that it is effective but not at all easy to ensure that each students has complete access to all curriculum, but it is what is necessary for academic success to be made.
    As someone that has had numerous long-term assignments in elementary schools, I know that the students that i was responsible made progress with me as the teacher because I was extremely organized, used all the latest professional materials that were available to me, and made sure that I collaborated with collegues on grade level, and was prepared with data driven lessons for the curriculum, and made sure I did planning well before the daily lessons were taught preparation in place before the students arrived for class.
    I find it amazing that teachers often are perplexed as to why their students are not progressing during the school year when they have not utilized professional data to drive their instruction and are not using authentic instruction and higher level lessons to allow high expectations to be in place at all times.

  5. With all due respect Mr. Adkins-Dutro, while I’m not in the education field, I find it very difficult to not question the strength and value of an evaluation mechanism such as that in the Danielson literature which suggests “(b)y shifting the focus of evaluation from “inspection” to “collaborative reflection” educators can ensure the maximum benefit from the evaluation activities.” Perhaps it does, but it certainly doesn’t jump out at me. To me, the Danielson materials seem to provide more in the way of potentially improving one’s teaching abilities rather than a measuring device concerning such.
    Value-added assesments at least seem to provide a more objective basis. I certainly don’t think that value-added should be the sole determinant but it appears to provide a pretty significant starting point.

  6. If schools are a factory and children the raw material that is shaped and processed into a product we call students, and then that student is packaged and sold to the market as an employee… THIS IS A GREAT WAY to evaluate the system.

    We love the business model… it works for everything… just don’t forget to slap a price tag on it!

  7. Yeah . . . we’re not 100% sold on Danielson . . . It is one of the better thought out evaluation systems, however. Your point is well taken . . . and I anticipate such arguments / points being made by teachers and administrators as we work towards a new system for evaluation. The biggest positive (in my opinion) is that we’re going to do away with the old and try to bring in something new and much, much more relevant / meaningful.

  8. adkinsdutro: WHAT? You mean we are getting rid of the current evauluation tool? That one being “the biggest principal suck-ups get excellent evaluations even though they stink as educators”. I have worked for more than one principal who gave out excellent evaluations to the “teachers” who did her bidding and told HER how fab she was on a daily basis…..NOT. It’s about time for a REAL tool for evaluating teachers, not based on asskissing, but educating children.

  9. Just found this out from a posting on EmergePeoria’s blog. I was able to confirm it with Friendship House.

    “Dr. Lathan and the City of Peoria are scheduled to appear at a Community Forum at the Peoria Friendship House (800 NE Madison) this Wednesday, August 18 @ 6:30 p.m.. The purpose of the forum is for East Bluff, near-north siders and Averyville residents to get to know the superintendent and ask school-related questions.”

    I talked to Jennifer at Friendship; she has some prepared questions based on concerns by parents; Tim Riggenbach is scheduled to be there as well.

  10. I was just hearing a bit more about this evaluation process on CNN. Now I have further questions and/or opinions. I can understand how primary school (maybe even middle school) teachers can be evaluated by student standardized tests scores. However, I’m not sure how it will be done at the high school level. English teachers and maybe math and science teachers could be evaluated in such a way, but there are many subject matter classes for which reliable (if any are) standardized tests are not available. Even science or math has many divisions (algebra, geometry, chemistry, biology, etc). For instance, some teachers teach only biology. I believe NCLB tests would cover all science courses in one class–not biology, physics, etc., as separate tests. Even now social students teachers rarely get too much flack about kids failing the NCLB test because their subject isn’t tested. The same goes for foreign language, business courses, and on and on. How will all those teachers be evaluated?

  11. Why should teachers be held responsible for standardized test scores that decline after administrators change the curriculum and refuse to change it back (4, 5, 6 years later)? It really doesn’t take a mental giant to figure this stuff out (IMO). D150 doesn’t have room for those who let the teachers, kids, and parents take the heat because they lack the ability to own up to mistakes and make corrections.

  12. The usual invitation: District Watch on Sunday, August 22, at 6 p.m. at Monical’s on Knoxville and Lake.

  13. District 150 board packets–Until today I have never seen one of these packets (available to the public upon request). However, I believe there is much missing–I can understand that, however… There are two letters addressed to the board; one entitled “Civic Center Facility Lease Agreement” and the other entitled “Civic Center Banquet Contract.” Neither of these letters reveals the price tag (for the civic center lease, if there is one, or for the food). I can understand why that information might not be in the “public” packet, but I certainly hope that board members are able to think about the cost ahead of time before showing up to vote on what appears to be a done deal. Two things that I believe should be in the board packet: the organizational chart and the information about the $6 million grant that Manual will receive (both mentioned earlier by a board member on Emerge’s blog).
    These Civic Center events are not common practice. To my knowledge, only Dr. Royster held these events. Personally, I don’t believe now is the time to let taxpayers know that taxpayer money goes for this purpose–to feed District 150 employees. I am very willing to bet that teachers would gladly forego the free breakfast to spend the morning in their classrooms getting ready for students to arrive. Peoria High teachers have not been able to get into their rooms because of construction. Many teachers, new to the building, surely would welcome any time available to put up bulletin boards, organize supplies, etc. I believe not giving them this time is insensitive to many people who are trying to adjust to the closing of Woodruff.

  14. So far Edgar, I have not revealed teachers’ scores on any test–I have only revealed the test scores of students. (Just joking; I get your message, I think). At the last BOE meeting I “revealed” student attendance records of last year’s Manual seniors. They are horrendous. I do not believe that teachers can or should be held responsible for what a student doesn’t learn when he/she is absent. I know that there is very little we can do to make parents accountable for their children’s performance at school; however, I believe they can and should be made responsible for seeing to it that the children are in school (yes, even high school students). That would be a start at leveling out the accountability factor. Then maybe I wouldn’t consider it to be so unfair that teachers take all the blame for test scores.
    Right now all teaching is geared to that one test–NCLB, a test for which students suffer absolutely no consequences. There is so much “learning” that can never be measured by a test–that kind of learning just isn’t happening any more in most American schools. We are too busy measuring.

  15. So far Edgar, I have not revealed teachers’ scores on any test–I have only revealed the test scores of students. (Just joking; I get your message, I think). At the last BOE meeting I “revealed” student attendance records of last year’s Manual seniors. They are horrendous. I do not believe that teachers can or should be held responsible for what a student doesn’t learn when he/she is absent. I know that there is very little we can do to make parents accountable for their children’s performance at school; however, I believe they can and should be made responsible for seeing to it that the children are in school (yes, even high school students). That would be a start at leveling out the accountability factor. Then maybe I wouldn’t consider it to be so unfair that teachers take all the blame for test scores.
    Right now all teaching is geared to that one test–NCLB, a test for which students suffer absolutely no consequences. There is so much “learning” that can never be measured by a test–that kind of learning just isn’t happening any more in most American schools. We are too busy measuring.
    At the BOE meeting, I did make light of the high report card grades students were given. I always want to make it very clear that I believe that all manner of “methods” are used to insure that grades are higher than teachers would choose to make them. You rightly question the fact of retaliation for not going along with the program of administrators. Manual has had so many first year teachers–this is their first experience of teaching and they wanted good records and a chance for a second year.

  16. PHS won’t be ready…..and even if it is ready, I’m not wrong – they are just being insensitive, but if they are sensitive, I’m not wrong, again. Hold it…..I feel as if I’m about to pass another complaint…….ahhhh much better.

  17. Sharon,
    There you go again slamming Kay Royster. Both times when the Civic Center was used to introduce her to all district employees and then the second year for a welcome back was done with money and supplies in kind donated from the business community and vendors. No taxpayer dollars were involved!

  18. Sharon–there is a lot I want to say before the start of school, but first I have to do a FOIA and follow proper channels (regarding testing and test reporting). Thank you for asking the tough and obvious questions. You are helping a lot of parents, students, and teachers who can’t take the risk or heat of stepping out.

  19. TR64, that is my intent–to put some information out there that everybody should know. As I have explained to some board members, for 43 years I was in the dark about so many things in District 150 because there was always so much swept under the rugs. Now we have access to information–and that access and exposure should be able to create a better district.
    Without malice, I don’t think I was putting Dr. Royster down–and if I didn’t mention it on this blog, I did mention on Emerge’s that Dr. Royster’s event didn’t cost the district anything. Also, I was totally in awe and impressed by Dr. Royster at that first event–and thought the idea was a great innovation. I wish I could have maintained that feeling.

  20. Elaine is discriminating on her site by not writing up the comments of Sierra. She also doesn’t think comments offered by a teacher speaking to the merits of collaboration and interaction with administrators is a “highlight”…..hmmmm.

  21. The speaker advocating the merits of collaboration and interaction with administrators is now an administrator, so, of course, he interacts with administrators. However, all of our speeches can be heard on Elaine’s PeoriaStory.com. Remember that District 150 doesn’t value the ideas of any public speakers as they are no longer televised. The district doesn’t even believe that the public should have immediate access to their decisions, opinions, etc., as the public can only watch the meetings a week later. Who is discriminating? I do agree that Sammy represents the southside well.

  22. Sharon- I am not sure he is an administrator, but he is not listed as teaching in any of our schools in the district as of this posting for the upcoming 2010-2011 school year.

    So in an effort to serve the union he,Mr. Darling and anyone else not in the classroom should excuse themselves from union dealings and union membership per the PFT constitution.

    Some people may take this as trying to divide the union, however, it is an effort to pose a question from our constitution that hopefully will unify those that are truly union members and whose postitions are typical of the constituency of the PFT.

    Time to look out for the good of the many and not just for the few. We shouldn’t expect any less!!!

  23. Skeptical1, thanks for clarifying–I meant to make the distinction, also, that whatever his title, he is not in a classroom and he does spend his days on Wisconsin Avenue, I believe. I do not think ill of him or Bobbie, at all. However, I wish they would both understand that being true to the constitution does require they step down now that they have new positions with the district–which, of course, could be considered as having a conflict of interest.

  24. I see that Illinois has lost its bid for Race to the Top–I guess we can debate whether or not that is a win or a loss for District 150.

  25. Sharon- I only want those that are showing a perceived conflict of interest(which is defined in the constitution) to be honest and own up to it by doing what is right and remove themselves from the union or return to the classroom and follow the constitution of the PFT.

    The statements like,”Be glad you have a job.” does not support the union members (to which there are many) that this statement has been directed at in the PFT nor the PFT constitution.

    Sharon thanks for all that you, Terry and the District Watch do for our kids and community to get the best out of a less than honorable district.

    Change is greatly needed!!!

  26. skeptical – with all the “change” at hand, please share what it is you want to see? Shorter days, fewer students, and a bigger check? That is part of what got us into this mess. Thanks to Terry, Rhonda Hunt and her fellow board members…..

  27. Change is at hand (I admit it sounds like a Stowell cliche–is that you, Jim)? I have gone to the podium regularly at board meetings in the last few months. I have called for change. However, never once have I asked for shorter days for children or a bigger check for teachers. I did ask the district to take an honest look at class size before they closed Woodruff. There are signs now that District 150 will have fewer students as a result of the closing of Woodruff and the opening of a charter school (and, therefore, less state per pupil aid). We hear that several have already enrolled in Pekin High (high school students that may have younger siblings, too) and some star athletes playing on the East Peoria football team.

  28. Can someone please explain WHY the PFT Union President, the PFT Union Secretary, and a PFT member are ALL working in administrative positions at the District 150 office? The PFT Union Secretary is now in charge of the Universal Leadership Team for the District’s teachers, doesn’t that make her an administrator. the other PFT member is working in the HR department, doesn’t that make him administration? The PFT president’s office is also at the administration building. What gives? I thought they were supposed to be TEACHERS to hold those positions in the Union. Again, who are they working for? Remember, the teachers of D150 pay HUGE union dues…..not much bang for the buck since Bobby got elected. Funny, the one person who did stand up for the teachers had her adult education classes ELIMINATED and was returned to a classroom. Oh, what tangled webs we weave, when first we practice to DECIEVE!

  29. Just got email from Stacey Shangraw, Dist. 150 PR director…The Dist. 150 staff meeting at Civic Center next Monday, August 30th is a staff meeting, so it is not open to the media. No more than 2 board members will be present at any one time.

    Bummer, I would have loved to video Dr. Lathan’s rallying speech. I
    am sure it would be inspirational to not only teachers, but perhaps
    parents, if they could view it.

  30. I would think Dr. Lathan would want the public to know she is reaching out to them as well–especially, when the event is to be held at such a public place as the Civic Center. If it had been at Richwoods, I would find “no media” easier to understand.

  31. We’ll see what she has to say to parents and public when she appears on our CAPtions show. Taping is scheduled for Sept. 22nd, and airing 1st Sunday in October.

  32. Her Highness Lathan doesn’t want any coverage but her coverage. Some of you still just don’t get it that she is not the miracle worker you seek. In three years she has been in North Carolina, Chicago, San Diego and now Peoria. What makes you think that Peoria is going to be the final place?

  33. Did they, Jesus didn’t stay in any one place long when he was doing his work. Just sayin.

  34. I think when you speak to staff, the media shouldn’t be there. Just my opinion.

  35. Could it be that Dr. Lathan may be a bit camera shy? It seems that just before she took over, the BoE meetings were no longer televised live and she doesn’t seem to ever speak on camera to the news media. Just asking.

  36. District 150 Observor–to how many staff people have you spoken? Does “staff” include teachers. I can’t imagine why anyone would object to the media being present.

  37. Is the media present at City of Peoria staff meetings? More to the point, is the media present at the staff meetings of any other taxing body?

  38. I am wondering why the press (being what it is by definition) wouldn’t consider the lack of an invitation to be an invitation to show up as close to the event as possible.

  39. Emerge, you really can’t equate this event at the Civic Center with a staff meeting of any sort, can you? Why do you think that no more than two board meetings will be present at a time? That measure is being taken to insure that no one call this a “meeting.” This is more a social event for which I still want to know the cost to the taxpayers. Of course, Dr. Lathan doesn’t have to invite the press, but I think she is making a very good PR mistake if she doesn’t. It certainly speaks volumes as to her lack of desire for transparency. Dr. Lathan (by tacit agreement and not speaking out against the policy) and the board have already made it clear that they do not want portions of board meetings to be made public and made it clear that they do not want the public to have immediate access to what happens at board meetings. I don’t believe that can be construed as putting one’s best foot forward.

  40. To “truth”

    You are seriously comparing Lathan to Jesus Christ? You have been baptized in the Koolaid tank, haven’t you?

  41. Sharon that is a good point that her not wanting the media at the Civic Center is right in line with delayed airings of school board meetings.

  42. My error: than two board meetings will be present (of course, I meant “board members.” and good PR mistake – I meant “big” mistake–I didn’t mean to imply that there is ever a “good” mistake. 🙂 I need a proofreader

  43. Did they, no I was just presenting counterexample of somebody that did not stay in one place long and still did some great things in each place that He was at. I have not drank flavorade, but I also have not sharpened my pitchfork as it seems you have.

  44. • Sharon, the notice to employees, on the employee section of the D150 website, called it a Staff Meeting. It was not publicized to the public – it was posted on the D150 website in the employee section, under staff news.

    Sounds like it’s a Staff Meeting to me.

  45. I believe Pam Adams was at the first all school meeting when Royster was the supt. She even was able to ask questions, afterwards wrote a story for the Pjstar about it. I WAS THERE>
    PS: Emerge, it is an energizer…..not a staff meeting.

  46. Emerge, Dr. Lathan can call it whatever she wants to call it but the truth is that it is an event to which the media should be invited because it is being held at the Civic Center. I will be very, very surprised if the media doesn’t show up anyway. Coco is right–the media did attend Dr. Royster’s event because they were invited–she set the precedent and now Dr. Lathan is saying that “behind closed doors” is the way she wants to operate even for something as innoxious as a “meet and greet.”

  47. Since more than 2 board members will be present, I believe it will be posted Friday as an open meeting. Sharon – Dr. Lathan is “saying” no such thing as it relates to “how she wants to operate”. If she has stated that directly, please provide details and context so I might apologize. Glad to know you now purport to speak for her though:) (Does she know?)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.