L. A. Times uses value-added analysis to rate public school teachers

Public school students are graded and tested all the time. Schools are scored too — California rates them in an annual index.

Not so with teachers.

Nationally, the vast majority who seek tenure get it after a few years on the job, practically ensuring a position for life. After that, pay and job protections depend mostly on seniority, not performance.

That’s from The Los Angeles Times, which recently published a fascinating article about evaluating teachers (read it here). They used a statistical method known as “value-added analysis” to rate teacher effectiveness in Los Angeles public schools. They explained that, “Value-added analysis offers a rigorous approach. In essence, a student’s past performance on tests is used to project his or her future results. The difference between the prediction and the student’s actual performance after a year is the ‘value’ that the teacher added or subtracted.”

The Times obtained seven years of math and English test scores from the Los Angeles Unified School District and used the information to estimate the effectiveness of L.A. teachers — something the district could do but has not.[…]

Among the findings:

  • Highly effective teachers routinely propel students from below grade level to advanced in a single year. There is a substantial gap at year’s end between students whose teachers were in the top 10% in effectiveness and the bottom 10%. The fortunate students ranked 17 percentile points higher in English and 25 points higher in math.
  • Some students landed in the classrooms of the poorest-performing instructors year after year — a potentially devastating setback that the district could have avoided. Over the period analyzed, more than 8,000 students got such a math or English teacher at least twice in a row.
  • Contrary to popular belief, the best teachers were not concentrated in schools in the most affluent neighborhoods, nor were the weakest instructors bunched in poor areas. Rather, these teachers were scattered throughout the district. The quality of instruction typically varied far more within a school than between schools.
  • Although many parents fixate on picking the right school for their child, it matters far more which teacher the child gets. Teachers had three times as much influence on students’ academic development as the school they attend. Yet parents have no access to objective information about individual instructors, and they often have little say in which teacher their child gets.
  • Many of the factors commonly assumed to be important to teachers’ effectiveness were not. Although teachers are paid more for experience, education and training, none of this had much bearing on whether they improved their students’ performance.

I highly recommend reading the whole article. A question for teachers and administrators who read my blog: what do you think of value-added analysis, and using this as a tool to evaluate teachers? The article concedes that it should not be the sole method of evaluation, but suggests that it would be beneficial if it made up 30-50% of a teacher’s review.

195 thoughts on “L. A. Times uses value-added analysis to rate public school teachers”

  1. Dennis, I agree. Don’t you think parents and the public, also, deserve the right to have immediate TV access to board meetings and even the public comments (two sides of an issue–maybe three, four, etc.)? About the money–so far most of us are not convinced that the closing of Woodruff and the reorganization of administration will result in any cost savings. Also, I believe that any savings from the closing of Woodruff (which was estimated to be only 1.5 million) will be spent $20,000 at a time on questionable expenditures such as this event.

  2. Can’t help wondering how long it will be before we are asked to donate copy paper and other school supplies. That is why I question the wisdom behind the Civic Center event. I sure hope Dave FOIAs the true cost afterwards…just to be sure the estimates were reported accurately.

  3. At one school the kindergarten supply list already asks for 1 package of copy paper, 4 boxes of Kleenex, 2 dry erase markers, 5 bottles of hand sanitizer, 3 Ziploc bags, 4 packs of Lysol wipes, paper plates, 5 boxes of crayons, 2 12-pack boxes of pencils, 4 Elmer’s glue (besides the usual personal items for a student). All of these items are undoubtedly needed in the classroom. If parents don’t pay for them, teachers will have to do so. As a high school teacher, I needed some of these items (or items more appropriate for my high school classroom), but I would never have been able to put them on a list for parents to buy–and District 150 certainly didn’t buy much of anything for my classroom. I know we talk about it often, but I doubt that anyone would believe how much of a teacher’s own money goes into classroom supplies. I think it would be great if the union would ask teachers to tally all their expenses for one year–I believe the amount spent would be staggering. I think the district should then be embarrassed by some of the money they spend on programs and “events,” that have little or no impact on the actual job of teaching–the only purpose for the existence of schools.

  4. Well, for those of you that attended the Civic Center event, do you think it was a frugal investment?

  5. It looks like the “journalists” that are pro-Lathan no matter what didn’t have any problem getting coverage on the civic center meeting.

  6. Donuts and muffins……coffee, tea, o.j. Can’t see how that stuff cost $20,000.

  7. After sitting and listening today, all I have to say is WOW! I hope she is a better leader than a speaker. We are in trouble!!

  8. Unforunate that our new leader preaches change and one of her “respect’s” was to respect finances, yet she ok’d spending funds on today’s event. She then was tight-lipped on the subject with WMBD-31.

    It seems we may be in for the same leadership we have had in the past… with just a change in faces and names.

    Maybe we should change the motto to… “We want change, but it will cost you and we don’t have a plan.”

    This would at least be honest!!!

    Remarkable!!

  9. UMMn…the line about status quo is getting a little tired. (OK–A LOT)

    R.E.S.P.E.C.T.–r u prepared to deal with your reality?

    The district doing the things that need to get done–vague and broad–PLEASE be specific:>

  10. She preaches respect, which is a very good thing, but did anyone see how rude she was to the reporter on Fox 43 when she was questioned about the cost of the Civic Center meeting?

  11. I was surprised by her snarky comment to the reporter. Itwas very awkward and left me with a very bad impression of her. If you cant’t stand the heat get the heck out of the kitchen. I predict a major public meltdown in the near future.

  12. “respect the customer” ???

    Okay… now we got 6 sigma voodoo in 150.

    Students are not customers. Parents are not customers.
    ‘someone who pays for goods or services’
    wordnetweb.princeton.edu/perl/webwn

    “Education is not a commodity to be bought and sold.
    There is a dangerous tendency to assume that when people use the same words, they perceive a situation in the same way. This is rarely the case. Once one gets beyond a dictionary definition—a meaning that is often of little practical value—the meaning we assign to a word is a belief, not an absolute fact. Here are a couple of examples.”
    http://www.teachersmind.com/education.htm

    We already know what Grenita thinks ‘education’ is… it is why she was hired; “workforce education”. She believes providing Caterpillar, Walmart and Lone Star Steakhouse employees is the purpose of education. (And of course, to provide her and her a friends a fat salary)

    AS W.E.B. Dubois said, the function of education is not to make carpenters out of men, but to make men out of carpenters.

    We have plenty of carpenters in this society…we need more men (and women).

  13. Charlie, you beat me to the punch–I was preparing these comments in my mind while I was getting my day started. I cringe whenever I hear students called “customers.” If administrators insist on continuing this comparison of education to business, then the comparisons should, at least, make sense. The customers are always the ones with the money to buy a product. Taxpayers are the customers who are paying to educate children in their community. Parents are customers only in their capacity as taxpayers, not as parents per se. Also, children are not products. The education offered them might be the product (if we have to continue this ridiculous analogy based solely on money). Remember the cliche “the customer is always right” is a dangerous cliche if we consider students to be the customers. Children do not know what is best for them–any good parent knows that.

  14. I think what she said makes sense. Aren’t public school districts in the business of educating children?

    I am a customer of District 150 because that is where I choose to send my child. I have the option of paying private school or home schooling. I choose to give them my business and my dollars (there are fees beyond taxes required to attend a public school).

    As a customer I always appreciate the attitude a business may have of giving good and respectful customer service, it doesn’t matter if it’s a school or a grocery store. If I don’t receive good service, I won’t patronize your place of business.

  15. I am kind of confused here as to why WMBD-TV 31 is the ONLY media reporting that $8,000 of the overall cost came from the sponsors listed on the banner at this rally. Where is the usual PJS top notch reporting? NEWS 25 didn’t have these numbers either.

  16. Ironic, Charlie, that you

    1) post a dictionary definition of “customer” and then
    2) post a link describing that “Once one gets beyond a dictionary definition—a meaning that is often of little practical value—the meaning we assign to a word is a belief, not an absolute fact.” and yet
    3) attempt to describe what you believe Lathan means by both “customer” and “education”.

    Of course, “customer” does not simply mean someone who “pays” or has “money to buy”.

    Here’s another definition of “customer” from a business dictionary:

    “1. General: (1) Entity that receives or consumes products (goods or services) and has the ability to choose between different products and suppliers.”

    http://www.businessdictionary.com/definition/customer.html

  17. Emerge and Jon, yes, educational programs and schools have, unfortunately, become a commodity to be sold through false advertising. Buyer beware! Educational programs and schools are now packaged with the same tactics used to present all manner of products to the public. And we all know how honest these advertisers are! The secret, of course, is to have schools like Edison (public) and Peoria Academy (private) where the “product” can choose its own “customers.” Yes, Emerge, you have a choice if you can afford to buy the product–the public schools are losing money right and left because everyone who can afford it is buying the more expensive “products.” In truth, the products (educational programs) seem better because the schools get to choose the students who will use the products. Read Ravitch’s book–you will find that you aren’t a customer but a pawn in this new money-making game of education.

  18. Dennis, have you posted the speeches on YouTube yet? Also, I am not sure that I am impressed to learn that another “not-for-profit” institution, Methodist Hospital, paid for part of yesterday’s event.

  19. Sharon, I will have it posted later this afternoon. I narrowed down the main segment of her speech, which is about 15 min. long.

  20. So, Sharon, how is Manual, for example, choosing the students who use the Johns Hopkins product? Who is being turned away and where are they going? What explains the double digit increases in reading and math scores?

  21. Sharon, a better example of a pawn would be an entire industry of folks who are forced to join a group and pay dues – a teacher’s union. I’m thinking you and your colleagues are the pawns in this game of education.

    Is there anything about that in Ravitch’s much ballyhooed book – if so, I just might read it.

  22. Much overanalysis taking place over this “customer” business. Either the new super will get the job done here or not. Parsing everything she says isn’t exactly constructive.

    She could have handled the 31 interview better, but that is just noise. Let’s give her a chance to succeed before we rip apart every move.

    It is exactly this kind of micro-criticism that causes leaders to shy away from the press and the public. That is just reality—these people are humans and get stung by 24/7 criticism over everything they do.

  23. Jon, I am not sure how Manual achieved the double digit increases in scores. I know that I have been told that the students (those in that middle group who can make or break the test scores) were pulled out of classes often to “teach to the test.” Maybe you approve of that method, but I don’t. The draw (if there is one) for students to choose Manual (with or without Johns Hopkins) might be the inflated grades. With 61% of senior grades being As or Bs, I imagine many students and their parents honestly believe these scores to be objective–that can be a draw to Manual.
    Jon, I do know that some former Woodruff students who were assigned to Manual found a way out of Manual. Of course, there might be some assigned to PHS who found a way to Manual. I can almost guarantee that Johns Hopkins had very little to do with any of those choices. Athletics and friends were probably the decision makers.
    Emerge, think what you will; teachers are probably going to be the pawns in anybody’s education game these days–might as well go with the union that promises some protection. However, when four of the current leaders of District 150’s union now hold administrative positions, the union might not be the union. Frankly, I think the administration has come up with an interesting, even clever strategy to break 150’s union. I wonder how many more administrative jobs are available for union leaders. Emerge, how many people do you know who are in a union and want out of the union?

  24. “Much overanalysis taking place over this “customer” business.”

    Hey….what do you mean by “business”? Are you implying that somehow Charlie, Sharon, Emerge and I are involved in a business together? That perhaps we are thus engaged in profit-making and false advertising – and that you don’t want to be a pawn to it?

    Or maybe you just meant the informal generalized version of the term? 🙂

  25. TeachingRocks, it is entitled, “The Death and Life of the Great American School System–How Testing and Choice Are Undermining Education” by Diane Ravitch. One of her chapters is called “NCLB: Measure and Punish.” Ravitch was the former assistant secretary of education and a leader in the drive to create a national curriculum. I believe that at first she advocated the goals of NCLB but (working in the Bush administration) saw the dangers of the direction that NCLB was taking American education.

  26. TeachingRocks, it is entitled, “The Death and Life of the Great American School System–How Testing and Choice Are Undermining Education” by Diane Ravitch. One of her chapters is called “NCLB: Measure and Punish.” Ravitch was the former assistant secretary of education and a leader in the drive to create a national curriculum. I believe that at first she advocated the goals of NCLB but (working in the Bush administration) saw the dangers of the direction that NCLB was taking American education.
    150 Observor, I haven’t heard Dr. Lathan using the term “customer” to apply to students (but I believe she did yesterday). That term is in common usage–it definitely is indicative of a philosophy that runs deeper than just a word and/or analogy. It is the philosophy behind Ravitch’s chapter “The Billionaire Boys’ Club” about how “private foundations assigned themselves the task of reconstructing the nation’s education system.” My interpretation is that big business is making big money on education–and the children aren’t necessarily the ones profitting.

  27. Yeah, Jon, the informal generalized version. 🙂

    I just think some can go overboard with the criticism of all things. Death by a thousand paper cuts.

  28. Misguided analogy. No one is FORCED to join the union. Without it, however, we teachers would often find ourselves pawns in the hands of absolute buffoons.

  29. Interesting Newsweek article on New Orleans’ school district and its majority privately-run charter schools (and Paul Vallas).

    “So far, the experiment appears to be working. Before Katrina, two thirds of students were attending schools deemed failing by state standards, notes Leslie Jacobs, a New Orleans education-reform advocate; in the 2010–11 academic year, she says, it will be less than one third. “The fact that we haven’t gotten everything right yet shouldn’t take away from the fact that we’re getting a whole lot more right,” she says. New Orleans schools are still performing below the state average on achievement tests, but according to Jacobs’s analysis of state data, the gap between New Orleans and the rest of the state has basically been cut in half.”

    http://www.newsweek.com/2010/08/26/new-orleans-s-charter-school-revolution.html

    There’s no cherry-picking argument there – the entire district has cut the achievement gap (with the rest of the state) in half in the past five years under the charter/choice system. And charter schools aren’t the only changes in the district.

    “Even in traditional schools, principals have unusual autonomy over the hiring—and firing—of teachers, since the city’s teachers’ union lost its collective-bargaining rights.”

    Of course, correlation does not mean causation (though it could be).

  30. adkins-dutro I have to differ with you about joining the union. A new teacher can choose to NOT join the union but they still have to pay the dues. If the new teacher decides to join the union they still pay dues but aren’t protected until they are tenured. This is what I have been told by new teachers and old teachers.

  31. Speaking on the union issue—as a part-time teacher in the district, I had a prorated amount of dues taken from my check. I wasn’t given a choice as to whether I wanted to join. Anytime there was talk of a strike, I was told I had ZERO union protection. What benefits was I getting exactly?

  32. ….you could be absolute buffoons in the hands of incompetent tyrants. Kids are losing either way. But, hey, have your union keep extracting cadillac benefits and see why the rest of America thinks your “us against management” rhetoric has remained way past its usefulness – especially since it is us taxpayers paying the price. What evaluation tool will be acceptible to the union? My guess will be one with little accountability, but hope springs eternal. Ask our President.

  33. If Paul Vallas’ name is attached to it, it is probably mostly PR bunk. Of course, Vallas is the one who said that he doesn’t want teachers to teach more than 10 years. I don’t know what he wants them to do with the rest of their lives, but he certainly doesn’t want to pay them much beyond starting pay. “Appears to be working” is the operative phrase–if you look at Vallas’ history, appearance has rarely been close to reality.
    Fortunately, most of Vallas’ employers have had the right to fire him–which has happened to him most of the time. One of his biggest problems has been “cherry-picking”–that’s what happened when he was running the Chicago schools. His second biggest problem was spending too much money on charter schools and the contracts with expensive programs to run the charters. I just love statements like “though we haven’t gotten it right yet.” That’s the standard comment for all these programs–give us time and we will prove our worth (after they have already walked away with enough money to have made their efforts profitable even if not successful). The above statement sounds so much like what Vallas did in Chicago (and much like the direction in which 150 is heading)–cherry pick students for enough schools to declare success and let the rest of the students and schools fail. Jon, of course, there is a cherry-picking argument here. Just keep talking about the successful schools and maybe everyone will forget that the rest are making no progress. Jon, does the article say anything about how the demographics in New Orleans has changed from the before Katrina era. For one thing, my guess is that there are fewer students.

  34. The lack of protection for teachers without tenure isn’t completely the union’s choice–probably a negotiated “given.” “Average teacher” has it right for starters–the salary and benefits of new teachers would be much lower if a contract hadn’t been negotiated–a contract from which starting teachers do benefit. I have often heard new teachers complain about getting no protection for four years. However, I don’t believe that is completely true. I remember Adkins-Dutro helping at least one new teacher to get a fair shake at Manual–I believe new teachers can be represented even though not “protected.” I believe the tenure restriction is the time period the administration does need to prove whether or not a teacher is a good or bad teacher. Protecting all new teachers from the getgo just wouldn’t be right. However, I do believe that union members (if not union leadership) can and have often gone to bat to support new teachers and/or to put in a good word for them with administrators when decisions are being made about which teachers should be let go. Personally (although this would not be advice the union leadership would ever give) I have always believed that new teachers should be given a “pass” in crossing the picket line during a strike. Their risk is so much greater than that of veteran teachers. With so many veteran teachers retiring over the last 10+ years protection is a problem in District 150–because so many are non-tenured–that is especially true at Manual in the last two years because so many were non-tenured.

  35. Average teacher: I’m aware I was getting a salary but the union wasn’t giving it to me. I was referring to what benefits I was getting from the union—the one in which I had a “choice” to join. If dues are deducted regardless of my wishes, I’m not sure how much of a choice that is. That was my point along with the fact that part-time teachers really get none of the benefits because we’re all getting pink slipped in April anyway.

  36. “Jon, does the article say anything about how the demographics in New Orleans has changed from the before Katrina era.”

    Wouldn’t it be more meaningful to actually read the article? (It’s Newsweek – not exactly War and Peace) I mean, before you criticize the achievements of the New Orleans district because of Paul Vallas – before you determine that cherry picking exists (even though I referenced the stats on the ENTIRE district) – maybe, just maybe, you would actually read the article?

    One key demographic (% of low income students) has risen from 77% before Katrina to 82% now. Here’s the website for the report:

    http://www.coweninstitute.com/spenofiveyearsafterkatrina/

    You have to click on the link to download the report – and then go to page 6 for demographics…..oh, why bother?

  37. Jon, I have already done quite a bit of research about Vallas, including his work in New Orleans. I have saved many of the articles on my computer. I will probably look at this new article. It will take more than one article to erase in my mind all the negatives that I have read previously about Vallas.
    Unionmember, I see your point about paying dues, etc. Of course, I have a problem with part-time teachers anyway. I don’t approve of the district hiring part-time teachers instead of full-time teachers–ICC does it all the time to save on salaries and benefits. However, the union does exist for the majority. It’s one of those situations where non-tenured teachers might complain about paying dues, but I doubt that they will want to stop being union members once they are tenured. It’s like teenagers wanting teenagers to have more rights until they become parents of teenagers.

  38. Sharon, I would say that from what I’ve seen this year the district isn’t hiring part-timers any more. They are doing a great job of having prep teachers working two schools or teaching two or more subjects. At least these teachers will qualify for tenure in a few years if they can actually stay around awhile.

  39. Jeff would never use that term and I am offended that this blog would allow it on here. It added nothing to the discussion.

  40. Jon, the article is very interesting. I will read it more thoroughly when I have time. I am impressed by the way the article presents both sides of every issue–you and I can find plenty of proof to support our own point of view. Right off the bat, I would say that going from 120,000 to 40,000 students gives New Orleans schools a great headstart. Starting completely new with so many federal dollars (which are about to run out) was a great advantage. I found this sentence interesting (in light of Vallas’ stated stance that he doesn’t want teachers for more than ten years: “At the same time, many new teachers leave teaching after a few years, just when research indicates they begin having the most impact on student learning.”
    Also, “These tests provide a decent way of comparing pre-storm and post-storm achievement in those grades. However, it is important to recognize that the pre- and post-storm public school populations in New Orleans are different. Similar ethnic and income demographics make the post-storm population of students roughly similar to the pre-storm population of students as a whole, but it is unclear how many of the current students were in school before the storm and how they differ from the students who never came back.”
    I love the “decent” tests–remember that every state gets to choose its own NCLB test–which accounts for many differences from state to state. Illinois is one of just a few that has chosen the most difficult tests.
    Average teacher–agreed. This person doesn’t know Jeff.

  41. Sharon, if you’re going to keep quoting Vallas, don’t you think you should accurately quote him?

    “I don’t want the majority of my staff to work more than 10 years. The cost of sustaining those individuals becomes so enormous,” he says.

    “Between retirement and health care and things like that, it means that you are constantly increasing class sizes and cutting programmes in order to sustain the cost of a veteran workforce, so I think you want a mix, you want a balance.”

    http://news.bbc.co.uk/2/hi/8608960.stm

    Sure, you probably still disagree with him (and I can understand why). But he didn’t simply say “he doesn’t want teachers for more than 10 years”.

  42. Jon, stop quibbling. All you added was the word “majority” and spelled out the “why” which anyone could have figured out.

  43. Sharon, it’s just that I disagree with your comments on the blogs.

    Oops! I left out the word “some”, but why quibble? 🙂

  44. Jon, I believe your original post made it sound as though everything in New Orleans was rosy–that isn’t the picture painted by the article. The article (as journalism is expected to be) is fair and balanced. As to my Vallas (indirect) quote, there was little with which to agree–therefore, I called it quibbling (and it was). I didn’t misinterpret Vallas’ stance about preferring to get rid of teachers before they have taught 10 years (for him). And, of course, cost is his reason. Cheap teachers allowed him to pay more money to the programs (like Edison) that run his charter schools. He was spending federal dollars in New Orleans. In Chicago and Philadelphia when he spent the school district dollars, he ran the districts into financial trouble.

  45. What, exactly, in my original comment implies “everything is rosy”?

    Was it quoting “The fact that we haven’t gotten everything right yet shouldn’t take away from the fact that we’re getting a whole lot more right”? Funny, because I thought that by NOT acknowledging mistakes, you would be more likely to paint a ROSY picture.

    Or was it talking about the improvement in the “achievement gap” with the rest of the state, which, if I have to spell it out, means they are still not performing as well as the rest of the state? Yep, just ROSY.

    Maybe it was the statistic that today 1/3 of New Orleans schools are still failing? If that’s not ROSY, I don’t know what is.

    You originally thought the article would be “PR Bunk”. Maybe you meant that as a compliment? You seem to be quite adept yourself at spin here, “Rosy”.

  46. Jon, I will go one more round with you. I just think that comparing schools with 120,000 students to schools with 40,000 students (before and after Katrina) is an iffy comparison. No, I said that anything Vallas said will be PR bunk–thankfully, the article was written by a journalist. Anyway, I’m not sure what point you are trying to make about New Orleans. How do you see the findings with regard to implications for District 150? I assume you are trying to make a case against unions–that is a pointless argument. Teachers in 150 aren’t going to voluntarily shut down the union. so whatever will be will be. Also, I will be curious to see how soon New Orleans teachers make a push to be unionized. Otherwise, they will continue to leave as the article suggests. Anyway New Orleans is a very, very unique situation that received many federal dollars–I don’t see much of a comparison with any other district.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.