Author Archive

As I posted yesterday, the tweets going around comparing Wisconsin’s SAT/ACT scores to five states where teachers have no right to unionize are based on bad data — it’s not true that Wisconsin’s SAT/ACT ranking is second in the nation, and that Texas, Virginia, Georgia, North Carolina, and South Carolina are all clustered at the bottom of the pile. The chart that says otherwise is based on outdated statistics and improper statistical analysis.

So what’s the truth? What would good data tell us about this question?

Well, it turns out that that’s kind of a complicated question. I can answer it, but you’ll have to bear with me for more than 140 characters.

It’s hard to measure SAT/ACT performance, because different numbers of students take the tests in every state, and comparing the strongest students from one state with a much bigger sample from another doesn’t tell us much that’s interesting. A 2000 study in the Harvard Educational Review, in fact, found that 85% of the difference in states’ performance on those tests is due to variation in participation rates.

Having said that, though, it’s clear from the numbers in my last post that once you’ve controlled for participation Wisconsin remains near the top of the country on SAT/ACT scores, Virginia is near the middle, and the rest of the no-union states from the tweet are near the bottom. High school graduation rates — the subject of another popular Wisconsin tweet meme in recent days — tell a similar story. It’s not as dramatic as best vs. worst, but it’s still dramatic.

Wisconsin does well on a third measure of student performance, too. Its scores on the National Assessment of Educational Progress (NAEP) in 2009 were above the national average in three of four measures (fourth grade math and eighth grade math and reading) and at the national average in the other (fourth grade reading). Of the ten states in the US without teachers’ unions, only one — Virginia — had NAEP results above the national average, and four — Arizona, Alabama, Louisiana, and Mississippi — were in the bottom quintile. (One scholar, in fact, found that the states with the strongest teachers unions tended to out-perform states with weaker unions too.)

But there’s another problem. Most of the states that don’t have teachers’ unions are poorer than Wisconsin, and have more English Language Learners in their schools, and rank higher for other demographic factors that make strong academic performance less likely. Rich kids in a school with a teacher’s union will do better than poor kids in a school without one, generally, but that doesn’t have much to do with the union itself. States with teachers’ unions do better, on average, than states without, but is this because of the unions, or state demographics?

There’s only been one scholarly effort to tackle this problem that I’m aware of. Back in 2000, three professors writing in the Harvard Educational Review did a statistical analysis of state SAT/ACT scores, controlling for factors like race, median income, and parental education. They found that the presence of teachers unions in a state did have a measurable and significant correlation with increased test scores — that going to school in a union state would, for instance, raise average SATs by about 50 points.

Two other findings leap out from the Harvard Educational Review study. First, they concluded that Southern states’ poor academic performance could be explained almost entirely by that region’s lack of unionization, even when you didn’t take socioeconomic differences into account.

And second, and to my mind far more interesting, they found that concrete improvements in the educational environment associated with teachers’ unions — lower class sizes, higher state spending on education, bigger teacher salaries — accounted for very little of the union/non-union variation. Teachers’ unions, in other words, don’t just help students by reducing class sizes or increasing educational spending. In their conclusion, they stated that

“other mechanism(s) (ie, better working conditions; greater worker autonomy, security, and dignity; improved administration; better training of teachers; greater levels of faculty professionalism) must be at work here.”

To sum up:

Yes, Wisconsin has great schools, with great outcomes. Yes, states without teachers’ unions lag behind. Yes, that lag persists even when you control for demographic variables. Yes, that difference seems to rest less on the quantifiable resources that unions fight to bring to the classroom than on the professionalism, positive working environment, and effective school administration that unions foster.

And yes, Virginia, (and Texas, Georgia, and North and South Carolina) unions do work.

Update | Matthew Di Carlo of the Shanker Blog has a new post up reviewing the state of scholarship on the relationship between teacher unionization and student performance, looking at (and linking to) several articles I missed. Di Carlo leans a little more heavily on an “on the one hand/on the other hand” approach than I would, but it’s definitely a worthwhile read for both his data and his analysis.

Update | Be sure to read this post on the truth behind the numbers — how teachers’ unions do improve student performance, and why. Seriously. Read it. If you don’t, you’re only getting half the story.

Also, if you’d like to follow me on Twitter or Facebook, feel free.

There’s a tweet circulating widely that says that Wisconsin ranks second in the nation in its students’ SAT/ACT scores, while five states that bar teachers from participating in collective bargaining all rank near the bottom of the pile. Here are a few examples:

@WisDems Only 5 states don’t have collective bargaining for educators. Their ACT/SAT rankings: SC-50th/NC-49th/GA-48th/TX-47th/VA-44th.

@PTraeder: 5 states have no collective bargaining & deemed it illegal. States & Their ranking on ACT/SAT scores: VA-44 TX-47 GA-48 NC-49 SC-50 #wiunion

@trek: 5 states forbid collective bargaining for educators: SC, NC, GA, TX, & VA. Their national rank in ACT scores: 50th, 49th, 48th, 47th, 44th.

The information presented here on which states bar collective bargaining in education is correct (although it only reflects the five states where teachers’ unions are illegal, leaving out the other five states where they’re legal but essentially don’t exist). What it doesn’t include, however, is any supporting documentation on the SAT and ACT rankings themselves.

So I’ve taken a look at the data.

State scores on the SAT and ACT are hard to compare directly, because there’s so much variation in how many students take the tests. In addition, I haven’t yet found a source that combines SAT and ACT scores into a composite ranking like the one in the tweet. Looking at charts for SAT and ACT results separately, however, here’s what I found:

Wisconsin ranks 3rd in the nation in SAT scores, but with a participation rate of just 4%. On the ACT, with a much more representative partcipation rate of 69%, it was tied for 17th. In comparison…

• Virginia was 34th on the SAT with 67% participation, 13th on the ACT with 22% participation.
• Texas was 45th on the SAT with 53% participation, 33rd on the ACT with 33% participation.
• Georgia was 48th on the SAT with 74% participation, 34th on the ACT with 44% participation.
• North Carolina was 38th on the SAT with 63% participation, 20th on the ACT with 16% participation.
• South Carolina was 49th on the SAT with 66% participation, 44th on the ACT with 52% participation.

Wisconsin is clearly above the other five states in both SAT and ACT scores, but the gap isn’t anywhere near as big as the pro-union tweets suggest. Among high ACT participation states, Wisconsin ranks something like 4th in the nation. But among high SAT participation states, Virginia ranks about 5th in the nation — almost all the states with better SAT scores than Virginia have far smaller participation rates, drawing on a far more elite test-taking group.

I’m open to seeing new data on this, but for now I’m going to mark this claim down as highly exaggerated.

Update | Several readers have posted links to the original source for the tweeted claim, which can be found here. Thanks!

Now, about that source. First, it’s from an analysis conducted in 1999, apparently by a University of Missouri law professor named Douglas O. Linder. Linder doesn’t say specifically what year the SAT/ACT numbers come from, but they’re obviously more than a decade old.

Second, the ranking methodology is really weird, and completely unreliable. What the author did was take each state’s ranking on the SAT, add it to each state’s ranking on the ACT, add those two numbers together, and then put them in order. In other words, Wisconsin scored 5th in the country on the SATs and 4th in the country on the ACTs, giving it a total of 9, and only Iowa had a lower total, so Wisconsin was 2nd in the country overall.

This is just silly. As I noted above, almost nobody in Wisconsin takes the SATs — the state has only a 4% participation rate on that test, with the only students taking it being those who are applying to competitive out-of-state colleges that don’t accept the ACTs. When you compare Wisconsin’s SAT average to Georgia’s, you’re comparing the performance of a tiny elite in one state with that of 74% of the graduating class of the other. And on top of that, this chart gives Wisconsin’s SAT score equal weight with its ACT score in determining which state is “better.”

It’s nonsense. It’s meaningless.

Second Update | The scores don’t seem to match the data, either. Linder gives Wisconsin’s average combined Math and Verbal SAT score as 1073, but Wisconsin’s average SAT scores in the late 1990s ranged from 1169 in 1996-97 to 1181 in 1999-2000. Something’s screwy with Linder’s numbers.

Third Update | As I said at the top of the post, I’ve put up a new essay this morning discussing exactly how teachers’ unions do improve student performance, and why. If you’ve read this far, you really should keep reading.

I was on KDVS radio last night, out of UC Davis, for an hour. Host Ed Martin and I talked about the history of student activism, campus radio, the privatization of public higher education, British fee protests, Wisconsin, Egypt, comparisons between Wisconsin and Egypt, the crisis at the University of Puerto Rico, prospects for mobilization on March 2, and a whole lot more.

You can listen to the show here.

As everyone knows by now, the Democrats in the Wisconsin state senate took a powder yesterday, fleeing the state to deny the body a quorum and force a pause in Governor Scott Walker’s plan to eviscerate the state’s public employee unions. Early yesterday evening the senate’s Reupublican leadership bowed to reality and put the senate into recess for the night.

Here’s the latest:

Wisconsin’s Dem state senators are still AWOL in Illinois, but Jesse Jackson made an appearance today in the capitol rotunda, addressing the crowd of activists who have packed the building in recent days. AFL-CIO President Richard Trumka also addressed the crowd.

The head of the state’s largest public employees union offered to compromise on benefits if Walker would back down on the union-busting component of the bill. Walker quickly rejected the proposal.

Milwaukee teachers staged a sick-in today, closing the city’s schools for the first time since the current crisis began. Madison’s schools were closed today for the third day in a row.

United Council, the Wisconsin statewide student group, said 11,000 UW students from across the state would be converging on Madison for an afternoon rally.

Update | Wow. Republicans in the state assembly just tried to pass the budget bill while most of the chamber’s Democrats were off the floor — apparently by starting the meeting earlier than scheduled and rushing through the preliminary steps. Democrats stormed into the room moments before passage, and managed to get the clock rolled back to the beginning of the process. The Assembly then abruptly adjourned until next Tuesday.

The Wisconsin budget bill is moving toward a vote in the state senate this afternoon, having passed in a party-line vote in that body’s budget committee late last night.

University of Wisconsin officials are pledging to keep the system’s campuses open today in the face of a planned walkout by faculty and students. Here’s an editorial from the University of Wisconsin Badger Herald supporting the walkout.

Public schools in Madison, Wisconsin’s capital, are closed for the second straight day today as teachers stage an ongoing sick-out. The closure has spread to more than a dozen districts in the capital region since yesterday.

A number of small changes were made to the bill yesterday, but none touched the core complaints that demonstrators have. Republican leaders say they have the votes in hand to pass the legislation in both houses today.

More to come…

Update | Organizers claim thirty thousand people joined yesterday’s protests.