Category Archive

December 5 Update | When I wrote this piece a week ago I was certain that Twitter wasn’t blocking Wikileaks from trending. Now I’m not so sure.

So the Wikileaks organization released the first batch of a promised quarter million US State Department cables today, and it seems like everyone on the planet is talking about it. The story is front-page news at media sites across the globe, it’s all over the television, and for a while this afternoon more than two percent of all Twitter traffic was about the leaks.

You read that right — one in every fifty tweets was about Wikileaks this afternoon. To put that in context, it’s about three times as many as mentioned Justin Bieber.

And yet “Wikileaks” hasn’t hit Twitter’s trending topics list all day. Right now #becauseofjustin, a Bieber-related hashtag, holds the top spot on Twitter’s global trending topics list, even though it’s running just one tenth the volume of #wikileaks itself.

Twitter being Twitter, there’s no shortage of speculation about why this is. Some suggest it’s a conspiracy of some kind. Others claim that because Wikileaks is a username on Twitter, it’s excluded from appearing on the trending topics list. (This is false, as a quick investigation will demonstrate.)

So what is it? Basically, it’s the algorithm.

It turns out it’s tougher than you’d think to put together a trending topics list that really means anything. If you just go by the raw frequency with which words appear, you’re going to wind up with stuff like “the,” “and,” and “RT” at the top of the charts forever. And even if you exclude words like those, you’re still going to wind up with “lunch” trending every lunchtime and Glee trending every Tuesday.

Which is fine if that’s what you’re interested in, but the folks at Twitter have decided that they’re interested in something else. What they’re interested in is finding out what’s breaking — what people are interested in today that they weren’t interested in yesterday. And to find that out you need to look beyond the raw numbers. You need an algorithm, and it needs to be a sophisticated one.

Because any time you create a system like this, there will be people who try to game it. Twitter doesn’t want its trending topics list to be a list of the day’s ten most successful “RT if you love puppies!!!!!” campaigns, and it doesn’t want the list — or its feed — to be clogged up with “Tweet about Velveeta to win a free iPad” garbage, either.

All of which means that Twitter doesn’t really care whether “wikileaks” was the most-tweeted word of the day. What they’re trying to measure is something more subtle and more complex. And they won’t tell us what exactly that is. They can’t tell us, because if they told us, we’d have a leg up in gaming the system again.

So why didn’t #wikileaks trend? It’s impossible to say for sure, but it’s most likely a combination of things. First, #wikileaks is a hashtag that sees considerable traffic on an ongoing basis, so it has to spike higher to make a splash than a less common word would. Second, a large portion of the traffic #wikileaks has seen today has been in the form of retweets, and Twitter gives retweets much less weight than original tweets in calculating trends. Third — and I’m truly guessing on this one — #wikileaks is a hashtag, and as a hashtag it necessarily reflects a co-ordinated, organized push to boost discussion of a topic, rather than an organic outpouring of interest. I wouldn’t be at all surprised to learn that putting a # in front of a word depresses that word’s weight in the trending topics algorithm, if only slightly.

Wikileaks rolled out a new hashtag today — #cablegate — and it quickly rose to the top of the trending topics list, despite producing far less traffic than #wikileaks. (That fact alone should quiet the conspiracy theorists, though it probably won’t.)

Some will take this last piece of info as a reason to spawn new hashtags every week, so that the tags’ novelty works in their favor. There may be something to be said for such an approach, but my own sense is that it’s counterproductive, for a few reasons.

For starters, trending topics are wildly overrated as an organizing tool. You constantly see people urging others to “get [whatever] trending,” or fretting about the fact that it’s not, but the reality is that having your hashtag show up briefly on a list in a sidebar isn’t going to do a hell of a lot for your movement. I’ll say more about why that is in a future post — this one is already ridiculously long.

Beyond that, there’s the fact of what you lose by switching hashtags — continuity and predictability.

Consider #wikileaks vs #cablegate. #Wikileaks drew huge traffic all day because people associated the tag with the topic, so using it came naturally to them. And they kept using it in the face of encouragement to switch to #cablegate. Even after #cablegate trended, in fact, #wikileaks stayed much more popular.

The point of hashtags — the point of Twitter itself — isn’t to get your tweets in front of random people. It’s to build a community of discussion. It’s to connect with people who are interested in what you’re interested in, and get them more interested. It’s to turn weak ties into stronger ties. And trending topics don’t have a whole hell of a lot to do with any of those projects.

Big week last week, in the US and overseas, and I’ve got lots of stuff to pass along. Most of it will wait until Monday, but there should be an update or two tomorrow.

The DREAM Act is going to be coming up for a vote in the US Congress in the next little while, and the contours of that vote are beginning to come into shape.

The DREAM Act would give undocumented immigrants who were brought to the United States as children a path to citizenship through college enrollment or military service. Polls show that it’s supported by a strong majority of Americans, and it’s expected to pass the House of Representatives easily.

It’s in the Senate, where a sixty-vote supermajority is required to invoke “cloture” and bring legislation to the floor for debate, where the real drama is expected.

Various media outfits and advocacy groups have released lists of the senators they believe to be in play, and though those lists are individually unreliable — Pro Publica’s list included a sponsor of the bill in their list of on-the-fence-senators — together they give a sense of the universe of possible wavering votes.

Thirty-five Democratic senators have signed on as sponsors of the DREAM Act, and another ten Democrats are understood to be reliable votes for its passage. On the other side, twenty-nine Republicans are known to be rock-solid in opposition. That leaves twenty-six senators who are at least theoretically up for grabs, and proponents of the bill need fifteen of those twenty-six to vote yes.

Here’s how those twenty-six votes break down, as of Friday, November 26:

Against:

• Lindsey Graham, Republican of South Carolina, was once seen as a possible “get” by DREAM Act but his spokesperson told the Daily Caller this week that he’ll be voting against cloture.
• Judd Gregg, Republican of New Hampshire, has told the National Review Online that he’ll also be voting no.
• George LeMieux, Republican of Florida, has announced that he’ll be voting against the bill, telling a reporter that he “cannot support consideration of the DREAM Act until we have taken substantial and effective measures to secure our borders.”
• Ben Nelson, Democrat of North Dakota, is also on record as a firm no.

Almost certainly against:

• Max Baucus, Democrat of Montana. Voted no in 2007. At that time he attacked the bill as “virtually the same” as “amnesty,” and he’s given no indication that he’s reconsidered his position.
• Scott Brown, Republican of Massachusetts, has repeatedly attacked the DREAM Act as an “amnesty” bill.
• Orrin Hatch, Republican of Utah, who has supported the bill in the past, now calls its consideration a “cynical” act, while his staff has laid the groundwork for a “no” vote with the National Review Online. He’s up for re-election in 2012, and unlikely to stick his neck out now.
• Kay Bailey Hutchison, Republican of Texas — who is the target of an ongoing hunger strike by DREAM Act advocates — said in a statement on Tuesday that she will not support the bill unless it is dramatically narrowed. Like Hatch, she’s up for re-election in 2012, and likely to stick with the conservative line for the next two years.
• John McCain, Republican of Arizona. Another past supporter of the bill who has distanced himself from it in recent months. One news story says that he has declared that he will not vote for it this year, though I haven’t been able to confirm that.
• Mark Pryor, Democrat of Arkansas. Voted no in 2007, said in September that he would “probably” oppose it again. Pryor was one of two Democrats who broke ranks in September and voted against bringing the defense bill to the floor with the DREAM Act and Don’t Ask Don’t Tell repeal added to it as a rider.

Uphill climbs:

• Kent Conrad, Democrat of North Dakota. Voted no in 2007, said in September he hadn’t made up his mind. Up for re-election in 2012.
• Byron Dorgan, Democrat of North Dakota. Voted no in 2007, said in September he hadn’t made up his mind. Retiring from the Senate, which may make him less predictable.
• Kay Hagan, Democrat of North Carolina. Listed as a “likely no” vote at DREAMact.info, she has said that the bill “should” be taken up as part of comprehensive immigration reform, but hasn’t said definitively that she’ll vote against it.
• Joe Manchin, Democrat of West Virginia. A conservative Democrat just elected in a conservative state. Almost certainly a tough vote to get, but not yet declared either way.
• Olympia Snowe, Republican of Maine. Voted yes in 2007, but is worried about a possible primary challenge from the right when she runs for re-election in 2012.
• John Tester, Democrat of Montana. Voted no in 2007, hasn’t made a statement this time around.

Up for grabs:

• Chris Coons, Democrat of Delaware. Elected this November, but took office early as he was running to fill Vice President Joe Biden’s seat for the remainder of his unexpired term. Hasn’t staked out a public position on the issue.
• Mary Landrieu, Democrat of Louisiana. Voted no in 2007, but one blogger said she was told by Landrieu’s staff this month that she’s a supporter this time, and she voted for cloture on the defense bill in September. Landrieu isn’t up for re-election again until 2016.
• Lisa Murkowski, Republican of Alaska. Is said by her staff to be “reviewing the bill.” This vote may well be a big one for Murkowski, who won re-election in November as a write-in candidate after losing in the Republican primary. If any Republican has a reason to want to poke the GOP in the eye right now, it’s Murkowski.
• George Voinovich, Republican of Ohio. Listed as a no vote on two bloggers’ scoresheets, though apparently hasn’t made a public statement. Is retiring from the Senate, and may be less concerned about maintaining party discipline as a result. Has been taking increasingly liberal stands on immigration issues in recent years. DREAMact.info lists him as undecided.

Ripe fruit:

• Sam Brownback, Republican of Kansas. Voted yes in 2007, no position this time. Retiring from the Senate to serve as governor of Kansas.
• Susan Collins, Republican of Maine. Voted yes in 2007, hasn’t taken a position yet this year.
• Claire McCaskill, Democrat of Missouri. Voted no in 2007, but described herself as “very sympathetic” to the bill’s aims this September. Voted for cloture on the defense bill in September.
• Jim Webb, Democrat of Virginia. Another vote for cloture in September, he’s listed at the DREAMact.info website as a solid “yes” vote on the bill.

Definitely for:

• Richard Lugar, Republican of Indiana. He’s a sponsor of the bill, and a longtime proponent of it, so he really shouldn’t be on the list at all. But Pro Publica listed him among the senators whose votes are “considered uncertain,” so here he is.
• Robert Bennett, Republican of Utah, has also announced that he’ll be voting for the bill.

So there you have it. To recap, there are twenty-six senators who are supposedly up for grabs, but six of them have already publicly declared their positions — four against, and two in favor. That leaves twenty actual undecideds, and the DREAMers need the support of thirteen of them in the cloture vote to win passage of the bill.

Of those twenty, four seem like solid pickup opportunities, and four more look like plausible gets. If all eight vote yes, that leaves five more votes needed out of the twelve remaining senators — but six of those twelve are almost certain “no” votes.

I’ll keep updating this list as the vote gets closer and more information emerges. If anyone has any additions or corrections, please let me know in comments.

Today’s massive student protests in Britain have produced a huge amount of Twitter traffic, much of it using the #demo2010 hashtag. But although a number of tags and phrases connected with the demonstrations have trended in the UK over the course of the day, #demo2010 has been strangely — and entirely — absent from the country’s trending topics list.

Why that’s the case still isn’t clear, but a comparison of the relative frequency of tweets using that tag with phrases that have trended in the UK today reveals that it’s not because of lack of traffic.

(click on graph to see it full size)

As of right now — nearly 3 o’clock in the afteroon NYC time — the top four trending topics on UK Twitter are #uksnow, #dayx, #wonderlandwednesday, and McKeith. But as this chart shows, #demo2010 has more than double the traffic of any of them at this hour, and has been far ahead of them all for the entire day. At around noon London time today #demo2010 peaked at 0.2% of total Twitter traffic, according to this chart — more than triple the volume of any of the current top four.

So what’s going on? I’m going to continue to dig, and others are on the case as well. More soon.

Update | Via Twitter, in response to this post, comes the suggestion that #demo2010 can’t trend because there’s a Twitter user with @demo2010 as their handle, and Twitter doesn’t allow strings that are identical to usernames to trend.

Second Update | The above explanation is incorrect. There’s a @dayx account, has been since 2008, and #dayx has been trending all day.

Third Update | Twitter honcho Del Harvey (@delbius) says, in response to a tweeted query, “we aren’t blocking” #demo2010. In followup tweets she guesses that the tag’s failure to trend may be an artifact of Twitter’s algorithms — “The more something is talked about, the more it needs to be talked about in order to be a trend” — but offers no inside info as to what’s going on.

If the trending algorithms are keeping #demo2010 off the list — and if the stats I’ve seen for #demo2010 tweets are accurate — then I must say that the algorithms strike me as seriously flawed. Take a look at what happens when you expand the graph linked above to cover the last thirty days, instead of the last 24 hours…

What this shows us is that #demo2010 spiked high on November 10, and about half as high — but still really high — today. Both spikes were dramatic, and both reflected huge interest in the topic. More importantly, both spikes were spikes — today’s pop in the numbers reflects a rebound from something close to zero between November 13 and November 22.

Compare that to “McKeith,” which did trend today. That term had two huge spikes in the last couple of weeks, and fairly consistent traffic in between and after. It trended even though its traffic today was actually quite similar to the traffic its been seeing over the last ten days.

I’m not taking a conspiratorial view of this, but I do think it’s weird, and it does strike me as bad coding — if in fact coding is the problem.

Fourth Update | Tweeter @adrianshort argues that the “diversity of tweets for TV programmes” versus the “huge numbers of RTs” for #demo2010 may explain the latter’s failure to trend. (Gillian McKeith is currently appearing on “I’m a Celebrity … Get Me Out of Here!” in the UK.) And though there have been a lot of unique #demo2010 tweets today, there have also been a lot of RTs, so this explanation strikes me as the most plausible I’ve heard so far.

In a separate tweet, Short suggests that trending topics may not actually be such a huge deal for an event like today’s demonstrations, since “You get massive network effects from normal RTing a popular tag.” I’m inclined to agree with that, too.

British students, building on protests that sent fifty thousand people into the streets of London two weeks ago, are staging demonstrations in dozens of cities today against government plans to slash funding for higher education and triple student fees.

Resources from around the web:

• I published an introduction to the protests earlier this week, and I’ve been tweeting all day at @studentactivism.

• The Guardian newspaper has a liveblog of the day’s events.

• The BBC’s television news coverage of the protests can be found online here.

• The main Twitter hashtag for the protests is #demo2010.

• The journalist @paul__lewis is tweeting from the scene in London.

• Britain’s Education Activist Network is covering the story on their blog.

Send along additional links in comments, and I’ll add them to this post.