Yesterday The Nation hosted an online roundtable about an incident at Brown University in which New York police chief Ray Kelly, the public face of that city’s racist, unconstitutional stop and frisk program, was heckled so severely at the start of a scheduled speech that he and the college cancelled the event.

I can see where both sides of the Nation debate are coming from, up to a point. On the one hand, I’m mostly a big fan of the idea that the cure for bad speech is more speech, and heckling is clearly a tactic that will turn a not-insignificant number of people off. On the other hand, there’s no question that it makes a powerful impression — Jesse Myerson is right to reject the idea that setting rhetorical traps for someone like Kelly in the Q&A period is likely to garner more publicity for your cause.

But I think that it’s a mistake to frame this discussion purely as a matter of organizing strategy. Because here’s the thing: Ray Kelly was not harmed in any material way by the students’ action. He hasn’t lost his job, hasn’t lost a paycheck, hasn’t lost a public platform. He hasn’t lost anything. The students who heckled him, on the other hand, are in danger of losing a lot.

Just yesterday, Brown’s president announced that the university is considering taking disciplinary action against “individuals or organizations involved” in the disruption. At UC Irvine three years ago, students who heckled an Israeli official were arrested. Ten were ultimately convicted of misdemeanors, and the Irvine Muslim Student Union’s charter and funding were temporarily revoked.

Even where such protests don’t result in criminal charges or loss of group recognition, moreover, disciplinary action has a chilling effect on campus activism. It has become common practice in recent years for administrators to use such charges to constrain students’ future organizing — holding the most serious punishments in abeyance with the threat of reinstating them if a student violates any university policy in the future. Given the ambiguity of typical campus disciplinary rules and the arbitrariness with which they are so often applied, such a threat can effectively sideline prominent activists for the remainder of their time on campus.

And that, ultimately, is the real civil liberties issue here.

In the Nation roundtable the students’ critics raised the specter of the tactic coming back around to be used against the left. But as someone who speaks on on college campuses with some regularity, I can say without hesitation that the prospect of being heckled — even heckled off the stage — doesn’t scare me. It doesn’t make me angry. What does scare me, and what does make me angry, is the thought that administrators might have such students arrested, might shut their funding down, might make it impossible for them to keep organizing on campus in the future.

Student hecklers don’t have the power to silence me, not really. I can engage them, or I can wait them out, or I can meet with interested students informally, or I can reschedule my speech, or I can give my talk online, or I can post its text here on my blog. I can get my word out, and if someone tries to shout me down, I expect I’ll wind up getting more attention — both supportive and critical — than I would have otherwise. But the administration does have the power to silence them.

That’s why those of us who occupy positions of relative prominence and security have a responsibility to be careful about how we describe these incidents. Not to hold back, necessarily — if you believe that the students who heckled Ray Kelly should be brought up on campus charges, or even arrested, by all means make your case. But if you don’t believe that, if you’re criticizing their failure to measure up to your standards of productive tactics or polite behavior, you should say so, clearly.

Unfortunately, a lot of the language that’s been used by the critics of the Brown students in the last few days has been anything but temperate. On Twitter, Katha Pollitt accused the students of bullying Kelly, while Richard Yeselson described their tactics as authoritarian. At the Daily Beast, Peter Beinart called them totalitarians. This kind of overheated rhetoric can only serve to demonize the students, and to pave the way for the use of punitive measures against them.

At the close of her contribution to the Nation roundtable, Pollitt said that the “abstract right to free expression” is “the best protection going for the left.” But too often these days campus activists who engage in rowdy, rambunctious speech are denied that protection. It’s not Ray Kelly’s right to free speech that’s in danger, it’s theirs.

And the threat is anything but theoretical.