At Texas A&M this Tuesday, a group of sixty students from Uplift Hampton Preparatory School, a predominantly black and Latino high school nearly two hundred miles away in Dallas, were visiting the campus when several of them were approached by white students from the college. One white student asked two of the Hampton girls for their opinion on her Confederate flag earrings, at which point, according to an account released by state senator Royce West,

“a group of white male and female students, who [were] within earshot of the first event, told a larger group of the high school visitors … to ‘go back where you came from.’ They continued their taunts by using the most well-known racial slur that’s directed toward African Americans and also made other references to the Confederate flag.”

I learned about this incident yesterday, when a guy I know on Twitter, a lawyer who works with the campus free-speech group FIRE, tweeted that while “racial slurs are ugly … expelling doesn’t fix the problem.” In a followup, he responded to Senator West’s suggestion that the students involved “should be strongly disciplined, if not expelled” by declaring that “the university can use its own speech to decry the racism, but it can’t ‘not tolerate’ racism by expelling” students.

Here we have one of the core recurring arguments connected to today’s student movement in a nutshell: A group of students do something bigoted, campus activists call for them to be punished, and someone points out that the First Amendment forbids it.

Over the last year in particular, lots of students have been pushing colleges to take action against  speech, and lots of civil libertarians have been criticizing them for doing so — like it or hate it, they say, you can’t punish free speech.

But it turns out that’s not quite true.

Not all speech is protected, particularly in an institutional context. Bigoted campus speech in particular may be actionable if it constitutes harassment under the terms of the Civil Rights Act of 1964, and free speech advocates do their cause no favors when they neglect that fact.

Under American law, colleges and universities have not only a right but an obligation to act in certain cases of student-on-student harassment. The exact contours of that obligation aren’t settled — there haven’t been enough Supreme Court cases to establish clear precedents on all the relevant questions — but a decision from 1999, Davis v. Monroe, laid out the basic outlines. Public schools, the Court said then, have a responsibility to act in cases in which student-on-student sexual harassment is “so severe, pervasive, and objectively offensive that it effectively bars the victim’s access to an educational opportunity or benefit.” (Subsequent decisions have extended that ruling to cover other kinds of bias, and some have replaced the “and” in “severe, pervasive, and objectively offensive” with an “or.”)

What does all this mean? It means that the United States Congress and the Supreme Court recognize that harassment of a student by other students may be so intense and abusive as to make it impossible for that student to participate fully in the educational process. And it means that when such a student is targeted on the basis of race, gender, disability, or certain other characteristics, the harassment constitutes a federal civil rights violation.

Consider that standard in the context of this week’s incident at Texas A&M. Prospective enrollees were, according to the state senator’s account, called vicious racial slurs, taunted by a group of students, and told they didn’t belong on campus. College officials, including campus police, were made aware of the incident but reportedly did little about it.

Clearly this behavior was offensive, and clearly the prospective students were targeted on the basis of their race. Which brings us to the big question: Was the harassment severe enough to “effectively bar [the high schoolers’] access to an educational opportunity or benefit”?

Again, this is the crucial point. Campus speech doesn’t lose its First Amendment protection because it’s obnoxious or offensive or wrong or hateful. It has to have a specific, concrete impact: It has to seriously compromise someone’s ability to obtain an education at the institution. And by that standard, it seems to me, one can reasonably argue that the reported treatment of those high schoolers at Texas A&M this week was racial harassment of a kind that the college has a right and even a responsibility to address.

These were black high school students visiting a college they were thinking of applying to. They arrived on campus and were barraged with racial epithets, Confederate flag taunts, and claims that they don’t belong at the university. College officials witnessed it happening and failed to effectively intervene.

Are the students of Uplift Hampton likely to apply to Texas A&M after all that? I suspect some of them may not be.

Now, I’m not a lawyer, and I’m not saying that a harassment case would be a slam dunk in this situation. But just as freedom of speech is a real thing and an important thing and a recognized fact of American law, so too is the right to pursue an education, and so too is the right to be free from discrimination on campus.

When bigoted, obnoxious campus speech is so aggressive and sustained that it materially interferes with students’ ability to become or remain a part of a campus community, colleges have a right and an obligation to take action. That’s a fact. And if we want a reconciliation between antiracist activists and civil libertarians, we need to acknowledge that fact and begin to work toward a shared understanding of how to translate it into policy.