When does a threat of violence posted on Facebook or Twitter become illegal? The Supreme Court will tackle that question in the case of a Pennsylvania man jailed for Facebook posts saying he would kill his wife and a class of kindergartners. Observers hope the high court will clarify the extent of free speech protections in the age of women stalked on social media and shooters who leave online manifestos behind. We examine legal questions and personal stories about online threats of violence.


  • Emma Llansó Policy Counsel, Center for Democracy and Technology
  • Robert Richards Professor of Journalism and co-director of the Pennsylvania Center for the First Amendment at Penn State University
  • Jennifer Golbeck Professor of Information Studies and Director of the Human-Computer Interaction Lab at the University of Maryland
  • Amanda Hess Staff writer, Slate


  • 12:06:39

    MR. KOJO NNAMDIFrom WAMU 88.5 at American University in Washington, welcome to "The Kojo Nnamdi Show," connecting your neighborhood with the world. It's Tech Tuesday. Here's what a Pennsylvania man wrote on Facebook about his estranged wife, "There is one way to love you but a thousand ways to kill you. I'm not going to rest until your body is a mess, soaked in blood and dying from all the little cuts."

  • 12:07:16

    MR. KOJO NNAMDIIt's one of many violent threats he posted after she filed a protection from abuse order against him. She was afraid, he was arrested and served three years in jail for using interstate communications to threaten someone. Now, the US Supreme Court has agreed to hear his case. He says his comments were rap lyrics, not threats of actual violence and that the court used an unfair test to convict him.

  • 12:07:41

    MR. KOJO NNAMDIThe case raises questions about what constitutes an illegal threat in the digital age. Should the law consider the writers intent or rely on the sense of fear the words instill in a reasonable person? It's a question with broad ramifications for women who sometimes feel the internet is an unsafe place for them. For champions of the First Amendment and Freedom of Speech and for law enforcement and Homeland Security officials trying to walk that thin line between venting on Facebook and threatening real harm.

  • 12:08:13

    MR. KOJO NNAMDIJoining me for this Tech Tuesday conversation about how we handle online threats is Emma Llanso of the Center for Democracy and Technology. Emma, thank you for joining us.

  • 12:08:24

    MS. EMMA LLANSOThanks for having me.

  • 12:08:25

    NNAMDIAlso joining, in studio, is Jenn Golbeck, professor of Information Studies and Director of the Human Computer Interaction Lab at the University of Maryland and occasional host of this broadcast, Jenn, good to see you again.

  • 12:08:38

    MS. JENNIFER GOLBECKGlad to be here.

  • 12:08:39

    NNAMDIJoining us from studios in Los Angeles is Robert Richards, professor of journalism and director of the Pennsylvania Center for the First Amendment at Penn State University. He's also a lawyer and a member of the Pennsylvania and US Supreme Court bar. Bob Richards, thank you for joining us.

  • 12:08:57

    MR. ROBERT RICHARDSGood to be here.

  • 12:08:58

    NNAMDIYou too can join the conversation. Give us a call 800-433-8850. Do you think online threats should be treated different then other, say, written or verbal test -- verbal threats? The number's 800-433-8850, the email address kojo@wamu.org. You can shoot us a tweet @kojoshow. Bob Richards, the Supreme Court has agreed to hear the case of Anthony Elonis who served time for breaking a federal law that makes it illegal to use interstate communications, in this case Facebook, to threaten to harm an individual. Could you please explain what he did?

  • 12:09:35

    RICHARDSSure. And I think it's important to look at, sort of, the factual background here. You have a guy who was splitting up from his wife. The wife had the kids, then he lost his job. And so he was sort of in a bad space in his life and he does what many people do now, they go online, go on their Facebook page and he begins to vent about this stuff. Problem in the old days, you might've gone to a bar and unloaded on the bartender but today you go online. And you're right about this stuff.

  • 12:10:02

    RICHARDSHe also, sort of, fancied himself as a rap lyricist, so his, his writings were all in, sort of, this -- that rap genre, rap format. He didn't -- importantly, he did not communicate that to anybody. He didn't send it to her, didn't call her up, didn't text it to her, didn't email her. It was, sort of, this passive posting on Facebook. Her family found out about it, called law enforcement authorities, the law enforcement authorities came to the door, talked to him. He then began to write about them as well. And eventually landed with five counts of this threat statue, which is federal threat statutes, which we call true threats.

  • 12:10:41

    RICHARDSNow, the important thing about true threats is that, if it is considered a true threat, if a court really says, it is a true threat, then the speech loses First Amendment protection. And that's where we are right now at the Supreme Court. We've gotta decide what constitutes a true threat, what tests should be used to determine if it's a true threat because only then it loses First Amendment protection and can be criminalized.

  • 12:11:04

    NNAMDIEmma Llanso, one of the biggest challenges with online threats is determining whether the speaker is serious about wanting to cause harm or is just venting. How are threats on social media different from face to face threats or telephone threats, phone threats?

  • 12:11:18

    LLANSOI think that's one of the really important questions that the Supreme Court is going to have to answer as they consider this case. You can easily imagine whether it's in the case of Mr. Elonis or from your own experience, people sharing comments, that to a small group of people, or an in-joke are supposed to be humorous, are hyperbolic and, maybe, use threatening language but are understood by the small community that they're shared with, not to have any serious intent carry -- to, to actually commit some kind of harmful action, you know, as part of that communication.

  • 12:11:52

    LLANSOBut online it's a lot easier for a communication that's made in one particular context to be copied, retweeted, reblogged, posts -- posted somewhere else and suddenly seen by a much wider audience of people. This really, radically, can change the context of a comment and can take something that was very easily understood by the initial audience as something that was joking or not serious and suddenly be put in a different context where other people may infer a threat that was never intended to be expressed.

  • 12:12:23

    NNAMDIJenn Golbeck, this case involves a man threatening a woman online, describe, if you will, the broader atmosphere of harassment that can make some women feel like the internet is a hostile place for them to be.

  • 12:12:34

    GOLBECKYeah, this is interesting, as I was getting ready for the show, I was reading a lot of things and I spend my time on social media and studying social media. So I knew that there had been this long history. And going back, I found articles from 1996, talking about how, is the internet really a place that's safe for women because we're threatened all the time? And those articles have continued all the way up until today.

  • 12:12:59

    GOLBECKAnd it's a place where there's a sense of anonymity, even if you have your name associated with an account, it feels like, well it's online and so maybe it doesn't really count and I'm not really doing it because it's just on the internet or on Facebook. And this seems to have given men, in particular, some feeling of license to discuss all of the things that they want to do to women who make them angry and a lot of female journalists and I think we'll talk to some on the show today...

  • 12:13:27

    NNAMDIAt least one.

  • 12:13:28

    GOLBECK...have written about the, the really violent and ongoing threats that they receive from, from readers, presumably men in most cases, about what they're gonna do to them because they're angry at some of the things these women have written. So I think the anonymity and the sense that it doesn't count 'cause it's on the internet makes a lot of people feel like they can go ahead and make these threats and it is an intimating place because of that.

  • 12:13:54

    NNAMDIIn case you're just joining us, it's a Tech Tuesday conversation on online threats. We're talking with Jenn Golbeck, professor of Information Studies and Director of the Human Computer Interaction Lab at the University of Maryland. She joins us in studio along with Emma Llanso of the Center For Democracy and Technology. Joining us from studios in Los Angeles is Robert Richards, he's a professor of journalism and Director of the Pennsylvania Center for the First Amendment at Penn State University. We're taking your calls at 800-433-8850, have you ever been threatened online or through social media?

  • 12:14:27

    NNAMDIWhat did you do? Give us a call, 800-433-8850 or you can go to our website, kojoshow.org, as a question or make a comment there. Bob Richards, you mentioned that this case hinges on what are called, true threats, which don't have First Amendment protection. Can you explain the three different legal interpretations of a true threat, depending on where one lives?

  • 12:14:50

    RICHARDSAbsolutely. There are three standards right now, depending on where you're charged with this one federal statute, same federal statute. If, in some parts of the country the way the jury will look at it or judge would look at it, is based on what a reasonable speaker would think his or her words are conveying. So that's from the perspective of the reasonable person in law. We have a reasonable person standard in negligence and, and also in certain criminal cases, such as this.

  • 12:15:19

    RICHARDSIn other parts of the country, you look at from the reasonable recipient of the language perspective. So the jury would say, okay, if I got this threat, if I read this threat online or, or, or got an email or whatever, would I feel threatened, would I perceive it as a threat? So that's from the, the perspective of the reasonable recipient. And in still another part of the country, we use what's called a subjective intent. There the prosecution would have to prove that the speaker actually intended to make a threat and to communicate a threat.

  • 12:15:52

    RICHARDSAnd that's what's in the Elonis case will be argued by Mr. Elonis and the supporters of his position, that the subjective intent is the most protective of speech because it requires the government to show a lot more then it would have to under a reasonable person standard. The reason we talk about a subject standard, that Supreme Court has only looked at this issue twice, once back in 1969, again in 2003.

  • 12:16:18

    RICHARDSAnd in the 2003 case, the Supreme Court talked about true threats and said, you know, that is a statement where the speaker means to communicate a serious expression of an intent to commit a act of violence to a particular person or group of persons. So that language means to communicate is a important part of that because that seems to suggest that there should be a specific intent here.

  • 12:16:40


  • 12:16:41

    GOLBECKSo, I am definitely not a legal scholar, but I was looking through some of these definitions last night and maybe Bob can straighten out my understanding here. But the Supreme Court ruling on subjective intent, I believe, was in response to the cross burning case in Virginia, right?

  • 12:16:57

    RICHARDSCorrect. That's correct, that's correct.

  • 12:16:58

    GOLBECKOkay. And I was thinking about this and, and I shouldn't be summarizing the law but I'll give my, my best understanding of it and then can get corrected. There was a, a case where Virginia had banned cross burning and the Supreme Court said, you can't just outright say it's illegal to burn a cross because it doesn't necessarily carry a threat with it. And so you can think of cases like the Klan is inducted new members and burns a cross. And they're not actually threatening anyone, it's part of some ceremony.

  • 12:17:25

    NNAMDIIt's significant that the one descending view in that case came from the states only African-American judge, Clarence Thomas.

  • 12:17:33

    GOLBECKI, I noticed that as well.

  • 12:17:35

    NNAMDII'm pretty sure, when he thinks of the Ku Klux Klan burning, he takes it personally, like most African-Americans do.

  • 12:17:40

    GOLBECKAnd, so even, if we give the benefit of the doubt in this case and say, okay, so there can be cases where one might burn a cross and not mean it as a threat. That makes sense in a way because you say, okay, well there, there's some ambiguity about whether or not it's a threat. But in this case, if you're saying you're gonna kill someone and he also, I believe, said he was gonna shoot up a kindergarten, that was one of his posts, it seems to me that that's missing a lot of what is behind this subjective intent, part of the law. That there's no ambiguity about whether or not there's a threat, if you say you're gonna shoot up a kindergarten.

  • 12:18:17

    NNAMDIBob Richards?

  • 12:18:17

    RICHARDSWell I think, well I think, if you -- first of all, you're correct with your interpretation of Virginia vs. Black, the case back in 2003. But I think, in terms of the language on a Facebook page, if I were to -- on my Facebook page, put a, an intro that says, to -- dedicated to the men and women of law enforcement. And then I were to take the 1992 song, "Cop Killer" by Ice-T, and just simply list the lyrics there. A reasonable person, looking at those lyrics, out of context, just basically looking at it, could say, wow, that's intended to be a threat to the law enforcement community.

  • 12:18:52

    RICHARDSNow, maybe I'm into community policing oversight and that sort of thing and, you know, all of these extraneous things that would have to be shown, if it were showing subject intent, then that would get me off the hook. I didn't really intend the threat to, to the law enforcement community. I was just simply putting that on there, venting my frustration. And that's the point that we're trying to make here, that you really should go, when you're talking about stripping somebody's words of First Amendment protection.

  • 12:19:19

    RICHARDSAnd literally putting them in jail because of their words alone, that is not too much to ask the government to have to prove that little higher standard. That they have to show that you intended to make the threat and not simply use an objective standard, looking at the words themselves and say, wow those are threatening. Let's get -- lets use, if you see a Facebook post like that or YouTube video, out here in California, we have the Isle of Vista case a couple of weeks ago where the kid posted on YouTube...

  • 12:19:44


  • 12:19:44

    RICHARDS...take, take that information and use it as a springboard to investigation. Absolutely investigate this guy. Go to his house, if you need it, get a warrant. If you use that, use your -- the Facebook post or the YouTube post as a, as a, as a help to establish probable cause for a warrant. Maybe turn it over to mental health asso -- professionals, if necessary. Use it as a springboard to investigation, don't make it the end of itself, into itself to arrest somebody just for their speech.

  • 12:20:12

    NNAMDIEmma Llanso.

  • 12:20:13

    LLANSOI agree. I mean, I think if the concern here is that we end up with a standard that allows prosecutors to kind of ignore the other contextual factors, and that was really one of the issues in the Virginia case, that, you know, the prosecutors have to look into the context of cross burning, which again, you know, may almost always be intended to intimidate. But the court was ruling that, you know, you need to do that investigation.

  • 12:20:39

    LLANSOAnd I think the concern of setting a standard where comments can just be treated as, you know, per say threatening is that you'll see disproportionate prosecution of kinds of expression that tend to rely more on hyperbole and violent threats in a way that could really chill speech of, you know, certain kinds of artists or certain communities.

  • 12:21:02

    NNAMDIWell, let me get back to the Ice-T question again, the question of context. Anthony Elonis says when his Facebook posts are taken out of context, his true intent is misunderstood. The people who have been threatened online say it's precisely the lack of context that makes social media threats more scary because the recipient has no in-person cues about the poster's state of mind. What do you think about this role of context, Jen?

  • 12:21:27

    RICHARDSI mean, I think that this is the critical point. If this guy, who by the way never had any experience listening to rap music or writing lyrics before...

  • 12:21:37

    NNAMDIHis wife testified she didn't know about this part of his artistic intention while they were married.

  • 12:21:42

    GOLBECKThat's right. So if he had, say, gone to some open-mic night and rapped these lyrics and someone had a YouTube video of him doing it, I could buy that it was artistic expression and he was just venting himself in this form. But putting it up on Facebook standalone, there's no context that's missing there. it's a standalone post.

  • 12:22:05

    GOLBECKAnd, you know, Bob had mentioned in the first question you asked him that he wasn't trying to communicate this to anyone. But I think that's a wild misunderstanding of social media. The whole point is that you're communicating to these large audiences.

  • 12:22:16

    GOLBECKIf I'm a woman and someone is saying to me the things that he wrote about his ex-wife, if they're standing in front of me I can understand if they're joking, if they're trying to do a bad rap in front of me. I can understand that. But when it's online and I'm the person who's being threatened or I'm the mother of one of those kindergartners that he said he was going to go shoot up a classroom of, there's no context for me to say, oh well, he's just joking or oh, it's just artistic expression.

  • 12:22:45

    NNAMDII think Bell in Silver Spring wants to underscore that point. Bell, you're on the air. Go ahead, please.

  • 12:22:50

    BELLHi. Thank you, Kojo. I'd just like to add an observation into this discussion about threatening or harassing language and communications online. And I think that this really is affecting very much the social development of young people going into adulthood today, which is that when people are speaking face to face their listener is giving feedback. Even if they're silent there's facial expressions and body language and cues that the speaker receives.

  • 12:23:15

    BELLBut in addition to the sense of anonymity, name posted or not online, the speaker is speaking with absolutely no response. It's just an opportunity to express with no communication back. And I think that that also promotes the idea of being able to say things that are not only inappropriate but the person is not learning how to interact with other people in an appropriate way. And I think that that plays in also here to the threats.

  • 12:23:45

    NNAMDIAs I said, underscoring the point that Jen Golbeck was making. Bell, thank you very much for your call. We're going to take a short break. And when we come back, we will talk with one of the journalists, that Jen mentioned earlier, who has had personal experience with this. But you can still call us at 800-433-8850. Where do you see the line between free speech and illegal harassment online? Send us an email to kojo@wamu.org or a tweet @kojoshow. I'm Kojo Nnamdi.

  • 12:25:50

    NNAMDIIt's Tech Tuesday and we're talking online threats with Emma Llanso of the Center for Democracy and Technology, Jen Golbeck, professor of information studies and director of the Human Computer Interaction Lab at the University of Maryland, and Robert Richards, professor of journalism and director of the Pennsylvania Center for the First Amendment at Penn State University. I'd like to start this segment by going to the phones where Hassan has a question maybe for you Bob Richards. Hassan, you're on the air. Go ahead, please.

  • 12:26:18

    HASSANYeah, thank you, Kojo. My question is this. Isn't intent what is in the mind? And when the gentleman published in public for everybody to read, eliminate this intent?

  • 12:26:32

    RICHARDSWell, you're correct. I mean, obviously when you're talking about subjective intent you want to make sure that you are getting into the mindset of the defendant. So the best way to kind of explain specific intent crimes is to look at theft, okay. Theft is a specific intent crime, meaning I have to have the mindset to permanently deprive somebody of their property. So if I take a stapler off your desk but I intended to return it, I don't have the specific intent to actually commit that crime.

  • 12:27:02

    RICHARDSSo now you bring that back to the situation at hand, you say, okay this guy writes his rap lyric song, puts it on his Facebook page. Did he really have the intent to threaten his wife, threaten law enforcement, threaten the kindergarten class? He maintains he doesn't, okay. And so if the law says that the prosecution has to prove that specific intent, then they're going to have to use other evidence to come up with the -- enough to convince the jury that he did specifically intend that. Rather than the objective standard, which would be just to look at the words themselves and kind of keep context out of it.

  • 12:27:38

    RICHARDSAnd I think it was Jen earlier, or it might've been Emma earlier, talking about context. And she's absolutely right. Context is key in the area of threats because I could say to somebody -- you know, I could be playing a basketball game with somebody and they score something. And I say, I'm going to kill you, man. Or I could be in a bar and be intoxicated and have a weapon on me and say, I'm going to kill you, man, say the same words and they're two different things. Context is key.

  • 12:28:01

    NNAMDIOkay. Thank you very much for your call, Hassan. We move on now to Steven in Baltimore, Md. Steven, you're on the air. Go ahead, please.

  • 12:28:10

    STEVENHi. Two really quick questions. Social media has been around for a few years now and I know that the government keeps track of violent occurrences by gun and knife and things like that. Have they started keeping a dataset of folks that actually post on social media their violent acts before they actually do it? And secondly, if he -- if the person posts something from say rap lyrics from Eminem or N.W.A against the police or something like that, would that have the same zeal for the police to come to his house to knock on his door?

  • 12:28:54

    NNAMDIAny studies conducted as far as you know, first Emma Llanso?

  • 12:28:57

    LLANSOI'm not sure.

  • 12:29:00

    NNAMDIAnd Jen?

  • 12:29:01

    GOLBECKSo I know that -- so I don't think the government is keeping track of specific individuals who are making one-on-one threats like we're talking about here. But there are places like the Southern Poverty Law Center that track hate groups and hate speech in things like social media and internet forums even before an act has been committed. So they have a repository of the kind of discussions that are going on there that could then be used in an investigation if something does happen.

  • 12:29:28

    NNAMDI800-433-8850 is the number to call if you'd like to join the conversation. Should someone who posts comments online threatening a killing spree like the one near UC Santa Barbara last month be arrested for making that threat? Threatening violence online also affects society at large. In several recent shootings the perpetrator posted videos or manifestos about their intention. One of them was Elliot Rodger who killed six people near UC Santa Barbara last month. Should he have been arrested for the disturbing YouTube videos his mother discovered and warned police about, Bob Richards?

  • 12:30:04

    RICHARDSWell, the police used those videos to actually go to his apartment. And they interviewed him. They determined at the time that he wasn't a threat. In hindsight that probably was not a -- they should've gone a little bit further. Maybe they should've gotten a warrant and gone in and found that he had stockpiled a number of weapons. In fact, he even said in his little last manifesto that he was very nervous when the police came to his apartment because he thought his plan was going to be, you know, spoiled because he had those weapons at the time at the apartment.

  • 12:30:38

    RICHARDSI think, as I said earlier, this should be a springboard absolutely. if you see videos online like that, if you see Mr. Elonis' posting, that sort of thing, I don't think it's at all inappropriate for the police or law enforcement authorities to say, all right, we should look into this a little bit further and use that as a springboard. Use it -- maybe get somebody into the mental health system, maybe get somebody into, you know, the -- get a warrant to look at their place. And also, if it is considered to be or if the law enforcement believes it's a true threat, by all means you can even arrest them for a true threat.

  • 12:31:10

    RICHARDSMy point is that when you get to court you should have to prove a little bit more than just, oh if you just read this posting online or look at this video you know that he's -- this is a true threat so therefore he should go to jail. I think if we're going to be serious about protecting speech in this country -- and we do have a First Amendment that does that, then I think that it's not unreasonable to say the government should have to go to the harder step to prove that speech alone is enough to go to jail.

  • 12:31:37

    NNAMDIJen Golbeck.

  • 12:31:38

    GOLBECKI understand the point Bob is making here but I'm not sure, like, what he actually means by a springboard to investigate someone's intent. So if I write a threat to Kojo on my Facebook page, I hate that Kojo guy, like, I'm going to find him after the show and shoot him down, I mean, what is there to prove that I'm going to do it or not, that I actually had intent? Like, I can say, no, I didn't actually intend it as a threat. But what proof is law enforcement going to find?

  • 12:32:06

    GOLBECKEven if I have guns I believe I have a Second Amendment right to keep those. So you can't just go into my house and say, what, she had a gun and she threatened Kojo and so she had subjective intent to do it? I think that there's this whole space of you have to get into my mindset. There's no investigation to just get into my mindset. And so I think we're opening up this space where people -- if we really have to prove intent of the mind when we're making a threat, we open up this space where people can make violent and terrifying threats willy-nilly and just claim that they had no actual intent to threaten anyone.

  • 12:32:47

    NNAMDIBefore we deal with that hypothetical case, allow me to deal -- to have us deal with a real one because joining us now by phone is Amanda Hess, a staff writer at Slate who had a real such experience. Amanda Hess, thank you for joining us.

  • 12:32:58

    MS. AMANDA HESSThank you so much for having me.

  • 12:33:00

    NNAMDIAmanda, you wrote a widely-read piece in Pacific Standard magazine about your own experience of being repeatedly threatened by a man online through twitter and through his blog. What did he do and how did you respond to those threats?

  • 12:33:14

    HESSSo this has happened to me a couple of times over the course of my career. And one of these sort of frustrating things about online threats is that I have no way of knowing whether all of the people who are threatening me are the same person or if they're different. And so that makes it really difficult, you know, if we're talking about understanding the intent of a threat and also makes it difficult for me as the victim to understand whether I should truly feel threatened by someone or if it's just sort of a drive-by threat by someone who doesn't really have much of an investment in harassing me.

  • 12:33:50

    HESSBut the most recent time this happened I was outside of my local jurisdiction. I was on vacation and I woke up to find that someone had created a Twitter account that was dedicated to threatening me. So they were saying that they knew where I lived and they had read my work. And they were going to come to my house and rape me and cut off my head.

  • 12:34:14

    HESSAnd as a victim, you know, this is obviously a space where people aren't sure how threats like this should be dealt with. And as the victim I too didn't really know what to do, so I called the police. And when the cop came to my hotel room, I told him what happened and he said, what is Twitter? He had no idea what this platform was, where this threat was happening. And over the course of me trying to A. understand what to do personally and B. report it to the fullest extent that I thought I could, I found so many contradictions in the way that tech companies and police forces are dealing with this stuff.

  • 12:34:56

    HESSAnd really both Twitter and the police had kind of passed the buck between each other saying that it was the other institution's problem to deal with. And it ended up just sort of putting it all on me as the victim to take the time and the emotional resources and the economic resources to try to figure out who this person was and also to protect myself.

  • 12:35:20

    NNAMDIBut you also contacted the FBI. What was the response there?

  • 12:35:26

    HESSSo I called the FBI because I wasn't sure where the threat was coming from. I didn't know if they were somebody who lived next door or they lived across the country or in a different country. And I called the FBI and they took down my information and said that an agent would get in touch with me if they were interested in taking the case. And I didn't hear back from them.

  • 12:35:46

    HESSSomething I've heard from a lot of victims is that when they contact local police these people don't necessarily have the tools to even understand the platforms where the abuse is happening and they don't have the resources to investigate them. But when you call a larger entity, you know, they have all the tools and the institutional knowledge to deal with this stuff, but they don't necessarily have the bandwidth to take on every threat that's being made...

  • 12:36:13

    NNAMDIWhich brings it back to you. The headline of your article is Why Women Aren't Welcome on the Internet. Can you describe how online threats poison the atmosphere for some women on the internet in ways that have, certainly in your case, professional and personal consequences?

  • 12:36:30

    HESSSure. So I think we talk a lot about how the internet is just, you know, a vile place in a lot of ways that people sort of use language online that they don't use in real life. But what I found in my experience in talking with a lot of women is that we experience threats in a similar way online in the way that we do sexual harassment in the workplace or harassment on the street. It's gendered in that, you know, someone isn't just threatening to kill me. They're threatening to rape me also. And these are sort of threats that we're experiencing online that we also experience offline.

  • 12:37:07

    HESSAnd what I found was because there is no sort of recourse for us, it ends up really discriminating against women online to, you know, even today are poorly represented in tech companies, we're poorly represented in online journalism. And because we have to deal with this extra sort of discriminatory threatening behavior, it makes it more difficult for us to succeed and to have our voices heard.

  • 12:37:35

    NNAMDINot to mention what it takes up in terms of your time, money for legal fees, online protection services and the like, correct?

  • 12:37:43

    HESSThat's right. That's right.

  • 12:37:45

    NNAMDIHow are -- well, go ahead, please.

  • 12:37:48

    HESSI'm sorry, say that again.

  • 12:37:50

    NNAMDIGo ahead.

  • 12:37:51

    HESSOh, so this is actually only the most recent time that this happened to me. The first time that someone threatened me over the internet I actually could figure out who the person was because they had been using their real name in some of the forums that they were threatening me on. And I called the police in that time as well. This happened in D.C. actually. They declined to take a report. And so I proceeded by getting a protection order against this guy, which took some money to get the paperwork sent to him. It took time for me to take off from my job to sit in a courtroom and wait for our case to come up.

  • 12:38:34

    HESSAnd then eventually I did get a protection order against him. It lasted for a year. And after that year was over he started contacting me again. And now because police refuse to investigate the threats that I've gotten more recently, I really have no idea if this sort of longtime stalker is still threatening me or if it's someone else. And so on a personal level because it's not really seen as a real crime, I don't even really have the tools necessary to protect myself individually if that's what I want to do.

  • 12:39:05

    NNAMDIJust quickly Amanda, how are tech companies and social media sites, maybe also law enforcement, beginning to address these concerns about online threats?

  • 12:39:16

    HESSSo they do it differently. Twitter, which is the company that I was dealing with, their general rule is to delete anything that doesn't conform to their guidelines. So when this was reported to them, they reacted by deleting this anonymous Twitter account that this guy had created and that was it. And some people -- some victims want that. They don't want declamatory material about them being on the internet. So if they apply for a job they -- you know, their Google searches don't result in a bunch of vile commentary.

  • 12:39:52

    HESSBut for me it made it more difficult for me to report the crime to the police because, you know, police sort of were confused about what was happening anyway. And when they went to the URL I gave them, now the material had been deleted. So it was even more confusing for police who were not equipped to deal with this to understand that this stuff had actually happened to me.

  • 12:40:15

    NNAMDII'm going to have to interrupt 'cause we have to take a short break, but Amanda, thank you so much for joining us.

  • 12:40:19

    HESSThank you.

  • 12:40:20

    NNAMDIAmanda Hess is a staff writer at Slate. When we come back, we'll get the responses of our panelists to Amanda's experience and take your calls at 800-433-8850. Should someone who posts comments online threatening a killing spree like the one near UC Santa Barbara last month be arrested for making that threat? What do you think? Send us an email to kojo@wamu.org or a tweet @kojoshow. I'm Kojo Nnamdi.

  • 12:42:27

    NNAMDIIt's Tech Tuesday. We're discussing online threats with Jen Golbeck, professor of information studies and director of the Human Computer Interaction Lab at the University of Maryland and occasional host of this broadcast. Emma Llanso is with the Center For Democracy and Technology and Robert Richards is a professor of journalist and director of the Pennsylvania Center For the First Amendment at Penn State University.

  • 12:42:51

    NNAMDIHe's also a lawyer, member of the Pennsylvania and U.S. Supreme Court Bar. Bob, you have been a victim of online threats, too, like Amanda, for your staunch defense of the First Amendment, which has included supporting the rights of Westboro Baptist Church members to protest at funerals. How have you handled those threats?

  • 12:43:08

    RICHARDSWell, I mean, it's, you know, Amanda's case demonstrates a really important point and that is the law, and by extension law enforcement, always lags behind the development of technology. In her situation -- and I was hearing her explain these things about going to the police and they didn't even know what Twitter was or being, you know, sort of nonchalant in their response, is a sort of the classic response that the law and thus law enforcement has to the new technology. They don't understand it.

  • 12:43:33

    RICHARDSBut I think that's beginning to change. Every social network now, Facebook, Twitter, Instagram, all of these things have policies on how to respond to law enforcement subpoenas. So if, for example, you are unable to identify the person who is threatening you, you can get -- the police can issue a subpoena on the social network and try to get some URL identifying information and so forth.

  • 12:43:59

    RICHARDSSo that's a little bit better. I've been around the block a long time and before, you know, prior to the internet had a Center For the First Amendment beginning in 1992 and some of the hate mail and threats that I got actually came in, you know, by the post office. They came in on a postcard or on a letter, you know. Obviously, it was anonymous. If there was something that I thought was concerning, I would turn it over to sort of campus law enforcement authorities to take a look at first.

  • 12:44:27

    RICHARDSA lot of it I felt was just sort of venting, but I think absolutely that people should take seriously -- I mean, the kind of thing that she was explaining, I would do exactly as she did. If I got that threat, I would call the police and I would hound the police. And I think as time goes on and the law sort of catches up a little bit as much as it can to technology, things will get better in terms of the response by law enforcement.

  • 12:44:51

    RICHARDSAnd they do have the tools to be able to identify people.

  • 12:44:53

    NNAMDIEmma Llanso, what do you take from Amanda Hess' story?

  • 12:44:56

    LLANSOWell, I think I'd echo Robert's point that it's -- I think law enforcement is still really struggling with how to deal with these kinds of threats and how seriously to take them. So you have a case like what Amanda's talking about here you have repeated instance of threats directed at a specific person over a course of time. That seems like absolutely the sort of thing that should be taken seriously and should be investigated, should, as Robert was saying, go to the different social media platforms, know what those platforms are and how to serve them with a subpoena, to preserve data about who's posting those comments.

  • 12:45:28

    LLANSOBut I think the concern is that if we start looking at making a legal definition of true threat that would allow law enforcement to prosecute people for things that just seem kind of out of context, treated as a single standalone message, like a threatening remark, then that's going to lead to prosecution of, you know, entirely the wrong people and it's not going to focus necessarily on the actual instances of repeated abuse and harassment that people like Amanda have been victims of, but it will potentially just open up to unexpected interactions with law enforcements by people who are just posting, you know, a stupid YouTube video for their friends.

  • 12:46:12

    NNAMDIJen, your response to Emma.

  • 12:46:14

    RICHARDSAbsolutely and that -- I just said that goes back to the point of context and she's absolutely right. I mean, context is key here.

  • 12:46:19


  • 12:46:19

    GOLBECKYeah, so, again, I would agree on context and when Amanda was telling her story, I was thinking about -- I've been focusing on this issue a lot and keeping track and it's an average of two or three times a day, every day, that somebody makes some harassing comment to me. You know, I'm out for a run. I'm out walking my dog. The landscaping guy driving past, like, yells something out the window.

  • 12:46:44

    GOLBECKThat's a threat in a way, that if he were standing in front of me saying it, I would be more threatened than him driving past. But if I'm in a parking lot and someone talks about the sexual thing he's going to do to me and is following when I walk to my car, I have a context to know that that's much more threatening than the guy walking past. On social media, we don't have any of that context so it could be a guy who knows where I live and is going to stand outside my house or it could be some guy in another country.

  • 12:47:12

    GOLBECKAnd without that, we create this huge environment of fear, regardless of where the person is because the recipient of the threat has no way of knowing how serious that threat actually is.

  • 12:47:26

    NNAMDIWe got an email from Patrick in Arlington, Virginia, who says the problem is that Congress did not define the term threat in the interstate communication statute. The solution is for Congress to repeal the statute. Bob Richards, you know what Patrick is referring to?

  • 12:47:43

    RICHARDSAbsolutely. He's absolutely right. I mean, it's a very brief statement basically. And that's why it's been subject to interpretation. So I don't know if we have to go back to Congress 'cause, you know, we all know how long that takes. I think what we're going to hopefully get is some clarification by the Supreme Court in the Elonis case. This is the, you know, the third crack at this notion of threat that the Supreme Court will have and at this point, I think, the reason the court took the case is that we do have this sort of triple standards out there, depending on where you are in the country.

  • 12:48:14

    RICHARDSAnd you can't have that. I mean, the very inscription above the U.S. Supreme Court is "Equal justice under law." And if you have the same statute be interpreted three different ways depending on where you live, that's not equal justice. So I think Patrick's correct The statute is, you know, not very good as written, but I think the answer will come when the Supreme Court decides the Elonis case.

  • 12:48:33

    NNAMDIAnd we got another email from William who says, "I play a game called 'Second Life' where someone has accused me of making threats towards her based on signs I made to counter protest her behavior towards me. Is there an absolute test for determining context and motive and understanding if those signs were true threats? Are there specific words or symbols that provide substantial evidence for a threat and what resources are available to learn more?" Can you comment on that, Emma Llanso?

  • 12:48:59

    LLANSOSo I think when we're -- and this goes a bit back to a question that Jennifer was raising earlier of how do you establish somebody's subjective intent to threaten and it's about making a factual case that will convince a jury. So, you know, there's not, to my knowledge, specific, you know, we have the Supreme Court case about cross burning. It's hard to think of something that could fit more into a category of always prescribable under the First Amendment and there, the Supreme Court said, well, no, you've still got to look at context.

  • 12:49:30

    LLANSOSo it's a malleable kind of standard. It's going to depend on, you know, where, to whom, what kind of audience is a comment directed. Is there a history of certain kinds of comments? How do the -- what is the perception of the audience? In the Supreme Court case about the cross burning -- or, no, sorry. This is about the earlier case, about threatening the president, the fact that the man who made the threat against LBJ, his comment was received with laughter from his audience was a big part of the context that the court considered, that, you know, this was not a statement that was taken seriously by the audience.

  • 12:50:14

    LLANSOSo there are going to be a number of different kind of factual pieces that a prosecutor is going to have to try to establish to convince a jury that, yes, all of this taken together clearly indicates this person was threatening or something else.

  • 12:50:30

    NNAMDILast week, a University of Washington student was arrested at his dorm for posting threats to kill women on campus. He had also posted comments on YouTube videos defending Elliot Roger in the Santa Barbara killings. Were officials right to arrest him, in your view, Jen Golbeck?

  • 12:50:44

    GOLBECKSo I looked at the specifics of his case so I can't comment on that specifically, but if we go back to the previous case where this guy did go and kill a lot of people, I've watched all of those videos that he posted, which are really disturbing, and, you know, I think it's a gray area in that case because he was making some threats, but they weren't very specific. You know, he didn't say, I'm going to go to this place. I'm going to do this thing.

  • 12:51:16

    GOLBECKAnd I agree, you know, the police went and they looked and they didn't feel like he was actually...

  • 12:51:23

    NNAMDIWell, they didn't really look. They said he was soft spoken and polite, didn't they, I think and they didn't truly search, yes?

  • 12:51:27

    GOLBECKI think that's right. Yes, no. I'm sorry. I didn't mean to imply that they searched, right, but they did go talk to him and they found that there was nothing they could do. If this guy was doing the same kind of thing, you know, I think it's an open question of how serious those threats actually seem when they're posted online. But I think that's very different than this Facebook case where the guy was threatening his wife to the point where she was so afraid that she got an order of protection from his threats and then he continued posting things online after that.

  • 12:51:58

    GOLBECKI think there's no gray area in that space, that she clearly felt threatened and he knew she felt threatened and by continuing to post violent things, there's no question that he was trying to intimidate her more.

  • 12:52:12

    LLANSOAnd I think that's exactly the set of circumstances that the prosecutor should present to the jury and see, does the jury agree that based on, you know, this pattern of continual comments directed toward his wife, it's, you know, if the court comes down in this case that a subjective intent standard is the right standard to apply, which I hope they do, the case will get remanded back to, you know, sent back to a lower court.

  • 12:52:35

    LLANSOThat standard could be applied and he very well could still be convicted.

  • 12:52:39

    NNAMDIBut the broader question, Bob Richards, is how do we strike a balance between free speech rights and the right to live without fear of being harmed?

  • 12:52:48

    RICHARDSYou know, the First Amendment is really all about striking a balance. It has been throughout its history. I mean, we have certain categories of speech that are not protected under the First Amendment, obscenity, child pornography, incitement to violence and true threats. So I think there is a mechanism there for balancing this and I think the -- I believe Emma was just talking, the subjective intense standard, to me, is a good balance here.

  • 12:53:13

    RICHARDSWe are not just doing a knee-jerk response to seeing a video online or a Facebook posting and arresting somebody on the basis of their speech. We do an investigation. If we see there is this context, that there is a sort of an escalation of the violent remarks and that sort of thing, then, by all means, you can arrest somebody for their speech. You can put them through that prosecution. If you can prove that subjective intent, you can criminalize the speech and the person goes to jail.

  • 12:53:40

    RICHARDSBut I think we really have to be careful about just worrying about, well, this internet is sort of the wild West and people say anything they want when they get online and it's sort of without consequence so we really have to come down hard and just be able to, you know, arrest people if they're putting videos out there. That's just a dangerous path to go down and I just hope we don’t go down it.

  • 12:53:59

    NNAMDIWell, you mentioned the wild West and I remember seeing a lot of stories where, in certain towns, people had to check their guns before they could even really step into the town or step into a bar in the town, which brings me to this, Jen Golbeck. In the internet age, it's easy to set up a new email account or Twitter account without revealing your identity. People who create accounts for the express purpose of online harassment are often referred to as trolls. Do we need better legal protection against them?

  • 12:54:27

    GOLBECKThat's, I think, a great and open question. The kind of famous...

  • 12:54:32

    NNAMDIThey check their guns at the door. Go ahead.

  • 12:54:34

    GOLBECKYeah. I mean, the famous trolling case was this guy on Reddit, Violent Acres, who would post, I mean, just a whole series of really vile things, young girls in, you know, swimwear in a rape bait section so guys could talk about the things they wanted to do to these girls. You know, is it child pornography, not exactly. Is it a threat? If you saw your picture up there, you would certainly feel like it was threat.

  • 12:55:03

    GOLBECKBut, you know, I think this is -- independent of the Supreme Court case, the question of anonymity on the internet is one that we are going to continue to grapple with because having that anonymity gives people a license to say things that they would never say in person. At the same time, taking away anonymity and tying identity to everything gives lots of people the ability to track us in all kinds of ways that we don't necessarily want, which is a whole separate issue that you and I have talked about before.

  • 12:55:36


  • 12:55:36

    GOLBECKAnd so it's not necessarily that we want everyone to be identified all the time, either. So lots of great internet thinkers have not come up with a good solution to this yet and I'm not gonna try to offer one here, but I think it's an important question.

  • 12:55:49

    NNAMDII'd like to combine an email we got from Keith and a tweet that we got from Adam. Keith writes, "I'm confused about whether there's a legal difference between intent to threaten and intent to harm. It seems that many of these examples describe threats where any reasonable person would feel threatened. Whether or not there was an actual intent to follow through with that threat seems like a separate question."

  • 12:56:14

    NNAMDIAnd the tweet from Adam says, "is your guest really saying you can't arrest someone for making clear threats unless he follows through or you can read his mind?" To which, Bob Richards, you say what?

  • 12:56:23

    RICHARDSI say that your emailer's exactly right. There are two different things here. The threat statute says you have to have an intent to threaten, not the intent to carry out the act. Okay. Those are two different things. It's the intent to threaten that is enough for the crime there. No, I'm not saying that you have to have the means to carry out the act. You may have no means to carry out the violence and you can still be arrested for threats.

  • 12:56:50

    RICHARDSAnd, you know, that's perfectly fine. I don't have a problem with that. I just say when you do that, we ought to have one standard. It ought to be a subjective intent standard that says that you had the intention to actually make a threat and you weren't simply venting your anger, frustrations or emotions. So, no, you do not have to have the ability or the intent to carry out the threat.

  • 12:57:09

    NNAMDIWhat do you say, Emma Llanso?

  • 12:57:11

    LLANSOWell, I did want to pick up on the point that Jen had made about anonymity online.

  • 12:57:17

    NNAMDIHave one minute left.

  • 12:57:17

    LLANSOAnd I think that is absolutely going to come up in what the court is looking at and I think it's going to be very important to not just keep in mind the kind of bad rap that anonymity has gotten online and the way it allows people to say things they might not say in person, but to also really understand and respect the way that it frees people to say things that they wouldn't ever feel they were able to say in person, whether because their local community wouldn't support them in discussing these things, or what have you.

  • 12:57:47

    LLANSOSo I think when we look at what the context is and how anonymity plays into that, we need to keep both sides of the benefits of anonymity as well as potential concerns.

  • 12:57:55

    NNAMDIEmma Llanso is with the Center For Democracy and Technology. Emma Llanso, thank you for joining us.

  • 12:58:00

    LLANSOThank you.

  • 12:58:00

    NNAMDIBob Richards is a professor of journalism and director of the Pennsylvania Center for the First Amendment at Penn State University. Bob Richards, thank you joining us.

  • 12:58:08

    RICHARDSThank you.

  • 12:58:09

    NNAMDIAnd Jen Golbeck is a professor of informational studies and director of the Human Computer Interaction lab at the University of Maryland. Jen, always a pleasure.

  • 12:58:16

    GOLBECKThank you.

  • 12:58:17

    NNAMDIThank you all for listening. I'm Kojo Nnamdi.

Related Links

Topics + Tags


comments powered by Disqus
Most Recent Shows

The Politics Hour – September 21, 2018

Friday, Sep 21 2018New proposed legislation threatens some of the power D.C. Mayor Muriel Bowser exercises over education in the District. Rep. Jamie Raskin is running for a second term in Congress, pledging to protect Maryland's air and federal workers. They both join us in studio.