Bullying is a big problem for Instagram, one that the company’s head, Adam Mosseri, has vowed to take on. As TIME explains in a new feature, the company is trying to build artificial intelligence that is capable of rooting out complex behaviors — ranging from identity attacks to betrayals — that make users feel victimized. That will likely take years. In the meantime, the company is developing ways for users to fight this scourge themselves, with the assistance of machines.
Enter Instagram’s newest feature: Restrict.
There are already some options for dealing with bullies on the platform, like flagging content that will be sent to moderators for review. But in user research, Instagram has found that teenagers — a demographic at high risk — are reluctant to report or block peers who bully them. “They are often seen as very harsh options,” says Francesco Fogu, a designer who works on well-being. Both can betray hurt feelings. And while blocking a bully essentially shuts down contact between two accounts, it also means that a user can no longer monitor what the bully is doing.
Restrict, which has not been previously reported on, is more like a one-way mirror. While it’s obvious to someone when they’ve been blocked — because they can no longer find that user on the platform — it won’t be obvious when they’ve been restricted. They will see that user’s posts in their feed like they usually do. But they will no longer see when the user is online or has read their messages. And, behind the scenes, that user will have more oversight. “The goal here,” Fogu says, “is to basically put some space between you and them.”
When it comes to posts, the user will have the ability to review any comment that the bully writes on one of their photos or videos. They can approve the comment so everyone can see it, delete it so no one can see it, or leave it forever in a pending state: invisible to everyone but the bully, who will have no indication that anything is out of the ordinary. The user doesn’t have to read the comment either; it will show up behind what Instagram calls a “sensitivity screen,” requiring the user to tap on the comment to approve or delete it. Do nothing, and it stays hidden in a pending state.
When it comes to direct messages, the restricted person will still see conversation threads they’ve had with the user in their main inbox. But, on the other end, the user won’t be notified when the bully sends them a message and won’t see the thread in their main inbox; instead, restricted threads will be diverted to the area reserved for message requests, where things like spam are typically relegated. If a user seeks the messages out and wants to respond, they’ll have to un-restrict the bully to do so.
Restrict is also designed to create “friction” when it comes to tags and mentions, “to make it harder for a bully to find you and to harass you,” Fogu says. Normally, Instagram will automatically suggest handles when someone is trying to tag or mention another user. If a bully is trying to tag a user who has restricted them, however, they’ll have to know the user’s handle and type it out completely.
Fogu describes Restrict as a “holistic” approach to combatting bullying. However encompassing it might be, one hurdle Instagram has to overcome is making sure users actually know about it. The company has launched other tools meant to help users protect themselves from bullies in the past — like a filter that will hide comments containing certain keywords that a user designates — but many remain unaware that such tools exist.
To overcome this, engineers and designers are experimenting with ways they can alert users to the new tools while they’re using the app, rather than ferreting them away as additional options in user settings. If a user seems to delete comment after comment that another user makes on their posts, for example, that might be the moment for a pop-up that tells them about Restrict.
Mosseri, who sat down with TIME to talk about bullying in mid-May, has said he wants to lead the industry in the fight against this perennial problem. By doing extensive user research, his hope is to develop a deep understanding of what bullying looks like on social media and create tools that other companies may eventually adopt. He believes that getting into the messy details, rather than slinging the equivalent of a sledgehammer, is key.
“You have to go deep into the nuances if you’re going to come up with more sophisticated answers,” he says. “On the issue of bullying, we want to be generative. We want to think about new and interesting ways to address the issue.”
Instagram will start testing Restrict in the next few weeks and plans to roll it out to all users before the end of the year.
Correction, July 8
The original version of this article misspelled the last name of designer Francesco Fogu. It is Fogu, not Fugo.
)
