United Federation of Ridiculous Plot Devices
If you think human laws are harsh, be glad you don’t live on the planet Rubicun III. In a 1987 episode of Star Trek: The Next Generation, teenage ensign/mascot Wesley Crusher goes there on shore leave, trips over a fence, and smashes a small greenhouse. This turns out to be against the law, and the punishment for every crime on Rubicun III is immediate execution by “mediators” who act as on-site police and judges. Hijinks ensue.
This is obviously meant to be absurd, but online platforms today aren’t that different. If you violate their policies, you may get a warning or a temporary suspension first, but the ultimate punishment is almost always account termination.
Platforms will tell you this is simply the online equivalent of a real-world business refusing service to rude or unsafe customers. The analogy might have worked once, but today’s large platforms are nothing like corner stores.
We have spent decades becoming dependent on platforms as the home of our local and global communities, including, critically, all of our contact information. We trust them with our important files and to manage our primary computing device. Recent antitheft features of Android and iPhone mean that if your account is terminated, you may not be able to log in to your phone at all!
This reliance means that losing one of your primary online accounts is much more disruptive to your life than getting kicked out of a Starbucks for stealing all the sugar. When your online storage account is cut off, it’s the equivalent of a devastating house fire where you lose all your family photos and financial records. What if your boss usually emails you your shift schedule? Could you lose your job? What if you share custody of a child and you can’t receive urgent text messages about a crisis?
We are no longer fully ourselves without our online identities.
One punishment good, four punishments better?
Account termination is a messy combination of four real-world forms of punishment.
Exile forces you to leave your native country. Think Napoleon or the Dalai Lama. When you lose your account, you aren’t supposed to be able to come back in again. (Deportation is related, but generally refers to non-citizens.)
Denaturalization strips you of your citizenship and all associated rights. In Google’s case, you can’t even log in with a terminated account to file an appeal. You need to effectively get citizenship elsewhere just to fill out the appeal form, by setting up an alternate email account with another provider.
Forfeiture is when the government takes away your property. Services like Facebook export, Google Takeout, and YouTube Studio don’t work on terminated accounts, so losing your account means losing all of your stuff, even if some of it had nothing to do with the policy violation in the first place.
Unpersoning is a term from George Orwell’s 1984. All images and records of you are removed from society, as if you never happened. Stalin is infamous for airbrushing people out of photos once they fell out of favor. When you lose your social media account, the digital airbrushes go to work and rewrite history. All of your past posts are deleted and your account no longer appears in search. Most of the time, there’s no way for anyone to know you once existed but were banned.
People simply disappeared, always during the night. Your name was removed from the registers, every record of everything you had ever done was wiped out, your one-time existence was denied and then forgotten. You were abolished, annihilated: vaporized was the usual word.
This is too high a price to pay for safety
Platforms are ubiquitous and essential to our lives, which means our online accounts have become part of who we are. They should no longer be treated as an arms-length transaction that can be entered into lightly and taken away at will.
Some terminated accounts belong to egregious violators, and I am not suggesting we tolerate harmful behavior, but even the worst real-world criminals have rights. Plus, we have plenty of public evidence that the platforms get it wrong a lot. The New York Times has written about parents who lost their Google accounts (sometimes due to their children’s behavior), and the BBC has reported on over 500 complaints about inexplicable Instagram terminations.
Because platforms don’t share much information about their enforcement statistics, we don’t know how many incorrect terminations never make the news. We should not accept platforms asking us to blindly accept that the current level of false positives are an acceptable side effect of online safety. The decision about where to draw the line between group safety and individual rights is a significant policy issue that affects the lives of billions of people. It should be debated openly with public participation, not determined behind closed doors by profit-driven platform executives.
Leaving the front door open is an explanation, not an excuse
Platforms would respond that my idealism is impossible at scale. Platforms see millions of new accounts every day, and the vast majority of terminations remove obviously fraudulent accounts. In recent weeks Meta reported that it took down 10 million Facebook profiles just for impersonating large content producers and 6.8 million WhatsApp accounts linked to scam operations. A lengthy judicial process for every termination could never keep up, and the platforms would die under the weight of unblocked junk and scams.
The catch is that platforms are doing this to themselves. The Internet is flooded with fraudulent accounts because platforms make it so easy to get one. Growth teams love easy account creation, because you get more usage when you let anybody (or any bot) in. So what if some bad guys slip through? Hire a safety team and make them sort it out later. If that leads to pervasive fraud, unchecked surveillance in the name of safety, and alienating our best users, so be it. The numbers must go up.
Maybe on the Internet, everybody should know if you’re a dog
Let’s review. Platforms’ open front door policy leads to massive numbers of fraudulent accounts, which requires a scaled-up approach to rapid account termination, which ends up causing an unknown number of real people to lose their entire online lives. Sometimes this is overly harsh punishment for a real violation, but sometimes it’s a mistake that hurts an honest person. If we care about justice, both outcomes are bad.
If we want to protect the rights of real people, it should be much harder to take away someone’s account and online property. But that only works if we have a way to know which accounts belong to a real and specific human with an authentic connection to the community. That implies more stringent validation requirements to get and keep an account.
This brings us to the problem of online identity. My quick sketch above makes the answer sound obvious, but equally important human values point in other directions. For example, see the UK’s recent, much-criticized rollout of age verification. This is a social and human dilemma that does not have a simple technical solution. After decades online, however, I don’t see a path forward that doesn’t require us to grapple with the problem of identity. So, I might as well get into it here. More next week.
I have a favor to ask. If you’ve read this far, you’re one of my most loyal readers and I am grateful for your support. Could you let a friend or colleague know about Platformocracy? That could be as easy as forwarding this email (or link if you’re reading on the Web) to someone you think would find it interesting or useful. Thank you.
Ideas? Feedback? Criticism? I want to hear it, because I am sure that I am going to get a lot of things wrong along the way. I will share what I learn with the community as we go. Reach out any time at [email protected].
