If an account is not a person, then you don’t have to treat it with respect

Last week, I kicked off a series on the potential of using real-world identity to defend ourselves and our rights online. This week, I make the positive case. Next week, I will look at the negatives and risks. Note that this is my argument, not an effort to catalog all aspects of this ongoing industry debate.

According to the Dead Internet theory, nothing you see online has been real since 2016. It’s just an endless series of bots creating and commenting on each other’s fake content, enabled by social media and search algorithms designed to keep you distracted. Just a funny meme / crazy conspiracy theory, right?

Here’s the thing, though. How do you know that anything online is real? Consider this newsletter. How do you know the author is who they claim to be? Even if you have met Jonathan Bellack in person, how can you be sure that he is actually behind this newsletter? You can’t, because nothing authoritatively links an online account to a specific, real human being.

This is not a new problem. In The Invention of News (2014), Andrew Pettegree explains that in the middle ages, “a news report delivered verbally by a trusted friend or messenger was far more likely to be believed than an anonymous written report.” Even in the mid-16th century, when printed pamphlets were everywhere, “News was now a commercial transaction. Did this not undermine the credit of the information? How could one know what to believe from these unknown anonymous correspondents?”

Without assurance of real-world identity, we are vulnerable. Fake accounts are a gift to harassers, who can simulate an army of haters from their basement. Phishing attackers can easily impersonate official company representatives in order to steal from you. Pig butchering scams earn millions of dollars, because victims have no way to know that the beautiful woman with financial connections who has fallen in love with them over text chats is actually a human trafficking victim being forced to lie.

This is, of course, exponentially more dangerous for children. The New York Times covered a disturbing story of an adult predator pretending for years to be just another child on Roblox. Or consider the case of the woman who used easily-available fake accounts and burner phone numbers to cyberbully her own teenage daughter and her boyfriend.

Fake accounts are also a national security threat. The Russian Internet Research Agency is infamous for trying to influence Western elections, and even now, Russian bots are targeting Ukraine on Telegram. North Koreans are earning hundreds of millions of dollars for the regime by using fake identities to work remote tech jobs at US companies. 

Platforms don’t verify the real-world identity of every new account holder because their business model is built around growth. Many platform policies explicitly allow multiple accounts from the same device, because it encourages more usage.

Bad actors abuse these open front doors. They make themselves look like different people to bulk-create hundreds of thousands of accounts automatically. They operate data centers stacked with remote-controlled physical phones that simulate huge populations of real people with believable usage patterns.

Platform safety teams work hard and catch the majority of bad accounts quickly, but because there’s no way to uniquely identify repeat offenders, bad guys can just tweak their evasion settings and try again, so eventually sheer volume means a lot of bad accounts get through undetected.

I saw this myself at Google. There is an active and well-supplied black market for compromised online accounts. For example, social media accounts only cost about the same as a Starbucks latte. To Google’s credit, our accounts went for at least 10 times that rate, but the bad accounts were still out there. For one large country in Asia, 97% of new AdSense accounts ended up being terminated for fraud. We spent months debating how to make the abuse harder, or if we should even exit the market, but ultimately nothing changed, because the strategy was openness to support growth. As they say in Dune, the spice must flow.

Ignoring real-world identity also helps platforms evade accountability. If an account isn’t a person, then you don’t have to treat it with respect. A row in a database doesn’t have rights, and doesn’t get a say in decision-making.

I think this is the underlying, perhaps subconscious, reason that blue-check verification programs are so spotty and inconsistent. Platforms need to avoid the embarrassment of obviously fake accounts for famous real-world people, but it’s easier in the long run to keep verification rare and expensive. It’s much cheaper for the platforms if they can treat most accounts as disposable abstractions. Imagine if Facebook had to run their cross-check program for three billion people.

If online platforms required strong proof of real-world identity to open an account, it would not stop all fraud, but it would be much harder for bad actors to attack us at scale. If we had confidence that every account we interacted with online belonged to a real-world person, we could request more information to judge their trustworthiness – location, employer, age, track record, etc. People could also choose to proactively share provable real-world credentials, such as degrees or certifications, to establish their authority on important topics of the day.

Real-world identity would also give us a leg to stand on against bad platform decisions. They would need to be more careful about terminating real people, and would have a harder time justifying secret processes that ignore our legitimate voice.

This is not an unreasonable standard. Lots of real-world interactions require us to share our real-world identity — taking out a mortgage, renting a vehicle, boarding a plane, checking into a hotel, and more. Passports and driver’s licenses can be faked, but countries work hard to make it harder every year.

At a minimum, this would help us avoid the fate of bankruptcy attorney Mark Zuckerberg, who has had to resort to suing Facebook over repeated account shutdowns because Meta can’t seem to wrap its head around the simultaneous existence of more than one person with the same name.

I know that a bunch of you have serious objections to this optimistic pitch. This week was about the affirmative case, so I glossed over a lot of potential problems. I promise that next week, I will air as many objections as I can. If you want to help me rip my own piece to shreds, please feel free to send me thoughts and critiques at [email protected].