Let’s build due process! Part 3: Transparency should be the default, not the exception

A special welcome to the many new readers who found me last week through Alice Hunsberger’s updated mega list of AI safety/responsible AI resources on LinkedIn. Alice writes a weekly newsletter for Everything in Moderation called Trust & Safety Insider, which I highly recommend.

Welcome back to my series on due process (1, 2, 3, 4). Last week I proposed three practical due process improvements to the account termination procedure that would improve user trust:

This week, I suggest three transparency improvements that would cement these trust gains, and also provide durable benefits to the broader community.

Platforms could indicate whether your account has achieved valid user status (i.e. the platform believes it’s not fraudulent). This would increase user loyalty, and quite possibly usage, by reassuring people that they can rely on due process protections against losing their account and content unfairly. 

Going a step further, disclosing user status on every account’s profile as a trust signal could be an effective abuse-fighting tactic. Imagine seeing a warning like this in your social media user interface: “If you do not directly know the person behind this account, we recommend that you use caution about any claims or requests that they make. It has not yet been confirmed as a valid member of this community (based on time since creation, patterns of posts and interactions, etc).”

Making validity status public probably horrifies a lot of safety teams. This would be an obvious signpost for bad actors trying to evade platform defenses. Safety teams are already underfunded and overwhelmed. The last thing they need is to give their opponents another advantage.

I am sympathetic, but I can’t accept that the only solution to fighting the bad guys is to keep every real person in the dark. People who are building communities (and thus enterprise value) on platforms deserve to know when they have earned the company’s trust.

There is no obvious answer to who’s right in this debate. Secrecy, called security through obscurity in the professional safety community, has always been controversial. For me, the argument itself is another reason we need transparency. This should be a community discussion, not something that corporate employees decide for us behind closed doors.

Platforms could leave traces of suspended and terminated accounts in their user interfaces. External links could still work, and accounts could be findable via search, leading to redacted stubs that indicate their current status. [To be clear, this would not share violative content or profile details, and would only apply to previously-valid accounts, not the massive number of fraudulent accounts terminated in bulk.]

Keeping suspended accounts (and their content) visible during due process, even if redacted, would build trust in the platform’s due process. If users can see that a popular creator was suspended and pending judgement, not terminated, it will help to control the rumors and protests that swirl today when someone’s account vanishes abruptly without any indication from the platform of what happened.

Keeping traces of terminated accounts in the system would allow communities to build a sense of history. Knowing that an account was part of a community, but was terminated, is important information about the community’s boundaries and rules. When someone is unpersoned like in 1984, we lose that knowledge. Online communities shouldn’t be scrubbed to an eternal present like a Disney theme park.

Platforms could publish suspensions, judgements, and terminations to a publicly-available database. This would include the same information from the logged-in due process workflow – the policy in question, the platform’s evidence, the user’s response, and the final judgement. [Egregious cases and illegal content would need special consideration, but real-world courts have processes for sealing records in sensitive cases – see this guide from the Federal Judicial Center.]

A robust case library would build trust that judgements were fair and help to educate the community. Users could get a better understanding of what behavior crosses a given policy line by reviewing the history of related cases. Journalists, academics, and civil society organizations could learn more about how platforms operate, and how enforcement has evolved over time. And anyone who suspects a platform of biased enforcement could review the case for themselves, which might reduce the prevalence of paranoid conspiracy theories.

This repository could also help platforms improve outcomes in a cost-efficient way. Interested community members and researchers could analyze the data to detect patterns of abuse, repeat offenders, and policy mistakes. This is a proven approach – the Berkman Klein Center’s Lumen database of online content takedowns has been used repeatedly to discover attempts to abuse the DMCA (Digital Millennium Copyright Act) as a censorship tool.

Finally, persistent records could improve social trust by making it harder for bad actors to become repeat offenders. A public record means someone can’t lie about their conduct, or pretend that nothing happened.

Isn’t wiping the slate clean what we want sometimes, though, so people’s reputations aren’t permanently damaged? Yes, which is why paths exist to give people a fresh start. Online records can already be purged thanks to court cases around the world affirming a right to be forgotten, most prominently in the EU. And real-world criminal courts in many countries will expunge criminal records after a certain number of years with continued good behavior (see The Clean Slate Initiative).

Most transparency efforts will have objections and edge cases like those I mentioned above. Too often, these are used as excuses to withhold information from the community. This default secrecy isn’t just bad for people. It hurts the platform, too.

No matter how careful safety teams are, there are always going to be mistakes, controversial decisions, or angry users who stir up bad press. When everything has been done in secret until the public eruption, communications and legal teams are put on the back foot, running around internally to figure out what happened, why people are mad about it, and what to say or do. All this churn takes people away from their day jobs and hurts internal morale, as safety teams feel besieged by scrutiny and criticism.

What’s also remarkable is how often fire drills lead to giving the user their account back. This suggests platforms are in big trouble internally. It implies that either A) platforms are getting lots of decisions wrong, and we are only learning about the few that make the news, or B) platforms are not able to make principled decisions, and will violate their own policies given enough pressure.

Either way, the best thing platforms could do to prove they are operating in good faith and addressing internal problems is to be more open about what they are doing. Establishing transparent due process on account terminations would be a great place to start.

Ideas? Feedback? Criticism? I want to hear it, because I am sure that I am going to get a lot of things wrong along the way. I will share what I learn with the community as we go. Reach out any time at [email protected].