Opinions about AI are like AI logos
I’ve been trying to avoid the AI Discourse™ in this newsletter. There are plenty of pundits and industry insiders more than ready to jump on that grenade. Just look at my old employer at Harvard, the Berkman Klein Center for Internet and Society, who used to focus on Internet-related legal issues like copyright and censorship. They hired a Chief AI Scientist last year, and now the majority of their homepage focuses on AI news and events. (They mention it nine times!)
I am making an exception this week because of my father. He’s a distinguished retired clinical psychologist who keeps up with the news and occasionally sends me articles about the tech industry to get my point of view. Last week he asked me to comment on “Why Tech Oligarchs Are Willing to Risk Apocalypse” by New York Times opinion columnist Michelle Goldberg. I am publishing my response to him here verbatim, as a way to share my baseline thinking toward AI without diverting into a long deep-dive series. [If you haven’t read Goldberg’s piece, see below for context.]
Explaining AI doomerism to a retired psychologist over email
Here’s what I wrote to my father:
Tech leaders are too rich and too disconnected from the societies they are supposed to be part of. Compare it to Carnegie building libraries. The thing is, rich businessmen have always tended toward horrible behavior in one form or another. Think of railroad trusts, violent union busting, strip mining, dumping chemicals in the Hudson, etc. This is why we need regulation and oversight. Without it, the incentive to make ever larger amounts of money leads to irresponsible and dangerous behavior. See: thalidomide babies, the Ford Pinto, etc.
On the other hand, I am exhausted by people who try to wipe away AI as entirely bad for the world. They either claim that it doesn’t work, or that it’s somehow inherently corrupting. For me that’s exactly like medieval priests complaining about printing versus hand-written manuscripts.
My own view is that AI hype is beyond its current capabilities, but over time it will be a profound change. It is basically the equivalent of mass production but for human thought. Machines can’t build everything that a human can, and there is a market for handmade products. But many things that used to be time consuming and expensive to make by hand are now made easily by machines. Many types of human thinking are sufficiently simple and reproducible that we can program machines to do it quickly and automatically. Do that fast enough and at scale and it’s better. Like how chess machines can now beat the best human players through brute force calculations.
The argument is over how far up the ladder of human thought AI will get over time. Will AI be able to do work that previously required a PhD? Will it be able to replicate something like creativity? Will brute force thinking at enormous scale be the equivalent of genius? At a certain point, we are building the equivalent of infinite monkeys on infinite typewriters. Can we actually write Shakespeare that way?
Coming full circle, the real danger is the irresponsibility of tech oligarchs combined with this potential for scaled effects. There is a famous thought experiment that the world could end if you programmed an AI to make paper clips, and it doesn’t have any guardrails, so it ends up making everything it can find into paper clips, and eventually turns the whole world into paper clips.
The way I understand AIs are built, it is true that they do not have any inherent guardrails. The underlying large language model contains everything that was in the source text that trained it - helpful things like parsing spreadsheets, but also how to build weapons, how to do criminal behavior, how to make someone hate minorities, etc. All of the safety work called “alignment” is about trying to build a safety layer on top of the underlying model, to stop it talking about all the bad stuff it knows. The closest analogy would be to a sociopath. You might be able to teach them how to appear to treat people with kindness and respect, and even why society expects this of them, but they are only ever imitating the desired behavior, they cannot actually internalize it.
My personal opinion is that the issue is not AI safety in particular, but software safety more broadly. We have treated computer software as something that can be governed by market forces and liability in the courts. This creates risks everywhere. Construction used to be like this. Cities burned down and bridges collapsed. So over time we decided that safety required licensed architects, stamping plans, and expert inspections before anyone enters a building. Similar for medicine, or even food safety. It should not be easier to build an app that serves fifty million people than to open a restaurant serving fifty.
Context
For those of you who didn’t read Goldberg’s piece, it criticizes the messianic obsession with innovation held by tech billionaires like Peter Thiel and Marc Andreessen. She touches on all the greatest hits like transhumanism, long-termism, and the Techno-Optimist’s Manifesto. Goldberg also drops in a drive-by attack on the whole product category:
Part of the reason the public has turned so quickly and decisively against A.I. is the bizarre mismatch between the costs they’re being asked to bear and the rewards they expect to receive. Data centers blight the landscape, A.I. slop degrades the culture, chatbots help kids cheat in school and sometimes drive people mad, and technology executives prophesy mass joblessness. For this, ordinary people get improved search engines and help writing emails. Great benefits could be on the horizon — we are constantly assured that A.I. will enable huge medical advances — but they’re not here yet to justify the hazards.
Self-regulation is not representation
Long-time readers will know I am not a fan of billionaires who have gotten high on their own supply. I just don’t think it’s as simple as saying AI is bad because the cost/reward tradeoff is unfavorable. The problem here is, like in social media, people aren’t being given a voice or authority over technology that massively affects their lives.
The AI industry is grappling toward recognizing this and trying to define regulation on their own terms.
In July, Demis Hassabis from Google published A Framework for Frontier AI and the Dawning of a New Age, calling for “a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organisation, much like the Financial Industry Regulatory Authority (FINRA), with a board that includes independent leading technical experts and open-source representatives.”
Earlier this month Dario Amodei from Anthropic published We Must Pace the Frontier, proposing that “each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.”
The Information reported yesterday that they, along with OpenAI, are moving forward with plans for a self-regulatory body, tentatively called the Standards Authority for Frontier AI.
These proposals leave me cold because the economic relationship would be between AI companies and the standards / evaluator body. This reminds me of the Enron / Arthur Andersen scandal, where the accounting firm’s financial incentives led it to turn a blind eye to rampant fraud. These companies are not earning our trust when we have to read stories like ChatGPT Helped Tumbler Ridge Shooter Focus on Guns, Tactics, and Terror, in particular that they “had decided not to alert authorities, despite the urging of some employees on OpenAI’s safety team.”
FINRA is also a lousy model. FINRA is self-regulation on the behavior of broker-dealers and individual brokers. That falls far short of evaluating the safety of pervasive, powerful software products like frontier models. They should be overseen and inspected by a funded government agency, more like the SEC or the FDA.
Effective software safety and oversight needs to be independent from industry, not on their payroll.
Postscript: Gemini thinks I’m popular!
Amusingly, when using Google search to double-check if I’d written about Andreessen before, its AI mode wrote a succinct contrast of my worldview with Andreessen’s: “Bellack's Platformocracy is a critique of the exact techno-centric power structure that Andreessen defends and actively tries to build.” I also appreciate that Gemini called Platformocracy a “popular newsletter,” even when I checked the link in Incognito Mode to be sure it wasn’t just buttering me up personally. Thanks, friendly LLM! I guess AI is fine, after all.
Ideas? Feedback? Criticism? I want to hear it, because I am sure that I am going to get a lot of things wrong along the way. I will share what I learn with the community as we go. Reach out any time at [email protected].
