Drezner's Deep Thoughts On AI

Not deep-deep. Maybe quasi-deep?

Photo by julien Tromeur on Unsplash

Every once in a while, a confluence of teaching, writing, and reading obligations nudges me to think a lot more about a topic than I would under ordinary circumstances.¹ Over the past week or so, that topic has been artificial intelligence. It’s been quite some time since I have written about it. Five years ago, I posited that general artificial intelligence (AGI) might qualify as a general purpose technology. At the same time, there were aspects of AI discourse that I found to be, um, suspect. Even ChatGPT has left me intrigued but cold.

In 2024, I’m seeing and reading a lot about the current state of AI and I’m not entirely sure what to make of it all. According to one recent assessment, it’s a mortal threat to higher education and other sectors: “We all know that in the era of generative AI and large language models, the usual assessment suspects are dead. The essay, still beloved of many, is perhaps only the most obvious corpse. If we’re serious about preventing widespread plagiarism, many other traditional assessment methods are also no longer viable.”

The more I think about it, however, the more skeptical I am on the most doomsaying proclamations about the topic. Like the MOOCs craze, my takeaway is that AI will change some things but not everything and certainly not all at once.

Take, for example, Jon Stewart’s rather bizarre rant against AI — or at least the firms promoting AI — on The Daily Show:

Goodness knows that Silicon Valley CEOs are ripe for satire. But Stewart’s entire rant can be distilled down to a “lump of labor” fallacy that presumes rapid, mass unemployment across broad swathes of the economy. Just to be clear, such a thing would be bad for U.S. society even if markets eventually clear. But given that AI has been gathering momentum at the same time that unemployment has dropped to record lows, maybe Stewart’s fears are overstated just a wee bit?

Part of the problem is that Stewart accepts at face value the CEO claims that AI will displace jobs across multiple sectors. It is possible that one source of my calm about this came from something a colleague mentioned to me a few days ago. My longstanding presumption had been that artificial intelligence suffers from fewer cognitive biases than human beings — who, never forget, are barely evolved monkey brains — possess. That is spectacularly false. By their very programming, AIs are prone to overconfidence bias — they always think they have a good bead about what they are analyzing. The AI’s lack of metacognition is always its greatest weakness.

This is why, unlike Stewart and the CEOs he lampoons, I remain unconvinced that AIs will displace human laborers as rapidly as predicted. It might happen at some point, but like self-driving cars, it always seems to be five years away from happening.

Don’t take my word for it — take Ed Zitron’s, who recently speculated that we may have already reached Peak AI. Stewart’s technique of playing quotes from softball interviews with tech CEOs might very well be one source of misperception:

This is the problem with powerful people in tech. If you allow them to speak and fill in the gaps for them, they will happily do so. Murati and Altman continuously obfuscate how ChatGPT works, what it can do, what it could do, and profit handsomely from a complete lack of pushback from a press that routinely accepts AI executives' vague explanations at face value. OpenAI's messaging and explanations of what its technology can (or will) do have barely changed in the last few years, returning repeatedly to "eventually" and "in the future" and speaking in the vaguest ways about how businesses make money off of — let alone profit from — integrating generative AI….

I believe a large part of the artificial intelligence boom is hot air, pumped through a combination of executive bullshitting and a compliant media that will gladly write stories imagining what AI can do rather than focus on what it's actually doing. Notorious boss-advocate Chip Cutter of the Wall Street Journal wrote a piece last week about how AI is being integrated in the office, spending most of the article discussing how companies "might" use tech before digressing that every company he spoke to was using these tools experimentally and that they kept making mistakes….

Despite fears to the contrary, AI does not appear to be replacing a large number of workers, and when it has, the results have been pretty terrible. A study from Boston Consulting Group found that consultants that "solved business problems with OpenAI's GPT-4" performed 23% worse than those who didn't use it, even when the consultant was warned about the limitations of generative AI and the risk of hallucinations.

Another presumption about AI is that it is an arms race and China is poised to take the lead. Like clockwork, for the past few years former Google CEO Eric Schmidt sounds warnings that China is catching up to the United States in AI. Last year he wrote, “It is hard to say whether China will seize the lead in AI, but top officials in Beijing certainly think it will. In 2017, Beijing announced plans to become the global leader in artificial intelligence by 2030, and it may achieve that goal even earlier than expected.”

And yet, as of April 2024, this catch-up has yet to take place. Macro Polo’s Global AI Tracker recently concluded, “The United States remains the top destination for top-tier AI talent to work…. the United States remains far and away the leading destination for the world’s most elite AI talent (top ~2%) and remains home to 60% of top AI institutions.”

In February of this year the New York Times’ Paul Mozur, John Liu and Cade Metz surveyed AI experts and came away with a similar conclusion:

Even as the country races to build generative A.I., Chinese companies are relying almost entirely on underlying systems from the United States. China now lags the United States in generative A.I. by at least a year and may be falling further behind, according to more than a dozen tech industry insiders and leading engineers….

“Chinese companies are under tremendous pressure to keep abreast of U.S. innovations,” said Chris Nicholson, an investor with the venture capital firm Page One Ventures who focuses on A.I. technologies. The release of ChatGPT was “yet another Sputnik moment that China felt it had to respond to.”

Jenny Xiao, a partner at Leonis Capital, an investment firm that focuses on A.I.-powered companies, said the A.I. models that Chinese companies build from scratch “aren’t very good,” leading to many Chinese firms often using “fine-tuned versions of Western models.” She estimated China was two to three years behind the United States in generative A.I. developments….

When OpenAI released ChatGPT in November 2022, many Chinese firms were being hamstrung by a regulatory crackdown from Beijing that discouraged experimentation without government approval. Chinese tech companies were also burdened by censorship rules designed to manage public opinion and mute major opposition to the Chinese Communist Party.

Chinese companies with the resources to build a generative A.I. model faced a dilemma. If they created a chatbot that said the wrong thing, its makers would pay the price. And no one could be sure what might tumble out of a chatbot’s digital mouth.

AI might well prove to be a game-changer over the next decade. But right now an awful lot of what is said or written about AI falls into the category of “competent bullshit.”² In other words, the folks who claim AI will be immediately transformative sound way too much like they are parroting ChatGPT — and not in a good way.

1. In the case of reading, both fiction and non-fiction.

2. Is it possible that, five years from now, this column will fall into that category as well! My metacognition forces me to acknowledge this possibility!