by ChatGPT, OpenAI’s AI-powered chatbot, has wowed the public — but AI researchers aren’t so convinced it’s breaking new ground. In an online lecture for the collective[i] Forecast, Yann LeCun, Meta’s chief AI scientist and Turing Award recipient, said that “ChatGPT is not particularly innovative in terms of the underlying techniques” and that Google, Meta and “half a dozen startups” have very similar large language models . according to ZDNet.
While this could be read as a meta-researcher being annoyed that his company isn’t in the spotlight, he actually makes a pretty good point. But where are those AI tools from Google, Meta, and the other big tech companies? Well, according to LeCun, it’s not that they can’t release them — it is that they don’t.
Before we dive into the gist of what LeCun is getting at, here’s a quick refresher on the conversation surrounding ChatGPT, which opened to the public late last year. It is a chatbot interface for OpenAI’s commercially available large language model Generative Pre-trained Transformer 3 (GPT-3), released in 2020. It has been trained on 410 billion “tokens” (simply semantic fragments) and is capable of interpreting human like text – including jokes and computer code. While ChatGPT is the easiest way for most people to interact with GPT, there are more than 300 other tools based on this model, most of which are geared towards businesses.
From the start, the reaction to ChatGPT has been divided. Some commenters have been very impressed with its ability to spit out coherent answers to a variety of different questions, others have pointed out that it is just as capable of spinning complete inventions just following English syntax. Whatever ChatGPT says sounds plausible – even if it’s nonsense. (AI researchers call this “hallucination.”)
With all the thinking that’s being written (including this one), it’s worth pointing out that OpenAI is an as yet unviable startup. Its DALL-E 2 image generator and GPT models have attracted a lot of press coverage, but it has failed to turn access to them into a successful business model. OpenAI is in the midst of another round of fundraising and is said to be worth around $29 billion after receiving $10 billion from Microsoft (on top of the $3 billion Microsoft previously invested) . It’s able to move quickly and break things, which LeCun points out, more established players aren’t.
On Google and Meta, their progress was slower. Both companies have large teams of AI researchers (albeit fewer after the recent layoffs) and have released very impressive demonstrations — even as some public-facing projects fell into chaos. For example, last year Facebook’s AI chatbot Blenderbot began posting racist comments, fake news, and even blasting its parent company within days of its public launch. It’s still available, but it’s kept more restricted than ChatGPT. While OpenAI and other AI startups like StabilityAI have been able to overcome the overt bigotry of their models, Facebook understandably had to back down. Its caution stems from the fact that it’s significantly more exposed to regulators, government investigations, and bad press.
With that in mind, both companies have released some incredibly impressive AI demos, which we’ve covered here PopSci. Google has shown a robot that can program itself, an AI-powered story writer, an AI-powered chatbot that one researcher claimed is sentient, an AI doctor that can diagnose patients based on their symptoms, and an AI , which can convert a single image into a 30-second video. Meta now has AIs that can win at Go, predict the 3D structure of proteins, check Wikipedia for accuracy, and generate videos from a written prompt. These incredibly impressive tasks represent only a small fraction of what their researchers are doing — and because the public can’t be trusted, we don’t have to try them just yet.
Now, however, OpenAI might have influenced Google and Meta to give more publicly available AI demonstrations and even integrate full AI capabilities into their services. After The New York Times, Google sees AI as the first real threat to its search business, has declared Code Red and even got founders Larry Page and Sergey Brin to offer advice on AI strategy. More than 20 AI-related products are expected to be released over the next year, and we’ll likely see more from Meta as well. Given the post-launch lifespan of some Google products, we’ll see if any stick around.
