In a live demo that OpenAI gave me yesterday, ChatGPT didn’t shine. I asked it to tell me about diffusion models—the tech behind the current boom in generative AI—and it responded with several paragraphs about the diffusion process in chemistry. Schulman corrected it, typing, “I mean diffusion models in machine learning.” ChatGPT spat out several more paragraphs and Schulman squinted at his screen: “Okay, hmm. It’s talking about something totally different.”
“Let’s say ‘generative image models like DALL-E,’” says Schulman. He looks at the response: “It’s totally wrong. It says DALL-E is a GAN.” But because ChatGPT is a chatbot, we can keep going. Schulman types: “I’ve read that DALL-E is a diffusion model.” This time ChatGPT gets it right, nailing it on the fourth try.
Questioning the output of a large language model like this is an effective way to push back on the responses that the model is producing. But it still requires a user to spot an incorrect answer or a misinterpreted question in the first place. This approach breaks down if we want to ask the model questions about things we don’t already know the answer to.
OpenAI acknowledges that fixing this flaw is hard. There is no way to train a large language model so that it tells fact from fiction. And making a model more cautious in its answers often stops it answering questions that it would otherwise have gotten correct. “We know that these models have real capabilities,” says Murati. “But it’s hard to know what’s useful and what’s not. It’s hard to trust their advice.”
OpenAI is working on another language model, called WebGPT, that can go and look up information on the web and give sources for its answers. Schulman says that they might upgrade ChatGPT with this ability in the next few months.
Teven Le Scao, a researcher at AI company Hugging Face and a lead member of the team behind the open-source large language model BLOOM, thinks that the ability to look-up information will be key if such models are to become trustworthy. “Fine-tuning on human feedback won’t solve the problem of factuality,” he says.
Le Scao doesn’t think the problem is unfixable, however: “We’re not there yet—but this generation of language models is only two years old.”
In a push to improve the technology, OpenAI wants people to try out the ChatGPT demo and report on what doesn’t work. It’s a good way to find flaws—and, perhaps one day, to fix them. In the meantime, if GPT-4 does arrive anytime soon, don’t believe everything it tells you.