Researchers have developed a mathematical formula that can predict when AI chatbots suddenly switch from providing helpful answers to potentially harmful ones.
The AI revolution of recent years has been technologically impressive, but it now seems as though the genie may be getting out of the bottle. Leading developers and scientists are warning about this, too. We need control, insight, and the awareness not to hand over every task and every field to AI.
AI makes mistakes because it does not think like a human.
Chatbots follow complex mathematical and statistical patterns. They do not “understand” the true meaning of words; they simply predict the most likely next word. That is why it makes sense to use mathematical formulas to try to predict their behavior.
In a new study, physics professor Neil Johnson and doctoral researcher Frank Yingjie Huo, from George Washington University, developed mathematical formulas to investigate this phenomenon. Their theory suggests that context may explain why AI makes mistakes.
Wei Xing, an artificial intelligence lecturer at the University of Sheffield, was not involved in the research but agreed that concerns about unsupervised offline systems are well-founded. However, he urged caution, pointing out that Johnson and Huo focused on a very narrow aspect of the problem and tested their theory on older models — GPT-2, EleutherAI Pythia, and Meta OPT, all open-source models — rather than on the AI assistants currently in use.
Further testing on the latest models will certainly be needed to determine whether this early-warning system for artificial intelligence still holds up. For now, however, Johnson and Huo’s research has highlighted an important fact: artificial intelligence can switch to undesirable behavior in the blink of an eye.
Interestingly, the AI itself agrees with the article’s central argument, although it believes that some of the claims need to be put into perspective. In particular, it questions the assertion that AI does not think in the same way humans do, as well as the suggestion that a mathematical formula could predict its mistakes in advance. This is a fascinating direction for research, it says, but it does not mean that every mistake can be predicted. — the AI itself, commenting on an article about AI.
