Hallucinations generated by Large Language Models (LLMs) are a feature of their operations, not a bug. Hallucinations are the phenomenon of what appears to us as models making up facts. Because this is exactly what is happening! Models use vast amounts of training data (including incorrect information) to generalize reasoning and then generate output using probabilistic algorithms. No matter the model, the unpredictable nature of output carries the risk of inaccuracy. LLMs do not deal in the world of facts - they deal in the world of probabilistic output using a system of parameters and weights.
The fundamental issue, however, is that they do this with such an air of authority! This is a significant trap for the unwary and a serious risk for organizations moving output throughout their organization and out into the marketplace absent safeguards in place. Caveat emptor.
Without going into the technical detail here, suffice it to say that various operational techniques including refined and structured prompting, Retrieval Augmented Generation from vector databases, knowledge graph retrieval, specific domain models, and AI validation with current research, can all help to alleviate this problem. With new research from Lamini outlined in this thoughtful piece from Dr. Ian Cutress, it is good to see that strides are being made with new approaches to minimize hallucinations at the model level.
Ultimately, however, nothing is risk-free. And so it is with Generative AI, which is why effective procedures and human oversight are so critical with the use of this technology, particularly with the wide variety of models available and the use cases for their application. The human-in-the-loop requirement is here to stay for a multitude of reasons, not least of which are self-protection and institutional learning. This is especially necessary with an evolving technology of emerging and currently unknown risks. A business deployer of AI is already exposed to a number of risks including inaccuracy, intellectual property violation, bias, malevolent actors and data breach. Governance is therefore vital both to ethics and security and is a hallmark of responsible AI use.
About R. Scott Jones
I am a Partner in Generative Consulting, an attorney and CEO of Veritai. I am a frequent writer on matters relating to Generative AI and its successful deployment, both from a user perspective and that of the wider community.