With analytics now an indispensable component of value generation, the issue of information reliability has become critical. So it’s no surprise that AI hallucination (when GenAI produces nonsensical or inaccurate output) has recently taken center stage. The public learned that ChatGPT erroneously claimed a law professor had sexually harassed a student in a class, Google’s Gemini generated images deemed to be “woke” that were historically inaccurate, and Meta’s Galactica was shut down because of inaccuracies and bias.

Blind reliance on fabricated data from these hallucinations can have severe consequences. Blind reliance on any information source is never advisable, and ensuring data accuracy is fundamental in any analytical process. Ensuring data reliability is particularly challenging when GenAI presents fabricated information as factual, often with limited transparency about the source. AI hallucination is especially problematic for novice users who lack the technical expertise to verify such data.

Should You Care About GenAI Hallucinating?

Amid the ongoing debate about the potential benefits of GenAI versus possible areas for concern, there are decisions to be made. GenAI is here to stay, and lingering on the debate stage without acting will set us back as the market moves forward.

As discussed in our recent Amplify article, competing factors come into play to determine our tolerance to AI hallucination, so a “Here’s what you should do” statement addressing this phenomenon is not the answer. We believe that understanding the context is crucial to making informed decisions about strategically adopting and implementing emerging technologies. This understanding can help us balance the potential value of the opportunity with our risk tolerance.

Here are several key takeaways:

• Consider the national context around your firm’s operations. What regulations are in place to guide GenAI’s adoption and use? Do they align with your organization’s culture, or do they require a cultural shift? Are there practices in other countries that closely resemble your own that you could adopt?

• Consider where your firm falls within each cultural dimension to determine how accepting your company is of new systems and its tolerance to AI errors.

• Get to know the individuals that make up your organization; many have probably informally adopted these technologies. Consider their experiences and personalities when devising adoption strategies that align with their current practices.

• Establish clear regulations that encourage transparency when AI-generated data is being leveraged for decision-making, but ensure their enforcement aligns with your organization’s culture and your employee’s personalities.

• Foster education on GenAI use and its potential pitfalls to promote a culture of responsible use.

We must not dismiss the issues surrounding AI hallucinations. The potential benefits of GenAI are virtually endless, but the repercussions from misuse or blind reliance on its output can be highly detrimental. The good news is that a balance is possible. Key to that balance are: (1) having context at the fore of your adoption strategies, (2) accounting for tolerance levels, and (3) mitigating the risk of misuse while capitalizing on the many opportunities this technology offers.

[For more from the authors on this topic, see: “AI Hallucination in the Wake of GenAI.”]