LLM Hallucination and a Continental Divide
Mitigating risk and reducing validation cost; watersheds for Newton's method
This morning I wrote a post about something I often think about but have not articulated before. When I evaluate the risk of using generative AI, I have in the back of my mind the cost of validation and the cost of error.
Sometimes the hard part of solving a problem is proposing a solution, not validating it. It may not matter than an AI has a high error rate as long as it’s easy to tell whether it has made an error. And sometimes it just doesn’t matter that much if an AI is wrong, whether or not the result is hard to validate.
The danger zone is when a errors are consequential and hard to detect. Then you need to either mitigate the consequences of errors or find more economical ways of validating outputs.
Newton’s method does a great job if you start with an initial guess close enough to the solution you’re looking for. But how close is close enough? If you’re not close enough, will you converge to a different solution or not converge to anything at all?
In general these questions are hard to answer, but for quadratic polynomials there’s a satisfying characterization of what happens from any initial guess. There’s a line that acts as a sort of continental divide, separating two watersheds.
If your company could use some help with problems such as mitigating AI hallucination risk, or with computational problems such as improving the convergence of a numerical method, let’s talk.