Hacker News new | past | comments | ask | show | jobs | submit login

for 1/2, surprised to hear this because debugging models is usually a total black box and practically impossible. for 2, it's a similar problem where getting performance and accuracy using the same model over and over again on different problem sets can be challenging. not an AI expert or anything this has been my experience on the product side.



Responded to the same sentiment elsewhere but my general sense is that for many use cases users simply do not care about high 9s accuracy/consistency. A 95% solution using AI is "good enough" if you can ship it quickly and give them the tools to iterate on that last 5%.


95% solution might work for small startup X or small biz y but at large company scale 5% is a huge deviation to correct on. Maybe just depends on the client and how touchy they are. At my company, we measure metrics in bps and moving something 50 bps is a huge win. 500 bps would be unheard of.


IMO it's less about the size of the company and moreso the nature of the integration. Users are more forgiving of 95% accuracy when it's used to enhance/complement an existing (manual?) workflow than when it's used to wholesale replace it. The comparison would be building an AI tool to make data entry easier/faster for a human employee (making them say, 2x as productive even at 95%) versus an AI tool that bills itself as a full replacement for hiring a data entry function at all (requiring human or superhuman accuracy, edge case handling, maddening LLM debugging, etc).

In the long run the latter is of course more valuable and has a larger market, so it's understandable large corps would try to "shoot for the moon" and unlock that value, but for now the former is far far more practical. It's just a more natural way for the tech to get integrated and come to market, in most large corp settings per-head productivity is already a measurable and well understood metric. "Hands off" LLM workflows are totally new and are a much less certain value proposition, there will be some hesitation at adoption until solutions are proven and mature.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: