Hacker News new | past | comments | ask | show | jobs | submit login

You edited this in after I replied, or maybe I missed it:

> If you have a condition that takes, on average, 1000 hours to occur, you have a 9 in 10 chance of missing it based on 100 error-free hours observed, and yet it will still affect nearly 100% of all of your users after a bit more than a month!

I understand the point you're trying to make here, but 1000 is just an incredibly unrealistically small number if we're modelling bugs like that. The real number might be on the order of millions. The effect you're describing in real life might take decades: weeks is unrealistic.




I agree that a realistic error rate for this particular bug is much lower than 1 in 1000 hours (or it would have long been caught by others).

But that makes your evidence of 100 error-free hours even less useful to make any predictions about stability!


> But that makes your evidence of 100 error-free hours even less useful to make any predictions about stability!

You're still conflating probabilities an event occurs across a group with the probability an event happens to one specific individual in that group (in this case, me). I'm talking about the second thing, and it's very much not the same.

If I could rewind the universe and replay it many many times, some portion of those times I will either be very lucky or very unlucky, and get an initial testing result that badly mispredicts my personal future. But we know that most of the time, I won't.

I can actually prove that. Because of the simple assumptions we're making, we can directly compute the probabilities we are initially that wrong:

  Odds we test 1000-hour bug for 1000 hours without tripping: 0.999^1000 = 36.7%
  Odds we test 50-hour bug for 100 hours without tripping: 0.98^100 = 13.3%
  Odds we test 10-hour bug for 100 hours without tripping: 0.9^100 = 0.002%
Under our spherical cow assumptions, my 100 hours is a very convincing demonstration the real bug rate is less than one per 10 hours.

Of course, in the real world, you might never hit the bug because you have to pat yourself on the head while singing Louie Louie and making three concurrent statx() calls on prime numbered CPUs with buffers off-by-one from 1GB alignment while Mars is in the fifth house to trigger it... it's just a model, after all.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: