Peter Watts has a series called Rifters that explores this a little. "Smart gels" which are neural nets made up of a mishmash of cultured neruons and silicon that run most of society. They're trained just like neural nets today, and therefore their decision making process is basically a black box. They do a great job, but no one is really sure how they get there, but they work great and they're so much cheaper, so they who cares.

Anyhow spoiler alert, the neural nets running the virus response have been inadvertently trained to prefer simple systems over complex ones without anyone realizing, and decide that a planet with no life on it after being wiped out from the virus is infinitely more simple than the present one and starts helping it out instead of stopping it.

So short answer to your question is I would not place much if any trust and systems like that, in as far as anything that has high stakes, real world consequences.