the problem is a layperson doesn't necessarily know what a prediction necessarily means without a deep understanding of how the system is making its predictions, let alone how to apply it.

worse is that since the prediction is coming from computer that lends the prediction an air of authority another article called "bias laundering". the general belief is that computers are objective and cannot have bias, which in a sense is true, but people don't tend to think a step further about the problems and biases in the people who programmed the computer.

so that is definitely a thing usually missing from these discussions is that the people using these systems generally don't know how they work, and believe they predict or imply things that they don't