You already elect politicians. If that system is producing people you don't trust to manage the affairs of state, why would electing prosecutors lead to different results?
Hard to say, really. It's one of those compromises, quis custodiet and all that.
I very much agree with you: a government has a monopoly on violence and ultimately we all end up trusting it. Too many checks and balances lead to gridlock. Too few lead to oppression. Much of it ends up being decided on inertia. We do it both ways in different jurisdictions, with successes and failures in both.