The thing is, some people did not commit any crimes. They get arrested because of false accusations, corrupt cops, and overzealous prosecutors. Many people have served sentences and then get exonerated. Our judicial system along with the prison system is broken...
Given the overwhelming incarceration of the US populace (beyond any other country), we could imagine that most prisoners are probably innocent, by the standards of more civilized societies.