You don't need a strong math background. I got through a few books (PDSH included) and a couple of MOOCs on data sci/ML over the last couple of years with high school level math skills + some extra reading. Not everything is explained in minute detail but there are plenty of other sources to supplement it if you want to go deeper.