There's one thing I don't understand. These chemicals are ubiquitous in the environment because they're non-reactive - nothing breaks them down. But if they're non-reactive, how do they affect our health?
Chemicals don't need to chemically react. Especially for biology, merely existing is enough. Lots of biological regulation is done by influencing three-dimensional shapes of the huge highly complex structures called proteins. Organic chemistry especially is a science of complex three dimensional structures and its dynamic changes, at least as important as actually altering the chemical structure. Often they are not static but have moving parts, because not all bonds restrict movement of the atoms connected through that bond in relation to the other part.
Many have what's call binding sites - other molecules fitting to those sites won't be altered or alter the chemical structure of the protein. They alter the shape. Proteins are machines, pretty much literally. For example, there are channel proteins, and a molecule docking on a binding site of such a protein might open or close that channel for certain molecules. The variety of those molecular machinery (really, literally) is huge. You can influence machines not just by breaking them down, you can just throw a spanner between their gears.
Things that matter are not just shape and size, but also polarity of the parts. Polar components of a molecule influence polar components of proteins if they can get close enough, which influence the shape.
