It could be, yeah, in an alternate universe where the Evangelical and more recently, Catholic churches weren't leading the movement; where the politicians advancing anti-abortion laws weren't literally arguing that it's god's will; where the same religious activists and politicians weren't passing laws to harm LGBT youth arguing that it's god's will; where censorship and library closures weren't being enacted and justified on the basis of god's will; where there wasn't a significant proportion of politicians and even justices describing the US as a Christian state. In that universe, one could argue that the movement against women's rights wasn't based on religion. But in this universe, you must be a liar or a fool to believe such a thing.
All I'm saying is that religion isn't the only basis for being pro-abortion. The debate must still happen in a 100% secular society.
For example, why are ~50% of US men anti-abortion and only ~33% of women against abortion? Is it because men are more religious? Are 33% of women against abortion only because they are religious while 67% of women are athiests?
It's naive to think that a secular society is a unanimous one.
To give you an example, let's just consider people who support abortion: do they all agree that you should be able to abort at any point up until birth?
Opinions are a mixture of different influences. It seems fairly logical that fewer women would be against abortion since it is something they have skin in the game with. A person can be religious, but if one of their beliefs is tested by their circumstances, they might be much more flexible with it and remain inflexible with untested beliefs.