as a person with a toddler, I would LOVE if more people wanted to hang out. having a young child is very tough if you already have a job, but when my friends join me for walks where I push my daughter around in a stroller, its great. Having a toddler does not make people want to be alone!
It gets exponentially worse because every parent is aware of how their kid can be difficult, and so you really start to have this inclination to not inflict them on your friends, which makes it worse and a cycle begins.