Django is pretty mainstream. Its not as popular as Rails but its hardly a maverick.
I think the reason why Python never went all in on a single web framework is just because the Python community itself is much larger and more diverse than the Ruby community. Consequently there were alot more opinions and alot more people willing to develop those opinions into separate projects.
Maybe you're right that there are advantages to converging on a single framework, but I'm not so sure. Rails is omakase, so if there's some part of it you don't like its not really easy or a good idea to change it, and you don't have any other options in Ruby. Django is also omakase but if there's some part of Django that you really disagree with you can just move to web.py or flask or pyramid. these are all built on the common core of Python's wsgi module so these are just different opinions on how to accomplish the same task.
With Django in specific you can swap out a component. For example, if (like many people) you don't like the HTML template syntax, just swap it out for Jinja2 or Mako. In Django it is also usually easy to customize something minor by use of overrides (OOP).
This is also true of Rails. That's what the Gemfile is--a list of optional packages that you can swap out. (Almost) all of your configuration happens in that one file where you tell Rails what packages (gems) you want to use with your application.
this is very true for some of django's components. the template engine is probably the most opinionated part of django and is also one of the easiest parts to swap out. other parts of django are not so easily disposed of. for example, the ORM is very tightly integrated. its possible to swap it out for SQLAlchemy or something but this is really a huge headache and not worth doing.
I think the reason why Python never went all in on a single web framework is just because the Python community itself is much larger and more diverse than the Ruby community. Consequently there were alot more opinions and alot more people willing to develop those opinions into separate projects.
Maybe you're right that there are advantages to converging on a single framework, but I'm not so sure. Rails is omakase, so if there's some part of it you don't like its not really easy or a good idea to change it, and you don't have any other options in Ruby. Django is also omakase but if there's some part of Django that you really disagree with you can just move to web.py or flask or pyramid. these are all built on the common core of Python's wsgi module so these are just different opinions on how to accomplish the same task.