I used to be dogmatic about TDD. Then I joined a team where management doesn't value unit testing that much, and so I got a lot less religious about it.
What I am missing now isn't the test-first mentality. I honestly don't think there's a quality advantage to writing the tests before the code. And the danger of ardent TDD was always investing a ton of time ironing out a unit to perfection, only to realize you missed the forest for the trees and the unit, while perfect, has no place in the overall solution.
Rather, I miss the side benefits that come with it:
- Automated tests actually get written.
- It improves development cadence when you're writing a chunk of code you don't really want to be writing (by setting mini milestones).
So now I occasionally practice TDD for those reasons.
Interesting! Those benefits are also part of it for me.
The quality advantage I have noticed for TDD comes in better coverage. If I'm doing test-after, I already think the code works, so it's harder for me to notice the places where my thinking is wrong. If I'm doing test-first, I start out more skeptical and clear-headed.
I also think I design better, and for similar reasons. TDD starts me focused on the how the code appears from the outside, and then I make the implementation conform to that. If I start with the implementation, it's easier for me to get a little slack about the API; because my head is full with how it works internally, more of that can end up in the external interface.
The book Growing Object Oriented Software, Guided By Tests does a good job explaining how to avoid the "missing forests for the trees" problem. The short summary is to use integration and unit tests in tandem, which keeps you focused on the entire feature while also developing carefully tested units. The book explains it better than I do.
I've had great luck writing quality code by writing the tests first. Testing code is in a higher level -- simpler -- dialect of your main code, thus easier to understand. Gradually the tests cover more and more code, until I'm confident to go ahead and connect the functions into the main line of the system. If the code isn't 100% covered, that's fine, and if code is simple enough to not have tests at first, that's fine too.
Doing testing first helps ensure code is testable. I haven't found OP's issue of "overly complex web of intermediary objects" -- the Fudge mocking library helps alleviate that. Example: test code creates a fake urlopen(), which the receiving code uses instead of doing real I/O. No intermediates.
What I am missing now isn't the test-first mentality. I honestly don't think there's a quality advantage to writing the tests before the code. And the danger of ardent TDD was always investing a ton of time ironing out a unit to perfection, only to realize you missed the forest for the trees and the unit, while perfect, has no place in the overall solution.
Rather, I miss the side benefits that come with it:
- Automated tests actually get written.
- It improves development cadence when you're writing a chunk of code you don't really want to be writing (by setting mini milestones).
So now I occasionally practice TDD for those reasons.