<i> Should you decide that you don't want to use the DateTime library any longer you will have to not just change the code which is using it but the tests as well. </i>
This is not a bad thing. Ostensibly, you're making a change to code which implements a business function for a reason. If you're not using DateTime anymore, there should be a reason why.
Tests are supposed to be the proof that a business function is performed correctly.
If date/time is such a fundamental piece of logic that you have many tests that verify various conditions based on what day or time it is, and you change the fundamental WAY you represent date and time, then why is it bad that you have to make a lot of test changes as well as code changes?
Having those tests, and changing them over should be worth the effort to give you the confidence that your new implementation of how a fundamental of your code works the same way as before.
As far as the concept of checking "this method on this collaborator will be called X times," that can be painful to get correct the first time, but say you again change business logic and add a new condition. Is it not worth the effort to either think about "Oh, now I will have to call getHour one more time" and fix the test, or discover a broken test AND CONSIDER WHY you're being told "expected 2 calls but got 3" ?
Granted, if you're just changing times(2) to times(3) and going on, well, then don't bother. But that SHOULD be telling you something valuable. It's up to individual developers whether they choose to see that value or not.
Business logic doesn't require that you use DateTime. Business logic requires that the software understand dates in a certain way.
Some unit test styles enforce a given API at all levels in the code. Others restrict themselves to tests through certain restricted APIs.
For example, the SQLite code tests through the API, so it's possible to do major revisions of the code without changing the tests. The choice of how date logic works is invisible to the tests. Date tests can be done through SQL, without testing the actual code functionally equivalent to DateTime.
I prefer the latter. I liken it to building a bridge. The main test is "can you safely get traffic from X to Y", with other tests related to maintenance, beauty, environmental impact, and so on. This can start with a stone bridge, then a truss bridge, or a suspension bridge.
It's also possible to have tests like "there must be a pillar here, a truss there, an arch up there, and a screw over there", which requires that the bridge must be an iron bridge using a Pratt truss design, with three support columns made of granite.
Rebuilding/refactoring to even, say, a Parker truss would require rewriting some of the tests. Rebuilding/refactoring to, say, a truss arch requires a lot of rewritten tests.
But under the high-level tests, like for SQLite, major redesigns don't require major changes in the tests.
If at the start you know that you want a specific bridge, and you just have to work on it, then go ahead and write these very specific tests. Just remember that major refactors will require a lot of rewritten tests.
This is not a bad thing. Ostensibly, you're making a change to code which implements a business function for a reason. If you're not using DateTime anymore, there should be a reason why.
Tests are supposed to be the proof that a business function is performed correctly.
If date/time is such a fundamental piece of logic that you have many tests that verify various conditions based on what day or time it is, and you change the fundamental WAY you represent date and time, then why is it bad that you have to make a lot of test changes as well as code changes?
Having those tests, and changing them over should be worth the effort to give you the confidence that your new implementation of how a fundamental of your code works the same way as before.
As far as the concept of checking "this method on this collaborator will be called X times," that can be painful to get correct the first time, but say you again change business logic and add a new condition. Is it not worth the effort to either think about "Oh, now I will have to call getHour one more time" and fix the test, or discover a broken test AND CONSIDER WHY you're being told "expected 2 calls but got 3" ?
Granted, if you're just changing times(2) to times(3) and going on, well, then don't bother. But that SHOULD be telling you something valuable. It's up to individual developers whether they choose to see that value or not.