> Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test?
So what we do where I work is for unit tests, we mock services and repositories, unit tests don't go anywhere near the database.
For integration tests, we use an in memory database.
BUT! Be careful before you embark down the path of running integration tests against a different database than the one you use when running the application. There are SO many pitfalls, nasty bugs, and other warts along that road. EntityFramework alone has quite some weirdness there. Expect these kinds of integration tests to cost a lot of developer effort to build and maintain. For us, it took months of effort to get these kinds of tests working usefully.
Personally after working on applications that have a solid test pyramid, I would recommend:
* Write unit tests as per the standard advice (bottom of pyramid kind of quantity), but try and keep it sane (don't go for 90% coverage just for the sake of hitting an arbitrary number; don't pointlessly unit test your framework/libraries/other dependencies)
* Write some integration tests where they really add value (interaction between 2 or a few complicated components in your system, for example places where the state of a component changes a lot depending on input from another component). Make sure when you start out writing an integration test that it doesn't turn into an "almost" end-to-end test along the way. They have a habit of doing this and it can really cost you later. Integration tests should still be focused.
* Write end-to-end tests that test as much of your system as possible, including a database (preferably the same vendor; one approach is to truncate all tables before each run). IME it is very good here to have one e2e test that covers a big chunk of functionality in one run, than lots of separate tests covering different things. Why? Because no matter how "different" the things being tested by the e2e test there will still be LOTS of overlap by its very nature, and changing anything where that overlap is will tend to break ALL your e2e tests. Not a fun workflow.
One last comment on E2E tests. This is more opinionated. But try and limit E2E tests to things that are really business critical (e.g. your "sign up" process and your "renew subscription" process, but not every single form in your dashboard). This is just a cost-benefit thing. E2E tests help, but they also slow you down, and the more you have, the more slowly you will be able to change your software. Sometimes minor bugs slipping through is an acceptable cost if it means you release 1 week sooner.
> For integration tests, we use an in memory database.
I think this can work for running the tests locally when speed is what you want, but you should still have the tests run on the actual database on commits. It requires keeping a database running just for tests to run (it can be a smaller instance and startup/shutdown on demand), but it will save so much pain later.
"BUT! Be careful before you embark down the path of running integration tests against a different database than the one you use when running the application. There are SO many pitfalls"
With docker and the other arguments the article points out I find there's very little value in an in-memory database. Just use either a stub/mock/dummy for unit test or spin up the real thing for integration test.
I've yet to have a good testing experience when using anything more than a trivial database if it has EntityFramework on top. The de facto standard seems to be Sqlite for unit tests, which causes some blocking issues if you have the same database name in two different schemas, or composite primary keys, to use my two most recent examples.
So what we do where I work is for unit tests, we mock services and repositories, unit tests don't go anywhere near the database.
For integration tests, we use an in memory database.
BUT! Be careful before you embark down the path of running integration tests against a different database than the one you use when running the application. There are SO many pitfalls, nasty bugs, and other warts along that road. EntityFramework alone has quite some weirdness there. Expect these kinds of integration tests to cost a lot of developer effort to build and maintain. For us, it took months of effort to get these kinds of tests working usefully.
Personally after working on applications that have a solid test pyramid, I would recommend:
* Write unit tests as per the standard advice (bottom of pyramid kind of quantity), but try and keep it sane (don't go for 90% coverage just for the sake of hitting an arbitrary number; don't pointlessly unit test your framework/libraries/other dependencies)
* Write some integration tests where they really add value (interaction between 2 or a few complicated components in your system, for example places where the state of a component changes a lot depending on input from another component). Make sure when you start out writing an integration test that it doesn't turn into an "almost" end-to-end test along the way. They have a habit of doing this and it can really cost you later. Integration tests should still be focused.
* Write end-to-end tests that test as much of your system as possible, including a database (preferably the same vendor; one approach is to truncate all tables before each run). IME it is very good here to have one e2e test that covers a big chunk of functionality in one run, than lots of separate tests covering different things. Why? Because no matter how "different" the things being tested by the e2e test there will still be LOTS of overlap by its very nature, and changing anything where that overlap is will tend to break ALL your e2e tests. Not a fun workflow.
One last comment on E2E tests. This is more opinionated. But try and limit E2E tests to things that are really business critical (e.g. your "sign up" process and your "renew subscription" process, but not every single form in your dashboard). This is just a cost-benefit thing. E2E tests help, but they also slow you down, and the more you have, the more slowly you will be able to change your software. Sometimes minor bugs slipping through is an acceptable cost if it means you release 1 week sooner.