I had heard that Capyabara-based tests were going to be incorporated in Rails. As the multi-threaded concurrent nature of these always makes things a pain, hard to get right, leading to much pain for devs with non-trivial test suites -- I was curious to see how they'd handle it, if there'd be an 'official' Rails solution, ideally backed by some deep Rails knowledge.
I was surprised that they've chosen to use the 'shared database connection accross two threads' approach. While this was for a while popular approach amongst people trying to figure out the best way to use Capybara-based tests with Rails -- mostly because you get to keep transaction-based database cleanup -- it was largely determined through experience to be too flaky. ActiveRecord's architecture just wasn't designed for a Connection object (representing a single actual network connection) to be shared between two threads, it's not a thread-safe object.
José Valim originally proposed the approach in a gist, and you can see a long discussion among people having hard-to-debug problems with it here: https://gist.github.com/josevalim/470808
There are other such threads in various places from over the years.
I thought I remembered Valim actually posting somewhere himself "Yeah, turns out, that wasn't a great idea", but now I can't find it. (It may be that José deleted most of his tweets when he left twitter?)
So I'm surprised Rails took this approach. I can't tell for sure if the approach now in Rails just basically takes Valim's approach (violating AR's concurrency contracts and expectations), or actually takes a new approach that will truly be concurrency-safe, while somehow still sharing the connection (another lock somewhere? Are AR connections in general now safe for sharing between threads? Cause that'd be huge if it was true in the general case). https://github.com/rails/rails/pull/28083 Can anyone else?
Either way, reliable Capybara has been a real challenge to a lot of devs due to race conditions, it would be shocking if Rails gets it right on the first try, when many devs haven't managed to figure out a right way to do it in years of trying! I expect lots and lots of frustrated devs running into race conditions and filing Issues. It will be interesting to see if this ends up a maintained feature, or another Rails internal abandonware. If dhh and his team use it, and run into the race conditions, then it will certainly be addressed; otherwise, some 'difficult' Rails features seem to sometimes end up basically abandonware.
What kind of race conditions are you talking about? I ran this setup for years, and just about the only thing that has been stable in the whole setup is ActiveRecord.
You ran the shared connection situation with no problems? Great, but I linked to examples of people who have had different experiences, you can follow the links to see. Like all race conditions, sometimes they appear and sometimes not, and it can depend on your particular app.
They are not surprising, because the concurrency guarantees of AR do not include sharing a connection object between threads, it is not meant to be thread-safe.
With postgres, the race conditions on sharing a connection between two concurrent threads sometimes look like "PG::Error: connection is closed" or "PG::UnableToSend: another command is already in progress", with mysql sometimes like "Mysql2::Error: This connection is in use by: #<Thread:...>"
The Rails app runs in one thread, the test in another one. There are no problems as long as you don't use the database from the tests, which would be against the spirit of integration testing anyway. Only browser actions, even to check if the assertions are valid. A little inconvenient but safe and true to what users will do and see. I never got any problems in that way.
While integration tests may ideally not use the database (although I find in practice, I think you often need or want to do some 'manual' setup first. But even if you don't, the _other_ tests do db setup and teardown, and the problems start happening when the Rails app is still busy doing something triggered by a test that the test thread considers 'over'. Easy to say "Well, don't do that", but in practice it is very difficult to diagnose, debug, and stop. Using fancy new front-end frameworks like React or Angular tends to make it orders of magnitude worse.
If you haven't run into problems, consider yourself lucky and I don't hold it against you, but many have.
I was surprised that they've chosen to use the 'shared database connection accross two threads' approach. While this was for a while popular approach amongst people trying to figure out the best way to use Capybara-based tests with Rails -- mostly because you get to keep transaction-based database cleanup -- it was largely determined through experience to be too flaky. ActiveRecord's architecture just wasn't designed for a Connection object (representing a single actual network connection) to be shared between two threads, it's not a thread-safe object.
José Valim originally proposed the approach in a gist, and you can see a long discussion among people having hard-to-debug problems with it here: https://gist.github.com/josevalim/470808
There are other such threads in various places from over the years.
Here's one blogger explaining the race condition issue, with the ActiveRecord architecture not actually being concurrency-safe in this use case: http://influitive.github.io/news/2015/09/02/dont-use-shared-...
I thought I remembered Valim actually posting somewhere himself "Yeah, turns out, that wasn't a great idea", but now I can't find it. (It may be that José deleted most of his tweets when he left twitter?)
So I'm surprised Rails took this approach. I can't tell for sure if the approach now in Rails just basically takes Valim's approach (violating AR's concurrency contracts and expectations), or actually takes a new approach that will truly be concurrency-safe, while somehow still sharing the connection (another lock somewhere? Are AR connections in general now safe for sharing between threads? Cause that'd be huge if it was true in the general case). https://github.com/rails/rails/pull/28083 Can anyone else?
Either way, reliable Capybara has been a real challenge to a lot of devs due to race conditions, it would be shocking if Rails gets it right on the first try, when many devs haven't managed to figure out a right way to do it in years of trying! I expect lots and lots of frustrated devs running into race conditions and filing Issues. It will be interesting to see if this ends up a maintained feature, or another Rails internal abandonware. If dhh and his team use it, and run into the race conditions, then it will certainly be addressed; otherwise, some 'difficult' Rails features seem to sometimes end up basically abandonware.