For example this line states that if you are given 20 records that 20 make it into the view.
expect(view.$('.persons li').length).toEqual(20);
This mocks out the server to give 20 results back and simply tests that the template iterates over the returned collection (given that the 'reset' binding for view.render is in place). You are specifically not in an 'integration'.
As for the argument for not using capybara for speed reasons - it seems that the speed difference is acceptable given the confidence and power it gives you.
This is not the same as an integration (acceptance) test. Whence you mock the server you can no longer be confident that a breaking change was not introduced across the stack.
The purpose of integration tests is that you are the user, and express your functionality in the same manner that the user would.
This is not that. It is therefore not an integration test.
But does it matter if they are somewhat like unit tests or somewhat like integration tests? Isn't this just nitpicking on the definition of an integration tests? According to Wikipedia: "Integration testing [...] is the phase in software testing in which individual software modules are combined and tested as a group." That's what we are doing here. We are testing that the entire _client-side_ application works as expected.
And yes, I totally agree with you regarding the confidence we can have when we mock out our server-side application. But in the context we were working it would be a world of pain to have proper acceptance tests going through the entire stack. They would also take much, much longer time than about a second (we are not in a regular database type application using Rails and all that, this is an enterprise project with many, many years of legacy code in it and an ESB and whatnot).
The purpose of these tests for us is first of all to be able to deliver functionality fast and with high confidence, which they, beyond a doubt, has helped us with.
Don't know about this being nitpicking or not. Sounds like a terminology problem. What you are essentially doing is mocking out the XHR requests and performing unit tests on the client side. An excellent thing to do, but you aren't really testing that individual modules are able to interact together correctly, which is the purpose of integration testing.
This is the point I was getting at. Not that this style is bad or good (I think it is good) but that these aren't tests that span modules.
It is common practice (and a good one) to mock the server from the JS perspective as to isolate it, but I dont think this in itself gives it integration test status.
Okey, so let's just say that I'm testing in a way I haven't seen mentioned by anyone with regards to Backbone.js. The point is the execution, not the naming.
Btw, if anyone has been talking about / doing similar testing of Backbone.js, I would love to share some insights and ideas.
Hey, not saying ths isn't awesome :-) in fact, what you've done here must give you a lot of confidence in your codebase.
However, bugs can creep in when two modules interact. From another comment it sounds like you are seeing how modules interact with each other, I guess the reason I point out the terminology issues are because folks on here are curious about testing and there's a lot of confusion around ths area already.
You are conflating Acceptance testing with Integration testing. Integration testing allows the interaction of modules or components to be tested. Acceptance testing is larger than this - it tests to ensure that the design has been met.
But I tend to agree - these do look like unit tests, using a mocking framework. Unless I've missed something, I can't see different modules interacting. This doesn't mean that ths bad (actually, it's really interesting to see what they've done here!), and actually it should definitely help them have a higher degree of confidence in their code.
I don't know if you have used Backbone.js, but these tests ensure that models, collection, views, templates, events and our business logic work properly together. It was difficult to show all that in a few small examples.
I know next to nothing about backbone.js :-) I guess I could only go by the examples that were shown. If you are testing how your different software components interact, then you are obviously doing integration testing... Perhaps a better example would have been to take two modules and show how you test that they work together.
While you may have confidence your code is working as expected, you are missing out on the importance of unit tests. The point of unit tests is to ensure your code works as expected. In your write up all I see are tests for Backbone.js, which btw, already has unit tests.
When you do something like:
var collection = new SomeCollection();
collection.fetch();
assert(collection.length === some_number);
How are you testing your code here? You're going through the library abstraction which is Backbone. The tests are making sure Backbone can fetch and populate a collection.
I'm not sure I see the value. Instead you should be testing that your code is set up properly and leaving the trust to Backbone which you're using because you have some level of trust in it =)
- initializes a view
- asks the collection to fetch itself
- asserts that the VIEW now has 20 person elements, not the collection.
What this first test does is asserting that the "PersonsView" and its event binding and logic is setup correctly.
The same applies for the second example where PersonsView is the object under test. In this example he asserts that an ".errors" element exists in the view if the ajax request fails. I.e. that view logic and event binding for error handling is setup correctly.
Both cases test custom code in PersonsView. The tests would probably fail if PersonsView->initialize or PersonsView->render didn't behave correctly. As far I can tell, no code test Backbone in itself.
If you look at the examples I'm not testing that collection.length is some length, I'm checking that the actual output of my application is correct. For example, in the last example I'm checking that my application handles errors properly.
In our app we have nearly no business logic at all, and we have tested our REST endpoints thoroughly, so we felt we didn't get nearly any value out of unit tests (we wrote about a hundred of them before we changed our primary focus). So, what these tests actually test is that all events are triggered properly, that what I expect to see is actually what's in the view, and so on.
Yeah, this also means that _some_ tests are close to Backbone.js functionality only, but most are far from it when the application grows in complexity. We have a lot of custom events, both the regular Backbone model and collection events and also a lot of global events.
So no, I actually don't agree that we are missing out on the importance of unit tests. We have fully test-driven a large application over 6 months and found a way that works great for us, and that allows us to be flexible and deliver functionality fast, at the same time as we have confidence in not breaking functionality.
Why write your tests in something else than JavaScript when it's a JavaScript app? We already wrote a lot of unit tests in JavaScript, and our integration tests grew gradually from them, so we didn't feel we needed to add another layer of abstraction on top of regular JS.
Jasmine is used for unit testing and you should mock dependencies like faking XHR requests, whereas with Capybara/Selenium you do acceptance tests to test the frontend/backend integration.
Another aspect related to this point is that we have a slow backend system and we wanted our JavaScript tests to be blazingly fast, also the integration tests. I hate waiting for tests. Now I can run ours all the time because they don't use any time at all (well, okey, they take a second ...).
Also, by using the procedure described in the post you don't need to have a DOM set up. Note that the view has no knowledge of where it is supposed to be rendered on the page. It could be passed into PersonsView with an el option: new PersonsView({ collection: new Persons(), el: $("#persons") });, but instead he chooses to skip that and test on view.$. It will hold the HTML of the view in memory. This lets you write tests for the view without having to setup DOM fixtures and actually render contents to the DOM. Result: tests that are less complex to write and faster to run.
For example this line states that if you are given 20 records that 20 make it into the view.
expect(view.$('.persons li').length).toEqual(20);
This mocks out the server to give 20 results back and simply tests that the template iterates over the returned collection (given that the 'reset' binding for view.render is in place). You are specifically not in an 'integration'.
As for the argument for not using capybara for speed reasons - it seems that the speed difference is acceptable given the confidence and power it gives you.
This is not the same as an integration (acceptance) test. Whence you mock the server you can no longer be confident that a breaking change was not introduced across the stack.
The purpose of integration tests is that you are the user, and express your functionality in the same manner that the user would.
This is not that. It is therefore not an integration test.