Copy-paste-change tests may be interesting, though. Too much fancyness makes tests difficult to follow in my experience.

Ideally, tests should take three steps:

  - setup
  - perform a single operation (the test)
  - check operation result is the expected one

If each step is simple and readable, is actually easy to follow, even if a lot of cases uses almost the same initial setup and expected result, making copy-paste-change a good way of adding tests.

If a change in functionality if big enough to require change a lot of tests, this way makes obvious is changing the behaviour in lots of cases, something good to keep in mind. Most times a simple editor replace suffices, but shows up how big the change is.

Refactoring should be focused in making those steps easy to write and read...

Yeah, I've done the same under something I've learned as AAA; Arrange, Act and Assert, which I think is the more familiar naming of a system like this. C2 has some more information about it over here: http://wiki.c2.com/?ArrangeActAssert

I often find it interesting the propensity to name things. In my opinion, this approach should be self-evident once the concept of unit tests is correctly understood - prepare the data, run the code, check the output.

I suppose this is another case where a common approach is given a name so that people can refer to it to each other and pass it onto others. I wouldn't have thought to name this one, though - I would rather teach a junior engineer how to think about what a test should do, instead of telling him to do "AAA".

With experience, I have seen the effect of something being given a name eventually leading to less experienced people treating it as a universal principle and wielding it as if all they have is a hammer, perhaps before having formed the habits of checking assumptions before making decisions, etc.

Quite a bit of the time, you don't want to rely on application logic to do your setup for you. You end up needing the application logic to put things into the DB, without any bugs, in order to be sure your test is actually testing the part of the code it's meant to be testing.

Allowing your application logic to do this for you creates all sorts of opportunity for hard to debug issues that have nothing to do with the thing you were actually testing. Putting data into the test DB directly makes for more setup, but makes for much more isolated test cases and much more certainty that your tests are doing what you think they're doing.

It also means you can write tests as you go without having to build a whole level of application logic to support your single test.

I think a problem with this is that when you bring an actual database into the test, it ceases to be a unit test and becomes an integration test (not that integration tests aren't useful). I would prefer for the application to be separated enough from the database that I can test the business logic entirely on its own.

I would suggest that, aside from at the boundaries of a system, if you find yourself requiring complex setup code then you should evaluate whether your abstractions are neatly decoupled and reduced to minimal necessary complexity.

I disagree. IMHO, modifying your app code for the sake of test code is an anti pattern. It should be viewed as an unfortunate yet necessary, rather than celebrated, action. It taints your design and makes your application harder to understand.

Juniors make plenty of mistakes, I would rather they make them then explain why they should change their code rather than them relying on "testable code = better code". I would admit that a junior working alone, or without code review/a mentor, is probably better off relying on that mantra though. Or, perhaps even anyone designing an application significantly larger than they have in the past (without any outside input).

However, everything has a cost, even abstractions. The more powerful the abstraction, the higher the cost. I find that for most of our projects, the sorts of abstractions unit testing forces on us is much more powerful - and thus higher cost - than necessary for the task at hand. If you are not taking advantage of that power, then all you really have left is the high cost.

If I have some object I want to test in multiple ways, I find it more convenient to create a single instance during setup and then testing various aspects of just that single instance. Since the constructor of the object in question needs a bunch of values passed in, and each unit test only checks the effects of one at a time, I'm just going to copy and paste a bunch of object initialization code if I don't have a setup step first.

This is not a good practice because a unit test should be able to assume that no other code can possibly modify the data it is using. Later, another engineer could possibly come in with the intent of modifying a single unit test in accordance with an isolated functional change in such a way that the data being used by other unit tests gets modified.

Then some poor fellow ends up redesigning the test suite to actually follow the assumptions of a unit test's test data being only ever accessed by that unit test. I have been that fellow, and it can be, bluntly, quite a pain in the ass; as you're doing it, you can't help but think, "whoever wrote this was too lazy to set up the test data properly, I shouldn't have to spend the time to fix this now."

Ah yes, I should have stated that the object is immutable.

Where are you getting the object to which you're going to send messages?

From the class.

    expect(Klass.new.special_case).to have_this_behavior

Anything more complicated in test code should be considered an anti-pattern.

Careful getting too DRY with your tests. Far too many projects have test that should be failing but aren't because of some silly side effect, caused by DRYness and lack of isolation.

I'd go as far to say that attempts at DRY cause more problems in test suites than copy-paste coding, many of which never even get caught. In a test suite, this is very bad.

We actually ran into this as a problem: our tests were so DRY that it was really hard to adapt them as the behaviour of our application changed.

We've since gone more DAMP (http://stackoverflow.com/questions/6453235/what-does-damp-no...): tests should be descriptive and meaningful by themselves, not abstract, concise and DRY.

It makes for more lines of test code but they are simpler to understand and adapt.

When you abstract away test code, you are abstracting code written to test a software application's design, and thus your abstractions (and test data) necessarily mirror your application design.

This is why I commented elsewhere that DRY should be applied not as a coding principle to all code, but as a design principle to software design.

Going too crazy on non-dry tests can make tests failures hard to track down, though. Meta-programming to generate tests has only lead to pain in my experience.

If you're following the red-green-refactor cycle properly, then you'll have seen test failures on DRYed tests. Most testing frameworks let you customize the failure message. It's usually a simple matter of adding more information about which part of it failed.

I won't meta-program for tests, but I will do things like make a list of classes, or symbols or whatever, to pass to a loop. Just keep it simple.

> I won't meta-program for tests, but I will do things like make a list of classes, or symbols or whatever, to pass to a loop. Just keep it simple.

This is a case where metaprogramming for tests came in handy.

I had a bug recently that involved someone making a change that violated an invariant property of a class. To codify this invariant, I was tempted to do what you did, to make a list of symbols to feed into my test to ensure the invariant was obeyed. However this bug was caused precisely by someone adding a new symbol, a method, that didn't obey this property. The test using this design wouldn't catch this failure. I instead opted to do some introspection (it's Python, so it was dead simple) on the class to ensure all of its methods obeyed this invariant. It took a little extra time to implement but in the end it worked.

With the caveat that I haven't invested the full amount of time necessary to strongly state this opinion publicly, I do feel that metaprogrammed tests are a case of adding abstraction unnecessarily (which, incidentally, is why I haven't taken the time to try to extensively use them on a project.)

A test should be dead simple to read and understand when someone else new to the project needs to understand what it tests. Further, when a test fails, the output should clearly indicate exactly on what line of code an error occurred.

Metaprogramming tests feels to me like a case of a desire for or predilection for cleverness getting in the way of what the task is actually for.

Tests should tell a story. They should not be subjected to the same methods of abstraction used in the code they themselves are supposed to test and verify.

>Meta-programming to generate tests has only lead to pain in my experience.

Likewise, although I view it as a deficiency of testing frameworks that require metaprogramming for parameterized testing. They shouldn't.

And on the other hand, I'm always suspicious of code that does something programming-like that is specific to the test framework.

If you need a loop, ideally you write the loop using the language, rather than using a test framework's special loop construct. For parameterization, ideally you'd use a function, rather than a special test framework concept for parameterization.

Having duals of every abstraction feature of a language in the test framework increases the cognitive load and ramp up time to become effective in a codebase, and there's always the risk of incomplete knowledge, leading to mixing and matching different abstractions - and test framework abstractions are usually more leaky and less well thought out than language abstractions.

Furthermore, they decrease readability and clarity of tests themselves. I am of the strong opinion that if you need to test 20 inputs, you should write 20 asserts, not a loop - when the test fails, you should be able to immediately tell _which_ input(s) failed, not just that a failure has occurred.

I came to this opinion by way of desiring to be clever in my tests and then finding myself having to insert `printf(input)` inside a loop in order to quickly figure out which input failed.

Code should be written so that it doesn't need to be modified. Feelings of aesthetics of desires to be clever should be avoided and are, ultimately, I think, often just vanity.

I'm thinking of rspec in particular, as it's the framework I write most tests in. It's trivial to include the loop variable in the string interpolation that describes the "context" block for the loop inner test setup. But usually I wouldn't be using a plain loop, I'd be iterating over entries from a hash map.

The same thing in something like junit is less workable, and copy and paste becomes more viable simply because the language is so clumsy and inexpressive. I've seen various extensions and annotations that let you parameterize junit tests but I don't think they make things much clearer.

A key benefit of driving the test cases off a table of inputs to outputs is that missing cases in cross products are much easier to spot. Visualising the state space and its coverage is easier when you can see a map of the covered space in one block of code.

The one place where I've had good results from meta programming / DRY for tests is table-based validation; where you have a large cross-product of different states to test, mapping to expected outcomes.

I disagree with this mindset and perceive it as an example of applying a common principle without thinking about what it was meant for.

The Don't Repeat Yourself principle, speaking from my internalized understanding without going back to wherever it was originally described and named as such, embodies the following general principles:

* A specific functionality should only be implemented in one place; implementing the same logic in multiple places creates the possibility of one or more of those implementations being modified without the rest being modified identically. * The total amount of code that must be maintained should be minimized reasonably, such as by eliminating duplicate code. * If the same functionality is being implemented times, that indicates that it may be an essential or otherwise important abstraction in the overall design of an application, and therefore should be abstracted.

That's the gist of it. Those principle apply to software design and architecture. Test code, however, is different in nature. It does not have abstraction and decoupling as a design goal; it is meant to test a software design in such a way as to be able to indicate specifically where an error is occurring. DRY actually works _against_ those goals - if you abstract away a test setup and make a mistake doing so, it is possible to, in the course of refactoring all relevant unit tests to use that abstraction, modify the unit tests to pass. If, in contrast, you keep setup functions entirely contained in each unit test class, you cannot possibly reach this situation, because you can only break one test at a time. The tests retain their function of testing specific source code and only breaking when that source code breaks.

DRY should not be applied without thinking to testing. It can be executed carefully and provide a benefit, but it is difficult to avoid the possibility of adding bugs to the abstractions you choose to make.

Follow up as it is too late to edit and I want to note an observation: this is why I dislike naming things unnecessarily. Giving an approach a name leads many people to treat it as some sort of absolute, because It Has Been Named, and to use it without question.

We should try to understand why we do the things we do instead of applying Named Methods because since they are named they must be absolute.

I am being hyperbolic for effect, but I do believe that this is a real effect, and I personally do hesitate to name things that don't really need a name.