I looked into TAP a while ago, but had a couple of problems with it:
1) The whole point is to allow language/harness agnostic tooling, but the only tool of note seems to be smoulder. Everything else seems to be trivial things like showing pass/fail with emoji tick/cross symbols, etc.
2) The TAP standard allows tests to be numbered, and a total count to be given up-front. In practice, all tools seem to require this, and barf when given unnumbered tests. This makes TAP far less useful, since it requires some global coordinating process to know how many tests we're going to run.
I was interested in TAP since I could write test scripts in any language which spit out a simple format to stdout, and they could be nested; e.g. we can have an overall "test everything" script which runs the test script for each of our projects/repos; projects might choose to have different scripts for unit tests, functional tests, integration tests, etc. and those tests might be spread across many files. Ideally we shouldn't have to care about any of that structure. With TAP we need some way to count the tests before running them (or else we have to wait for all of the tests to finish first, which is naff); this requires some sort of declarative structure, at which point our scripts can no longer be opaque black-boxes, so we might as well use a "proper" test harness like xUnit.
Was going to reply this too. The idea of TAP was to allow tests to be streamed too. So in theory a producer could output/emit events of test results. And a consumer would be able to parse/interpret the results. Upon a number like 1..2, it would know that the test set is done.
It gets more complicated with subtests, but that was part of TAP 13 I think, which wasn't formally released.
> The whole point is to allow language/harness agnostic tooling, but the only tool of note seems to be smoulder. Everything else seems to be trivial things like showing pass/fail with emoji tick/cross symbols, etc.
Interesting. In the Node.js ecosystem there are loads of different interfaces (https://github.com/substack/tape/#pretty-reporters), but my favorite is node-tap (which also comes with coverage tests and such).
1) The whole point is to allow language/harness agnostic tooling, but the only tool of note seems to be smoulder. Everything else seems to be trivial things like showing pass/fail with emoji tick/cross symbols, etc.
2) The TAP standard allows tests to be numbered, and a total count to be given up-front. In practice, all tools seem to require this, and barf when given unnumbered tests. This makes TAP far less useful, since it requires some global coordinating process to know how many tests we're going to run.
I was interested in TAP since I could write test scripts in any language which spit out a simple format to stdout, and they could be nested; e.g. we can have an overall "test everything" script which runs the test script for each of our projects/repos; projects might choose to have different scripts for unit tests, functional tests, integration tests, etc. and those tests might be spread across many files. Ideally we shouldn't have to care about any of that structure. With TAP we need some way to count the tests before running them (or else we have to wait for all of the tests to finish first, which is naff); this requires some sort of declarative structure, at which point our scripts can no longer be opaque black-boxes, so we might as well use a "proper" test harness like xUnit.