This is partially fixed in Julia 0.5 (or available now as BaseTestNext.jl in Julia 0.4). Its basically a backwards-compatible rewrite of the former Base.Test that adds the notion of test sets - enabling counting tests by default, but also not immediately failing out on the first bad test and even logging.