But anyone interested in a next generation scalable build tool, check out Google Bazel. https://bazel.io/ (Ex-Googlers built similar build tools elsewhere -- Buck and Pants -- but I'm most optimistic about Bazel.)

Like Make, Bazel uses incremental tasks at its core. But where Make uses timestamps, Make hashes inputs, allowing for shared caches. It's also been optimized to work with Google-scale build definitions. In beta now, 1.0 planned for end of 2017.

My experience with Google software is that its usually not as good as Google employees think it is. See Go, Dart, GWT, Android, Google Groups, Angular. I think this can be partially contributed to the sect-like structure of the company where everything created at Google is the best thing ever, while everything invented in the rest of the world is discarded or ignored. See this hilarious reply from Googlers to a feature that many people take for granted, but Googlers are completely unaware of: https://github.com/bazelbuild/bazel/issues/1394

Bazel is yet another thing where Googlers think they have invented the best thing ever, and then have to realize that the world around them, which they tried to ignore so hard, has moved on since 1970, making their "really great, special tool" neither great nor special.

There is an additional distinction between "top down" and "bottom up" incremental builds as mentioned in this article about tup.

NEITHER Make or Bazel do bottom up builds. They are top down. That means that incremental builds requires O(n) stat() or hashes. You have to check if ANY input is out of date.

In contrast, tup does O(1) work for incremental builds by using inotify.

In practice, I'm not sure how much this matters, but it is interesting -- and it is the central point of the linked article. Ninja has a reputation for being fast, and it also does top-down builds. As I recall the Linux kernel is quite good at caching inodes for stat() calls, so you can basically stat millions of files in under a second. (Windows may be a different story.)

But if you are using hashes rather than timestamps, the work to check if any of a million files are out of date is much larger.

I actually switched from Windows to Linux because of Tup—although to their credit, they have fixed the issues that I filed.

I still use Tup on Windows for production at work, and in place of `tup monitor` I use a service based on .NET's file system watcher, and just tail monitor the log.[0] The config looks like this:

    <job at="%HOME%\some\project">
      <watch>
        <path>script/*.js</path>
        <path>stylus/*</path>
        <path>src/**</path>
        <path>tools/**.cs</path>
      </watch>
      <do>tup --no-environ-check</do>
    </job>

[0] https://bitbucket.org/gavinpc/reactor/

Best I can find. I was referring to kqueue and FSEvent though for free bad and macOS.

Sorry traveling and phone only.

I wonder if the Linux on Windows stuffs will make it easy to use inotify on Windows? I've only heard of it. Haven't researched how that works.

TTBOMK, the only two things it doesn't do well are processes with multiple outputs (e.g., yacc has both .c and .h as output), and distinct output directories. Both can be worked around, but ... it's clunky. Other than that, it's ridiculously simple, extremely robust and generally awesome. Way better than Make at properly expressing dependencies.

But of course, gittup takes the prize - by virtue of watching the build process from inside, it doesn't need you to tell it about dependencies at all. It knows better than you do, and can thus do the fastest possible perfectly incremental builds.

I think you mean "Bazel" hashes inputs.

Anyway, you don't mention Tup... have you used it? It might not be Google-scale, but it can build Mozilla central and a Linux distro. I use it at work and for personal projects. All I can say about it is that there's no going back. Maybe Bazel is that way, too. I'll check it out.

Tup also does output-based short-circuiting, which I assume is based on hashes. In other words you can say

    : some_input |> ^o^ process %f %o |> some_output

I will say this: it's an adjustment, if you're used to mutating files during a build. (I'm looking at you, pdflatex.) But it's sooo worth it.

Every time I've encountered Bazel thus far in the wild I've found it an atrocious experience.

I haven't yet found anything I prefer to simple Make, and even auto tools while awful, are at least completely a known quantity.

And at least the internal version makes you specify your dependencies twice for C++, rather than using something like gcc -M. (And I believe that is true for Java too.)

I'm not sure if SBT supports an incremental tasks execution, but it does support incremental compilation of Scala which can be more than enough for many devs.

But it was still listing the up-to-date targets. Incremental builds in Tup don't even list what's not being done, because there are no "targets" in Tup. That's the whole point of Mike Shal's paper.

Has anyone attempted that approach? simply put a JSON wrapper around make?

XML is definitely not better than JSON IMO. Are you hinting at either S-expression, or something like YAML?

(For any other "custom syntax" we're better off sticking to make syntax itself)

However, in my opinion, from your list only s-expressions make a reasonable syntax that a human would write an maintain.

And, if you want that, well :) https://www.gnu.org/software/make/manual/html_node/Guile-Int...

I think if you want a simple serialization format to replace Make's syntax, s-expressions would be a better fit.

Regarding JSON, it is more verbose than Make. You need to use quote around strings, commands, everything. Compare:

foo.o: foo.c gcc -c foo.c

with

{ 'foo.o' => ['foo.c'; 'gcc -c foo.c'] }

It was originally written for Chrome, but has been adopted as an output format for CMake and is used for many other projects as well.