But abstractions are a good thing! They let you separate a system into layers, so that things programmed at a higher layer don't need to know all the details of the lower layers. It's the way we separate interface from implementation.

Let me give an example from Bazel. Suppose you are compiling some C or C++, and you need to define a preprocessor symbol. There are two ways of doing this, "defines" and "local_defines":

    cc_library(
        name = "my_lib",
        srcs = ["my_lib.cc"],
        hdrs = ["my_lib.h"],
        # This will pass -DFOO when compiling this library
        # *or* any library that depends on it.
        defines = ["FOO"],
    )

    cc_library(
        name = "my_lib",
        srcs = ["my_lib.cc"],
        hdrs = ["my_lib.h"],
        # This will pass -DFOO when compiling this library
        # only.  Libraries that depend on us will *not* get
        # the symbol.
        local_defines = ["FOO"],
    )

I did not have to think about how defines will propagate to the libraries that depend on me, I just specified what I need and the build system handles the rest. I did not have to define my own CXXFLAGS variable and handle concatenating all of the right things together. The lower layer takes care of that.

What Bazel lets you do is create abstraction boundaries within the build system. People can write rules that define a self-contained set of build logic, then other people can use those abstractions without knowing all the details of how they are implemented.

Bazel is not perfect, and I have found myself banging my head against the wall many times. But overall the ability to define abstractions in a build system is, I think, one of the biggest things it gets right.

Disclosure: I work for Google, but not on Bazel.

They only fully support Javascript. The complex stuff, like defines, C++ toolchain, dynamic libs, etc.. is all out of scope.

That's very unfair to JS devs. We're well aware that other things exist...

...because we're trying to replace them all with JS.

As if all JS developers can be grouped into a single entity with the same thoughts. This sounds a bit like saying "the legions of Makita Tools carpenters think" or "the legions of Volvo truck drivers think".

Interfaces are VERY GOOD for migrations.

If you decide that you want to stop using some compiler flag, or maybe use a different compiler, or change your python version, or...

You can right a regex to go over all your shell invocations, change them, then test, or you can do something like:

``` def build_thing(name, srcs, hdrs, migration=False): if migration: ...

build_thing( name = "my_lib", srcs = ["my_lib.cc"], hdrs = ["my_lib.h"], ) ```

You can then manually flip that to test stuff, write a script to flip it for every team, sending out a PR (because your targets can have oncalls) and they can land it, and then at the end you can flip the default and manually add a False to the holdouts.

All this stuff gives you the ability to do hard stuff at scale.

This means that #ifdef FOO will be evaluate to true _inside the .cc and .h files listed in srcs_, when building this target or dependent targets, but might be something else when downstream targets' object files (.o) are built.

I'm incredibly skeptical of this.

I'm ex-meta and have worked a lot with the enterprise solutions you're talking about and the choice of starlark (originally python) as the build definition language is one of the killer features of the systems.

People want to create macros, target generators, etc. It's a common use case for a lot of engineers and IMO is a pretty killer feature.

Being able to say "This is an X86 Python Binary", "This is an M1 python binary" and then bundle those into "this is how you build either of those binaries based on inputs" without ever touching the internals or anything other than (more or less) a blob of python is why those tools scale organizationally.

It allows the teams that need to do weird stuff to unblock themselves without drowning the tools org. Sure, it has draw backs. Super deep macro layers are kinda a crime against humanity and debugging/evolving them can be quite expensive, but I think that's just the cost of software.

If that logic isn't in the build definitions it'll expand into a meta layer that generates configuration (I've seen giant "Translate this definition into 30 configs to run stuff" systems time and time again).

I may just be super biased from past mistakes and wins, but I think what you're doing is just moving the complexity out of your tool into neighboring tools and selling it as a win isn't really true, it's shuffling complexity around not removing it.

To synthesis my comment down (because it's easier once I've written it once, poorly):

The complexity you see in Bazel/Buck/Pants build files may seem like a result of their decision to use a programming language. That's a red herring. That complexity is fundamental to the problems people need to solve to build software. If you remove the ability to solve it in your build system the complexity will move to other systems.

Wrappers, CI, whatever. The complexity is just the problem space rather than a result of bad tooling.

Glad someone else brought up this topic and phrased it way better than I ever could have!

This other comment further down puts it very well, too: https://news.ycombinator.com/item?id=34892724

> If you remove the ability to solve it in your build system the complexity will move to other systems.

Adding to this: I think one of the main limitations of purely declarative (e.g. YAML-based) configurations is that a declarative DSL (as opposed to an imperative DSL) means you (only) get to use config options the inventors of the config language have thought of and implemented (i.e. "defined"). You cannot put into words things that require vocabulary that doesn't exist in your language.

Meanwhile, an imperative language gives you the ability to define on your own what you would like the machine to do.

As an example, consider Gitlab CI configs (YAML-based) or Ansible playbooks. In both cases, once you hit the boundaries of the DSL spec (which, in my experience, typically happens on day one) you can't do anything other than 1) give up or 2) leave the realms of the DSL and as you say,

> move [the complexity] to other systems

The other differentiation I think is worth on the following part, also which I'd liked a lot btw:

> move [the complexity] to other systems

which stemmed from:

> If you remove the ability to solve it in your build system the complexity will move to other systems.

I think it is even _more_ beneficial to consider that this does not start at the build system:

If you don't have and don't add the ability at development (make it a property of the project), the complexity will continue to _increase_ each time you move it into other systems after it (build, version control, CI, ... etc.).

To add to everyone else, please don't use YAML. Starlark is great _precisely_ because it is a readable, well known (nearly Python) language that is limited at the same time (no unbounded for loops, no way to do non-deterministic things like get the current time or `random()`).

[1]: https://aurora.apache.org/

You get an interpreter with access to 0 functions by default and can add stuff on top.

Maybe Python already has it, I'm not aware of it.

In the same vein as those questions how does caching work? Is it content based like Bazel or mtime like Nx et al? If there is no sandboxing does it do input tracking or is there manual "cache key" shenanigans?

If the configuration language is YAML how am I expected to implement new behavior? Is that in Rust? Is there a plugin architecture? Do I need to rebuild the tool itself and redistribute it to my build machines and developers? The main appeal of Starlark/Python in build systems is ability to create macros or in many cases define entirely new rulesets to support new tools and languages without needing to modify the complex and performance sensitive core.

Sorry for the skeptisicm but build systems are very complex beasts and new entrants like Nx don't measure up to tools like Bazel very well.

If you don't have correct caching of intermediate build artifacts, a system can't handle the compile and test requirements of large codebases.

I say this as someone who wants to love bazel... I just can't understand why it picks up impure toolchains from the system at all.

Combine that with the fact that C++ toolchains at least make assumptions all over the place about being spewed over `/usr`... I think they just bowed out to practicality at that point.

You're definitely right it should use hermetic toolchains. I'm curious which other Bazel-like build system does that by default?

Given its Google-internal predecessors, it’s not clear how priority should be established (the general guideline in science is that non-public is not science, but that might not be what we care about here). The public Bazel, though, is from 2015, while the first releases of Nix date back all the way to 2004[1] and the thesis is from 2006.

Now Nix is not really at its best if used as a general-purpose build system, but it certainly has and uses hermetic toolchains.

[1] https://releases.nixos.org/nix/nix-0.5/manual/manual.html

I don't think it was easy, though.

I wouldn't say moon is hermetic, nor are we trying to be. We don't use the sandbox approach for tasks, and run against the original files. For the languages we support, this works best.

As for deterministic, we try to be. We have a "toolchain" where we download languages/tools in the background, and run tasks using these tools. This ensures, at minimum, that the same version/variant of a tool is used across machines.

> In the same vein as those questions how does caching work?

It's content based. We also hash other kinds of inputs depending on the language that is running.

> If the configuration language is YAML how am I expected to implement new behavior? (and other questions)

Our focus right now is on non-compiled languages, primarily web languages, where custom behavior (like Starlark) is not necessary. In the future, this may change.

Also a bit disingenuous to describe it as language agnostic if it only really works with non-compiled languages.

It's also language agnostic because you can technically run anything through moon, but support may vary. https://moonrepo.dev/docs/faq#can-we-run-other-languages

But you can only reliably know what is affected through hermeticity.

> It's also language agnostic because you can technically run anything through moon

That's just calling out to another build system though. I don't think it really counts.

[1] https://en.wikipedia.org/wiki/Pile_driver

Most likely you did this because you felt all the other ones are too complicated.

But the reason the “enterprise” ones are so complicated is to serve their enterprise customers, who need “just this one feature” so they can use it. But those customers pay the bills.

So basically you have to choose complexity and profit or simplicity and less (or no) profit.

Good luck! But make sure you’re ready to make that choice.

Almost none of the complexity comes from what configuration options I've registered ahead of time. It comes almost entirely from, "Well darn, this code depends on this completely unrelated part of the project. I wish it didn't, but now the build tool either needs to sometimes fail to rebuild something correctly, or it needs to build way too much to run quickly."

This runs counter to how most people think of things like Bazel which are tools you should only reach for when the situation has already grown out of control.

I'm a tech lead for the web monorepo at Uber. We talked to the Turborepo guy a few years ago, and he admitted that he wasn't sure if it could handle our scale in terms of all the bells and whistles that a repo of our size leverages - and his is one of the more feature packed commercial offerings in this space.

As a random example: we see thousands of commits a week, so invalidating the whole build graph when the lockfile changes is a complete non-starter, yet most turn-key solutions target small teams and are not equipped to cope with this problem. Phantom dependencies[0] are a problem with disjointed build graphs. Etc.

As someone who's evaluated a ton of these systems, I would love to see where your experience in this space is coming from. A new kid in the block will have a lot to prove.

[0] https://rushjs.io/pages/advanced/phantom_deps/

In regards to build graph invalidation, we attempt to hash pieces at the granular level. This includes per file content hashing, and for dependencies (those in `package.json`), we parse the lockfile and extract the resolved version/integrity hashes. We can probably improve this further, but this has been working great so far.

As for phantom dependencies, this isn't something moon solves directly. We encourage users to use pnpm or yarn 3, where these phantom problems are less likely. We don't believe moon should solve this, and instead, the package managers should.

If you have any other questions or concerns, would love to hear them.

My IDE solves this in that I see red underlines, but I also want my build to fail as well. Not everyone uses an IDE that would show this error.

If I have package A only list its own sources as inputs, how do you prevent the Node runtime from happily doing require('../B')? If you don't, how do you prevent differences in the state of B from poisoning the remote cache?

We do not invalidate the whole graph anymore for a lockfile change. We now have a sophisticated parser that can calculate if a change within the lockfile should actually alter the hash of a given target. In addition to higher cache hit rates, this is what powers our `turbo prune` command which allows teams to create slices of their monorepo for a target and its dependencies…useful for those building in Docker.

Prune docs: https://turbo.build/repo/docs/reference/command-line-referen...

Turborepo is much more scalable now than when we spoke pre-Vercel acquisition. It now powers the core web codebases at Netflix, Snap, Disney Streaming, Hearst, Plex and thousands of other high-performance teams. You can see a full list of users here: https://turbo.build/showcase

Would be happy to reconnect about Uber’s web monorepo sometime.

I build my packages regularly (not pruned), then I prune with scope "backend". Apparently the pruned directory contains node_modules with empty packages, not sure what reason is for that; I just ignored it. In the resulting directory I then run `pnpm install --prod`. Only the regular dependencies will be installed. I think this is enough for my usecase. I am not sure if prune is supposed to be used for this approach though.

If I don't need build caching I'm not using any tool but PNPM and its workspace toolset - that's literally all most people need for a monorepo. I've looked into Turborepo, and its simplicity versus Nx is a strength. However, it's not the taskrunner that I want.

I now work in a monorepo where build caching is required, so I'm excited about moon and keeping a watchful eye on the project's progress. From my evaluation so far, it fixes all of my gripes about Nx and I'm keen on it not trying to do too much, while allowing me to make it as flexible and extensible as I need. Extending configs is chefs kiss

NX has been on my "to learn" list for a long time and high enough that I have given it a few stabs with personal project but something have failed every single time. I chalked it down to my setup or my inexperience but maybe it just isn't as good as I thought.

Could you elaborate a bit on what you mean here? Is this saying Turborepo's task runner is flawed in some way?

You lost me there. Buck/Blaze aren't perfect but configuration was never an issue.

Moon like it is currently is a glorified task runner.

How easy will it be for layers of abstractions to emerge in this system? That’s the ultimate limitation of markup-language-based / configuration-based build systems.

Look up the difference between external DSLs and internal DSLs. The YAML config file is an external DSL. I prefer to express configuration using internal DSLs.

Aside from the classic `country_code: NO`, the other day I ran into issues with scientific notations. Now, guess, which of the following are strings, and which are numbers:

- 1e+10

- 1.e+10

- 1.0e10

As I understand it, the primary reason these build systems leverage these Python-variants is so that the build rules, toolchains, constraints, and build definitions can all be written in the same language (since build rules often require some programmatic behavior). Perhaps with a future vision of them being totally interoperable across build systems.

I'm generally happy with Blaze/Bazel, so I'm not necessarily in the target market for Moonrepo, I guess.

EDIT: This isn't really competing with Blaze/Bazel either when I look at the execution model. It goes back to imperatively defined tasks instead of declaratively defined dependencies, which feels more spirtually aligned with Make than Blaze/Bazel.

Clearly, you've never used Gradle.

Have you actually talked to Blaze/Bazel users to understand what frustrations (if any) they have with their current build system? Have these users asked for a Bazel-lite? If so, and you still want to position yourself as Bazel-lite, then you should include some of their direct feedback and write your messaging accordingly.

As a daily Blaze/Bazel user, I don't have a desire for a Bazel-lite. I've worked at a midsize company that used Bazel, and I'm working at the company that created Blaze/Bazel.

Disclaimer: Opinions expressed are my own; not representative of my employer.

- Distributed cache of built artifacts. Work with my company's network topology and security requirements. Make this easy to setup, operate, and seamless for developers to opt into

- Seamless monorepo support (multiple languages)

That could be considered Bazel-lite. And yes, I'd take it over Bazel currently.

If you don't work at Google, convincing your engineering organization into learning Bazel is almost always a non-starter. Who uses Bazel in the wild? Xooglers primarily.

Their value proposition is sound.

[1]: https://www.youtube.com/watch?v=t_3bckhV_YI

[2]: https://www.youtube.com/watch?v=H67uuwVO1tc

[3]: https://www.databricks.com/blog/2019/02/27/speedy-scala-buil...

[4]: https://stripe.com/blog/fast-secure-builds-choose-two

[5]: https://www.uber.com/en-SE/blog/go-monorepo-bazel/

Large companies, with established platform / ops teams, who can support Bazel and push for its adoption within an engineering organization definitely exist. There's probably a few Xooglers working there who'd like to see it happen too.

My response shows these hypotheses to be false. You’re correct, no one’s arguing whether Bazel is useful or worthwhile, not even I.

> Large companies […] who can support Basel within an engineering organization definitely exist.

Ok great, we agree. You’ve revised your point from earlier.

> There’s probably a few Xooglers […]

A baseless hypothesis that’s confirmed false by all the references shared with you. Feel free to plug all the lead engineers’ names into LinkedIn to verify that the majority are not ex-Google.

> Bazel-lite may be enough.

Sure, it may be. That’s why I’m offering the constructive feedback to go talk to the thousands of active Bazel users to see whether and what they want from a Bazel-lite, if that’s the positioning Moonrepo wants to choose.

Usually, there’s a strong rationale and need behind an organization’s adoption of Bazel. Some of that rationale is captured in the references shared earlier. These organizations need Bazel; they’re generally happier with Bazel than their previous systems. These baseline needs represent table stakes for any build system that wants to compete with Bazel.

If Moonrepo wants to compete where Bazel is weak, then I’m suggesting that they need to sharpen their communication such that an engineer well-versed in Bazel has an “aha!” moment within 2 minutes. The proverbial elevator pitch.

Once again, I’m not dissuading Moonrepo from pursuing their vision. I’m offering constructive feedback on their messaging/positioning — the type of feedback I’d like to hear as a founder.

Even in my original reply:

> If you don't work at Google, convincing your engineering organization into learning Bazel is almost always a non-starter. Who uses Bazel in the wild? Xooglers primarily.

Emphasis on almost since you've missed it three replies now.

What are you even getting at here? I freely admit that some companies adopt Bazel. Many do not. You doing a Bing search to find the ones that do doesn't change that fact.

> Ok great, we agree. You’ve revised your point from earlier.

I haven't revised anything. You misread, but feel free to be needlessly hostile defending a product your current employer makes against a newly launched competitor. I use the word competitor loosely. It's not a good look either way.

> That’s why I’m offering the constructive feedback to go talk to the thousands of active Bazel users to see whether and what they want from a Bazel-lite, if that’s the positioning Moonrepo wants to choose.

Why would they need to talk to people who are actively using Bazel to see if they want Bazel-lite? It's a small population that's already, presumably, well served.

There's a whole other, much, much larger segment of customers who'd love to use Bazel-lite that aren't using Bazel, for many reasons, including the high cost of adopting and supporting it. I'm one of those customers. Unsure why this point escapes you.

Your advice, by the way comes off as attacking their idea and trying to tear it down regardless of how many times you preamble it's constructive criticism.

Bazel doesn't work well for every language, but for those languages that it does, it definitely makes sense to use Bazel. For those languages that work better with something more lightweight, that's where moon comes in. I'm assuming you work at Google, so you're experience around Bazel is probably much better than those that don't work at Google.

I appreciate all the feedback and comments though, very much appreciated.

Better “monorepo-ish” (whatever that means in the frontend world) tools are still valuable, but as GP said, it's pretty crowded here.

I don't know if you have talked with people working on C++ or Java projects (I see that these are not among your supported languages), but if you do, you get to see why people are talking about Bazel.

The second reason is we wanted something language agnostic and not proprietary. It also helps that many other tools, like GitHub actions, are written in YAML.

And lastly, we wanted a format that developers are familiar with, and won't need to spend time learning. Requiring developers to understand a special syntax simply to define a task to run is something we want to avoid.

However, I think for describing actual build toolchains this way of implementing things might end up being significantly more complicated (and require even more arcane end-user cognitive load) than the Starlark/Python-based build rule / toolchain / constraint approach for actually assembling libraries, binaries, and other compiled artifacts. There are a combinatorial number of backends, conditional options settings, etc. which will be hard to capture with a purely declarative system. For instance, a C++ binary might needs platform-specific compiler flags, or some #ifdef nonsense. YAML doesn't have a clean way of implementing conditionals based on some constraint. So for heterogenous ecosystems (C/C++, Python, container assembly, GPU development, microcontrollers, etc.) this pushes a ton of complexity into the build rules themselves which may be opaque to end users (and thus introduce a ton of cognitive load, steep learning curves, hard-to-debug errors, etc.).

Once we starts supporting more languages, especially compile based ones (probably Rust first), our decision around YAML will probably change. Off the top of my head, a non-YAML format would probably be used as a secondary format, kind of like how a Dockerfile works.

I can see the benefits of "generating code" within the BUILD file, but honestly, we haven't required it yet, and have been able to do most things with explicit configuration. Our token syntax helps with some dynamic aspects of this.

Maybe in the future we'll revisit this.

It's really 'do you make config an artifact, or do you make it code' sort of argument.

You will end up developing your own yaml-based DSL that is incompatible with everything else, has weird limitations and poorly documented constraints... just like any other YAML-based system.

And by the time you'll have written a 1000-line YAML (where half of the file is nothing more than awkwardly quoted and escaped bash scripts) and realise that YAML cannot be split into reusable files, it will already be too late.

Oh. I just saw that you already use a custom non-standard extension of YAML with `extends` so there you are

- https://www.buildbuddy.io/

- https://github.com/buildbuddy-io/buildbuddy

I've primarily worked in typescript codebases and have used raw yarn workspaces, lerna, nx and recently evaluated moon and turbo.

The funky part is I eventually simply went with nx, not just because I've used it before, but also because I felt like the configuration is just simpler and more lightweight. Esp. since you can pretty much roll with it without defining much more than single simple config file - while moon required some 2-3 separate config files, plus config files in each project (I understand the per-project config is not required, I don't remember why I needed it - something to do with my build tasks)..

In any case I intend to give it another shot soon.

As for the whole config in yaml vs json vs toml or whatever, not a deal breaker since most of the time these sorts of configs are perhaps not something you end up interacting with programmatically - I know a lot of people enjoy editing yaml more than json files

Besides, yes, nx comes with a single configuration file for each project, but alongside of it are jest, eslint, babel and app/lib/spec/ide tsconfig configs - that’s a lot!

All of this shouldn’t be visible to the developer. Initially, when trying to find myself in this mess, I thought that the solution lies in autogenerated ide workspaces - for vscode and sublime. But in practice it wasn’t that helpful, because there is always something which is not handled by autogenerate multiroot structure but is needed, so one needs to have multiple windows open anyways.

Hopefully typescript 5.0 is going to help reduce some of this boilerplate with multiple tsconfig base classes, so lib/app/spec/ide tsconfigs will be able to extend from common base, which currently is not possible.

The worst part of Nx is that there is lack of ssr support, so I had to patch nx with patch-package to generate obfuscated css classes in prod, because currently it’s not possible with their webpack executor! Recently they tried to simplify it a little bit, reducing this webpack boilerplate, which introduced few bugs but it's better than not doing anything!

However, saying all of that and as someone mentioned in previous comments, dx is not primarily improved by lack of config boilerplate, but by having semantic structure of the actual code, clearly knowing what depends on what and where one can find it and put it. Right now we are on our own, because there is lack of information on how to structure cross platform codebase properly. I strongly believe it will change though. Best of luck to the people at moon!

Ps. Nx has a nice blog post why they do t use bazel under the hood https://blog.nrwl.io/on-bazel-support-6be3b3ceba29

Could you explain why any existing project using Turborepo/Nx should switch to Moonrepo? What are the advantages and disadvantages? The support for multiple languages seems like a big advantage.

I'll start with Turborepo. Turbo is primarily a task runner for `package.json` scripts with some caching... and that's basically it. If that's all you need, then great, but if you're looking for more functionality, that's where moon comes in. moon is more than just a task runner, we're aiming to be a repository management tool as a whole. This includes project/code ownership, direct CI support, future CD support, code generation, hooks management, constraints, release workflows, and much more. With that being said, we do have a comparison article against Turbo: https://moonrepo.dev/docs/comparison#turborepo

As for Nx, they're more of a competitor than Turborepo. Nx and moon are aiming to solve the same problems, but go about it in different ways. Nx is Node.js based and requires heavy adoption of their ecosystem (@nrwl packages) and their executors pattern. In the long run, this becomes a heavy source of tech debt, as your dependencies are now tightly coupled to their packages and release timelines. With moon, we wanted to avoid this all together. There are no coupled dependencies, and tasks are ran as if you ran them yourself on the command line. No abstraction layer necessary. We also want to embrace a language's ecosystem as much as possible, so moon adoption should be rather simple and transparent (at most each project has a moon.yml file).

But to your last point, we agree, multi-language support is a massive advantage. Having both backend and frontend code in the same repository, powered by the same build system, is a massive win in maintenance costs and developer time saved.

> release workflows

Looking forward for this, especially if that also means auto-publishing of NPM packages, Rust crates, etc.

To deploy the services locally, we use Tilt[2] (K8s for local). We want to be able to reproduce production as much as possible and remove developer overhead on how things work locally and in production.

Then come the issues with Docker and large node code bases:

1 challenge with large monorepos is the huge node_modules folder (rush among other tools put packages into a single large node_modules folder, and symlink every dependency there. It can contain millions of files and GBs, depending on how many 3rd-party libraries you use). On Linux, you can mount it without issues, but Mac has performance issues[3] with large folders.

We pre-build that huge node_modules into a "base" image, and each service in the monorepo pulls that base image and only mounts what's necessary (a few mb). So we can save that npm build time and don't need to copy all those files inside the containers. It is fine because package.json does not change that often. You need to do this pre-build locally then in your CI/CD -> dockerhub, so everyone can get it.

Another challenge is that you need to "watch" files to rebuild them. Watching all your files inside the monorepo isn't really viable. We use a dependency graph to know what services to watch and then copy the built files inside the Tilt containers.

Hope this can help people.

[1] https://rushjs.io

[2] https://tilt.dev/

[3] https://github.com/docker/roadmap/issues/7

Does that mean most of the automated features you mentioned here only apply to JS?

This makes me sad. A product designed to be developer friendly and automate things should create a parser and custom config language, or at least borrow one. The whole reason there are so many is they are built for purpose to make a specific task easier. If you need features like templating and logic, you can slap in existing solutions like Jinja2 and Go templates. YAML is not a configuration format, it's a data serialization format. If you don't want to write a parser, use TOML.

My final note is: don't reinvent the wheel. There is a fully open source alternative to Drone.io (I can't remember the name) that forked a while ago. Drone is the best CI system for developers, hands down. Build on the open source codebase, build some proprietary extensions and a new UI, sell your product there. It's proven (because Harness now owns it) and you can help build the existing open source product at the same time.

Otherwise, I have no need for yet another build system. All build systems are pretty much the same to me (except Jenkins, which is pretty much the tenth circle of Hell) and I'd much rather use a fully open source one that I could pay for support + extensions for at work. Maybe you could make a better Jenkins?

What you will find is the vast majority of your configurations will invoke a make.sh script that does everything that you want to support in your system.

Feel like any data definition language eventually bolts on a shitty form of templates.

It's incredibly frustrating to have to configure a build with something as shitty as YAML. There's a tangible amount of money I have wasted in organizations on it.

A build is not actually a static configuration of another system. It's a program and deserves everything we need from programming languages.

That said, Starlark is way closer to a real language (Python) than YAML.

The reason you might want a Turing-incomplete language is that you can prove the build terminates. However that's a lie, since you will always need to escape to running programs outside the language!

The thing that annoys me most about CircleCI, TravisCI, Github Actions and Appveyor is that there is no simple way to run the same thing locally to debug or test workflows without creating either git history mess or temporary hacks like taking off branch restrictions in the yaml.

This seems pretty cool. I particularly like how 'gradual' it seems to be relative to things like Bazel, i.e. you can take some shell scripts and migrate things over. I did have a play and hit an initial problem around project caching I think, which I raised at [0].

One comment, from the paranoid point of view of someone who has built distributed caching build systems before is that your caching is very pessimistic! I understand why you hash outputs by default (as well as inputs), but I think that will massively reduce hit rate a lot of the time when it may not be necessary? I raised [1].

Edit: for any future readers, I spotted an additional issue around the cache not being pessimistic enough [3]

As an aside, I do wish build systems moved beyond the 'file-based' approach to inputs/outputs to something more abstract/extensible. For example, when creating docker images I'd prefer to define an extension that informs the build system of the docker image hash, rather than create marker files on disk (the same is true of initiating rebuilds on environment variable change, which I see moon has some limited support for). It just feels like language agnostic build systems saw the file-based nature of Make and said 'good enough for us' (honorable mention to Shake, which is an exception [2]).

[0] https://github.com/moonrepo/moon/issues/637

[1] https://github.com/moonrepo/moon/issues/638

[2] https://shakebuild.com/why#expresses-many-types-of-build-rul...

[3] https://github.com/moonrepo/moon/issues/640

Just is a task runner (with full shell integration) that calls our turborepo tasks.

We define all of our tasks in one justfile (things like repo setup, syncing env vars, and compiling dependencies) and then link them to turbo processes which cache the result.

Massively reduced our cognitive load working with our monorepo, and is lightning fast.

If we ever want to change it will be simple to remove both, so we're not tied to the ecosystem.

[1]https://github.com/casey/just

Most of that repository is a series of modules in their own directories (e.g., "src/modules/character").

The second repository is a Node.js API. The majority of its functionality comes from one of the modules from the first repository that I copy in its entirety to the other repository whenever I change it.

- Use git submodules. Have the node repo have a submodule on the app repo.

- Publish the shared code to a private registry (like github's package registry), and pull it in as an npm package.

I'm not going to click "learn more" because you've given me absolutely no reason to. The page may as well just be "log in with github" and nothing else. That'd serve the same functionality, and have an equally convincing pitch to prospective customers.

For the languages we currently support, this is more than enough. Once we dive deeper into compiled languages (probably starting with Rust), we'll look into more granular reactivity and possible use something like sccache.

E.g. with Turborepo the cache server spec is somewhat known and there are plenty of various implementations in the wild.

also: are you looking for any other founders?

It's a fair bit of effort to change one's already established code repository and build system, and I don't really understand from your pitch what you are offering that is sufficiently advantageous for a decision maker to choose to switch.

For example, in CI, tasks are only ran if they are affected by files changed in the pull request. No more running everything unnecessarily. We also support remote caching of artifacts, which helps to speed up CI even further, by avoiding long build times.

This seems to be a cyclical thing with build tools. Things are great until you expect customers to pay.

(I also worked for Google a while back, and they were very conscious about not using Google as a verb internally, losing a slow battle against the tide of trademark dilution.)