Huh? I really don't understand your argument. Jenkins is better than scripts because you can do more in Jenkins than in scripts? O_O

You can do a heck of a lot more in scripts; you've got a full language with all of the libraries it comes with in order to do stuff.

We've build our current build pipeline with Python scripts:

0. Bootstrap the build environment (installs everything you need to build). 1. Build projects order, order is derived from the intermodule dependency graph. 2. Execute unit test sets based on build parameter + save results in output. 3. Execute integration tests based on build parameter + save results in output. 4. Build distribution packages + save in output. 5. Deploy distribution packages to server based on parameters. 6. Run UI tests against the deployed packages + save results in output. 7. Publish the bits if they're OK.

Most of these steps are in different scripts, which can be called either from orchestration scripts or one-for-one in Jenkins.

All we have Jenkins do is invoke the script(s) then process and report the results.

We've basically got it wrong with the design of our CI tools; they do too much. They should just provide scheduling, invocation, reporting and dashboards over the build process. Expose that as an API and let scripts integrate with it.

So, let's say you have 8 projects. In each of the 8 projects, you want to run the same static analysis tool. If the violations go up, you want an email sent out. If any critical errors are there, you want the build failed.

In any project, if the build fails, you want the latest commit automatically reverted.

Now, do you just copy and paste to each script, or do you have a chain of build processes and logic in the build system?

I was basically answering "If you're putting lots of logic into ANY CI server you're doing it wrong!".

There is a separation of concern: the CI server logic is a refactoring of things that are common to any project. If your project have scripts that sends email on build failure, I think you're doing something wrong.

You mention at some point "orchestration scripts", this is exactly what Jenkins is supposed to provide help with: orchestration.

The important thing IMO is that the boundaries of responsibilities are clearly defined between what a project does and what the orchestration does.

There are good reasons to prefer free and open systems, even for "production" software. Those reasons are part of why people choose to use GNU/Linux, nginx, and all the rest of the open infrastructure.

It was not clear whether it was free as in beer or speech. I do understand there are benefits to using open software.

Too often I see engineers bias heavily toward inexpensive, free/cheap, or home-built solutions simply because they personally don't think things should cost money, when the understated anchoring costs of building and maintaining, plus the opportunity cost of valuable people time, might shift the toward other alternatives if a wider business context is taken into consideration.

Yeah, I parsed them as wanting a free system as in one that's open source, but I could be wrong.

If we're talking about hosted solutions, I agree it's mostly silly to refuse to pay... unless we're talking about open source projects, where many CI vendors do have a free tier...

Anyway, I also see an opposite bias toward costly hosted proprietary solutions and a refusal to even think about how one could accomplish one's goal with, say, a couple of shell scripts and cron jobs—or a self-hosted open source system.

That bias can be very sensible, but it can also be a hindrance, especially when it turns out that the proprietary solutions aren't quite what you want anyway, but since you're opposed to writing your own scripts, or running a more flexible system on your own servers, you settle for a suboptimal setup...

This bias I'm talking about is also a way of undervaluing something. It's saying "I'll pay $99/month for some hosted thing, but I'll never spend a week of engineer effort on really getting it right."

It's never the cost for me, it's the mutability, oos I can hack on, and make it do exactly what I want it to do, proprietary code I only ever seem to get somewhat close but never exactly there.