I need to deal with system with complex dependency relationship. The system need to stop executing when some step fails. I used to look at Airflow. I remember at the time it was the first result if you Google DAG something. But it has the same problem as data-engineer centric system that it was somewhat over-engineered and it relies too much on a center server. It’s heavily relying on a Python runtime is also something I don’t like, feels like everything else is a second-class citizen. The nature of our job is that we have deal with complex legacy codes, some Fortran program, some need a conda environment etc. Later I found what I should look at is just some makefile-like system, and I settled on Snakemake, which has a nice DSL that forces you to be explicit about input / output / etc.. Probably Airflow is just not the right tool for a one man team.