I am getting downvoted here without anybody actually addressing my arguments - TFX/Beam being replaced by TFX/KubeflowPipelines and the new platform being plugged in tightly to Google's managed services (AI platform, BigQuery, GCS, etc). I didn't think my tone was negative - they are observations that nobody in this subthread is addressing.
As a systems person, TFX to me was a layer on top of Beam to tie ML pipelines together. Now, Beam is gone - dataflow appears as a managed service you can use in your pipelines (see the figure on the blog post). TFX is being repackaged as the API for Kubeflow pipelines. That was my take and what i wanted to discuss.
I don't know the _eventual_ direction of Kubeflow/TFX -- but in our TFX pipelines you still get to choose where it runs. From the docs:
> Apache Beam is an open source, unified model for defining both batch and streaming data-parallel processing pipelines. TFX uses Apache Beam to implement data-parallel pipelines. The pipeline is then executed by one of Beam's supported distributed processing back-ends, which include Apache Flink, Apache Spark, Google Cloud Dataflow, and others.
