My title is still software engineer, but I effectively do data engineering, and I work closely with data scientists.
I love a lot of it, but there's still plenty of bullshit to deal with. Just in the technical side, dealing with Python is a perpetual gong show, and most of my team's work seems to revolve around configuration of secrets and K8s.
I'm fortunate to be the guy that nerds out about performant code, so when something inevitably turns out to be a perf bottleneck, I can turn back into a regular old software engineer who trades in big data. Which I think is a better title/charge than data engineer, anyway.
I've talked with plenty of ML engineers, and they seem to immensely enjoy what they do. It seems that the periphery of data engineering is great; the core of it, not so much.
I think "The Gong Show" was an old tv show about amateur talents. Sometimes good, most of the time terrible and hilariously unaware. Not sure if that was what was intended here.
So is the GP criticizing Python? If yes, I am curious to know why. No, I am not here to defend Python. The constant runtime exceptions due to typing mistakes is so tiring.
I love a lot of it, but there's still plenty of bullshit to deal with. Just in the technical side, dealing with Python is a perpetual gong show, and most of my team's work seems to revolve around configuration of secrets and K8s.
I'm fortunate to be the guy that nerds out about performant code, so when something inevitably turns out to be a perf bottleneck, I can turn back into a regular old software engineer who trades in big data. Which I think is a better title/charge than data engineer, anyway.
I've talked with plenty of ML engineers, and they seem to immensely enjoy what they do. It seems that the periphery of data engineering is great; the core of it, not so much.