Most (but not all) overlay networks are implemented in kernel. If you compromise one node in a cluster, you can fairly trivially snoop traffic, bias other nodes to send traffic through you, or listen via various mechanisms such that you can intercept traffic flowing between workloads not actually located on the compromised node.

So always encrypt everything unless you’re in a very rare environment with central network control that cannot be compromised or intercepted from a given machine.

This would be less of a concern if the cluster's pods were Firecrackers, yes?

AWS EKS on Fargate has a dedicated ENI and kernel per pod; the only way to intercept the traffic is when it crosses a network, or with flow control logs. Or if somebody hacked the control plane, but that's always "Game over man, game over!"

And if you've been in that kind of rare environment, those people encrypt everything. They'd encrypt their license plate if they could. You want paranoid, look up laser microphones.

Although with many clusters, compromising a single node is likely to lead to cluster compromise as it allows for all the service account tokens assigned to workloads running on the compromised node to be used by the attacker :)