Oh! Can you give some pointers where to read about it more?

The wikipedia page for automatic differentiation and its references would be a good start. https://en.wikipedia.org/wiki/Automatic_differentiation

In particular, the section "Beyond forward and reverse accumulation" tells you how hard it is to deal with the graph in optimal ways, hence the popularity of simpler forwards and backwards traversals of the graph.

The original 2017 PyTorch paper is probably a good one.