If you care about going deep under the surface I would think that the underlying optimization problems have many ties to DG. Not sure about the OPs pdf specifically
>In some sense DG is the foundation of all modern AI/ML/DL approach.
in what sense? i love when people say pretentious things like this to sound authoritative.
just because you take derivatives doesn't mean it's calculus. DG is not calculus - DG is calculus in spaces aren't globally flat but are locally flat. very different.
>The bar is just too high for normal people
in most schools that have DG classes they're junior level. certainly this class is a junior level class
Per Nash embedding https://en.m.wikipedia.org/wiki/Nash_embedding_theorem DG actually is multivariate calculus. However DG can provide an intrinsic point of view of curved spaces without explicit reference to such an embedding. It would indeed be interesting to see a coherent description of ML in a DG framework.
lol so backwards. i can put a fish in a fish tank in my living room it doesn't make it a mammal.
>It would indeed be interesting to see a coherent description of ML in a DG framework.
there is no need for such a thing - you don't need the machinery of connections, bundles, christoffel symbols, whatever else in order to take derivatives. you use those tools to be able to take derivatives in places where you can't do freshman calculus, not the other way around (bring those tools to places where you do do freshman calculus). it makes no sense.
it's like reasoning that because wiles used algebraic geometry to resolve fermat's last theorem that solving quadratic equations is really about algebraic geometry.
There are quite a few places differential geometry is very useful to know. Generally, if you want to know at a fundamental level "what does it mean to learn, what is inference truly?". You will find yourself in dire need of learning differential geometry. The easiest example is: the deeper your understanding of differential geometry, the more you'll be able to reason about hamiltonian monte carlo algorithms.
Information geometry also applies differential geometry, where you can think of learning as trajectories on a statistical manifold.
K-FAC, mirror descent and the natural gradient also derive from or are closely connected to work in information geometry. There's recent work connecting optimal transport. Optimal transport is an important idea and pops up in many surprising places, from GANs to programming language theory via way of modeling concurrency, for example (Kantorovich metric for bisimulation). Understanding differential geometry allows you to see and navigate such rich connections at a deep level. I heartily recommend it. A good place to start is: https://metacademy.org/roadmaps/rgrosse/dgml
This is beyond my expertise and involves some words that I don’t understand, but my understanding is that quite a few people in ML (eg Michael Jordan) specifically care about things like gradient flows in the space of probability measures, and have research questions involving quite challenging differential geometry. You’d have to browse his papers to get a better understanding as I’m not qualified to expand much on this subject.
