Could you recommend one or two of these short stories? Thank you!

Pick up Axiomatic, try the first story (The Infinite Assassin) and see if it grabs you. It's got the best parts of something like Snow Crash, which drops you in without much exposition, and lets the visuals/action lead. Learning to Be Me from the same collection really stood out to me, it's a unique take on the "uploading your consciousness" trope.

My recollection from Infinite Assassin is that for someone not pretty literate in mathematics it would nearly be nonsense.

It's not a short story, but this is the first work of Egan's that I read and I loved it: https://en.wikipedia.org/wiki/Diaspora_(novel)

Into Darkness was another great one from Axiomatic. Also great action, and a seriously intense brooding atmosphere.