Game development has really intense math and engineering stuff, and it's extremely fun and addicting for some people. If you can get yourself into that world, it'll set you on a track to learning what you're asking for here.
Of course, it's not for everyone, but I'd say it's worth giving it a shot. Definitely start with C/C++ if you haven't already, and try not to stay with existing game engines like Unreal or Unity for too long (unless you're specifically trying to start a career in game development; a bad idea)
FWIW, I think Unity/Unreal can provide that feedback loop to really enjoy building games. There's still plenty of opportunity for mathematics at the scripting level, and the advantage is you won't spend days or weeks trying to draw a square in OpenGL. I took that path and vastly prefer the engine approach.
Of course, it's not for everyone, but I'd say it's worth giving it a shot. Definitely start with C/C++ if you haven't already, and try not to stay with existing game engines like Unreal or Unity for too long (unless you're specifically trying to start a career in game development; a bad idea)