A bit of algebra is good to know but more math knowledge is really only required if you want to work in a domain which requires it like Machine Learning, graphics engines, Scientific Computing, algorithm development and some other domains.

So I would explore and see how far you can get in math before investing to heavily into a specific language. If you want to learn one that’s useful in many domains and is easy to learn while focusing your studies on math my suggestion is Python.