- Have crystal clear Mathematical foundations, as in why this formula/method the way it is, rather than being able to solve college/HS test problems. Really solid footing in Differential Calculus and Linear Algebra is necessary.
- Know the Statistical language that you learn from a basic college-level Stat 101 course. Be able to translate normal sentences into those using Statistical notation, and be able to read easily. Also, know basic Statistics.
- You already know programming, I assume. Learn Python if you don't know already. It's really easy.
- There are a number of paths you can go from there. Here's what I did.
-- IBM Data Science Professional Certificate (not deep at all, but lays out the landscape well; did it in a week)
-- Machine Learning for Absolute Beginners by Oliver Theobald which you can finish in an evening.
-- Machine Learning Specialization by Andrew Ng on Coursera.
-- Deep Learning Specialization by Andrew Ng on Coursera.
-- fast.ai course.
- Learn PyTorch really well. I suggest Sebastian Raschka's book.
Now from here, you can chart your own path. You can choose NLProc, Vision, RL, or something else.
I went towards Vision. And I do Edge AI as hobby.
I was in the last year of college as a Physics undergrad, when I was hired to do Vision modelling/research for a non-flashy company in 2021. Finishing my CS Master's next month and starting to look for PhD. I worked in the same company for the ~2.5 years.
EDIT: If you want a job in big tech, grind Leetcode, and learn about system design, study Machine Learning systems, and be able to design them. Chip Huyen has a good book as I hear. 6-7 rounds of interview is common in Meta/Google. DL hackathon awards, open source contributions are significantly helpful.
Highjacking for self-edification as you led with math...
Is math going to become more important for software engineers in general, as AI is adopted in to more and more aspects of the SDLC, even if said engineers aren't working directly on the AI systems themselves?
I ask because I'm quite math averse (dyscalulia). I am self-taught engineer, largely because it became glaringly obvious I wasn't going to be able to complete a CS program, so I pivoted my educational focus.
I still manage a relatively successful and effective career in software. I love the engineering aspects, e.g. programming, system design, problem solving, pattern recognition, etc.
When it comes down to it, I couldn't calculate my way out of a wet paper sack. Are my days (pun intended) numbered?
I've been an ML/data engineer for the majority of my nearly decade-long career. You don't need hardcore math unless you're building models themselves, and even then it's pretty variable.
There's a LOT that happens around just building models, and posts like the top post ignore that, and it's more akin to software engineering than anything else.
Only in the sense that ultimately everyone's days are numbered. LLMs can suggest bits of code but there's no reason to believe that it can undertake large-scale, novel software engineering projects. In fact the lack of an underlying mental model makes me doubt that the LLM paradigm will ever get to that point.
For the forseeable future I think there will always be a role for the person who can turn around and say "I see what you're getting at, but it just doesn't fit conceptually with the system. What are you actually trying to achieve? What if we did it this way instead?".
Copilot can absolutely do the jobs of Data Scientists who got hired based on some minimal skills that were considered valuable even three years back.
I have tried applying Copilot to my own work where it spews- garbage. But I use Copilot a lot to generate some example data, or write repetitive code- I needed a basic Flask frontend and I used Copilot to do 70% of the job. But it cannot yet solve Vision tasks.
> " I think there will always be a role for the person who can turn around and say"
These roles, if and when they start to exist, will exist in extremely tiny numbers. And they will all likely be hired from Stanford/Oxbridge/MIT/Caltech, and they will have PhDs.
If this scenario plays out, then, I am afraid for my future, too.
>Is math going to become more important for software engineers in general, as AI is adopted in to more and more aspects of the SDLC, even if said engineers aren't working directly on the AI systems themselves?
I'm definitely not any authority on the matter -- also self taught, but I don't think so.
Having a conceptual understanding of the math is probably a really good idea, but I imagine there's quite a bit that can be hidden behind an api that you can essentially say "does xyz by means of magic".
I imagine this winds up creating three distinct roles:
- Focusing on improving the model's usefulness
- Focusing on implementing the model for optimal performance
- Focusing on using the model within a larger application
I say "creating", but I imagine this is also probably roughly the current status quo.
The maths are not actually that complicated. 99% of what pytorch is doing is matrix multiplication and addition with n-dimensional arrays (tensors). For backprop you need a very limited understanding of calculus. [1] Stuff like sigmoid/softmax is also straightforward and you can grok it from a 5 minute youtube video. [2] If something doesn't make sense, just ask ChatGPT to walk you through it.
There is a lot of other complicated math that you will need when reading papers because people writing AI papers like to overcomplicate a lot of things that can be represented in 5 lines of pytorch.
Til I learnt about discalulia!! Interesting. I know math can cause anxiety in kids as/when there is an external "reward/loss" contingent on it. I wonder if that exists when those external incentives are removed?
I did a ee and cs degree in college and I hated the eng part of it. It was because I hated the exams which was mostly rote stuff. When I finished college though and started working in a job that had no linkage to ee I one day just picked up courses (which were just arbitrary web 1.0 sites by random folks) and started going through the (signal processing/kalman filtering/convolution) problems and implementing them in software which was a lot of fun to see in action.
Reason for that long winded one was, I was thinking in your case, would the anxiety still be there as now you (might) have a reason to develop passion for the math than back in school where it was forced without a reason?
I have attempted a couple times to re-learn and the result is always the same. It goes beyond anxiety. If I try to read so much as a three digit number, my brain turns to mush and things start to get blurry. I have trouble reading ticket numbers on a daily basis, and struggle to help my grade school kids with their homework. I _can_ do formulae and calculations when I really hunker down, but it takes me considerable time, and I'm often wrong.
Have you tried recently? I was one of these people in high school but now in my 30s, stuff that seemed impossible no longer does. Maybe me or the context is different.
I have. I am disappointingly crippled by dyslexia. It takes a tremendous amount of energy for me to work with numbers and formulae which is one of the reasons I was drawn to programming. Once I realized I could abstract away the numbers and only really needed to get the calculation right once before putting it into a function, it was like jet fuel.
What did you try? I was the same, also in my 30s, did some online stat and calc classes, passed those classes, and I don't think I learned much. If I did, I've already forgotten it I'm afraid.
- Know the Statistical language that you learn from a basic college-level Stat 101 course. Be able to translate normal sentences into those using Statistical notation, and be able to read easily. Also, know basic Statistics.
- You already know programming, I assume. Learn Python if you don't know already. It's really easy.
- There are a number of paths you can go from there. Here's what I did.
-- IBM Data Science Professional Certificate (not deep at all, but lays out the landscape well; did it in a week)
-- Machine Learning for Absolute Beginners by Oliver Theobald which you can finish in an evening.
-- Machine Learning Specialization by Andrew Ng on Coursera.
-- Deep Learning Specialization by Andrew Ng on Coursera.
-- fast.ai course.
- Learn PyTorch really well. I suggest Sebastian Raschka's book.
Now from here, you can chart your own path. You can choose NLProc, Vision, RL, or something else.
I went towards Vision. And I do Edge AI as hobby.
I was in the last year of college as a Physics undergrad, when I was hired to do Vision modelling/research for a non-flashy company in 2021. Finishing my CS Master's next month and starting to look for PhD. I worked in the same company for the ~2.5 years.
EDIT: If you want a job in big tech, grind Leetcode, and learn about system design, study Machine Learning systems, and be able to design them. Chip Huyen has a good book as I hear. 6-7 rounds of interview is common in Meta/Google. DL hackathon awards, open source contributions are significantly helpful.