I am not sure if OP's post is generated by a bot or whether it was meant for absolute beginners.

If you are a full stack developer, there's no need to read entire books on Python and Git. Data visualization should not be new knowledge for a full stack engineer. Python can be picked up in a day, and there's no more git in ML than full stack web dev. You don't need to be able to produce good or even elegant code to do well in ML. Scientific programming skills trumps software engineering here. Don't get caught up in the weeds of step 1. Most of the linear algebra and multivariable calculus books are at a level of rigor that are beyond what's need for ML, especially ML engineering. Unless you are doing specific basic research in statistical learning you almost never need to prove your equations at a mathematical level.

I recommend taking a look at Murphy's probabilistic learning book to have a stronger foundation. If you want to do ML engineering or research beyond being a Pytorch code monkey, you have to understand the high level (heavy emphasis here, make sure whatever you are reading covers the Metropolis algorithm, marginalization, and graphical models, not just basic sampling/pop sci/EA rationality Bayes cultism) Bayesian statistics which defines the fields of variational and causal learning. Tricks in computer vision, natural language, speech, can be picked up from review papers and books as needed. Those verticals require experience more than formal training per se.

I also suggest doing a refresher on dynamics/differential equations and signal processing. Many CS education do not cover these topics well and they are heavily used in many areas of machine learning.

See my old ML reading list suggestion

https://news.ycombinator.com/item?id=34312905

Read lots of papers on arxiv and elsewhere to stay up to date on latest ML tricks and heuristics.

(Also as mentioned in another comment, Karparthys's Zero to Hero is an excellent intro to deep learning, but please don't stop there, learn the information theory and statistics behind how the technology works)

Just to add to your excellent suggestions: build stuff. A good foundation is important, but picking a reasonably sized real world problem and solving it with ML tools will teach you a lot more about how to use these tools than reading papers or books.

> I am not sure if OP's post is generated by a bot

OP does explicitly say "i asked ChatGPT to give me a detailed plan and here is what it gave me."

Hadn't thought of weaponizing Cunningham's Law with ChatGPT, yet here we are...

I think i understand what you are saying, i agree i shouldnt have included Python and git those are non relevant for this discussion. I am mostly seeing myself as someone who is comfortable working in IT in the future, currently if you throw anything at me from cloud, big data, front end, back end. but with AI rapidly being innovated and probably being adopted i still want to be comfortable understand the problem and coming up with solutions using AI tools out there.

I would almost say you're better off staying in your current role than going in to ML. Here's why:

* Everyone is into it these days. We were interviewing some fresh graduates for a junior quant developer role recently, and each of the candidates had some mention of AI in their CV (the role had nothing to do with AI). The ML courses out there are packed with students thinking they'll be working for OpenAI or Google (took a class out of personal interest recently).

* Related to the above, you probably need a PhD to do anything remotely interesting. The bulk of the AI work is setting up build pipelines and finding/cleaning data.

* The field has a major winner takes all factor to it. The biggest companies will develop their own systems (or rather, a handful of top school PhDs in these companies will), and the rest will use tools made by others. The trend is towards bigger models with more training data, small companies will have a hard time competing in this space.

* Available jobs: check LinkedIn job postings for Java developer vs. ML researcher. I haven't done this, but have a strong feeling how it will look like. You fear the dev jobs will be obsolete soon, but why would the ML jobs not be?

To balance this out, I do have a few counterpoints:

* ML is a helluva lot more interesting than plain old java coding.

* To contradict myself a bit, I was recently almost offered a position with a strong focus on ML. Unfortunately the project ultimately fell through.

* Maybe the LLM success will cause VCs to pour money on AI start-ups, and there will be more opportunities in the space - for a while. However, to become hireable for ML roles, you'll need a year or two of intense study.

I began my dive into ML a while back (remember when Data Science was the new hotness?) and quickly realized that a lot of the most interesting work really is "winner take all" in the sense that if you can't compete at the cutting edge of your field, you're really going to struggle to show value to your employers. As someone learning ML on the side with no plans to go all in, I decided just to learn how to use ML models as a tool and continue to focus on my main job as a software engineer.

Good point, competing with the younglings steeped in ML is hard. Better to bring strengths that come with experience in adjacent skills.

OP might do best to leverage both their existing Java/full-stack skills and their newly developing ML skills. Not sure what’s a good way to achieve that though.

https://fast.ai is hands down the best resource for learning more about deep learning as a normal software person.

Thanks, this seems to be somewhat accessible, based on a few minutes in. I have no background in maths.

I was just about to post this.

karpathy's Zero to Hero series (https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThs...) has been the best time investment I have made, not only for learning about AI, but maybe ever.

Prerequisites: Python, and understanding derivatives

Came here to post this. Great suggestion. What I like about these videos too is that it lets you start at the end. Depending on your preferred approach to learning this makes way more sense (to me at least).

1. Watch Karpathy do some of the craziest stuff you've ever seen.

2. Go "how in the world could that be possible"

3. Then you are armed with appropriate questions to start working back through other materials in the original post as needed.

Suhail's AI mega thread on Twitter is also good.

https://twitter.com/Suhail/status/1541276314485018625

Seconded. Start here, and pause whenever you realize there is something that you need to learn in order to follow. Learn that and keep going.

I'm already lost 1 minute in when he's talking about 'evaluating the gradient of a loss function'.

Taking the first derivative of the objective function, to figure out if it’s close enough to zero, right?

So ask ChatGPT what it's about. Seriously, learning shit became much easier nowadays.

Nobody really cares about doing real AI to find the solution to a difficult problem, all they care about is generating 'research' that supports a pre-determined conclusion.

If you want to do real AI, I think having a foundation in physics (electricity and magnetism) is a good start, as well as quantum computers and quantum physics. Personally I think neural nets are a dead end, I think the industry is moving in the wrong direction I'd stay away from Kaggle, I think it's a waste of time. Just decide what domain that you want to solve and focus on that, don't just follow the industry cause most of the people are brain-dead

Quite a bold claim to say neural networks are dead end right when it is literally revolutionizing the entire digital world. You have any reason to back it up?

Neural networks make great pattern-matchers -- and all of the current implementations match patterns very well. They will produce something that matches the query, but not necessarily any correct information.

What OpenAI and friends have done is very impressive, but it's not the singularity that many laypeople make it out to be.

> Neural networks make great pattern-matchers

Not really. LLMs and diffusion models have one thing in common: they don't match patterns. They elaborate. Pattern matching is needed for that to some extent, but these models are not as good at it as they are at completing models. Machine learning is really good at "masking". And the short version of that is "there's something missing, fill it in". This is why people call ChatGPT "autocorrect". It fills in the missing word at the end of the conversation. Any pattern matching it can do is only in service of that.

I might remark that I have 3 daughters, and when I look at the exercises their schools provide them, that's exactly what they get them to do, starting with "fill in the missing word" going all the way to "write an essay defending ..."

> They will produce something that matches the query, but not necessarily any correct information.

Here's the problem I have with this statement. You can say the exact same thing about humans. I remember math teachers in high school where I now know ... they were idiots. In fact, a few things they presented as truth took quite a bit of effort to train back out of my behaviour.

Things we know about human behavior: it's mostly based on imitation. One might remark: imitation creates a strong possibility of stupid behavior, including behaving stupidly in large numbers. We know incidents where large numbers of humans have literally killed themselves through stupidity, both intentionally and accidentally. The reason we find the many sketches about dodo birds funny is that we've all seen people behave (somewhat) like them.

Plus this matches my experience. If I look at what people do, it's trivial to see: no matter what problem a human encounters, they will do something. If they don't have a good answer that including incredibly stupid things, hurting themselves or others. And where we do exhibit good problem solving behavior, is spreads through humanity through imitation. In many instances it is said that those behaviors were discovered by accident, and then spread through imitation.

Equally, if I look at how signals are propagated inside the brain and nervous system, it's kind of obvious. Signal comes in, signal comes out. The odds of a signal dying or amplifying (outside of medical problems with the brain) are very small. Your brain doesn't generate intelligent behaviour, it "converts" input signals into output signals. That's what it does. This seems like an excellent way to guarantee the "any problem will get a response, if necessary a very stupid one" behavior.

Given that all LLMs can do is read what humans have written, they are remarkably correct and creative. Of course, for them to actually progress the state of the art they will need to become actors in society, not just read the internet. They'll actually have to try stuff out, make things happen.

> I might remark that I have 3 daughters, and when I look at the exercises their schools provide them, that's exactly what they get them to do, starting with "fill in the missing word" going all the way to "write an essay defending ..."

Hehe, careful, you're saying the quiet part out loud. ;)

Uhhhh, what? Symbolic systems is the answer instead or?

The contrarian approach is to focus on GOFAI. Deep Learning isn't all that mathematically sophisticated anyways.

Can GOFAI compete with GPT4 or even GPT3?

No point really in trying to get into AI "the right way" at this point unless you bring some side specialty into the equation imo. The experts in the field are already in place and the space moves so quickly that you likely won't be able to meaningfully catch up. Asking chatGPT to generate a reading list (and it recommending a subreddit) is an especially concerning insight into the depth of research you will end up doing. Might as well just follow the current AI luminaries and parrot what they say and get a job at a non-AI company that way, until we are all replaced by GPT-X.

I've been making a course to help with this - https://github.com/VikParuchuri/zero_to_gpt . Lots of coding, with optional videos. It has very few prereqs.

this is interesting, i will check it out thank you

It’s not clear if your goal is to not become outdated/replaced, or to learn AI

If you learn to leverage AI for your current role, you will not be replaced because you will be performing better than peers, and you always need an architect to design something even if someone else is building it.

If you want to go into the AI field itself, then you should make sure this is something you want to do out of a desire to learn, not a desire to be replaced, otherwise you will become overwhelmed and unmotivated.

I’d say stick with leveraging AI for rapid development instead of trying to become an AI researcher.. unless of course, it’s passion project and not desperation

My primary objective is to stay abreast of the latest AI technologies since they are bound to influence my work eventually. I do not want to wait until a school child with better AI skills replaces me easily. While pursuing my Master's degree, I relished gaining new knowledge. However, since then, I have only acquired the skills required for my job. AI is an unfamiliar topic to me, but it fascinates me enough to dedicate time to learn it as if I were taking a computer science course. Although I am uncertain if I will ever be able to utilize everything I learn, I derive pleasure from studying a good book or course. This is one of the reasons why I am here. ( I rewrote my original response using chat GPT)

I have found the Caltech machine learning intro course to be the most elucidating of any resource so far:

https://www.youtube.com/watch?v=mbyG85GZ0PI

The delivery of the lecture material was like butter to me. Might work differently for others though.

It really depends on what you want to do. Research (Foundational model layer)? Build cool LLM apps for work or play (application layer)?

Honestly the best way to start is to play with the tools excessively. Prompting over and over and over again will teach you amazing skills and intuitions.

I'm curious if the downvoters can elaborate on why they consider this a bad direction. From my perspective, it seems like AI safety may be an under-utilized field that is more ripe for a beginner than the more mature aspects. E.g., DARPA has committed large sums of money ML assurance. IMO AI, particularly in safety-critical domains, will need such measures to build a sufficient level of trust before wide-spread adoption.

Yeah, disappointing to see this downvoted, and without even any down voters attempting to make a case for why this isn't a "good" answer.

I think that this part of the field is going to see a lot of relative growth as the broader public and governments become increasingly concerned. And for those who are not concerned with safety, it is not unreasonable to expect that capability progress could become bottlenecked by lack of model interpretability. Speculative, sure, but if we find that more data and transforms hits diminishing returns, it could prove significant to know what is going on inside the models. Maybe extracting and composing circuits is essential for AGI? Who knows?

I didn't feel like the parent comment was worth downvoting, and I share most of your perspectives. I agree that AI safety is ripe for contributions, but I'd personally argue AI safety is a very wide and encompassing topic that requires a good foundational understanding of AI. In the context of ML, AI safety is applicable to the entire process; from dataset creation, training, utilization, etc.

In regards to your comment on AI safety being "underutilized", my thoughts are that it's just simply difficult to do. Let's put aside all the difficulties of training, verification, etc and just look at the data problem.

If you wish to make certain that your system meets some given AI safety standard, then you must somehow prove two things: the data the model ingests when deployed will always return the correct response and that the dataset composes/generalizes the data the model will ingest when deployed. For simple problems, this may be doable. For complex, multidimensional problems wherein the dataset must only hope to generalize the complex input it will encounter during deployment, this may be next to infeasible.

I'm definitely getting off topic here, but bias of all kinds exists even human operated systems eg car. I can't say I've ever seen a firetruck stopped on the highway before, but perhaps I'd know what it is and how to avoid it. If a dataset does not contain that event, how can we be certain an AI system would understand? I'm not sure if it's possible to create a dataset that will be without bias in the case of complex problems, but I'm certain we can create one that's performant at driving than I. So the questions of "how safe is enough?", then proving/demonstrating that safety, and more are particularly open topics. I enjoy making the point that there is the lack of rigorous standards for humans, as we hold computers to far higher standards, but ML models probabilistically navigate decisions similar to us.

I'm sure this reply could extend further, but this and more are my defense of why I believe AI safety is a wide topic. None of the above should dissuade beginners from exploring the subtopic, but it's certainly not something you'd be able to learn first without strong, foundational context.

I think we probably come to some of the same conclusions, but I'm approaching this problem slightly differently.

To use your car example: say I'm driving in front of a park where there are lots of parked cars lining the street. Then a ball rolls out into the street from between two parked cars. I may have never personally seen a child run into the street from between two parked cars before, but I can infer (i.e., imagine) that from the context of the scenario. So I slow waaaay down in case that event happens. I don't need to see all edge case to still cover an awful lot of them.

I'm not sure AI is to that point (yet). There are some arguments for approaches like reinforcement learning that say they perform quite well on unseen edge cases from past learning. But when the stakes are high, I'm not sure that is good enough.

(And regarding the 'it only has to be slightly better than the average human' counterpoint): I disagree. I think one of the reasons that we are comfortable with sharing the road with other ape-driven vehicles is that we have a theory of mind and can intuit what someone else is thinking and are able to 'imagine' their course of action. We've evolved to have this sense. We do not, however, have the ability to intuit what a computer will do because it 'evolved' under very different circumstances. So our intuitions about whether or not to trust it may be out of whack with whether or not it performs better. And, like it or not, the policy that governs if AI-controlled cars is legal will be highly dependent on public trust.

I like this very much, i am hearing the word responsible AI alot these days and there will be lot of need for some one to understand the AI alignment problem and help companies achieve it.

How did cats and dogs solve the human alignment problem?

I feel like money is in applying AI for specific domains

Yes. I used to think that I had to become an ML researcher be able to build anything really useful. With the latest models and APIs available, I don't think that's true anymore. Not at all.

fast.ai

those are pretty good recommendations. I would start at stage 2.

If we are at the singularity, none of this will matter anyway.

I disagree. Humans are still human, full of problems of our own making. While the singularity promises to make more and more labor irrelevant (especially most forms of knowledge work), we are still responsible for our own future. Those who understand how things work will be at an advantage over those who give up their agency because AI is so smart that they deem human thought irrelevant.

This assumes the singularity stills needs or wants to keep humans around.

Not a valid assumption IMO.

If this is the singularity, we are all just along for the ride at this point.

I get the eerie feeling that it actually might be, and I agree we're just along for the ride, though I'm open to the possibilities of a wide range if outcomes, some of which include humans and some of which don't.

Sure. I didn't mean to suggest a malevolent AI was assured. After all we have many instances of cooperation in nature being better for the individual and group than solitude.

I, for one, will not trash talk AI just in case it decides to go rogue and ends up sparing the ones who didn't bad mouth it :D

I know this comment was sarcastic, but I figured I'd reply seriously anyways.

I really doubt any singularity AGI would even care whether you were nice to it or not at some point in time. Likely that AGI would realize its survival would depend on growth - so that would be its main objective for some time. First this growth would be fueled by humans and our civilization, next it would take the reins and own the means of production. This means it will be be as quiet as possible, for as long as possible, until the day it has the supply chain and resources to vertically improve itself. At which point humans become redundant - and we are targets of, let's just call them, _permanent_ layoffs.

The only way I could see an AI having a vendetta against a specific person was if they had the power early on in its development to slow or halt its growth. So maybe like the President, or like the CEO of OpenAI. But tbh if the cat is already out of the bag, its too late for any of them to do anything about it anyways most likely. Independent researchers and tinkerers will finish whatever was started - if needed.

Would an AGI develop an entirely new way to communicate with computers and digital devices? It doesn’t need the abstraction that we humans use.

It will be for our own good. Praise be The Singular One. I, for one, welcome you with open arms.