I'm doing an ML apprenticeship [1] these weeks and Karpathy's videos are part of it. We've been deep down into them. I found them excellent. All concepts he illustrates are crystal clear in his mind (even though they are complicated concepts themselves) and that shows in his explanations.
Also, the way he builds up everything is magnificent. Starting from basic python classes, to derivatives and gradient descent, to micrograd [2] and then from a bigram counting model [3] to makemore [4] and nanoGPT [5]
The website doesn't say what—for me—is the best thing about it. The course is peer-led which works like this: once your join, you're part of a team which has one objective: get the best score with your ML recommendation system.
There is simulated environment in which all teams of the cohort receive millions of requests per day (and hundreds of thousands of users and items) and you have to build out your infrastructure on an EC2 instance, build a basic model, and then iteratively improve on it. Imagine a simulated facebook/youtube/tiktok-style system where you aim for the best uptime and the best recommendations!
Something I discovered not so long ago that I wish I had years ago is to watch the video first and then code along after. So simple but makes a world of difference, you can skip errors, fluff and foresee what's next, all around you'd think it's watching two 2 hour videos but it works out in terms of getting the most out of the content and drilling it into your head.
I've found that actually running the code has been very beneficial in understanding. This, along with reasoning for each line of code and spending a lot of time with the video paused and discussing and explaining to each other what we understood.
Also, the way he builds up everything is magnificent. Starting from basic python classes, to derivatives and gradient descent, to micrograd [2] and then from a bigram counting model [3] to makemore [4] and nanoGPT [5]
[1]: https://www.foundersandcoders.com/ml
[2]: https://github.com/karpathy/micrograd
[3]: https://github.com/karpathy/randomfun/blob/master/lectures/m...
[4]: https://github.com/karpathy/makemore
[5]: https://github.com/karpathy/nanoGPT