The only time I got my M1 Air to actually somewhat heat up was when I was compiling Node.js from scratch, right after I bought it (prebuilt binaries weren't available yet apparently). So my experience matches yours.

I also do a lot of AI + Audio stuff, and it gets somewhat warm but not as much as when compiling heavy stuff.