The heuristic I use for distinguishing between statistical modelling, machine learning and AI is is through feature engineering and model specification:
- Statistical modelling: Manual feature engineering, manual model specification (y = ax + b)
- Machine learning: Manual feature engineering, automated model specification (y = ax + b or y = ax^2 + b, I don't care, the algorithm should figure it out).
- AI: Automated feature engineering (e.g. CNN), automated model specification
IDK about this. The model y = ax + b is not specified, parameters a and b have to be chosen by optimization. Now add regularization that some of those parameters are shrinked to 0 (lasso) and you have "automatated model specification" where only some parameters are left and others discarded.
And furthermore the models are always chosen from a predefined hypothesis set, so there can never be truly automated specification.
