> outside of trade show demos, kernels are condegened. What is true is there are recurring "themes/patterns" that are handled by engineers for a class of products. Lately this is flash attention
I never said they weren't using code generation. I said that each one requires a manual tune. You will set various parameters, determine if the generated code does well enough and then if there's performance to squeeze out, you modify the code generator.
They require one person or team to engineer and then a whole bunch of people to use...? That doesn't resemble in the least what you were describing where each kernel is hand-tuned for each shape and device. But please do continue to insist you're still somehow right
Sigh. I use to be an engineering manager for a kernels team. I think I know what I'm talking about. Yes, each kernel is paid individual attention to even if many are basically the same and require little rework. It's a lot of work. Now I work as an ic in the same field . I don't need to insist I'm right because it's what I do all day
I work in DB kernels, everything gets hand tuned as there is economic reason for hand tuning it. The expectation in many of these systems is that there are no wasted cycles. You can codegen a decent kernel, but then someone will find a better approach - do you want the slower version of the product, or the faster one?
You can see this in action with matrix math libraries, folks have been hand tuning those for decades at this point.
Your claim is that there are automated methods (which I mentioned in my original post) to manage the complexity. My claim is that it requires a large team of engineers working on it. I'm not really sure what you think you've refuted.
I never said they weren't using code generation. I said that each one requires a manual tune. You will set various parameters, determine if the generated code does well enough and then if there's performance to squeeze out, you modify the code generator.
> I guess you work at AMD.
Close but not quite