Wow I'm a total novice but as far as I know most of the optimization methods for models (Pruning, Weight Sharing, Quantization, Low Rank Approximation, Binary / Ternary Net, Winograd Transformation) aren't really implemented in most ML frameworks. Would be fun to try it out one of these weekends
Winograd transform probably wouldn't be exposed at the framework layer. I wouldn't be surprised if cuDNN/TensorRT/TPU driver already use it internally.
It would actually be interesting to quantize Winograd transformed weights directly and then see whether we can squeeze some accuracy out (since Winograd is more accurate numerically than direct convolution). Maybe just too much work for potentially marginal improvement and you will get better result when incorporating quantization into your training process anyway.
Assuming you mean Fourier domain. It is not very useful to keep the activation in that domain. Winograd transform operates at 4x4 tiles (it is effective because 16 FMA of that 4x4 tiles can generate 2x2 activations (roughly speaking), while direct convolution requires 3x3 FMA to generate 1 activation). If you keep the activation in that domain, you are looking at keeping 4x4 / 2x2 thus 4x more information than it is needed, which is not useful or desirable.
Thus, the most optimizations are just keep the transformed weights (you are still looking at 4x4 / 3x3, 1.9x more data), but that can be somewhat justified by the computation cost. The transformed activations are not useful.
Not an answer, but I've never heard it called the "Winograd domain." The Winograd transform is an algorithm for computing small discrete Fourier transforms, so its the same Fourier domain.
cuDNN certainly does, if you use the CUDNN_CONVOLUTION_BWD_DATA_ALGO_WINOGRAD option (and you can ask it to automatically pick the best approach if you like).
