Oh yes, but that might be comparable to attention-based models for NLP, complexity-wise. Using C3D is quite common since 2015 actually:

http://vlg.cs.dartmouth.edu/c3d/

https://arxiv.org/pdf/1412.0767.pdf

Training a great model is expensive these days, FB routinely pays 6 figures per training run.