I'm not an expert and would appreciate being corrected if I'm wrong, but I'm under the impression using a neural network after it has been trained typically requires relatively little computation and data. It's training it that takes the big compute and requires lots of data.

I think many services like Grammarly would be perfectly possible to implement without sending your data off device. There are just massive incentives not to.