I'm not understanding this as homomorphic privacy.
They take pains to say:
> your device downloads the current model, improves it by learning from data on your phone, and then summarizes the changes as a small focused update. Only this update to the model is sent to the cloud, using encrypted communication, where it is immediately averaged with other user updates to improve the shared model. All the training data remains on your device, and no individual updates are stored in the cloud.
(my emphasis on the word stored)
Now there are lots of scholarly articles on reverse-engineering and rule-extraction from neural nets.
So Google, having the diff can actually get some idea what it is you are trying to teach the net.
In the secure aggregation paper (linked in the post: http://eprint.iacr.org/2017/281 ) they indeed hint at the possibility of extracting information from the diffs. So the protocol they propose cryptographically ensures that the individual diffs can not be learned. They do this by having each pair of clients generate a secret the size of the diff, and have one add it to their diff and the other subtract it. The clients then send that result to the server where they are summed, which cancels out all the secrets. The bulk of the protocol then deals with setting up the secrets and dealing with client drop outs, which appears to be the real challenge.
Well, that's my fallible summary anyway, go read the paper. :-)
They say they have a technique called Secure Aggregation as well so that it can only decrypt the update if many users have sent their updates. I don't know how possible that would be to reverse engineer.
Plus this would seem a computationally expensive way of conducting mass surveillance
They take pains to say:
> your device downloads the current model, improves it by learning from data on your phone, and then summarizes the changes as a small focused update. Only this update to the model is sent to the cloud, using encrypted communication, where it is immediately averaged with other user updates to improve the shared model. All the training data remains on your device, and no individual updates are stored in the cloud.
(my emphasis on the word stored)
Now there are lots of scholarly articles on reverse-engineering and rule-extraction from neural nets.
So Google, having the diff can actually get some idea what it is you are trying to teach the net.
They just promise not to.