You would encrypt the data before giving it to a rogue server using a key that reflects the access you want to permit.

So if you want your friends to be able to use it then you'll need a shared key with your friends.

But only the clients have the key, so the servers pushing the data around can't access it.

Edit: Check out how Keybase handles this.