You're right that Chrome OS doesn't store any of that at the moment. But this is the sort of integration that would be fairly easy for Google to add. Chrome already has some minimal integration with Google web services, especially on Android where Chrome will automatically log in to Google for you. How much bigger a step would it be to fetch your Google contacts while they're at it?
The concept of permissions (to contacts, etc) doesn't exist on Chrome OS and will likely never exist on Chrome OS. But both Chrome and Android apps can access the Internet where all this stuff is stored. But this has nothing to do with security of Chrome OS itself.
Sure, nothing stops a website (or app) from going to Google to ask for your contacts. This is the classic "invite your contacts" strategy that Facebook, LinkedIn, etc. have employed since the beginning. But granting that access is almost never mandatory with such services. Why? Because people just wouldn't sign up. It's not even necessarily a question of people not trusting the service, because the UX hurdle of forcing the user to enter yet-one-more-password is enough to hurt conversion rates.
On Android, this is not the case; if you use the Facebook or LinkedIn apps, you will be forced to give up access to your contacts (unless you root your device). The reasons for this are twofold. First, Android permissions are all-or-nothing, which means that apps have to over-approximate their required permissions. Second, because granting access is so easy (no UX hurdle to speak of), there's really no downside to doing so (which is to say, conversion rates will not decrease substantially).
The issue of how the device gets access to the data (e.g. whether contacts are cached locally or are purely remote) is really not the issue. You could easily implement the Android API with no local caching of data, and aside from the performance hit the behavior would be identical.
I could certainly imagine a version of ChromeOS which supports Android apps without the Android permission model, essentially by returning empty data sets whenever the app asks. This would be compatible with ChromeOS's current security policies, and would protect user privacy. But it would be so easy for Google to break that line in order to promote "better UX" for apps in the style of Android, that I can't honestly convince myself that they'll never go down that road. And that's why I can't be entirely comfortable with believing the ChromeOS security model will remain intact.
