Does it take the same amount of time to load on subsequent runs as on the initial run? This should not happen. Are you running in incognito mode, perhaps? (in that case, caching doesn't happen for obvious reasons). Feel free to submit a report on the native-client-discuss mailing list or through the bug tracker.
It is faster, say less than a half second and not the 10 seconds it was the first time, but still very noticeable. Normal latest chrome on a Windows 2008 box.
