Hacker News new | past | comments | ask | show | jobs | submit login

Those are not the same category. In communicators you can store the user / channel from a given time period in a specific partition and that's about it. It's likely never going to be queried after delivery since everyone has a local copy. If it does, you just download a slice based on a client-side generated hash. And the metadata basically doesn't change either.

Compare to Twitter where new people continuously query old messages with quickly changing replies, likes, retweet, mutes, blocks, etc to account for. But you don't want to materialise everyones timeline on every change either. However you likely do want to keep highly replicated snapshots of popular tags and popstars instead of processing actual events. And then you need some system that can query both and merge them into a usable personalised view.




Doesn’t Google search basically all of Twitter Anyway? Doesn’t that mean whatever Google has could easily itself be Twitter?


It's not indexed in realtime. (tweets from 20min ago are still not searchable) It has no interactivity either which makes the problem few orders of magnitude easier. And they don't care about subscriptions.

Implementing search: just chuck the last hour of fire hose into s3 and update the ngrams.

Implementing real timeish view: ok, so we need to distribute+replicate the data so timeline queries can happen in parallel and get fresh info depending on subscriptions.

Implementing the number of likes ticking up as you watch the tweet: basically alien technology. (I assume it's optimised somehow because otherwise that would convert a single tweet view into 1 + however long someone stays on the page / 2s times the load in naive implementation)




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: