Hacker News new | past | comments | ask | show | jobs | submit login

Such as?



Concider what a database does. It provides ACID properties and ability to query data. If all you need is writing data, the fastest you can do it write it directly to the disk, without the overhead a database comes with.

Using a loadbalancer in front of a farm of cheap logging machines, and aggreate the data you need for analysis to a suitable machine.


they'd still need to query their stuff, I guess, so you'd need to trow in there somewhere something to aggregate logs and get the metrics they're tracking out of it - which can totally be done in streaming, without the need of going trough the logs every time, for most metrics.


With that amount of data, streaming and only saving aggregated data is the only sane way. With 1.2TB/hour there is a limit to how much historical data that can be saved anyway, and we're talking about 30% utilization of a 10gbps network interface, so it's beyond using single machines for most usecases.


Query? Most likely not. At least not in the traditional "lets on the fly create a dashboard" sense.


query as in 'how much I bill this guy for it's click' - doesn't have to be sql nor on the fly of course




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: