A part of the problem is that the ingestion is not vector compressed, so they're... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

jiggawatts 56 days ago | parent | context | favorite | on: Lossless Log Aggregation – Reduce Log Volume by 99...

A part of the problem is that the ingestion is not vector compressed, so they're charging you for the CPU overhead of this data rearrangement.

It would cut costs a lot if the source agents did this (pre)processing locally before sending it down the wire.

piterrro 55 days ago [–]

We should distinct between compression in transit and at rest. Compressing a larger corpus should yield better results in comparison to smaller chunks because dictionaries can be reused (zstd for example)

Consider applying for YC's Spring batch! Applications are open till Feb 11.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact