> This makes me very thankful for the work that the Go team has put into the go GC.
If you read that part of the piece again, the problem with the GC wasn't necessarily with Java's GC implementation but with Cassandra's storage architecture combined with a mass delete, which tombstoned millions of records and then Cassandra had to scan them and pass them over: "Cassandra had to effectively scan millions of message tombstones (generating garbage faster than the JVM could collect it)"
Even if you had an incremental GC, you'd be eating CPU and thrashing the cache like crazy doing that. And you may never catch up.
Actually even if you didn't have a GC language, you'd likely still have scan or free operations.
Writing databases is hard. And actually the various JVM's GCs are top knotch and have had hundreds of thousands of engineering hours put into perfecting them for various workloads.
Still, these days I'm glad not to be near it and I work in Rust or C++.
I'm not discounting that the GC had an overwhelming amount of work to do.
My comment wasn't intended to convey that the go gc is somehow more efficient, but rather that I'm thankful for the tradeoffs they've made prioritizing latency and responsiveness, and minimizing the STW phase. I can see how I didn't give enough context for it to be interpreted correctly.
It's been my experience that with larger STW pauses, you wind up with other effects -- to the outside observer it's impossible to tell if the service is even available or not. You could argue that if you're thrashing memory that hard, no, it's not available. In general though, it makes for higher variance in latency distributions.
> Writing databases is hard. And actually the various JVM's GCs are top knotch and have had hundreds of thousands of engineering hours put into perfecting them for various workloads.
no doubt on both accounts. again, thankful for the design tradeoffs.
> Still, these days I'm glad not to be near it and I work in Rust or C++.
For anything that really needs to be very consistent, or have full control execution, I don't blame you. Cassandra is a prime example of what happens when you don't have that full control.
I think they're actually talking about both at once.
Cassandra definitely has compaction (similar LSM storage as RocksDB), and it causes pauses.
It's been over 10 years since I touched Cassandra, but when I did there was often a royal dumpster fire when compaction & JVM GC ganged up to make a big mess.
If you read that part of the piece again, the problem with the GC wasn't necessarily with Java's GC implementation but with Cassandra's storage architecture combined with a mass delete, which tombstoned millions of records and then Cassandra had to scan them and pass them over: "Cassandra had to effectively scan millions of message tombstones (generating garbage faster than the JVM could collect it)"
Even if you had an incremental GC, you'd be eating CPU and thrashing the cache like crazy doing that. And you may never catch up.
Actually even if you didn't have a GC language, you'd likely still have scan or free operations.
Writing databases is hard. And actually the various JVM's GCs are top knotch and have had hundreds of thousands of engineering hours put into perfecting them for various workloads.
Still, these days I'm glad not to be near it and I work in Rust or C++.