I’d be interested to see a benchmark of actually selecting rows by manually specifying the partition vs not.
This benchmark seems to be pure computation of the hash value, which I don’t think is helpful to test the hypothesis. A lot can happen at actual query time that this benchmark does not account for.
That's fair. I had an EXPLAIN ANALYZE which shows the time it takes for planning and execution. And sending the query directly to the table outperforms that.
We don’t want to round trip to a network endpoint in the ideal path, but we run multiple instances of our monolith and we want a shared cache tier for efficiency.
To be honest, I'm not sure I can recommend anything specific here.
1. How much data do you have and how many entries? If you have lots of data with very small records, you might need an off-heap based cache solution. The only ready-made implementation I know is Olric [1].
2. If you can use an on-heap cache, you might want to look at groupcache [2]. It's not "blazingly-fast", but it's battle-tested. Potential drawbacks include LRU eviction and lack of generics (meaning extra GC pressure from using `interface{}` for keys/values). It's also barely maintained, though you can find active forks on GitHub.
3. You could implement your own solution, though I doubt you'd want to go that route. Architecturally, segcache [3] looks interesting.
The original groupcache is basically unmaintained, but there's at least two forks that have carried on active development and support additional nice features (like eviction), and should probably be preferred for most projects.
Hm, without more details on the use case and assuming no "round trip to a network" means everything is running on a single host I see a couple of options:
1) Shared memory - use a cache/key-value lib which allows you to swap the backend to some shmem implementation
2) File-system based - managing concurrent writes is the challenge here, maybe best to use something battle tested (sqlite was mentioned in a sibling)
3) Local sockets - not strictly "no network", but at least no inter-node communication. Start valkey/redis and talk to it via local socket?
Would be interested in the actual use case though, if the monolith is written in anything even slightly modern the language/runtime should give you primitives to parallelize over cores without worrying about something like this at all... And when it comes to horizontal scaling with multiple nodes there is no avoiding networking anyway.
perhaps a NATS server colocated on each monolith server (or even embedded in your app, if it is written in golang, meaning that all communication is in-process) and use NATS KV?
Or if you just want it all to be in-memory, perhaps use some other non-distributed caching library and do the replication via NATS? Im sure there's lots of gotchas with something like that, but Marmot is an example of doing SQLite replication via NATS Jetstream
Since you mention no network endpoint, I assume it's on a single server. If so, have you considered SQLite? Assuming your cache is not massive, the file is likely to end up in Filesystem cache so most of reads will come from memory and writes on modern SSD will be fine as well.
It's easy to understand system with well battle tested library and getting rid of cache is easy, delete the file.
EDIT: I will say for most use cases, the database cache is probably plenty. Don't add power until you really need it.
Could you add a bit more to the “distributed cache” concept without a “network endpoint”? Would this mean running multiple processes of the same binary with a shared memory cache on a single system?
If so, that’s not how I’d normally think of a distributed cache. When I think of a distributed cache, I’m thinking of multiple instances, likely (but not necessarily) running on multiple nodes. So, I’m having a bit of a disconnect…
I haven’t benchmarked this, but I have recently benchmarked Spark Streaming vs self-rolled Go vs Bento vs RisingWave (which is also in Rust) and RW matched/exceeded self-rolled, and absolutely demolished Bento and Spark. Not even in the same ballpark.
Highly recommend checking RisingWave out if you have real time streaming transformation use cases. It’s open source too.
The benchmark was some high throughput low latency JSON transformations.
It works but it shows phone numbers rather than contact names and you can’t assign a name to a number without giving access to your entire contacts … it ticks me off.
No, it isn’t. Browser root programs have certificate transparency (SCT embedding) requirements which would immediately reveal to the world if an ISP started to use a trusted root to MITM its users.
I think the title buries the most horrifying part of this. The HiCA certificate authority is relying on an RCE to do an end-run around the semantics of the ACME HTTP-01 validation method.
Fucked up and they should be booted from every root program for this.
They aren't in any root programs. They're just taking certificate requests and relaying them to real CAs, which is why they need to exploit an RCE in the ACME client, since the ACME client wouldn't otherwise be able to complete the validations required by the actual CA.
When confronted they just flat out shut down the service. They also donated $1000 to the project, and they've redirected requests to their payment site to the US White House's website, and they're from China.
They were also suggesting that user's ran the utility as root...
There is years and years and years of amazing content and community on Reddit. Rallying people around a replacement looks challenging. I am hopeful that it happens.
They're solving the same "high level problem", but with very different approaches.
TensorRT is proprietary to Nvidia and Nvidia hardware. You'd take a {PyTorch, Tensorflow, <insert some other ML framework>} model and "export / convert" it into essentially a binary. Assuming all goes well (and in practice rarely does, at least on first try - more on this later), you now automatically leverage other Nvidia card features such as Tensor cores and can serve a model that runs significantly faster.
The problem is TensorRT being exclusive to Nvidia. The APIs for doing more advanced ML techniques like deep learning optimization requires significant lock-in to their APIs, if they are even available in the first place. And all these assuming they work as documented.
OpenXLA (and other players in the ecosystem like TVM) aim to "democratize" this so there are more support both upstream (# of supported ML frameworks) and downstream (# of hardware accelerators other than Nvidia). It's yet another layer or two that ML compiler engineers will need to stitch together, but once implemented, they in theory can do a lot of optimization techniques largely independent of the hardware targets underneath.
Note that further down in the article they mention other compiler frameworks like MLIR. You can then hypothetically lower (compiler terminology) it to a TensorRT MLIR dialect that then in turn runs on the Nvidia GPU.
>You can then hypothetically lower (compiler terminology) it to a TensorRT MLIR dialect that then in turn runs on the Nvidia GPU.
there's no tensorrt dialect (there are nvgpu and nvvm dialects) nor would there be as tensorrt is primarily a runtime (although arguably dialects like omp and spirv basically model runtime calls).
TensorFlow is also a runtime, yet we model its dataflow graph (the input to the runtime) as a dialect, same for ONNX. TensorRT isn't that different actually.
OpenXLA is an open-source library for accelerating linear algebra computations on a variety of hardware platforms, while TensorRT is a proprietary library from NVIDIA that's specifically designed for optimizing neural network inference performance on NVIDIA GPUs.
openxla is a ML-ish compiler ecosystem built primarily around mlir that can target (through nvptx backend in llvm) and run on nvidia devices (on iree). tensorrt is a runtime for cuda programs. certainly they have features in common as a reflection of their common goals ("fast nn program training/inference") but the scope of tensorrt is much narrower.
This benchmark seems to be pure computation of the hash value, which I don’t think is helpful to test the hypothesis. A lot can happen at actual query time that this benchmark does not account for.