Using offheap memory allocation in Java is generally a bad idea.
Java is a GC'd language. If you need to manage memory manually, use a language designed for that. The lengths people go to NOT learn another language (while having to learn stuff like this and do primitive memory management in your application code - which itself may use the GC if you're not careful) amazes me.
It’s the same as using “unsafe” in rust, it shouldn’t be necessary by default, but when you do need it, you can have a well-defined part that safely encapsulates all the logic of dealing manually with memory. The end result will be a safe program with the assumed memory/performance-bottleneck of doing it naively solved.
>Using offheap memory allocation in Java is generally a bad idea.
Memory mapped files and direct buffers do work well. I'd never consider it a bad idea; technically both would be freed if there are no strong references to them (not much different than allocating massive arrays).
Think of it like that the jdk standard library has to use native memory for any IO or even a trivial zip compression. ByteBuffers were introduced in 1.4 (around 20y back) to allow operations outside the managed system, the tools are available to anyone, e.g. implementing zstd with direct buffers is not hard.
Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones.
I saw a pretty jawdropping speedup moving from bytebuffers to the new foreign memory stuff with in the marginalia search index code. Most operations run about 50-100% faster, even discounting the rigmarole needed to deal with mmapping >2 GB files. Haven't dug too deep into why this is so I'm not entirely sure why, may have something to do with being able to declare non-shared memory ranges in the arena allocator, saves a bunch of synchronization maybe?
I can't wait for this stuff to leave experimental. It's such a quality of life boon for dealing with off-heap memory, having explicit lifecycle control is.
> Unfortunately, you do pay a fairly large price for bytebuffers, even off-heap ones.
I'd disagree. Heap based bytebuffers - I don't consider them interesting, they are just byte arrays. The direct ones suffer from unmapping (munmap on linux) - it's a slow process that has to flush the TLB. So the only sane way to use them allocate once/reuse. Anything else I'd consider a programmer error.
mmap on large files indeed does suck (a bit) w/ ByteBuffers as it'd require an array of them.
Personally I have been using direct buffer since 1.4, so I guess I am also quite used to them as well.
Right, I'm not talking about mapping or allocation costs or anything surrounding the access, just the impact of addressing off heap bytebuffers them seems quite a bit slower than what you get with the new foreign memory API... for whatever reason.
You mean using directBuffers in java - that could be, if the JIT fails to remove the bound checks.
It should never be an issue in native code, of course. But yes - it's possible that it happens with Java code. In that case you'd need print assembly and looking at the generated code.
Some libraries have 'switched' to straight unsafe use (which has no bound checks, of course), so there is that.
While I fully agree as a rule you shouldn't have to do manual memory management in Java, the functionality is there and useful in exceptional situations.
I've seen integrations where there's no '100% pure Java' implementation available, so you have to choose between picking a battle-tested native library or reimplementing from scratch in Java (which sometimes isn't even possible with closed source systems)
It is called taking your cake and eating it too, in terms of productivity, IDE tooling and library ecosystem, for the same reasons ML frameworks use Python with bindings to C++ libraries, instead of being 100% written in C++.
