LMDB looks awesome. I lost a lot of performance (~5x) moving from a memory mapped architecture to LevelDB in a database I wrote (http://skydb.io/). I'll definitely be checking out lmdb to see if I can get that performance back. Thanks!
SkyDB is definitely most interesting thing I seen this year in DB field, mostly because it's a rare example of out-of-the-box thinking and considering higher level use cases than usual (data semantics level I'd say) may give you an orders of magnitude speedup.
But should I suggest, considering SkyDB highly experimental, maybe port it to more flexible backend like Mapkeeper[1] that gives for free not only easy switchable storage backends, but (not switchable, but changeable with a little effort) network/API backends (now it's Thrift), event engine (now it's libevent), etc. It would be nice to focus on core SkyDB things (which I think are data organization, API, stored procs etc.) while "outsource" the non-core low-level jobs to things like Mapkeeper.
And by the way, lmdb backend have being merged into Mapkeeper not so long ago :-)
Mapkeeper looks cool. I'll definitely read up on it some more.
Sky originally started out as a project to learn how to write a database and programming language. The storage was originally a block-based mmap format and the query engine was built on LLVM. It was a great learning experience but I am trying to take a higher level approach by delegating storage to LevelDB (or possibly LMDB instead) and the query engine uses LuaJIT. Focusing on a higher level use case (i.e. behavioral analytics) has given some crazy performance gains and great scalability.
I think once Sky matures more then it would be more interesting to be able to swap out the backends. I worry about having too many moving pieces. I'd also be concerned about copying data with the extra layers. It looks like LDMB does a zero copy which would be awesome.
Yeah, zero-copying is a great LMDB feature, even using LuaJIT stored procs you can pass pointer to mmap window and process the data in that window using ffi (did I say LuaJIT is just damn great?) and (optionally) DynASM.
