1) Obviously it's about tradeoffs - if you don't have the capital then operating with last gen environments is a perfectly reasonable decision. For others who do - it is not.
2) What kind of queries do you have that take that long? If it's scientific computing - no way around it. If it's just DB slicing/aggregating - if you use last gen structures you'll get last gen perf.
3 ) Cache locality on repeatable computable units on local in memory data do go better - continuous batched background updates to indices with a real time layer like the OP suggested will become the new standard.
if you don't have the capital then operating with last gen environments is a perfectly reasonable decision.
I am indeed operating in an environment of limited resources. It's called "The Real World". Reading marketing language like "last gen" makes me lose interest in a debate very quickly.
I also operate in this "Real World" and I'm not constrained by your limitations - i.e. large clusters, lots of data etc.
Just because you happen to be constrained by your resources it does not follow that your world is any more "Real" than mine - it's just different - which is what I said.
What part of "last gen" is marketing speak? Original Xbox is "last gen", the iPod is "last gen" - quite literally the last generation.
It is marketing speak because marketing people use these kinds of phrases to stop readers thinking about the merits of different choices they have, and instead focus their minds on the idea that old technologies are superseded by new ones. Other popular phrases are "legacy" or "conventional".
The original Xbox is no longer available in the stores. That's because it has been superseded by the new model, which does everything the old one did, only better! Poor me who just doesn't have "the capital" to get myself the shiny new one just yet.
Apparently, that's the way you want me to think about immutability versus mutability in data structures. Makes no sense.
2) What kind of queries do you have that take that long? If it's scientific computing - no way around it. If it's just DB slicing/aggregating - if you use last gen structures you'll get last gen perf.
3 ) Cache locality on repeatable computable units on local in memory data do go better - continuous batched background updates to indices with a real time layer like the OP suggested will become the new standard.