For those interested MIT 6.851 Advanced Data Structures covers Cache-Oblivious algorithms if your interested in the general CS/Coding techniques to maximize cache usage

OpenCourseWare https://www.youtube.com/watch?v=bY8f4DSkQ6M

Take what is fixed and make it flexible.

Take what is one and make it many.

Take what is repeated and make it one, and fast.

Where is this from?

These are common patterns I have extracted from things that are innovative. Most of the time, the innovation is a simple perspective transformation.

Read 1000 patents and extract the abstract refactorings that made the idea possible.

Also; Take what is flexible and make it fixed.

FPGA BlockRAMs have been configurable for the past two decades! The CPU architecture world is borrowing some good ideas from the FPGA architects.

To be fair, BRAMs aren't reconfigurable on the fly. That is, there's no assumption of data preservation when you change them.

Processors would need to temporarily hang to reconfigure hierarchy. Execution pipelines would stop issuing and resolve all in-process instructions and push registers to a stack, the memory system would need to quiesce the memory system and flush all structures, and finally the processor could rewrite low-level addressing crossbars. Once that's done, pop the registers back into the stack and hope there's no weird interactions with any number of legacy features in any older processor architecture. :)

That is a great point. Yes, most of the configuration parameters for FPGA RAMs are statically configured and would need to be "reconfigured" safely. Certain parameters are dynamically controlled (can be changed during execution without reconfiguration) and do not affect data safety, like cascade control if you want to stitch together multiple RAMs on-the-fly.

I think it can be done in a way that is quite less "stop and go". When you want to reassign a chunk, mark it as such and interrupt the core currently owning that chunk so it can do what it needs to do. Once it signal that it no longer have any data in that chuck, you simply reassign it and the new owner can start using it.

I wonder if projects like openblas can make use of this. Maybe it could be baked into the cpu too somehow.

On the other hand, when reading the title I was preparing for a gory Amiga article. In this case it should be on-chip memory or is my non native english fooling me here? But it was at least in the same spirit ;)

I wonder how the OS is supposed to deal with this complexity. It probably needs an API for processes to reconfigure the cache. But then the cache can be private to a process or to a group of processes. Complicated stuff.

Ideally, it will be selfcontained at the silicon level, perhaps with support for the OS to report statistics to the hardware.