There's just been a twitter post by Omar Khattab (@lateinteraction) on encoding documents into a scoring function instead of a simple vector for the work on ColBERT - and maybe at some point using a DNN as scoring function.

So, yes, maybe there's a way to "distribute" RAG. (I still wonder if that isn't just MoE taken to its logical conclusion)

So, dig for ColBERT papers, might be helpful. (I wish I had the time to do that)