I don't think it's a dumb question. And I think you're right: the closer you get to the way the CPU you're targeting likes its memory optimally managed, you're basically heading towards assembly.
I have notions that there can be something 'between' assembly and, for instance, C, but that's a daydream of mine without any details worked out yet.
I’m not sure you’re actually heading down the path of assembly in terms of the language itself, but you’re definitely moving towards the architectural lock-in of assembly.
But maybe thats fine if the language can express architectural-specific decisions (like memory layout) and abstract it away in a more sane fashion than ifdefsc and the rest of the codebase can maintain the pretense of portability.
I have notions that there can be something 'between' assembly and, for instance, C, but that's a daydream of mine without any details worked out yet.