Have you considered an Analogue Pocket? It's like an "upgraded Game Boy" with a better screen that can also play games from the Game Gear, Neo Geo, etc.
I bought an Analogue Pocket last November and already the buttons on the front are starting to wear out. The D-pad sometimes doesn't register a click and other times registers two clicks.
Nice device, but for $290 it should last longer IMHO.
Yeah, the trouble is acquiring cartridges in 2024 is just a hot mess navigating knockoffs, etc and expensive.
And if I am going to emulate I might as well get them an Anbernic or Miyoo emulation device. But, those can have complex and annoying user interfaces. I guess it could work and I could make SD cards that just have a handful of games.
FYI the Analogue Pocket supports a number of OpenFPGA emulators quite well, I have several carts but also plenty of harder to find NES, SNES and GBA roms loaded right on.