> (I will also forever wonder why Sega's 16-bit console didn't have hardware sprite scaling and rotation, since that was the main attraction of Sega games at the time...)
An interview with one of the hardware designers on the Mega Drive/Genesis VDP suggests they wanted this, but that didn't have enough die space. It's also not clear how comparable to their arcade hardware this would have actually been even if it was present. Scaling and rotation requires non-sequential access to the source data which the VRAM used in the MD/Gen is not very good at. My guess is that the limitations would have been similar to what you saw with the scaling hardware on the Sega/Mega CD (i.e. relatively low frame rate due to the limited bandwidth available for updating VRAM).
> They could have made the console a bit bigger, and a bit more flexible inside.
They could have gone to a 2 chip solution for the VDP and implemented scaling, but it would have substantially increased costs. Now, Nintendo did notably go with a 2 chip solution for the PPU, but they were using a lower cost CPU, launched 2 years later and sold for a higher price.
There are other places they skimped that hold up much worse than dropping scaling hardware. For instance, having seen photos of the VDP die it seems pretty clear they could have made a nice improvement to the palette hardware if they took more time to optimize the layout (though I imagine that could have taken quite a while with 80's IC design tools).
An interview with one of the hardware designers on the Mega Drive/Genesis VDP suggests they wanted this, but that didn't have enough die space. It's also not clear how comparable to their arcade hardware this would have actually been even if it was present. Scaling and rotation requires non-sequential access to the source data which the VRAM used in the MD/Gen is not very good at. My guess is that the limitations would have been similar to what you saw with the scaling hardware on the Sega/Mega CD (i.e. relatively low frame rate due to the limited bandwidth available for updating VRAM).