It's not just a matter of using random system functionality; I think Explorer uses the same UI code as Open/Save dialogs, in which case the UI itself necessarily lives in system DLLs. (But I'm a novice when it comes to Windows, so I could be wrong.)
Yeah, all the file tree view/list view stuff, shell namespace etc. etc. is implemented in shell32.dll, so we should probably compare with explorer.exe + shell32.dll. Everything else are basic widgets and OS interaction which clearly exists in both environments so shouldn't be counted.
And with that it is still only 4.27MB+6.81MB = 11.08MB on my system. So yeah, the bloat is real no matter how you look at it.
That said, the bloat is real.