We didn't have "help" growing up but when there was a dinner party, there was still the idea that you had the dining room and that all the food prep/cleanup happened out of sight in a different room. You certainly didn't have mingling of guests and food prep/delivery.
This wasn't really upper class but it was influenced by British norms, albeit in the US, even if there weren't servants involved.
The cooking is still exclusively done by "the help" in restaurants, but the article indicates that the theatre, so to speak, of food preparation has come to be considered a desirable trait by those occupying the dining room.
So this doesn't quite answer why they settled on keeping it hidden instead of a display of entertainment. There is no reason to think people in that time wouldn't have found the same enjoyment in watching the food get made.
Most likely the answer is that it's just what one notable person happened to decide, by random chance, to do with his kitchen arrangement, and everyone else copied it in hopes that it would make them equally notable. Pretty much all human behaviour can be explained that way.
> it's just what one notable person happened to decide
Oh no. Chances are the real engine of this change is the recent mass-media discovery of food-prep as a heroic and cool activity.
Celebrity chefs used to be obnoxious, "always-be-prepared", by-the-book home-goddess archetypes like Delia Smith; at some point over the last 25 years, they turned into sexy, improvising, creative rockstars. The creativity-impoverished masses, bored to death by day after day looking at computer screens, now get their kicks by eating and cooking.
You are quite right that another notable person came along and, to buck the trend, decided it would be entertaining to watch "the help", and then the rest started to follow thinking that must be what is required to become notable.
But it could have gone the other way. The first notable person could have recognized the entertainment, and the later notable person could have decided it was better to keep hidden. We don't know why the first one decided to go with hidden, and likely it was just a random choice.
Not long ago, cooking was often done exclusively by “the help”.