I own an online apparel company that I launched on HN three years ago, but before this, I built tech companies, so maybe I can weigh in with what I've found (and have been really surprised by).

Fit is almost a philosophical decision. Most companies only have five sizes (XS-XL) and the goal is probably to accommodate as large of a % of your customer base as you can with each of those five sizes. Most designers either use themselves to build their sizing from, or use a fit model, someone that literally is paid to stay the same size year round and give input on the fit.

After the decision on who, exactly, you want to design to, you need to decide how the individual garment should fit. eg. some streetwear might have a 4" longer shirt from the high point shoulder than mine, if they like that long, baggy look. Some styles, like James Perse, may want a laid back, beach vibe, or cater to an older demo that might want a more forgiving shirt, so they'll want a few extra inches around the waist.

In our case, I'm a very consistent medium -- almost every brand I'd expect my friends to buy from fits me well at a medium -- so I tailored our shirts to myself. We then started with what is a fairly standard grade rule to go up and down the sizes, and then after many, many batches, fine tuned each size to feedback from our customers.

So -- a lot going on. There is no way to standardize what a "medium" is across brands (and some companies don't even use the same terminology!)

The frustrating part, as a consumer, however, is when a brand doesn't have consistency between batches of the same style, colors within the same style, or between styles!! eg. I have a pair of Levis (huge fan..) on right now that I have the same exact wash, and style, but the 31 waist is MUCH bigger than the 32 waist. In this case, they were probably sewn in different factories, and they don't have tight enough tolerances in QC, or something. Between styles happen because a big company like Nike, for instance, may have 100 designers, who may not want to re-use design elements/blocks from one style to the next - and would prefer to fit / design their new tee how they see fit. But as a consumer, I expect all Nike medium shirts to fit the same, so that is frustrating.

Turns out, sewing things by hand, and dying, pre-shrinking natural fabrics, etc. is really hard. We've been working on our processes for three years, and are incredibly proud of what we've built... but we learn new ways to improve every day!