In terms of product-market fit, great fit in a small market still limits potential success of a company in the absolute sense. (i.e. "dominating" the now-$5M a year "pog" milkcap industry)

That's why VC's sometimes look for startups that have seemingly strange ideas to "normals", but the potential to address problems in a massive market. Achieving product-market fit can take time, but doing so in a large market can mean more in the end. (i.e. Spanx - a "minor player" in the womens underwear sector when compared to Victoria' Secret)