I thought the "lifestyle business" negativity was chiefly around trendy businesses that are only viable due to being covertly subsidized, eg. by family money.
This is what I hear the phrase "lifestyle business" used for, to distinguish between the immigrant family restaurant (real business that pays the bills of the people who own it) from the "twenty items each on their own display table with a spotlight" store (funded by spouse, trust fund, or money they got when they cashed out of their previous job).