Insider trading takes a lot out of the pockets of literally everybody else, there’s barely any punishment when you get caught, and you’re really quite unlikely to get caught at all. It is a problem you should care about.
As an extreme example, look what happened with Gamestop. As people piled on to purchase more and more out of the money calls, it forced an infinite gamma squeeze where market markers selling the call options were forced to purchase the stock underlyer to hedge, driving up the price.
The effect isnt always that obvious, but it is there. It is also unfair to the person selling the option, if they are doing it with normal knowledge (not insider knowledge)