The world consists of more than two people. It isn't just about them. Am I misunderstanding your point?
Many individuals care about society as a useful construct -- a construct that is not easily calculable from individual utilities. [1] This would suggest that even utilitarians should care about society -- unless they think they get to define what matters to their precious individuals. [2]
[1] Sure, one can say society is _causally_ derived from individual actions, but... (1) the derivation of what society looks like is not predictable enough for the time scales we care about; (2) individuals are influenced by society, as a matter of perception
[2] If I may attempt some satire, I wouldn't at all be surprised if some utilitarians are a sort of "mini-autocrat" at heart -- in the sense they get to decide what counts in every individual's utility function. e.g. "I value you, individuals!, yes I do!... but I get to tell you what really matters for your happiness! and after I do that I decree that the summation operator is how we put it all together!"
Many individuals care about society as a useful construct -- a construct that is not easily calculable from individual utilities. [1] This would suggest that even utilitarians should care about society -- unless they think they get to define what matters to their precious individuals. [2]
[1] Sure, one can say society is _causally_ derived from individual actions, but... (1) the derivation of what society looks like is not predictable enough for the time scales we care about; (2) individuals are influenced by society, as a matter of perception
[2] If I may attempt some satire, I wouldn't at all be surprised if some utilitarians are a sort of "mini-autocrat" at heart -- in the sense they get to decide what counts in every individual's utility function. e.g. "I value you, individuals!, yes I do!... but I get to tell you what really matters for your happiness! and after I do that I decree that the summation operator is how we put it all together!"