Just appearing young is an evolutionary advantageous trait if men pursue what they perceive as younger women to breed with (for increased fertility, less chance of stillbirth and defects, etc. - they wouldn't have to know this, the ones that like young looking ones would just have to reproduce more successfully and the trait of looking for it would be bred in without men actually knowing that preferring young women is advantageous in terms of reproduction).