My Dad, when he was young, worked at the Institute for Advanced Study while von Neumann and Einstein were there. AFAIR, my Dad said that von Neumann tended to work late at night (I am visiting him this weekend and I'll ask about this).