Define "come to work". Am I considered at work if I'm going through that annoying bug during my morning shower, or spending my weekends networking online with technical colleagues?
I doubt this is a good comeback as the other person might very well also be thinking about some company related stuff and talking to colleagues on weekends.