This is where being a clear and confident communicator, and a planner comes in handy. Decide how much time off you deserve: four, five weeks, whatever, and then schedule and broadcast it to the company. Now when anyone has a problem with it, justify it based on your productivity and nothing else. Bam, ball is out of your court.