I read that a problem the pollsters were having was simply that they didn't have any precedents. There was a referendum last century asking a different question but that wasn't that helpful. Essentially you can't make up a model without observations and opinion polls don't count
This is my understanding of the issue too. Polls don't just ask the question. Each cycle they take their input, the actual results, and refine their model. Over iterations the model should get more and more accurate.
This event is a one-off. No cycles, no iterations, leaving a model which is essentially a best-guess.