If they bring in permanent staff to fill a short term role and lay them off as soon as they find the right person, that will cost a lot of money. Layoffs aren't cheap in the UK, even in the private sector.

When you pay a lot of money for a contractor, part of what you are paying for is the right to get rid of them easily.