Strange. In France you don't have to contribute VAT for the services you invoiced abroad (well, outside the EU. Inside the EU is another matter). You don't receive it, you don't pay it.
It seems absurd to make a company pay back a VAT amount that it has not perceived.
I know, but I really went over with a comb on the HRMC website, asked them, asked all around and everyone agreed. These transactions are 'zero rated' -- therefore, you need to pay 1[34].5% VAT back... Stupid, but the rule is clear.
Incredibly bizarre that as a small company, you try to 'export' and thus bring cash back into the economy, and are in fact actively prevented from doing so when on that scheme.
It seems absurd to make a company pay back a VAT amount that it has not perceived.