How much will Brexit cost?

The Office for Budget Responsibility says the vote to leave the EU will add £58.7bn to government borrowing by 2020-21. Is that too gloomy?