The report, which was commissioned by TheCityUK and carried out by PwC, discovered that up to 100,000 jobs could be lost from the financial services sector by 2020, while the industry's contribution to the economy could also be lowered by £12bn.
By 2030, the study predict that, while the sector would be showing signs of recovery, there would still be up to 30,000 fewer jobs and the sector could be creating less value than if Britain had stayed in the EU.
The report found that a Brexit would lead to a period of uncertainty while the terms of leaving were being negotiated, which could cause financing costs to rocket and many firms to rethink their investment strategy.
"Major firms from across the world come to London to access Europe’s single market, bringing with them jobs and investment," said Chris Cummings, chief executive of TheCityUK. "While Brexit may not be ruinous for the UK economy, it does risk damaging the UK-based financial services sector, particularly over the short-term, delaying investment decisions and reducing activity. It also threatens the overall competitiveness of the UK as a place to do business."
Andrew Sentance, senior economic adviser at PwC, added: "In all the scenarios we have examined, the financial services industries will be less successful for at least a decade if the UK leaves the EU."
Read more: EU business leaders: Don't go for Brexit
However, Matthew Elliott, chief executive of Vote Leave, said: "This talk of a mass exodus from the City of London is completely unrealistic. London was one of the greatest financial centres in the world before we joined the EU and will still be after we vote leave. People come to London because it is a great place to do business – it is a shame TheCityUK are talking Britain down. They were wrong then and they are wrong now."
Calling the suggestions from the report "drivel", David Buik, market commentator at Panmure Gordon & Co, added: "London is the centre of the timezone. London is the largest and best financial centre. London is the world's leading centre for the law and accountancy. No meaningful jobs will be lost as a result of Brexit!"
Last month, a report by PwC for the Confederation of British Industry (CBI) discovered that leaving the EU could cost the country as a whole up to 950,000 jobs by 2020, although it also noted that lower levels of migration would be a big contributor this figure.