Blackrock president Rob Kapito noted that multinationals with a UK presence would now be carefully considering their options relating to how much of their business they kept in the country.
"I don't think there's any firm, any good firm, that has not already started looking at real estate in different areas outside the UK in case they have to move larger operations," Kapito told the BBC's Today programme.
Also speaking to the Today programme, Morgan Stanley president Colm Kelleher cautioned that, although London is likely to retain its standing as a financial capital, some firms will probably need to cart certain parts of their offering to elsewhere in Europe following the Brexit decision.
"I am convinced that London will retain its reputation and prestige as a global financial services centre, but clearly some size of our businesses will have to be moved out of London and into Europe with the absence of any passporting agreement," said Kelleher.
"It's very hard to ascertain what that means at the moment. I do think generally though that capital markets in Europe will shrink as a result of this."
Kelleher's and Kapito's comments come the day after figures from the Financial Conduct Authority revealed over 13,000 firms relied on passporting, with 5,476 of those firms being based in the UK and using the rights to carry out business in other parts of the EEA.
Speaking at a conference in Frankfurt last month, Deutsche Bank boss John Cryan warned that, although London was likely to stay a key centre for banking for at least the next decade, it might end up looking a very different place to how we know it today, as the City's relationship with the rest of Europe continues to morph.