Just 31 per cent of British housebuilders said they are worried about a so-called Brexit, compared to 49 per cent who are not worried, according to research by McBains Cooper.
In London the proportion who are concerned is slightly higher, at 38 per cent.
The housing crisis has become a major political issue in London in recently with an estimated 50,000 new homes needed to be built every year to keep up with population growth.
A lack of supply in the capital has been attributed to a number of factors such as strict planning permission and green belt rules, a lack of skills in the construction sector, a smaller number of construction firms, and housing companies restricting land use to push up prices.
Michael Thirkettle, chief executive of McBains Cooper, said:
London firms may be among those more worried about Brexit because, more than the rest of the country, the capital is in the middle of building and infrastructure boom which requires skilled workers from the rest of the EU. In this scenario, a Brexit could lead to a fall in house-building in London, hitting the prospects of those wanting to get on the property ladder.
There have recently been a raft of warnings over a potential slowdown in the construction sector if the UK does leave the EU.
Last week data from IHS Global Insight found output from the UK construction industry in February dropped by an estimated 0.3 per cent compared with January, which itself was down 0.4 per cent on December.
IHS blamed uncertainty over the Brexit vote for the weakened figures.
Accounting giant KPMG found two-thirds of the 25 global property investors, with combined portfolios of more than $400bn (£281bn), would trim investment in UK property if the country opts to leave the trading bloc.
Meanwhile, the construction industry was found to be expanding at its slowest pace since 2013, according to the closely-watched Markit/CIPS purchasing managers' index (PMI) out earlier this month.
The survey, which measures business activity across the construction sector, came in at 54.2 for March – on an index where a score above 50 indicates expansion – though Markit said this was sluggish compared to the last few years.
Housebuilders have often recruited from EU countries because of skills shortages in the UK and last week last week deputy London mayor Stephen Greenhalgh warned that London risks losing the large numbers of European construction workers it needs to build new homes if Britain leaves the EU.
The UK’s construction industry will need an extra 80,000 workers in 2016 alone, according to the Construction Industry Training Board (CITB).
Last month UBS Wealth Management warned that house-builders would suffer badly from the effects of a Brexit, due to the knock on effect to consumer sentiment and credit conditions.
The companies were asked whether they were worried a vote to leave on 23 June would make it more difficult for them to recruit workers from EU countries.
Regions showing the least concern were the West Midlands, with 61 per cent expressing concern, Eastern England, with 60 per cent, and East Midlands, with 56 per cent.