Conservative cabinet minister Michael Fallon has said some towns on the east coast of England feel they are "under siege" and "swamped" by migrant workers.
The defence secretary was responding to German Chancellor Angela Merkel's rejection of any substantive change to Britain's EU membership to limit the free movement of people.
“Germany will not tamper with the fundamental principles of free movement in the EU,” Merkel said.
This sets the German leader on a collision course with David Cameron should the Tories win the general election next year, with the PM pledging to make immigration a "red line" in the UK's renegotiations of its EU membership terms.
Under the 1957 Treaty of Rome, the free movement of people was enshrined as one of the four fundamental freedoms of the EU and is seen as non-negotiable by many continental leaders.
Speaking to Sky News' Murnaghan programme, Fallon said:
The Germans haven't seen our proposal yet, and we haven't seen our proposal yet.
That is still being worked on at the moment to see what we can do to prevent whole towns and communities being swamped by huge numbers of migrant workers.
The government is hoping to work with sympathetic EU member states to limit migration within the 28-member bloc. Fallon's comments come three days after a ComRes survey showed the Conservatives are set to lose the Rochester and Strood by-election to Ukip defector Mark Reckless.
Tensions are already running high between Cameron and the European Commission after Britain was ordered to pay an extra €2.1bn (£1.7bn) to the EU. The UK was told it must cough up the money by 1 December. The extra contribution is thanks to the strength of UK employment and growth in gross national income.
Cameron said on Friday:
It’s a €2bn bill. It gets presented with a month to go. That is not an acceptable way to behave, and it’s not an acceptable sum of money.
I am not paying that bill on December 1. If people think I am, they have got another think coming.
While Britain is on course to hand over €2.1bn to Brussels, France and Germany are are set to be the winners of this year's EU budget, receiving rebates of €1bn and €779m respectively.