The move could scupper the European Commission's plans - it had hoped the new setup would be in place by the end of this year, an unusually tight timeframe.
Meanwhile European Central Bank boss Mario Draghi said he will go to the German parliament to explain how his bond buying plan will work, in an effort to calm opposition from the Eurozone's biggest economy.
The Swedish government objected to plans for the ECB to take supervisory powers over the country's largest bank, Nordea.
Sweden is not in the Eurozone and is not expected to take part in the banking union, which would give the ECB regulatory powers over all banks in the currency area. But as Nordea operates in Finland, which uses the euro, it will face regulation.
"As long as it remains that we are not participating in the banking union, it is unreasonable that we would be subject to the ECB's inspection, since we do not have a voting right," Anders Borg said according to the Finnish national broadcaster.
"It is totally out of the question."
The German government has also objected to the EU plans, arguing that only the largest banks should be subject to ECB rule, as the central bank lacks the resources to regulate all 6,000 banks in the Eurozone.
Some Germans have also objected to ECB plans to buy government bonds to lower borrowing costs for troubled Eurozone states, and so Mario Draghi will give evidence to the Bundestag in an effort to overcome that opposition.
If the new proposals do indeed progress as planned, analysts believe they could form the basis of a turning point for the Eurozone in the crisis.
The announcements are "a sign that activity could be stabilising and recent policy efforts paying off," said Barclays' Fabrice Montagne. Indeed, he argues "with the recent market improvement, the risk is that Spain considers it no longer necessary and delays the decision" to ask for help from the bailout fund, which was approved by Germany's top court last week.
However if the Spanish government did not ask for help, "this could give birth to renewed tensions," said Montagne.