British overseas territories and crown dependencies would be included.
Speaking at the International Monetary Fund's (IMF) spring meeting, Osborne pushed for the international community to join the UK in making a list of globally recognised countries that are still allowing individuals to avoid paying tax.
"We could develop an international blacklist of tax havens and once you have that internationally agreed list – that would be the first time in our history as a world that you had an internationally agreed blacklist – then all sorts of countermeasures could be deployed against non-compliant regimes [and] tax havens that are on that blacklist," the Guardian reported him to have said on the sidelines of the IMF meetings on Friday.
The chancellor added that the Organisation for Economic Co-operation and Development should take on the task of creating the list.
Osborne said the UK would withhold funds that its citizens or companies tried to transfer to countries on the blacklist.
Such measures would act to incentivise countries on the list, or posed with the threat of being added to the list, to reform.
British overseas territories and crown dependencies are reforming and as such can avoid inclusion on the list.
"There’s no reason why they would be on the blacklist but there would be a clear threat and clear sanctions that if they didn’t do those things, so that would add to the pressure on all of these ultra low tax jurisdictions to put their houses in order," Osborne said.
Prime Minister David Cameron said last week that overseas territories and crown dependencies agreed to provide UK tax and law enforcement agencies with full access to company ownership details, after a campaign by the government.
The comments come after 11.5m confidential documents from one of the world's largest offshore law firms, Mossack Fonseca, were leaked and revealed how the world's rich and powerful have been using tax havens to hide their wealth.
Last year Sir Eric Pickles said that the government could end up forcing the UK's tax havens to reveal the owners behind offshore companies.
But, also last year, the crown dependencies and overseas territories had resisted pressure from the Prime Minister to adopt a public register.
Last week Osborne agreed to regulations that are designed to counter tax evasion.
In partnership with other European finance ministers, the Chancellor announced regulations that will lead to the automatic sharing of information about the owners behind shell companies and overseas trusts.