Addressing the Labour fringe for the first time in more than seven years, Miliband said it was "nonsense" for a political party to seek to represent the 48 per cent of voters who backed EU membership.
"Of course we have got to speak to Remainers but the Labour party has got to speak for people who voted Leave as much if not more, because they feel that they're not being heard in politics," he said, adding the party must accept the result in order to push the government to secure the best possible Brexit deal.
And the former party leader also called on his fellow MPs to "unite behind" the re-selected Jeremy Corbyn as part of that process.
Miliband added that he expected to remain on the backbenches, rather than taking up a post in Corbyn's shadow cabinet, but refused to rule out a top job, stating "those matters are for Jeremy".
His comments were echoed by Streatham MP Chuka Umunna, who said it would be "incredibly patronising" to view Leave voters as "brainwashed by the Daily Mail", and Treasury Select Committee member Rachel Reeves, both of whom said Labour must also engage in a debate around free movement.
And despite claims to the contrary from European governments, Reeves added this would not necessarily preclude Single Market membership.
"It's not going to be in the interests of French businesses to have tariffs on things that they are trying to sell into the UK," Reeves said, adding that European politicians would face growing pressure from their local industries to reflect this.
"There are more exporters from the EU who would lose out from tariff than there would be exporters from the UK," she said, adding: "I think there are ways of appealing to the European Commission and member-states to do a deal that is in all of our interests."