Speaking in Parliament, Mr Whittingdale told MPs that charging all viewers the same annual fee forced those on lower incomes to pay more for the BBC's television services.
It is his most critical comment on the BBC as a minister, since joining the government after May's General Election.
However, as a backbencher, Mr Whittingdale was a regular critic of the licence fee - calling it "worst than the poll tax".
The move may indicate that the minister wants to scrap the £145.50 annual licence fee in favour of another funding system.
In response to a question asked by Chris Bryant, Mr Whittingdale responded:
As for the licence fee, he’ll have to await our conclusions. But I would say I very much agree with him when he observed that elements of the licence fee are regressive because everyone has to pay it and so it falls as a greater percentage of the income on the poorest people.