One of the five ministers to go against the government's official position of staying in a reformed EU, Gove said the European Court of Justice (ECJ) could just throw out a number of the changes.
He said that while Cameron had achieved an "international law declaration", without treaty change the deal the Prime Minister struck could be subject to legal challenges by the ECJ.
"The facts are that the ECJ is not bound by this agreement until treaties are changed and we don't know when that will be," he told the BBC.
Gove added that while the 28-nation states agreed in good faith to the agreement, "I do think it's important that people also realise that the ECJ stands above every nation state, and ultimately it will decide on the basis of the treaties and this deal is not yet in the treaties".
But European Council President Donald Tusk disagreed, saying the package of reforms negotiated by Cameron can't be reversed by European judges. Speaking in Brussels, Tusk said the deal is "legally binding and irreversible.
"[It] cannot be annulled by the ECJ," Tusk said.
Attorney General Jeremy Wright also said that Gove was not correct: "It has legal effect from the point the UK says it intends to remain in the EU, and the European Court must take it into account. The job of the European Court is to interpret the agreements between the 28 nation states of the EU. This is one of those agreements, with equivalent legal force to other agreements such as treaties.
"That is not just my opinion – it is the opinion of this government's lawyers, lawyers for the EU, and, I suspect, the majority of lawyers in this country," he added.
Lawyers could be expected to argue the case both ways, as the reforms are not formally ratified by treaty, but are intended by the 28-member states to be stuck by. The ECJ is not bound by international agreements between member states.
The justice secretary's comments come after the Prime Minister said on Monday to the House of Commons that the reforms would be legally binding in international law: "They cannot be unpicked without the agreement of Britain and every other EU country."
But Gove's comments were backed up by Conservative backbencher Dominic Raab MP, who said:
The EU's own legal advice makes clear the UK deal is based on vague assurances. Ultimately, it is crystal clear the Luxembourg Court will have the last word as to what sticks, and it is free to not to enforce the British deal.