Saudi Aramco, a petroleum giant comprising the oil wealth of Saudi Arabia, is planning a listing that could value the company at $2 trillion.
London and New York are the frontrunners for the listing, but a float on the London Stock Exchange would require a modification of listing rules because of the size of Saudi Aramco.
Critics of the IPO argue that the City should not relax its rules to allow Saudi Aramco to come to London, but Grimstone said today that there was no evidence that the firm was run badly.
Speaking at the Conservative party conference, Grimstone said: "I think that company should be listed here.
"No one would say that Saudi Aramco is not well-run...If a company is well-run we need to encourage that company to come to London."
If London wants to compete in the global marketplace, it must be able to attract firms such as Saudi Aramco to the Square Mile, he said.
Politicians are scrutinising whether the Financial Conduct Authority (FCA) should change its rules.
Chairs of the influential Treasury and Business, Energy and Industrial Strategy select committees, Nicky Morgan and Rachel Reeves, have written to the FCA demanding more information on the regulator's plans.
On executive pay
The government's proposed reforms on corporate governance have not impressed Grimstone, who said today that they were "incrementalism rather than radicalism" and would not be enough to fix public perceptions of business.
"We would like to see the government taking much stronger action," Grimstone said.
"I have no problem with Gary Lineker making more than two million a year if he hits the ball in the back of the net...What I am absolutely against is high, inappropriate pay."
Business minister Margot James responded by saying that radical government interventions were risky, and that corporate governance in the UK had improved vastly since the Thatcher era.
However, James suggested the government was ready to take more action if required, saying: "That is not to say we won't go further if that does not hit the mark."