Rail commuters will be able to use an Oyster-style pay-as-you-go smart card to travel anywhere in the country by the end of 2018.
Writing in the Telegraph, Grayling said: "We need smart ticketing and particularly pay as you go smart cards on commuter routes all around the country. I want that to be happening in the next couple of years."
London commuters can pay as they go with Oyster cards so there's no need for paper tickets ahead of travel and the transport secretary wants this rolled out across the nation.
Oyster cards were introduced back in 2003 and by 2012 over 43m had been issued, with more than 80 per cent of journeys on public transport in London made using them.
The smart cards will be brought in alongside changes to the compensation process which will see travellers paid out after a train is delayed for 15 minutes or more.
It comes amid the biggest revamp of the rail industry for two decades, as Grayling pledged to "put passengers first" to establish a "golden era for rail". Under these plans, Network Rail won't have such a monopoly over the nation's train tracks.
The winners of new rail franchises will be given a say over maintaining tracks in their area, while Network Rail will be stopped from building a new line line between Oxford and Cambridge.
A new East West Rail organisation will lead on the completion of the lines between the two cities before the franchise is handed to a new integrated operation, while new franchises due to be let on Southeastern and the East Midlands will follow suit.
Grayling said: "Train companies take the blame for the problems of Network Rail and Network Rail has little or no contact with passengers, and so has had little reason to focus on the best possible customer service."