What can steam trains teach us about progress in AI?
Contrary to popular belief, Stephenson’s Rocket did not hail the dawn of the steam age. Likewise, AI has actually been in development for decades, but progress could now begin to move faster than steam trains ever could, says Paul Ormerod
The economics Nobel Laureate Robert Solow famously stated in 1987: “you can see the computer age everywhere but in the productivity statistics”.
Although 40 years ago seems more or less the Dark Ages now in terms of technology, in the years leading up to Solow’s remark there had been major developments in computing. In particular, the PC had spread widely, ending the dependency on large mainframe computers for analytical work and enabling products like Word and Wordstar to replace the office typing pool.
A similar phenomenon seems to be taking place with AI. It is hardly possible to open a newspaper or open a media link without seeing some AI-related story. But we have not yet seen a surge in productivity from this technology.
One reason is simply that what we might term “modern” AI is very much in its infancy. There has been very little time for its effects to come through.
Back 200 years ago, the revolutionary technology of the railways emerged. These opening decades of the 19th century give us a useful insight into the evolution of AI in our own times.
The dawn of the steam age
In 1829, a famous set of trials was conducted at Rainhill to compare the performance of the steam locomotives which had been developed. Stephenson’s Rocket eclipsed the field.
But there were not just contemporary rivals, there was already a history of steam locomotion.
In 1804, for example, a steam engine developed by Richard Trevithick had hauled coal on rails in the rapidly developing South Wales coalfields. Its maximum speed was 4mph.
In 1825, the Stockton and Darlington railway was opened with Locomotion no 1. It hauled both freight and, for the first time, passengers over the 26 mile route. Its top speed was 15mph, though in practice it averaged 8mph.
The Rocket was far superior. It was in fact the first truly modern steam engine. Its then revolutionary engineering principles lasted as the basis for all engines until the very end of steam in the late 20th century.
Its very success led to it becoming obsolete, and by 1840 locomotives had become seven or eight times more powerful.
AI has a similar history. Neural networks underpin the modern large language models. The idea can be traced back to the 1940s, but neural nets did not really become operational until the publication of the technique of backpropagation in 1986 by David Rumelhart and Geoffrey Hinton.
It was an exciting time. For example, in the early 1990s my small company led, with the computer science department at UCL, a large consortium financed by government and large firms to develop practical applications of neural network technology.
The outcome was disappointing. It was rather like Trevithick’s engine. Neural nets seemed to be able to do a few new things, but they were not obviously superior to well established tools of analysis. Everyone had the same experience, and enthusiasm waned.
Interest revived gradually with a paper published by Yoshua Bengio of Montreal in 2003 on a “Neural Probabilistic Language Model”. Further boosts were provided by papers in the 2010s by Bengio and Geoffrey Hinton. But by 2020 we had probably reached a stage equivalent to Locomotion no 1. We could certainly do valuable and useful things which had not previously been possible.
But the real “Rocket moment” for AI came with the publication of ChatGPT at the end of 2022. This was indeed a revolutionary advance.
Just like Rocket, these first versions are obsolete. Progress has been so phenomenal that the current worry is that AI may lead to the extinction of humanity.
If it does, we will no longer need to worry about productivity. But in the meantime, truly modern AI will give a massive boost over the coming decades.
Paul Ormerod is an Honorary Professor at the Alliance Business School at the University of Manchester. You can follow him on Instagram @profpaulormerod