King's last day will be 4 March, according to Barclays.
Commenting on his decision to retire, King said: "After nearly 25 years in my career, and having now seen the Investment Bank through a period of extraordinary change and on to a solid footing for the future, I feel the moment is right for me to pass the baton.
"I am so proud of what we have achieved together at Barclays through what has been a hugely challenging period, and I shall leave the firm with fond memories of great clients and great colleagues."
King is understood to have come out on top in an internal struggle last year over the future of Barclays's investment banking business, having been at loggerheads with the bank's former chief executive Antony Jenkins.
Jenkins was ousted from his job last July, and former JP Morgan investment banker Jes Staley took over the top role at Barclays in December.
Staley said today he was "sad" to see King leave Barclays, but added: "I understand and respect Tom’s decision to retire.
"Tom’s work in particular in leading the change in our investment bank strategy before and since May of 2014 has been very important to ensuring Barclays’ future success. Tom’s trademark has always been his focus on the needs of clients and colleagues, and we are grateful to him for his stewardship."
King, who is based in New York and has worked for Barclays since 2009, has long been expected to step down from his post at the top of the bank's investment banking business.
King's departure follows hot on the heels of Jonathan Moulds, who left his role as chief operating officer just over one year after taking the job. Moulds will be replaced by Paul Compton, who like Staley, comes from JP Morgan.