The claim comes after a week in which Livingstone was accused of making anti-Semitic comments and suspended from the Labour Party.
Khan earlier this week described the comments as "appalling and inexcusable", adding: "There must be no place for this in our party."
But Goldsmith this morning backed current Mayor of London Boris Johnson's claim that there is an "ideological continuum between the views of Ken Livingstone about Israel and the position of Jeremy Corbyn and indeed the views of their candidate for London Mayor Sadiq Khan".
Speaking to BBC Radio 4's Today this morning, Goldsmith said: “They're certainly part of the same movement within the Labour Party. Ken Livingstone has been a strong supporter of Sadiq Khan and vice-versa.
Read more: Sadiq Khan has been caught short
“Ken Livingstone hasn't suddenly decided to make outrageous comments – he's been making them for a very long time. On previous occasions he's done so, Sadiq Khan has rushed publicly to his defence.”
He added: “This is not a Ken Livingstone problem. This is a Labour Party problem. There is a streak running through the Labour Party – I don't think anyone now denies it, I don't think the Labour Party now denies it – a streak running through the party where anti-Semitism of the most aggressive form has become normalised.”
Goldmsith made similar claims in an interview with City A.M. to be published in full next week. He said: "There is no doubting that the party has become a home for anti-Semites, and not just mild anti-Semites – vicious anti-Semites."
On Radio 4, Goldsmith added: “I'm certainly not suggesting that Sadiq Khan is anti-Semitic – I've not made that claim.”