Without a point from their opening two group games, Celtic took the lead on the stroke of half-time when Forrest converted a penalty after Stefano Denswil was penalized for a foul on Anthony Stokes.
The home side doubled their advantage on 53 minutes when Georgios Samaras’s shot was blocked by Joel Veltman into the path of Kayal, whose placed finish from 18 yards deflected in.
Celtic had to ride out late pressure after substitute Nir Biton was shown a straight red card for a late challenge on Thulani Serero.
Lasse Schone fired in from the edge of the box deep into second-half stoppage time, but the Bhoys hung on.
Elsewhere in their group, Lionel Messi cancelled out Robinho’s opener as AC Milan drew 1-1 with Barcelona.
Chelsea’s 3-0 win in Schalke moved them to the top of Group E, with Basel held to a 1-1 draw by Steaua Bucharest thanks to an 88th-minute equaliser from Leandro Tatu for the Romanians after Marcelo Diaz had given the visitors the lead.
Napoli won 2-1 at Marseille in Group F to draw level on points with Arsenal, who were beaten by the same score at home to Borussia Dortmund.
Meanwhile in Group G, Diego Costa scored twice to help Atletico Madrid win 3-0 away to Austria Vienna and Zenit Saint Petersburg won 1-0 at 10-man FC Porto, who had Hector Herrera sent off after incurring two bookings in the first six minutes.