Watford manager Walter Mazzarri suffered an embarrassing early blow to his tenure as third-tier side Gillingham caused an upset in the second round of the EFL Cup.
Hornets striker Odion Ighalo looked to have put the hosts on track for victory but Mark Byrne equalised in the 82nd minute and Bradley Dack made it 2-1 in the first half of extra time.
West Brom were the other Premier League team to crash out, also against the third tier opposition, the Baggies losing 4-3 on penalties to Northampton after a 2-2 draw at Sixfields.
Forward Saido Berahino and midfielder James Morrison missed spot-kicks for the top-flight side.
England striker Daniel Sturridge scored his first goals of the season as Liverpool overwhelmed Championship side Burton Albion 5-0.
Sturridge came off the bench to net twice after strikes from forwards Divock Origi and Roberto Firmino and an own goal from Burton’s Tom Naylor had put the Reds firmly in control.
Crystal Palace handed £27m record signing Christian Benteke a debut as they eased past League Two outfit Blackpool 2-0 at Selhurst Park.
Defender Scott Dann fired the hosts into a 25th-minute lead and striker Connor Wickham, who replaced Benteke at half-time, wrapped up the win two minutes after the interval.
Everton substitute Arouna Kone struck twice in the last 10 minutes as the Toffees beat fourth-tier Yeovil Town 4-0 at Goodison Park.
Winger Aaron Lennon and midfielder Ross Barkley scored the other goals while summer signings Yannick Bolasie and Ashley Williams were handed their first starts.
Former England striker Peter Crouch plundered a hat-trick as Stoke ran out 4-0 winners at League Two Stevenage. Defender Phil Bardsley netted the visitors’ other goal from 30 yards.
Hull continued their 100 per cent start to the season, despite a dearth of players, with a 3-1 victory at Exeter. Striker Adama Diomande scored twice and Robert Snodgrass grabbed the other for the Tigers.
Celtic will be in Thursday’s Champions League group stage draw after they progressed 5-4 on aggregate, despite losing their play-off second leg 2-0 at Hapoel Beer Sheva.
Irish club Dundalk were knocked out 2-1 on aggregate after drawing 1-1 with 10-man Legia Warsaw in Poland.