Keith Andrews' Brentford drew 1-1 with Aston Villa before winning 4-2 on penalties in West London, while Palace were held to a 1-1 draw by second-tier Millwall prior to their 4-2 victory in the shoot-out.
Harvey Elliott, signed on deadline day on loan from Liverpool, scored Villa's first goal of the season on his full debut thanks to an error from Bees goalkeeper Hakon Valdimarsson.
But after Aaron Hickey scored a spectacular equaliser for Brentford, Valdimarsson made amends by saving from John McGinn and Matty Cash in the shoot-out.
Mikkel Damsgaard rolled in the winning penalty as Brentford advanced to add to the problems for Villa boss Unai Emery, whose side draw a blank in their first four Premier League matches.
At Selhurst Park, FA Cup holders Palace survived a scare after Camiel Neghli and Josh Coburn missed golden opportunities to put Millwall ahead.
Palace defender Chris Richards netted with a close-range strike in the 72nd minute when Millwall were unable to clear a corner.
The Lions equalised through Ryan Leonard's glancing header in stoppage time before Palace striker Jean-Philippe Mateta produced a candidate for miss of the season when he somehow blazed wide from six yards.
The South London derby turned Palace's way in the shoot-out as debutant goalkeeper Walter Benitez saved spot-kicks from Tristan Crama and Aidomo Emakhu.
Grimsby march on
Fourth-tier Grimsby reached the League Cup fourth round for the first time in 24 years with a 1-0 win at Championship strugglers Sheffield Wednesday.
Jaze Kabia headed the 49th-minute winner from a corner as Grimsby extended their run in the competition following a stunning penalty shoot-out victory over Manchester United in the previous round.
The Mariners, riding high in League Two after losing only one of their first eight matches, added Wednesday to their list of scalps with a composed display.
Wednesday fans were outnumbered in their own stadium as they continue to boycott certain games in protest at club owner Dejphon Chansiri, and over 6,000 travelling Grimsby fans revelled in the spotlight.