Goals either side of the break from Paul Ifill and Sean Scannell ensured victory for Neil Warnock's men who now travel to Vicarage Road a week on Saturday.
Max Gradel scored a late consolation for the League One side with the final kick of the match.
After a truly awful start to the game, Palace created the first opening, Alan Lee blazing over from 14 yards after a neat Victor Moses pass in the tenth minute.
The former Ipswich striker then headed narrowly over from Danny Butterfield's pinpoint cross as the home side began to take control of the game.
Both sides were dealt injury blows with defender Aleksandar Tunchev and Palace's Moses being forced off.
Despite enjoying the majority of possession, Palace failed to create any real clear-cut opportunities.
However, with five minutes left in the opening period the Eagles took the lead when Ifill's shot from the edge of the box took a wicked deflection off Andy King before rolling past the wrong-footed Dave Martin.
Palace made sure of their passage to the fourth round when substitute Scannell slotted his shot past Martin ten minutes into the second half after a superb Lee pass.
Jack Hobbs missed a half-chance to pull one back for Nigel Pearson's men soon after - but his header from Matt Oakley's corner sailed well over.
But Palace went on to dominate proceedings and missed good chances through Ifill and Butterfield to increase their lead before Gradel slotted home in the final minute of stoppage time.