The result lifted PSG to seventh place on nine points from five games, four behind leaders Rennes, while Reims stayed 19th on one point.
Icardi fired the visitors ahead in the ninth minute with a crisp low shot into the far corner after an assist by Kylian Mbappe.
Reims goalkeeper Predrag Rajkovic then kept the home side afloat with several fine saves, with Neymar spurning a gilt-edged chance in the 43rd minute when he shot wide after beating three defenders with a solo run.
The hosts missed their best opportunity on the hour when substitute Boulaye Dia hit the post and they were punished barely a minute later as good work by Mbappe and Julian Draxler left Icardi with a simple finish.
Mbappe notched his second assist of the night, exchanging a one-two with Draxler before squaring the ball to Icardi for a tap-in from close range into an empty net.
Rajkovic produced another fine stop midway through the second half to deny Mbappe, who was guilty of poor finishing in the 82nd minute as he shot wide with only the goalkeeper to beat after a fast break.
Pablo Sarabia blazed a shot over the bar and Neymar's free kick sailed inches wide in the closing stages.