The result left second-placed Lille, who impressed in a 3-0 win at AC Milan in the Europa League on Thursday, on 19 points, five behind PSG who beat Rennes 3-0 at home on Saturday.
Brest moved up to 13th on 12 points.
The home side opened the scoring in the 15th minute when Pierre-Gabriel headed home Perraud's cross.
Perraud made it 2-0 four minutes later with a fine half volley as Lille struggled to cope with the hosts' early pressure.
Three minutes from the break, Brest added another as Cardona lobbed Mike Maignan with a header, only for Yilmaz to give Lille a lifeline on the stroke of halftime with a penalty after Paul Lasne fouled Jose Fonte.
Yilmaz further reduced the arrears 12 minutes into the second half with a deflected shot for his sixth Ligue 1 goal.
The striker had a golden opportunity to salvage a point for the visitors but his last-minute strike hit the crossbar.