The victory left them four points behind Group G leaders the Netherlands and two points behind Norway with two matches left to play.
Hosts Latvia took the lead in the 70th minute when Turkey's Merih Demiral fired into his own net while attempting to clear a cross from Roberts Savalnieks. Serdar Dursun equalised for Turkey in the 75th minute when he headed in a cross from Cengiz Under.
Turkey dominated possession throughout, with Latvia repeatedly looking to hit back on the counter attack. The hosts looked to have held on until Yilmaz was brought down in the penalty area in the ninth minute of injury time.
After consulting VAR, the referee pointed to the spot and Yilmaz fired the ball into the net to seal victory.