Sweden still top the group on 15 points but second-placed Spain can overtake them with victory in Greece later on Thursday. Spain host Sweden in the final qualifier on Sunday.
Zlatan Ibrahimovic returned to the Sweden side after a knee injury that kept him out of Euro 2020, but not even the presence of their record goalscorer could not save them from an embarrassing defeat.
The Swedes racked up 15 efforts on goal, including gilt-edged chances for Alexander Isak and Viktor Claesson, but it was the home side who took the lead through Khvicha Kvaratskhelia in the 61st minute and the winger added a second 16 minutes later to secure a famous victory.
The Swedes continued to create chances but Kvaratskhelia struck again, blocking a clearance from Victor Lindelof and racing through to fire home in the 77th minute.
"I'm disappointed, very disappointed. We should win that game. We had so many chances, we just had to score ... we should win the game, we're much better than them, no doubt about it," Ibrahimovic told broadcaster TV4.
Sweden coach Janne Andersson, who led his side to a dour scoreless draw against Spain in the group stages of Euro 2020 in June, rued the missed opportunities.
"It's an old truth in football - if you don't score from your chances, sooner or later your opponent will get one. They got it and they took it, it bounced their way, this was one of those days," Andersson said.