City -- who went behind by two goals in the match -- dropped points for the first time this season, leaving Arsenal as the only Premier League side to have won their opening three matches.
After the match, City boss Pep Guardiola said the result was proof of just how hard it is to compete in the Premier League week in and week out.
"The Premier League is so difficult for everyone, and we know that we have to live this kind of experience to realise how amazing we have done the last five years and how difficult will be if want to continue," Guardiola said.
But Almiron did deservedly draw the game level in the 28th minute. His improvised finish from Saint-Maximin's cross was initially ruled out for offside but a VAR check overturned the decision and granted Newcastle their leveller.
Newcastle had the better of their opponents in the first half and they were rewarded six minutes before the break. More probing by Saint-Maximin allowed the France winger to play in Callum Wilson, who cut inside and fired past Ederson with the outside of his boot.
Erling Haaland, who had been subdued in the first half, struck the post seven minutes after the interval.
However it was Newcastle who struck next and extended their lead against Pep Guardiola's side, Kieran Trippier sending a free-kick from 25 yards soaring into the top corner.
St James' Park was soon silenced, though, as two quickfire goals got City back level by the 64th minute. The prolific Haaland sparked the comeback, latching onto Rodri's flick to slam home a volley from close range with 30 minutes left to play. Three minutes later, Kevin De Bruyne's smart ball in behind found Bernardo Silva, who slid the ball past Nick Pope.
"After 3-1 down, to make this comeback and the chances we created in the second half ... we had the much clearer chances," Guardiola said. "Newcastle are becoming one of the toughest opponents for sure, their quality is very strong."
Both sides pushed for a winner in the frantic affair, but neither could find the breakthrough -- Newcastle were boosted by keeping goalscorer Trippier on the pitch, who was shown a red card by referee Jarred Gillett until a VAR check turned the verdict to yellow.
Guardiola's opposite number Eddie Howe came close to pulling off a shock result, but City, who have not lost away since the opening day of last season, were able to salvage a point.
"It was an incredible game, a rollercoaster of emotions, we went 1-0 down which is the worst possible start against this team because then you have to chase the game, but I think the players responded brilliantly," Howe said.
"We wanted to put them under pressure and leave here feeling they had been in a game. Certainly we were in their faces and we committed very well."
Information from Reuters was used in this report.