Lars Eller, Jonathan Gruden and Jeff Carter scored, and Jarry made 16 saves for the Penguins (27-23-8), who have lost two straight after winning three in a row.
"We have to keep fighting," Pittsburgh forward Drew O'Connor said. "It's a tough way to end that game. We know how important each game is, and we need a lot of them right now. To lose that one when we're in control hurts a lot, but we have another one tomorrow (at the Edmonton Oilers) we have to get ready for."
Prior to the game, the Flames held a ceremony for the retirement of Miikka Kiprusoff's No. 34 jersey. The goaltender played for Calgary from 2003-2013 and is the franchise's all-time leader in wins (305), shutouts (41), goals against average (2.46), save percentage (.913) and games played by a goalie (576).
"Obviously, one to get the special night," Coleman said. "Everybody's really happy for 'Kipper' and his family, and it's really cool to see how much support he received tonight and how much he meant to this city. To honor him with a win is special.
"On the business side of things, we're chasing teams that are winning games. We can't slip up and we talked about that going into the third and how important these points were. Give the guys credit on this team. They dug in and found a way."
Kadri cut the Penguins' lead to 3-2 at 10:09 of the third period with a power-play goal just as a 4-on-4 expired. He stickhandled around Kris Letang before cutting across the crease and tucking the puck around Jarry's outstretched right pad.
"It's go time," Kadri said. "I think that's what everyone should be saying. The third period is an opportunity, really, whatever the situation is ... crawl back, build on a lead, tie the game ... whatever the situation is you always want to finish strong, and that's what we've been doing this year."