Crosby then made it 2-0 at 10:39 when he deflected a point shot from Pierre-Olivier Joseph. The Rangers used a coach's challenge for offside, but the call on the ice was upheld after video review.
"It's remarkable," Penguins coach Mike Sullivan said. "To perform at an elite level for as long as he has is remarkable, and I think his body of work speaks for itself, every milestone that he gets. I wasn't aware of that one. And so it doesn't surprise me. I just think all of these milestones that he's reaching at this point are just more evidence that he's one of the greatest players of all time, not just of his generation, but he's in elite company in all these categories."
Nedeljkovic made 21 saves in the first two periods.
"We knew it was going to be a tough test, even if we were 100 percent (healthy)," he said. "That's a really good hockey team over there. They had a lot to play for tonight. ... I mean, so do we though. We're still fighting for points and the guys battled."
Mika Zibanejad appeared to score for the Rangers at 3:09 of the second period, but the goal was disallowed because Chris Kreider interfered with Nedeljkovic.
"Obviously, in the first period, we get down 2-0," Zibanejad. "Not playing a good period, good hockey in the first. I think we cleaned some stuff up in the second. The push in the third comes a little bit too late, but overall, not our best game."
Emil Bemstrom scored on a breakaway after a turnover at the Penguins' blue line to extend the lead to 3-0 at 9:51 of the third period.
Kakko got the Rangers to within 3-1 at 10:42 with a one-timer from the top of the right circle. His shot was stopped by Nedeljkovic, but a video review determined that the puck, which was in Nedejkovic's glove, completely crossed the goal line.
Roslovic scored to make it 3-2 with 3:07 left, but Rust scored an empty-net goal to make it 4-2 with 2:39 remaining.
"It seemed like we were kind of chasing right from the start and you never want to do that,” Rangers defenseman Ryan Lindgren said. “We started to get a couple in the third, kind of feel the momentum, the building got a little more lively. We had a chance to tie it but just didn't happen."
Crosby scored at 19:15 for the 5-2 final. He has 13 points in a five-game point streak (four goals, nine assists)
"Yeah, it's huge. I mean, I think we know the circumstances, so nobody's going to feel sorry for us and you got to find ways in times like these where you have last-minute guys in and out," Crosby said. "Whatever we got to do to get wins. So, that was a great, great team win and something that hopefully we can build off of."
NOTES: Panarin had an assist on Kakko's goal to pass Mark Messier for sole possession of the third-most points in a single season in Rangers history. He has 108 points (44 goals, 64 assists) in 75 games this season. Jaromir Jagr is first with 123 points in 2005-06. … Crosby is three assists from 1,000 in his career. … Nedeljkovic is 4-0-2 with a 2.45 goals-against average and .917 save percentage in his past seven games.