Drake Batherson broke a tie at 17:51 of the third period, tapping in a pass from Brady Tkachuk on a power play from atop the crease for the 2-1 final.
Thomas Chabot also scored for the Senators (34-31-5), who ended a five-game skid (0-4-1) and moved six points behind the Panthers for the second wild card from the East.
Rickard Rakell tied it 1-1 at 14:39 of the third by sneaking a shot between Ferguson and the right post from the slot off a centering pass by Evgeni Malkin from the right corner.
"The way we were playing, it felt like once we got one, there would be more to come, hopefully," Penguins captain Sidney Crosby. "Get a big one there. Unfortunately, they come right back and get one. It's tough. We fought hard."
Chabot put Ottawa ahead 1-0 at 16:46 of the first period. After receiving a pass from Claude Giroux in the high slot, Chabot drove past Malkin to the bottom of the right face-off circle and scored on a wrist shot blocker side.
The Senators were outshot 32-13 in the first two periods. Ferguson made 18 saves in the first, including two key saves during a power play after
Ridly Greig
was called for tripping at 5:33. He closed the five-hole when Rakell tipped a pass from Malkin from just outside the crease at 6:10 and then made a sliding save to his left on a Jake Guentzel one-timer from the right circle at 6:59.
"It's just a perfect ending for a guy like that," Ottawa coach D.J. Smith said. "When he started his summer training, I would think that he didn't think he would be giving up one on that many shots in Pittsburgh at the end of the year. That just shows you, if you stay with it, you never know when your chance comes."