Zack MacEwen redirected Nick Seeler's shot at 10:17 of the second period to give Philadelphia a 3-2 lead.
Moments later, Hayes skated in on a breakaway but lost the puck. He retrieved it off the end boards and sent it toward the front of the net, where Farabee knocked it in to make it 4-2 at 11:29.
"We started slow, [but I] thought we picked it up parts of the second period and then we just hung in there," Flyers coach John Tortorella said.
Montour scored a power-play goal 36 seconds into the third for the 4-3 final. It was the Panthers' first goal with the man-advantage in six games, ending an 0-for-26 skid.
"We got some good looks, some clean looks," Montour said. "I'd like us to be a little quicker still. I think we're trying to make a little too perfect plays. We've got some good shooters. It was nice to get one there early in the third, but still a lot to work on there."