Panarin tied it 3-3 at 11:04 on a quick wrist shot after Trocheck won a face-off against Erik Haula in the right face-off circle.
"I just see the puck and it's probably 26 years playing in a hockey rink ... so I know exactly where the net is (on the shot)," Panarin said. "I just shot to that (top short) side. [Trocheck] did an unbelievable battle for me and I just rip the puck into the net."
Vesey gave New York a 4-3 lead at 17:11 with his second of the game, scoring on a rebound with the backhand.
Blake Wheeler scored into an empty net at 18:27 for the 5-3 final.
Shesterkin stopped 11 of 12 shots in the third in his first start since Nov. 2.
"Every minute that went by, he looked better and better," Laviolette said. "That's expected since he was out for a little bit. We needed him and I thought he really stepped up."
Panarin gave the Rangers a 1-0 lead on the power play at 8:37 of the first period with a sharp-angled shot from the left circle.
Hughes tied it 1-1 at 9:25 from the top of the right circle.
Vesey put New York ahead 2-1 at 14:11 when he knocked in a rebound.
Ondrej Palat then tied it 2-2 on the power play at 19:12 with his first goal of the season, scoring on a snap shot from the high slot.
Haula gave the Devils a 3-2 lead at 1:17 of the third when he got to a loose puck before scoring on a backhand from the slot.
"The game was right there for us," New Jersey coach Lindy Ruff said. "We've been in these games where after 20 or 40, we haven't been able to finish. Our last game we did (a 5-2 win at the Pittsburgh Penguins on Thursday). I think where we capitalized last game were the opportunities to put the game away. We didn't do it tonight."
NOTES: The Rangers have outscored the opposition 18-9 in the third period this season. ... New York became the third team this season with at least four third-period comeback wins, joining the Anaheim Ducks (six) and Toronto Maple Leafs (four). … The Rangers have their first point streak of at least 11 games since a 13-game run (11-0-2) from Oct. 18-Nov. 15, 2015. … Devils forward Tomas Nosek (upper body) exited the game after a hit from Rangers defenseman Jacob Trouba in front of the New York net at 13:43 of the first, leaving the Devils with 10 forwards (New Jersey dressed 11 forwards and seven defensemen). Ruff had no update on Nosek. ... Jack Hughes had 14 shot attempts, eight on goal, in 22:40 of ice time. ... Devils defenseman Kevin Bahl had a game-high eight hits. … It was the first game between the teams since New Jersey’s seven-game victory in the Eastern Conference First Round last season.