Jack Hughes had two goals and an assist for the Devils (25-12-3), who ended an eight-game home losing streak (0-7-1). Vitek Vanecek made 31 saves.
"I'm extremely proud of our road record, but the home record (11-10-2), I mean, imagine if you pour that 14-2-1 road record into this building, the noise that was in here today," New Jersey coach Lindy Ruff said. "Just the atmosphere, you know, I think it pushes the players, too. As soon as we got the second one, I thought the building was fantastic."
Jimmy Vesey and Chris Kreider scored, and Igor Shesterkin made 39 saves for the Rangers (22-12-7), who had won three in a row.
"I thought we were pretty good for two periods, but we've got to get the job done," Vesey said. "We've got to finish the job. It's one game out of 82. ... We'll take the point."
Jesper Bratt pulled New Jersey to within 3-2 at 8:28 of the third period with a one-timer from the left circle on a power play.
"I just felt it was a good opportunity for me to shoot it and get something going in motion on the power play, and luckily it went in," Bratt said. "We've been playing the right way, so leaving home like this, going on a [season-long five-game road trip] with a win, it's always a good start."
Hughes tied it 3-3 at 10:32, quickly scoring from the right face-off dot after a shot from Ryan Graves deflected to him off Dawson Mercer in slot.
"It was a good way to win and probably not how we wanted to win," Hughes said. "But a comeback win, that's huge for our confidence. … A win is a win. … We wanted to take anything we could get. So it happened to be a comeback win. Like I said, that was a pretty, pretty crazy crowd today. It's a lot of fun playing the Rangers, so when you get to beat them, that's kind of a statement win, too.
"Obviously, they have one of the top goalies in the League. He stood on his head the first two periods. We were able to get a few behind him in the third. That's another good team. That's a big one for us."
New Jersey outshot New York 33-22 in the first two periods. Shesterkin saved all 19 shots he faced in the first.
"Back and forth, a lot of good scoring chances," Rangers coach Gerard Gallant said. "Obviously, 'Shesty' was great tonight. He was our best player. He gave us a chance to win the hockey game. … We had our chances, but we couldn't hold the lead."
Vesey gave New York a 1-0 lead 1:34 into the first period. After receiving a pass off the boards from Lafreniere, Vesey skated around New Jersey defenseman Brendan Smith and scored from the left circle.
Julien Gauthier made it 2-0 at 3:28 of the second period when he chipped the puck past Vanecek after receiving a drop pass from Adam Fox.