The Rangers are 3-0-0 against the Devils this season. They play for the final time at the Garden on April 3.
Kaapo Kahkonen made 23 saves in his Devils debut. He was acquired in a trade from the San Jose Sharks on Friday.
"Good to get the first one out of the way, obviously, but tough to not come up with the two points," Kahkonen said.
New Jersey (31-30-4) has lost two in a row and five of six. The Devils are six points out of a Stanley Cup Playoff spot with 17 games to play.
"I thought we came up light a little bit in a couple areas, little loose puck battles," Devils coach Travis Green said. "A lot of nights those little battles determine the game. Obviously, our power play needs to get us one and we didn't convert on some really good chances where we had some good looks or even backdoor plays that didn't connect that we're capable of. It was a tough loss against a good team."
Zibanejad gave the Rangers a 1-0 lead at 12:25 of the second period.
Zac Jones sent the puck down the ice and Jack Roslovic chipped it into the zone before going into the right corner on the forecheck against Jonas Siegenthaler.
The forecheck allowed the puck to come free to Curtis Lazar, but he gave it away to Chris Kreider behind the net. Kreider found Zibanejad in front. With the puck on his backhand, he carried it across the slot, around John Marino, and scored with a shot under Kahkonen's blocker.
Zibanejad glided backward into the left corner boards and glass, took a deep breath, a sigh of relief and smiled as his teammates came over to celebrate his 20th goal of the season.
"I think my face said it all," Zibanejad said.
Rangers coach Peter Laviolette said the forecheck from Roslovic is an example of the speed he has brought to the Rangers in the two games he's played since they acquired him from the Columbus Blue Jackets on Friday.
He said the same about Alex Wennberg, who has played his first two games since being traded from the Seattle Kraken as the center on the third line between Will Cuylle and Kaapo Kakko.
"They bring a lot of speed and they hang onto pucks and they look to make plays," Laviolette said. "They can put it behind them and go to work or they can attack off the rush, which they've done as well. They're noticeable and I think they've had positive impacts on their lines."
Gustafsson made it 2-0 at 19:08 of the second with a wrist shot through traffic.
Matt Rempe, who won the face-off in the right circle of the offensive zone, screened Kahkonen in front as Gustafsson's shot went over the goalie's glove.