Dougie Hamilton scored, and Vitek Vanecek made 18 saves for New Jersey (16-4-0), which had three goals disallowed.
"It's definitely frustrating, but nothing we can do because the refs make the call and there's nothing that's going to change it, so we've got to stick with it," Devils captain Nico Hischier said. "I think we did a good job but came up short."
Tavares gave the Maple Leafs a 1-0 lead at 11:41 of the first period from the slot after Mitchell Marner's backhand pass from behind the net.
Holmberg's first NHL goal made it 2-0 at 14:03. He scored from the bottom of the left face-off circle off a turnover along the left boards. The forward was playing his sixth NHL game.
"I think we lost the game in the first period," Hischier said. "We were not good enough with the puck and had too many turnovers."
Murray made 16 saves in the second period, stopping nine shots on two New Jersey power plays.
"I think we played unbelievable defensively, and I thought our first period kind of set us up for the win tonight," Murray said. "They had momentum throughout the second with a lot of power plays, but our first period is where we really got the job done and were able to hold on.
"At the end of the game, they're just gritty so anything you can do to try to keep the puck out of the net, because they were really pushing."
Hamilton scored on a deflection from the slot at 14:52 of the third period for the 2-1 final.
"Nobody feels good, I don't feel good, don't like the feeling," Devils coach Lindy Ruff said. "I don't like losing and I know it's been a while, but you don't want that feeling. Now you've got to get over that, get ready for your next game, be able to answer the bell and continue to prove we can be the team that was able to run all those games together."