Mika Zibanejad and Artemi Panarin each had a goal and an assist, Vincent Trocheck scored, and Jonathan Quick made 25 saves for his 398th NHL win.
The Rangers (13-9-1) went 3-for-6 on the power play and 3-for-3 on the penalty kill and won for the first time since Nov. 19.
"I think it's been hard lately for us," Kakko said. "All teams have ups and downs and we are not doing well right now, and the third period was pretty bad, but that's something you need, maybe lucky sometimes. They almost got a breakaway, but we scored in the end."
Suzuki and Caufield each assisted on the other's goal in the third period. Mike Matheson scored, and Sam Montembeault made 24 saves for the Canadiens (8-12-3).
"I thought we deserved better," Suzuki said. "I thought we played better than them. It's a tough way to end a game that way giving up a goal that late. Special teams was the big difference. We've done well in those situations so far this season. We'll clean it up tomorrow."
Kakko gave the Rangers a 4-3 lead at 19:36, scoring from below the right face-off circle off a drop pass from Will Cuylle, who got the puck in front of the net off a pass from Zac Jones.
Filip Chytil kept the puck in at the blue line, preventing Jake Evans from going on a potential short-handed breakaway. Jones carried the puck into the left circle, spun and his pass to the middle went off Kaiden Guhle and onto Cuylle's stick.
"I knew Kaapo was behind me," Cuylle said. "I looked to pull their goalie to one side, get it back so he had an open net."
Said Kakko, "When 'Cools' got it, I'm thinking I'm wide open, hopefully he's going to find me. Nice pass by him. Big goal for us."
Caufield made it 3-2 at 4:16, scoring with a low shot through Quick's five-hole from the bottom of the left circle after the Rangers were unable to clear the puck out of the zone with Suzuki keeping it in at the blue line.