Jakub Voracek's power-play goal from the top of the slot put Columbus ahead 1-0 at 8:59 of the first period. It was his first goal of the season in his 12th game.
Panarin tied it 1-1 at 17:29 from the left face-off circle off a cross-ice pass by Adam Fox. Merzlikins dived to his right and got his stick on the shot, but the puck continued across the line.
Panarin entered the game with two goals and said earlier in the week he needed to shoot more. The former Blue Jackets forward had 10 shot attempts and six shots on goal.
"Maybe because Columbus, old team, I tried to score," he said. "That's why I shoot. … The situation I'm in now makes me hungry."
Eric Robinson gave Columbus a 2-1 lead at 6:56 of the second period with a shot from the left face-off circle that went off the right post.
Kreider tied it 2-2 with a redirection of a Trouba pass out of the right corner at 7:39.
"The first period was a bit sloppy," Gallant said. "It looked like we had four days off (since a 4-3 win against the Florida Panthers on Monday). We didn't have any jump, but after that I thought we played the exact way I wanted us to play.
"After they scored that second goal, I thought we were real good the rest of the night."
Panarin was alone in the slot to put the Rangers ahead 3-2 at 8:15, and Trouba extended the lead to 4-2 at 8:42, giving New York three goals in 1:03. They were the quickest three goals by the Rangers in a regular-season game since they scored three in 1:00 in a 4-2 win at the Panthers on Nov. 16, 1993.
"It was the end result of us doing the right things," Kreider said. "There's going to be stretches where pucks aren't going to go in but we're getting looks. Then they come in bunches like that, especially with the skill we have."
Blue Jackets coach Brad Larsen said, "You have to gather yourself. You have to calm your mind when the storm hits. We were not able to do that tonight."
Kreider scored the Rangers' fourth straight goal when he tapped in a pass from Mika Zibanejad on the power play 52 seconds into the third period for a 5-2 lead. Kreider has scored 11 goals this season, seven on the power play.