Artemi Panarin had two assists to extend his season-opening point streak to 14 games, and Kreider had a goal and an assist for the Rangers (11-2-1), who are 9-0-1 during the streak. Quick made 25 saves.
Elvis Merzlikins made 37 saves, including 16 in the third period, for the Blue Jackets (4-7-4), who have lost five in a row (0-3-2). Erik Gudbranson had two assists.
“We don't have room for error right now,” Columbus forward Sean Kuraly said. “It's tough to get two points for us. It's going to be until we figure it out. We're going to have to learn how to make that slim margin of error, and right now it's just not coming easy and we're going to have to battle through.”
Justin Danforth gave Columbus a 1-0 lead at 16:26 of the first period, deflecting in Gudbranson's shot from along the right boards.
Kreider tied it 1-1 at 18:46 with a one-timer from the right circle. It was his 10th goal this season.
Lafreniere put the Rangers in front 2-1 at 11:27 of the second period with a one-timer from the left circle off a cross-ice pass from Vincent Trocheck.
Panarin, who had the secondary assist on the play, has 24 points (eight goals, 16 assists) during his point streak, which is tied with Rod Gilbert (1972-73) for the longest to begin a season in Rangers history.
“It means, honestly, a lot to me,” Panarin said. “It’s a good feeling. Thanks to my partners and the support of the fans.”
Adam Fantilli tied it 2-2 at 12:35 of the second, scoring blocker side from the right circle after he stole the puck in the neutral zone.