"Struggles in the game, yeah, but [Bobrovsky] made a couple of huge saves," Blue Jackets coach John Tortorella said.
Atkinson redirected Zach Werenski's point shot at 14:00 of the second period to make it 3-2, and Lukas Sedlak extended the lead to 4-2 at 17:49.
Zuccarello cut it to 4-3 at 18:58 off a face-off win by Mika Zibanejad, but Foligno scored with two seconds left in the period to make it 5-3.
David Savard pushed the lead to 6-3 at 5:29 of the third period off a pass from Panarin from behind the net.
Zuccarello (15:00) and Jimmy Vesey (16:48) pulled the Rangers to within 6-5, but Foligno scored his second of the game at 18:14 to make it 7-5.
"It's one of those games too many were going in against us, but we found a way to stick with it and get some back for ourselves too," Foligno said. "It's a testament to a lot of us. Every line contributed."
Rick Nash, who retired Friday after 15 NHL seasons because of medical reasons, dropped the ceremonial puck. Nash was selected with the No. 1 pick by the Blue Jackets in the 2002 NHL Draft and is their all-time leader in games played (674), goals (289), assists (258) and points (547). He also played more than five seasons with the Rangers after he was acquired by them in a trade with Columbus on July 23, 2012.