New York scored twice in 1:33 for a 3-1 lead, but Columbus scored 32 seconds apart to tie it 3-3 before two more goals, one apiece, made it 4-4 after two periods.
Buchnevich scored off another odd-man rush for a 2-1 lead at 6:57.
Kreider made it 3-1 at 8:30 on a tight tic-tac-toe play from Tony DeAngelo to Hayes to Kreider for his eighth goal on a tap-in.
"We had stretches of good minutes, stretches of not-so-good minutes," Jones said. "For the most part we played a well-checked game. When they went up 3-1, we responded well."
The Rangers led the Detroit Red Wings 2-0 in the third period Friday but lost 3-2 in overtime, and they let a lead slip away again.
Dubois' seventh made it 3-2 at 11:49 with a tip of a shot by Jones.
Foligno tied it 3-3 at 12:21 on a backhand move at the crease from a Boone Jenner pass.
Wennberg's first of the season came shorthanded at 16:03 to put the Blue Jackets ahead 4-3.
Jones picked off a Neal Pionk pass and passed ahead to Wennberg for his shot from the left circle that went in at the near post.
It was the first shorthanded goal of the season for Columbus.