Savard and Brandon Dubinsky scored, and Sergei Bobrovsky made 22 saves for Columbus (27-22-4), which has lost four straight (0-3-1).
"We'd like to get more than two goals tonight, but we played a hard game," Blue Jackets defenseman Seth Jones said. "You see the importance of every play. Against a team like this, with 45 seconds left, you can't afford defensive plays like that.
"These last two games I feel like we've thrown away some points."
Dubinsky tied the game 2-2 with 6:05 left in the third period when he put in his rebound. He redirected a pass from Markus Nutivaara toward the net, Holtby made the save with his left pad, but the rebound went off the right post to Dubinsky, who tipped it in.
Savard made it 1-0 at 1:58 of the first period, taking a pass from Alex Wennberg in the high slot and shooting past a screened Holtby.