In his second straight start, Filip Gustavsson made 33 saves for the Senators (30-41-7) whose lone goal came from Brady Tkachuk. Jack Roslovic scored for Columbus (35-36-7) while Elvis Merzlikins stopped 25 shots.
"As the game went on I thought we got a little hungrier," Sens head coach D.J. Smith said. "It was a pretty even game both ways but we find a way to win the game in the shootout."
Tim Stützle had the only goal in the shootout as Gustavsson turned aside Roslovic, Gustav Nyquist and Jakub Voracek.
"I always think shootouts are pretty fun," Gustavsson said. "I've been the backup a lot of games this year and we always do the shootouts in the pre-game skate so maybe [my success is] from there."
Columbus took a 1-0 lead 11:20 into the first as on a broken play on the man advantage, Cole Sillinger was able to locate the puck and find Roslovic who had an almost empty-net to deposit his 20th goal of the season into.
Kelly thought he had tied the game at 10:02 of the second as on a 2-on-0 breakaway, he buried Dylan Gambrell's pass but Blue Jackets head coach Brad Larsen challenged for offside where it was overturned after Tkachuk was still in the offensive zone as he was changing.
The Sens did tie the game at 7:06 of the third and it came on a gorgeous passing sequence from Josh Norris, Tim Stützle and finally Tkachuk who buried his 29th of the season to send the game to overtime, and ultimately a shootout.
"I felt more comfortable tonight," Gustavsson said. "This game was relaxing and I felt like I could just play my game."
Ottawa is back home Saturday to host Montreal. Puck drop is 7 p.m.
