Cam Atkinson made it 2-0 at 19:05 when he scored on the power play.
"We let them play their game, do their thing kind of, and it cost us," Johansson said. "Even though it didn't end the way we wanted it to, we showed a lot of character and got a point."
Johansson made it 2-1 at 1:32 of the second when he scored on the power play. It was his first goal since signing with the Sabres as a free agent July 6.
Ullmark stopped Gustav Nyquist point-blank as a Blue Jackets power play expired at 10:14 of the second period, and Skinner tied it 2-2 at 11:58 with an unassisted goal.
Defenseman Ryan Murray skated the puck from behind the Columbus net, but Skinner stole it from behind and snapped a shot from the slot past Korpisalo.
"Our composure against Pittsburgh, it just blew up on us," Tortorella said. "We've been talking about that since then. Tonight, in the third period especially, going into it 2-2 after being up 2-0, was a really good test for us as far as staying within ourselves and not trying to do too much. I thought we did a good job."