The Blue Jackets, who won for the first time on their three-game California trip after losses to the Los Angeles Kings and Anaheim Ducks, are one point ahead of the Florida Panthers and two ahead of the Carolina Hurricanes.
Artemi Panarin scored two goals, and Sergei Bobrovsky made 33 saves for Columbus (33-28-5), which took a 3-0 lead early in the second period that shrunk to 3-2 in the third before Panarin scored into an empty net with 53 seconds left to make it 4-2.
"It was a must-win game tonight," said Blue Jackets captain Nick Foligno, who had a goal. "We talked about that before the game. We just have to start getting that mentality where you're in a tight game and they're coming and they're pushing. That's what teams do this time of year.
WATCH: [All Blue Jackets vs. Sharks highlights\]
"We found a way with some huge blocks, good sticks, great saves obviously by [Bobrovsky]. We found a way to get it done. That's what we need to do this time of year."
Joonas Donskoi and Evander Kane scored for the Sharks (35-22-9), who are in second place in the Pacific Division, 10 points behind the Vegas Golden Knights and one ahead of the Anaheim Ducks.
The goal was Kane's first since being traded to San Jose from the Buffalo Sabres on Feb. 26.
"You get down by three goals, it's tough to come back," said Kane, who also had an assist. "We came up a little short. Late in the second period, we started breaking down their defense and getting pucks to the net. We need to start doing that earlier."