Filip Zadina scored, and Kaapo Kahkonen made 28 saves for the Sharks in their season opener.
"We're still coming together as a team, and there were a lot of things I liked about our game tonight," San Jose coach David Quinn said. "For 38 minutes, it's a 1-1 hockey game, and then we didn't handle the adversity the way that we need to."
Michael Amadio gave the Golden Knights a 1-0 lead at 15:25 of the first period after a cross-ice feed from Karlsson.
Zadina tied it 1-1 at 17:48 on a wraparound. It was his first goal with the Sharks after signing as an unrestricted free agent during the offseason.
"We were trying to put the pucks on net to create some chances, some rebounds," Zadina said. "It was kind of like a lucky bounce. It hit Tommy (Hertl), and I was the net-front guy and just was in the right spot at the right time."
Hague made it 2-1 at 18:31 of the second period on a one-timer from the right face-off circle, and Nicolas Roy made it 3-1 at 19:24.
"Obviously, they had a push,” Roy said. “They had some energy for their first game, but we found a way and found the back of the net. At the end of the second, we played our best hockey."