Mats Zuccarello had a goal and an assist for New York (4-7-1), which won a game on the road for the first time this season.
"We did a lot of good things, played hard," Rangers coach David Quinn said. "I thought our gaps were great all over the ice. And listen, we're learning. We've got to learn to close games out. We've got to understand situational hockey, but to come in here and get two points against a really good hockey team I think says an awful lot about our guys.
"The last three games we think we've struggled a little bit, haven't been playing the way we think we're capable of, and I thought we got back to doing things we were doing earlier in the year and maybe turned it up a notch."
Jones made 34 saves for the Sharks, who extended their point streak to six games (4-0-2), thanks to Hertl's late goal.
"I looked up, saw the zeroes and thought it might have been over, but the goal came just in time," Hertl said. "It was a huge point because we didn't have our game tonight."