Brett Leason scored, and Lukas Dostal made 34 saves for the Ducks (10-19-0), who lost their fifth straight game and have lost 13 of 14 since Nov. 15.
"Frustration, trying not to let it get to that," Anaheim forward Troy Terry said. "It feels like a similar story. We get into penalty trouble or whatever the case is, and when the game ends you feel like you generated a lot and played pretty even 5-on-5. It's hard to continue doing it."
The Ducks gave the Rangers' six power-play opportunities; they scored on two. Anaheim is the most penalized team in the NHL with 406 penalty minutes in 29 games, an average of 14:00 per game, and has been short-handed a League-high 129 times.
"I thought we played hard and then we get into penalty trouble again, and then it's bingo ball, who's going to get the next penalty," Ducks coach Greg Cronin said. "It's unfortunate. It's a 2-1 game and then we take some penalties and before you know it it's 4-1."
Leason gave Anaheim a 1-0 lead at 13:30 of first period, scoring with a low shot from between the circles.
Kreider tied it 1-1 at 17:06. Ducks defenseman Jackson LaCombe poked the puck loose in the right face-off circle, and Kreider came off the wall, skated onto it and quickly shot it past Dostal.
Vesey gave New York a 2-1 lead at 11:45 of the second period with a shot from the left circle off a pass from Fox out of the right circle.
Kreider's power-play goal made it 3-1 at 16:19. From the side of the right post, he angled his stick blade and redirected Panarin's pass from the point.
"It's an impressive play," Zibanejad said. "I would say the angle is the most difficult one to try to get it by. It's not like he's done it once or twice where it's kind of just bouncing in or a lucky bounce. Like everything else that he does, he works on it every day. There's no luck involved in his game."