Ottawa took a 1-0 lead at 2:44 of the first period. Artem Zub took a wrist shot from the right point, and Kastelic tipped it into the net for his third goal of the season.
"It's a good feeling getting on the board early and the rest of the game just comes to you," Kastelic said.
Matthew Highmore, who was recalled from Belleville of the American Hockey League on Wednesday, shortly before the Senators traded Vladimir Tarasenko to the Florida Panthers, was credited with an assist in his season debut.
Highmore left the game in the third period with an upper-body injury.
The Senators had 12 shots on goal through the first two periods before totaling 18 in the third.
"We're best when we get a lot of pucks to the net and we go to the net for rebounds and screens," Ottawa interim coach Jacques Martin said. "We did in the third period and at the end of the game we ended up outshooting them by a fair margin, but I think we need to have more of a shooting mentality from the beginning."
Dostal made a left-toe save on Shane Pinto at 3:59 of the second period before the Ducks tied it 1-1 at 4:42.
"He's playing awesome," Zellweger said of Dostal. "He plays the puck well too, so lots of things he's been doing to give us confidence."
Strome tried to feed Frank Vatrano cutting through the slot, but the puck went off Ottawa defenseman Jake Sanderson and into his own net.