With Sergio Aguero injured, Manuel Pellegrini opted to go without a recognised striker, employing Sterling, Silva and Kevin De Bruyne as a rather fluid front three.
The early half chances on the night fell to the Germans, but it was actually City who took the lead when Silva latched onto Sterling's flick and fired past Yann Sommer from a tight angle 15 minutes in.
Defending has been an issue all season when Vincent Kompany has been missing and it was again. It took less than three minutes for Gladbach to get back on level terms as Korb, who caught a sight of goal earlier, clinically converted an equaliser after good work from Lars Stindl.
For a while, neither side could find their shooting boots after those two goals. Sterling, Yaya Toure and Eliaquim Mangala all missed the target for City, while the away side couldn't find a way to test Joe Hart either.
Gladbach turned the game on its head in the closing stages before the interval when Raffael pounced in the penalty, deflecting Fabian Johnson's shot beyond the grasp of a helpless Hart.
Looking for a way back into the game, City turned up the heat when the teams returned for the second half. Sommer was playing the role of hero for Gladbach, keeping out Sterling and Silva on a number of occasions around the hour mark.
Around the same time news filtered through that Sevilla had taken the lead against Juventus, handing City the opportunity to win the group if they could only get back ahead themselves.
Granit Xhaka and Stindl both threatened, but the vital goal was City's as Sterling put the sky blues back level with a first time finish. Moments later the 21-year-old was at it again, putting City ahead with just 10 minutes to play after being played in by the substitute Bony.
Bony himself then made absolutely sure of the win with a goal of his own in the closing stages, dispatching at the second attempt after his first effort had been blocked.
Manchester City: Hart, Clichy (Sagna - 80'), Otamendi, Mangala, Kolarov, Fernandinho, Delph (Bony - 65'), Toure, Sterling, Silva, De Bruyne (Navas - 65')
Borussia Monchengladbach: Sommer, Elvedi, Christensen, Nordtveit, Korb, Xhaka, Dahoud (Schulz - 67'), Wendt (Hazard - 84'), Stindl, Raffael, Johnson (Drmic - 72')