Bayern – who have won the title for the past seven years – are now in sixth, seven points behind Gladbach.
They had led when Ivan Perisic smashed home from the edge of the box.
But Ramy Bensebaini powered in a header to level and then scored an injury-time penalty to seal a dramatic victory after Javi Martinez fouled Marcus Thuram in the box and was sent off.
Bayern – who lost Corentin Tolisso to a first-half hamstring injury – have now lost two league games in a row under interim boss Hans-Dieter Flic.
Monchengladbach, who last won the title in 1977, are one point above RB Leipzig – who beat Hoffenheim 3-1.