Aston Villa needed less than two minutes to open the scoring in Germany. Matty Cash swung in a cross from the right which Ollie Watkins headed back to John McGinn to steer home from close range.
The visitors took control and should have doubled their lead but Watkins, Morgan Rogers and Youri Tielemans were all unable to convert chances. Villa were then made to pay for their misses when an awful error from Emi Martinez gifted Leipzig an unlikely equaliser.
The Argentina goalkeeper came racing out of his goal but was caught out by Lois Openda who nicked the ball away, went past the stopper and rolled the ball into an empty net just before the half-hour mark.
Unai Emery opted to send on Duran for Watkins at the break and his substitution proved to be inspired. The Colombian star needed just minutes to restore Villa’s lead with a brilliant effort from range that curled over goalkeeper Peter Gulacsi.
Duran had the ball in the back of the net again shortly afterwards only to see it chalked off for offside, and Leipzig quickly levelled again when a long ball forward found Openda who crossed for Baumgartner to guide a volley across goal and past Martinez.
Yet there was still time for more drama and another crucial contribution from a substitute. Ross Barkley's strike from outside of the box took a huge deflection off Lukas Klostermann to fly into the back of the net and hand Unai Emery's side all three points.
GOAL rates Aston Villa's players from the Red Bull Arena...