PSG, who were crowned Ligue 1 champions after the French league was cancelled in April, have never reached the final of the Champions League.
And it looked as though that wait would continue after Atalanta, who are playing their first-ever season in the Champions League, held a 1-0 lead for the majority of the match.
PSG were punished for that costly error as Duvan Zapata played the ball into the path of Mario Pasalic, who produced a superb finish to beat Keylor Navas from 16 yards out.
Kylian Mbappe was brought on in the second half as PSG tried their hardest to produce an equaliser.
The French side made their breakthrough in the 90th minute after Marquinhos' shot went in off Atalanta defender Mattia Caldara.
PSG sealed their stunning comeback minutes later after Mbappe set up Eric Maxim Choupo-Moting, who tapped in from six yards out.
Tuchel's side will now face either RB Leipzig or Atletico Madrid in the semi-finals of the Champions League.
RB Leipzig clash with Atletico on Thursday evening, while Barcelona take on Bayern Munich in the quarter-finals on Friday.
Manchester City continue with their Champions League ambitions on Saturday in their quarter-final clash with Lyon.
The two semi-final matches will be played on the 18th and 19th August, with the final taking place on 23rd at the Estadio Da Luz in Lisbon.
Featured Image Credit: BT Sport
- PSG slammed over ‘insulting’ offer for Lyon star Rayan Cherki
- Lionel Messi overlooked by Christophe Galtier for promotion in PSG first team
- Hakim Ziyech's move to PSG collapsed after 'Chelsea sent the wrong documents THREE times'
- Kylian Mbappe produced hilarious reaction to Neymar’s free-kick during PSG warmup