The two sides played out a 1-1 draw in Germany last week, and any thoughts Xavi's men had of brushing aside their opponents on home turf were dispelled in the fourth minute when Filip Kostic scored from the penalty spot.
Rafael Borre and Kostic added further goals to give the scoreline a scarcely believable look, before a fine Sergio Busquets strike and Memphis Depay penalty reduced the deficit in stoppage-time.
But Frankfurt had done enough for a famous 4-3 aggregate win and Oliver Glasner's men will now face West Ham United in the last four after the Premier League side overcame Lyon.
Barca started dismally and fell behind in the fourth minute when Kostic slammed home from the spot after Eric Garcia had hauled down Jesper Lindstrom in the area.
It went 2-0 down in the 36th minute, though, courtesy of a stunning 25-yard strike from Borre that flashed past Marc-Andre ter Stegen and into the roof of the net.
Ansgar Knauff fizzed a strike narrowly over Ter Stegen's crossbar soon after as Frankfurt threatened to put the tie beyond Barca's reach before the interval.
Frankfurt then sent its fans into dreamland when Kostic drilled across Ter Stegen for his second of the game in the 67th minute.
Busquets thundered home from outside the penalty area in stoppage time moments after having an effort ruled out by VAR (video assistant referee) for offside, before Depay scored from the spot with almost the last kick of the game after Evan Ndicka had clattered into Luuk de Jong.
Few expected Barca's 15-game unbeaten streak to come to an end against a team sitting ninth in the Bundesliga, but Frankfurt was excellent value for its win.
Glasner's side, which had 15 shots to Barca's 10, is now unbeaten in its past 10 games against Spanish sides in European competition (won six drawn four), including the past five away from home (won four dranw one).
Barca will need to lift itself when it hosts Cadiz in LaLiga on Tuesday (AEST), while a buoyant Frankfurt visits Union Berlin in the Bundesliga a day earlier.