Badosa fired seven aces – three more than Barty – and saved 12 of 14 break points she faced on the way to a 6-4, 6-3 victory.
It's the biggest win of Badosa's career, the Spaniard having notched her first top-20 victory when she beat No.5 seed Belinda Bencic in the second round.
"I'm still a little bit shocked," Badosa admitted after the match, adding that when Barty's final forehand reeled out of court "I was, like, 'What just happened?'
"But I'm quite happy with my match, the way that I managed my nerves. It was quite important because sometimes these matches are tough to close."
Serving for the opening set, Badosa was down 0-40 but won five straight points on the way to sealing it with an overhead winner.
They traded breaks in the first two games of the second set, Badosa gaining the advantage again with a break for 4-3 as she won the final four games of the match.
In the semi-finals Badosa will face either American Sloane Sephens or Russian Veronika Kudermetova, who met in the last quarter-final of the day.
Badosa, a former French Open junior champion, is the lowest-ranked player to beat Barty since September 2019.
Barty had made her first overseas appearance since the WTA's pandemic shutdown last year with a run to the title in Miami last week – where she was the defending champion thanks to her 2019 triumph after the tournament was cancelled in 2020.
In other matches, Tunisia's Ons Jabeur defeated American Coco Gauff 6-2 7-6 (7-2) to book a semi-final meeting with Danka Kovinic of Montenegro, a 6-7 (2-7) 7-5 6-1 winner over Kazakhstan's Yulia Putintseva.