After the two nations had split the singles at one win apiece, they combined to play out an absorbing, see-saw doubles encounter for the right to face Canada in the last four.
Serbia's pairing of Troicki and Novak Djokovic failed on three occasions to get over the line, allowing Russia duo Andrey Rublev and Karen Khachanov to battle back from the brink and seal a dramatic victory.
Djokovic, who had beaten Khachanov earlier to level the tie, attempted to console his distraught compatriot on the court in the immediate aftermath, while members of the Serbia squad were unable to hold back the tears during a poignant post-match press conference.
For Troicki, the hero in his country's 2010 Davis Cup success against France, the result was tough to take.
"I never experienced such a moment in my career, in my life," Troicki said. "I let my team down and I apologise to them.
"We had chances to finish it. We didn't do it. I messed up in the crucial moments.
"God gave me once to be the hero, to win the Davis Cup in the deciding rubber. Now he took it away. I'm really, really disappointed in myself that I couldn't hold my focus until the end and finish."
Serbia had been hoping to send off Janko Tipsarevic on a winning note, the 35-year-old having already announced this year that he would retire after the new-look tournament in Spain.
Team captain Nenad Zimonjic admitted it was not the finish he wanted for a group of players he described as his country's golden generation.
"I see it as an end, because it is Janko's last match, but he's obviously going to be here for us all the way, not just for this tie and in the future, because we are really good friends," Zimonjic said.
"You dream maybe to go all the way, to celebrate with a victory, but sometimes it doesn’t happen what you want to happen.
"But the main thing that everybody knows here [is] how much we care for each other and love each other, and that's what got us here."