The Spaniard, who has signed a three-year deal, replaces compatriot Pep Clotet, who was dismissed with four games of the Championship remaining.
Karanka led Middlesbrough out of the second tier in the 2015-2016 campaign and spent a year as Nottingham Forest boss before departing in January 2019.
"I'm delighted to be joining a club with the history and fanbase that Birmingham City has," Karanka said.
"I share the excitement and determination of everyone here to start working together on all the challenges that lie ahead."
Birmingham finished just two places and two points above the relegation zone this season, and it would have been just one spot had Wigan Athletic not been docked points for entering administration.
As a player, Karanka won a LaLiga title and three UEFA Champions League medals with Real Madrid, for whom he was also an assistant coach under Jose Mourinho.