Juventus is seeking to win the tournament for the fourth year in a row but they were made to battle against its local rival to set up a last four meeting with Atalanta, which shocked Serie A leader Napoli a day earlier.
Costa gave Juventus a deserved early lead with a powerful volleyed finish but Torino almost equalised within two minutes, M'Baye Niang's shot after a rapid break hitting the post.
Mandzukic spurned two presentable chances to extend Juve's lead early in the second half, before the Croatia striker settled the tie after 66 minutes with a cute chipped finish, the goal awarded after a video assistant referee (VAR) referral.
Despite the host progressing, it was another frustrating game for Paulo Dybala, who could not extend his run of scoring in all six of his home appearances in the Coppa Italia.
Torino had scored 11 goals in this season's competition to be the tournament's most prolific side, but it could not find a way through a Juve defence that has conceded one goal in 10 games in all competitions.