The much-respected 70-year-old was appointed by Dortmund in June as part of Edin Terzic's backroom staff.
Hermann has spent more than three decades coaching in German football, appearing on the touchline for over 1,000 games in the Bundesliga, 2.Bundesliga and DFB-Pokal.
He previously had two spells as assistant at Bayern Munich, where he played a part in the club's famous treble alongside Jupp Heynckes in the 2012-13 season.
Dortmund announced in a statement on its official website on Thursday that Hermann has left his role with immediate effect.
"Peter has asked us for his release for health reasons. It goes without saying that we have complied with his wish," Dortmund sporting director Sebastian Kehl said.
"We were incredibly happy last summer that Peter – although his personal life plans didn't include it at the time – decided to spend another year in professional football for BVB.
"In him we are not only losing an experienced, technically excellent coach, but also a great person."
Hermann also enjoyed spells in the dugout with Bayer Leverkusen, Schalke, Hamburg, Nurnberg and Fortuna Dusseldorf.
"My time at BVB was short and yet very intense," Hermann said. "I've come to appreciate this club, the team and the entire environment over the past few months.
"I've enjoyed working for a club that loves and lives football so much. Unfortunately, I'm not allowed to work longer for Dortmund, because of course health always comes first."
Dortmund, which is sixth in the Bundesliga after 15 matches, confirmed it is searching for Hermann's successor and a decision will be made in the coming weeks.