Former Huddersfield Town boss Wagner was appointed ahead of the 2019-20 season and Schalke lost just twice in the first half of that campaign.
But their form plummeted from there, with Schalke ultimately finishing 12th and just seven points above the relegation zone last term.
An 8-0 hammering at champions Bayern Munich in their Bundesliga opener was followed by another poor showing in a 3-1 defeat at home to Werder Bremen on Saturday.
Schalke have failed to win their last 18 Bundesliga matches, which represents a club-record and the third worst streak for a coach in the competition's history.
"We had all hoped that we could improve on the pitch together with David Wagner. Unfortunately, we haven't seen the right performances and results from the first two games of the season for this to happen," said Jochen Schneider, Schalke's head of sport.
"We have therefore decided to make a fresh start. Despite the disappointing results, this wasn't an easy decision for us to make.
"I would like to thank David Wagner, Christoph Buhler and Frank Frohling, who, up until their departure, have done everything they can to get Schalke back on track."
Buhler and Frohling, Wagner's assistants, have been placed on gardening leave, with Schalke saying a successor will be named in the coming days.