Holders Bayern Munich twice gave up a the lead before crashing out of the German Cup 6-5 on penalties to second-tier Holstein Kiel on Wednesday following a 2-2 draw after extra time in the second-round match.
"We lost to the underdogs Kiel and they did everything they could," Muller said. "We had chances in the first half. But we made mistakes and Kiel punished us.
"They were very efficient but we should have won the game. We are out and it is bitter. It is difficult to say we did not invest everything. That's positive but it is frustrating to be eliminated."
The champions' defence has been leaking throughout the season with their most goals conceded after 15 games in the league since 1981, and Kiel struck on the break in the 37th minute when Bartels beat defender Niklas Sule in a sprint to score.
Despite making five changes to the team that squandered a two-goal lead to lose 3-2 at Borussia Monchengladbach in the Bundesliga on Saturday, coach Hansi Flick's Bayern continued to be vulnerable at the back.
Bayern went ahead again through Leroy Sane's 48th-minute curled free kick but offered up several good chances for the hosts, who found ways past their opponents' shaky backline.
Bayern, who won five titles in 2020 including the Champions League, hit the woodwork through Jamal Musiala before the hosts struck with Hauke Wahl's last-gasp header to send the game into extra time.
Kiel, who next face Darmstadt 98, kept their cool in the shootout to knock Bayern out at this stage of the German Cup for the first time since 2000.
"We believed throughout that we could make the spring a big surprise here tonight," Bartels said. "We said we will make all our penalties and we did. But overall I think it capped a very good performance from us and we deserved it."