The midfielder spent four years playing alongside Bale at Tottenham, and seven in Madrid, forming part of a team that won four Champions Leagues and two La Liga titles before Bale returned to Spurs on a season-long loan deal last month.
"I've been with Gareth for a long time, almost my entire professional career," Modric told COPE radio. "He's a great guy... I think that what people have said about him lately is unfair.
"Maybe they judge him for the last few years, but what Gareth did at Madrid is impressive. For me, he'll always be remembered as a player who has done a lot for the team and the club."
Bale endured a difficult final season in Madrid, scoring just two league goals from 16 appearances in 2019-20.
His relationship with coach Zinedine Zidane deteriorated, with sources telling ESPN that it had "become something personal." He was also whistled by fans over the "Wales. Golf. Madrid. In that order" flag incident before being left out of the squad entirely at the end of the season.
"People say a lot of things, but Gareth never had a problem in the dressing room with anyone," Modric added. "He spoke Spanish, he just didn't give interviews. In the dressing room, he'd joke and speak with us a bit.
"We were all really happy with him... People forget everything he's done, but with the passing of time people will remember that he was an incredible, important player for Real Madrid."
The Wales international was repeatedly criticised by the Madrid media for an alleged failure to integrate into Spanish life during his time at the Bernabeu.
"Everyone has their own behaviour and way of doing things," Modric said. "Gareth was like that. He didn't socialise much, but in the dressing room he was totally fine.
"It's a shame that perhaps everything he's done has been forgotten because of the last year, or year-and-a-half."
Bale is yet to feature for Spurs since returning to the club, having sustained a knee injury while playing for Wales, and is expected to be fit after the international break.