The American will face five-time champion Novak Djokovic in the semifinals.
The match went with serve until the final set when Fritz broke twice in the final three games, either side of holding to love, sealing his spot in the next round on the first of two match points as fellow tournament debutant Auger-Aliassime sent a backhand wide.
Before the sixth game of the final set, the only other break points had come in the second game of the second set when Fritz had to stave off three of them against his Canadian opponent.
The top-seeded Nadal had lost his first two matches in the tournament and avoided equaling his career-worst streak of five straight defeats.
"I have been practicing well," Nadal said. "Just probably not enough matches to be at the level that I needed to be. Not enough confidence, probably, after six tough months. That's how it is.
"I accept that the season didn't end the way that I wanted, at least I finished with a positive victory," he continued. "It's important, the last official match of the season, so I am happy for that."
Nadal won the Australian Open and French Open to take his tally of Grand Slam titles to a record 22, but he has played just nine singles matches since withdrawing from Wimbledon a day before his semifinals match because of a torn abdominal muscle.
"I can't ask for more: 2022 has had a tough six months, two Grand Slams, and finishing the year in a high spot in the rankings. So, I can't complain at all," the 36-year-old Nadal said. "At my age, to be able to achieve and be competitive means a lot for me."
For 2023, the Spaniard said he wants to "start the season with the right energy, the right attitude, to reach the level that I need to be competitive from the beginning. Let's try it, I am excited about it."