Greenwood scored late after James Tarkowski had equalised moments after the youngster had put the hosts ahead in the second half.
It was United's first home win against Burnley in the Premier League since February 2015.
"Very happy with three points, we haven't beaten Burnley here since 2015. Good result, five wins on the bounce, that's good form," Solskjaer told Sky Sports.
"We've always said this year has to be progress and improvement, we've got as many points as last season and then if we get some more, we've got more than last season and that's progress.
"We want to finish with a trophy in the Europa League, we've got loads of games and loads to focus on."
Burnley had the ball in the back of the net after 18 seconds but it was ruled out with the scorer Chris Wood in an offside position.
Both teams had chances to score before the break with Wood heading just wide before Greenwood was unable to get on the end of a Marcus Rashford cross.
The hosts went ahead three minutes into the second half when Greenwood fired home from Rashford's low cross after a clever Bruno Fernandes dummy.
Burnley responded less than two minutes later when Tarkowski scored an equaliser from Ashley Westwood's corner.
United had a great chance to retake the lead after 70 minutes but Bailey Peacock-Farrell stopped Fernandes' header.
The host did go back in front with six minutes remaining when Greenwood's deflected strike wrong-footed Peacock-Farrell.
United grabbed a third in stoppage time as substitutes Cavani and Donny van de Beek combined which saw the Uruguayan striker score into an empty net.
"Very happy for him," Solskaer told the BBC about Greenwood. "He's mixing up his game. He goes inside and outside, he's maturing all the time.
"It's lovely to see. He's put the work in on the training ground and reaps the rewards."