Flanker Luke Reimer's try seven minutes from time looked to have given the Brumbies a fourth straight win from the start of the season but winger Filipo Daugunu crossed in the 78th minute to put the Reds back within one score.
The visitors broke out of defence and laid siege to the Brumbies line as the clock ticked past 80 minutes before former Wallabies flyhalf Gordon went over in the corner to snatch a first victory for the Reds in Canberra since 2021.
"Pretty much the whole game it felt like we were on the back foot," said Gordon, who returned to union this season after a spell in rugby league.
"It felt like we were down and out towards the end, but the boys just dug in ... and we ended up winning."
It was an impressive recovery after the Brumbies scored two tries in the first 10 minutes once the match finally got started following a 90-minute delay because of a lightning storm.
The only other perfect start in the competition was brought to a shuddering halt on Friday night when the New South Wales Waratahs were crushed 59–19 by the Wellington Hurricanes at Sydney Football Stadium
The Waratahs lost centre Joseph Suaalii to a hamstring strain before kickoff and although in-form winger Max Jorgensen scored two tries, the Hurricanes ran in nine for a ninth straight win over the Sydney-based side.
Two tries in four minutes from All Blacks centre Billy Proctor straight after the break put the powerful Hurricanes comfortably ahead and two yellow cards in the second half kept the Waratahs firmly on the back foot.
In the opening game of the round, the Waikato Chiefs welcomed back flyhalf Damian McKenzie from paternity leave with an equally emphatic 57–24 victory over Moana Pasifika to give them 13 points, two behind the Brumbies in second place in the standings.
While a big win for his old team would have pleased newly-appointed All Blacks coach Dave Rennie, he would have learned a bit more about his pool of players from Saturday's heavyweight clash between the Auckland Blues and Canterbury Crusaders.
Test winger Caleb Clarke scored a hat-trick of tries and flyhalf Beauden Barrett kicked nine points as the Blues condemned the reigning champions to a third loss in four games this season with a 29-13 victory at Eden Park.
"We knew we had to turn up tonight against a classy side, it wasn't perfect but we're happy to win at home," said Barrett, who was playing the first game of his 16th Super Rugby season.
Earlier on Saturday, the Otago Highlanders edged Western Force 39–31 in a tight contest in Dunedin, taking the lead through winger Caleb Tangitau's second try 10 minutes from time before Reesjan Pasitoa's 77th minute penalty sealed the deal.