World number three Zverev, playing in his first match since his first-round loss at Wimbledon, was far from his best but emerged with a 7-6(6), 6-4 win over Adam Walton under the lights in Toronto.
Momentum shifted firmly in Zverev's favour when he won a 52-shot rally in the first set tiebreak, and he sealed the win on a double fault by the Australian in the second set.
"It was a very important moment, very important point for me," Zverev said. "Lucky to get through in the first set and finish it off in two."
Zverez said he did not play his "prettiest" match but was proud of how he battled after taking a break from the game following his early Wimbledon exit.
"I took some time off, which I needed also for myself. I’m happy to be playing again," he added.
Zverev next plays 32nd seed Matteo Arnaldi on Thursday.
On the women's side, top seed Gauff was two points from defeat but battled back to beat fellow American Danielle Collins 7-5, 4-6, 7-6(2) in a nearly three-hour battle to reach the third round.
French Open champion Gauff overcame 23 double faults and 74 unforced errors to beat Collins in their first career meeting.
"I was practicing well and I don't think I transferred it today, but hopefully I got my bad match of the tournament out of the way," Gauff said.
Several top players withdrew ahead of the Masters and WTA 1000 tournament, including world number ones Aryna Sabalenka and Jannik Sinner, Carlos Alcaraz, Novak Djokovic and Jack Draper.