Markram (110) scored his fourth ODI hundred, forming crucial partnerships with captain Temba Bavuma and Matthew Breetzke, before Dewald Brevis scored a swift fifty as South Africa pulled off their third-biggest successful run chase in ODI history.
"Happy to have crossed the line," Bavuma said in the post-match presentation. "Unbelievable game... and it showed how tough it is to play against this Indian side."
India, put to bat after losing their 20th ODI toss in a row, lost openers Rohit Sharma (14) and Yashasvi Jaiswal (22) within 10 overs. Kohli, whose century helped India win the first ODI on Sunday, focused on rotating strikes with Ruturaj Gaikwad, guiding India to 158-2 in 25 overs.
The duo added 97 runs in the next 10 overs as Gaikwad reached his maiden ODI hundred in 77 balls, hitting two sixes and 12 fours along the way.
Gaikwad (105) tried to pull Marco Jansen's (2-63) offcutter only to get caught in deep fine leg, ending the 195-run partnership for the third wicket. But Kohli, along with captain KL Rahul, continued the flow of runs.
Kohli's record-extending 53rd ODI hundred came up in 90 balls, with the help of two sixes and seven fours, before he fell to Lungi Ngidi at 102.
Rahul (66) and Ravindra Jadeja's (24) unbeaten 69-run partnership helped India post 358-5.
Markram sets the stage
South Africa lost Quinton de Kock early, but his fellow opener Markram steadied the innings with captain Temba Bavuma (46), adding 101 in the next 16 overs and making sure the asking rate never got out of hand.
Matthew Breetzke, in at four, held the other end as Markram reached his fourth hundred in ODIs in 88 balls, including four sixes and nine boundaries. Markram got caught trying to hit Harshit Rana down long on when South Africa needed another 162 from 20 overs.
But Dewald Brevis (54 off 34) launched a blitz, hitting Rana for back-to-back sixes en route to his maiden ODI fifty, as South Africa added 85 runs in the next 10 overs.
South Africa then lost three wickets within four overs, including Brevis and Breetzke (66), as Indian bowlers tried to make a late comeback on a track made further batting-friendly by the evening dew.
But the visitors were not dissuaded, even as Tony de Zorzi (17 off 11) was forced to retire due to a hamstring injury, as they could smell victory, needing only 27 runs from the last five overs.
Corbin Bosch (29 off 15) and Keshav Maharaj (10) calmly guided South Africa to victory with four balls to spare, as Bosch scored the winning runs with a drive down long on, getting the visitors to 362-6.
"Proud of the boys to get the job done under pressure," said player-of-the-match Markram.
Rahul said the evening dew made it difficult for the Indian bowlers to be effective. Asked if he could have done anything differently, he joked, "Yeah, win the toss."
The decisive third match of the series will be played in Visakhapatnam on Saturday.