Both officials had been criticised by Gers fans following a poor start to the season that led to manager Russell Martin's dismissal in October after just one win from his first seven league matches in charge.
Former Manchester United chief executive Stewart only arrived at Ibrox in December, while Thelwell's appointment was announced in April, although he did not take up his post until leaving Everton at the end of last season.
Rangers have since been taken over by a US-based consortium led by Andrew Cavenagh, who has now appointed director Fraser Thornton as interim chief executive while the club seek replacements for Stewart and Thelwell.
Rangers have won four straight league games under new boss Danny Rohl but are still nine points adrift of Scottish Premiership leaders Hearts.