I simply wouldn't credit Carter for rescuing them, when the thing his presidency was most notorious for was failing to rescue them. That failure, probably more than anything else, cost him re-election.
Iran released the remaining hostages on Ronald Reagan's inauguration day. Either the Iranians were more afraid of what Reagan might do than they were of Carter, or they just wanted to rub Carter's nose in it.
I was 9 at the time and this was literally the first international news story I was aware of. So my memory for details will naturally be a bit hazy. But it was widely criticized as Carter's failure, not merely a military accident.