Boris Johnson was accused of ducking personal responsibility for the exams fiasco after sacking the senior official at the Department for Education and blaming the debacle on a "mutant algorithm".
The Prime Minister acknowledged the stress caused by the situation – which eventually resulted in a U-turn with A-level and GCSE grades in England awarded based on teachers' assessments rather than the algorithm.
The Department for Education announced that permanent secretary Jonathan Slater will stand down because "the Prime Minister has concluded that there is a need for fresh official leadership".
The announcement came a day after Sally Collier resigned from her role as head of exams regulator Ofqual.
Dave Penman, leader of the FDA union which represents senior public servants, said: "If it wasn't clear before, then it certainly is now – this administration will throw civil service leaders under a bus without a moment's hesitation to shield ministers from any kind of accountability."
Relations between Number 10 and Whitehall have been strained, with Mr Johnson's senior aide Dominic Cummings backing a major shake-up of the civil service.
Mr Penman said: "After this Government's continuous anonymous briefings to the press, trust between ministers and civil servants is already at an all-time low and this will only damage it further."
Education mandarin Mr Slater will stand down on September 1, in advance of the end of his tenure in spring 2021.
Susan Acland-Hood, currently interim second permanent secretary, will take over as acting permanent secretary.
Education Secretary Gavin Williamson thanked Mr Slater for his "commitment to public service".
In a statement, he said: "I would like to thank Jonathan Slater for his commitment to public service, including over four years spent as permanent secretary in the Department for Education.
"Like the Prime Minister, I appreciate the hard work of officials across government, particularly during this unprecedented time.
"I look forward to working with Susan Acland-Hood as the acting permanent secretary to continue to drive forward this Government's ambitious agenda to level up education across the country, giving every child an equal opportunity to succeed.
"Our immediate focus remains on making sure every child returns to the classroom full-time at the start of term."
The Prime Minister, addressing pupils at a school in Coalville, Leicestershire, acknowledged that the exams situation had been "stressful" for those awaiting A-level and GCSE results.
"I'm afraid your grades were almost derailed by a mutant algorithm," he told them.
"I know how stressful that must have been for pupils up and down the country.
"I'm very, very glad that it has finally been sorted out."
Mr Johnson had previously claimed that the algorithm-based grades would be "robust" and "dependable".
Shadow education secretary Kate Green said: "Boris Johnson is shamelessly trying to avoid taking responsibility for the exams fiasco that his government created.
"Responsibility for this shambles lies squarely with Downing Street and the Department for Education, who set out how they wanted the algorithm to work and were warned weeks in advance of issues, but repeatedly refused to address the problems they had created."
On Mr Slater's departure, she added: "Under this Government, civil servants have time and time again taken the fall for the incompetence and failures of ministers.
"Parents will be looking on in dismay at a government in complete chaos just a matter of days before children will return to schools.
"Leadership requires a sense of responsibility and a willingness to be held accountable, qualities this Prime Minister and his ministers utterly lack."
The departure of Mr Slater follows the resignation of Ms Collier from her role as Ofqual's chief regulator on Tuesday.
Education Secretary Gavin Williamson denied forcing her out of her post at the watchdog.
"No, this was a decision that Sally made, an incredibly dedicated and committed public servant, and in discussions with the Ofqual board, which, as you're aware, is a non-ministerial Government department," he told BBC's Breakfast.
"That was a decision between them."
Mr Williamson indicated he had not considered resigning over the U-turn.
But Labour frontbencher Bill Esterson said: "Why is the secretary of state still in post? Two scapegoats can't save him.
"The buck stops with Williamson. Sooner or later, he has to go too."
General secretary of the National Association of Head Teachers (NAHT) Paul Whiteman said: "Our experience is that civil servants have been working incredibly hard to get things right during this pandemic.
"Our civil service is impartial, professional and prepared to handle the difficult issues in good faith.
"We hope that the events of the last 24 hours will not serve to erode trust in the department at a time when it is needed most."