Nearly 6,000 foreign criminals were sent home last year - the highest number since records began, the Government has said.
But there remain 9,000 people from overseas in British jails and immigration detention centres, Justice Minister Sam Gyimah said.
He made the comments in justice questions in the Commons as he came under pressure from backbench Tories to do everything possible to send foreign offenders back to their native countries.
Mr Gyimah said: "As of the 30 September 2016 there were 6,618 foreign national offenders (FNOs) serving a custodial sentence in our prisons, a further 2,374 foreign nationals are being held in prison on remand or (in) immigration detention centres.
"We are committed to increasing the number of foreign national offenders removed from our prisons - whether they are removed under the prisoner transfer agreement or through the early removal scheme.
"In 2015/16, 5,810 FNOs were removed from prisons and immigration removal centres, this is the highest number since records began. And since 2010, 33,000 have been removed."
Conservative MP Philip Hollobone (Kettering) said many foreign offenders in British prisons are from Poland.
He said Poland's "derogation" from the compulsory EU prisoner transfer directive is due to end in December 2016, paving the way for more foreign convicts to be ejected from the UK.
He added: "Are we now in a position to send these Polish prisoners back to prison in their own country?"
Mr Gyimah said: "All eligible Polish nationals have been identified and deportation orders sought.
"We are referring cases to the Polish courts and transfers will take place once Polish legal proceedings have been completed."