Philip Hammond has vowed to maintain a "disciplined" approach to the public finances as he prepares for a "challenging" Budget later this year.
The Chancellor has come under pressure from Cabinet colleagues to find extra cash to fund pay rises for public sector workers, but he indicated that his priority was on tackling the deficit.
He said the UK was spending "eye-wateringly large" sums on servicing the national debt but insisted progress was being made.
Mr Hammond also hit out at Labour, claiming Jeremy Corbyn's spending plans would add an extra £5.8 billion a year to debt interest payments.
On BBC Radio 4's Today programme he said "I'm sure the next Budget will be challenging" - but acknowledged that "all budgets are challenging".
Setting out his priorities, he said: "It's very, very important that we deal with Britain's debt because we are spending eye-wateringly large amounts of taxpayers' money every year servicing a debt which is still growing.
"Fortunately, because we have got the deficit under control, next year that debt will start to fall. For the first time in 10 years we are going to see the debt beginning to fall.
"My concern is that we continue to maintain a disciplined path towards getting our debt down."
Debt as a share of GDP is forecast to peak in 2017-18 and then fall, but because the deficit has not been eliminated the Government will have to continue borrowing to make ends meet.
Hitting out at Mr Corbyn's alternative approach, Mr Hammond said: "In the general election campaign we saw Labour coming forward with plans that would add £250 billion to our debt over the course of this Parliament, on top of the current trajectory.
"That would require them to find another £5.8 billion a year to pay for that debt. That's the salary of nearly 200,000 public workers and we have had no indication at all from the Labour Party of what they would cut in order to meet that additional debt interest cost."