Poor savings rates, tough employment conditions and high living costs are part of the "painful adjustment" needed to rebalance the economy, Sir Mervyn King said.
The Bank of England Governor said he had "deep sympathy" with savers who are receiving "negligible" returns through no fault of their own. But he warned there was "no easy remedy" to get the economy back on track and raising interest rates would only make the situation worse.
The Bank of England's latest quarterly inflation report fuelled predictions of low interest rates lasting for another two years.
This would spell further bad news for savers, who have seen little return on their cash pots as the base rate is kept at its historic 0.5% low, while household budgets have been squeezed by inflation, which is beginning to ease.
Sir Mervyn denied suggestions that the Bank was giving the impression it was not worth saving, saying: "No, we're not sending out any such message at all and indeed you can see that households believe that it is, because their savings have gone up, not gone down.
"One of the reasons for slow growth in the last year was weakness of consumer spending and higher savings by households. I have deep sympathy with those who, totally unconnected with the origins of this crisis, suddenly find that the returns on their savings have reached, as I said, negligible levels."
He said increasing interest rates to 4% or 5% would give the impression that savings returns had gone up, but this would lead to the value of assets decreasing, falls in investment and consumer spending and a return to recession.
He continued: "I think that many savers would find that the value of their wealth would fall more than enough to offset the apparently higher yield and everyone would be worse off.
"So difficult though it is I think what we have to do is to make what is a difficult judgment about the right course of action for the economy as a whole. All groups in society are suffering from the consequences of the financial crisis. Our judgment has to be what is the right course of action eventually to steer the economy back to low inflation close to the target and steady growth."
Simon Rose, spokesman for campaign group Save Our Savers, said that the harmful effects on people's savings were undermining investment and growth. Mr Rose said of Sir Mervyn's comments: "Perhaps he needs to go back and look at his textbooks again. Prolonged low interest rates are a terrible thing. Savers are angry and they are dismayed."