More than 100 million calls by the public to central Government departments were charged at a premium rate and the practice must be stopped, an influential committee of MPs has said.
Margaret Hodge, chairwoman of the Public Accounts Committee, said taxpayers should be able to contact Government services "easily and cheaply".
But in a report, the committee said of 208 million calls in 2012/13, some 63% were made to higher rate numbers at an estimated total cost of £56 million.
The Department for Work and Pensions received 100 million of the calls and HM Revenue and Customs took 68 million calls.
Mrs Hodge said: "Customers of Government services should be able to contact those services easily and cheaply. Charging customers higher rates by making them use 0845 or other high rate numbers is not acceptable, especially when the customers are often vulnerable people.
"We found that one third of customer telephone lines across Central Government used higher rate numbers. Half of those lines serve the poorest people.
"Customers spent an estimated £56 million on calls using higher rate numbers, from the lines run by the Department for Work and Pensions, to helplines for victim support and the Bereavement Service and the inquiries and complaints line of the Student Loans Company."
In its report, the Public Accounts Committee also said calls to Government departments take too long to answer.
It found most departments have no targets at all, despite a normal industry benchmark demanding calls be answered within 20 seconds.
It said HM Revenue and Customs only answered 16% of calls made to its tax credit helpline on July 31, the deadline day for notifying change of circumstances.
It said across the first quarter of 2013/14, average call waits at HM Revenue and Customs were seven minutes.
Mrs Hodge said: "Callers must be informed of the costs involved in calling a particular number. Costs to callers can be even higher when they are left waiting to speak to someone.
"Performance by departments varies but is often astonishingly bad. HMRC managed to answer only 16% of the calls it received on its tax credits helpline on the deadline day for notifying the department of changes of circumstances.
"The industry benchmark is to answer 80% of calls in 20 seconds but most departments do not have such a target and their performance falls wide of accepted industry standards.
"Citizens should not as a matter of principle have to put up with standards of service from government which are significantly worse than industry standards."
Matthew Sinclair, chief executive of the TaxPayers' Alliance, said: "Charging people via the back door to contact departments and public bodies is unacceptable. Premium rate lines not only cost a fortune to use but hit those on low incomes particularly hard.
"It's time ministers scrapped these rip-off charges as we pay enough for government bureaucracy already without a stealth telephone tax adding to that."
A Government spokesman said: "Our digital by default agenda is creating online services which are so good that people prefer to use them.
"We are transforming a first wave of 25 exemplar services, to be digital by 2015.
"We agree that it is inappropriate for vulnerable people to pay high charges for accessing vital public services and we are clear that a more consistent approach is needed.
"The Cabinet Office now runs a cross-departmental group to consider customer telephone lines. This group has made good progress in drafting guidance on prefix number selection and establishing best practice.
"We will publish this guidance and have a standing remit to ensure it is kept up to date."
Richard Lloyd, executive director of consumer group Which?, said: "It's ridiculous that people face a bumper bill to call a public body, especially when they have to wait on hold.
"Almost 60,000 people have supported our Costly Calls campaign for all companies and public bodies to provide either freephone or local rate numbers for their customer service and complaints lines.
"This practice has been costing cash-strapped consumers millions of pounds.
"The Cabinet Office must now act fast to ensure the Government and public bodies lead by example and put an end to costly calls."
The report said higher rate telephone lines "have a disproportionate impact on vulnerable and l ow-income groups who are deterred from calling, limiting access to essential services".
It said: " We are concerned that 59 of the 120 higher rate phone numbers used by departments are for telephone lines that serve vulnerable and low-income groups.
"These include a Victim Support helpline, the inquiries and complaints lines of the Student Loans Company, and 35 lines run by the Department for Work and Pensions, including the Bereavement Service helpline.
"Higher rate 084 numbers are usually more expensive than geographic number types (such as those with 01, 02, or 03 prefixes), particularly where a mobile telephone is used."
The Victim Support charity, which uses an 0845 number, said it was looking at the issue.
Spokesman Jeff Gardner said: "We are currently looking at possible ways of reducing the cost to people calling Victim Support's Supportline, and are working with partners to establish how this could be done.
"Victim Support offers a call back service to anybody leaving a message or raising concerns about the cost of contacting us.
"People can also contact their local Victim Support in office hours on local rate numbers or email the Supportline as an alternative to ringing us."
Shadow Cabinet Office minister Michael Dugher said: "Ministers should take action against rip-off phone charges to Government. It makes no sense to have the most expensive charges for vital phone calls. Those who seek access to Government services are often those most in need and yet they are being made to pay extortionate call rates.
"The Cabinet Office may have set up a group to 'consider' customer telephone lines, but no action has been taken. It is essential that we now see concrete steps to bring down costs for the public. As well as cheaper charges, this needs to include less waiting times and more accessible information on the charges people can expect."