HM Revenue and Customs is planning to close 137 offices under a modernisation programme.
A total of 13 new regional centres will be opened over the next five years.
The Public and Commercial Services (PCS) union said the cuts would pose a "significant threat" to the operation of HMRC as well as to the working lives of staff.
The union called for full public and parliamentary scrutiny of the decision.
HMRC said its modernisation programme includes investment in new online and other services to make it easier for people to pay their tax.
Lin Homer, HMRC's chief executive, said: "HMRC is committed to modern, regional centres serving every region and nation in the UK, with skilled and varied jobs and development opportunities, while also ensuring jobs are spread throughout the UK and not concentrated in the capital.
"HMRC has too many expensive, isolated and outdated offices. This makes it difficult for us to collaborate, modernise our ways of working, and make the changes we need to transform our service to customers and clamp down further on the minority who try to cheat the system.
"The new regional centres will bring our staff together in more modern and cost-effective buildings in areas with lower rents. They will also make a big contribution to the cities where they are based, providing high-quality, skilled jobs and supporting the Government's commitment for a national recovery that benefits all parts of the UK."