The practice, known as 'dynamic currency conversion', allows retailers and foreign banks to set their own conversation - adding up to 10 per cent on bills.
Approximately £26.4 billion will be spent by Brits overseas on debit and credit cards this year - as a third of holidaymakers use plastic to make purchases.
The cost of paying in pounds on cards while abroad was £633.6 million a year - equivalent to £100 per trip for every person - levied by third-party currency exchange providers in a service called dynamic currency conversion.
Fees average 6% each transaction but can be has high as 10%.
Other costs that are easy to dodge
Some fees are easier to dodge than others
Banks also charge holidaymakers to use cards abroad with the average fee of 2.8% for debit and 3.5% for credit cards - an additional cost of £293m and £350m.
And another £190 million in fees will come out of the pockets of unsuspecting Brits for withdrawing cash, according to research carried out by Fairfx. The average ATM fee is 2.9% for debit cards and up to 4.9% on credit.
Now travellers have been warned to avoid the ruse in the wake of poor state of the pound which has plunged post Brexit and after the instability of the recent snap election.
Just like Money Saving Expert's Guy Anker who was almost charged a 18% mark up at a cash machine in Corfu.
He said: "When taking out €200 I was presented with two options. One was to pay in euros and let my bank do the transfer. The other was to pay in pounds, and let the Greek bank do conversion.
"By choosing euros, as you always should, I got that day's rate of €1.23 to the pound, with no foreign load fee added as I've a cheap overseas card.
"That meant I was charged £162.61. But if I'd chosen to pay in pounds it would have cost me £191.84 at a pathetic €1.04-to-the-pound exchange rate. That means the Greek bank was effectively charging its own 18% load."
Ian Strafford-Taylor, chief executive of FairFX, commented: "Consumers may be forgiven for thinking that bad currency decisions only cost a couple of pounds here and this reveals the true extent of how much we're missing out.
"Fortunately, it's not too late and a few smart moves can ensure that we take back control of our currency choices."
How to beat the rip-off
Get more from your pounds
If you're asked what currency you want to pay in, always choose local currency. And the same if you're using an ATM.
Be vigilant: If you're billed in pounds, refuse it. Write 'DCC rejected' on the receipt and insist on being charged in local currency.
Debit and credit cards are a good back up for travel money but if you want to guarantee your rate, use a prepaid currency card which locks in your exchange rate when you buy. If you're using a prepaid currency card loaded with pounds, similarly always choose to pay in local currency to avoid third party charges and rates.
To get the best value for money before you book, pick a destination where you get more bang for your buck with more favourable long-term exchange rates.
Set up a currency alert with an online currency provider so that you are alerted to the best time to buy when the rate moves in your favour.
Whatever you do, don't leave sorting out your travel money until the last minute as you could be left severely out of pocket. Exchange rates at the airport can be more than 10 percent more expensive than elsewhere meaning that you could lose £100 for every £1,000 you exchange.