Citizens Advice said it had seen cases in recent months which included payday lending to people who were aged under 18, had mental health issues or were drunk when they took out the loan.
The findings come at a time when Britain's biggest payday lenders are under threat of being put out of action if they fail to prove to the Office of Fair Trading (OFT) that their practices are up to scratch.
The OFT is expected to announce whether it will refer the payday market for an investigation by the Competition Commission in June.
Citizens Advice said the OFT must take "tough and immediate action" after the trading watchdog handed 50 payday lenders a 12-week deadline in early March to prove their good behaviour or risk losing their licences, which they need in order to trade. The OFT has written to lenders in waves, telling them where they need to step up, and the responses will be analysed before it decides what to do next.
Struggling borrowers' bank accounts were being drained of cash "without any warning" by lenders using a type of payment agreement called a continuous payment authority (CPA), the charity said. It found evidence that in some cases, lenders also took more than they were owed and refused to give a refund.
Citizens Advice also looked at customer feedback on 2,000 payday loans from more than 100 lenders, which was provided between November and May. It said that in almost nine out of 10 (87%) cases, borrowers were not asked to hand over documents to show they could afford the loan.
Three-quarters of people questioned said that they struggled to repay the loan and 84% of people with repayment problems were not offered the chance to have their interest and charges frozen.
Citizens Advice chief executive Gillian Guy described the payday loan industry as "out of control" and said it has shown a "complete disregard" for customers. She said: "It is vital that, following the investigation, the OFT takes swift action to protect consumers from the harm caused by these unscrupulous lenders."