An investigation by Which? surveyed 4,800 members with car or home insurance and looked at policies from insurers which were commonly used.
More than one in 10 (12%) consumers who made a home insurance claim, and one in 20 (5%) who made a car insurance claim, had it fully or partly rejected.
Which? found some insurers are taking rules to the "extreme". For example, members told the consumer group their insurer would not pay out for the contents of their freezer without a receipt.
Home insurance policies looked at by the consumer group included a requirement for the property to be kept in good condition, but none was found to explain what this meant.
In one case, a consumer told the study that their insurer refused a claim because they had not regrouted their bathroom tiles every year.
Which? said it saw "many cases" where car insurers settled claims 50/50 even when the driver believed they had strong evidence that a car accident was not their fault, meaning they faced paying an excess and the possibility of losing their no-claims bonus.
A spokesman for the Association of British Insurers said: "Insurers continually review their policy wordings and claims procedures so that the expectations of customers are met.
"Motor insurers pay out the equivalent of £19 million every day to policyholders, with £9 million a day paid to homeowners under buildings and contents insurances. The industry is also stepping up its fight to tackle fraudulent claims to protect honest customers."