The National Audit Office also said it had not been given the evidence it needed to assess if the scheme's £144 billion liability, the estimated amount that is needed to pay to members in the future, was reasonable.
Auditors have refused to fully sign off the accounts of a civil service pension fund after Whitehall failed to provide records to show some pensioners were receiving the correct payments.
Amyas Morse, the comptroller and auditor general, qualified the 2011/12 civil superannuation accounts, which report the financial results of the Principal Civil Service Pension Scheme and some small public sector schemes, in both areas.
He also warned that although there has been "significant" improvements in record-keeping, current reforms of the pension system could increase the risk of the problems getting worse in future years.
Pensions in the scheme are linked to length of service and salary but six per cent of payments tested could not be backed up with the relevant records by the Cabinet Office.
In his report, Mr Morse said the Cabinet Office should continue to develop its programme to improve record-keeping.
"It should, as part of that programme, identify gaps in its historic records and take practicable action to address these," he said.
"The Cabinet Office has agreed the need to take such action to address the issues I have raised."