Disadvantage can follow someone like a "shadow down the years" affecting their degree and job prospects, Professor Les Ebdon has warned.
The director of the Office for Fair Access (OFFA) said universities and colleges should do more to engage with students throughout their entire time in education to ensure everyone gets the same support and opportunities.
He also suggested that there are still some who believe that there is a "conflict" between maintaining academic excellence and ensuring fair access to university, but that academic potential is not just about someone's existing grades.
In a foreword to OFFA's annual report and accounts, Prof Ebdon said that institutions need to be involved in "the whole student lifecycle" - raising the aspirations and achievements of children from an early age, supporting them while they study for a degree and when they go into work or further
"Disadvantage can follow you like a shadow down the years, affecting the degree you end up with and your ensuing postgraduate study or search for a job," he said.
"For access to be meaningful, there must be appropriate support for students as they progress through their studies and continue to employment or postgraduate study. OFFA therefore takes a broad view of access, challenging universities and colleges to look not just at how they can diversify their student intake, but also at how they can engage across the whole student lifecycle."
"We may be winning hearts and minds among young people but there's still work to be done to convince those who feel there is a conflict between fair access and academic excellence," Prof Ebdon said.
"There is no conflict between fair access and academic excellence. Nor should there be. Fair access is about searching out academic potential wherever it is found - in every type of neighbourhood, every type of school, and every age group, ethnic group and gender.
"It's about recognising that academic potential is not only reflected in existing qualifications.
"It's about acknowledging that a wide range of people have the potential to become the excellent graduates who will later run our businesses and lead our country - not just the privileged few but also, for example, the 30-something worker who wants to improve his or her skills, the working-class boy who dreams of a career in law, or the daughter of first-generation immigrants who has won a scholarship to an independent school."