The second instalment of The X Factor attracted 6.3 million viewers on Sunday night - a drop of 500,000 from the launch show.
The episode featured emotional contestants doing their best to impress judges Simon Cowell, Louis Walsh, Sharon Osbourne and Nicole Scherzinger.
ITV said The X Factor was the most watched programme of Sunday night, with 6.3 million viewers - peaking at 7.1 million - and a 31% audience share.
It was a drop from the 6.8 million (34% share) and peak of 7.5 million who watched the launch episode.
Last year's equivalent Sunday show attracted 6.5 million viewers and a 30% share.
The ratings for Saturday night's series opener were the show's lowest audience for a launch episode in 10 years.
The drop in numbers comes despite show boss Simon bringing back old favourites as judges and reinstating long-running host Dermot O'Leary after a year away.
The series will face a fresh battle in the ratings when the BBC's rival show Strictly Come Dancing kicks off on September 3.
Numbers for the first episode of the current series of The X Factor are set to rise as consolidated figures will be calculated for TV viewers watching on catch-up and other services.