Culture Secretary John Whittingdale is under pressure to withdraw from decisions about press regulation after disclosing that he had a relationship with a sex worker.
Mr Whittingdale, who is single, said he had been unaware of the woman's occupation and had broken off the relationship when he discovered someone was trying to sell the story to the press.
He received support from Downing Street who said he was "a single man who is entitled to a private life" and that Prime Minister David Cameron had "full confidence in him".
But he faced calls from campaigners to recuse himself from decisions about press regulation amid claims that the revelations had left him vulnerable.
Although the relationship occurred before he was made a minister following the 2015 general election, he was chairman of the influential Commons Culture, Media and Sport Committee which had held a series of high-profile hearings on the phone-hacking scandal.
BBC2's Newsnight reported that four newspapers - The People, The Mail on Sunday, The Sun and The Independent - had investigated the claims at the time but concluded it was not a public interest story.
Mr Whittingdale insisted that events had no bearing on any decisions he took in office.
"This is an old story which was a bit embarrassing at the time. The events occurred long before I took up my present position and it has never had any influence on the decisions I have made as Culture Secretary," he said.
But shadow culture secretary Maria Eagle said it was essential that Mr Whittingdale acted to ensure there was no perception of "undue influence" in his dealings with the press.
"Everyone is entitled to a private life. However, these revelations raise serious questions about why the Secretary of State has reneged on the Government's promise to deliver the cross-party agreement on Leveson when this is something he was previously committed to as chair of the Culture, Media and Sport Select Committee," she said.
"In order for the public to have any confidence in the Government's approach to press regulation and to allay any concerns about perceptions of any undue influence, the Secretary of State must now recuse himself from any decision-making over this matter, just as Vince Cable was removed from deciding media policy in the last parliament."
Brian Cathcart, from the Hacked Off campaign group, said that since becoming Culture Secretary with responsibility for the media, Mr Whittingdale had taken a number of decisions which had been welcomed by the press.
He told the Press Association: "He is compromised. There is absolutely no doubt about it, he is compromised. He knew the press were on to him and he didn't tell the Prime Minister, and we would like to know more about how much he knew.
"He knew that at least one newspaper had this story - was he approached by other newspapers? Did he ever wonder why they didn't run the story?"
Mr Cathcart said he thought the newspapers had "stocked up" the story for future use and as a way to possibly put "pressure on him".
The executive director of the Society of Editors, Bob Satchwell, said it is a "preposterous conspiracy theory too far" to say newspapers and broadcasters "jointly decided not to publish" the story.
"The idea that the newspapers and broadcasters could all get together and say 'we are not running the story' is just silly," he said.
"Since the Leveson report and the establishment of a new and tougher press regulator, papers have become extremely careful about stories involving anyone in public life."
In a statement, Mr Whittingdale confirmed he had a six-month relationship with the woman who he met through a dating website.
"Between August 2013 and February 2014, I had a relationship with someone who I first met through Match.com. She was a similar age and lived close to me," he said.
"At no time did she give me any indication of her real occupation and I only discovered this when I was made aware that someone was trying to sell a story about me to tabloid newspapers. As soon as I discovered, I ended the relationship."