Amber Rudd has said Boris Johnson has a "sort of language which he's quite rightly nervous using in front of women" as the ex-home secretary hit out at the "boys' club" atmosphere in the Commons.
Ms Rudd also accused the Prime Minister of going "backwards" on promoting women.
The former Cabinet minister said she quit Mr Johnson's top team as work and pensions secretary in September 2019 because she did not like his style of government over Brexit.
The comments came as part of a project by the Institute for Government (IfG) aimed at getting ministers to reflect on their time in power.
In the study, former leader of the Commons Andrea Leadsom revealed that relations with ex-Speaker John Bercow reached such a low that she would not hold regular weekly meetings with him alone.
Ms Rudd expressed concern that the situation regarding the promotion of women was going into reverse.
She said: "There is a kind of boys' club-type behaviour in parliament because it is still more like a public school or a university club than anywhere else you'll ever go.
"I fear that it's going backwards a bit at the moment because unless you have the leadership really making an effort to ensure that women are promoted as equals, all the time – not just because, oh, let's promote the women, we forgot about the women – it's going to be a problem.
"I see that in Boris Johnson, I'm afraid.
"Even though I don't dislike him at all.
"He's come from that establishment group.
"And also, he has that sort of language, which he's – quite rightly – nervous of using in front of women."
Ms Rudd, who resigned as home secretary in 2018 over the Windrush scandal, said she quit the Cabinet again after a return to the top table as work and pensions secretary because of the way Mr Johnson treated people in order to get his Brexit agenda through.
She said: "It was Boris Johnson's style of government, really.
"It was the way he treated other people and his determination to deliver Brexit, whatever the cost in terms of the economy and, I thought, the consequences to people's lives."
Ms Rudd, a prominent Remainer in the 2016 Brexit campaign, famously remarked of Mr Johnson in a referendum TV debate: "Boris is the life and soul of the party, but he is not the man you want driving you home at the end of the evening."
Ms Leadsom and Mr Bercow regularly clashed when he was Commons Speaker.
Recalling their weekly meetings, she said: "It reached a point where I had to take someone with me, because of the level of vitriol in those meetings.
"And he, likewise, said he needed to have someone there, because apparently I was extraordinarily difficult."
The two had a public spat after Mr Bercow was accused of calling Ms Leadsom a "stupid woman".
Former attorney general Jeremy Wright told the IfG that David Cameron would become more publicly irritated at situations as prime minister than Theresa May did after she succeeded him.
Mr Wright said: "David Cameron let his irritation show more often and more visibly than Theresa ever did, getting the advice that she was being given."
The ex-Cabinet minister also insisted the government should have done more to defend judges hearing high-profile Brexit cases from media criticism.
He said: "Where I think we were going wrong... was to allow the sort of abuse to play out in the newspapers, of those who were doing their job as judges and as lawyers."
Former business secretary Greg Clark said some international firms were "alarmed" at how Britain was handling the post-Brexit vote turmoil.
He said: "The further that you go from these shores, the more people were alarmed at what they thought was uncertainty and unpredictability in a country that they had always thought of as a byword for predictability."