Sir Keir Starmer faced a series of resignations as he suffered a major rebellion over the so-called "spy cops" law.
The Labour leader ordered his MPs to abstain on the third reading of the Covert Human Intelligence Sources (CHIS) Bill.
But 34 decided to oppose the legislation, including former leader Jeremy Corbyn, ex-shadow chancellor John McDonnell and former shadow home secretary Diane Abbott.
As a matter of conscience, I must vote against the #CHISBill tonight.
— Dan Carden MP (@DanCardenMP) October 15, 2020
Margaret Greenwood tendered her resignation as shadow schools minister and Dan Carden quit his post as a shadow Treasury minister so they could vote against the Bill.
Liverpool Walton MP Mr Carden said he voted against the Bill as a "matter of conscience", saying the legislation sets "dangerous new precedents" on the rule of law and civil liberties.
Wirral West MP Ms Greenwood echoed those sentiments, while Navendu Mishra, Labour MP for Stockport, also resigned from his role as a parliamentary aide to the party's deputy leader Angela Rayner ahead of the vote.
Mr Mishra said he believed voting against the Bill "sends a clearer message about the strength of our concerns".
The Bill would grant wide-ranging legal rights to undercover agents to commit crimes in the course of their work.
I have voted against the Covert Human Intelligence Sources bill and so have tendered my resignation from my position as Shadow Schools Minister. I wrote to Keir Starmer before the vote. I'd like to thank Keir for having given me the opportunity to serve on Labour's front bench. pic.twitter.com/9BmfqhciHz
— Margaret Greenwood MP (@MGreenwoodWW) October 15, 2020
Closing the committee stage debate for the Opposition, shadow Home Office minister Conor McGinn said rebels in his party do not have a "monopoly on principles".
He told the Commons: "I think they're wrong but it doesn't mean I don't respect the arguments they have put forward.
"All I would say gently is that those who oppose the Bill in its entirety don't have the monopoly on principles and nor are they the sole moral arbiters when it comes to forming a view on the measures in the Bill."
Mr Carden, in his letter to Sir Keir, said: "We have spoken at length on these matters and I know you have settled on yours and the party's position from your own experience and with sincerity.
"You will understand that as a Liverpool MP and trade unionist, I share the deep concerns about this legislation from across the labour movement, human rights organisations, and so many who have suffered the abuse of state power, from blacklisted workers to the Hillsborough families and survivors."
He said he fully supported the Labour leader, adding: "My focus now and in the months ahead will remain on representing my Liverpool Walton constituency and fighting for the people of my city as we face the huge challenges posed by the coronavirus pandemic."
Home Secretary Priti Patel said: "Once again, Labour has refused to stand up for those who protect our country and keep us all safe.
"Their leader may have changed, but Labour still can't be trusted on national security."