Lord Janner's family have expressed outrage at his "inexplicable inclusion" in the Independent Inquiry into Child Sexual Abuse.
His three children said the existence of a strand of the inquiry examining allegations against Lord Janner - the only individual to be "singled out" - created an "assumption of guilt".
Their comments came ahead of a meeting with lawyers from the inquiry on Wednesday, where they planned to discuss their concerns.
Lord Janner's son Daniel Janner QC and two daughters, Marion Janner and Rabbi Laura Janner-Klausner, have also indicated their intention to apply for core participant status, which would grant them a formal role in the inquiry process.
Speaking outside the inquiry's headquarters in Millbank, central London, Mr Janner said: "We are here to protest.
"We will state our outrage to this inquiry that our late father is the only individual singled out for separate treatment.
"As long as there is a strand in his name there is an assumption of guilt.
"The inquiry was meant to investigate institutions and not individuals.
"Our father was entirely innocent of any wrongdoing. We know that because we have seen the evidence."
Thirteen different investigative strands spanning several decades and examining a host of different institutions are being pursued by the inquiry.
An internal review in December concluded the inquiry should not be scaled back and would retain all investigations.
But changes were proposed to the scope and timing of the public hearing into responses to allegations of sexual abuse involving Lord Janner, with a stronger focus on institutions.
Lord Janner, who died in December 2015, is alleged to have abused youngsters over a period spanning more than 30 years dating back to the 1950s.
Mr Janner dismissed the accusations as "false" and alleged they had been made "with the intention of making fraudulent claims for compensation".
He added: "Yet this shambolic and discredited inquiry denies us the right even to cross-examine. That is not justice."
Mr Janner said the family were "proud to fight relentlessly for our late father's good name".
He said: "His inexplicable inclusion must be stopped before it further stains the integrity of the inquiry and of genuine complainants."