But it looks like the Liverpudlian baker made a smart move as he's now worth a sweet £10million.
See also: BBC cuts top talent pay bill
See also: The I'm A Celebrity effect: five contestants who saw their fortunes rise
The blue-eyed chef agreed a six-figure deal to follow the Bake Off to Channel 4.
And now he's certainly come a long way from 3am starts six days a week where he started out working in his dad's Merseyside bakery.
The baker has almost £8.8million of assets in his new company HJP Media LLP, according to The Sun.
He has a further £1.2million in a company called Paul Hollywood Limited.
It's a huge turn up for the books after his bread-making business Paul Hollywood Artisan Bread went into liquidation in 2014 with debts of over £50,000.
"Bake Off's success has made Paul incredibly wealthy and a household name. It's also opened doors and opportunities for him," a source said.
"He has released a series of books and has his own range of bread rolls in the shops."
Earlier this month it was revealed veteran cook Prue Leith has finally signed her contract for the new version of the Great British Bake Off on Channel 4.
The 77-year-old agreed a six-figure contract to star alongside Paul.
Bosses at makers Love Productions and Channel 4 boss Jay Hunt were both delighted to land Prue, who they see as a worthy successor to Mary Berry, with a long history of cooking on screen and writing about food and drink.
A TV source confirmed: "The deal has taken its time to iron out the details, but it has now been signed and Prue is going to be on the show.
"We think her and Paul will be another winning combination. Prue is excited about the show and can't wait to get started."
Prue Leith has finally signed her contract
Prue will join Paul Hollywood as a judge
Signing the deal came just a few days after Prue oddly appeared to distance herself from the baking series.
She was linked to the programme last month but at that stage had only held initial talks over the role.
She then wrote an article for the latest issue of The Spectator magazine where she insisted "a fortnight ago I got a taste of what being far too famous might feel like".
Paul Hollywood, Sue Perkins, Mary Berry and Mel Giedroyc on The Great Christmas Bake Off
She then dubbed the story "fake news" suggesting she was not on board with Bake Off, but it is understood that the article was written some time ago and she has now signed her contract after more rounds of talks.
Channel 4 have now set their sights on finding a duo to host the series, which is due on air this Autumn.
They have the almost impossible task of picking a pair of presenters to follow in the footsteps of Mel and Sue.
Despite a backlash from some viewers over the move from BBC to Channel 4, it is understood the broadcaster has received huge interest from advertisers.
The series is also likely to have its own sponsor which will earn them millions of pounds.
A Channel 4 spokesman said: "We have not confirmed the line up for Bake Off."