Paul Hollywood has said the female stars of the Great British Bake Off "abandoned" the series when it moved from the BBC to Channel 4.
Co-judge Mary Berry and presenters Mel Giedroyc and Sue Perkins chose not to go with the baking programme when it was sold last year.
Hollywood faced a backlash from fans after he decided to make the move, declaring that the tent is "where I belong".
But he told Radio Times magazine: "I stayed with Bake Off. The girls abandoned it. But I was the one put under siege.
"I became the most hated man in the country.
"It's not fun for someone that doesn't like being in the limelight.
"I didn't set out to be on the telly, I set out to be a good baker. And I didn't want this."
He added: "If you give me a cheque, you'll never see me again."
Asked how big a cheque it would take, he said: "Not much, to be honest."
Hollywood now judges the cooking contest alongside Prue Leith, with Sandi Toksvig and Noel Fielding as the presenters.
It has continued to do well in the ratings, with 10 million people tuning in to see Sophie Faldo win the last series - the first on Channel 4.
But Hollywood suggested he is frustrated with those viewers who have not accompanied the show to its new home.
"There are people who can't get past channel one, the older generation, who don't understand how that works," he said.
"They only flick a couple of buttons and it takes three clicks to watch Channel 4."