Sir Jacob Rees-Mogg said it is not “wise” for the Conservative Party to change leaders "in the way people seem to change their socks” as he criticised the “self-obsession of some politicians”.
The former business secretary said the party needed to get behind Rishi Sunak and "recognise an election is close at hand."
Speaking on GB News, Sir Jacob Rees-Mogg said: "There have been rumours, stories, speculation abounding in our newspapers over the last 48 hours about a challenge to Rishi Sunak.
“It's been claimed the Leader of the House of Commons, Penny Mordaunt is about to be shoehorned in as a unity candidate.
“I think this is crazy stuff. It's madness. The Conservative Party has already changed leader too many times since the last election.
“The mandate that we have for Prime Minister gets thinner and thinner every time we change it. And we need, as conservatives, to calm down and support the Prime Minister.
“Why do I say that? Well, I think there are some positive signs that the economy is beginning to pick up, that the recession didn't seem to last very long. If the Bank of England were halfway competent, it would be beginning to cut interest rates and that would be helping people who have mortgages; the economy would be beginning to grow.
“We are getting ahead with some conservative policies and the Rwanda one is an example of that.
“But we also have to show to voters that we are a serious party of government and changing our leader in the way people seem to change their socks cannot be wise, it cannot be sensible, it cannot imbue the nation with that level of confidence that it needs to have.
“And so, although I've been a critic of Rishi Sunak over the years, I haven't always larded him with praise, I think now is the time for the party to be loyal: To back its leader to recognise an election is close at hand.
“We need to concentrate on that and the manifesto for the British people that we will deliver in the next few months, rather than the vanity, the vain glory, the self-obsession of some politicians who think it's about them.
“It's not. It’s about the country and it's about the King’s government being carried on, as the Duke of Wellington always used to say.”