Conservative Party Deputy Chairman Lee Anderson told GB News he didn't find the joke funny, adding: "I can assure him I have had more fun at the dentist."
Meanwhile, Ben Bradley said the language he alluded to isn't "acceptable in any forum, never mind a headline conference speech".
MP Jonathan Gullis simply quipped: "Most people thankfully are spared from knowing who Ed Davey is."
WATCH NOW: Ed Davey makes a risky joke in his party conference address
Addressing the Liberal Democrat Conference in Bournemouth, Davey said: "I'm going to have to start today with an apology.
"You might remember after our incredible victory in Somerton and Frome in July, when the amazing Sarah Dyke overturned a Conservative majority of 19,000, I said: 'It's time to get these clowns out of Number 10'. We even wrote it on the side of a caravan. Do you remember?
"Well, a party member got in touch afterwards to say he is an actual clown. And he took great offence at being compared to the Conservative Government.
"And on reflection, I have to admit he's got a point.
"Clowns didn't crash our economy and send interest rates soaring. Clowns didn't let water companies make billions in profits while dumping sewage into our rivers and onto our beaches. Clowns didn't plunge our NHS into crisis. Clowns didn't waste billions of pounds of our money on dodgy PPE contracts."
He added: "Clowns didn't do it. The Conservatives did."
Davey said: "I'd like to apologise unreservedly to that party member - and to the whole clowning community. I'm sorry - I used the wrong c-word."
Conservative Deputy Chairman Lee Anderson tore Davey apart for the remarks, telling GB News: "It's not the only c*** up this week for dismal Davey and his merry band of out-of-touch MPs who have been defeated over housing targets by their own members."
He added: "The good news is their Parliamentary mini bus only seats 16 MPs and with no plans to buy a bigger bus no one should take a blind bit of notice of what Ed says. Not even his own members do.
"If Ed thinks he is being humorous then I can assure him I have had more fun at the dentist."
Tory MP Ben Bradley also hit out at Davey for his comments, saying the language he alluded to isn't "acceptable in any forum, never mind a headline conference speech".
He told GB News: "I think a party leader who claims to be a serious person worthy of attracting people's votes around the country can probably do better than referring the millions of people who have and will vote Conservative as c***s, which let's face it isn't a word that's acceptable really in any forum, never mind a headline conference speech."
The MP for Mansfield added: "Safe to say, whatever his view of my constituency and my party, Mansfield Conservatives aren't exactly quaking in their boots following the Lib Dems 'impressive' 1.4 per cent of the vote in our constituency last time."
Meanwhile, Conservative MP Brendan Clarke-Smith pointed to previous remarks made by Davey when he said: "We need to get rid of these children on the Tory backbenches who can't have a grown-up debate."
Clarke-Smith said Davey's comments today show the Lib Dems are "not really that grown up...".