“She spoke to me and said look, ‘will you work for me?’. I've always said no.
“A number of MPs in the past have said will you work for me and I’ve always said no. I couldn't think of anything worse.
“The ego and everyone else that comes. Speaking now, hopefully I’m a different example, but I'm just very different, you know.”
He continued: “A working class woman - she’s quite fiery, she’s incredibly bright, incredibly bright.
“She's offering what the Labour Party is about, isn't she really, in terms of social mobility?
“And obviously I relate to aspects of the background and the back story.
“So yeah, I worked for her for a couple of years, left on good terms, and she's still a good friend.”
Ms Rayner was criticised last month for failing to wade in on the transgender row.
When asked whether a women could have a penis, Ms Rayner said: “When we debase it to whether or not… what genitalia you’ve got, I think all that does is damage people and it doesn’t help us go forward on some of the real issues that people are facing.
“I think we should be protecting women’s spaces – of course we should – and we should be supporting people who are going through transition.”