The former PM wrote about his days at Eton – the country’s most exclusive public school – in his new book being serialised in The Times.
He also described how he once woke to members of the infamous Bullingdon Club smashing bottles with a golf club and says he once swore in front of the Queen.
On his cannabis smoking, he wrote: “Three of us used to hire one of the school’s double scull rowing boats and head off to a small island in the middle of the Thames called Queen’s Eyot.
“Being quite small back then, I was the cox. Once there, we would roll up and spend a summer’s afternoon gently off our heads.”
The group’s trips were brought to an abrupt halt however when teachers found out and his rowing friends were kicked out.
Mr Cameron said he thought he was in the clear until he was summoned in the middle of a maths class, calling it “without doubt the worst moment of my life so far”.
Hauled in front of the headmaster, Mr Cameron managed to avoid being expelled by telling “a more and more elaborate set of lies” and claiming he had only smoked the drug once at the school.
The former Tory leader, like many at the elite institution, ended up at Oxford University – at the same time as Boris Johnson.
He wrote: “I made friends. I had fun. I argued. I gossiped. And I fell in love. Lots of times.”
“I can’t, of course, write about Oxford without three dreaded words that haunted me for most of my political life: the Bullingdon Club,” added Mr Cameron.
“When I look now at the much-reproduced photograph taken of our group of appallingly, over-self-confident ‘sons of privilege’, I cringe. If I had known at the time the grief I would get for that picture, of course I would never have joined. But life isn’t like that.”
The all-male dining club is known for its boisterous behaviour and Mr Cameron was pictured with Boris Johnson in a group photograph of its members.
However, he says stories of its members trashing restaurants and being paralytic from alcohol are exaggerated.
“I was never arrested. I was never completely incomprehensible from drink,” he wrote.
But he does confirm a ritual that involved being woken in the night to members wrecking his room.
“I have a pretty clear memory of walking from my bedroom into my sitting room to find a group of people making a terrible racket, with one of them standing on the legs of an upended table, using a golf club to smash bottles as they were thrown at him,” said Mr Cameron.
“I can’t swear that one of these people was Boris Johnson, but he was certainly a member at the time.
“Boris has claimed subsequently that he was unable to climb over the wall into my college. I’m not sure I believe his story. But I’m not totally certain of my own, either. So perhaps I should leave it there.”
Mr Cameron also told of how he was packed off to a boarding school at age seven, where Prince Edward was among the pupils.
Unfortunately, his first time in the presence of the Queen did not go to plan.
The 52-year-old wrote: “I was asked to read one of the lessons at our carol service – Isaiah, I think – and Her Majesty was in the front row.
“I did OK, but crucially forgot to say ‘Thanks be to God’ at the end. I remembered as I stepped away from the lectern, started to turn back, then realised it was too late to go back, panicked, and said, ‘Oh s***’.”
As well as tales of his school days, the first extracts from the ex-PM’s book focus on his profound regrets over Brexit as well as some stinging criticism of Mr Johnson for behaving “appallingly” during the referendum campaign.
He also wrote about his pride on delivering on one of his key policies, gay marriage, calling it “one of the most contentious, hard-fought and divisive issues” he dealt with.
Mr Cameron said backing the policy was “a giant leap for our party”.