The grime star had been working with Ed Sheeran and the 99 Problems rapper on the song Take Me Back To London, which features on Sheeran’s No 6 Collaborations Project album.
Speaking about the hit on The Jonathan Ross Show, Stormzy said having Jay-Z on the record would have changed “everything”.
The London-born rapper said: “We were all in the studio writing; me, Ed Sheeran and Jay-Z.
“Before that I loved Take Me Back To London because I thought it was just going to be me and Ed and then Ed was like, ‘Jay is getting on it.’
“It blew my mind. Jay-Z is my number one hero inspiration.
“We got into the studio and we were writing for this song and then he just stops the music and he said, ‘Tell me about London, I need some inspiration.'”
Stormzy said he and his manager began telling Jay-Z about London and they had “this powerful, beautiful, greatest conversation of my entire life”.
The rapper said they talked about “everything from black culture, community, music, his journey, my journey”.
He continued: “At the end he’s smiling, we were probably talking for an hour and he goes, ‘So what do you think of the song?’… I had verbal diarrhoea.
“I was like, ‘I love this song but you being on the song changes everything.’
“Take Me Back To London with me and Ed is one thing, but now that Jay-Z is on it… This is Jay-Z and Stormzy, what he means to culture, what I mean to culture… I said, ‘Is this the song?’ I didn’t think it was the song…”
The rapper, whose real name is Michael Ebenazer Kwadjo Omari Owuo Jr, said Jay-Z was “super proud and super shocked and taken aback by the fact that this kid told him”.
He added: “I felt proper bad because I was like, ‘You’re saying no to Jay-Z!’
“I was like, ‘If you didn’t ask me this, I would have just written my verse and we would have done it and I would have never questioned it but now that you’ve asked me, no.’
“I was like, ‘I know how it goes, Mr Z. You are the most brilliant, busy man and I will probably never get this opportunity ever again but hand on my heart, I don’t think this is the right song for us.
“I don’t know why I’m saying this to you but this is not the song.'”
Describing the encounter as the “greatest story” of his life, Stormzy said the US rapper “fully understood”.
“We spoke and he made me feel [proud of what I did]. I left there thinking [I was an idiot] but then I had the honour of seeing him again and having the conversation and he let me know to my face that what I did [was courageous].
“He couldn’t believe that I did it.”
Take Me Back To London topped the charts earlier this year and remained at number one for five weeks.
Stormzy also recently announced that he will be touring the world next year, playing 55 dates in total, 13 of them across the UK and Ireland.
His new album, Heavy Is The Head, is released on the 7 December, a day after tickets for his tour go on sale.
Stormzy will appear on The Jonathan Ross Show at 10.10pm on 30 November on ITV.