Whitney, directed by Scottish film-maker Kevin Macdonald, alleges that Houston’s cousin, the soul singer Dee Dee Warwick, abused her.
The film debuted at the Cannes Film Festival and includes interviews with Houston’s half-brother, the former NBA basketball player Gary Garland-Houston, and her assistant, Mary Jones.
The pair make the allegations against Warwick.
According to the documentary, Houston never told her mother about the alleged abuse by Warwick, The Times reported.
The newspaper reports that in the film Garland-Houston says he had been sexually abused “by a female relative” between the ages of seven and nine but does not identify her.
Jones says in the film that Houston had told her: “Mary, I was [abused] too. It was a woman.”
Ms Jones, when asked if Houston had named the person, replied: “It was Dee Dee Warwick.”
Houston, who sold millions of records and starred in the box-office hit film The Bodyguard, died in 2012 aged 48.
She was found in a bathtub in the Beverly Hilton Hotel, with the coroner ruling that she died of drowning and the effects of heart disease and cocaine use.
Houston ended her volatile 15-year marriage to rhythm and blues singer Bobby Brown in 2007.
Dee Dee Warwick is the younger sister of soul singing icon Dionne Warwick and was the niece of Houston’s mother.
She enjoyed a successful career as a soul singer in the 1960s and 70s and died at the age of 63 in 2008.
The Warwick family have so far not released a comment on the allegations.