A decision by the Parole Board to release the rapist John Worboys has been quashed, prompting the board's chairman Nick Hardwick to resign.
A legal challenge by two victims was upheld by the High Court, and the Parole Board praised the "bravery" of the women who brought the action.
Worboys, 60, has served 10 years, including remand time, of an indeterminate prison sentence.
"A lot of women will be sleeping a lot easier tonight," one victim said.
The victim – known as DSD – said she was thrilled with the result and was confident the Parole Board would "make the right decision next time because the information will be put in front of them".
Mr Worboys' case will now be referred back to the Parole Board, where a new panel will make a "fresh determination".
In Mr Hardwick's letter of resignation, he said he was "sorry for the mistakes that were made in this case", adding that Justice Secretary David Gauke had told him his position was now "untenable".
That the Parole Board did not take into account the dozens of alleged offences that Worboys was not prosecuted for was acknowledged by the High Court.
But barrister Phillippa Kaufmann QC, who represented the two victims, said the problem did not lie "solely at the feet of the Parole Board". She said Mr Hardwick had been "scapegoated" for wider failings in the system.
After the High Court hearing Ms Kaufmann said: "That material [on Worboys' alleged offences] was not put in by the Ministry of Justice itself.
"So Gauke has a responsibility here; the Ministry of Justice has a responsibility here," she said.
Worboys, who is now known as John Radford, was convicted of one rape, five sexual assaults, one attempted assault and 12 drugging charges – but police believe he committed crimes against more than 100 women between 2002 and 2008.
In November, the Parole Board decided to approve his release with "stringent" licence conditions, arguing its decision was based on appropriate evidence.
But the High Court judges said the Parole Board "should have undertaken further inquiry into the circumstances of his offending".
The case will now be referred back to the Parole Board and considered by a new panel, while Worboys remains in prison.
Justice Secretary Mr Gauke congratulated the victims on the "unprecedented legal action" and announced plans to make the Parole Board system more transparent.
Responding to a question at PMQs, Prime Minister Theresa May said the ruling raised "serious concerns" and agreed the victims should never have had to face their attacker in court during their legal challenge.
Mrs May also said she had full confidence in Mr Gauke, who had decided it "would not be appropriate" for the government to seek a judicial review earlier in the year.
But shadow justice secretary Richard Burgon has called for an independent review of the Parole Board's handling of the case.
He added: "It can't be right that women victims have to go to judicial review before the reasons for the release of John Worboys became available."
London Mayor Sadiq Khan, who brought the challenge with the two victims, said the decision would bring "some reassurance to the victims and to all Londoners".
At a court hearing this month, lawyers for Worboys' victims and Mr Khan had argued the former cab driver had been dishonest with the Parole Board and had crafted an account to convince the panel he was a changed man.
- Victims told Worboys would serve life-sentence
- Worboys review: What we have learned
They said the "wider allegations" against Worboys should have been taken into account.
During his original trial, jurors heard Worboys – who became known as the black-cab rapist – had picked up his victims in London's West End and given them champagne laced with sedatives, claiming he had won the lottery or had won money at casinos.
Analysis by BBC home affairs correspondent, Danny Shaw
This is a landmark ruling that will shape the way future parole hearings are conducted.
It makes clear that when assessing a prisoner's suitability for release, details of alleged offences, as well as convictions, can be considered – as long as they're dealt with fairly.
In the John Worboys case, the judges were clearly concerned that allegations for which he was not prosecuted weren't addressed by the panel that directed his release.
As a result, his apparent "honesty" wasn't subjected to sufficient scrutiny.
It remains something of a puzzle why the psychologists and parole panel members didn't examine those issues. Were they confused about the legality of doing so? Or were they taken in by the arch manipulator, Worboys?
Some may regard it as unfortunate that Nick Hardwick, who'd steered through changes to reduce parole delays, won't be in place to see through the further reforms necessary.
But the high-profile nature of the case, and the clear failings highlighted in the judgment, made it difficult for him to continue in such a febrile atmosphere.
In a further move on Wednesday, the High Court judges found the rules which stop Parole Board decisions from being made public were unlawful.
It came after the Sun newspaper, along with Mr Khan and the victims, also challenged the Parole Board's lack of openness and its failure to give reasons for its decisions.
Following the judgement, Mr Gauke said such rules would be abolished.
A spokesman from the Sun praised it as "a victory for transparency and for the free press".
The Victims' Commissioner, Baroness Newlove, also said the parole process should be more victim-focussed.
Conservative MP Zac Goldsmith praised the victims who brought the case forward, but said the decision "must now provoke a rethink of a criminal justice system" in which many lacked "confidence".
Fellow Conservative Dominic Grieve said some aspects of the case were "quite astonishing".
He said it was difficult to understand why the Parole Board did not probe into Worboys' background.
But he added that because of the number of indeterminate sentences – where no date is set for a prisoner's release – the Parole Board deals with more cases than it was designed to handle.
What does this mean for the Parole Board?
Making a statement in the Commons, the justice secretary announced a host of proposals regarding the parole process:
- Parole Board decisions can be challenged and re-assessed by a judge-led panel of different Parole Board members under a new mechanism
- Rule 25 – which prohibits the details of Parole decisions from being made public – will be abolished
- All relevant evidence of past offending should be included in the dossiers submitted to the Parole Board
- Procedures to check that every dossier sent by HM Prison and Probation Service to the Parole Board contains every necessary piece of evidence
- Boost the role of the secretary of state's representative at Parole Board hearings
- Work with the Parole Board to review the composition of panels so that the Parole Board includes greater judicial expertise for complex, high profile cases
- Develop more specialist training for Parole Board panel members