The Ministry of Justice (MOJ) must also "accept its role" in the decision to free rapist John Worboys, according to the former Parole Board chairman.
Nick Hardwick quit after the High Court quashed the board's decision to release Worboys nine years after he was jailed.
Professor Hardwick said the MOJ had not given details of other allegations.
A review of at least six other cases has now been ordered by the MOJ. It has also apologised for not providing the information "it should have done".
Worboys, who is now known as John Radford, has served 10 years, including remand time, of an indeterminate prison sentence.
In November, the Parole Board decided to approve his release with "stringent" licence conditions, arguing its decision was based on appropriate evidence.
However, following a legal challenge by two victims and the mayor of London, High Court judges overturned the decision on Wednesday, saying the board "should have undertaken further inquiry into the circumstances of his offending".
Justice Secretary David Gauke then outlined a series of changes to the Parole Board system, saying the court's findings had highlighted "serious failings".
Speaking to BBC Radio 4's Today programme on Thursday, Prof Hardwick said the board had "got it wrong" by not considering other alleged offences – but that it had been "the widely-held practice at the time".
"I thought at the time it was correct that we couldn't go into the alleged offences (Worboys) hadn't been convicted of."
Prof Hardwick, who was forced to quit in the wake of the High Court ruling, refused to accept he had been "scapegoated", as claimed by lawyers representing Worboys' victims.
But he added: "I accept the Parole Board was at fault, I don't accept we were any more at fault than the MOJ.
"I don't think the right lessons will be learned if the only people accepting responsibility is us."
'Failings were serious'
Analysis BBC home affairs correspondent Danny Shaw
Nick Hardwick has a long and distinguished record working in criminal justice and there is force in his argument that he would have been well placed to steer through the changes the Parole Board needs to make.
And he's right to say the Ministry of Justice has questions to answer about the information it failed to supply to the parole panel which considered the Worboys case.
But the Parole Board's own failings, which he has accepted, were serious.
Common sense appeared to desert the panel when it granted Worboys' release – his sudden admission of responsibility after years of denial should have raised enough alarm bells on its own.
And it's deeply concerning that, as its then chairman, Prof Hardwick didn't know the correct position as to whether alleged offending, as well as convictions, could be taken into account when assessing risk.
It raises the possibility that other prisoners have been released who pose more of a threat than the Parole Board had realised.
Prof Hardwick said Mr Gauke should not resign but, as the Parole Board "accepts its role in this, so should others".
Prof Hardwick also said the High Court judgement had "turned" on the fact that other allegations were not considered, and though the judges had "some doubts" about the decision to free Worboys, the judges did not believe the Parole Board's conclusion to release the rapist had been "irrational".
The dossier provided to the panel by the MOJ "did not contain sufficient information about those other alleged offences and therefore they were not considered", he added.
Mr Gauke apologised for the information not being included but said the Parole Board had been "insufficiently inquisitive".
An MOJ spokesman said: "The department's representatives at the hearing made clear that they had analysed the offences of which (Worboys) had been convicted and opposed release on the basis that he remained high risk.
"They did not feel the need to analyse any other offending in order to reach this firm conclusion."
Prof Hardwick also said an independent review had found the probation service rather than the Parole Board was to blame for not keeping victims informed of the decision to release Worboys.
Marie McCourt, mother of Helen McCourt who was murdered in an unrelated case in 1988, told BBC Breakfast she too had been frustrated by a lack of communication from the Parole Board dealing with her daughter's killer.
Worboys will remain in prison while a "fresh determination" is carried out by a new Parole Board panel.
Mr Gauke has now also ordered a review of cases in which other prisoners have been released directly from high security prisons.
The Parole Board said there were six Category A offenders released in the 12 months up to April 2017.
Who is David Gauke?
David Gauke became the sixth justice secretary in six years – and the first solicitor to take the role – in Theresa May's January 2018 reshuffle.
His appointment breaks a run of four consecutive non-legally qualified MPs to hold the position of the government's chief law officer.
He was previously work and pensions secretary and chief secretary to the Treasury.
The former City lawyer has been the MP for Hertfordshire South West since 2005.