In 1977 I walked through the gates of Hendon Police Training School with a cardboard suitcase, a pretty average law degree and a total ignorance as to what the next 30 years would hold for me. Nevertheless, I was optimistic and full of hope. As it turned out, I spent much of those 30 years doing my very best to put people into prison.

Asked why I thought it was a worthwhile thing to be doing, I would have talked about the need to protect the public, enforce the law, prevent more crimes – and victims.

I’m pretty sure I would never have said that it would be a good idea to hold people in conditions that would almost certainly guarantee they would emerge from prison more angry, more embittered, more violent, better schooled in the ways of crime, and more ravaged by drugs than if they had never been sent to prison in the first place.

Nor did I ever imagine that 41 years later I would be inspecting prisons and reporting, publicly and sometimes in very stark terms, but I hope in strict accordance with my statutory remit, that because of the treatment and conditions experienced by prisoners, all too often prisons are failing in one of their primary purposes – that of protecting the public.

After so many years working in law enforcement, in the inner city, international drug dealing, security and counter terrorism, I didn’t approach the role of Chief Inspector of Prisons as a dewy-eyed optimist. I do not labour under any illusion that some of the most difficult, dangerous and challenging people in our society will emerge at the end of sentences full of remorse; contrite and eager to repay their debt to society.

Nevertheless, I have always believed that prison should be a place where those who need to be there should be held safely, securely, and in decent conditions. Those who wish to do so should have the opportunity to mend their ways, learn new skills and attitudes, and those who don’t wish to do so should at least have some encouragement or incentive to change.

Sadly, my experience over the past three years has been that all too often, our prisons are failing in these basic aspirations. I have seen many prisoners who want to be better equipped to play a useful role in the community when they are released, but who are currently denied the opportunity to do so.

When the state chooses to take away a person’s liberty, it also accepts the incredibly serious responsibility for every part of that person’s life. We all, in our lives, however difficult or challenging, need hope. Prisoners more than most. Allowing them hope should be part of the state’s responsibility.

But far from giving hope, prisons are too often taking it away. Does this matter? Why should prisoners have hope for their futures, when all too often they have removed hope from their victims? Quite apart from the moral and ethical dimension of how we should treat fellow human beings, there is a very pragmatic reason why we should not extinguish hope in our jails. The things that snuff out hope in jails make it far more likely that prisoners will re-offend after their release.

I firmly believe there is an overriding public interest in holding prisoners in conditions that make it less likely they will commit more crimes. Throughout my years of trying to gather evidence that would put people into jail, I never did so with the intention that they should be held in squalid conditions, live in fear of their fellow prisoners, locked in their cells for 23 hours or more every day, with little or no access to any purposeful activity, and come out of prison with a drug habit they didn’t have when they went in. At the moment, too often, this is precisely what happens. The extraordinarily high re-offending rates speak for themselves.

Despair’s cause and effect

What is it about life in our prisons that has taken away hope? Quite simply, we are holding people in conditions that are indecent and degrading. Two or more people being held in tiny cells designed for one. Those cells, usually locked for much if not all the day, serve as the bedroom, lavatory and dining room for their occupants. I shall not describe in detail some of the more grotesque things I have seen in jails, but my recent letter to the Secretary of State invoking the so-called Urgent Notification protocol at Bedford prison, which described an inmate luring rats into his cell before killing them, actually during the course of an inspection, gives a flavour. I am very aware of the irony that Bedford prison was where John Howard started out on his journey of prison reform nearly 250 years ago. I fear he would have been as appalled as I was by what I saw there last month.

Sometimes as a cause of despair and sometimes as an effect of it, drugs are dominating life in many jails. They cause debt, bullying and violence. They are directly linked to dozens of deaths, suicides and cases of self-harm. At the prison I recently inspected, nearly a third of prisoners we surveyed said they had acquired a drug habit since entering the jail. This, sadly, is far from unique.

The violence means prisoners and staff alike live in fear. Some prisoners refuse to come out of their cells. Staff shortages mean that often they can’t be unlocked. The chance to reform or rehabilitate is lost.

I often see what feels like a vicious circle of despair. Totally unreasonable lengths of time locked in cells give rise to frustration, boredom and anger. In response, prisoners often turn to drugs – sometimes referred to as the ‘bird killer’. The drugs in turn give rise to debt, bullying and violence. Prisoners are then too afraid, or the wings too unsafe, to allow them to get training, education or employment – so they are left locked in their cells and the circle is complete. Breaking into that circle and re-injecting hope is vital. But how is that to be done? What can give hope?

Breaking the vicious circle

First, prisons above all are about people. Those who are held there and those who work there. In the past few years the numbers of staff dropped below the level at which they could offer decent, purposeful detention. Staff numbers are now rising again. What I find really encouraging is that enlightened governors see their new staff not as an inexperienced liability but as a huge opportunity: to get prisoners out of their cells, develop personal relationships, and to make jails safer.

Achieving the right balance between prisoner and staff numbers is vital. As Chief Inspector I don’t take a position on how many people should be in prison – that is a matter for policymakers – I just say that the balance between staff and prisoner numbers must be such as to allow for decent, purposeful detention.

Secondly, living conditions need to be improved, and the squalor we see all too often done away with. This will take time, but there is a growing realisation that degrading conditions are neither sensible nor inevitable.

Thirdly, the large number of prisoners suffering from one form or another of mental illness need to receive proper care – very often not available in a prison. Many should not be in prison at all – but the lack of alternatives has often left the courts with little or no choice.

Fourthly, we need to find a way of providing appropriate security and proper care for the rapidly growing population of older prisoners.

Fifthly, we must harness the goodwill, dedication and extraordinary generosity of spirit not only of prison staff but of the voluntary sector. The levels of commitment, care, courtesy and compassion that I see in prisons is often humbling. It is people who can forge the type of relationships that too many prisoners have never enjoyed. Many have never experienced another human being caring for them. It is people who can bring hope back to so many who thought they had none.

So, if 41 years ago today I was optimistic about policing, today I am cautiously optimistic about our prisons. Of course, there is much that is terribly wrong. Not all of our prisons are or have been in crisis. But many are still in a very dark place. I have seen things that I never thought I would see in institutions being run by the government for the public benefit. But I can just begin to see some shafts of light at the end of what has been a long, dark tunnel.

We need to keep hope alive in our prisons. Hope for prisoners that they will at least have the opportunity to turn their lives around. Hope for staff that they can achieve more than simply keeping the lid on a bubbling pot of despair and disorder. Hope that they can achieve what so many joined the prison service to achieve – to make a real and positive difference to the lives of prisoners. There is hope, I am sure. And I hope I am right.

Peter Clarke is HM Chief Inspector of Prisons. This article is based on an address to Temple Church, Sung Evensong on 10 October 2018.