Nothing humbles history’s great thinkers more quickly than reading their declarations on the causes of madness. Over the centuries, mental illness has been attributed to everything from a “badness of spirit” (Aristotle) and a “humoral imbalance” (Galen) to autoerotic fixation (Freud) and the weakness of the hierarchical state of the ego (Jung).

The arrival of biological psychiatry, in the past few decades, was expected to clarify matters, by detailing how abnormalities in the brain gave rise to all variety of mental distress. But that goal hasn’t been achieved — nor is it likely to be, in this lifetime.

Still, the futility of the effort promises to inspire a change in the culture of behavioral science in the coming decades. The way forward will require a closer collaboration between scientists and the individuals they’re trying to understand, a mutual endeavor based on a shared appreciation of where the science stands, and why it hasn’t progressed further. One reason the psychiatric field struggles is due to anti-psychiatry views held by individuals. BetterHelp.com has a great article on how this mentality can do more harm than good.

“There has to be far more give and take between researchers and the people suffering with these disorders,” said Dr. Steven Hyman, director of the Stanley Center for Psychiatric Research at the Broad Institute of MIT and Harvard. “The research cannot happen without them, and they need to be convinced it’s promising.”

The course of Science Times coincides almost exactly with the tear-down and rebuilding of psychiatry. Over the past 40 years, the field remade itself from the inside out, radically altering how researchers and the public talked about the root causes of persistent mental distress.

The blueprint for reassembly was the revision in 1980 of psychiatry’s field guide, the Diagnostic and Statistical Manual of Mental Disorders, which effectively excluded psychological explanations.

Gone was the rich Freudian language about hidden conflicts, along with the empty theories about incorrect or insufficient “mothering.” Depression became a cluster of symptoms and behaviors; so did obsessive-compulsive disorder, bipolar disorder, schizophrenia, autism and the rest.

This modernized edifice struck many therapists as a behavioral McMansion: an eyesore, crude and grandiose. But there was no denying that the plumbing worked, the lighting was better, and the occupants had a clear, agreed-upon language.

Researchers now had tidier labels to work with; more sophisticated tools, including MRIs, animal models, and genetic analysis, to guide their investigations of the brain; and a better understanding of why the available drugs and forms of psychotherapy relieved symptoms for many patients.

Science journalists, and their readers, also had an easier time understanding the new vocabulary. In time, mental problems became mental disorders, then brain disorders, perhaps caused by faulty wiring, a “chemical imbalance” or genes.

But the actual science didn’t back up those interpretations. Despite billions of dollars in research funding, and thousands of journal articles, biological psychiatry has given doctors and patients little of practical value, never mind a cause or a cure.

Nonetheless, that failure offers two valuable guideposts for the next 40 years of research.

One is that psychiatry’s now-standard diagnostic system — the well-lighted structure, with all its labels — does not map well onto any shared biology. Depression is not one ailment but many, expressing different faces in different people. Likewise for persistent anxiety, post-traumatic stress, and personality issues such as borderline personality disorder.

As a result, the best place for biological scientists to find traction is with individuals who have highly heritable, narrowly defined problems. This research area has run into many blind alleys, but there are promising leads.

In 2016, researchers at the Broad Institute found strong evidence that the development of schizophrenia is tied to genes that regulate synaptic pruning, a natural process of brain reorganization that ramps up during adolescence and young adulthood.

“We are now following up hard on that finding,” said Hyman. “We owe it those who are suffering with this diagnosis.”

Scientists also foresee a breakthrough in understanding the genetics of autism. Dr. Matthew State, chief of psychiatry at the University of California, San Francisco, said that in a subset of people on the autism spectrum, “the top 10 associated genes have huge effects, so a clinical trial using gene therapies is in plausible reach.”

The second guidepost concerns the impact of biology.

Although there are several important exceptions, measurable differences in brain biology appear to contribute only a fraction of added risk for developing persistent mental problems. Genetic inheritance surely plays a role, but it falls well short of a stand-alone “cause” in most people who receive a diagnosis.

The remainder of the risk is supplied by experience: the messy combination of trauma, substance use, loss and identity crises that make up an individual’s intimate, personal history. Biology has nothing to say about those factors, but people do. Millions of individuals who develop a disabling mental illness either recover entirely or learn to manage their distress in ways that give them back a full life. Together, they constitute a deep reservoir of scientific data that until recently has not been tapped.

Gail Hornstein, a professor of psychology at Mount Holyoke College, is now running a study of people who attend meetings of the Hearing Voices Network, a grassroots, Alcoholics Anonymous-like group where people can talk with one another about their mental health struggles.

Many participants are veterans of the psychiatric system, people who have received multiple diagnoses and decided to leave medical care behind. The study will analyze their experiences, their personal techniques to manage distress, and the distinctive characteristics of the Hearing Voices groups that account for their effectiveness.

“When people have an opportunity to engage in ongoing, in-depth conversation with others with similar experiences, their lives are transformed,” said Hornstein, who has chronicled the network and its growth in the United States. “We start with a person’s own framework of understanding and move from there.”

She added: “We have underestimated the power of social interactions. We see people who’ve been in the system for years, on every med there is. How is it possible that such people have recovered, through the process of talking with others? How has that occurred? That is the question we need to answer.”

To push beyond the futility of the last 40 years, scientists will need to work not only from the bottom up, with genetics, but also from the top down, guided by individuals who have struggled with mental illness and come out the other side.

Their expertise is fraught with the pain of having been misunderstood and, often, mistreated. But it’s also the kind of expertise that researchers will need if they hope to build a science that even remotely describes, much less predicts, the fullness of human mental suffering.