How Israeli algorithms spot signs of depression

A patient with depression sits down with a psychiatrist as special high-resolution cameras record their conversation. Over the next hour their words, voice and facial expressions are captured and processed by algorithms created by Israeli experts in artificial intelligence, machine-learning and natural language processing (NLP), a sub-field of linguistics. At the hour’s end the patient gets a score of four. Last month they were 3.5. It means the treatment isn’t working and the psychiatrist changes tack.

It seems otherworldly, the stuff of sci-fi, but that scenario may not be too far away, because scientists funded by the Dangoor Centre for Personalised Medicine at Bar-Ilan University in Israel are leading a global race to develop game-changing AI programmes applied to depression, and are currently busy gathering preliminary data for what they call their Depression Meter.

“We’re developing an algorithm that can identify and predict the level of depression, to be used by psychiatrists and psychologists in treatment,” says the centre’s vice-president of research Prof Shulamit Michaeli. “We run a clinic as a charity, and from this we have many hours of audio and visual recordings from sessions. The camera that captures facial expressions is very sophisticated.”

Get The Jewish News Daily Edition by email and never miss our top stories Free Sign Up

Hang on a minute, let’s go back a step… How can algorithms help when it comes to a complex psychological condition here, often seen in combination with other conditions such as anxiety or personality disorders? What can computer code offer a highly-trained psychiatrist who knows the inner workings and frailties of the human mind?

Clinical psychologists Prof Eva Gilboa-Schechtman and Dr Dana Atzil think it could be a lot. They’re using the session recordings and working with Dr Yossi Keshet, a machine learning expert who developed the IDF software that identifies keywords in conversations. Together with Bar-Ilan’s engineering department and two scientists at UCLA in California, the team are analysing patients’ speech using NLP to create a self-learning system.

Clinical psychologists analyse thousands of hours of therapy sessions with the aim that machines will predict what will work

“Depression is getting worse and worse in the modern world,” says Atzil, speaking to Jewish News. “More than 30 million Europeans are diagnosed, 260 million around the world.” Yet despite advances, recovery rates are still just 50 percent. In part, she says, that’s because there have been no big breakthroughs in this area for the last 40 years.

“Treatments are mainly based on the building blocks of psychotherapy, such as self-reports and clinician assessments, but they’re not enough,” says Atzil. “They’re subjective – clients don’t always know how to capture what they feel in a certain moment – and clinician assessments are expensive. For 40 years we’ve been in the same situation, not improving.”

The treatments are numerous – psychodynamic, cognitive behavioural therapy, medications etc – but “all get similar results,” she says. “At the same time we know that there is a lot of variability between clients. They react differently to different treatments, so it’s important to capture what works for whom and when. It’s not just one disease. It’s complicated and we need a method to capture that complexity.”

It sounds like an impossible task, but they’re convinced it isn’t as they use novel techniques and apply it to AI and machine-learning to analyse “a very unique data set”.

Atzil says: “We transcribed sessions with 500 patients over five years, took physiological data, plus data before treatment, such as diary data, where clients write five times a day how they feel, so we have a lot of information on triggers and reactions, such as self-criticism or comments from a spouse that they can find insulting.

Jewish News editor discusses this week’s mental health issue of the newspaper

“It’s a very rich, emotional data-set including text, voice and facial expressions and spans pre-treatment to post-treatment. It tells us a lot about diagnosis and fluctuations in symptoms, which is very valuable. It lets us say ‘this is how the person looks and sounds when they’re depressed, this is what the person says when they’re depressed.’”

Fine, but they’re still asking a machine to capture subtle human nuance. “Machine-learning can do that,” says Atzil. “Just as when you talk to your loved one and pick up that something’s not right, so the machine also picks up on differences in the human voice or facial features to tell if this person is distressed.”

Elements of the voice that the team has analysed include pitch, energy and quality. “All these features, when integrated together, can capture nuanced differences over time,” says Atzil. “It tells us if the client is getting better within the session. It also tells us about the interaction between the client and therapist, how synchronised they are, and whether the therapist is helping the client to feel better.

“Some depressed clients feel flat or detached from their emotions. Others feel extremely negative emotions. Both experience depression in very different ways, so what helps will also be different. For emotionally-aroused clients, therapists help to down-regulate their emotions, but for those who feel flat, therapists need to feel it for them, to help up-regulate their emotions. It’s different processes at work within the session.”

Watching them at work is fascinating. The team code the session moment-by-moment, providing labels for each speech turn, interaction and intervention. “We can train the machine to learn the codes, learn how specific moments are moments of change, and learn what intervention the therapist enacted earlier to bring about that change,” says Atzil. “It learns from thousands of thousands of sessions then predicts what will work in any specific moment.”

It’s all still work-in-progress, but they’re now using deep-learning models to find “signatures of depression,” says Michaeli. The results feed back into the algorithm which updates and improves it. “This truly is machine-learning applied to health.”

This is just one of the projects supported by British Jewish philanthropists David and Judy Dangoor, the family interest in the centre having first begun with David’s father Naim, but it is among the most promising. Once the team have enough preliminary data to show “proof of concept” they hope to land a bigger private donation or a grant from the EU or Israeli government. Either way, it’s a 6-7 year task.

“David’s funding is a glue grant,” says Michaeli. “It glues interdisciplinary scientists together around a major health programme, in this case using AI and NLP in the treatment of depression. We hope to initiate collaborations with major Israeli medical centres such as Sheba and Hadassah.”

The Israeli team is not alone in working on AI-enabled depression diagnosis, but Michaeli is not worried. “We have a saying in Hebrew, that competition makes more wisdom. When you find people competing, it shows that it is an important question and a major need.”

Support your Jewish community. Support your Jewish News

Thank you for helping to make Jewish News the leading source of news and opinion for the UK Jewish community. Today we're asking for your invaluable help to continue putting our community first in everything we do.

For as little as £5 a month you can help sustain the vital work we do in celebrating and standing up for Jewish life in Britain.

Read more

Jewish News holds our community together and keeps us connected. Like a synagogue, it’s where people turn to feel part of something bigger. It also proudly shows the rest of Britain the vibrancy and rich culture of modern Jewish life.

You can make a quick and easy one-off or monthly contribution of £5, £10, £20 or any other sum you’re comfortable with.

100% of your donation will help us continue celebrating our community, in all its dynamic diversity...

Engaging

Being a community platform means so much more than producing a newspaper and website. One of our proudest roles is media partnering with our invaluable charities to amplify the outstanding work they do to help us all.

Celebrating

There’s no shortage of oys in the world but Jewish News takes every opportunity to celebrate the joys too, through projects like Night of Heroes, 40 Under 40 and other compelling countdowns that make the community kvell with pride.

Pioneering

In the first collaboration between media outlets from different faiths, Jewish News worked with British Muslim TV and Church Times to produce a list of young activists leading the way on interfaith understanding.

Campaigning

Royal Mail issued a stamp honouring Holocaust hero Sir Nicholas Winton after a Jewish News campaign attracted more than 100,000 backers. Jewish Newsalso produces special editions of the paper highlighting pressing issues including mental health and Holocaust remembrance.

Easy access

In an age when news is readily accessible, Jewish News provides high-quality content free online and offline, removing any financial barriers to connecting people.

Voice of our community to wider society

The Jewish News team regularly appears on TV, radio and on the pages of the national press to comment on stories about the Jewish community. Easy access to the paper on the streets of London also means Jewish News provides an invaluable window into the community for the country at large.

We hope you agree all this is worth preserving.

Help us here