Discover SPARK

Are Digital Tools the Future of Autism Research?

Carolyn Sayre

Children and adults diagnosed with autism spectrum disorder (ASD) have difficulty with communicating and social interactions and display repetitive behaviors. While great progress has been made in finding the genetic causes of ASD, researchers still do not have a great way to measure these key symptoms.

In other conditions, the telltale signs are easier to observe, count, and analyze. For example, in epilepsy, physicians can record abnormal electrical activity in the brain. However, in autism, behaviors such as anxiety or speech deficits are much harder to measure. Finding ways to quantify these features will help researchers develop new drugs and therapies that improve the quality of life for individuals with autism and their families.

Researchers working in other areas of health care have started using digital tools such as smartphones and computers to test ideas about the mind and brain. For example, they are able to record and analyze videos and voices to help screen and describe individuals with such conditions as depression, schizophrenia and Parkinson’s disease. Could that same technology transform how researchers measure behaviors in ASD?

Leading researchers and technologists came together to discuss this question at a workshop in February of 2018 hosted by the Simons Foundation Autism Research Initiative (SFARI), the organization that funds Simons Foundation Powering Autism Research (SPARK). “The goal was to bring together experts who have worked out interesting technologies, many of them not in the autism field, so we can begin to develop a framework for measuring autism with digital tools,” says Pamela Feliciano, Ph.D., scientific director for SPARK.


Typically, when autism researchers collect data for a study, a trained clinician uses a test such as the Autism Diagnostic Observation Schedule (ADOS) to look for autism behaviors. The clinician might observe the amount that the child smiled or made eye contact. These tests take place at a clinic or doctor’s office, even though the range of behaviors exhibited by the person with autism in that type of setting might be very different from his or her behavior at home.

However, research that uses digital tools has the opportunity to be more objective, accessible and realistic because it can take place in the individual’s everyday environment at home, school or work. Many of the technologies being developed are known as passive digital tools because they are downloaded to the user’s phone, sit in the background and do not require interaction from the user.

Health care has been one of the last industries to adapt to the digital age. But consider the exponential growth in smartphone users over the past decade. “It would be silly not to think about this as a recipe for what could happen in the way that we describe behavior, cognition and affect,” says Thomas Insel, M.D., psychiatrist, neuroscientist and co-founder and president of Mindstrong Health.

All you have to do is reach into your pocket. There are tons of sensors already built into smartphones that can monitor such things as touch, light and noise. Once this information is collected, it can tell us things about the phone’s user. For example, keyboard strokes can give us clues about an individual’s attention, voice recordings can help us understand how well someone speaks, and location trackers can determine how social is the person’s behavior.

But how exactly do researchers draw conclusions from the information collected? Welcome to the world of machine-learning algorithms, where highly specialized computer programs ingest large amounts of data and make predictions based on what they have learned. It is the same technology Amazon uses to suggest what book you might like next based on your ordering history.

Machine-learning algorithms could also help identify patterns and features in autism behaviors. Says Robert Schultz, Ph.D., director of the Center for Autism Research at Children’s Hospital of Philadelphia, “It is now possible to digitize every behavior that experts observe.”


Early studies to screen and detect mental health conditions digitally have taken interesting first steps. For example, psychiatrists know that individuals with schizophrenia display unusual speech patterns. A study published in World Psychiatry used a machine-learning computer to classify speech patterns in individuals with schizophrenia and was 83 percent accurate in predicting when psychosis would occur.

Similar technologies that track mood and behavior are also being developed as screening tools for individuals at risk for depression. For example, a tool called EARS (Effortless Assessment of Risk States) uses smartphone data to identify people in psychological distress and may someday help flag individuals at risk of suicide.

Another benefit of digital tools is the potential to help larger numbers of people. They can help automate tasks normally done by clinicians.

For example, in one study, a virtual human conducted interviews with real people in emotional distress. Distinct speech patterns, such as slurring vowel sounds, and patterns in body language, such as the direction someone is looking, were analyzed. If a machine learns that people who are depressed do not open their mouth as wide as someone who is not depressed, it can use speech analysis to identify people who are more likely to be depressed.

Such technology has the power to dramatically improve research and treatment. “If you ask a therapist to do this, they might only be able to capture the number of smiles. Artificial agents, such as virtual humans or robots, could help gather unbiased and massive amounts of data,” says Stefan Scherer, Ph.D., chief technology officer at Embodied, Inc., in Pasadena, California. Research published in Studies in Health Technology and Informatics found that more symptoms of post-traumatic stress disorder were identified in service members who spoke to a virtual human using facial expression analysis than in service members receiving a post-deployment health assessment.

Furthermore, motion-capture software, which digitally records patterns of movement, shows promising results. This technology is used in movies and video games to replicate human movement and make characters look more realistic. Motion-capture software is of particular interest to SPARK, since motor impairments and atypical movements are common in autism.


Digital tools have also been found to be valuable in early autism screenings. Schultz, at Children’s Hospital of Philadelphia, has presented research that uses sensors to record facial movements, including head posture and eye movement, during a three-minute get-to-know-you conversation. This tool claims to be approximately 89 percent accurate at predicting autism. Similarly, an app called Autism & Beyond, which records a child’s emotional responses and behaviors while he or she watches a short video, has helped identify more than 1,000 children with autism characteristics.

“The power is that you are giving a tool with an autism expert to people who are not autism experts,” says Guillermo Sapiro, Ph.D., Edmund T. Pratt, Jr. School Professor of Electrical and Computer Engineering at Duke University, who used the app in sub-Saharan Africa, where 99 percent of the population lacks access to a trained autism clinician.


While still in their infancy, digital tools offer endless possibilities. For autism, the path forward is likely a combination of technologies that integrate information from digital tools and traditional in-person clinic visits.

Autism treatment will never be a one-size-fits-all endeavor. But if these technologies can help identify and measure patterns in autism behaviors, researchers can more readily develop treatments for people with ASD.

Language is one of the first areas being studied that could lead to new therapies. Researchers funded by SFARI have already been using smartphones and automatic transcription software to record speech and find patterns in children with autism.

“The idea that you have treatments and diagnostics is an old idea. We want to have interventions we are learning about in real time,” says Insel, president of Mindstrong. “The opportunity to develop social prosthetics will have an enormous impact for kids with autism. We need to think about how the tools we are developing can make a real difference in the lives of people with autism.”