Autism Research Via Smartphone

One of the most effective means of investigating and understanding autism is eye tracking. Participants are shown photos or videos, and computer software records where their gaze rests. Autistic individuals are more likely to focus on nonsocial aspects of an image, such as objects or background patterns, while neurotypical subjects have an increased propensity to focus on people's faces.

Ralph Adolphs, the Bren Professor of Psychology, Neuroscience, and Biology and an affiliated faculty member of the Tianqiao and Chrissy Chen Institute for Neuroscience, has been researching autism for decades as part of a larger project aimed at understanding the neuroscience of human social behavior. In his Emotion and Social Cognition Lab, researchers get a finer grasp on the mechanics of the brain when processing emotion and interacting with others by studying both neurotypical individuals and those who have brain damage or brain malformations or who have neuropsychiatric conditions such as obsessive compulsive disorder (OCD) or autism spectrum disorder (ASD).

Autism is a particularly rich field for research into emotion and social cognition since it is characterized by, among other things, differences in social behavior. Adolphs has been exploring its features by bringing adults with autism into the lab to track their eye movements when they are exposed to a variety of visual stimuli.

This research has yielded many interesting findings but has been inherently limited by the expense of laboratory eye tracking technology. "Eye tracking is a sensitive measure that gives us insight into some cognitive processes that are thought to be different in autism," Adolphs explains. "But previous studies have required a desktop eye tracker, which can cost $30,000 or $40,000, and the assistance of a graduate student or postdoc to calibrate the equipment and set up the research subjects for tests. It's very time intensive and money intensive research."

Adolphs and others have asked whether smartphones, which are able to display images and video and use camera technology to record, display, or share elements of the user's face or environment, might be able to capture the same information that established eye-tracking technology already does but at considerably less expense.

In a recent proof-of-concept study, Adolphs's lab recruited participants with and without autism spectrum disorder to undergo eye-tracking experiments, first with established desktop eye-tracking technology (the Tobii Pro Spectrum eye tracker), then with smartphone eye tracking administered in the lab with the assistance of researchers who adjust the smartphones and the participants' angle of view, and finally with the same participants participating in eye-tracking experiments at home via smartphone. Impressively, similar results were found across all three modalities.

This holds enormous promise for research into autism. "If we can only get a dozen or 20 people into the lab at Caltech at a time, our sample size is obviously limited," Adolphs says. "Not only that, it is biased: Participants are people who can travel on their own, who are in the Los Angeles area, and who are high functioning. With smartphones we can scale research to much larger sample sizes and include participants from underserved communities. This will help us get a much better understanding of the features of autism."

There is a saying about autism: "If you've seen one person with autism, you've seen one person with autism." Because the characteristics of autism can vary so widely, small sample sizes limit the conclusions that can be drawn from research. Adolphs hopes that with the implementation of smartphone eye-tracking technology and the larger sample sizes it permits, "we will have the statistical power to look at a lot of questions about autism. There might be two or three or four or a dozen different types of autism that can be identified, which could greatly improve diagnosis and treatment for autistic individuals."

Smartphone eye tracking can also benefit autism research by enabling longitudinal studies—those that collect data about specific individuals over a longer period of time. "Sometimes, when a person comes into the lab for a study, they're nervous, or they've taken a medication, or they haven't slept well the night before, and all these things can give results that might be quite different on another day," Adolphs says. "It's too impractical for people to come to the lab repeatedly, but if they can perform these tests at home alone, week after week, we can establish a baseline and then document changes due to development, treatment, or aging."

"There are still a lot of practical hurdles with smartphone research," Adolphs cautions. "Getting people to reliably do these tests is not trivial. They have to remember to do it, follow instructions, hold the phone in a certain way, and upload the data to us."

A potentially bigger problem concerns privacy, especially as this technology is commercialized, as it inevitably will be (and to some extent, already is). "If you have an app that advertises itself as diagnosing or tracking autism, it will take a video of your face, send it to some machine on the internet, and then give you results," Adolphs explains. "This is identifiable data. Whoever gets the video can tell who you are." In Adolphs's study, videos of participants were cropped to show only the eyes. "The data became immediately anonymized," Adolphs says.

These difficulties notwithstanding, Adolphs's experiments with smartphone eye tracking "have the potential to scale sample size by several orders of magnitude and include participants from all over the world."

"Smartphone-based gaze estimation for in-home autism research" was published in the journal Autism Research. The authors on the paper include postdoc Na Yeon Kim and graduate student Qianying Wu, who together led the study, as well as co-authors Jasmin Turner, Lynn K. Paul, and Adolphs of Caltech; Daniel P. Kennedy of Indiana University; and Junfeng He, Kai Kohlhoff, Na Dai, and Vidhya Navalpakkam, who all work for Google Research in Mountain View, California, and who were responsible for analyzing the data. The study was funded by the Della Martin Foundation, the Simons Foundation Autism Research Initiative, and Google.