Carnegie Mellon University

Machine learning's Assistant Professor Leila Wehbe Portrayed

December 03, 2019

Wehbe Works Machine Learning Magic To Understand How We Think

By Aisha Rashid (DC 2019)

Byron Spice
  • Director of Media Relations
  • 412-268-9068

Readers around the world adore J.K. Rowling's Harry Potter series for the fascinating world of Hogwarts, its lively characters and vivid storytelling. And that's exactly why the first book of the series, "Harry Potter and the Sorcerer's Stone," was the perfect stimulus for Leila Wehbe to use in her studies to better understand how the brain interprets language.

But first, a little background.

Wehbe, currently an assistant professor in CMU's Machine Learning (ML) Department, has always been fascinated by the human brain. After studying electrical engineering at the American University of Beirut, she decided to combine her passions for neuroscience and computational studies at CMU, where she began a six-year journey to earn her Ph.D. in machine learning in 2009.

"Coming to the ML Department was perfect for me, especially when I met my advisor, Tom Mitchell, because he was working on exactly what I was interested in — the intersection between machine learning and the neuroscience of language," Wehbe said. "I was not just in the Machine Learning Department, but could expand my interests through the Center for the Neural Basis of Cognition (CNBC). I could take more courses about the brain, get to know more researchers who share similar interdisciplinary research interests, and work on different aspects of research and analysis."

Wehbe's research interests posed huge logistical challenges, though.

"Fields like vision have been studied for a long time in animals, where one could perform different invasive procedures," she said. "But the challenge with studying language is that you can only study humans, and, for good reason, we can't just open up the brains of humans and record from there (except in rare medical situations). We're limited to noninvasive methods."

To circumvent this challenge, Wehbe relies on two cutting-edge techniques that help her observe human brain activity: functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG).

The first method uses magnetic fields and radio waves to provide a detailed image of blood and oxygen flow through the brain.

"fMRI is an indirect way to observe the changes in oxygen levels due to brain activity," Wehbe said. "The oxygen levels change slowly, however, so we can't track them quickly. The response takes a few seconds to rise to a peak, even though we think and use language at a much faster time scale."

MEG is a more direct method than fMRI. While fMRI indirectly reflects brain activity through oxygen levels, MEG provides signals directly related to the electrical activity of neurons — specialized cells in the nervous system used to communicate and transmit information. But while these two methods differ in their mechanisms, they are both important in mapping the fast-paced activity of the brain, especially in the case of language processing. According to Wehbe, language is a complex process in that it interacts with many other cognitive domains, such as visual imagery, planning or social reasoning. So studying language could be a window to understanding the brain basis of these cognitive domains.

"If you look at an individual word, like 'apple,' for instance, it has a specific meaning to us, and there are multiple theories about how that meaning is stored in our brain," she said. "One theory is that the meaning of the word 'apple' is based on how we interact with the apple. If you look at the apple, specific areas of your brain are activated that may be associated with the color red, or the taste of something sweet."

"And from words to sentences, you start going broader and broader until you end up engaging the brain areas you use to interact with the world in everyday life. When you read, you start taking the perspective of story characters, or guessing the outcome of certain events. And it seems that you use the same brain areas when you perform these tasks while reading as you do when you perform them naturally in everyday life."

Which brings us back to Harry Potter.

In 2014, Wehbe and then-advisor Mitchell created a study to simulate this everyday life scenario through storytelling and observe how participants' brains were activated when they interacted with fictional characters. They performed fMRI scans of eight participants as they read a chapter of "Harry Potter and the Sorcerer's Stone." In the first iteration of the study, the researchers tried to observe all of the possible information the participants could have associated with the story, like the syntactic structure of the sentences, how many letters were in each word, and more complex associations like which characters were being mentioned and how they were feeling.

They then used the corresponding brain activity to develop a computational model that expressed the features of the text at any given point as a function of what was going on in the brain. Wehbe and Mitchell could then use the model to predict what the brain activity would look like for passages the readers hadn't yet read. The model also reported other key features of the text like syntax, semantic properties, narrative text, and which brain regions modulate the activity of those features. The robustness of these findings was tested using statistical tools.

"The model helps us detect where exactly natural language processing increases brain activity, and provides insight into what type of information is encoded in each one of the regions of the brain that respond to language," Wehbe said.

Wehbe believes computer modeling and statistical analysis provide significant insight into the brain's robust ability to process language. She's especially grateful for her time at CMU for giving her an avenue to explore this interaction.

"It's made me see more of a connection between the brain and machine learning," she said. "The ability to work with people from so many different departments and having the freedom of collaboration makes it a productive environment to explore these different ideas and interactions."

And this is only the beginning of Wehbe's journey to unlock the inner workings of the human brain. More recently, she has been interested in exploring whether brain activity of people reading natural text could help design better artificial intelligence algorithms that can solve language tasks, such as translating text from one language to another. Artificial intelligence algorithms do not yet understand language, and the hope is that since the human brain is the only system that does understand language, brain activity recordings could provide insight into how to design better algorithms.

Wehbe is also interested in using her approach for medical applications. She believes her research provides the groundwork to explore cognitive disorders and their treatments.

"Based on our results across different subjects, we can still see that there's a high level of consistency between one subject and another. In general, this means that healthy brains are similarly organized," she said. "It might be possible to find hidden patterns in how our brains organize information, which can help diagnose and suggest a specific way to treat a disorder. Because the brain is a complex machine, you might not be able to just fix all of the parts of the machine, but vary one part at a time and understand how the machine works. A promising alternative approach is to perform rich, natural interactive tasks in the scanner and observe how the brain works in real-life simulations, which can offer a rich picture of the different parts of the brain working together."

For More Information
Byron Spice | 412-268-9068 |
Virginia Alvino Young | 412-268-8356 |