Unleashing the Krakencoder

The early history of neuroscience research teems with mind-altering incidents that illuminated how the brain operates. Like Phineas Gage, a railroad worker who underwent an instant personality change when an explosion sent a meter-long iron rod through his left frontal lobe. Or HM, a young man whose hippocampus was removed to treat his intractable epilepsy — leaving him completely unable to form new memories of people, places or events.

These legendary accounts provided tantalizing evidence that different brain regions govern specific functions — from social interaction to memory consolidation. But scientists are still learning about how these regions interact to guide how we think and behave.

The modern approach to mapping the brain’s neural circuitry relies more on imaging machines and computers than on brain-penetrating projectiles. But the work is no less adventurous. Researchers at Weill Cornell Medicine are using innovative computational tools — including a program they call the Krakencoder — to analyze treasure troves of imaging data in an effort to comprehensively chart the brain’s structural and functional connections.  

“Seeing how these anatomical and physiological connections map to behavior is crucial for understanding the mechanisms of disease and designing interventions that will support better outcomes,” says Dr. Amy Kuceyeski, professor of mathematics in radiology and in neuroscience at Weill Cornell Medicine.

Unleashing the Krakencoder

To get a more comprehensive representation, Dr. Kuceyeski and her team built a tool that could take the structural and functional connectomes produced by all of these disparate approaches and collapse them together to produce a more unified interpretation.

Enter the Krakencoder. “In my head, I saw it as some sort of monster with multiple arms that could reach out and grab different brain representations and digest and congeal them into one unified connectome,” says Dr. Keith Jamison, a staff associate in Dr. Kuceyeski’s lab. The resulting program, an autoencoder that compresses and reconstructs more than a dozen different “flavors” of input data, thus came to be called the Krakencoder, in honor of the mythical, multitentacled sea creature.

The team trained the Krakencoder on data collected from 700 participants in the NIH’s Human Connectome Project (HCP). As part of that inaugural study, volunteers between the ages of 22 and 35 underwent extensive structural and functional MRI scanning.

The researchers found that the Krakencoder allowed them to take an individual’s structural connectome and correctly generate that person’s functional connectome (and vice versa) about 20 times more accurately than previously published approaches. Their findings appeared in June in the journal Nature Methods.  

The Krakencoder’s combined and compressed representation also allowed the team to predict the individual’s age, gender and the scores received on a battery of memory and cognitive tests administered along with the imaging scans. These scores, Dr. Kuceyeski notes, are notoriously difficult to gauge based on brain imaging alone.

Such information can be critical for assessing how disease or injury affects brain function. For example, Christie Gillies, a student in the Kuceyeski lab, has found that the functional connectome produced by the Krakencoder does a better job at predicting an individual’s motor and languages scores following a stroke than does looking at the person’s original fMRI.

Making those connections is something that artificial intelligence (AI) is really good at. “Just like ChatGPT predicts the next word in a sentence, AI can take high-dimensional connectome data extracted from MRIs and predict some outcome, like an individual’s cognitive score,” says Dr. Kuceyeski.