A self-supervised transformer model for property prediction

For decades, metal-organic frameworks (MOFs) have been captivating researchers because of their wide range of applications: gas absorption, water harvesting, energy storage and desalination. Until now, quickly and inexpensively selecting the top performing MOFs for specific tasks has been challenging. Enter MOFormer, a machine learning model that can achieve higher accuracy on prediction tasks than leading models without explicitly relying on 3D atomic structures.

“We recognized that relying on the 3D structures of MOFs led to the additional cost. To work around this we used MOFids to make accurate predictions,” explained Yuyang Wang, a Ph.D. student in Professor of Mechanical Engineering Amir Barati Farimani’s research group.

This research was first published in The Journal of the American Chemical Society.

A MOFid is a text string representation of MOFs building blocks—a combination of metal nodes, organic linkers and topologies—that enables machine learning models to output property predictions. Because of the countless combinations of these building blocks, discovering optimal MOFs is complicated. In Barati Farimani’s MOFormer, researchers can perform faster screening of MOFs by hypothetically creating new MOFids.

“To train MOFormer, we use self-supervised learning (SSL), which leverages both the structure based approach with graph neural network and structure agnostic approach with MOformer. SSL improves the performance of MOFormer on downstream property prediction tasks. Using such an approach. we explore how structure based and structure agnostic approach can utilize mutual information from each other,” explains Rishikesh Magar, a Ph.D. student in Barati’s lab.

“This paves the way for researchers to build MOFs more efficiently. Say there are one million hypothetical MOF structures that might seem like a fit for water harvesting. The MOFormer can narrow them down by predicting which will have the most desirable properties. From there, researchers may only need to test 100 variations in the lab,” Zhonglin Cao, a Ph.D. student in Barati’s group, illustrated.

Compared with other structure-agnostic methods, MOFormer achieves 35-48% higher accuracy on various gas absorption prediction tasks, and 21.4% higher on band gap prediction.

“We expect the MOFormer to expedite the exploration of MOFs,” said Farmini. “It can serve as a tool for exploring the vast chemical space of hypothetical MOFs.”