Nature

Medium-sized protein language models perform well at transfer learning on realistic datasets

Protein language models (pLMs) can offer deep insights into evolutionary and structural properties of proteins. While larger models, such as the 15 billion parameter model ESM-2, promise to capture ...
Nature

Transformer-generated atomic embeddings to enhance prediction accuracy of crystal properties with machine learning

Accelerating the discovery of novel crystal materials by machine learning is crucial for advancing various technologies from clean energy to information processing. The machine-learning models for ...
Quanta Magazine

How ‘Embeddings’ Encode What Words Mean — Sort Of

A picture may be worth a thousand words, but how many numbers is a word worth? The question may sound silly, but it happens to be the foundation that underlies large language models, or LLMs — and ...