The ESM repository from Meta's Fundamental AI Research Protein Team contains pretrained Transformer language models designed specifically for protein sequence analysis and structure prediction. The repository implements Evolutionary Scale Modeling, a framework that applies large-scale unsupervised learning to protein sequences, with the codebase and pretrained weights made publicly available for research and applications in computational biology.
The primary models available in this repository include ESM-2, which represents state-of-the-art performance across protein structure prediction tasks and can predict protein properties directly from individual sequences. ESMFold provides end-to-end three-dimensional structure prediction from protein sequences alone, without requiring multiple sequence alignments. The repository also includes MSA Transformer for extracting embeddings from multiple sequence alignments, ESM-1v specialized for predicting the effects of sequence variants, and ESM-IF1 for inverse folding tasks that enable fixed backbone sequence design. These models were trained on datasets including UniRef50, UniRef90, and combinations with structural data from the Protein Data Bank and CATH.
Beyond the core models, the repository provides access to the ESM Metagenomic Atlas, a resource containing over 600 million predicted protein structures from metagenomic sequences. The Atlas was initially released in November 2022 with 617 million structures and subsequently updated in March 2023 to include 150 million additional predicted structures and precomputed ESM-2 embeddings. This atlas represents a significant resource for exploring protein structure space beyond experimentally determined structures.
The repository includes code for protein design applications released in April 2023, with two simultaneous preprints demonstrating how language models can generalize beyond natural proteins and how ESMFold can be used within a high-level programming language framework for generative protein design. These examples are provided under dedicated directories within the repository structure.
Installation is straightforward through pip, with optional dependencies for ESMFold functionality that require Python 3.9 or earlier and CUDA-compatible PyTorch. The repository supports multiple access methods including direct usage through the esm package, integration with HuggingFace transformers library, PyTorch Hub, and a command-line interface for bulk structure prediction from FASTA files. ColabFold integration also enables browser-based execution through Google Colab.
According to GitGenius activity tracking, the repository shows median issue and pull request response latency of approximately 5839.6 hours with a mean of 8081.9 hours across 17 tracked items. The most active issue labels include questions, enhancement requests, and good first issue designations. Primary contributors tracked include Arkadiy-Garber, WrViajreo, and luiswyss. The repository shares overlapping contributors with ray-project/ray, lightgbm-org/lightgbm, and google-deepmind/alphafold3, indicating cross-pollination with other major machine learning and structural biology projects.
The codebase is written primarily in Python and encompasses comprehensive functionality for protein sequence modeling, from embedding generation and variant effect prediction to structure prediction and design applications. The repository provides notebooks demonstrating various use cases and maintains detailed documentation of available models, datasets, and comparative performance metrics against related approaches in the field.
