{"success":true,"data":{"author":"Chandar Research Lab","bucketAccessType":"public","bucketName":"models/chandar-lab/AMPLIFY_350M","bucketPrefix":"","category":"Biology","cloudProvider":"huggingface","createdAt":"2026-11-16T15:25:49.059Z","description":"Open weights for AMPLIFY 350M, an efficient protein language model from Chandar Lab pretrained with masked language modelling on UR100P, a corpus combining UniRef100, the Observed Antibody Space (OAS) and SCOP. The model has 32 layers, a hidden size of 960 and 15 attention heads, supports sequences up to 2,048 residues, and produces residue and protein embeddings, suggests mutations and scores sequences. The mounted repo holds model.safetensors (1.4 GB), config.json, tokenizer files and the custom modelling code (amplify.py, rotary.py, rmsnorm.py), loaded with transformers and trust_remote_code from the mount path. MIT licence.","displayTitle":"AMPLIFY 350M protein language model weights","huggingFaceUrl":"https://huggingface.co/chandar-lab/AMPLIFY_350M","id":"432da359-0ac7-50b4-abf8-e1efd422b7f3","license":"MIT","licenseEvidenceUrl":"https://github.com/chandar-lab/AMPLIFY/blob/main/LICENSE","objectCountEstimate":10,"probeStatus":"ok","referenceUrl":"https://www.biorxiv.org/content/10.1101/2024.09.23.614603v1","region":"us","schemaCard":{"accessHints":[{"language":"python-hub","snippet":"from huggingface_hub import snapshot_download\nsnapshot_download(repo_id=\"chandar-lab/AMPLIFY_350M\")  # or read /mnt/archive directly"}],"caveats":["HuggingFace metadata: author=chandar-lab, license:mit, downloads=2726","Weights are read lazily from the mount. Copying them to local disk first is faster when a model is loaded many times."],"format":"binary","generatedAt":"2026-11-17T19:17:20.515Z","layout":{"example":"model.safetensors","pattern":"{repoId}/{filename}"},"mountPath":"/mnt/archive","sampleObjects":["config.json","model.safetensors"],"sidecars":[{"bytes":"---\nlicense: mit\ndatasets:\n  - chandar-lab/UR100P\nlanguage:\n  - en\ntags:\n  - biology\n---\n\n## AMPLIFY\n\nAMPLIFY is an efficient, state-of-the-art protein language model pre-trained using masked language modeling on UniRef100, OAS, and SCOP ([UR100P](https://huggingface.co/datasets/chandar-lab/UR100P)). AMPLIFY can generate residue and protein embeddings, suggest mutations, differentiate disordered proteins from non-protein sequences, and much more. AMPLIFY is available in two sizes, 120M and 350M parameters, with the `_base` models not extended beyond 512 residues (Stage 1). The model architecture and pre-training procedure are detailed below. For more details, please refer to the [accompanying paper](https://www.biorxiv.org/content/10.1101/2024.09.23.614603v1).\n\n- [`AMPLIFY_350M`](https://huggingface.co/chandar-lab/AMPLIFY_350M)\n- [`AMPLIFY_350M_base`](https://huggingface.co/chandar-lab/AMPLIFY_350M_base)\n- [`AMPLIFY_120M`](https://huggingface.co/chandar-lab/AMPLIFY_120M)\n- [`AMPLIFY_120M_base`](https://huggingface.co/chandar-lab/AMPLIFY_120M_base)\n\n### Model Descritpion\n\n|                                | AMPLIFY 120M | AMPLIFY 350M |\n| :----------------------------- | -----------: | -----------: |\n| `hidden-size`                  |          640 |          960 |\n| `num-hidden-layers`            |           24 |           32 |\n| `num-attention-heads`          |           10 |           15 |\n| `intermediate-size`            |         2560 |         3840 |\n| `max-position-embeddings`      |         2048 |         2048 |\n| `vocab-size`                   |           27 |           27 |\n| `rope-theta`                   |        10000 |        10000 |\n| `dropout-prob`                 |            0 |            0 |\n| `embedding-init-range`         |         0.02 |         0.02 |\n| `norm-eps`                     |      1.0e-05 |      1.0e-05 |\n| `hidden-act`                   |       swiglu |       swiglu |\n| `pre-activation-layer-norm`    |         true |         true |\n| `layer-norm-after-embedding`   |        false |        false |\n| `layer-norm-before-last-layer` |         true |         true |\n| `rms-norm`                     |         true |         true |\n| `ffn-bias`                     |        false |        false |\n| `attn-bias`                    |        false |        false |\n\n### Training Descritpion\n\n|                     |     Stage 1 |                      Stage 2 |\n| :------------------ | ----------: | ---------------------------: |\n| `dataset`           |      UR100P |                       UR100P |\n| `max-steps`         |     1000000 | 25000 (120M) or 50000 (350M) |\n| `max-length`        |         512 |                         2048 |\n| `optimizer`         |       adamw |                        adamw |\n| `lr`                |       0.001 |                       0.0001 |\n| `betas`             | (0.9, 0.95) |                  (0.9, 0.95) |\n| `eps`               |     1.0e-08 |                      1.0e-08 |\n| `weight-decay`      |        0.01 |                         0.01 |\n| `scheduler`         | cosinedecay |                         none |\n| `warmup-steps`      |       1,000 |                         none |\n| `final-step`        |     900,000 |                         none |\n| `warmup-steps`      |       1,000 |                         none |\n| `gradient-clipping` |         1.0 |                          1.0 |\n| `tf32`              |        true |                         true |\n| `mixed-precision`   |        bf16 |                         bf16 |\n| `padding`           |  max-length |                   max-length |\n| `random-truncate`   |        true |                         true |\n| `mask-probability`  |        0.15 |                         0.15 |\n| `total-batch-size`  |        4096 |                         4096 |\n| `deepspeed`         |        true |                         true |\n| `zero-stage`        |           3 |                            3 |\n\n## Get Started\n\n```python\nfrom transformers import AutoModel\nfrom transformers import AutoTokenizer\nfrom datasets import load_dataset\n\n# Load AMPLIFY and toke","encoding":"utf8","key":"README.md","name":"README.md","size":5579,"truncated":true}],"sourceObjectKey":"model.safetensors","starterCell":"import os, json\nbase = \"/mnt/archive\"   # this model repo, mounted read-only\nfiles = sorted(os.listdir(base)); print(files)\nprint(json.dumps(json.load(open(os.path.join(base, \"config.json\"))), indent=1)[:800])\nimport struct\nwith open(os.path.join(base, \"model.safetensors\"), \"rb\") as f:\n    hdr = json.loads(f.read(struct.unpack(\"<Q\", f.read(8))[0]))\ntensors = {k: v for k, v in hdr.items() if k != \"__metadata__\"}\nprint(len(tensors), \"tensors, e.g.\", list(tensors.items())[:3])\n# To load them: pip install safetensors torch, then safetensors.torch.load_file(path)","totalObjects":10},"summary":"Open weights for AMPLIFY 350M, an efficient protein language model from Chandar Lab pretrained with masked language modelling on UR100P, a corpus combining UniRef100, the Observed…","title":"AMPLIFY 350M protein language model weights","updatedAt":"2026-11-17T19:17:20.753Z","visibility":"public","websiteUrl":"https://huggingface.co/chandar-lab/AMPLIFY_350M"}}