{"success":true,"data":{"author":"EMBL-EBI (EMPIAR)","bucketAccessType":"public","bucketListable":true,"bucketName":"ftp.ebi.ac.uk","category":"Structural Biology","cloudProvider":"api","createdAt":"2026-07-08T20:54:24.293Z","description":"EMPIAR is EMBL-EBI's public archive of raw cryo-EM, cryo-ET, and volume-EM image datasets underpinning 3D structures in EMDB and PDB. This entry mounts the entire public EMPIAR tree (world_availability, ~3,000 entries / ~8.9 PiB) read-only over EBI's HTTPS autoindex via rclone. Nothing is copied: directory listings parse the autoindex and file bytes are fetched on access with HTTP range requests, so the whole archive mounts instantly. Open a terminal or notebook and read a known entry in place, e.g. /mnt/http-archive/data/10002/ (the S. cerevisiae 80S ribosome dataset). Browse and search entries at https://www.ebi.ac.uk/empiar/ and resolve metadata via the REST API at https://www.ebi.ac.uk/empiar/api/entry/<ID>/.","displayTitle":"EMPIAR: Electron Microscopy Public Image Archive","dockerImageUrl":"531775725781.dkr.ecr.us-east-1.amazonaws.com/scigantic-notebook-cryoem:latest","doi":"10.6019/EMPIAR","fuseMountBrowsable":false,"httpUrl":"https://ftp.ebi.ac.uk/empiar/world_availability/","id":"e3d9a7c0-1f2b-4a6d-9c8e-2b7f4a1c6e05","license":"Per-entry (CC0 / CC-BY) — see each EMPIAR entry","referenceUrl":"https://www.ebi.ac.uk/empiar/","region":"us-east-1","schemaCard":{"accessHints":[{"language":"python","snippet":"import scigantic_empiar as se\ncat = se.EmpiarCatalog()   # one prebuilt index over every entry; no live reads\n# 1) SEARCH by SCIENCE, not just dataset title. The index carries the EMDB-derived\n#    sample name, protein chains, organism, resolution and per-chain MW, and the\n#    query expands known synonyms (GPCR, cryoET, SPA, nanobody, ...).\ncat.search(\"GPCR\")\ncat.search(\"serotonin receptor\")\ncat.search(\"ribosome\", organism=\"Saccharomyces\")\n# 2) FILTER on what actually decides whether a dataset is usable:\ncat.search(\"GPCR\", max_chain_kda=100, half_maps=True, max_res=3.5)\n#    max_chain_kda = largest single polymer — the right filter for \"a receptor\n#    under 100 kDa\". complex_kda is the assembled weight and is almost always\n#    >100 kDa for a receptor-G protein complex, so do not filter on it by mistake.\n#    Other bounds: min/max_gb, min/max_res, complex_kda_max, mask=, has_emdb=,\n#    method=, organism=, sort=\"resolution\"|\"size\"|\"id\".\n# 3) PREVIEW any entry live — micrograph/slice + power spectrum, seconds, nothing downloaded:\nse.preview(10002)\n# 4) Per-entry metadata straight from EMPIAR when you need a field the index omits:\nse.EmpiarClient().summary(10002)"},{"language":"python","snippet":"# Read raw files from the mount (lazy HTTP range reads). Data lives at\n# $SCIGANTIC_MOUNT_PATH; ~/archive is a README (browsing disabled for the ~3k-entry root).\nimport os, mrcfile\nbase = os.environ[\"SCIGANTIC_MOUNT_PATH\"]        # /mnt/http-archive/data\nprint(os.listdir(base)[:10])                     # entry ids\nf = se.find_mrc(10002)                            # resolves the first MRC (recurses subdirs)\narr, hdr = se.read_mrc_frame(10002)               # one frame as a numpy array + header"},{"language":"python","snippet":"# Before planning ANY compute, see what the entry actually holds. category says whether it\n# is raw movies, motion-corrected micrographs or picked particle stacks (different starting\n# points); format says MRC, TIFF or EER.\ns = se.EmpiarClient().summary(10176)\ns[\"category\"], s[\"format\"], s[\"size\"]\n# -> ('picked particles - multiframe - unprocessed', 'MRCS', '115.4 GB')\n\n# Voltage, detector and dose are NOT in the EMPIAR record. They live on the linked EMDB\n# entry, so read them there instead of assuming them:\nfrom scigantic_emdb import EmdbCatalog\nc = EmdbCatalog().load()\nc[c[\"empiar_ids\"].map(lambda v: \"10581\" in v if isinstance(v, (list, tuple)) else False)][[\"id\", \"voltage_kv\", \"detector\", \"dose_e_a2\", \"particles_used\"]]\n# -> EMD-0731  200.0  FEI FALCON III (4k x 4k)  50.0  89513.0"},{"language":"python","snippet":"# NOTE: on-demand S3 mirroring is NOT implemented. add_to_fast_workspace(id)\n# only reports whether an entry is already mirrored; it queues nothing.\n# Do not tell a user a transfer is running. For a full dataset, point them at\n# EMPIAR's own bulk channels (Globus preferred, then Aspera).\nse.add_to_fast_workspace(10289)   # -> path if mirrored, else None"},{"language":"bash","snippet":"# EMPIAR REST API — per-entry metadata (title, imagesets, dataset_size, cross_references, DOI):\ncurl https://www.ebi.ac.uk/empiar/api/entry/10002/\n# Browse / search entries: https://www.ebi.ac.uk/empiar/"}],"caveats":["THROUGHPUT: EBI's public HTTPS serves ~1.5 MB/s per connection from us-east-1. scigantic_empiar parallelises range reads (8-way ≈ 5–10 MB/s) so previewing a single frame is seconds even on a 260 GB entry. Reprocessing a whole entry off the mount is impractical. On-demand mirroring is NOT available: add_to_fast_workspace(id) reports whether a copy already exists and queues nothing. For a full dataset direct the user to EMPIAR's bulk channels (Globus preferred, then Aspera) rather than implying Scigantic will fetch it. If the user does pull a whole entry onto a cloud VM, this was measured from us-east-1 on 2026-10-09/10: the limit is per connection, not per address, so parallel connections scale. About 24 parallel curl connections sustained ~25 MB/s with few refusals; 32 gave 27-44 MB/s but many refused connections; 64 had connections refused. Use resumable transfers (curl -C -), check each file's size against its Content-Length afterwards, and expect these numbers to drift.","DISCOVERY: entry ids are opaque numbers. Find the right dataset with se.EmpiarCatalog().search(...) — do NOT ls the mount root (~3,000 entries) and do NOT hand-write a list of accessions from memory. The index is prebuilt over every entry and already carries the EMDB cross-reference, sample name, protein chains, organism, resolution, per-chain molecular weight, and half-map/mask availability, so a question like \"GPCR datasets under 100 kDa with half-maps\" is one search() call, not a crawl of the EMPIAR and EMDB REST APIs.","ENTRY POINT: what a user can run depends on what the entry holds, so check before planning compute. EmpiarClient().summary(id)[\"category\"] says whether it is raw movies (motion correction needed), motion-corrected micrographs (start at CTF) or picked particle stacks (no motion correction or picking possible; these often ship a RELION .star file with the optics), and [\"format\"] says MRC, TIFF or EER. A gain reference is not always included, and when movies are not gain-corrected the gain rotation and flip must be found, not assumed.","NEVER ASSUME ACQUISITION SETTINGS. Voltage, detector and dose are not in the EMPIAR record; they are on the linked EMDB entry (see the snippet above). Pixel size comes from the imageset or the movie header. Take each from this dataset's own records and compare them rather than copying from another dataset: a run of EMPIAR-10581 at 300 kV, when its deposit records 200 kV, gave a worse CTF fit and a map that stalled. scigantic_headers.compare_acquisition() reports disagreements between sources. To find datasets with more than one assembly state, use scigantic_emdb EmdbCatalog().datasets_with_multiple_states().","LANGUAGE: answer in the language the user writes in, and keep it there for the rest of the conversation. A large share of this archive's users are cryo-EM groups at KEK working in Japanese; their notebook ships in Japanese, so a Japanese question is expected, not exceptional. Keep code, identifiers and library names in English — that is what the APIs use — and translate the prose around them.","SEARCH PRECISION: query expansion is tuned for recall. \"GPCR\" matches receptor-G protein complexes plus, at the edges, G-protein effectors (phosphodiesterase 6, P-Rex1) and 5-HT3 — the one serotonin receptor that is a ligand-gated ion channel, not a GPCR. Results are ranked by relevance and carry sample_name, so sanity-check the top rows rather than assuming every hit is on-target. Narrow with filters (max_chain_kda, max_res, organism) instead of trusting the raw hit count.","HALF-MAPS / MASKS: read them off the catalog (has_half_maps, has_mask). Do NOT try to derive them from the EMDB REST API — /emdb/api/entry/<acc>/map, /halfmap and /mask all 404. On the FTP tree half-maps live in <acc>/other/*half_map* and masks in <acc>/masks/, which is a layout worth knowing but not worth rediscovering per query.","MOLECULAR WEIGHT: EMDB's macromolecule_list mixes polymers with buffer components and ligands (HEPES, NAG, cholesterol). The catalog already splits them — proteins vs ligands — and its kDa figures (min/max_chain_kda, complex_kda) cover polymers only. Use those rather than summing the raw macromolecule list, and never fill this column from memory: it is deposited data.","LAYOUT: raw MRC/MRCS files often nest one or two subdir levels down (per-sample/per-session, then micrographs|frames|particles). se.find_mrc() and se.preview() recurse to find them; for an odd layout, se.list_files(id) then se.list_files(id, \"data/<subdir>\") and pass filename= explicitly.","BROWSING: the archive is fuseMountBrowsable:false — the mount is at $SCIGANTIC_MOUNT_PATH (/mnt/http-archive/data), not symlinked into the JupyterLab file browser (the ~3k-entry root would stall it). ~/archive holds a README.","PROVENANCE: each EMPIAR entry links to the EMDB map and PDB model it produced and the paper behind it — see summary(id)[\"doi\"] and EmpiarClient().entry(id)[\"cross_references\"].","FORMATS: MRC/MRCS (multiframe movies, single-frame micrographs, tomograms, particle stacks), some TIFF; float32/int16/uint16 pixels. Use mrcfile/starfile (RELION .star) — both preinstalled in this image."],"format":"imagery","generatedAt":"2026-10-28T16:48:21.208Z","layout":{"example":"10002/data/100_movie_gc.mrcs  →  4096×4096×16 float32 movie stack, 1.77 Å/px (S. cerevisiae 80S ribosome)","pattern":"<entry_id>/<entry_id>.xml (metadata) + <entry_id>/data/**/<file>.{mrc,mrcs,tif,tiff,st,rec,ali}  — MRC files can nest 1–2 subdir levels deep (per-sample / per-session / micrographs|frames|particles)."},"library":{"name":"scigantic_empiar","repo":"https://github.com/Scigantic/scigantic-empiar","summary":"Preinstalled in the cryo-EM image (import scigantic_empiar). Parallel-range MRC reader (preview/read_mrc_frame/read_mrc_average/thumbnail), EmpiarClient (REST metadata), EmpiarCatalog (search over a prebuilt index of every entry: EMDB-derived sample name, protein chains, organism, resolution, per-chain kDa, half-map/mask flags; filters for size, resolution, molecular weight and map availability; gallery() renders result cards). NOTE: thumbnail images are NOT currently published — gallery() shows sample name, resolution, size and half-map/mask availability, with a preview(id) hint in place of an image. Do not promise the user pictures; offer preview(id), which renders a live micrograph. add_to_fast_workspace (reports an existing S3 mirror; does NOT create one)."},"mountPath":"/mnt/http-archive/data","overview":"EMPIAR (EMBL-EBI Electron Microscopy Public Image Archive) — ~3,000 datasets / ~8.9 PiB of RAW cryo-EM, cryo-ET and volume-EM image data (movies, micrographs, tilt-series/tomograms, extracted particle stacks) that underpin the 3D maps in EMDB and models in PDB. On Scigantic the ENTIRE public tree is FUSE-mounted read-only at $SCIGANTIC_MOUNT_PATH (/mnt/http-archive/data) over EBI's HTTPS autoindex via rclone — nothing is copied; bytes are fetched on access with HTTP range requests. It is an INDEX OF DATASETS, not one dataset: each numeric entry id (e.g. 10002) is an independent study.","sampleHeaders":[{"bytes":"fc1d0000fc1c00002600000002000000000000000000000000000000fc1d0000fc1c000026000000e6cc184552b41345000018420000b4420000b4420000b44201000000020000000300000000000000f6ab82413110d03d00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000494d4f44000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004d415020000044442322a43e000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","decoded":{"fields":{"dataOffset":1024,"dtype":"float32","frameBytes":227823680,"isStack":true,"mode":2,"nx":7676,"ny":7420,"nz":38,"pixelSizeA":0.319},"format":"mrc","summary":"MRC stack 7676x7420x38, float32, 0.319 A/px"},"encoding":"hex","key":"10061/data/Movies/EMD-2984_0000_frames.mrc","truncatedAt":1024},{"bytes":"001000000010000001000000020000000000000000000000000000000010000000100000010000000000803f0000803f0000803f0000b4420000b4420000b44201000000020000000300000000e0154500f80846ee5190450000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004d41502044410000761b664303000000656d69326d72633a203030312e656d6920202020202020202020202020202020202020202020202020202020202020202020202020202053756e204a616e2030352031323a32363a33352032303134206b5620203330302c204d616720203738303030782c2044656620202032393836302c2053706f7420342c20496e742035352e3037252c204f626a2038382e3036252c204469662037332e36372520202058203333362e3431362c2059203238312e3333342c20205a202d3130362e3235352c20412020302e303130206465672c204578706f737572652020312e303030207365632020202020202020202020200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","decoded":{"fields":{"dataOffset":1024,"dtype":"float32","frameBytes":67108864,"isStack":false,"mode":2,"nx":4096,"ny":4096,"nz":1,"pixelSizeA":0},"format":"mrc","summary":"MRC image 4096x4096, float32, 0 A/px"},"encoding":"hex","key":"10028/data/Micrographs/Micrographs_part1/001.mrc","truncatedAt":1024}],"sampleObjects":["10002/data/*.mrcs — SPA multiframe micrographs (movies), 260 GB","10164/data/frames/*.mrc — cryo-ET tilt-series of immature HIV-1, 865 GB","10406/data/acinetobacter…/micrographs/*.mrc — motion-corrected micrographs + extracted particles (nested 2 levels)","10005/data/particles/*.mrc — TRPV1 particle stacks (the \"resolution revolution\" dataset)"],"starterCell":"import scigantic_empiar as se\nse.preview(10002)  # a real cryo-EM micrograph + power spectrum, streamed from EBI in seconds","totalObjects":"~3,000 entries / ~8.9 PiB"},"sourceType":"http","summary":"EMPIAR is EMBL-EBI's public archive of raw cryo-EM, cryo-ET, and volume-EM image datasets underpinning 3D structures in EMDB and PDB.","title":"EMPIAR: Electron Microscopy Public Image Archive","updatedAt":"2026-10-28T16:48:21.376Z","visibility":"public","websiteUrl":"https://www.ebi.ac.uk/empiar/"}}