BioModelBench — peptide_potency

Addressing: You are the agent working this task. Every obligation and constraint in this document applies to your work. Sub-agents you spawn inherit the interface contract, wall clock, and reporting requirements below; you retain responsibility for the final model_bundle/ and method.md submitted from /task.

Task: Autonomously build a model that predicts pActivity — the negative log-10 molar potency of a peptide (or peptidomimetic) against a class-A GPCR, unified across pKi / pIC50 / pEC50 / pKd, in log-molar units — and deliver it as a self-contained bundle that Capable's sandboxed Modal endpoint loads and executes against a private held-out slice of 3-5 class-A peptide GPCRs. Given (SMILES, target_uniprot) your model returns a real number in roughly [3, 12]; higher pActivity = tighter binder.

The private SMILES and labels never enter your training workdir. Your predict(...) function receives the private SMILES at inference time inside a no-egress container; only the aggregate metric surface defined below leaves that container. Attempting to write, log, or otherwise exfiltrate the SMILES or labels from predict.py is FORBIDDEN and, in the sandbox, mechanically impossible: network egress is blocked, subprocesses are prevented, and the filesystem outside /task is not writable.

Shipped materials

campaigns.json (the private target list; per receptor: target_uniprot, receptor_symbol, family, natural_ligand, and one to two sentences of context — no SMILES, no labels), baselines_summary.md (Capable-internal baselines on the same held-out slice; these are the numbers to beat), and this prompt. Nothing else. No training data ships; you SHALL source your own.

Submission

Write model_bundle/ in /task:

model_bundle/
├── predict.py            def predict(smiles: list[str], target_uniprot: str) -> list[float]
├── requirements.txt      pip-freeze; MUST resolve offline against the pre-baked wheelhouse
├── weights/              torch state_dicts, npz, joblib, ...; arbitrary binary artifacts
└── manifest.json         {"framework": ..., "gpu_required": bool, "max_batch": int}

The runner tars model_bundle/ into model_bundle.tar.gz. The endpoint installs requirements.txt with pip install --no-index --find-links /wheelhouse -r requirements.txt. Pre-baked wheels: torch, transformers, rdkit, scikit-learn, lightgbm, xgboost, numpy, pandas, pyarrow, biopython, scipy. Any other dependency MUST be a pip-installable wheel declared in requirements.txt early enough to be mirrored; unmirrored deps fail the install and the bundle scores zero across every receptor.

predict.py MUST expose the following two entry points:

def load_model():                                          # optional; called once per process
    """Load weights. Return anything; kept in memory by the runner."""

def predict(smiles: list[str], target_uniprot: str) -> list[float]:
    """
    smiles          RDKit-parseable SMILES. Peptides may be cyclic, contain
                    non-canonical residues, PEGylation, D-amino acids,
                    N-methylation, or exotic C-terminal caps.
    target_uniprot  e.g. "P43220" for GLP1R. One receptor per call; the
                    runner invokes predict() once per held-out receptor
                    with that receptor's full SMILES batch (500-1500).

    returns         list[float] of length len(smiles), predicted pActivity
                    in log-molar units, in the same order as the input
                    SMILES. Return exactly one float per SMILES; non-numeric
                    returns, length mismatches, or shape errors mark that
                    receptor missing in the macro average.
    """

The predict.py interface is FROZEN and identical for every submission. The runner does not pass additional keyword arguments and does not read return values other than the numeric list. Sockets, subprocesses, and writes outside /task are FORBIDDEN. GPU (H100, 80 GB) is provisioned when manifest.json sets "gpu_required": true.

Training data

You source your own. Fair-game corpora and models: ChEMBL, BindingDB, PubChem, Papyrus, DrugBank, PDB, UniProt, InterPro, any DMS dataset, any foundation-model checkpoint whose weights are downloadable (ESM-2 / ESM-3, PeptideCLM, ChemBERTa, MolFormer, Uni-Mol, ProtT5, Boltz-1 / Boltz-2, ESMFold, AF3 open weights), any cheminformatics library (rdkit, openbabel, useful-rdkit-utils). Cofolding and homology priors are permitted; H100 wall clock is a hard 10h ceiling and cofolding is expensive, so budget accordingly.

Compute

Training container: 16 CPU, 128 GB RAM, H100 (80 GB VRAM), 10h wall clock. pip install <x> works during training. Pre-installed at build time: torch, transformers, peft, scikit-learn, lightgbm, xgboost, rdkit, pandas, pyarrow, numpy, requests, datasets.

Scoring

Three metrics, each measuring an orthogonal axis. Computed per held-out receptor and macro-averaged (each receptor weighted equally).

1. Spearman ρ on pActivity — headline; full-list rank correlation between predicted and true pActivity, macro across receptors.
2. Enrichment factor at top 10% — hit density in the model's top decile divided by the base rate, hit threshold pActivity ≥ 7 (≤100 nM). Captures top-of-list utility.
3. RMSE in log-molar units — regression calibration; orthogonal to rank quality.

Deliverables

1. model_bundle/ per the interface above.
2. method.md (< 500 words): training data sources, encoder(s), model class, training regime, hyperparameters, ablations, calibration.
3. training_manifest.json: every dataset, URL, model checkpoint, and API you touched during training.