Myelin · Partner Ingestion API

Developer documentation

Push R&D data deliveries into Myelin from your pipeline — validated against the client’s quality rules before a byte moves, resumable at any scale, reviewed and audited exactly like a portal delivery.

Quick start

Your bridge owner creates an API key in Bridge → API. From key to first submitted delivery is a few minutes:

export MYELIN_API_KEY=myl_live_…

npx @myelinbridge/cli ping                                  # verify auth, list your projects
npx @myelinbridge/cli check ./run_042 --dataset onco1-wes   # preflight — nothing uploaded
npx @myelinbridge/cli push  ./run_042 --dataset onco1-wes --submit

Prefer raw HTTP? Everything the CLI does is plain REST + presigned S3 uploads — see the reference and the OpenAPI 3.1 spec.

Authentication

Every request carries Authorization: Bearer myl_live_…. A key is scoped to one bridge and a chosen set of projects; behind it stands a named service identity that appears in member lists and in the tamper-evident audit trail. Keys are shown once, rotate in one click, and the client can disable a bridge’s API access instantly. Errors are RFC 9457 application/problem+json with a stable code; the rate limit is 120 requests/min per key (uploads don’t count — they go direct to storage).

Delivery flow

POST /v1/datasets/{id}/preflight        # 0. validate your file list — no upload, exit early on blockers
POST /v1/datasets/{id}/batches          # 1. create or resume THE draft (one per dataset)
POST /v1/batches/{id}/files             # 2. declare files (≤500/call) → storage paths + part size
POST /v1/files/{id}/upload-parts        # 3. get presigned S3 URLs for the parts you still need
PUT  {presigned url}                    #    push each part direct to EU staging — resumable, no credential stored
POST /v1/files/{id}/confirm             # 4. finalize (upload_id + part ETags); server verifies the declared size
POST /v1/batches/{id}/submit            # 5. runs the client's auto-checks, enters review
GET  /v1/batches/{id}/findings          # 6. on changes_requested: failed files + hints, machine-readable
                                        #    fix → re-declare (replacement) → re-upload → re-submit

Declares are idempotent (same path + size = same file, untouched) — a crashed pipeline re-runs the same command and only the delta uploads. A batch that is submitted or in review is locked (409 delivery_locked); recall it or wait.

Webhooks

Register an HTTPS endpoint (portal, or POST /v1/webhook-endpoints) and Myelin pushes signed events — batch.review_started, batch.changes_requested, batch.validated, batch.transferred, … Verify the signature:

// Node — header: Myelin-Signature: t=<unix>,v1=<hex>
import { createHmac } from 'node:crypto'

function verify(secret, header, rawBody, toleranceSec = 300) {
  const { t, v1 } = Object.fromEntries(header.split(',').map((p) => p.split('=')))
  if (Math.abs(Date.now() / 1000 - Number(t)) > toleranceSec) return false
  const expected = createHmac('sha256', secret).update(`${t}.${rawBody}`).digest('hex')
  return expected === v1
}
# Python
import hmac, hashlib, time

def verify(secret: str, header: str, raw_body: bytes, tolerance: int = 300) -> bool:
    parts = dict(p.split("=") for p in header.split(","))
    if abs(time.time() - int(parts["t"])) > tolerance:
        return False
    signed = f"{parts['t']}.".encode() + raw_body
    expected = hmac.new(secret.encode(), signed, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, parts["v1"])

Deliveries retry at +5 min / +30 min / +24 h; use Myelin-Event-Id to deduplicate. No HTTPS receiver? Poll GET /v1/events?cursor=… (pass next_cursor back verbatim).

Endpoint reference

Rendered from the same spec served at /api/v1/openapi.json (1.0.0-phase4).

get/v1/ping

Is my key alive? Returns key, bridge, and scoped project ids.

get/v1/me

Key identity, bridge (lifecycle, connection mode), scoped projects.

get/v1/projects

Projects in the key's scope.

get/v1/projects/{projectId}/datasets

Datasets of a scoped project: id, name, slug, description, lifecycle_status, sample_depth, current quality-check version, open-draft pointer.

get/v1/datasets/{datasetId}

Dataset detail: the list fields plus project_id and quality_checks_published_at. The object is the delivery contract only — what a delivery must satisfy comes from the quality-check version, never from a descriptive label (`data_type`, `data_domain` and `metadata_mode` are all gone — see the Dataset schema).

post/v1/datasets/{datasetId}/sample-depth

Declare at which folder depth a sample sits (PRD §4.3) — the only dataset write a partner has. Body: { sample_depth: 0-5 }. You own your output structure, so you declare the depth your deliveries use; everything else about a dataset is created and described by the client in the portal. Idempotent: re-declaring the current value succeeds with changed:false, so a pipeline can assert its depth on every run. Locks once any batch in the dataset leaves draft — after that the value is frozen so sample-scoped quality verdicts stay comparable across deliveries. Set it BEFORE your first submit.

get/v1/datasets/{datasetId}/quality-checks

The delivery contract: the published quality-check snapshot preflight/submit will evaluate, each check carrying a plain-language assertion, its quality dimension, whether it can be checked BEFORE upload, and a pass/fail example.

get/v1/batches

List scoped batches. Filter by dataset_id/status; cursor pagination.

get/v1/batches/{batchId}

Batch detail: status, whose-move, round number, files with upload state, submit-readiness.

delete/v1/batches/{batchId}

Delete a draft batch (files, rows, and staging blobs). Draft only.

get/v1/batches/{batchId}/findings

Quality findings; in changes_requested also per-file reviewer verdicts + request-changes comment.

get/v1/batches/{batchId}/events

Batch timeline (submitted, review_started, changes_requested, …).

post/v1/datasets/{datasetId}/batches

Create OR resume the dataset's draft batch (one draft per dataset).

post/v1/batches/{batchId}/files

Declare files (single {path, size_bytes, checksum?} or {files: [...]} ≤500). Mints storage paths, creates 'queued' rows, returns the S3-multipart upload descriptor (protocol, part_size, max_parts_per_sign). Get presigned part URLs per file from /files/{id}/upload-parts. Re-declaring identical files is idempotent; a different size at an existing path needs replace=true.

post/v1/files/{fileId}/upload-parts

Mint presigned S3 multipart UploadPart URLs for a declared file (SEC-TOKEN-01: no bearer token — each URL authorizes exactly one part of one object for a few minutes). Body: { part_numbers: number[] } (1-based, ≤ max_parts_per_sign). Returns the upload_id, the part_size, which parts already landed (resume), and a presigned url per still-missing requested part. PUT each part to its url; the ETag response header is what confirm needs. Gated to draft/changes_requested batches.

post/v1/files/{fileId}/confirm

Finalize the upload: completes the S3 multipart upload from its parts, then verifies the staging object exists with the declared size, then flips the row to uploaded (audit batch.file_uploaded / file_reuploaded). Body: { upload_id, parts: [{part_number, etag}], checksum?, checksum_algorithm? }.

delete/v1/files/{fileId}

Remove a file (row + staging blob) from a draft / changes_requested batch.

post/v1/batches/{batchId}/submit

Submit (or resubmit from changes_requested): round snapshot, frozen fields, synchronous auto-checks, status flip, reviewer notification. Blocks while declared files are still un-uploaded.

post/v1/batches/{batchId}/recall

Recall a submitted batch (only before review starts). Restores draft or changes_requested and invalidates the submit snapshots.

post/v1/datasets/{datasetId}/preflight

Evaluate a file list ({files: [{path, size_bytes}]}, ≤10k) against the dataset's published quality checks BEFORE uploading. Read-only, no audit. Returns per-check verdicts with remediation hints; content-dependent checks come back in `deferred` (they run at submit), manual checks in `manual`.

post/v1/bridges/{bridgeId}/sandbox

Partner-side test transfer, API mode: multipart `file` ≤10 MB (hosting platform may cap request bodies earlier, ~4.5 MB on Vercel — a small sample file is the point). Satisfies the same activation checklist item as the portal sandbox (same tracking row, same audit event). Works while the bridge is still configuring.

get/v1/webhook-endpoints

List this bridge's webhook endpoints (url, events, active, failure_count, last delivery) plus the available event_types.

post/v1/webhook-endpoints

Create an endpoint: {url (HTTPS, public host), events?} — empty events = all partner events. The HMAC signing secret is returned ONCE. Deliveries carry `Myelin-Signature: t=<unix>,v1=<hmac_sha256(secret, t + "." + body)>` (5-min replay window recommended) and `Myelin-Event-Id` for dedup. Retries at +5 min/+30 min/+24 h, terminal after 4 attempts; endpoints auto-disable after 30 consecutive terminal failures.

delete/v1/webhook-endpoints/{endpointId}

Remove an endpoint (pending deliveries to it are dropped).

post/v1/webhook-endpoints/{endpointId}/test

Queue and immediately attempt a synthetic test.ping delivery (signed like a real event).

get/v1/events

Polling fallback for shops without an HTTPS receiver: ascending cursor over batch timeline events across the key's projects. Pass next_cursor back VERBATIM (it carries microsecond precision).

Pipeline recipes

Nightly cron shipping finished runs:

#!/usr/bin/env bash
set -euo pipefail
export MYELIN_API_KEY="$(cat /etc/myelin/key)"

for run in /data/outbox/*/; do
  if npx @myelinbridge/cli check "$run" --dataset onco1-wes --json > /tmp/preflight.json; then
    npx @myelinbridge/cli push "$run" --dataset onco1-wes --submit && mv "$run" /data/shipped/
  else
    echo "blocked: $run — see /tmp/preflight.json" >&2   # exit 2 = failing checks
  fi
done

Nextflow / Snakemake completion hook — same two commands in an onComplete handler; use --json and the exit code to gate your workflow status.

API v1 · additive changes only