For self-driving and robot teams. Mutational breaks your system on purpose, inside the simulator you already run, and shows which tests notice - which pass no matter what, which you can retire, and what a release quietly stopped catching.
Runs against
A simulation suite is the evidence a release is safe. Three things make it evidence of less than it looks - and none of them turns anything red.
A scenario whose checks no fault can trip still passes on every merge. On fifty Meta-World tasks, every one of 265 caught faults failed through a single check - did the task finish - while an arm could sweep through anything else on the table.
When the stack improves, the scenario kept for last year's weakness stops testing it - and still passes. Every other gate fires when the system gets worse. This one fires when the suite did.
Models now write scenarios faster than anyone can read them. Close to half of what a capable model proposes tests nothing and passes. Only a sweep can tell which half.
Self-driving teams
Robot teams
mutational standin measures where a twin rehearsal says go, or a judge says done, and the real run fails. In a stand-in for a model-driven home humanoid, the robot's own report claimed success on every false grasp.From January 2027 the EU Machinery Regulation requires a notified body to assess machinery whose safety functions rely on self-evolving machine learning. Every Mutational report can be replayed from its recording by someone who was not there.
Book a robotics pilotEvidence
SimCloud Twins
No install, no simulator, no sign-in. The real binary, compiled to WebAssembly, replaying a real sweep: six Meta-World manipulation tasks in MuJoCo, 542 simulator runs.
====================================================================
MUTATIONAL REPORT
====================================================================
runner replay: 542 recorded runs
discrimination 31.9% (23 of 72 injected defects detected)
scenarios 6 scored, 0 skipped, 10 inconclusive defects
undeliverable 26 defects could not be expressed in the scenarios aimed at,
and are excluded: not exposed, and not missed either.
PER SCENARIO
? reach-v3 0.0% 0/10 flake 0%
drawer-close-v3 15.4% 2/13 flake 0%
door-open-v3 16.7% 2/12 flake 0%
button-press-v3 45.5% 5/11 flake 0%
pick-place-v3 50.0% 7/14 flake 0%
push-v3 58.3% 7/12 flake 0%
Press "Run it here" to load the binary (3.7 MB) and produce this report
yourself, from the recording, in this tab.A catalogue of ways your system gets worse - a blinded sensor, a stale picture, a weak grip, a skill that reports success it did not have - delivered through your own adapter.
Each scenario's own pass/fail checks are the oracle. Verdicts are held against that scenario's own flake rate, so noise is never read as a catch.
Keep, delete, or cannot tell - each with the evidence behind it. And it says plainly when it cannot tell.
Hazards
A catch counts for the check that actually failed - not for what a fault was aimed at. See which hazards rest on a single scenario.
$ mutational gate --fingerprint v3
against arm-v1
heavy_mug_far_edge
no longer catches weak-grip x0.5
no longer catches weak-grip x0.75
exit 1 - the corpus lost coverageRelease after release, the gate fires on what a build stopped catching - not on a rate that moved. An unanswered question is never a pass.
Record a sweep once where the simulator lives. Replay it anywhere into the full report - in CI, on a laptop, or in this browser.
Worklist
What to do to the corpus next, for whoever - or whatever - is improving it. Field incidents raise demand; only a sweep is evidence.
POST /v1/sweeps curl https://collector.mutational.in/v1/sweeps --header 'Authorization: Bearer mutational_...' --json @sweep.json
Everything the CLI does is an API call or an MCP tool. Post sweeps from CI, ask the gate from a pipeline, drive it all from an agent.
This tab's worker removed
In the browser your files never leave the tab: the worker that reads your files deletes its own network first. On the collector, adapter output is never stored.
Your simulator stays where it is and your scenarios stay in your checkout. Mutational says what they do not prove yet, and is the one thing in the loop that decides whether a new scenario helped.
Anyone may propose. Only a sweep accepts.
What your tests do not prove yet, most urgent first: a hazard with nothing behind it, a fault nothing catches, a scenario that stopped testing what it was kept for. Field incidents raise an item. Only a sweep closes one.
An engineer, or a coding agent working in your checkout through mutational mcp. Either way it is a claim: describing a scenario differently changes nothing, and changing what it asserts does.
On your laptop, in CI, or on a worker next to the simulator that dials out, behind your VPN. It runs the adapter its own project file pins, and nothing the cloud sends can change your corpus.
accept answers from that sweep: earns its place, duplicate, or catches nothing. Close to half of what a capable model writes tests nothing and looks fine; this is where it is found out.
The gate fails the build that stopped catching something, by name, and not the one whose pass rate moved - including the build where the system got better and a scenario quietly stopped testing anything.
Any agent that speaks MCP: over stdio beside the simulator, or over HTTP to the cloud with an agent token.
A model reads the worklist, writes a scenario, and asks a sweep whether it earned its place.
The model operating the real robot gets three tools and no more: to say what happened.
{
"mcpServers": {
"mutational": {
"command": "mutational",
"args": ["mcp", "--project", "mutational.project.json"]
}
}
}
tools orient · next · sweep_start · job_status · accept · gate
triage · asserts · catalogue · history · field_add · ...suite.json
Every CI build posts its sweep to one collector. Two builds line up as one series, whatever machine ran them.
API tokens
An agent's token adds. It cannot delete, or replace a sweep with one that catches less.
Give the model that keeps your corpus a token that cannot rewrite the history its work is gated on.
DELETE /v1/sweeps/swp_2fb1... 200 the sweep, its rows, its recording at rest, per run: key · injected · outcome · reason
What the adapter printed - paths, serial numbers - is stripped before upload. Delete a sweep and its recording goes with it.
You keep your simulator, your scenarios and your stack where they are. We connect to them, measure them, and hand you what we found. The pilot's workspace runs at Programme's limits for 56 days, so there is time to use it.
We write the adapter with you: one script that runs a scenario with a fault and exits 0 or 1. mutational doctor proves it injects before a campaign is spent.
Every scenario against your fault catalogue, on your machines. You get what each one catches, what nothing catches, and what can go.
The gate on lost coverage in your pipeline, a hosted history every build is measured against - and, for robots, your rehearsal or success judge measured against the real run.
What you get at the end looks like this: a sample pilot report, made on a stand-in for a humanoid home robot, with every figure linked to the run that produced it.
Or write to admin@mutational.in. Tell us your simulator and roughly how many scenarios you run.
You pay for how much of a corpus is under measurement - never per seat, and never per finding. A tool paid for every regression it reports is paid to report regressions. Billed yearly; the binary runs beside your simulator and checks its licence offline.
Open
$0
Sign in with your email · no card
Universities, open source, evaluation.
Team
$180 /year
One validation team
One corpus, measured whole.
Programme
$600 /year
An AV or robotics programme
Several corpora, and models driving them.
Enterprise
From $1,500
A year, organisation-wide
Air-gapped if it has to be.
Add-ons, on Team or Programme: another corpus for $150 a year; 1,000 more scenarios for $60 a year.
Start with a pilot: $200 once, for 56 days at Programme's limits on your own simulator - your triage report and a CI gate at the end - and what it cost comes off your first year of Team or Programme. Buy a pilot or talk to us first. No account? The playground runs every read-only command in your browser.
Can't find what you are looking for? Write to us.
Whether your simulation scenarios would fail if the system under test got worse. It injects faults from a catalogue - a blinded sensor, a lagging picture, a weak grip - and watches whether each scenario's own pass/fail checks object. A scenario that no fault can fail costs compute on every merge and is evidence of nothing.
No. It sits over whatever you run - CARLA, Isaac Lab, MuJoCo, an in-house simulator, a vendor platform - through one script, and asks it a question it cannot ask itself: if the system got worse, would any of these scenarios fail? It simulates nothing, and your simulator stays where it is.
No, and it never says so. It measures what your tests would catch, against a fault catalogue it prints the limits of, and records it so that someone else can check it: every report replays from its recording. Deciding what that evidence is worth is your safety team's, or your assessor's - not ours.
A check that decides from something cheaper than the world: a rehearsal of the plan in a digital twin before the robot moves, or a judge - a skill's own result, a vision-language model looking at the scene - that says the task succeeded. Both can say go, or done, when the real run fails. mutational standin sweeps the stand-in beside the real run and names every fault where the two disagree.
One script that runs a scenario and honours an exit code: 0 if it passed, 1 if it failed, 3 if the fault means nothing in that scenario, anything else if the harness broke. The fault arrives as JSON in an environment variable. It has been run through reference adapters for CARLA, Isaac Lab and Meta-World on MuJoCo, written this way.
Sweeps run where your simulator is, and scenario source never leaves it: the collector refuses it by name, along with your adapter's command line, its environment and any path. What --post sends is a summary - scenario names, scores, which faults were caught - and, if you ask, the recording with everything your adapter printed stripped out.
A hosted worklist needs more than a summary, and gets it only when you say so: mutational project push sends the fault catalogue, what each scenario asserts and the format a model may write in, and says what it is sending every time it runs.
In the browser, nothing leaves the tab. The worker that reads your files deletes its own network access before it accepts one, and this site's account API refuses any request larger than four kilobytes - there is nowhere here to send a file.
Open is free: a sweep of up to 100 scenarios runs with no licence, and a workspace keeps 1 corpus, 500 sweeps posted a month, 5 GB at rest - recordings, full records and project documents together - 5 API tokens and 365 days of history. Past 100 scenarios a sweep needs a licence: a file your dashboard issues once a plan is paid for, which the binary checks offline against a key built into it and never phones home about. Universities and open-source maintainers can ask for a free academic licence. A sweep refused for a limit is still on the machine that ran it, and can be posted later from its recording.
Sign in with your email and download it from your dashboard: one static binary for macOS or Linux, on Apple silicon, Intel or Arm, with its checksum. There is an install one-liner for CI.
Yes. mutational mcp serves everything over the Model Context Protocol beside your simulator, and the collector serves the same tools over HTTP for an agent that is somewhere else. The rule both enforce is the product: a model may propose anything, and only a sweep accepts it. A coding agent working in your checkout can read the worklist, write a scenario and ask a sweep whether it earned its place. Give it an agent token and it can add to your history and queue campaigns - and cannot delete anything, or replace a build's sweep with one that catches less. Nothing here holds a model key of yours.
Delete any sweep through the API and its recording goes with it. Delete your account from the dashboard and the whole workspace goes: recordings first, then every row.
Results in two weeks, on your simulator, with your scenarios where they are. Or sign in and start on your own: Open is free, and measures up to 100 scenarios a sweep with no licence.