FAQ
Questions a reviewer asks first.
Last updated 30 October 2026
Short answers, taken from what the service does today.
Which pull requests can I check?
Public GitHub pull requests of Node.js projects. Noether installs the repository's dependencies with npm in a Node 24 sandbox and runs checks with node --test. A private repository is refused, and a project without a package.json cannot be run.
How long does a run take, and what does it cost me?
About a minute for a small pull request. A run is stopped after 270 seconds. It is free; to keep the cost bounded, one address may start 3 live runs per hour and the service accepts a fixed number per day. The replay of the sealed Hopper run is a real earlier run and never counts against that.
What do the verdicts mean?
- BROKEN: a behaviour that held on the base commit fails on the head, and the pull request does not announce the change.
- CONSERVED: every pinned behaviour still holds on the head.
- INCONCLUSIVE: nothing could be pinned, or the run could not finish. The reason is always given and nothing is claimed.
Can a CONSERVED verdict be wrong?
Yes. Only behaviours that were pinned, and green twice on the base, are compared, and a language model writes the checks. A break nobody pinned stays invisible, and two runs of the same pull request can pin different checks. The benchmark lists the catches, the misses and the false alarms.
What is a seal, and how do I check one?
Each finished run is stored as one JSON bundle with its sha256. Every run page re-hashes the bundle in your browser and compares it with the recorded hash. Download the JSON from the same page to hash it yourself.
Who sees my code?
For a live run, GitHub, Nebius (models and sandbox) and the server that hosts the site. The privacy page lists each one and what it receives.
Can I run it on my own repository?
Yes, as a GitHub Action with two Nebius secrets. The Action page has the workflow file, every input and output, and when it fails a job.