Noether

FAQ

Questions a reviewer asks first.

Last updated 30 October 2026

Short answers, taken from what the service does today.

01

Which pull requests can I check?

Public GitHub pull requests of Node.js projects. Noether installs the repository's dependencies with npm in a Node 24 sandbox and runs checks with node --test. A private repository is refused, and a project without a package.json cannot be run.

02

How long does a run take, and what does it cost me?

About a minute for a small pull request. A run is stopped after 270 seconds. It is free; to keep the cost bounded, one address may start 3 live runs per hour and the service accepts a fixed number per day. The replay of the sealed Hopper run is a real earlier run and never counts against that.

03

What do the verdicts mean?

  • BROKEN: a behaviour that held on the base commit fails on the head, and the pull request does not announce the change.
  • CONSERVED: every pinned behaviour still holds on the head.
  • INCONCLUSIVE: nothing could be pinned, or the run could not finish. The reason is always given and nothing is claimed.
04

Can a CONSERVED verdict be wrong?

Yes. Only behaviours that were pinned, and green twice on the base, are compared, and a language model writes the checks. A break nobody pinned stays invisible, and two runs of the same pull request can pin different checks. The benchmark lists the catches, the misses and the false alarms.

05

What is a seal, and how do I check one?

Each finished run is stored as one JSON bundle with its sha256. Every run page re-hashes the bundle in your browser and compares it with the recorded hash. Download the JSON from the same page to hash it yourself.

06

Who sees my code?

For a live run, GitHub, Nebius (models and sandbox) and the server that hosts the site. The privacy page lists each one and what it receives.

07

Can I run it on my own repository?

Yes, as a GitHub Action with two Nebius secrets. The Action page has the workflow file, every input and output, and when it fails a job.