Simantic reviews your pull requests

Overview

The reviewer is an on-demand assistant, not CI: nothing runs on a push. A simantic.json in the project’s folder tells it the MCU and how to build; without one, it works out both from the repository. It builds the pull request’s head (and its base when a comparison helps), and runs the firmware in the Simantic simulator with scenarios such as “press the button on PC13 at 100 ms and expect button pressed on USART2”. Every claim about runtime behavior in its review comes from one of those runs.

It is part of every plan, Community included, and runs on Simantic’s own agent: you need no key of your own.

Install the app

In the dashboard’s Simantic Bot tab, press Connect GitHub. GitHub asks which account or organization to install the simantic app on and which repositories it may see; pick the firmware repositories you want reviewed. An installation on an organization you do not own is sent to its owners for approval; once they approve it, add it again from the dashboard. Always start from the dashboard: the install link is made for your account and lasts thirty minutes, and GitHub hands it back so Simantic knows which account to link.

Once one is connected, Add an organization installs the app on another organization, or on your own account, the same way: connect as many as you like. Each installation is connected to one Simantic account, the one whose plan its reviews come out of.

If the app is already installed on the account you pick, after you disconnected it say, GitHub opens the installation’s settings instead of installing it again. Save them, and GitHub sends you back to authorize Simantic, which connects it.

An enterprise group has a Simantic Bot tab of its own, under the group’s dashboard. An installation connected there is the group’s: every member sees its repositories and the reviews of them, any member can disconnect it or connect it again, and the reviews are billed to the group, separately from its Enterprise plan, rather than to anyone’s own plan. An installation is connected to one account or one group at a time; connecting it again from the other dashboard moves it.

Billing and allowance

Each review is billed for the work it took, and comes out of your plan’s monthly included usage, which simulation runs share: $10 a month on the Community plan, $200 on Developer. Enterprise reviews are billed separately. The Simantic Bot tab’s billing card shows what this month’s reviews were billed and what is left; View charges lists what each review was billed, a month at a time.

Once the included usage is used up, reviews are refused, unless you set a monthly review overage budget in your settings; reviews may then go past the allowance up to that budget, and the overage is charged to you. A review is never billed more than what is left when it starts; with less than $1.50 left, it is refused rather than started. The comment on the pull request says when a review was refused and why.

Ask for a review

Mention the bot in a comment on the pull request; any mention asks, for example:

@simantic-bot review

Anyone with write access to the repository can ask. Simantic reacts to the comment, posts a note that it is working and a link to the review’s page in the dashboard, and posts the review when it is done, usually within a few minutes. To review new commits, mention it again. The words after the mention are passed to the reviewer as your request, for example @simantic-bot review the UART changes only. Which MCU a project targets comes from its simantic.json; a repository without one can say it here: @simantic-bot review --mcu STM32F401RE.

What it does

• Clones the pull request into a container of its own.
• Reads each project’s simantic.json for the MCU, the build system (a CubeMX or plain Makefile, CMake, pico-sdk, ESP-IDF or Zephyr) and the ELF, detects what the file leaves out, and builds the firmware, keeping the compiler’s warnings.
• Reads the diff and designs a few scenarios for what changed, each with a stimulus and an expectation, and runs them in the simulator.
• Posts one review: a summary, a table of the scenarios and their results, and inline comments with suggested changes where it found a problem. Anything it could not simulate is reported as inconclusive, naming what was missing; when nothing was simulated at all, because the MCU is one the simulator does not run or the firmware could not be built and run, the whole review is static and says why.

The review’s page in the dashboard holds the full report: each scenario’s UART output, the build log and the reviewer’s log, and the firmware it built (below).

The Simantic Bot tab lists your reviews by pull request: each pull request once, with its latest review and what all of its reviews were billed. Open one to see every review of it, each with what it was billed. Show all lists every review instead, newest first.

The firmware it built

Simantic keeps the ELF files a review builds, encrypted, so you can download them: the firmware of the commit it reviewed, on the review’s page, and of the pull request’s base (where its branch started), when the reviewer built that to compare with. A pull request keeps one base, its current one, on the pull request’s page: after a rebase, the next review replaces it with the new base’s firmware, or deletes it if that review did not need the base.

Delete either from its page. Firmware storage in your Cloud Sim settings covers it with your runs’ firmware: turned off, it deletes all of it and reviews keep none. People you share a review with do not get its firmware.

Sharing a review

A review is private: only the account that connected the installation sees it. On the review’s page, Share opens it to the people you name, by email, or to anyone at Simantic who has the link. The same choice for every review of a repository is in the same dialog, and in the Simantic Bot tab’s repository list.

An invited person signs in to Simantic with that email and finds the review under Shared with me on the Simantic Bot tab; a review open to anyone with the link is listed nowhere. Either way they can read the report, the log and the scenarios, but not cancel the review, and they do not see what it cost you. Withdraw the sharing at any time from the same dialog. A review of an enterprise group’s installation needs no sharing within the group: every member reads it from the group’s Simantic Bot tab, with what it was billed.

Limits

• MCUs: the ones the simulator runs, which sim models and the Cloud Sim API’s GET /v1/mcus list; a pull request for any other MCU gets a static review, without simulation and never approved.
• Build systems: CubeMX and plain make, CMake, pico-sdk, Zephyr (west) and ESP-IDF.
• A review has half an hour and an agent budget of its own, capped by what is left of your allowance; it stops with what it has.
• A personal account has a few reviews unfinished at a time; another is refused until one finishes. An enterprise group’s installation has no such cap: its reviews queue up.

Privacy and cost

The repository is cloned into a container that is removed when the review ends. The diff, the build output and the simulator output are sent to Anthropic, Simantic’s model provider, under Simantic’s agreement with them; Simantic keeps the review’s report and log, what it cost, whom you shared it with, and, unless you turn firmware storage off, the firmware it built. See the Privacy Policy and the Terms.