A review layer for every PR
Sid wants every piece of code our team pushes to GitHub to pass through an evaluation step run by a capable coding model before a person reviews it, and he wants it to work across the whole organization.
The easiest way to do that is to plug in a hosted coding agent. By Achal’s estimate, running each review through Claude Code would cost about $25, and running it through Dots with Codex would likely cost even more. Per-review pricing works against the exact habit we’re trying to build, because the smaller and more frequent our PRs get, the more reviews we run and the bigger the bill becomes.
“If it's $25 every time, that will just rack up the bill.”
—Achal Aggarwal, CPO, Born West
Why Qwen 3 fits the way we work
Sid is testing Qwen 3, an open-weight model, as a self-hosted alternative. In his experience it offers strong performance for its cost on small and medium PRs, and it struggles when it’s handed a very large diff.
That limit rarely matters for us, because our engineers already keep their PRs small. We don’t submit massive diffs, so Qwen 3 almost never sees the kind of work it struggles with.
“It falls down when you have really massive code to review. But our best practice is small chunks of PRs, so it works well.” Siddhaarth Verma, CTO, Born West
What self-hosting gives us
The model runs on servers we provision in our own AWS account, so we pay for the server and the cost doesn’t change with each review. Achal describes that as unlimited tokens, which is true in the way that matters for us: the bill stays the same until we decide to scale the machine up.
Our codebase also stays on our own infrastructure and isn’t sent to a third party for processing.
“This is our codebase, and it's running on our provisioned servers.”
—Achal Aggarwal, CPO, Born West
We can also swap the model whenever something better comes along, and move the server up or down as the load changes.
What it costs us
A self-hosted model on our own hardware won’t match the speed of a hosted service. We’re comfortable with that because code review is asynchronous. Once an engineer submits a PR, they move on to the next task, so if the review takes 20 minutes, nobody is sitting and waiting for it.
The bigger cost is the work of running it. Someone has to provision the server, monitor it, and keep it updated, and that ongoing effort is the main reason hosted tools appeal to teams without infrastructure experience.
Where it stands
This is still an experiment. Sid is testing whether it can roll out across the organization and comparing it against the hosted options. We’ll report back either way, even if a hosted tool wins.
Lessons you can steal
- Count how many PRs your team ships each week before you choose a per-review tool.
- If your team already ships small PRs, a smaller self-hosted model may be all you need.
- Don’t worry much about review speed when the review happens asynchronously.
- Keep your code on infrastructure you control whenever that’s practical.
- Choose a setup that lets you swap the model later.
Small PRs make cheap reviews possible. Big PRs make every option expensive.
If your AI tooling bill grows faster than your team, we're happy to look at it with you. Book a 15-minute intro.
