Every tool starts at zero and earns points for what it does and what it costs. Here is exactly what earns them.
The short version
- These tools do different jobs, so each kind has its own checklist: AI coding assistants on 5 things, code review and security tools on 5 things and AI app builders on 3 things. Each checklist is worth 100 points, and a tool is only ranked against its own kind.
- The final score is those points out of 10. A tool with 84 points scores 8.4.
- Everything comes from the tool's own website. We don't take payment for points.
AI coding assistants
Agents and your codebasePricingModel freedomPrivacy and securityWhere it works
What earns points
1. Agents and your codebase
25 of 100 pointsCan it do real work on its own? Agents that work in the cloud and open a pull request, understanding your whole codebase, reviewing pull requests, and suggesting code as you type.
Agents that work in the cloud and open pull requestsup to 3
Understands your whole codebaseup to 2.5
Reviews pull requestsup to 2
Suggests code as you typeup to 2.5
Each section adds up to 10, then counts for 25 of the 100.
2. Pricing
20 of 100 pointsWhat the cheapest paid plan costs each month, and whether there is a free plan. Tools that sell no plan, where you pay your AI provider directly, are judged on the free plan alone.
Monthly price, cheapest paid planup to 6
| Up to $10 | +6 |
| Up to $20 | +5 |
| Up to $30 | +4 |
| Up to $40 | +3 |
| Over $40 | +1.5 |
Free trialup to 1
| None | 0 |
| 1 to 7 days | +0.5 |
| 8 days or more | +1 |
Each section adds up to 10, then counts for 20 of the 100.
3. Model freedom
20 of 100 pointsCan you choose the AI model, use your own API key, or run a model on your own machine?
AI model families you can chooseup to 5
| None | 0 |
| 1 familie | +1.5 |
| 2 families | +2.5 |
| 3 families | +3.5 |
| 4 families | +4.5 |
| 5 families or more | +5 |
Bring your own API keyup to 3
Runs models on your own machineup to 2
Each section adds up to 10, then counts for 20 of the 100.
4. Privacy and security
20 of 100 pointsWill your code stay private? Whether it trains AI on your code, can keep no copy of it, runs on your own servers, holds SOC 2, and covers you legally for the code it writes.
Does not train AI on your codeup to 3
| Yes | +3 |
| Only if you opt out | +1.5 |
| No | 0 |
Option to keep no copy of your codeup to 2
Runs on your own servers or cloudup to 2
Legal cover for generated codeup to 1
Each section adds up to 10, then counts for 20 of the 100.
5. Where it works
15 of 100 pointsDoes it work where you already code: VS Code, JetBrains, Visual Studio, Neovim, Xcode, the terminal or its own editor? And is it open source?
Editors and terminals it works inup to 8
| None | 0 |
| 1 place | +2 |
| 2 places | +4 |
| 3 places | +6 |
| 4 places | +7 |
| 5 places or more | +8 |
Each section adds up to 10, then counts for 15 of the 100.
A real example: GitHub
Here is how GitHub got its 8.5.
| Section | Points |
|---|
| Agents and your codebase | 25 of 25 |
| Pricing | 20 of 20 |
| Model freedom | 15 of 20 |
| Privacy and security | 13 of 20 |
| Where it works | 12 of 15 |
| Total | 85 of 100 = 8.5 out of 10 |
See every fact behind GitHub's score.
Code review and security tools
Review qualitySecurity scanningPricingFits your stackPrivacy and deployment
What earns points
1. Review quality
25 of 100 pointsDoes it review like a colleague? Comments on the exact lines that need work, and learning from what your team accepts or rejects.
Comments on specific lines of codeup to 6
Learns from your team's feedbackup to 4
Each section adds up to 10, then counts for 25 of the 100.
2. Security scanning
25 of 100 pointsDoes it catch security problems before they ship: flaws in your code, vulnerable dependencies, leaked keys and misconfigured infrastructure?
Finds security flaws in your codeup to 3.5
Checks open-source dependenciesup to 2.5
Finds leaked keys and passwordsup to 2
Scans infrastructure codeup to 2
Each section adds up to 10, then counts for 25 of the 100.
3. Pricing
20 of 100 pointsWhat the cheapest paid plan costs per developer each month, and whether there is a free plan.
Monthly price per developer, cheapest paid planup to 6
| Up to $20 | +6 |
| Up to $25 | +5 |
| Up to $30 | +4 |
| Up to $50 | +3 |
| Over $50 | +1.5 |
Free trialup to 1
| None | 0 |
| 1 to 7 days | +0.5 |
| 8 days or more | +1 |
Each section adds up to 10, then counts for 20 of the 100.
4. Fits your stack
15 of 100 pointsDoes it work with where your code lives: GitHub, GitLab, Bitbucket and Azure DevOps?
Git hosts it works withup to 10
| None | 0 |
| 1 host | +2.5 |
| 2 hosts | +5 |
| 3 hosts | +7.5 |
| 4 hosts or more | +10 |
Each section adds up to 10, then counts for 15 of the 100.
5. Privacy and deployment
15 of 100 pointsCan you run it on your own servers, does it keep your code out of AI training, and does it hold ISO 27001 and HIPAA?
Runs on your own servers or cloudup to 4
Does not train AI on your codeup to 3
| Yes | +3 |
| Only if you opt out | +1.5 |
| No | 0 |
Each section adds up to 10, then counts for 15 of the 100.
A real example: CodeRabbit
Here is how CodeRabbit got its 9.1.
| Section | Points |
|---|
| Review quality | 25 of 25 |
| Security scanning | 25 of 25 |
| Pricing | 15.6 of 20 |
| Fits your stack | 15 of 15 |
| Privacy and deployment | 10.5 of 15 |
| Total | 91.1 of 100 = 9.1 out of 10 |
See every fact behind CodeRabbit's score.
AI app builders
PricingWhat you can shipTrust and compliance
What earns points
1. Pricing
20 of 100 pointsWhat the cheapest paid plan costs each month. All four have a free plan, so that earns no extra points here.
Monthly price, cheapest paid planup to 10
| Up to $20 | +10 |
| Up to $25 | +8.5 |
| Up to $30 | +7 |
| Up to $50 | +5 |
| Over $50 | +2.5 |
Each section adds up to 10, then counts for 20 of the 100.
2. What you can ship
45 of 100 pointsNearly everything else is the same on all four: hosting, a database, logins and GitHub. What differs is whether it builds mobile apps, has an API, and can run on your own servers.
Builds mobile appsup to 5
Runs on your own servers or cloudup to 2
Each section adds up to 10, then counts for 45 of the 100.
3. Trust and compliance
35 of 100 pointsCan the apps you build handle health data (HIPAA), and does the vendor hold ISO 27001?
Each section adds up to 10, then counts for 35 of the 100.
A real example: bolt.new
Here is how bolt.new got its 6.6.
| Section | Points |
|---|
| Pricing | 17 of 20 |
| What you can ship | 31.5 of 45 |
| Trust and compliance | 17.5 of 35 |
| Total | 66 of 100 = 6.6 out of 10 |
See every fact behind bolt.new's score.
When a tool doesn't say
If a tool's website doesn't answer a yes-or-no question, it gets half the points for that item. Not zero, because a missing answer isn't the same as a no, and not full marks, because silence shouldn't beat a vendor who answers honestly. For a number, like a price, it gets the typical value of the tools that do publish one. Every item scored this way is marked on the tool's scorecard.
If a tool leaves out more than a third of what we check, we don't show a score for it at all. A number built mostly on guesses wouldn't be fair to you or to the tool. Right now 20 of the 21 tools we checked have enough information to be scored.
What the score doesn't tell you
We score what a tool offers and what it costs, as its own website describes it. That's not a hands-on test. The score can't tell you whether a tool is easy to use, whether its data is accurate, or whether support answers quickly. The review covers that.
See the AI Developer tools ranked or read how we review tools in general.