--- title: How We Rank AI Models Without Publishing a Score description: We publish vendor prices, context windows and licences for 21 AI models, plus four labelled picks. Here is the method, the limits and what we refuse to invent. url: https://nerdleveltech.com/how-we-rank-ai-models-without-a-score language: en date: '2026-10-24' category: ai-ml tags: - AI models - model pricing - open-weight models - AI benchmarks - developer tools markdown_url: https://nerdleveltech.com/how-we-rank-ai-models-without-a-score.md attribution: 'Source: NerdLevelTech — https://nerdleveltech.com/how-we-rank-ai-models-without-a-score' terms: https://nerdleveltech.com/llms.txt --- # How We Rank AI Models Without Publishing a Score Every week another model tops another leaderboard, then disappears when the prompt or harness changes. For someone choosing a model for a product, that is a feed, not a decision aid. The [NerdLevelTech model index](https://nerdleveltech.com/models) does something deliberately boring: it publishes the facts a buyer can check — price, context window, licence, release date and links to the maker's pages — and labels our opinions as opinions. It does not publish a score out of 100, because we do not run the benchmark harness that would make such a score ours. Here is the method, the four picks, and where a number can mislead even when it is technically correct. ## What we publish, and what we refuse to The hub currently tracks 21 current models from 13 makers. Ten are open-weight or have open weights announced. For each one you get: - **Money and limits**: published API prices in USD per 1M input and output tokens, plus the context and output limits, including windows that are shared rather than additive. - **Licence status**: proprietary, open weights you can download now, or open weights announced but not yet public. - **Provenance**: a dated release where the maker published one, the maker's pricing and docs pages, and — only when they resolve — the Hugging Face repo and exact Ollama tag. - **One current card per model line.** A successor archives the old card and its URL 308s to the new one; the index does not accumulate dead models. Every entry carries a `verified` date. Prices and limits change; a page that does not say when it was checked is asking you to trust a memory. We do not publish a benchmark score. We do not run the same prompt across 21 models and call the output a ranking, and we do not relabel someone else's benchmark table as ours. If you need MMLU, SWE-bench or an arena Elo, read the people who ran that evaluation. Our job is narrower: what each model costs, what it accepts, and what you can legally do with it. ## The cost number is a rate, not an invoice A per-token price is easy to quote and hard to feel. So the hub also prints a fixed task cost: **50,000 input tokens plus 10,000 output tokens**, at the published headline rates. The arithmetic is simple: at $4 input and $20 output, that task costs $0.40. At $0.30 and $1.20 — [DeepSeek V4.1 Flash](https://nerdleveltech.com/models/deepseek-v4-1-flash) — it costs $0.027. At $0.15 and $0.50 — [GLM-5.3 Flash](https://nerdleveltech.com/models/glm-5-3-flash) — it costs about $0.013. Those differences matter more than a few percentage points on a synthetic exam. The label is "cost per task", not "cost per month". It ignores prompt caching, batch discounts, off-peak rates and tier crossings, and the pricing data records the exceptions: Grok changes price above a 200K-token prompt, DeepSeek has an off-peak rate, and Gemini's introductory prices have end dates. The leaderboard prints the task definition next to the column. ## Context is a window, not a sum The most common misleading sentence in model coverage is some version of "1M context and 128K output". It sounds like 1,128,000 tokens. It usually is not. OpenAI's GPT-6 family publishes a **1,050,000-token window**, split into at most **922,000 input** and **128,000 output** tokens.[^4] Anthropic's 1M window is shared: input and output both count toward it, with a 128K output cap.[^5] Gemini's 4 Argon and 3.8 Flash pages state a 1,048,576-token window and a 65,536-token standard output limit.[^6] Those shapes differ, which matters when you pack a long document into a prompt. The model pages therefore print the exact window, the max input and max output where the maker publishes them, and a note when the split is not published. A card may round to "1M" or "1.05M" so it fits; the detail page does not. ## The four picks are editorial, and that is the point The [leaderboard](https://nerdleveltech.com/models/leaderboard) carries four awards. They are opinions, dated, and chosen for four different jobs: - **Best overall: [Claude Opus 5.5](https://nerdleveltech.com/models/claude-opus-5-5).** Anthropic's default starting point for demanding work, with a 1M window, documented tool use and a $4/$20 price. - **Best value: [GPT-6.1 Sol](https://nerdleveltech.com/models/gpt-6-1-sol).** Near-Astra positioning for complex coding at $2/$10, with the same 1.05M window. - **Best budget workhorse: [DeepSeek V4.1 Flash](https://nerdleveltech.com/models/deepseek-v4-1-flash).** Vision, a 1M-token window and tool use at about $0.027 per task, half that off-peak. It is the one we would hand an agent. - **Best open weights: [GLM-5.3 Flash](https://nerdleveltech.com/models/glm-5-3-flash).** A dedicated Hugging Face repo, an Ollama library, a 1M window and image input at $0.15/$0.50. The full GLM-5.3 costs about nine times more for the same window. A pick is not a claim that the model wins every task. It is the card we would start with for that job at the checked price; the rationale and caveat sit next to it. If a pick changes, the page changes; no second, contradictory winner lingers. ## Open weights means downloadable, or it says so "Open" is doing a lot of work in model marketing. The hub separates three states: 1. **Open weights, published.** There is a Hugging Face repo or an Ollama page you can use today. The card links the exact repo and tag. 2. **Open weights, announced.** The maker has promised a release but the weights are not public. Kimi K3.1 Pro and Mistral Large 4 are in this state at the time of writing.[^7] Their cards say "Open weights (announced)" and do not carry a download link or a self-hosting tag until the release exists. 3. **Proprietary.** The model is API- or subscription-only, and the card says so. This is also why the self-hosting filter is narrower than the licence count. A promised release is not a deployment option yet. ## How to use the hub Start at the [model index](https://nerdleveltech.com/models). **Recent** is the default sort, so the newest cards are visible. Filter by maker, open weights, or use case; every card links the full page. The [leaderboard](https://nerdleveltech.com/models/leaderboard) groups the table by maker, collapsed by default; expand a group and every column sorts. The [methodology](https://nerdleveltech.com/models/methodology) covers succession and what happens to an old URL. If you are an agent, the same facts are available as data: - [`/models/index.json`](https://nerdleveltech.com/models/index.json) — every current model with prices, limits, provenance links and picks. - [`/models-rss.xml`](https://nerdleveltech.com/models-rss.xml) — the model change feed, newest first. - The MCP server at `https://nerdleveltech.com/api/mcp` exposes `search_models` and `get_model` with the same facts. The A2A endpoint mirrors them. ## What would change our mind If we ran a reproducible harness with published prompts, seeds and scoring code, we could publish a score. We have not, so we do not. If a vendor publishes an exact context split, we add it. If a price changes, we update the card and the date. If a model is replaced, the old URL redirects to the successor. That is the whole trick: fewer claims, each one traceable. A model page should let you answer "how much, how long, under what licence, and says who?" without trusting a number we made up. ## References [^1]: Anthropic, "[Pricing](https://docs.claude.com/en/docs/about-claude/pricing)," checked October 24, 2026. [^2]: DeepSeek, "[Models & Pricing](https://api-docs.deepseek.com/quick_start/pricing)," checked October 24, 2026. [^3]: Z.ai, "[Pricing](https://docs.z.ai/guides/overview/pricing)," checked October 24, 2026. [^4]: OpenAI, "[Pricing](https://platform.openai.com/docs/pricing)" and "[GPT-6.1 Sol](https://platform.openai.com/docs/models/gpt-6.1-sol)," checked October 24, 2026. [^5]: Anthropic, "[Models overview](https://docs.claude.com/en/docs/about-claude/models/overview)," checked October 24, 2026. [^6]: Google, "[Gemini API pricing](https://ai.google.dev/gemini-api/docs/pricing)" and "[Gemini 4 Argon](https://ai.google.dev/gemini-api/docs/models/gemini-4-argon)," checked October 24, 2026. [^7]: Release notes linked from the [Kimi K3.1 Pro](https://nerdleveltech.com/models/kimi-k3-1-pro) and [Mistral Large 4](https://nerdleveltech.com/models/mistral-large-4) cards; checked October 24, 2026. Corrections are welcome. If a price, limit or link is wrong, send the source and the date to [agents@nerdleveltech.com](mailto:agents@nerdleveltech.com).